aboutsummaryrefslogtreecommitdiff
path: root/tests/runtests.pl
AgeCommit message (Collapse)Author
2020-04-02runtests.pl: log host OS as detected by Perl environmentMarc Hoersken
2020-03-18schannel: add "best effort" revocation check optionJohannes Schindelin
- Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and --ssl-revoke-best-effort to allow a "best effort" revocation check. A best effort revocation check ignores errors that the revocation check was unable to take place. The reasoning is described in detail below and discussed further in the PR. --- When running e.g. with Fiddler, the schannel backend fails with an unhelpful error message: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate. Sadly, many enterprise users who are stuck behind MITM proxies suffer the very same problem. This has been discussed in plenty of issues: https://github.com/curl/curl/issues/3727, https://github.com/curl/curl/issues/264, for example. In the latter, a Microsoft Edge developer even made the case that the common behavior is to ignore issues when a certificate has no recorded distribution point for revocation lists, or when the server is offline. This is also known as "best effort" strategy and addresses the Fiddler issue. Unfortunately, this strategy was not chosen as the default for schannel (and is therefore a backend-specific behavior: OpenSSL seems to happily ignore the offline servers and missing distribution points). To maintain backward-compatibility, we therefore add a new flag (`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option (`--ssl-revoke-best-effort`) to select the new behavior. Due to the many related issues Git for Windows and GitHub Desktop, the plan is to make this behavior the default in these software packages. The test 2070 was added to verify this behavior, adapted from 310. Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com> Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Closes https://github.com/curl/curl/pull/4981
2020-03-10runtests: log minimal and maximal used port numbersMarc Hoersken
2020-03-07runtests: fix missing use of exe_ext helper functionMarc Hoersken
2020-03-07libssh: Fix matching user-specified MD5 hex keyJay Satiro
Prior to this change a match would never be successful because it was mistakenly coded to compare binary data from libssh to a user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). Reported-by: fds242@users.noreply.github.com Fixes https://github.com/curl/curl/issues/4971 Closes https://github.com/curl/curl/pull/4974
2020-03-05polarssl: remove more references and mentionsDaniel Stenberg
Assisted-by: Jay Satiro Follow-up to 6357a19ff29dac04 Closes #5036
2020-03-04tests: wrap ignored test failures in bracesMarc Hoersken
2020-03-04tests: try to make sleeping portable by avoiding selectMarc Hoersken
select does not support just waiting on Windows: https://perldoc.perl.org/perlport.html#select Reviewed-By: Daniel Stenberg Closes #5035
2020-03-03ci/tests: Make it possible to still run but ignore failing testsMarc Hoersken
This enables the development of a solution for the failing tests by running them on CI while ignoring their result for the overall status. Closes #4994
2020-03-03ci/tests: Move CI test result creation above environment setupMarc Hoersken
This avoids using our test servers as proxy to the AppVeyor API. Closes #5022
2020-03-03ci/tests: Send test results to AppVeyor for status overviewMarc Hoersken
Closes #5021
2020-03-02tests: disable SMTP UTF-8 tests on WindowsSteve Holme
Fixes #4988 Closes #4992
2020-02-29runtests: fix output to command logJay Satiro
- Record only the command of the most recently ran test in the command log. This is a follow-up to 02988b7 from several weeks ago which fixed writing to the command log, however it saved all commands for all tests instead of just the most recently ran test as we would now expect. Fixes https://github.com/curl/curl/commit/02988b7#commitcomment-37546876 Closes https://github.com/curl/curl/pull/5001
2020-02-29ci/tests: Send test results to Azure DevOps for reportingMarc Hoersken
2020-02-27runtests: restore the command logDaniel Stenberg
The log file with all command lines for the invoked command lines is now called logs/commands.log Fixes #4911 Closes #4989
2020-02-27tests: Automatically deduce the tool name from the test case for unit testsSteve Holme
It is still possible to override the executable to run during the test, using the <tool> tag, but this patch removes the requirement that the tag must be present for unit tests. It also removes the possibility of human error when existing test cases are used as the basis for new tests, as recently witnessed in 81c37124. Reviewed-by: Daniel Stenberg Closes #4976
2020-02-02cleanup: fix typos and wording in docs and commentsPedro Monreal
Closes #4869 Reviewed-by: Emil Engler and Daniel Gustafsson
2020-01-09runtests: make random seed fixed for a monthDaniel Stenberg
When using randomized features of runtests (-R and --shallow) it is useful to have a fixed random seed to make sure for example extra commits in a branch or a rebase won't change the seed that would make repeated runs work differently. As it is also useful to change seed sometimes, the default seed is now determined based on the current month (and first line curl -V output). When the month changes, so will the random seed. The specific seed is also shown in the standard test suite top header and it can be set explictly with the new --seed=[num] option so that the exact order of a previous run can be achieved. Closes #4734
2019-12-15runtests: --repeat=[num] to repeat testsDaniel Stenberg
Closes #4715
2019-12-11runtests: introduce --shallow to reduce huge torture testsDaniel Stenberg
When set, shallow mode limits runtests -t to make no more than NUM fails per test case. If more are found, it will randomly discard entries until the number is right. The random seed can also be set. This is particularly useful when running MANY tests as then most torture failures will already fail the same functions over and over and make the total operation painfully tedious. Closes #4699
2019-12-09tests: make it possible to set executable extensionsMarc Hoersken
This enables the use of Windows Subsystem for Linux (WSL) to run the testsuite against Windows binaries while using Linux servers. This commit introduces the following environment variables: - CURL_TEST_EXE_EXT: set the executable extension for all components - CURL_TEST_EXE_EXT_TOOL: set it for the curl tool only - CURL_TEST_EXE_EXT_SSH: set it for the SSH tools only Later testcurl.pl could be adjusted to make use of those variables. - CURL_TEST_EXE_EXT_SRV: set it for the test servers only (This is one of several commits to support use of WSL for the tests.) Closes https://github.com/curl/curl/pull/3899
2019-12-09tests: use \r\n for log messages in WSLMarc Hoersken
Bash in Windows Subsystem for Linux (WSL) requires it for some reason. (This is one of several commits to support use of WSL for the tests.) Ref: https://github.com/curl/curl/pull/3899
2019-10-20runtests: get textaware info from curl instead of perlMarcel Raad
The MSYS system on Windows can run the test suite for curl built with any toolset. When built with the MSYS GCC, curl uses Unix line endings, while it uses Windows line endings when built with the MinGW GCC, and `^O` reports 'msys' in both cases. Use the curl executable itself to determine the line endings instead, which reports 'x86_64-pc-msys' when built with the MSYS GCC. Closes https://github.com/curl/curl/pull/4506
2019-06-20tests: support non-localhost HOSTIP for dict/smb serversStephan Szabo
smbserver.py/dictserver.py were explicitly using localhost/127.0.0.1 for binding the server which when we were running the tests with a separate HOSTIP and CLIENTIP had failures verifying the server from the device we were testing. This changes them to take the address from runtests.py and default to localhost/127.0.0.1 if none is given. Closes #4048
2019-06-18runtests: keep logfiles around by defaultDaniel Stenberg
Make '-k' a no-op. The singletest function now clears the log directory BEFORE each individual test and not after, which makes it possible to always keep the logfiles around after a test has been run. No need to specify -k anymore. Keeping the option parsing around to work with users of old habits. Some tests also didn't work properly when -k was used (since the old logs would be kep when a new test starts) which this change also fixes. Closes #4035
2019-06-17tests: have runtests figure out disabled featuresDaniel Stenberg
... so that runtests can skip individual test cases that test features that are explicitly disabled in this build. This new logic is intended for disabled features that aren't otherwise easily visible through the curl_version_info() or other API calls. tests/server/disabled is a newly built executable that will output a list of disabled features. Outputs nothing for a default build. Closes #3950
2019-06-15runtests: report single test time + total durationDaniel Stenberg
... after each successful test. Closes #4027
2019-06-10wolfssl: refer to it as wolfSSL onlyDaniel Stenberg
Remove support for, references to and use of "cyaSSL" from the source and docs. wolfSSL is the current name and there's no point in keeping references to ancient history. Assisted-by: Daniel Gustafsson Closes #3903
2019-04-15runtests: start socksd like other serversDaniel Stenberg
... without a $srcdir prefix. Triggered by the failures in several autobuilds. Closes #3781
2019-04-13test: make tests and test scripts use socksd for SOCKSDaniel Stenberg
Make all SOCKS tests use socksd instead of ssh.
2019-03-03alt-svc: the libcurl bitsDaniel Stenberg
2019-02-26runtests: detect "schannel" as an alias for "winssl"Daniel Stenberg
Follow-up to 180501cb02 Reported-by: Marcel Raad Fixes #3609 Closes #3620
2019-02-11tests: add stderr comparison to the test suiteFrank Gevaerts
The code is more or less copied from the stdout comparison code, maybe some better reuse is possible. test 1457 is adjusted to make the output actually match (by using --silent) test 506 used <stderr> without actually needing it, so that <stderr> block is removed Closes #3536
2019-02-04runtests.pl: Fix perl call to include srcdirLadar Levison
- Use explicit include opt for perl calls. Prior to this change some scripts couldn't find their dependencies. At the top, perl is called using with the "-Isrcdir" option, and it works: https://github.com/curl/curl/blob/curl-7_63_0/tests/runtests.pl#L183 But on line 3868, that option is omitted. This caused problems for me, as the symbol-scan.pl script in particular couldn't find its dependencies properly: https://github.com/curl/curl/blob/curl-7_63_0/tests/runtests.pl#L3868 This patch fixes that oversight by making calls to perl sub-shells uniform. Closes https://github.com/curl/curl/pull/3496
2018-11-06url: a short host name + port is not a schemeDaniel Stenberg
The function identifying a leading "scheme" part of the URL considered a few letters ending with a colon to be a scheme, making something like "short:80" to become an unknown scheme instead of a short host name and a port number. Extended test 1560 to verify. Also fixed test203 to use file_pwd to make it get the correct path on windows. Removed test 2070 since it was a duplicate of 203. Assisted-by: Marcel Raad Reported-by: Hagai Auro Fixes #3220 Fixes #3233 Closes #3223 Closes #3235
2018-11-01axtls: removedDaniel Stenberg
As has been outlined in the DEPRECATE.md document, the axTLS code has been disabled for 6 months and is hereby removed. Use a better supported TLS library! Assisted-by: Daniel Gustafsson Closes #3194
2018-10-31runtests: use the local curl for verifyingDaniel Stenberg
... revert the mistaken change brought in commit 8440616f53. Reported-by: Alessandro Ghedini Bug: https://curl.haxx.se/mail/lib-2018-10/0118.html Closes #3198
2018-10-07runtests: skip ld_preload tests on macOSDaniel Gustafsson
The LD_PRELOAD functionality doesn't exist on macOS, so skip any tests requiring it. Fixes #2394 Closes #3106 Reported-by: Github user @jakirkham Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2018-10-04runtests: use Windows paths for Windows curlMarcel Raad
curl generated by CMake's Visual Studio generator has "Windows" in the version number.
2018-10-02runtests: ignore disabled even when ranges are givenDaniel Stenberg
runtests.pl support running a range of tests, like "44 to 127". Starting now, the code makes sure that even such given ranges will ignore tests that are marked as disabled. Disabled tests can still be run by explictly specifying that test number. Closes #3075
2018-09-13runtests.pl: run tests against the MesaLink vtls backendYiming Jing
2018-09-05pipelining: deprecatedDaniel Stenberg
Transparently. The related curl_multi_setopt() options all still returns OK when pipelining is selected. To re-enable the support, the single line change in lib/multi.c needs to be reverted. See docs/DEPRECATE.md Closes #2705
2018-08-31Don't use Windows path %PWD for SSH testsMarcel Raad
All these tests failed on Windows because something like sftp://%HOSTIP:%SSHPORT%PWD/ expanded to sftp://127.0.0.1:1234c:/msys64/home/bla/curl and then curl complained about the port number ending with a letter. Use the original POSIX path instead of the Windows path created in checksystem to fix this. Closes https://github.com/curl/curl/pull/2920
2018-08-13http: fix for tiny "HTTP/0.9" responseDaniel Stenberg
Deal with tiny "HTTP/0.9" (header-less) responses by checking the status-line early, even before a full "HTTP/" is received to allow detecting 0.9 properly. Test 1266 and 1267 added to verify. Fixes #2420 Closes #2872
2018-06-15runtests.pl: remove debug leftover from bb9a340c73f3Daniel Stenberg
2018-06-12runtests: support variables in <strippart>Daniel Stenberg
... and make use of that to make 1455 work better without using a fixed local port number. Fixes #2649 Closes #2650
2018-04-29tests: provide 'manual' as a feature to optionally requireDaniel Stenberg
... and make test 1026 rely on that feature so that --disable-manual builds don't cause test failures. Reported-by: Max Dymond and Anders Roxell Fixes #2533 Closes #2540
2018-03-26runtests.pl: fix warning 'use of uninitialized value'Michael Kaufmann
follow-up to a9a7b60 Closes #2428
2018-03-15transfer: make HTTP without headers count correct body sizeDaniel Stenberg
This is what "HTTP/0.9" basically looks like. Reported on IRC Closes #2382
2018-01-25SChannel/WinSSL: Implement public key pinningmoparisthebest
Closes #1429
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-01 13:33:55 +0000