aboutsummaryrefslogtreecommitdiff
path: root/tests/runtests.pl
AgeCommit message (Collapse)Author
2020-04-25runtests: check for the disabled tests relative srcdirDaniel Stenberg
To make it work correctly for out-of-tree builds. Follow-up to 75e8feb6fb08b Bug: https://github.com/curl/curl/pull/5288#issuecomment-619346389 Reported-by: Marcel Raad Closes #5297
2020-04-25runtests: revert commenting out a line I did for debuggingDaniel Stenberg
Follow-up to 11091cd4d. It was not meant to be pushed!
2020-04-25runtests: when <killserver> mentions http, kill http/2 tooDaniel Stenberg
Since the http2 test server is a mere proxy that needs to know about the dynamic port the HTTP server is using, it too needs to get restarted when the http server is killed. A regression caused by 80d6515. Fixes #5289 Closes #5291
2020-04-24tests: make runtests check that disabled tests existsDaniel Stenberg
... and error out if so. Removed '536' from DISABLED as there is no such test file. Closes #5288
2020-04-20tests: run the RTSP test server on a dynamic port numberDaniel Stenberg
To avoid port collisions. Closes #5272
2020-04-20tests: add %NOLISTENPORT and use itDaniel Stenberg
The purpose with this variable is to provide a port number that is reasonably likely to not have a listener on the local host so that tests can try connect failures against it. It uses port 47 - "reserved" according to IANA. Updated six tests to use it instead of the previous different ports. Assisted-by: Emil Engler Closes #5270
2020-04-20tests: run the SOCKS test server on a dynamic port numberDaniel Stenberg
Closes #5266
2020-04-20tests: run the TFTP test server on a dynamic port numberDaniel Stenberg
Picking a dynamic unused port is better than a fixed to avoid the collision risk. Closes #5265
2020-04-20runtests: always put test number in servercmd fileDaniel Stenberg
2020-04-19runtests: use a unix domain socket path with the pid in the nameDaniel Stenberg
To make it impossible for test cases to access the file name without using the proper variable for the purpose. Closes #5264
2020-04-19runtests: dummy init the ports variables to avoid warningsDaniel Stenberg
... and generate something that can help debug test cases.
2020-04-18tests: move pingpong server to dynamic listening portDaniel Stenberg
FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic ports Test 842-845 are unfortunately a bit hard to move over to this concept right now and require "default port" still...
2020-04-18tests: introduce preprocessed test casesDaniel Stenberg
The runtests script now always performs variable replacement on the entire test source file before the test gets executed, and saves the updated version in a temporary file (log/test[num]) so that all test case readers/servers can use that version (if present) and thus enjoy the powers of test case variable substitution. This is necessary to allow complete port number freedom. Test 309 is updated to work with a non-fixed port number thanks to this.
2020-04-18tests: run the sws server on "any port"Daniel Stenberg
Makes the test servers for HTTP and Gopher pop up on a currently unused port and runtests adapts to that! Closes #5247
2020-04-14tests: support hex encoded data and mqtt serverDaniel Stenberg
The mqtt server is started using a "random" port.
2020-04-11tests: add Windows compatible pidwait like pidkill and pidtermMarc Hoersken
Related to #5188
2020-04-09runtests: fix warning about using an undefined variableDaniel Stenberg
Follow-up from 4d939ef6ceb2db1
2020-04-08runtests: provide nicer errormsg when protocol "dump" file is emptyDaniel Stenberg
2020-04-02runtests.pl: log host OS as detected by Perl environmentMarc Hoersken
2020-03-18schannel: add "best effort" revocation check optionJohannes Schindelin
- Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and --ssl-revoke-best-effort to allow a "best effort" revocation check. A best effort revocation check ignores errors that the revocation check was unable to take place. The reasoning is described in detail below and discussed further in the PR. --- When running e.g. with Fiddler, the schannel backend fails with an unhelpful error message: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate. Sadly, many enterprise users who are stuck behind MITM proxies suffer the very same problem. This has been discussed in plenty of issues: https://github.com/curl/curl/issues/3727, https://github.com/curl/curl/issues/264, for example. In the latter, a Microsoft Edge developer even made the case that the common behavior is to ignore issues when a certificate has no recorded distribution point for revocation lists, or when the server is offline. This is also known as "best effort" strategy and addresses the Fiddler issue. Unfortunately, this strategy was not chosen as the default for schannel (and is therefore a backend-specific behavior: OpenSSL seems to happily ignore the offline servers and missing distribution points). To maintain backward-compatibility, we therefore add a new flag (`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option (`--ssl-revoke-best-effort`) to select the new behavior. Due to the many related issues Git for Windows and GitHub Desktop, the plan is to make this behavior the default in these software packages. The test 2070 was added to verify this behavior, adapted from 310. Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com> Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Closes https://github.com/curl/curl/pull/4981
2020-03-10runtests: log minimal and maximal used port numbersMarc Hoersken
2020-03-07runtests: fix missing use of exe_ext helper functionMarc Hoersken
2020-03-07libssh: Fix matching user-specified MD5 hex keyJay Satiro
Prior to this change a match would never be successful because it was mistakenly coded to compare binary data from libssh to a user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). Reported-by: fds242@users.noreply.github.com Fixes https://github.com/curl/curl/issues/4971 Closes https://github.com/curl/curl/pull/4974
2020-03-05polarssl: remove more references and mentionsDaniel Stenberg
Assisted-by: Jay Satiro Follow-up to 6357a19ff29dac04 Closes #5036
2020-03-04tests: wrap ignored test failures in bracesMarc Hoersken
2020-03-04tests: try to make sleeping portable by avoiding selectMarc Hoersken
select does not support just waiting on Windows: https://perldoc.perl.org/perlport.html#select Reviewed-By: Daniel Stenberg Closes #5035
2020-03-03ci/tests: Make it possible to still run but ignore failing testsMarc Hoersken
This enables the development of a solution for the failing tests by running them on CI while ignoring their result for the overall status. Closes #4994
2020-03-03ci/tests: Move CI test result creation above environment setupMarc Hoersken
This avoids using our test servers as proxy to the AppVeyor API. Closes #5022
2020-03-03ci/tests: Send test results to AppVeyor for status overviewMarc Hoersken
Closes #5021
2020-03-02tests: disable SMTP UTF-8 tests on WindowsSteve Holme
Fixes #4988 Closes #4992
2020-02-29runtests: fix output to command logJay Satiro
- Record only the command of the most recently ran test in the command log. This is a follow-up to 02988b7 from several weeks ago which fixed writing to the command log, however it saved all commands for all tests instead of just the most recently ran test as we would now expect. Fixes https://github.com/curl/curl/commit/02988b7#commitcomment-37546876 Closes https://github.com/curl/curl/pull/5001
2020-02-29ci/tests: Send test results to Azure DevOps for reportingMarc Hoersken
2020-02-27runtests: restore the command logDaniel Stenberg
The log file with all command lines for the invoked command lines is now called logs/commands.log Fixes #4911 Closes #4989
2020-02-27tests: Automatically deduce the tool name from the test case for unit testsSteve Holme
It is still possible to override the executable to run during the test, using the <tool> tag, but this patch removes the requirement that the tag must be present for unit tests. It also removes the possibility of human error when existing test cases are used as the basis for new tests, as recently witnessed in 81c37124. Reviewed-by: Daniel Stenberg Closes #4976
2020-02-02cleanup: fix typos and wording in docs and commentsPedro Monreal
Closes #4869 Reviewed-by: Emil Engler and Daniel Gustafsson
2020-01-09runtests: make random seed fixed for a monthDaniel Stenberg
When using randomized features of runtests (-R and --shallow) it is useful to have a fixed random seed to make sure for example extra commits in a branch or a rebase won't change the seed that would make repeated runs work differently. As it is also useful to change seed sometimes, the default seed is now determined based on the current month (and first line curl -V output). When the month changes, so will the random seed. The specific seed is also shown in the standard test suite top header and it can be set explictly with the new --seed=[num] option so that the exact order of a previous run can be achieved. Closes #4734
2019-12-15runtests: --repeat=[num] to repeat testsDaniel Stenberg
Closes #4715
2019-12-11runtests: introduce --shallow to reduce huge torture testsDaniel Stenberg
When set, shallow mode limits runtests -t to make no more than NUM fails per test case. If more are found, it will randomly discard entries until the number is right. The random seed can also be set. This is particularly useful when running MANY tests as then most torture failures will already fail the same functions over and over and make the total operation painfully tedious. Closes #4699
2019-12-09tests: make it possible to set executable extensionsMarc Hoersken
This enables the use of Windows Subsystem for Linux (WSL) to run the testsuite against Windows binaries while using Linux servers. This commit introduces the following environment variables: - CURL_TEST_EXE_EXT: set the executable extension for all components - CURL_TEST_EXE_EXT_TOOL: set it for the curl tool only - CURL_TEST_EXE_EXT_SSH: set it for the SSH tools only Later testcurl.pl could be adjusted to make use of those variables. - CURL_TEST_EXE_EXT_SRV: set it for the test servers only (This is one of several commits to support use of WSL for the tests.) Closes https://github.com/curl/curl/pull/3899
2019-12-09tests: use \r\n for log messages in WSLMarc Hoersken
Bash in Windows Subsystem for Linux (WSL) requires it for some reason. (This is one of several commits to support use of WSL for the tests.) Ref: https://github.com/curl/curl/pull/3899
2019-10-20runtests: get textaware info from curl instead of perlMarcel Raad
The MSYS system on Windows can run the test suite for curl built with any toolset. When built with the MSYS GCC, curl uses Unix line endings, while it uses Windows line endings when built with the MinGW GCC, and `^O` reports 'msys' in both cases. Use the curl executable itself to determine the line endings instead, which reports 'x86_64-pc-msys' when built with the MSYS GCC. Closes https://github.com/curl/curl/pull/4506
2019-06-20tests: support non-localhost HOSTIP for dict/smb serversStephan Szabo
smbserver.py/dictserver.py were explicitly using localhost/127.0.0.1 for binding the server which when we were running the tests with a separate HOSTIP and CLIENTIP had failures verifying the server from the device we were testing. This changes them to take the address from runtests.py and default to localhost/127.0.0.1 if none is given. Closes #4048
2019-06-18runtests: keep logfiles around by defaultDaniel Stenberg
Make '-k' a no-op. The singletest function now clears the log directory BEFORE each individual test and not after, which makes it possible to always keep the logfiles around after a test has been run. No need to specify -k anymore. Keeping the option parsing around to work with users of old habits. Some tests also didn't work properly when -k was used (since the old logs would be kep when a new test starts) which this change also fixes. Closes #4035
2019-06-17tests: have runtests figure out disabled featuresDaniel Stenberg
... so that runtests can skip individual test cases that test features that are explicitly disabled in this build. This new logic is intended for disabled features that aren't otherwise easily visible through the curl_version_info() or other API calls. tests/server/disabled is a newly built executable that will output a list of disabled features. Outputs nothing for a default build. Closes #3950
2019-06-15runtests: report single test time + total durationDaniel Stenberg
... after each successful test. Closes #4027
2019-06-10wolfssl: refer to it as wolfSSL onlyDaniel Stenberg
Remove support for, references to and use of "cyaSSL" from the source and docs. wolfSSL is the current name and there's no point in keeping references to ancient history. Assisted-by: Daniel Gustafsson Closes #3903
2019-04-15runtests: start socksd like other serversDaniel Stenberg
... without a $srcdir prefix. Triggered by the failures in several autobuilds. Closes #3781
2019-04-13test: make tests and test scripts use socksd for SOCKSDaniel Stenberg
Make all SOCKS tests use socksd instead of ssh.
2019-03-03alt-svc: the libcurl bitsDaniel Stenberg
2019-02-26runtests: detect "schannel" as an alias for "winssl"Daniel Stenberg
Follow-up to 180501cb02 Reported-by: Marcel Raad Fixes #3609 Closes #3620
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-24 16:32:14 +0000