aboutsummaryrefslogtreecommitdiff
path: root/lib/url.c
AgeCommit message (Collapse)Author
2017-11-09share: add support for sharing the connection cacheDaniel Stenberg
2017-11-06url: remove unncessary NULL-checkDaniel Stenberg
Since 'conn' won't be NULL in there and we also access the pointer in there without the check. Coverity CID 1420610
2017-11-05HTTP: support multiple Content-EncodingsPatrick Monnerat
This is implemented as an output streaming stack of unencoders, the last calling the client write procedure. New test 230 checks this feature. Bug: https://github.com/curl/curl/pull/2002 Reported-By: Daniel Bankhead
2017-11-04url: remove arg value check from CURLOPT_SSH_AUTH_TYPESJay Satiro
Since CURLSSH_AUTH_ANY (aka CURLSSH_AUTH_DEFAULT) is ~0 an arg value check on this option is incorrect; we have to accept any value. Prior to this change since f121575 (7.56.1+) CURLOPT_SSH_AUTH_TYPES erroneously rejected CURLSSH_AUTH_ANY with CURLE_BAD_FUNCTION_ARGUMENT. Bug: https://github.com/curl/curl/commit/f121575#commitcomment-25347120
2017-11-03CURLOPT_INFILESIZE: accept -1Daniel Stenberg
Regression since f121575 Reported-by: Petr Voytsik Fixes #2047
2017-11-02url: fix CURLOPT_DNS_CACHE_TIMEOUT arg value check to allow -1Jay Satiro
Prior to this change since f121575 (7.56.1+) CURLOPT_DNS_CACHE_TIMEOUT erroneously rejected -1 with CURLE_BAD_FUNCTION_ARGUMENT.
2017-11-01CURLOPT_MAXREDIRS: allow -1 as a valueDaniel Stenberg
... which is valid according to documentation. Regression since f121575c0b5f. Verified now in test 501. Reported-by: cbartl on github Fixes #2038 Closes #2039
2017-11-01url: fix CURLOPT_POSTFIELDSIZE arg value check to allow -1Jay Satiro
.. also add same arg value check to CURLOPT_POSTFIELDSIZE_LARGE. Prior to this change since f121575 (7.56.1+) CURLOPT_POSTFIELDSIZE erroneously rejected -1 value with CURLE_BAD_FUNCTION_ARGUMENT. Bug: https://curl.haxx.se/mail/lib-2017-11/0000.html Reported-by: Andrew Lambert
2017-10-28Curl_timeleft: change return type to timediff_tDaniel Stenberg
returning 'time_t' is problematic when that type is unsigned and we return values less than zero to signal "already expired", used in several places in the code. Closes #2021
2017-10-27setopt: fix CURLOPT_SSH_AUTH_TYPES option readDaniel Stenberg
Regression since f121575c0b5f Reported-by: Rob Cotrone
2017-10-26wildcards: don't use with non-supported protocolsMax Dymond
Fixes timeouts in the fuzzing tests for non-FTP protocols. Closes #2016
2017-10-25time: rename Curl_tvnow to Curl_nowDaniel Stenberg
... since the 'tv' stood for timeval and this function does not return a timeval struct anymore. Also, cleaned up the Curl_timediff*() functions to avoid typecasts and clean up the descriptive comments. Closes #2011
2017-10-25timediff: return timediff_t from the time diff functionsDaniel Stenberg
... to cater for systems with unsigned time_t variables. - Renamed the functions to curlx_timediff and Curl_timediff_us. - Added overflow protection for both of them in either direction for both 32 bit and 64 bit time_ts - Reprefixed the curlx_time functions to use Curl_* Reported-by: Peter Piekarski Fixes #2004 Closes #2005
2017-10-16setopt: range check most long optionsDaniel Stenberg
... filter early instead of risking "funny values" having to be dealt with elsewhere.
2017-10-16setopt: avoid integer overflows when setting millsecond valuesDaniel Stenberg
... that are multiplied by 1000 when stored. For 32 bit long systems, the max value accepted (2147483 seconds) is > 596 hours which is unlikely to ever be set by a legitimate application - and previously it didn't work either, it just caused undefined behavior. Also updated the man pages for these timeout options to mention the return code. Closes #1938
2017-10-11url: Update current connection SSL verify params in setoptArtak Galoyan
Now VERIFYHOST, VERIFYPEER and VERIFYSTATUS options change during active connection updates the current connection's (i.e.'connectdata' structure) appropriate ssl_config (and ssl_proxy_config) structures variables, making these options effective for ongoing connection. This functionality was available before and was broken by the following change: "proxy: Support HTTPS proxy and SOCKS+HTTP(s)" CommitId: cb4e2be7c6d42ca0780f8e0a747cecf9ba45f151. Bug: https://github.com/curl/curl/issues/1941 Closes https://github.com/curl/curl/pull/1951
2017-10-08mime: improve unbinding top multipart from easy handle.Patrick Monnerat
Also avoid dangling pointers in referencing parts.
2017-10-03idn: fix source code commentMichael Kaufmann
2017-10-03proxy: read the "no_proxy" variable only if necessaryMichael Kaufmann
Reviewed-by: Daniel Stenberg Closes #1919
2017-09-30Set and use more necessary options when some protocols are disabledDan Fandrich
When curl and libcurl are built with some protocols disabled, they stop setting and receiving some options that don't make sense with those protocols. In particular, when HTTP is disabled many options aren't set that are used only by HTTP. However, some options that appear to be HTTP-only are actually used by other protocols as well (some despite having HTTP in the name) and should be set, but weren't. This change now causes some of these options to be set and used for more (or for all) protocols. In particular, this fixes tests 646 through 649 in an HTTP-disabled build, which use the MIME API in the mail protocols.
2017-09-28reuse_conn: don't copy flags that are known to be equalMichael Kaufmann
A connection can only be reused if the flags "conn_to_host" and "conn_to_port" match. Therefore it is not necessary to copy these flags in reuse_conn(). Closes #1918
2017-09-14URL: on connection re-use, still pick the new remote portDaniel Stenberg
... as when a proxy connection is being re-used, it can still get a different remote port. Fixes #1887 Reported-by: Oli Kingshott
2017-09-11code style: use spaces around plusesDaniel Stenberg
2017-09-11code style: use spaces around equals signsDaniel Stenberg
2017-09-02mime: new MIME API.Patrick Monnerat
Available in HTTP, SMTP and IMAP. Deprecates the FORM API. See CURLOPT_MIMEPOST. Lib code and associated documentation.
2017-08-28vtls: encapsulate SSL backend-specific dataJohannes Schindelin
So far, all of the SSL backends' private data has been declared as part of the ssl_connect_data struct, in one big #if .. #elif .. #endif block. This can only work as long as the SSL backend is a compile-time option, something we want to change in the next commits. Therefore, let's encapsulate the exact data needed by each SSL backend into a private struct, and let's avoid bleeding any SSL backend-specific information into urldata.h. This is also necessary to allow multiple SSL backends to be compiled in at the same time, as e.g. OpenSSL's and CyaSSL's headers cannot be included in the same .c file. To avoid too many malloc() calls, we simply append the private structs to the connectdata struct in allocate_conn(). This requires us to take extra care of alignment issues: struct fields often need to be aligned on certain boundaries e.g. 32-bit values need to be stored at addresses that divide evenly by 4 (= 32 bit / 8 bit-per-byte). We do that by assuming that no SSL backend's private data contains any fields that need to be aligned on boundaries larger than `long long` (typically 64-bit) would need. Under this assumption, we simply add a dummy field of type `long long` to the `struct connectdata` struct. This field will never be accessed but acts as a placeholder for the four instances of ssl_backend_data instead. the size of each ssl_backend_data struct is stored in the SSL backend-specific metadata, to allow allocate_conn() to know how much extra space to allocate, and how to initialize the ssl[sockindex]->backend and proxy_ssl[sockindex]->backend pointers. This would appear to be a little complicated at first, but is really necessary to encapsulate the private data of each SSL backend correctly. And we need to encapsulate thusly if we ever want to allow selecting CyaSSL and OpenSSL at runtime, as their headers cannot be included within the same .c file (there are just too many conflicting definitions and declarations for that). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28vtls: move the SUPPORT_HTTPS_PROXY flag into the Curl_ssl structJohannes Schindelin
That will allow us to choose the SSL backend at runtime. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28vtls: convert the have_curlssl_* constants to runtime flagsJohannes Schindelin
The entire idea of introducing the Curl_ssl struct to describe SSL backends is to prepare for choosing the SSL backend at runtime. To that end, convert all the #ifdef have_curlssl_* style conditionals to use bit flags instead. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-17ssh: add the ability to enable compression (for SCP/SFTP)Viktor Szakats
The required low-level logic was already available as part of `libssh2` (via `LIBSSH2_FLAG_COMPRESS` `libssh2_session_flag()`[1] option.) This patch adds the new `libcurl` option `CURLOPT_SSH_COMPRESSION` (boolean) and the new `curl` command-line option `--compressed-ssh` to request this `libssh2` feature. To have compression enabled, it is required that the SSH server supports a (zlib) compatible compression method and that `libssh2` was built with `zlib` support enabled. [1] https://www.libssh2.org/libssh2_session_flag.html Ref: https://github.com/curl/curl/issues/1732 Closes https://github.com/curl/curl/pull/1735
2017-08-11parse_proxy(): fix memory leak in case of invalid proxy server nameEven Rouault
Fixes the below leak: $ valgrind --leak-check=full ~/install-curl-git/bin/curl --proxy "http://a:b@/x" http://127.0.0.1 curl: (5) Couldn't resolve proxy name ==5048== ==5048== HEAP SUMMARY: ==5048== in use at exit: 532 bytes in 12 blocks ==5048== total heap usage: 5,288 allocs, 5,276 frees, 445,271 bytes allocated ==5048== ==5048== 2 bytes in 1 blocks are definitely lost in loss record 1 of 12 ==5048== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==5048== by 0x4E6CB79: parse_login_details (url.c:5614) ==5048== by 0x4E6BA82: parse_proxy (url.c:5091) ==5048== by 0x4E6C46D: create_conn_helper_init_proxy (url.c:5346) ==5048== by 0x4E6EA18: create_conn (url.c:6498) ==5048== by 0x4E6F9B4: Curl_connect (url.c:6967) ==5048== by 0x4E86D05: multi_runsingle (multi.c:1436) ==5048== by 0x4E88432: curl_multi_perform (multi.c:2160) ==5048== by 0x4E7C515: easy_transfer (easy.c:708) ==5048== by 0x4E7C74A: easy_perform (easy.c:794) ==5048== by 0x4E7C7B1: curl_easy_perform (easy.c:813) ==5048== by 0x414025: operate_do (tool_operate.c:1563) ==5048== ==5048== 2 bytes in 1 blocks are definitely lost in loss record 2 of 12 ==5048== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==5048== by 0x4E6CBB6: parse_login_details (url.c:5621) ==5048== by 0x4E6BA82: parse_proxy (url.c:5091) ==5048== by 0x4E6C46D: create_conn_helper_init_proxy (url.c:5346) ==5048== by 0x4E6EA18: create_conn (url.c:6498) ==5048== by 0x4E6F9B4: Curl_connect (url.c:6967) ==5048== by 0x4E86D05: multi_runsingle (multi.c:1436) ==5048== by 0x4E88432: curl_multi_perform (multi.c:2160) ==5048== by 0x4E7C515: easy_transfer (easy.c:708) ==5048== by 0x4E7C74A: easy_perform (easy.c:794) ==5048== by 0x4E7C7B1: curl_easy_perform (easy.c:813) ==5048== by 0x414025: operate_do (tool_operate.c:1563) Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2984 Credit to OSS Fuzz for discovery Closes #1761
2017-07-28timeval: struct curltime is a struct timeval replacementDaniel Stenberg
... to make all libcurl internals able to use the same data types for the struct members. The timeval struct differs subtly on several platforms so it makes it cumbersome to use everywhere. Ref: #1652 Closes #1693
2017-06-30handler: refactor connection checkingMax Dymond
Add a new type of callback to Curl_handler which performs checks on the connection. Alter RTSP so that it uses this callback to do its own check on connection health.
2017-06-28CURLOPT_SOCKS5_AUTH: allowed methods for SOCKS5 proxy authKamil Dudka
If libcurl was built with GSS-API support, it unconditionally advertised GSS-API authentication while connecting to a SOCKS5 proxy. This caused problems in environments with improperly configured Kerberos: a stock libcurl failed to connect, despite libcurl built without GSS-API connected fine using username and password. This commit introduces the CURLOPT_SOCKS5_AUTH option to control the allowed methods for SOCKS5 authentication at run time. Note that a new option was preferred over reusing CURLOPT_PROXYAUTH for compatibility reasons because the set of authentication methods allowed by default was different for HTTP and SOCKS5 proxies. Bug: https://curl.haxx.se/mail/lib-2017-01/0005.html Closes https://github.com/curl/curl/pull/1454
2017-06-21--request-target: instead of --strip-path-slashDaniel Stenberg
... and CURLOPT_REQUEST_TARGET instead of CURLOPT_STRIP_PATH_SLASH. This option instead provides the full "alternative" target to use in the request, instead of extracting the path from the URL. Test 1298 and 1299 updated accordingly. Idea-by: Evert Pot Suggestion: https://daniel.haxx.se/blog/2017/06/19/options-with-curl/comment-page-1/#comment-18373 Closes #1593
2017-06-19http: add --strip-path-slash and CURLOPT_STRIP_PATH_SLASHDaniel Stenberg
... to enable sending "OPTIONS *" which wasn't possible previously. This option currently only works for HTTP. Added test cases 1298 + 1299 to verify Fixes #1280 Closes #1462
2017-06-16url: refactor the check for Windows drive letter in pathJay Satiro
- Move the logic to detect a Windows drive letter prefix (eg c: in c:foo) into a function-like macro. Closes https://github.com/curl/curl/pull/1571
2017-06-15http-proxy: only attempt FTP over HTTP proxyDaniel Stenberg
... all other non-HTTP protocol schemes are now defaulting to "tunnel trough" mode if a HTTP proxy is specified. In reality there are no HTTP proxies out there that allow those other schemes. Assisted-by: Ray Satiro, Michael Kaufmann Closes #1505
2017-06-14http-proxy: do the HTTP CONNECT process entirely non-blockingDaniel Stenberg
Mentioned as a problem since 2007 (8f87c15bdac63) and of course it existed even before that. Closes #1547
2017-06-13url: fix buffer overwrite with file protocol (CVE-2017-9502)Daniel Stenberg
Bug: https://github.com/curl/curl/issues/1540 Advisory: https://curl.haxx.se/docs/adv_20170614.html Assisted-by: Ray Satiro Reported-by: Marcel Raad
2017-06-09setopt: check CURLOPT_ADDRESS_SCOPE option rangeDaniel Stenberg
... and return error instead of triggering an assert() when being way out of range.
2017-06-05test1521: test *all* curl_easy_setopt optionsDaniel Stenberg
mk-lib1521.pl generates a test program (lib1521.c) that calls curl_easy_setopt() for every known option with a few typical values to make sure they work (ignoring the return codes). Some small changes were necessary to avoid asserts and NULL accesses when doing this. The perl script needs to be manually rerun when we add new options. Closes #1543
2017-05-23url: fix declaration of 'pipe' shadows a global declarationDaniel Stenberg
follow-up to 4cdb1be8246c
2017-05-13url.c: add a compile-time check that CURL_MAX_WRITE_SIZE is large enoughDan Fandrich
Some code (e.g. Curl_fillreadbuffer) assumes that this buffer is not exceedingly tiny and will break if it is. This same check is already done at run time in the CURLOPT_BUFFERSIZE option.
2017-05-12pipeline: fix mistakenly trying to pipeline POSTsDaniel Stenberg
The function IsPipeliningPossible() would return TRUE if either pipelining OR HTTP/2 were possible on a connection, which would lead to it returning TRUE even for POSTs on HTTP/1 connections. It now returns a bitmask so that the caller can differentiate which kind the connection allows. Fixes #1481 Closes #1483 Reported-by: stootill at github
2017-05-02url: let CURLOPT_BUFFERSIZE realloc to smaller sizes tooDaniel Stenberg
Closes #1449
2017-05-01BUFSIZE: rename to READBUFFER_*, make separate MASTERBUF_SIZEDaniel Stenberg
2017-05-01http-proxy: use a dedicated CONNECT response bufferDaniel Stenberg
To make it suitably independent of the receive buffer and its flexible size.
2017-05-01buffer_size: make sure it always has the correct sizeDaniel Stenberg
Removes the need for CURL_BUFSIZE
2017-05-01CURLOPT_BUFFERSIZE: 1024 bytes is now the minimum sizeDaniel Stenberg
The buffer is needed to receive FTP, HTTP CONNECT responses etc so already at this size things risk breaking and smaller is certainly not wise.
2017-04-29url: fixed a memory leak on OOM while setting CURLOPT_BUFFERSIZEDan Fandrich
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-02 12:00:03 +0000