aboutsummaryrefslogtreecommitdiff
path: root/lib/url.c
AgeCommit message (Collapse)Author
2017-09-30Set and use more necessary options when some protocols are disabledDan Fandrich
When curl and libcurl are built with some protocols disabled, they stop setting and receiving some options that don't make sense with those protocols. In particular, when HTTP is disabled many options aren't set that are used only by HTTP. However, some options that appear to be HTTP-only are actually used by other protocols as well (some despite having HTTP in the name) and should be set, but weren't. This change now causes some of these options to be set and used for more (or for all) protocols. In particular, this fixes tests 646 through 649 in an HTTP-disabled build, which use the MIME API in the mail protocols.
2017-09-28reuse_conn: don't copy flags that are known to be equalMichael Kaufmann
A connection can only be reused if the flags "conn_to_host" and "conn_to_port" match. Therefore it is not necessary to copy these flags in reuse_conn(). Closes #1918
2017-09-14URL: on connection re-use, still pick the new remote portDaniel Stenberg
... as when a proxy connection is being re-used, it can still get a different remote port. Fixes #1887 Reported-by: Oli Kingshott
2017-09-11code style: use spaces around plusesDaniel Stenberg
2017-09-11code style: use spaces around equals signsDaniel Stenberg
2017-09-02mime: new MIME API.Patrick Monnerat
Available in HTTP, SMTP and IMAP. Deprecates the FORM API. See CURLOPT_MIMEPOST. Lib code and associated documentation.
2017-08-28vtls: encapsulate SSL backend-specific dataJohannes Schindelin
So far, all of the SSL backends' private data has been declared as part of the ssl_connect_data struct, in one big #if .. #elif .. #endif block. This can only work as long as the SSL backend is a compile-time option, something we want to change in the next commits. Therefore, let's encapsulate the exact data needed by each SSL backend into a private struct, and let's avoid bleeding any SSL backend-specific information into urldata.h. This is also necessary to allow multiple SSL backends to be compiled in at the same time, as e.g. OpenSSL's and CyaSSL's headers cannot be included in the same .c file. To avoid too many malloc() calls, we simply append the private structs to the connectdata struct in allocate_conn(). This requires us to take extra care of alignment issues: struct fields often need to be aligned on certain boundaries e.g. 32-bit values need to be stored at addresses that divide evenly by 4 (= 32 bit / 8 bit-per-byte). We do that by assuming that no SSL backend's private data contains any fields that need to be aligned on boundaries larger than `long long` (typically 64-bit) would need. Under this assumption, we simply add a dummy field of type `long long` to the `struct connectdata` struct. This field will never be accessed but acts as a placeholder for the four instances of ssl_backend_data instead. the size of each ssl_backend_data struct is stored in the SSL backend-specific metadata, to allow allocate_conn() to know how much extra space to allocate, and how to initialize the ssl[sockindex]->backend and proxy_ssl[sockindex]->backend pointers. This would appear to be a little complicated at first, but is really necessary to encapsulate the private data of each SSL backend correctly. And we need to encapsulate thusly if we ever want to allow selecting CyaSSL and OpenSSL at runtime, as their headers cannot be included within the same .c file (there are just too many conflicting definitions and declarations for that). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28vtls: move the SUPPORT_HTTPS_PROXY flag into the Curl_ssl structJohannes Schindelin
That will allow us to choose the SSL backend at runtime. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28vtls: convert the have_curlssl_* constants to runtime flagsJohannes Schindelin
The entire idea of introducing the Curl_ssl struct to describe SSL backends is to prepare for choosing the SSL backend at runtime. To that end, convert all the #ifdef have_curlssl_* style conditionals to use bit flags instead. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-17ssh: add the ability to enable compression (for SCP/SFTP)Viktor Szakats
The required low-level logic was already available as part of `libssh2` (via `LIBSSH2_FLAG_COMPRESS` `libssh2_session_flag()`[1] option.) This patch adds the new `libcurl` option `CURLOPT_SSH_COMPRESSION` (boolean) and the new `curl` command-line option `--compressed-ssh` to request this `libssh2` feature. To have compression enabled, it is required that the SSH server supports a (zlib) compatible compression method and that `libssh2` was built with `zlib` support enabled. [1] https://www.libssh2.org/libssh2_session_flag.html Ref: https://github.com/curl/curl/issues/1732 Closes https://github.com/curl/curl/pull/1735
2017-08-11parse_proxy(): fix memory leak in case of invalid proxy server nameEven Rouault
Fixes the below leak: $ valgrind --leak-check=full ~/install-curl-git/bin/curl --proxy "http://a:b@/x" http://127.0.0.1 curl: (5) Couldn't resolve proxy name ==5048== ==5048== HEAP SUMMARY: ==5048== in use at exit: 532 bytes in 12 blocks ==5048== total heap usage: 5,288 allocs, 5,276 frees, 445,271 bytes allocated ==5048== ==5048== 2 bytes in 1 blocks are definitely lost in loss record 1 of 12 ==5048== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==5048== by 0x4E6CB79: parse_login_details (url.c:5614) ==5048== by 0x4E6BA82: parse_proxy (url.c:5091) ==5048== by 0x4E6C46D: create_conn_helper_init_proxy (url.c:5346) ==5048== by 0x4E6EA18: create_conn (url.c:6498) ==5048== by 0x4E6F9B4: Curl_connect (url.c:6967) ==5048== by 0x4E86D05: multi_runsingle (multi.c:1436) ==5048== by 0x4E88432: curl_multi_perform (multi.c:2160) ==5048== by 0x4E7C515: easy_transfer (easy.c:708) ==5048== by 0x4E7C74A: easy_perform (easy.c:794) ==5048== by 0x4E7C7B1: curl_easy_perform (easy.c:813) ==5048== by 0x414025: operate_do (tool_operate.c:1563) ==5048== ==5048== 2 bytes in 1 blocks are definitely lost in loss record 2 of 12 ==5048== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==5048== by 0x4E6CBB6: parse_login_details (url.c:5621) ==5048== by 0x4E6BA82: parse_proxy (url.c:5091) ==5048== by 0x4E6C46D: create_conn_helper_init_proxy (url.c:5346) ==5048== by 0x4E6EA18: create_conn (url.c:6498) ==5048== by 0x4E6F9B4: Curl_connect (url.c:6967) ==5048== by 0x4E86D05: multi_runsingle (multi.c:1436) ==5048== by 0x4E88432: curl_multi_perform (multi.c:2160) ==5048== by 0x4E7C515: easy_transfer (easy.c:708) ==5048== by 0x4E7C74A: easy_perform (easy.c:794) ==5048== by 0x4E7C7B1: curl_easy_perform (easy.c:813) ==5048== by 0x414025: operate_do (tool_operate.c:1563) Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2984 Credit to OSS Fuzz for discovery Closes #1761
2017-07-28timeval: struct curltime is a struct timeval replacementDaniel Stenberg
... to make all libcurl internals able to use the same data types for the struct members. The timeval struct differs subtly on several platforms so it makes it cumbersome to use everywhere. Ref: #1652 Closes #1693
2017-06-30handler: refactor connection checkingMax Dymond
Add a new type of callback to Curl_handler which performs checks on the connection. Alter RTSP so that it uses this callback to do its own check on connection health.
2017-06-28CURLOPT_SOCKS5_AUTH: allowed methods for SOCKS5 proxy authKamil Dudka
If libcurl was built with GSS-API support, it unconditionally advertised GSS-API authentication while connecting to a SOCKS5 proxy. This caused problems in environments with improperly configured Kerberos: a stock libcurl failed to connect, despite libcurl built without GSS-API connected fine using username and password. This commit introduces the CURLOPT_SOCKS5_AUTH option to control the allowed methods for SOCKS5 authentication at run time. Note that a new option was preferred over reusing CURLOPT_PROXYAUTH for compatibility reasons because the set of authentication methods allowed by default was different for HTTP and SOCKS5 proxies. Bug: https://curl.haxx.se/mail/lib-2017-01/0005.html Closes https://github.com/curl/curl/pull/1454
2017-06-21--request-target: instead of --strip-path-slashDaniel Stenberg
... and CURLOPT_REQUEST_TARGET instead of CURLOPT_STRIP_PATH_SLASH. This option instead provides the full "alternative" target to use in the request, instead of extracting the path from the URL. Test 1298 and 1299 updated accordingly. Idea-by: Evert Pot Suggestion: https://daniel.haxx.se/blog/2017/06/19/options-with-curl/comment-page-1/#comment-18373 Closes #1593
2017-06-19http: add --strip-path-slash and CURLOPT_STRIP_PATH_SLASHDaniel Stenberg
... to enable sending "OPTIONS *" which wasn't possible previously. This option currently only works for HTTP. Added test cases 1298 + 1299 to verify Fixes #1280 Closes #1462
2017-06-16url: refactor the check for Windows drive letter in pathJay Satiro
- Move the logic to detect a Windows drive letter prefix (eg c: in c:foo) into a function-like macro. Closes https://github.com/curl/curl/pull/1571
2017-06-15http-proxy: only attempt FTP over HTTP proxyDaniel Stenberg
... all other non-HTTP protocol schemes are now defaulting to "tunnel trough" mode if a HTTP proxy is specified. In reality there are no HTTP proxies out there that allow those other schemes. Assisted-by: Ray Satiro, Michael Kaufmann Closes #1505
2017-06-14http-proxy: do the HTTP CONNECT process entirely non-blockingDaniel Stenberg
Mentioned as a problem since 2007 (8f87c15bdac63) and of course it existed even before that. Closes #1547
2017-06-13url: fix buffer overwrite with file protocol (CVE-2017-9502)Daniel Stenberg
Bug: https://github.com/curl/curl/issues/1540 Advisory: https://curl.haxx.se/docs/adv_20170614.html Assisted-by: Ray Satiro Reported-by: Marcel Raad
2017-06-09setopt: check CURLOPT_ADDRESS_SCOPE option rangeDaniel Stenberg
... and return error instead of triggering an assert() when being way out of range.
2017-06-05test1521: test *all* curl_easy_setopt optionsDaniel Stenberg
mk-lib1521.pl generates a test program (lib1521.c) that calls curl_easy_setopt() for every known option with a few typical values to make sure they work (ignoring the return codes). Some small changes were necessary to avoid asserts and NULL accesses when doing this. The perl script needs to be manually rerun when we add new options. Closes #1543
2017-05-23url: fix declaration of 'pipe' shadows a global declarationDaniel Stenberg
follow-up to 4cdb1be8246c
2017-05-13url.c: add a compile-time check that CURL_MAX_WRITE_SIZE is large enoughDan Fandrich
Some code (e.g. Curl_fillreadbuffer) assumes that this buffer is not exceedingly tiny and will break if it is. This same check is already done at run time in the CURLOPT_BUFFERSIZE option.
2017-05-12pipeline: fix mistakenly trying to pipeline POSTsDaniel Stenberg
The function IsPipeliningPossible() would return TRUE if either pipelining OR HTTP/2 were possible on a connection, which would lead to it returning TRUE even for POSTs on HTTP/1 connections. It now returns a bitmask so that the caller can differentiate which kind the connection allows. Fixes #1481 Closes #1483 Reported-by: stootill at github
2017-05-02url: let CURLOPT_BUFFERSIZE realloc to smaller sizes tooDaniel Stenberg
Closes #1449
2017-05-01BUFSIZE: rename to READBUFFER_*, make separate MASTERBUF_SIZEDaniel Stenberg
2017-05-01http-proxy: use a dedicated CONNECT response bufferDaniel Stenberg
To make it suitably independent of the receive buffer and its flexible size.
2017-05-01buffer_size: make sure it always has the correct sizeDaniel Stenberg
Removes the need for CURL_BUFSIZE
2017-05-01CURLOPT_BUFFERSIZE: 1024 bytes is now the minimum sizeDaniel Stenberg
The buffer is needed to receive FTP, HTTP CONNECT responses etc so already at this size things risk breaking and smaller is certainly not wise.
2017-04-29url: fixed a memory leak on OOM while setting CURLOPT_BUFFERSIZEDan Fandrich
2017-04-26lib: remove unused codeMarcel Raad
This fixes the following clang warnings: macro is not used [-Wunused-macros] will never be executed [-Wunreachable-code] Closes https://github.com/curl/curl/pull/1448
2017-04-26url: declare get_protocol_family() staticMartin Kepplinger
get_protocol_family() is not defined static even though there is a static local forward declaration. Let's simply make the definition match it's declaration. Bug: https://curl.haxx.se/mail/lib-2017-04/0127.html
2017-04-25transfer: remove 'uploadbuf' pointer and cleanup readwrite_upload()Daniel Stenberg
The data->req.uploadbuf struct member served no good purpose, instead we use ->state.uploadbuffer directly. It makes it clearer in the code which buffer that's being used. Removed the 'SingleRequest *' argument from the readwrite_upload() proto as it can be derived from the Curl_easy struct. Also made the code in the readwrite_upload() function use the 'k->' shortcut to all references to struct fields in 'data->req', which previously was made with a mix of both.
2017-04-18TLS: Fix switching off SSL session id when client cert is usedJay Satiro
Move the sessionid flag to ssl_primary_config so that ssl and proxy_ssl will each have their own sessionid flag. Regression since HTTPS-Proxy support was added in cb4e2be. Prior to that this issue had been fixed in 247d890, CVE-2016-5419. Bug: https://github.com/curl/curl/issues/1341 Reported-by: lijian996@users.noreply.github.com The new incarnation of this bug is called CVE-2017-7468 and is documented here: https://curl.haxx.se/docs/adv_20170419.html
2017-04-04url: don't free postponed data on connection reuseAntti Hätälä
- Don't free postponed data on a connection that will be reused since doing so can cause data loss when pipelining. Only Windows builds are affected by this. Closes https://github.com/curl/curl/issues/1380
2017-04-04llist: replace Curl_llist_alloc with Curl_llist_initDaniel Stenberg
No longer allocate the curl_llist head struct for lists separately. Removes 17 (15%) tiny allocations in a normal "curl localhost" invoke. closes #1381
2017-03-28http: Fix proxy connection reuse with basic-authIsaac Boukris
When using basic-auth, connections and proxy connections can be re-used with different Authorization headers since it does not authenticate the connection (like NTLM does). For instance, the below command should re-use the proxy connection, but it currently doesn't: curl -v -U alice:a -x http://localhost:8181 http://localhost/ --next -U bob:b -x http://localhost:8181 http://localhost/ This is a regression since refactoring of ConnectionExists() as part of: cb4e2be7c6d42ca0780f8e0a747cecf9ba45f151 Fix the above by removing the username and password compare when re-using proxy connection at proxy_info_matches(). However, this fix brings back another bug would make curl to re-print the old proxy-authorization header of previous proxy basic-auth connection because it wasn't cleared. For instance, in the below command the second request should fail if the proxy requires authentication, but would succeed after the above fix (and before aforementioned commit): curl -v -U alice:a -x http://localhost:8181 http://localhost/ --next -x http://localhost:8181 http://localhost/ Fix this by clearing conn->allocptr.proxyuserpwd after use unconditionally, same as we do for conn->allocptr.userpwd. Also fix test 540 to not expect digest auth header to be resent when connection is reused. Signed-off-by: Isaac Boukris <iboukris@gmail.com> Closes https://github.com/curl/curl/pull/1350
2017-03-26spelling fixesklemens
Closes #1356
2017-03-13Improve code readbilitySylvestre Ledru
... by removing the else branch after a return, break or continue. Closes #1310
2017-03-12url: add option CURLOPT_SUPPRESS_CONNECT_HEADERSDesmond O. Chang
- Add new option CURLOPT_SUPPRESS_CONNECT_HEADERS to allow suppressing proxy CONNECT response headers from the user callback functions CURLOPT_HEADERFUNCTION and CURLOPT_WRITEFUNCTION. - Add new tool option --suppress-connect-headers to expose CURLOPT_SUPPRESS_CONNECT_HEADERS and allow suppressing proxy CONNECT response headers from --dump-header and --include. Assisted-by: Jay Satiro Assisted-by: CarloCannas@users.noreply.github.com Closes https://github.com/curl/curl/pull/783
2017-03-11url: don't compile detect_proxy if HTTP support is disabledDan Fandrich
2017-03-08vtls: add options to specify range of enabled TLS versionsJozef Kralik
This commit introduces the CURL_SSLVERSION_MAX_* constants as well as the --tls-max option of the curl tool. Closes https://github.com/curl/curl/pull/1166
2017-03-06URL: return error on malformed URLs with junk after port numberDaniel Stenberg
... because it causes confusion with users. Example URLs: "http://[127.0.0.1]:11211:80" which a lot of languages' URL parsers will parse and claim uses port number 80, while libcurl would use port number 11211. "http://user@example.com:80@localhost" which by the WHATWG URL spec will be treated to contain user name 'user@example.com' but according to RFC3986 is user name 'user' for the host 'example.com' and then port 80 is followed by "@localhost" Both these formats are now rejected, and verified so in test 1260. Reported-by: Orange Tsai
2017-03-03url: split off proxy init and parsing from create_connJay Satiro
Move the proxy parse/init into helper create_conn_helper_init_proxy to mitigate the chances some non-proxy code will be mistakenly added to it. Ref: https://github.com/curl/curl/issues/1274#issuecomment-281556510 Ref: https://github.com/curl/curl/pull/1293 Closes https://github.com/curl/curl/pull/1298
2017-02-25url: fix unix-socket support for proxy-disabled buildsIsaac Boukris
Prior to this change if curl was built with Unix Socket support (--enable-unix-sockets) and without Proxy support (--disable-proxy) then unix socket options would erroneously be ignored. Regression introduced in: 0b8d682f81ee9acb763dd4c9ad805fe08d1227c0 Bug: https://github.com/curl/curl/issues/1274 Reported-by: mccormickt12@users.noreply.github.com Closes https://github.com/curl/curl/pull/1289
2017-02-25proxy: fixed a memory leak on OOMDan Fandrich
2017-02-23url: Default the CA proxy bundle location to CURL_CA_BUNDLEJay Satiro
If the compile-time CURL_CA_BUNDLE location is defined use it as the default value for the proxy CA bundle location, which is the same as what we already do for the regular CA bundle location. Ref: https://github.com/curl/curl/pull/1257
2017-02-21url: Improve CURLOPT_PROXY_CAPATH error handlingJay Satiro
- Change CURLOPT_PROXY_CAPATH to return CURLE_NOT_BUILT_IN if the option is not supported, which is the same as what we already do for CURLOPT_CAPATH. - Change the curl tool to handle CURLOPT_PROXY_CAPATH error CURLE_NOT_BUILT_IN as a warning instead of as an error, which is the same as what we already do for CURLOPT_CAPATH. - Fix CAPATH docs to show that CURLE_NOT_BUILT_IN is returned when the respective CAPATH option is not supported by the SSL library. Ref: https://github.com/curl/curl/pull/1257
2017-02-21TLS: make SSL_VERIFYSTATUS work againDaniel Stenberg
The CURLOPT_SSL_VERIFYSTATUS option was not properly handled by libcurl and thus even if the status couldn't be verified, the connection would be allowed and the user would not be told about the failed verification. Regression since cb4e2be7c6d42ca CVE-2017-2629 Bug: https://curl.haxx.se/docs/adv_20170222.html Reported-by: Marcus Hoffmann
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-01 17:45:56 +0000