aboutsummaryrefslogtreecommitdiff
path: root/lib/url.c
AgeCommit message (Collapse)Author
2013-04-18url: Reworked URL parsing to allow overriding by CURLOPT_USERPWDSteve Holme
2013-04-13url: Added support for parsing login options from the URLSteve Holme
As well as parsing the username and password from the URL, added support for parsing the optional options part from the login details, to allow the following supported URL format: schema://username:password;options@example.com/path?q=foobar This will only be used by IMAP, POP3 and SMTP at present but any protocol that may be given login options in the URL will be able to add support for them.
2013-04-08proxy: make ConnectionExists() check credential of proxyconnections tooFabian Keil
Previously it only compared credentials if the requested needle connection wasn't using a proxy. This caused NTLM authentication failures when using proxies as the authentication code wasn't send on the connection where the challenge arrived. Added test 1215 to verify: NTLM server authentication through a proxy (This is a modified copy of test 67)
2013-04-05Curl_open: restore default MAXCONNECTS to 5Daniel Stenberg
At some point recently we lost the default value for the easy handle's connection cache, and this change puts it back to 5 - which is the former default value and it is documented in the curl_easy_setopt.3 man page.
2013-03-25NTLM: fix several NTLM code paths memory leaksYang Tse
2013-03-15HTTP proxy: insert slash in URL if missingDaniel Stenberg
curl has been accepting URLs using slightly wrong syntax for a long time, such as when completely missing as slash "http://example.org" or missing a slash when a query part is given "http://example.org?q=foobar". curl would translate these into a legitimate HTTP request to servers, although as was shown in bug #1206 it was not adjusted properly in the cases where a HTTP proxy was used. Test 1213 and 1214 were added to the test suite to verify this fix. The test HTTP server was adjusted to allow us to specify test number in the host name only without using any slashes in a given URL. Bug: http://curl.haxx.se/bug/view.cgi?id=1206 Reported by: ScottJi
2013-03-13Multiple pipelines and limiting the number of connections.Linus Nielsen Feltzing
Introducing a number of options to the multi interface that allows for multiple pipelines to the same host, in order to optimize the balance between the penalty for opening new connections and the potential pipelining latency. Two new options for limiting the number of connections: CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections to the same host. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished, so we can reuse the connection. CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished. The free connection will then be reused, if possible, or closed if the pending handle can't reuse it. Several new options for pipelining: CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a pipeline is "full" when a connection is to be reused, a new connection will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it. If not, the handle will be put in a pending state until a connection is ready (either free or a pipe got shorter). CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a transfer with a content length that is larger than this. CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a chunk larger than this. CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow pipelining. CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow pipelining. See the curl_multi_setopt() man page for details.
2013-02-22lib/url.c: Generic read/write data pointersAlexander Klauer
Always interprets the pointer passed with the CURLOPT_WRITEDATA or CURLOPT_READDATA options of curl_easy_setopt() as a void pointer in order to avoid problems in environments where FILE and void pointers have non-trivial conversion.
2013-02-11SOCKS: fix socks proxy when noproxy matchedulion
Test 1212 added to verify Bug: http://curl.haxx.se/bug/view.cgi?id=1190
2013-02-08DONE: consider callback-aborted transfers prematureDaniel Stenberg
This bug report properly identified that when doing SMTP and aborting the transfer with a callback, it must be considered aborted prematurely by the code to avoid QUIT etc to be attempted as that would cause a hang. The new test case 1507 verifies this behavior. Reported by: Patricia Muscalu Bug: http://curl.haxx.se/bug/view.cgi?id=1184
2013-02-05CURLMOPT_MAXCONNECTS: restore functionalityLinus Nielsen Feltzing
When a connection is no longer used, it is kept in the cache. If the cache is full, the oldest idle connection is closed. If no connection is idle, the current one is closed instead.
2013-01-17always-multi: always use non-blocking internalsDaniel Stenberg
Remove internal separated behavior of the easy vs multi intercace. curl_easy_perform() is now using the multi interface itself. Several minor multi interface quirks and bugs have been fixed in the process. Much help with debugging this has been provided by: Yang Tse
2013-01-17url.c: fix HTTP CONNECT tunnel establishment upon delayed responseYang Tse
Fixes initial proxy response being processed by the tunneled protocol handler instead of the HTTP wrapper handler. This issue would trigger upon delayed CONNECT response from the proxy. Additionally fixes a multi interface code-path in which connections would not time out properly. This does not fix known bug #39. URL: http://curl.haxx.se/mail/lib-2013-01/0191.html
2013-01-09build: fix circular header inclusion with other packagesYang Tse
This commit renames lib/setup.h to lib/curl_setup.h and renames lib/setup_once.h to lib/curl_setup_once.h. Removes the need and usage of a header inclusion guard foreign to libcurl. [1] Removes the need and presence of an alarming notice we carried in old setup_once.h [2] ---------------------------------------- 1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H, this single inclusion guard is enough to ensure that inclusion of lib/setup_once.h done from lib/setup.h is only done once. Additionally lib/setup.h has always used __SETUP_ONCE_H macro to protect inclusion of setup_once.h even after commit ec691ca3, this was to avoid a circular header inclusion triggered when building a c-ares enabled version with c-ares sources available which also has a setup_once.h header. Commit ec691ca3 exposes the real nature of __SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard foreign to libcurl belonging to c-ares's setup_once.h The renaming this commit does, fixes the circular header inclusion, and as such removes the need and usage of a header inclusion guard foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl. 2 - Due to the circular interdependency of old lib/setup_once.h and the c-ares setup_once.h header, old file lib/setup_once.h has carried back from 2006 up to now days an alarming and prominent notice about the need of keeping libcurl's and c-ares's setup_once.h in sync. Given that this commit fixes the circular interdependency, the need and presence of mentioned notice is removed. All mentioned interdependencies come back from now old days when the c-ares project lived inside a curl subdirectory. This commit removes last traces of such fact.
2013-01-06Revert changes relative to lib/*.[ch] recent renamingYang Tse
This reverts renaming and usage of lib/*.h header files done 28-12-2012, reverting 2 commits: f871de0... build: make use of 76 lib/*.h renamed files ffd8e12... build: rename 76 lib/*.h files This also reverts removal of redundant include guard (redundant thanks to changes in above commits) done 2-12-2013, reverting 1 commit: c087374... curl_setup.h: remove redundant include guard This also reverts renaming and usage of lib/*.c source files done 3-12-2013, reverting 3 commits: 13606bb... build: make use of 93 lib/*.c renamed files 5b6e792... build: rename 93 lib/*.c files 7d83dff... build: commit 13606bbfde follow-up 1 Start of related discussion thread: http://curl.haxx.se/mail/lib-2013-01/0012.html Asking for confirmation on pushing this revertion commit: http://curl.haxx.se/mail/lib-2013-01/0048.html Confirmation summary: http://curl.haxx.se/mail/lib-2013-01/0079.html NOTICE: The list of 2 files that have been modified by other intermixed commits, while renamed, and also by at least one of the 6 commits this one reverts follows below. These 2 files will exhibit a hole in history unless git's '--follow' option is used when viewing logs. lib/curl_imap.h lib/curl_smtp.h
2013-01-03build: rename 93 lib/*.c filesYang Tse
93 lib/*.c source files renamed to use our standard naming scheme. This commit only does the file renaming. ---------------------------------------- renamed: lib/amigaos.c -> lib/curl_amigaos.c renamed: lib/asyn-ares.c -> lib/curl_asyn_ares.c renamed: lib/asyn-thread.c -> lib/curl_asyn_thread.c renamed: lib/axtls.c -> lib/curl_axtls.c renamed: lib/base64.c -> lib/curl_base64.c renamed: lib/bundles.c -> lib/curl_bundles.c renamed: lib/conncache.c -> lib/curl_conncache.c renamed: lib/connect.c -> lib/curl_connect.c renamed: lib/content_encoding.c -> lib/curl_content_encoding.c renamed: lib/cookie.c -> lib/curl_cookie.c renamed: lib/cyassl.c -> lib/curl_cyassl.c renamed: lib/dict.c -> lib/curl_dict.c renamed: lib/easy.c -> lib/curl_easy.c renamed: lib/escape.c -> lib/curl_escape.c renamed: lib/file.c -> lib/curl_file.c renamed: lib/fileinfo.c -> lib/curl_fileinfo.c renamed: lib/formdata.c -> lib/curl_formdata.c renamed: lib/ftp.c -> lib/curl_ftp.c renamed: lib/ftplistparser.c -> lib/curl_ftplistparser.c renamed: lib/getenv.c -> lib/curl_getenv.c renamed: lib/getinfo.c -> lib/curl_getinfo.c renamed: lib/gopher.c -> lib/curl_gopher.c renamed: lib/gtls.c -> lib/curl_gtls.c renamed: lib/hash.c -> lib/curl_hash.c renamed: lib/hmac.c -> lib/curl_hmac.c renamed: lib/hostasyn.c -> lib/curl_hostasyn.c renamed: lib/hostcheck.c -> lib/curl_hostcheck.c renamed: lib/hostip.c -> lib/curl_hostip.c renamed: lib/hostip4.c -> lib/curl_hostip4.c renamed: lib/hostip6.c -> lib/curl_hostip6.c renamed: lib/hostsyn.c -> lib/curl_hostsyn.c renamed: lib/http.c -> lib/curl_http.c renamed: lib/http_chunks.c -> lib/curl_http_chunks.c renamed: lib/http_digest.c -> lib/curl_http_digest.c renamed: lib/http_negotiate.c -> lib/curl_http_negotiate.c renamed: lib/http_negotiate_sspi.c -> lib/curl_http_negotiate_sspi.c renamed: lib/http_proxy.c -> lib/curl_http_proxy.c renamed: lib/idn_win32.c -> lib/curl_idn_win32.c renamed: lib/if2ip.c -> lib/curl_if2ip.c renamed: lib/imap.c -> lib/curl_imap.c renamed: lib/inet_ntop.c -> lib/curl_inet_ntop.c renamed: lib/inet_pton.c -> lib/curl_inet_pton.c renamed: lib/krb4.c -> lib/curl_krb4.c renamed: lib/krb5.c -> lib/curl_krb5.c renamed: lib/ldap.c -> lib/curl_ldap.c renamed: lib/llist.c -> lib/curl_llist.c renamed: lib/md4.c -> lib/curl_md4.c renamed: lib/md5.c -> lib/curl_md5.c renamed: lib/memdebug.c -> lib/curl_memdebug.c renamed: lib/mprintf.c -> lib/curl_mprintf.c renamed: lib/multi.c -> lib/curl_multi.c renamed: lib/netrc.c -> lib/curl_netrc.c renamed: lib/non-ascii.c -> lib/curl_non_ascii.c renamed: lib/curl_non-ascii.h -> lib/curl_non_ascii.h renamed: lib/nonblock.c -> lib/curl_nonblock.c renamed: lib/nss.c -> lib/curl_nss.c renamed: lib/nwlib.c -> lib/curl_nwlib.c renamed: lib/nwos.c -> lib/curl_nwos.c renamed: lib/openldap.c -> lib/curl_openldap.c renamed: lib/parsedate.c -> lib/curl_parsedate.c renamed: lib/pingpong.c -> lib/curl_pingpong.c renamed: lib/polarssl.c -> lib/curl_polarssl.c renamed: lib/pop3.c -> lib/curl_pop3.c renamed: lib/progress.c -> lib/curl_progress.c renamed: lib/qssl.c -> lib/curl_qssl.c renamed: lib/rawstr.c -> lib/curl_rawstr.c renamed: lib/rtsp.c -> lib/curl_rtsp.c renamed: lib/security.c -> lib/curl_security.c renamed: lib/select.c -> lib/curl_select.c renamed: lib/sendf.c -> lib/curl_sendf.c renamed: lib/share.c -> lib/curl_share.c renamed: lib/slist.c -> lib/curl_slist.c renamed: lib/smtp.c -> lib/curl_smtp.c renamed: lib/socks.c -> lib/curl_socks.c renamed: lib/socks_gssapi.c -> lib/curl_socks_gssapi.c renamed: lib/socks_sspi.c -> lib/curl_socks_sspi.c renamed: lib/speedcheck.c -> lib/curl_speedcheck.c renamed: lib/splay.c -> lib/curl_splay.c renamed: lib/ssh.c -> lib/curl_ssh.c renamed: lib/sslgen.c -> lib/curl_sslgen.c renamed: lib/ssluse.c -> lib/curl_ssluse.c renamed: lib/strdup.c -> lib/curl_strdup.c renamed: lib/strequal.c -> lib/curl_strequal.c renamed: lib/strerror.c -> lib/curl_strerror.c renamed: lib/strtok.c -> lib/curl_strtok.c renamed: lib/strtoofft.c -> lib/curl_strtoofft.c renamed: lib/telnet.c -> lib/curl_telnet.c renamed: lib/tftp.c -> lib/curl_tftp.c renamed: lib/timeval.c -> lib/curl_timeval.c renamed: lib/transfer.c -> lib/curl_transfer.c renamed: lib/url.c -> lib/curl_url.c renamed: lib/version.c -> lib/curl_version.c renamed: lib/warnless.c -> lib/curl_warnless.c renamed: lib/wildcard.c -> lib/curl_wildcard.c ----------------------------------------
2013-01-03build: make use of 93 lib/*.c renamed filesYang Tse
93 *.c source files renamed to use our standard naming scheme. This change affects 77 files in libcurl's source tree.
2012-12-28build: make use of 76 lib/*.h renamed filesYang Tse
76 private header files renamed to use our standard naming scheme. This change affects 322 files in libcurl's source tree.
2012-12-19bundles connection caching: some out of memory handling fixesYang Tse
2012-12-14setup_once.h: refactor inclusion of <unistd.h> and <sys/socket.h>Yang Tse
Inclusion of top two most included header files now done in setup_once.h
2012-12-07Introducing a new persistent connection caching system using "bundles".Linus Nielsen Feltzing
A bundle is a list of all persistent connections to the same host. The connection cache consists of a hash of bundles, with the hostname as the key. The benefits may not be obvious, but they are two: 1) Faster search for connections to reuse, since the hash lookup only finds connections to the host in question. 2) It lays out the groundworks for an upcoming patch, which will introduce multiple HTTP pipelines. This patch also removes the awkward list of "closure handles", which were needed to send QUIT commands to the FTP server when closing a connection. Now we allocate a separate closure handle and use that one to close all connections. This has been tested in a live system for a few weeks, and of course passes the test suite.
2012-11-26avoid mixing of enumerated type with another typeYang Tse
2012-11-06URL parser: cut off '#' fragments from URLs (better)Daniel Stenberg
The existing logic only cut off the fragment from the separate 'path' buffer which is used when sending HTTP to hosts. The buffer that held the full URL used for proxies were not dealt with. It is now. Test case 5 was updated to use a fragment on a URL over a proxy. Bug: http://curl.haxx.se/bug/view.cgi?id=3579813
2012-11-06CURLOPT_SSL_VERIFYHOST: stop supporting the 1 valueDaniel Stenberg
After a research team wrote a document[1] that found several live source codes out there in the wild that misused the CURLOPT_SSL_VERIFYHOST option thinking it was a boolean, this change now bans 1 as a value and will make libcurl return error for it. 1 was never a sensible value to use in production but was introduced back in the days to help debugging. It was always documented clearly this way. 1 was never supported by all SSL backends in libcurl, so this cleanup makes the treatment of it unified. The report's list of mistakes for this option were all PHP code and while there's a binding layer between libcurl and PHP, the PHP team has decided that they have an as thin layer as possible on top of libcurl so they will not alter or specifically filter a 'TRUE' value for this particular option. I sympathize with that position. [1] = http://daniel.haxx.se/blog/2012/10/25/libcurl-claimed-to-be-dangerous/
2012-10-23tlsauthtype: deal with the string case insensitivelyDaniel Stenberg
When given a string as 'srp' it didn't work, but required 'SRP'. Starting now, the check disregards casing. Bug: http://curl.haxx.se/bug/view.cgi?id=3578418 Reported by: Jeff Connelly
2012-09-13parse_proxy: treat "socks://x" as a socks4 proxyDaniel Stenberg
Selected socks proxy in Google's Chrome browser. Resulting in the following environment variables: NO_PROXY=localhost,127.0.0.0/8 ALL_PROXY=socks://localhost:1080/ all_proxy=socks://localhost:1080/ no_proxy=localhost,127.0.0.0/8 ... and libcurl didn't treat 'socks://' as socks but instead picked HTTP proxy. Reported by: Scott Bailey Bug: http://curl.haxx.se/bug/view.cgi?id=3566860
2012-08-31NTLM: re-use existing connection betterJoe Mason
If we need an NTLM connection and one already exists, always choose that one.
2012-08-08verbose messages: fixed output of hostnames in re-used connectionsDaniel Stenberg
I suspect this is a regression introduced in commit 207cf150, included since 7.24.0. Avoid showing '(nil)' as hostname in verbose output by making sure the hostname fixup function is called early enough to set the pointers that are used for this. The name data is set again for each request even for re-used connections to handle multiple hostnames over the same connection (like with proxy) or that the casing etc of the host name is changed between requests (which has proven to be important at least once in the past). Test1011 was modified to use a redirect with a re-used a connection since it then showed the bug and now lo longer does. There's currently no easy way to have the test suite detect 'nil' texts in verbose ouputs so no tests will detect if this problem gets reintroduced. Bug: http://curl.haxx.se/mail/lib-2012-07/0111.html Reported by: Gisle Vanem
2012-08-07Avoid leak of local device string when reusing connectionMike Crowe
Ensure that the copy of the CURLOPT_INTERFACE string is freed if we decide we can reuse an existing connection.
2012-07-13parse_proxy: remove dead assignmentDaniel Stenberg
Spotted by clang-analyzer
2012-06-10Curl_pgrsDone: return int and acknowledge return codeDaniel Stenberg
Since Curl_pgrsDone() itself calls Curl_pgrsUpdate() which may return an abort instruction or similar we need to return that info back and subsequently properly handle return codes from Curl_pgrsDone() where used. (Spotted by a Coverity scan)
2012-04-23URL parse: reject numerical IPv6 addresses outside bracketsDaniel Stenberg
Roman Mamedov spotted (in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=670126) that curl would not complain when given a URL with an IPv6 numerical address without brackets. It would simply cut off the last ":[hex]" part and thus not work correctly. That's a URL using an illegal syntax and now libcurl will instead return a clear error code and error message detailing the error. The above mentioned bug report claims this to be a regression but libcurl does not guarantee functionality when given URLs that aren't following the URL spec (RFC3986 mostly). I consider the fact that it used to handle this differently a mere coincidence.
2012-04-19url.c: CURLOPT_HTTPAUTH and CURLOPT_PROXYAUTH fixesYang Tse
Fail with CURLE_NOT_BUILT_IN when none of requested auth methods is supported. Reject CURLAUTH_ONLY bit when given alone or with CURLAUTH_NONE.
2012-04-18Take in account that CURLAUTH_* bitmasks are now 'unsigned long'Yang Tse
Data type of internal vars holding CURLAUTH_* bitmasks changed from 'long' to 'unsigned long' for proper handling and operating.
2012-04-12url.c: fix compiler warningYang Tse
2012-04-05CURLOPT_POSTREDIR: also allow 303 to do POST on the redirected URLAndrei Cipu
As it turns out, some people do want that after all.
2012-03-22CONNECT: fix multi interface regressionDaniel Stenberg
The refactoring of HTTP CONNECT handling in commit 41b0237834232 that made it protocol independent broke it for the multi interface. This fix now introduce a better state handling and moved some logic to the http_proxy.c source file. Reported by: Yang Tse Bug: http://curl.haxx.se/mail/lib-2012-03/0162.html
2012-03-16fix some compiler warningsYang Tse
2012-03-08CONNECT: made generically not per-protocolDaniel Stenberg
Curl_protocol_connect() now does the tunneling through the HTTP proxy if requested instead of letting each protocol specific connection function do it.
2012-02-14smtp_mail: Added support to MAIL FROM for the optional AUTH parameterSteve Holme
Added a new CURLOPT_MAIL_AUTH option that allows the calling program to set the optional AUTH parameter in the MAIL FROM command. When this option is specified and an authentication mechanism is used to communicate with the mail server then the AUTH parameter will be included in the MAIL FROM command. This is particularly useful when the calling program is acting as a relay in a trusted environment and performing server to server communication, as it allows the relaying server to specify the address of the mailbox that was used to authenticate and send the original email.
2012-02-10parse_proxy: simply memory handlingDaniel Stenberg
... by making sure that the string is always freed after the invoke as parse_proxy will always copy the data and this way there's a single free() instead of multiple ones.
2012-02-10parse_proxy: bail out on zero-length proxy names!Daniel Stenberg
The proxy parser function strips off trailing slashes off the proxy name which could lead to a mistaken zero length proxy name which would be treated as no proxy at all by subsequent functions! This is now detected and an error is returned. Verified by the new test 1329. Reported by: Chandrakant Bagul Bug: http://curl.haxx.se/mail/lib-2012-02/0000.html
2012-02-09CURLOPT_SSL_OPTIONS: addedDaniel Stenberg
Allow an appliction to set libcurl specific SSL options. The first and only options supported right now is CURLSSLOPT_ALLOW_BEAST. It will make libcurl to disable any work-arounds the underlying SSL library may have to address a known security flaw in the SSL3 and TLS1.0 protocol versions. This is a reaction to us unconditionally removing that behavior after this security advisory: http://curl.haxx.se/docs/adv_20120124B.html ... it did however cause a lot of programs to fail because of old servers not liking this work-around. Now programs can opt to decrease the security in order to interoperate with old servers better.
2012-02-09add library support for tuning TCP_KEEPALIVEDave Reisner
This adds three new options to control the behavior of TCP keepalives: - CURLOPT_TCP_KEEPALIVE: enable/disable probes - CURLOPT_TCP_KEEPIDLE: idle time before sending first probe - CURLOPT_TCP_KEEPINTVL: delay between successive probes While not all operating systems support the TCP_KEEPIDLE and TCP_KEEPINTVL knobs, the library will still allow these options to be set by clients, silently ignoring the values.
2012-02-07curl_easy_reset: reset the referer stringDaniel Stenberg
When CURLOPT_REFERER has been used, curl_easy_reset() did not properly clear it. Verified with the new test 598 Bug: http://curl.haxx.se/bug/view.cgi?id=3481551 Reported by: Michael Day
2012-01-20URL parse: user name with ipv6 numerical addressDaniel Stenberg
Using a URL with embedded user name and password didn't work if the host was given as a numerical IPv6 string, like ftp://user:password@[::1]/ Reported by: Brandon Wang Bug: http://curl.haxx.se/mail/archive-2012-01/0047.html
2012-01-18ssl session caching: fix compiler warningsYang Tse
2012-01-16add LF termination to infof() trace stringYang Tse
2011-12-31changed case: use new host name for subsequent HTTP requestsDaniel Stenberg
When a HTTP connection is re-used for a subsequent request without proxy, it would always re-use the Host: header of the first request. As host names are case insensitive it would make curl send another host name case that what the particular request used. Now it will instead always use the most recent host name to always use the desired casing. Added test case 1318 to verify. Bug: http://curl.haxx.se/mail/lib-2011-12/0314.html Reported by: Alex Vinnik
2011-12-24Curl_do_more: fix typo logicDaniel Stenberg
In the recent do_more fix the new logic was mistakenly checking the pointer instead of what it points to. Reported by: Gokhan Sengun Bug: http://curl.haxx.se/mail/lib-2011-12/0250.html