aboutsummaryrefslogtreecommitdiff
path: root/lib/http.c
AgeCommit message (Collapse)Author
2014-02-17http2: Support HTTP POST/PUTTatsuhiro Tsujikawa
This patch enables HTTP POST/PUT in HTTP2. We disabled Expect header field and chunked transfer encoding since HTTP2 forbids them. In HTTP1, Curl sends small upload data with request headers, but HTTP2 requires upload data must be in DATA frame separately. So we added some conditionals to achieve this.
2014-02-05http2: rely on content-encoding headerFabian Frank
A server might respond with a content-encoding header and a response that was encoded accordingly in HTTP-draft-09/2.0 mode, even if the client did not send an accept-encoding header earlier. The server might not send a content-encoding header if the identity encoding was used to encode the response. See: http://tools.ietf.org/html/draft-ietf-httpbis-http2-09#section-9.3
2014-02-04HTTP2: add layer between existing http and socket(TLS) layerTatsuhiro Tsujikawa
This patch chooses different approach to integrate HTTP2 into HTTP curl stack. The idea is that we insert HTTP2 layer between HTTP code and socket(TLS) layer. When HTTP2 is initialized (either in NPN or Upgrade), we replace the Curl_recv/Curl_send callbacks with HTTP2's, but keep the original callbacks in http_conn struct. When sending serialized data by nghttp2, we use original Curl_send callback. Likewise, when reading data from network, we use original Curl_recv callback. In this way we can treat both TLS and non-TLS connections. With this patch, one can transfer contents from https://twitter.com and from nghttp2 test server in plain HTTP as well. The code still has rough edges. The notable one is I could not figure out how to call nghttp2_session_send() when underlying socket is writable.
2014-01-30http2: switch into http2 mode if NPN indicatesFabian Frank
Check the NPN result before preparing an HTTP request and switch into HTTP/2.0 mode if necessary. This is a work in progress, the actual code to prepare and send the request using nghttp2 is still missing from Curl_http2_send_request().
2014-01-30http2: s/Curl_http2_request/Curl_http2_request_upgradeDaniel Stenberg
To better reflect its purpose
2014-01-29http2: handle 101 responses and switch to HTTP2Daniel Stenberg
2014-01-20HTTP POST: omit Content-Length if data size is unknownCédric Deltheil
This prevents sending a `Content-Length: -1` header, e.g this ocurred with the following combination: * standard HTTP POST (no chunked encoding), * user-defined read function set, * `CURLOPT_POSTFIELDSIZE(_LARGE)` NOT set. With this fix it now behaves like HTTP PUT.
2013-12-31mprintf: Replaced internal usage of FORMAT_OFF_T and FORMAT_OFF_TUSteve Holme
Following commit 0aafd77fa4c6f2, replaced the internal usage of FORMAT_OFF_T and FORMAT_OFF_TU with the external versions that we expect API programmers to use. This negates the need for separate definitions which were subtly different under different platforms/compilers.
2013-12-20vtls: renamed sslgen.[ch] to vtls.[ch]Daniel Stenberg
2013-12-20vtls: created subdir, moved sslgen.[ch] there, updated all include linesDaniel Stenberg
2013-11-03http: Post base64 decoding tidy upSteve Holme
Renamed copy_header_value() to Curl_copy_header_value() as this function is now non static. Simplified proxy flag in Curl_http_input_auth() when calling sub-functions. Removed unnecessary white space removal when using negotiate as it had been missed in commit cdccb422671aeb.
2013-11-02http: Added proxy tunnel authentication message header value extractionSteve Holme
...following recent changes to Curl_base64_decode() rather than trying to parse a header line for the authentication mechanisms which is CRLF terminated and inline zero terminate it.
2013-10-31http: Added authentication message header value extractionSteve Holme
...following recent changes to Curl_base64_decode() rather than trying to parse a header line for the authentication mechanisms which is CRLF terminated and inline zero terminate it.
2013-09-07http2: first embryo toward Upgrade:Daniel Stenberg
2013-09-07http: rename use_http_1_1 to use_http_1_1plusDaniel Stenberg
Since it now actually says if 1.1 or a later version should be used.
2013-08-12SessionHandle: the protocol specific pointer is now a void *Daniel Stenberg
All protocol handler structs are now opaque (void *) in the SessionHandle struct and moved in the request-specific sub-struct 'SingleRequest'. The intension is to keep the protocol specific knowledge in their own dedicated source files [protocol].c etc. There's some "leakage" where this policy is violated, to be addressed at a later point in time.
2013-08-12urldata: clean up the use of the protocol specific structsDaniel Stenberg
1 - always allocate the struct in protocol->setup_connection. Some protocol handlers had to get this function added. 2 - always free at the end of a request. This is also an attempt to keep less memory in the handle after it is completed.
2013-03-25NTLM: fix several NTLM code paths memory leaksYang Tse
2013-03-13Multiple pipelines and limiting the number of connections.Linus Nielsen Feltzing
Introducing a number of options to the multi interface that allows for multiple pipelines to the same host, in order to optimize the balance between the penalty for opening new connections and the potential pipelining latency. Two new options for limiting the number of connections: CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections to the same host. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished, so we can reuse the connection. CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished. The free connection will then be reused, if possible, or closed if the pending handle can't reuse it. Several new options for pipelining: CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a pipeline is "full" when a connection is to be reused, a new connection will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it. If not, the handle will be put in a pending state until a connection is ready (either free or a pipe got shorter). CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a transfer with a content length that is larger than this. CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a chunk larger than this. CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow pipelining. CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow pipelining. See the curl_multi_setopt() man page for details.
2013-01-17always-multi: always use non-blocking internalsDaniel Stenberg
Remove internal separated behavior of the easy vs multi intercace. curl_easy_perform() is now using the multi interface itself. Several minor multi interface quirks and bugs have been fixed in the process. Much help with debugging this has been provided by: Yang Tse
2013-01-09build: fix circular header inclusion with other packagesYang Tse
This commit renames lib/setup.h to lib/curl_setup.h and renames lib/setup_once.h to lib/curl_setup_once.h. Removes the need and usage of a header inclusion guard foreign to libcurl. [1] Removes the need and presence of an alarming notice we carried in old setup_once.h [2] ---------------------------------------- 1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H, this single inclusion guard is enough to ensure that inclusion of lib/setup_once.h done from lib/setup.h is only done once. Additionally lib/setup.h has always used __SETUP_ONCE_H macro to protect inclusion of setup_once.h even after commit ec691ca3, this was to avoid a circular header inclusion triggered when building a c-ares enabled version with c-ares sources available which also has a setup_once.h header. Commit ec691ca3 exposes the real nature of __SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard foreign to libcurl belonging to c-ares's setup_once.h The renaming this commit does, fixes the circular header inclusion, and as such removes the need and usage of a header inclusion guard foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl. 2 - Due to the circular interdependency of old lib/setup_once.h and the c-ares setup_once.h header, old file lib/setup_once.h has carried back from 2006 up to now days an alarming and prominent notice about the need of keeping libcurl's and c-ares's setup_once.h in sync. Given that this commit fixes the circular interdependency, the need and presence of mentioned notice is removed. All mentioned interdependencies come back from now old days when the c-ares project lived inside a curl subdirectory. This commit removes last traces of such fact.
2013-01-06Revert changes relative to lib/*.[ch] recent renamingYang Tse
This reverts renaming and usage of lib/*.h header files done 28-12-2012, reverting 2 commits: f871de0... build: make use of 76 lib/*.h renamed files ffd8e12... build: rename 76 lib/*.h files This also reverts removal of redundant include guard (redundant thanks to changes in above commits) done 2-12-2013, reverting 1 commit: c087374... curl_setup.h: remove redundant include guard This also reverts renaming and usage of lib/*.c source files done 3-12-2013, reverting 3 commits: 13606bb... build: make use of 93 lib/*.c renamed files 5b6e792... build: rename 93 lib/*.c files 7d83dff... build: commit 13606bbfde follow-up 1 Start of related discussion thread: http://curl.haxx.se/mail/lib-2013-01/0012.html Asking for confirmation on pushing this revertion commit: http://curl.haxx.se/mail/lib-2013-01/0048.html Confirmation summary: http://curl.haxx.se/mail/lib-2013-01/0079.html NOTICE: The list of 2 files that have been modified by other intermixed commits, while renamed, and also by at least one of the 6 commits this one reverts follows below. These 2 files will exhibit a hole in history unless git's '--follow' option is used when viewing logs. lib/curl_imap.h lib/curl_smtp.h
2013-01-03build: rename 93 lib/*.c filesYang Tse
93 lib/*.c source files renamed to use our standard naming scheme. This commit only does the file renaming. ---------------------------------------- renamed: lib/amigaos.c -> lib/curl_amigaos.c renamed: lib/asyn-ares.c -> lib/curl_asyn_ares.c renamed: lib/asyn-thread.c -> lib/curl_asyn_thread.c renamed: lib/axtls.c -> lib/curl_axtls.c renamed: lib/base64.c -> lib/curl_base64.c renamed: lib/bundles.c -> lib/curl_bundles.c renamed: lib/conncache.c -> lib/curl_conncache.c renamed: lib/connect.c -> lib/curl_connect.c renamed: lib/content_encoding.c -> lib/curl_content_encoding.c renamed: lib/cookie.c -> lib/curl_cookie.c renamed: lib/cyassl.c -> lib/curl_cyassl.c renamed: lib/dict.c -> lib/curl_dict.c renamed: lib/easy.c -> lib/curl_easy.c renamed: lib/escape.c -> lib/curl_escape.c renamed: lib/file.c -> lib/curl_file.c renamed: lib/fileinfo.c -> lib/curl_fileinfo.c renamed: lib/formdata.c -> lib/curl_formdata.c renamed: lib/ftp.c -> lib/curl_ftp.c renamed: lib/ftplistparser.c -> lib/curl_ftplistparser.c renamed: lib/getenv.c -> lib/curl_getenv.c renamed: lib/getinfo.c -> lib/curl_getinfo.c renamed: lib/gopher.c -> lib/curl_gopher.c renamed: lib/gtls.c -> lib/curl_gtls.c renamed: lib/hash.c -> lib/curl_hash.c renamed: lib/hmac.c -> lib/curl_hmac.c renamed: lib/hostasyn.c -> lib/curl_hostasyn.c renamed: lib/hostcheck.c -> lib/curl_hostcheck.c renamed: lib/hostip.c -> lib/curl_hostip.c renamed: lib/hostip4.c -> lib/curl_hostip4.c renamed: lib/hostip6.c -> lib/curl_hostip6.c renamed: lib/hostsyn.c -> lib/curl_hostsyn.c renamed: lib/http.c -> lib/curl_http.c renamed: lib/http_chunks.c -> lib/curl_http_chunks.c renamed: lib/http_digest.c -> lib/curl_http_digest.c renamed: lib/http_negotiate.c -> lib/curl_http_negotiate.c renamed: lib/http_negotiate_sspi.c -> lib/curl_http_negotiate_sspi.c renamed: lib/http_proxy.c -> lib/curl_http_proxy.c renamed: lib/idn_win32.c -> lib/curl_idn_win32.c renamed: lib/if2ip.c -> lib/curl_if2ip.c renamed: lib/imap.c -> lib/curl_imap.c renamed: lib/inet_ntop.c -> lib/curl_inet_ntop.c renamed: lib/inet_pton.c -> lib/curl_inet_pton.c renamed: lib/krb4.c -> lib/curl_krb4.c renamed: lib/krb5.c -> lib/curl_krb5.c renamed: lib/ldap.c -> lib/curl_ldap.c renamed: lib/llist.c -> lib/curl_llist.c renamed: lib/md4.c -> lib/curl_md4.c renamed: lib/md5.c -> lib/curl_md5.c renamed: lib/memdebug.c -> lib/curl_memdebug.c renamed: lib/mprintf.c -> lib/curl_mprintf.c renamed: lib/multi.c -> lib/curl_multi.c renamed: lib/netrc.c -> lib/curl_netrc.c renamed: lib/non-ascii.c -> lib/curl_non_ascii.c renamed: lib/curl_non-ascii.h -> lib/curl_non_ascii.h renamed: lib/nonblock.c -> lib/curl_nonblock.c renamed: lib/nss.c -> lib/curl_nss.c renamed: lib/nwlib.c -> lib/curl_nwlib.c renamed: lib/nwos.c -> lib/curl_nwos.c renamed: lib/openldap.c -> lib/curl_openldap.c renamed: lib/parsedate.c -> lib/curl_parsedate.c renamed: lib/pingpong.c -> lib/curl_pingpong.c renamed: lib/polarssl.c -> lib/curl_polarssl.c renamed: lib/pop3.c -> lib/curl_pop3.c renamed: lib/progress.c -> lib/curl_progress.c renamed: lib/qssl.c -> lib/curl_qssl.c renamed: lib/rawstr.c -> lib/curl_rawstr.c renamed: lib/rtsp.c -> lib/curl_rtsp.c renamed: lib/security.c -> lib/curl_security.c renamed: lib/select.c -> lib/curl_select.c renamed: lib/sendf.c -> lib/curl_sendf.c renamed: lib/share.c -> lib/curl_share.c renamed: lib/slist.c -> lib/curl_slist.c renamed: lib/smtp.c -> lib/curl_smtp.c renamed: lib/socks.c -> lib/curl_socks.c renamed: lib/socks_gssapi.c -> lib/curl_socks_gssapi.c renamed: lib/socks_sspi.c -> lib/curl_socks_sspi.c renamed: lib/speedcheck.c -> lib/curl_speedcheck.c renamed: lib/splay.c -> lib/curl_splay.c renamed: lib/ssh.c -> lib/curl_ssh.c renamed: lib/sslgen.c -> lib/curl_sslgen.c renamed: lib/ssluse.c -> lib/curl_ssluse.c renamed: lib/strdup.c -> lib/curl_strdup.c renamed: lib/strequal.c -> lib/curl_strequal.c renamed: lib/strerror.c -> lib/curl_strerror.c renamed: lib/strtok.c -> lib/curl_strtok.c renamed: lib/strtoofft.c -> lib/curl_strtoofft.c renamed: lib/telnet.c -> lib/curl_telnet.c renamed: lib/tftp.c -> lib/curl_tftp.c renamed: lib/timeval.c -> lib/curl_timeval.c renamed: lib/transfer.c -> lib/curl_transfer.c renamed: lib/url.c -> lib/curl_url.c renamed: lib/version.c -> lib/curl_version.c renamed: lib/warnless.c -> lib/curl_warnless.c renamed: lib/wildcard.c -> lib/curl_wildcard.c ----------------------------------------
2013-01-03build: make use of 93 lib/*.c renamed filesYang Tse
93 *.c source files renamed to use our standard naming scheme. This change affects 77 files in libcurl's source tree.
2012-12-28build: make use of 76 lib/*.h renamed filesYang Tse
76 private header files renamed to use our standard naming scheme. This change affects 322 files in libcurl's source tree.
2012-12-14setup_once.h: refactor inclusion of <unistd.h> and <sys/socket.h>Yang Tse
Inclusion of top two most included header files now done in setup_once.h
2012-12-07Introducing a new persistent connection caching system using "bundles".Linus Nielsen Feltzing
A bundle is a list of all persistent connections to the same host. The connection cache consists of a hash of bundles, with the hostname as the key. The benefits may not be obvious, but they are two: 1) Faster search for connections to reuse, since the hash lookup only finds connections to the host in question. 2) It lays out the groundworks for an upcoming patch, which will introduce multiple HTTP pipelines. This patch also removes the awkward list of "closure handles", which were needed to send QUIT commands to the FTP server when closing a connection. Now we allocate a separate closure handle and use that one to close all connections. This has been tested in a live system for a few weeks, and of course passes the test suite.
2012-11-26Remove stray CRLF in chunk-encoded content-free request bodiesFabian Keil
.. that are sent when auth-negotiating before a chunked upload or when setting the 'Transfer-Encoding: chunked' header and intentionally sending no content. Adjust test565 and test1333 accordingly.
2012-11-07http_perhapsrewind: consider NTLM over proxy tooDaniel Stenberg
The logic previously checked for a started NTLM negotiation only for host and not also with proxy, leading to problems doing POSTs over a proxy NTLM that are larger than 2000 bytes. Now it includes proxy in the check. Bug: http://curl.haxx.se/bug/view.cgi?id=3582321 Reported by: John Suprock
2012-07-25darwinssl: fixed freeze involving the multi interfaceNick Zitzmann
Previously the curl_multi interface would freeze if darwinssl was enabled and at least one of the handles tried to connect to a Web site using HTTPS. Removed the "wouldblock" state darwinssl was using because I figured out a solution for our "would block but in which direction?" dilemma.
2012-07-22http: print reason phrase from HTTP status line on errorKamil Dudka
Bug: https://bugzilla.redhat.com/676596
2012-07-12HEAD: don't force-close after response-headersDaniel Stenberg
A HEAD response has no body length and gets the headers like the corresponding GET would so it should not get closed after the response based on the same rules. This mistake caused connections that did HEAD to get closed too often without a valid reason. Bug: http://curl.haxx.se/bug/view.cgi?id=3542731 Reported by: Eelco Dolstra
2012-07-09https_getsock: provided for schannel backend as wellChristian Hägele
The function https_getsock was only implemented properly when USE_SSLEAY or USE_GNUTLS is defined, but it is also necessary for USE_SCHANNEL. The problem occurs when Curl_read_plain or Curl_write_plain returns CURLE_AGAIN. In that case CURL_OK is returned to the multi-interface an the used socket is set to state CURL_POLL_REMOVE and the easy-state is set to CURLM_STATE_PROTOCONNECT. This is fine, because later the socket should be set to CURL_POLL_IN or CURL_POLL_OUT via multi_getsock. That's where https_getsock is called and doesn't return any sockets.
2012-06-14Fix bad failf() and info() usageYang Tse
Calls to failf() are not supposed to provide trailing newline. Calls to infof() must provide trailing newline. Fixed 30 or so strings.
2012-06-11http: Replaced specific SSL libraries list in https_getsock fallbackMarc Hoersken
2012-04-20HTTP: empty chunked POST ended up in two zero size chunksDaniel Stenberg
When doing a chunked-encoded POST with -d (CURLOPT_POSTFIELDS) and the size of the POST was zero length, it made libcurl first send a zero chunk and then the terminating one. This could confuse a receiver and it should rather just send the terminating chunk as it does with this fix. Test case 1333 is added to verify. Bug: http://curl.haxx.se/mail/archive-2012-04/0060.html Reported by: Arnaud Compan
2012-04-18Take in account that CURLAUTH_* bitmasks are now 'unsigned long'Yang Tse
Data type of internal vars holding CURLAUTH_* bitmasks changed from 'long' to 'unsigned long' for proper handling and operating.
2012-04-04IPv6 cookie domain: get rid of the first bracket before the second.Andrei Cipu
Commit 97b66ebe was copying a smaller buffer, thus duplicating the last character.
2012-03-22cookies: strip the numerical ipv6 host properlyAndrei Cipu
The commit e650dbde86d4 that stripped off [brackets] from ipv6-only host headers for the sake of cookie parsing wrongly incremented the host pointer which would cause a bad free() call later on.
2012-03-22CONNECT: fix multi interface regressionDaniel Stenberg
The refactoring of HTTP CONNECT handling in commit 41b0237834232 that made it protocol independent broke it for the multi interface. This fix now introduce a better state handling and moved some logic to the http_proxy.c source file. Reported by: Yang Tse Bug: http://curl.haxx.se/mail/lib-2012-03/0162.html
2012-03-10Curl_http: strip off [brackets] from ipv6-only host headersAndrei Cipu
Since the host name is passed in to the cookie engine it will not work correctly if the brackets are left in the name. Bug:http://curl.haxx.se/mail/lib-2012-03/0036.html
2012-03-08CONNECT: made generically not per-protocolDaniel Stenberg
Curl_protocol_connect() now does the tunneling through the HTTP proxy if requested instead of letting each protocol specific connection function do it.
2011-11-06HTTP auth: fix proxy Negotiate bugRene Bernhardt
If a proxy offers several Authentication schemes where NTLM and Negotiate are offered by the proxy and you tell libcurl not to use the Negotiate scheme then the request never returns when the proxy answers with its HTTP 407 reply. It is reproducible by the following steps: - Use a proxy that offers NTLM and Negotiate ( CURLOPT_PROXY and CURLOPT_PROXYPORT ) - Tell libcurl NOT to use Negotiate CURL_EASY_SETOPT(CURLOPT_PROXYAUTH, CURLAUTH_BASIC | CURLAUTH_DIGEST | CURLAUTH_NTLM ) - Start the request The call to CURL_EASY_PERFORM never returns. If you switch on debug logging you can see that libcurl issues a new request As soon as it received the 407 reply. Instead it should return and set the response code to 407. Bug: http://curl.haxx.se/mail/lib-2011-10/0323.html
2011-10-21curl_multi_fdset: correct fdset with FTP PORT useDaniel Stenberg
After a PORT has been issued, and the multi handle would switch to the CURLM_STATE_DO_MORE state (which is unique for FTP), libcurl would return the wrong fdset to wait for when curl_multi_fdset() is called. The code would blindly assume that it was waiting for a connect of the second connection, while that isn't true immediately after the PORT command. Also, the function multi.c:domore_getsock() was highly FTP-centric and therefore ugly to keep in protocol-agnostic code. I solved this problem by introducing a new function pointer in the Curl_handler struct called domore_getsock() which is only called during the DOMORE state for protocols that set that pointer. The new ftp.c:ftp_domore_getsock() function now returns fdset info about the control connection's command/response handling while such a state is in use, and goes over to waiting for a writable second connection first once the commands are done. The original problem could be seen by running test 525 and checking the time stamps in the FTP server log. I can verify that this fix at least fixes this problem. Bug: http://curl.haxx.se/mail/lib-2011-10/0250.html Reported by: Gokhan Sengun
2011-10-20Curl_http_input_auth: handle multiple auths in WWW-AuthenticateDaniel Stenberg
The fix is pretty much the one Nick Zitzmann provided, just edited to do the right indent levels and with test case 1204 added to verify the fix. Bug: http://curl.haxx.se/mail/lib-2011-10/0190.html Reported by: Nick Zitzmann
2011-10-12OOM handling/cleanup slight adjustmentsYang Tse
2011-09-24http header: allow Content-Length to be replacedDaniel Stenberg
In some cases Content-Length: couldn't be replaced by an application Also, indented some code properly
2011-09-22NTLM_WB: fix disabling of NTLM_WB when NTLM is disabledYang Tse
2011-09-21lib/http: add missing whitespace in verbose outputDave Reisner
Example: * upload completely sent off: 35out of 35 bytes Should be: * upload completely sent off: 35 out of 35 bytes
2011-09-13Curl_add_custom_headers: support headers with no datawarp kawada
A custom HTTP header ending in a semicolon instead of a colon will be treated as a header to be added without any data portion.