aboutsummaryrefslogtreecommitdiff
path: root/lib/transfer.c
AgeCommit message (Collapse)Author
2014-05-22bits.close: introduce connection close trackingDaniel Stenberg
Make all code use connclose() and connkeep() when changing the "close state" for a connection. These two macros take a string argument with an explanation, and debug builds of curl will include that in the debug output. Helps tracking connection re-use/close issues.
2014-04-26INFILESIZE: fields in UserDefined must not be changed run-timeDaniel Stenberg
set.infilesize in this case was modified in several places, which could lead to repeated requests using the same handle to get unintendent/wrong consequences based on what the previous request did!
2014-04-23handler: make 'protocol' always specified as a single bitDaniel Stenberg
This makes the findprotocol() function work as intended so that libcurl can properly be restricted to not support HTTP while still supporting HTTPS - since the HTTPS handler previously set both the HTTP and HTTPS bits in the protocol field. This fixes --proto and --proto-redir for most SSL protocols. This is done by adding a few new convenience defines that groups HTTP and HTTPS, FTP and FTPS etc that should then be used when the code wants to check for both protocols at once. PROTO_FAMILY_[protocol] style. Bug: https://github.com/bagder/curl/pull/97 Reported-by: drizzt
2014-03-14chunked-encoding: provide a readable error string for chunked errorsDaniel Stenberg
2014-02-28transfer.c: Fixed non-HTTP2 builds from commit cde0cf7c5eSteve Holme
2014-02-28Fix bug that HTTP/2 hangs if whole response body is read with headersTatsuhiro Tsujikawa
For HTTP/2, we may read up everything including responde body with header fields in Curl_http_readwrite_headers. If no content-length is provided, curl waits for the connection close, which we emulate it using conn->proto.httpc.closed = TRUE. The thing is if we read everything, then http2_recv won't be called and we cannot signal the HTTP/2 stream has closed. As a workaround, we return nonzero from data_pending to call http2_recv.
2014-02-13transfer: make Expect: 100-continue timeout configurable.Tiit Pikma
Replaced the #define CURL_TIMEOUT_EXPECT_100 in transfer.c with the CURLOPT_EXPECT_100_TIMEOUT_MS option to make the timeout configurable.
2014-02-06100-continue: fix timeout conditionRemi Gacogne
When using the multi socket interface, libcurl calls the curl_multi_timer_callback asking to be woken up after CURL_TIMEOUT_EXPECT_100 milliseconds. After the timeout has expired, calling curl_multi_socket_action with CURL_SOCKET_TIMEOUT as sockfd leads libcurl to check expired timeouts. When handling the 100-continue one, the following check in Curl_readwrite() fails if exactly CURL_TIMEOUT_EXPECT_100 milliseconds passed since the timeout has been set! It seems logical to consider that having waited for exactly CURL_TIMEOUT_EXPECT_100 ms is enough. Bug: http://curl.haxx.se/bug/view.cgi?id=1334
2014-01-12multi: remove MULTI_TIMEOUT_INACCURACYDaniel Stenberg
With the recently added timeout "reminder" functionality, there's no reason left for us to execute timeout code before the time is ripe. Simplifies the handling too. This will make the *TIMEOUT and *CONNECTTIMEOUT options more accurate again, which probably is most important when the *_MS versions are used. In multi_socket, make sure to update 'now' after having handled activity on a socket.
2013-12-31mprintf: Replaced internal usage of FORMAT_OFF_T and FORMAT_OFF_TUSteve Holme
Following commit 0aafd77fa4c6f2, replaced the internal usage of FORMAT_OFF_T and FORMAT_OFF_TU with the external versions that we expect API programmers to use. This negates the need for separate definitions which were subtly different under different platforms/compilers.
2013-12-26FILE: we don't support paused transfers using this protocolDaniel Stenberg
Make sure that we detect such attempts and return a proper error code instead of silently handling this in problematic ways. Updated the documentation to mention this limitation. Bug: http://curl.haxx.se/bug/view.cgi?id=1286
2013-12-20vtls: renamed sslgen.[ch] to vtls.[ch]Daniel Stenberg
2013-12-20vtls: created subdir, moved sslgen.[ch] there, updated all include linesDaniel Stenberg
2013-12-15multi: add timer inaccuracy margin to timeout/connecttimeoutDaniel Stenberg
Since all systems have inaccuracy in the timeout handling it is imperative that we add an inaccuracy margin to the general timeout and connecttimeout handling with the multi interface. This way, when the timeout fires we should be fairly sure that it has passed the timeout value and will be suitably detected. For cases where the timeout fire before the actual timeout, we would otherwise consume the timeout action and still not run the timeout code since the condition wasn't met. Reported-by: He Qin Bug: http://curl.haxx.se/bug/view.cgi?id=1298
2013-10-07HTTP: Output http response 304 when modified time is too oldDerek Higgins
When using the -w '%{http_code}' flag and simulating a Not Modified then 304 should be output.
2013-08-28multi_socket: improved 100-continue timeout handlingDaniel Stenberg
When waiting for a 100-continue response from the server, the Curl_readwrite() will refuse to run if called until the timeout has been reached. We timeout code in multi_socket() allows code to run slightly before the actual timeout time, so for test 154 it could lead to the function being executed but refused in Curl_readwrite() and then the application would just sit idling forever. This was detected with runtests.pl -e on test 154.
2013-08-20transfer: the recent sessionhandle change broke CURL_DOES_CONVERSIONSGisle Vanem
2013-08-12SessionHandle: the protocol specific pointer is now a void *Daniel Stenberg
All protocol handler structs are now opaque (void *) in the SessionHandle struct and moved in the request-specific sub-struct 'SingleRequest'. The intension is to keep the protocol specific knowledge in their own dedicated source files [protocol].c etc. There's some "leakage" where this policy is violated, to be addressed at a later point in time.
2013-03-13Multiple pipelines and limiting the number of connections.Linus Nielsen Feltzing
Introducing a number of options to the multi interface that allows for multiple pipelines to the same host, in order to optimize the balance between the penalty for opening new connections and the potential pipelining latency. Two new options for limiting the number of connections: CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections to the same host. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished, so we can reuse the connection. CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished. The free connection will then be reused, if possible, or closed if the pending handle can't reuse it. Several new options for pipelining: CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a pipeline is "full" when a connection is to be reused, a new connection will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it. If not, the handle will be put in a pending state until a connection is ready (either free or a pipe got shorter). CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a transfer with a content length that is larger than this. CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a chunk larger than this. CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow pipelining. CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow pipelining. See the curl_multi_setopt() man page for details.
2013-02-01Curl_perfom: removedDaniel Stenberg
Curl_perfom is no longer used anywhere since the always-multi commit c43127414d89ccb9, and some related functions were used only from within Curl_perfom.
2013-01-17always-multi: always use non-blocking internalsDaniel Stenberg
Remove internal separated behavior of the easy vs multi intercace. curl_easy_perform() is now using the multi interface itself. Several minor multi interface quirks and bugs have been fixed in the process. Much help with debugging this has been provided by: Yang Tse
2013-01-09build: fix circular header inclusion with other packagesYang Tse
This commit renames lib/setup.h to lib/curl_setup.h and renames lib/setup_once.h to lib/curl_setup_once.h. Removes the need and usage of a header inclusion guard foreign to libcurl. [1] Removes the need and presence of an alarming notice we carried in old setup_once.h [2] ---------------------------------------- 1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H, this single inclusion guard is enough to ensure that inclusion of lib/setup_once.h done from lib/setup.h is only done once. Additionally lib/setup.h has always used __SETUP_ONCE_H macro to protect inclusion of setup_once.h even after commit ec691ca3, this was to avoid a circular header inclusion triggered when building a c-ares enabled version with c-ares sources available which also has a setup_once.h header. Commit ec691ca3 exposes the real nature of __SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard foreign to libcurl belonging to c-ares's setup_once.h The renaming this commit does, fixes the circular header inclusion, and as such removes the need and usage of a header inclusion guard foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl. 2 - Due to the circular interdependency of old lib/setup_once.h and the c-ares setup_once.h header, old file lib/setup_once.h has carried back from 2006 up to now days an alarming and prominent notice about the need of keeping libcurl's and c-ares's setup_once.h in sync. Given that this commit fixes the circular interdependency, the need and presence of mentioned notice is removed. All mentioned interdependencies come back from now old days when the c-ares project lived inside a curl subdirectory. This commit removes last traces of such fact.
2013-01-06Revert changes relative to lib/*.[ch] recent renamingYang Tse
This reverts renaming and usage of lib/*.h header files done 28-12-2012, reverting 2 commits: f871de0... build: make use of 76 lib/*.h renamed files ffd8e12... build: rename 76 lib/*.h files This also reverts removal of redundant include guard (redundant thanks to changes in above commits) done 2-12-2013, reverting 1 commit: c087374... curl_setup.h: remove redundant include guard This also reverts renaming and usage of lib/*.c source files done 3-12-2013, reverting 3 commits: 13606bb... build: make use of 93 lib/*.c renamed files 5b6e792... build: rename 93 lib/*.c files 7d83dff... build: commit 13606bbfde follow-up 1 Start of related discussion thread: http://curl.haxx.se/mail/lib-2013-01/0012.html Asking for confirmation on pushing this revertion commit: http://curl.haxx.se/mail/lib-2013-01/0048.html Confirmation summary: http://curl.haxx.se/mail/lib-2013-01/0079.html NOTICE: The list of 2 files that have been modified by other intermixed commits, while renamed, and also by at least one of the 6 commits this one reverts follows below. These 2 files will exhibit a hole in history unless git's '--follow' option is used when viewing logs. lib/curl_imap.h lib/curl_smtp.h
2013-01-03build: rename 93 lib/*.c filesYang Tse
93 lib/*.c source files renamed to use our standard naming scheme. This commit only does the file renaming. ---------------------------------------- renamed: lib/amigaos.c -> lib/curl_amigaos.c renamed: lib/asyn-ares.c -> lib/curl_asyn_ares.c renamed: lib/asyn-thread.c -> lib/curl_asyn_thread.c renamed: lib/axtls.c -> lib/curl_axtls.c renamed: lib/base64.c -> lib/curl_base64.c renamed: lib/bundles.c -> lib/curl_bundles.c renamed: lib/conncache.c -> lib/curl_conncache.c renamed: lib/connect.c -> lib/curl_connect.c renamed: lib/content_encoding.c -> lib/curl_content_encoding.c renamed: lib/cookie.c -> lib/curl_cookie.c renamed: lib/cyassl.c -> lib/curl_cyassl.c renamed: lib/dict.c -> lib/curl_dict.c renamed: lib/easy.c -> lib/curl_easy.c renamed: lib/escape.c -> lib/curl_escape.c renamed: lib/file.c -> lib/curl_file.c renamed: lib/fileinfo.c -> lib/curl_fileinfo.c renamed: lib/formdata.c -> lib/curl_formdata.c renamed: lib/ftp.c -> lib/curl_ftp.c renamed: lib/ftplistparser.c -> lib/curl_ftplistparser.c renamed: lib/getenv.c -> lib/curl_getenv.c renamed: lib/getinfo.c -> lib/curl_getinfo.c renamed: lib/gopher.c -> lib/curl_gopher.c renamed: lib/gtls.c -> lib/curl_gtls.c renamed: lib/hash.c -> lib/curl_hash.c renamed: lib/hmac.c -> lib/curl_hmac.c renamed: lib/hostasyn.c -> lib/curl_hostasyn.c renamed: lib/hostcheck.c -> lib/curl_hostcheck.c renamed: lib/hostip.c -> lib/curl_hostip.c renamed: lib/hostip4.c -> lib/curl_hostip4.c renamed: lib/hostip6.c -> lib/curl_hostip6.c renamed: lib/hostsyn.c -> lib/curl_hostsyn.c renamed: lib/http.c -> lib/curl_http.c renamed: lib/http_chunks.c -> lib/curl_http_chunks.c renamed: lib/http_digest.c -> lib/curl_http_digest.c renamed: lib/http_negotiate.c -> lib/curl_http_negotiate.c renamed: lib/http_negotiate_sspi.c -> lib/curl_http_negotiate_sspi.c renamed: lib/http_proxy.c -> lib/curl_http_proxy.c renamed: lib/idn_win32.c -> lib/curl_idn_win32.c renamed: lib/if2ip.c -> lib/curl_if2ip.c renamed: lib/imap.c -> lib/curl_imap.c renamed: lib/inet_ntop.c -> lib/curl_inet_ntop.c renamed: lib/inet_pton.c -> lib/curl_inet_pton.c renamed: lib/krb4.c -> lib/curl_krb4.c renamed: lib/krb5.c -> lib/curl_krb5.c renamed: lib/ldap.c -> lib/curl_ldap.c renamed: lib/llist.c -> lib/curl_llist.c renamed: lib/md4.c -> lib/curl_md4.c renamed: lib/md5.c -> lib/curl_md5.c renamed: lib/memdebug.c -> lib/curl_memdebug.c renamed: lib/mprintf.c -> lib/curl_mprintf.c renamed: lib/multi.c -> lib/curl_multi.c renamed: lib/netrc.c -> lib/curl_netrc.c renamed: lib/non-ascii.c -> lib/curl_non_ascii.c renamed: lib/curl_non-ascii.h -> lib/curl_non_ascii.h renamed: lib/nonblock.c -> lib/curl_nonblock.c renamed: lib/nss.c -> lib/curl_nss.c renamed: lib/nwlib.c -> lib/curl_nwlib.c renamed: lib/nwos.c -> lib/curl_nwos.c renamed: lib/openldap.c -> lib/curl_openldap.c renamed: lib/parsedate.c -> lib/curl_parsedate.c renamed: lib/pingpong.c -> lib/curl_pingpong.c renamed: lib/polarssl.c -> lib/curl_polarssl.c renamed: lib/pop3.c -> lib/curl_pop3.c renamed: lib/progress.c -> lib/curl_progress.c renamed: lib/qssl.c -> lib/curl_qssl.c renamed: lib/rawstr.c -> lib/curl_rawstr.c renamed: lib/rtsp.c -> lib/curl_rtsp.c renamed: lib/security.c -> lib/curl_security.c renamed: lib/select.c -> lib/curl_select.c renamed: lib/sendf.c -> lib/curl_sendf.c renamed: lib/share.c -> lib/curl_share.c renamed: lib/slist.c -> lib/curl_slist.c renamed: lib/smtp.c -> lib/curl_smtp.c renamed: lib/socks.c -> lib/curl_socks.c renamed: lib/socks_gssapi.c -> lib/curl_socks_gssapi.c renamed: lib/socks_sspi.c -> lib/curl_socks_sspi.c renamed: lib/speedcheck.c -> lib/curl_speedcheck.c renamed: lib/splay.c -> lib/curl_splay.c renamed: lib/ssh.c -> lib/curl_ssh.c renamed: lib/sslgen.c -> lib/curl_sslgen.c renamed: lib/ssluse.c -> lib/curl_ssluse.c renamed: lib/strdup.c -> lib/curl_strdup.c renamed: lib/strequal.c -> lib/curl_strequal.c renamed: lib/strerror.c -> lib/curl_strerror.c renamed: lib/strtok.c -> lib/curl_strtok.c renamed: lib/strtoofft.c -> lib/curl_strtoofft.c renamed: lib/telnet.c -> lib/curl_telnet.c renamed: lib/tftp.c -> lib/curl_tftp.c renamed: lib/timeval.c -> lib/curl_timeval.c renamed: lib/transfer.c -> lib/curl_transfer.c renamed: lib/url.c -> lib/curl_url.c renamed: lib/version.c -> lib/curl_version.c renamed: lib/warnless.c -> lib/curl_warnless.c renamed: lib/wildcard.c -> lib/curl_wildcard.c ----------------------------------------
2013-01-03build: make use of 93 lib/*.c renamed filesYang Tse
93 *.c source files renamed to use our standard naming scheme. This change affects 77 files in libcurl's source tree.
2012-12-28build: make use of 76 lib/*.h renamed filesYang Tse
76 private header files renamed to use our standard naming scheme. This change affects 322 files in libcurl's source tree.
2012-12-14setup_once.h: refactor inclusion of <unistd.h> and <sys/socket.h>Yang Tse
Inclusion of top two most included header files now done in setup_once.h
2012-11-08Curl_readwrite: remove debug outputDaniel Stenberg
The text "additional stuff not fine" text was added for debug purposes a while ago, but it isn't really helping anyone and for some reason some Linux distributions provide their libcurls built with debug info still present and thus (far too many) users get to read this info.
2012-11-06Curl_pretransfer: clear out unwanted auth methodsDaniel Stenberg
As a handle can be re-used after having done HTTP auth in a previous request, it must make sure to clear out the HTTP types that aren't wanted in this new request.
2012-11-06Revert "Zero out auth structs before transfer"Daniel Stenberg
This reverts commit ce8311c7e49eca93c136b58efa6763853541ec97. The commit made test 2024 work but caused a regression with repeated Digest authentication. We need to fix this differently.
2012-09-28Curl_reconnect_request: clear pointer on failureDaniel Stenberg
The Curl_reconnect_request() function could end up returning a pointer to a free()d struct when Curl_done() failed inside. Clearing the pointer unconditionally after Curl_done() avoids this risk. Reported by: Ho-chi Chen Bug: http://curl.haxx.se/mail/lib-2012-09/0188.html
2012-08-07retry request: only access the HTTP data if in fact HTTPDaniel Stenberg
When figuring out if the data stream needs to be rewound when the request is to be resent, we must not access the HTTP struct unless the protocol used is indeed HTTP... Bug: http://curl.haxx.se/bug/view.cgi?id=3544688
2012-08-03Zero out auth structs before transferJoe Mason
2012-04-05CURLOPT_POSTREDIR: also allow 303 to do POST on the redirected URLAndrei Cipu
As it turns out, some people do want that after all.
2012-04-01HTTP: reset expected DL/UL sizes on redirectsDaniel Stenberg
With FOLLOWLOCATION enabled. When a 3xx page is downloaded and the download size was known (like with a Content-Length header), but the subsequent URL (transfered after the 3xx page) was chunked encoded, then the previous "known download size" would linger and cause the progress meter to get incorrect information, ie the former value would remain being sent in. This could easily result in downloads that were WAY larger than "expected" and would cause >100% outputs with the curl command line tool. Test case 599 was created and it was used to repeat the bug and then verify the fix. Bug: http://curl.haxx.se/bug/view.cgi?id=3510057 Reported by: Michael Wallner
2012-01-18ssl session caching: fix compiler warningsYang Tse
2012-01-16add LF termination to infof() trace stringYang Tse
2011-12-31CURLOPT_RESOLVE: avoid adding already present host namesDaniel Stenberg
The load host names to DNS cache function was moved to hostip.c and it now makes sure to not add host names that already are present in the cache. It would previously lead to memory leaks when for example using the --resolve and multiple URLs on the command line.
2011-12-25transfer: avoid unnecessary timeout event when waiting for 100-continueKamil Dudka
The commit 9dd85bc unintentionally changed the way we compute the time spent waiting for 100-continue. In particular, when using a SSL client certificate, the time spent by SSL handshake was included and could cause the CURL_TIMEOUT_EXPECT_100 timeout to be mistakenly fired up. Bug: https://bugzilla.redhat.com/767490 Reported by: Mamoru Tasaka
2011-12-25transfer.c: move a logging statement placementYang Tse
2011-11-02url.c and transfer.c: nullify connection pointer when free()'edYang Tse
2011-10-12OOM handling/cleanup slight adjustmentsYang Tse
2011-09-20Curl_follow: handle redirects to "//hostname/path"Daniel Stenberg
2011-09-11spell-checked commentsDaniel Stenberg
2011-09-05fix bool variables checking and assignmentYang Tse
2011-09-02MemoryTracking: fix logging of free() calls done where Curl_safefree is calledYang Tse
Just internal stuff... Curl_safefree is now a macro defined in memdebug.h instead of a function prototyped in url.h and implemented in url.c, so inclusion of url.h is no longer required in order to simply use Curl_safefree. Provide definition of macro WHILE_FALSE in setup_once.h in order to allow other macros such as DEBUGF and DEBUGASSERT, and code using it, to compile without 'conditional expression is constant' warnings. The WHILE_FALSE stuff fixes 150+ MSVC compiler warnings.
2011-09-02transfer.c: fix parentheses/braces placement for vi/emacs delimiter matchingYang Tse
2011-09-02transfer.c: fix loadhostpairs() OOM handlingYang Tse
2011-08-28NTLM: END of refactoring/splitting/movingYang Tse
First: File curl_ntlm.h renamed curl_ntlm_msgs.h File curl_ntlm.c renamed curl_ntlm_msgs.c Afterwards: File http_ntlm.c renamed curl_ntlm.c File http_ntlm.h renamed curl_ntlm.h
2011-08-13Curl_retry_request: check return code!Daniel Stenberg
Curl_readrewind() was called without checking its return code, which could lead to badness. Bug: http://curl.haxx.se/bug/view.cgi?id=3349227