aboutsummaryrefslogtreecommitdiff
path: root/lib/multi.c
AgeCommit message (Collapse)Author
2013-06-11multi_socket: reduce timeout inaccuracy marginDaniel Stenberg
Allow less room for "triggered too early" mistakes by applications / timers on non-windows platforms. Starting now, we assume that a timeout call is never made earlier than 3 milliseconds before the actual timeout. This greatly improves timeout accuracy on Linux. Bug: http://curl.haxx.se/bug/view.cgi?id=1228 Reported-by: Hang Su
2013-06-04curl_multi_wait: only use internal timer if not -1Daniel Stenberg
commit 29bf0598aad5 introduced a problem when the "internal" timeout is prefered to the given if shorter, as it didn't consider the case where -1 was returned. Now the internal timeout is only considered if not -1. Reported-by: Tor Arntsen Bug: http://curl.haxx.se/mail/lib-2013-06/0015.html
2013-06-03curl_multi_wait: reduce timeout if the multi handle wants toDaniel Stenberg
If the multi handle's pending timeout is less than what is passed into this function, it will now opt to use the shorter time anyway since it is a very good hint that the handle wants to process something in a shorter time than what otherwise would happen. curl_multi_wait.3 was updated accordingly to clarify This is the reason for bug #1224 Bug: http://curl.haxx.se/bug/view.cgi?id=1224 Reported-by: Andrii Moiseiev
2013-06-03multi_runsingle: switch an if() condition for readabilityDaniel Stenberg
... because there's an identical check right next to it so using the operators in the check in the same order increases readability.
2013-05-30multi_runsingle: add braces to clarify the codeDaniel Stenberg
2013-04-26curl_easy_init: use less mallocsDaniel Stenberg
By introducing an internal alternative to curl_multi_init() that accepts parameters to set the hash sizes, easy handles will now use tiny socket and connection hash tables since it will only ever add a single easy handle to that multi handle. This decreased the number mallocs in test 40 (which is a rather simple and typical easy interface use case) from 1142 to 138. The maximum amount of memory allocated used went down from 118969 to 78805.
2013-03-21multi.c: Corrected a couple of violations of the curl coding standardsSteve Holme
Corrected some incorrectly positioned pointer variable declarations to be "type *" rather than "type* ".
2013-03-21multi.c: Fix compilation warningSteve Holme
warning: an enumerated type is mixed with another type
2013-03-20multi.c: fix compilation errorSteve Holme
warning: conversion from enumeration type to different enumeration type
2013-03-13Multiple pipelines and limiting the number of connections.Linus Nielsen Feltzing
Introducing a number of options to the multi interface that allows for multiple pipelines to the same host, in order to optimize the balance between the penalty for opening new connections and the potential pipelining latency. Two new options for limiting the number of connections: CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections to the same host. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished, so we can reuse the connection. CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished. The free connection will then be reused, if possible, or closed if the pending handle can't reuse it. Several new options for pipelining: CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a pipeline is "full" when a connection is to be reused, a new connection will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it. If not, the handle will be put in a pending state until a connection is ready (either free or a pipe got shorter). CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a transfer with a content length that is larger than this. CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a chunk larger than this. CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow pipelining. CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow pipelining. See the curl_multi_setopt() man page for details.
2013-03-09curl_multi_wait: avoid second loop if nothing to doDaniel Stenberg
... hopefully this will also make clang-analyzer stop warning on potentional NULL dereferences (which were false positives anyway).
2013-03-09multi_runsingle: avoid NULL dereferenceDaniel Stenberg
When Curl_do() returns failure, the connection pointer could be NULL so the code path following needs to that that into account. Bug: http://curl.haxx.se/mail/lib-2013-03/0062.html Reported by: Eric Hu
2013-02-10Fix NULL pointer reference when closing an unused multi handle.Linus Nielsen Feltzing
2013-01-17always-multi: always use non-blocking internalsDaniel Stenberg
Remove internal separated behavior of the easy vs multi intercace. curl_easy_perform() is now using the multi interface itself. Several minor multi interface quirks and bugs have been fixed in the process. Much help with debugging this has been provided by: Yang Tse
2013-01-09build: fix circular header inclusion with other packagesYang Tse
This commit renames lib/setup.h to lib/curl_setup.h and renames lib/setup_once.h to lib/curl_setup_once.h. Removes the need and usage of a header inclusion guard foreign to libcurl. [1] Removes the need and presence of an alarming notice we carried in old setup_once.h [2] ---------------------------------------- 1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H, this single inclusion guard is enough to ensure that inclusion of lib/setup_once.h done from lib/setup.h is only done once. Additionally lib/setup.h has always used __SETUP_ONCE_H macro to protect inclusion of setup_once.h even after commit ec691ca3, this was to avoid a circular header inclusion triggered when building a c-ares enabled version with c-ares sources available which also has a setup_once.h header. Commit ec691ca3 exposes the real nature of __SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard foreign to libcurl belonging to c-ares's setup_once.h The renaming this commit does, fixes the circular header inclusion, and as such removes the need and usage of a header inclusion guard foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl. 2 - Due to the circular interdependency of old lib/setup_once.h and the c-ares setup_once.h header, old file lib/setup_once.h has carried back from 2006 up to now days an alarming and prominent notice about the need of keeping libcurl's and c-ares's setup_once.h in sync. Given that this commit fixes the circular interdependency, the need and presence of mentioned notice is removed. All mentioned interdependencies come back from now old days when the c-ares project lived inside a curl subdirectory. This commit removes last traces of such fact.
2013-01-06Revert changes relative to lib/*.[ch] recent renamingYang Tse
This reverts renaming and usage of lib/*.h header files done 28-12-2012, reverting 2 commits: f871de0... build: make use of 76 lib/*.h renamed files ffd8e12... build: rename 76 lib/*.h files This also reverts removal of redundant include guard (redundant thanks to changes in above commits) done 2-12-2013, reverting 1 commit: c087374... curl_setup.h: remove redundant include guard This also reverts renaming and usage of lib/*.c source files done 3-12-2013, reverting 3 commits: 13606bb... build: make use of 93 lib/*.c renamed files 5b6e792... build: rename 93 lib/*.c files 7d83dff... build: commit 13606bbfde follow-up 1 Start of related discussion thread: http://curl.haxx.se/mail/lib-2013-01/0012.html Asking for confirmation on pushing this revertion commit: http://curl.haxx.se/mail/lib-2013-01/0048.html Confirmation summary: http://curl.haxx.se/mail/lib-2013-01/0079.html NOTICE: The list of 2 files that have been modified by other intermixed commits, while renamed, and also by at least one of the 6 commits this one reverts follows below. These 2 files will exhibit a hole in history unless git's '--follow' option is used when viewing logs. lib/curl_imap.h lib/curl_smtp.h
2013-01-03build: rename 93 lib/*.c filesYang Tse
93 lib/*.c source files renamed to use our standard naming scheme. This commit only does the file renaming. ---------------------------------------- renamed: lib/amigaos.c -> lib/curl_amigaos.c renamed: lib/asyn-ares.c -> lib/curl_asyn_ares.c renamed: lib/asyn-thread.c -> lib/curl_asyn_thread.c renamed: lib/axtls.c -> lib/curl_axtls.c renamed: lib/base64.c -> lib/curl_base64.c renamed: lib/bundles.c -> lib/curl_bundles.c renamed: lib/conncache.c -> lib/curl_conncache.c renamed: lib/connect.c -> lib/curl_connect.c renamed: lib/content_encoding.c -> lib/curl_content_encoding.c renamed: lib/cookie.c -> lib/curl_cookie.c renamed: lib/cyassl.c -> lib/curl_cyassl.c renamed: lib/dict.c -> lib/curl_dict.c renamed: lib/easy.c -> lib/curl_easy.c renamed: lib/escape.c -> lib/curl_escape.c renamed: lib/file.c -> lib/curl_file.c renamed: lib/fileinfo.c -> lib/curl_fileinfo.c renamed: lib/formdata.c -> lib/curl_formdata.c renamed: lib/ftp.c -> lib/curl_ftp.c renamed: lib/ftplistparser.c -> lib/curl_ftplistparser.c renamed: lib/getenv.c -> lib/curl_getenv.c renamed: lib/getinfo.c -> lib/curl_getinfo.c renamed: lib/gopher.c -> lib/curl_gopher.c renamed: lib/gtls.c -> lib/curl_gtls.c renamed: lib/hash.c -> lib/curl_hash.c renamed: lib/hmac.c -> lib/curl_hmac.c renamed: lib/hostasyn.c -> lib/curl_hostasyn.c renamed: lib/hostcheck.c -> lib/curl_hostcheck.c renamed: lib/hostip.c -> lib/curl_hostip.c renamed: lib/hostip4.c -> lib/curl_hostip4.c renamed: lib/hostip6.c -> lib/curl_hostip6.c renamed: lib/hostsyn.c -> lib/curl_hostsyn.c renamed: lib/http.c -> lib/curl_http.c renamed: lib/http_chunks.c -> lib/curl_http_chunks.c renamed: lib/http_digest.c -> lib/curl_http_digest.c renamed: lib/http_negotiate.c -> lib/curl_http_negotiate.c renamed: lib/http_negotiate_sspi.c -> lib/curl_http_negotiate_sspi.c renamed: lib/http_proxy.c -> lib/curl_http_proxy.c renamed: lib/idn_win32.c -> lib/curl_idn_win32.c renamed: lib/if2ip.c -> lib/curl_if2ip.c renamed: lib/imap.c -> lib/curl_imap.c renamed: lib/inet_ntop.c -> lib/curl_inet_ntop.c renamed: lib/inet_pton.c -> lib/curl_inet_pton.c renamed: lib/krb4.c -> lib/curl_krb4.c renamed: lib/krb5.c -> lib/curl_krb5.c renamed: lib/ldap.c -> lib/curl_ldap.c renamed: lib/llist.c -> lib/curl_llist.c renamed: lib/md4.c -> lib/curl_md4.c renamed: lib/md5.c -> lib/curl_md5.c renamed: lib/memdebug.c -> lib/curl_memdebug.c renamed: lib/mprintf.c -> lib/curl_mprintf.c renamed: lib/multi.c -> lib/curl_multi.c renamed: lib/netrc.c -> lib/curl_netrc.c renamed: lib/non-ascii.c -> lib/curl_non_ascii.c renamed: lib/curl_non-ascii.h -> lib/curl_non_ascii.h renamed: lib/nonblock.c -> lib/curl_nonblock.c renamed: lib/nss.c -> lib/curl_nss.c renamed: lib/nwlib.c -> lib/curl_nwlib.c renamed: lib/nwos.c -> lib/curl_nwos.c renamed: lib/openldap.c -> lib/curl_openldap.c renamed: lib/parsedate.c -> lib/curl_parsedate.c renamed: lib/pingpong.c -> lib/curl_pingpong.c renamed: lib/polarssl.c -> lib/curl_polarssl.c renamed: lib/pop3.c -> lib/curl_pop3.c renamed: lib/progress.c -> lib/curl_progress.c renamed: lib/qssl.c -> lib/curl_qssl.c renamed: lib/rawstr.c -> lib/curl_rawstr.c renamed: lib/rtsp.c -> lib/curl_rtsp.c renamed: lib/security.c -> lib/curl_security.c renamed: lib/select.c -> lib/curl_select.c renamed: lib/sendf.c -> lib/curl_sendf.c renamed: lib/share.c -> lib/curl_share.c renamed: lib/slist.c -> lib/curl_slist.c renamed: lib/smtp.c -> lib/curl_smtp.c renamed: lib/socks.c -> lib/curl_socks.c renamed: lib/socks_gssapi.c -> lib/curl_socks_gssapi.c renamed: lib/socks_sspi.c -> lib/curl_socks_sspi.c renamed: lib/speedcheck.c -> lib/curl_speedcheck.c renamed: lib/splay.c -> lib/curl_splay.c renamed: lib/ssh.c -> lib/curl_ssh.c renamed: lib/sslgen.c -> lib/curl_sslgen.c renamed: lib/ssluse.c -> lib/curl_ssluse.c renamed: lib/strdup.c -> lib/curl_strdup.c renamed: lib/strequal.c -> lib/curl_strequal.c renamed: lib/strerror.c -> lib/curl_strerror.c renamed: lib/strtok.c -> lib/curl_strtok.c renamed: lib/strtoofft.c -> lib/curl_strtoofft.c renamed: lib/telnet.c -> lib/curl_telnet.c renamed: lib/tftp.c -> lib/curl_tftp.c renamed: lib/timeval.c -> lib/curl_timeval.c renamed: lib/transfer.c -> lib/curl_transfer.c renamed: lib/url.c -> lib/curl_url.c renamed: lib/version.c -> lib/curl_version.c renamed: lib/warnless.c -> lib/curl_warnless.c renamed: lib/wildcard.c -> lib/curl_wildcard.c ----------------------------------------
2013-01-03build: make use of 93 lib/*.c renamed filesYang Tse
93 *.c source files renamed to use our standard naming scheme. This change affects 77 files in libcurl's source tree.
2012-12-28build: make use of 76 lib/*.h renamed filesYang Tse
76 private header files renamed to use our standard naming scheme. This change affects 322 files in libcurl's source tree.
2012-12-26curl_multi_remove_handle: commit 0aabfd9963 follow-upYang Tse
2012-12-25curl_multi_remove_handle: fix memory leak triggered with CURLOPT_RESOLVEYang Tse
2012-12-23curl_multi_wait: OOM handling fixYang Tse
2012-12-23curl_multi_wait: avoid an unnecessary memory allocationDaniel Stenberg
2012-12-21multi.c: OOM handling fixYang Tse
2012-12-19bundles connection caching: some out of memory handling fixesYang Tse
2012-12-14setup_once.h: refactor inclusion of <unistd.h> and <sys/socket.h>Yang Tse
Inclusion of top two most included header files now done in setup_once.h
2012-12-07Introducing a new persistent connection caching system using "bundles".Linus Nielsen Feltzing
A bundle is a list of all persistent connections to the same host. The connection cache consists of a hash of bundles, with the hostname as the key. The benefits may not be obvious, but they are two: 1) Faster search for connections to reuse, since the hash lookup only finds connections to the host in question. 2) It lays out the groundworks for an upcoming patch, which will introduce multiple HTTP pipelines. This patch also removes the awkward list of "closure handles", which were needed to send QUIT commands to the FTP server when closing a connection. Now we allocate a separate closure handle and use that one to close all connections. This has been tested in a live system for a few weeks, and of course passes the test suite.
2012-12-04multi: fix re-sending request on early connection closeDaniel Stenberg
This handling already works with the easy-interface code. When a request is sent on a re-used connection that gets closed by the server at the same time as the request is sent, the situation may occur so that we can send the request and we discover the broken connection as a RECV_ERROR in the PERFORM state and then the request needs to be retried on a fresh connection. Test 64 broke with 'multi-always-internally'.
2012-11-26multi.c: disambiguate precedence of bitwise and relational operationYang Tse
2012-11-18fixed memory leak: CURLOPT_RESOLVE with multi interfaceDaniel Stenberg
DNS cache entries populated with CURLOPT_RESOLVE were not properly freed again when done using the multi interface. Test case 1502 added to verify. Bug: http://curl.haxx.se/bug/view.cgi?id=3575448 Reported by: Alex Gruz
2012-10-09curl_multi_wait: no wait if no descriptors to wait forDaniel Stenberg
This is a minor change in behavior after having been pointed out by Mark Tully and discussed on the list. Initially this case would internally call poll() with no sockets and a timeout which would equal a sleep for that specified time. Bug: http://curl.haxx.se/mail/lib-2012-10/0076.html Reported by: Mark Tully
2012-10-02multi_runsingle: CURLOPT_LOW_SPEED_* fix for rate limitationDaniel Stenberg
During the periods of rate limitation, the speedcheck function wasn't called and thus the values weren't updated accordingly and it would then easily trigger wrongly once data got transferred again. Also, the progress callback's return code was not acknowledged in this state so it could make an "abort" return code to get ignored and not have the documented effect of aborting an ongoing transfer. Bug: http://curl.haxx.se/mail/lib-2012-09/0081.html Reported by: Jie He
2012-09-16curl_multi_wait: Add parameter to return number of active socketsSara Golemon
Minor change to recently introduced function. BC breaking, but since curl_multi_wait() doesn't exist in any releases that should be fine.
2012-09-01multi: add curl_multi_wait()Sara Golemon
/* * Name: curl_multi_wait() * * Desc: Poll on all fds within a CURLM set as well as any * additional fds passed to the function. * * Returns: CURLMcode type, general multi error code. */ CURL_EXTERN CURLMcode curl_multi_wait(CURLM *multi_handle, struct curl_waitfd extra_fds[], unsigned int extra_nfds, int timeout_ms);
2012-08-08add_next_timeout: minor restructure of codeDaniel Stenberg
By reading the ->head pointer and using that instead of the ->size number to figure out if there's a list remaining we avoid the (false positive) clang-analyzer warning that we might dereference of a null pointer.
2012-07-15multi_runsingle: added precaution against easy_conn NULL pointerDaniel Stenberg
In many states the easy_conn pointer is referenced and just assumed to be working. This is an added extra check since analyzing indicates there's a risk we can end up in these states with a NULL pointer there.
2012-06-12singlesocket: remove dead codeDaniel Stenberg
No need to check if 'entry' is non-NULL in a spot where it is already checked and guaranteed to be non-NULL. (Spotted by a Coverity scan)
2012-03-22CONNECT: fix multi interface regressionDaniel Stenberg
The refactoring of HTTP CONNECT handling in commit 41b0237834232 that made it protocol independent broke it for the multi interface. This fix now introduce a better state handling and moved some logic to the http_proxy.c source file. Reported by: Yang Tse Bug: http://curl.haxx.se/mail/lib-2012-03/0162.html
2011-12-20non-blocking active FTP: cleanup multi state usageDaniel Stenberg
Backpedaled out the funny double-change of state in the multi state machine by adding a new argument to the do_more() function to signal completion. This way it can remain in the DO_MORE state properly until done. Long term, the entire DO_MORE logic should be moved into the FTP code and be hidden from the multi code as the logic is only used for FTP.
2011-12-20FTP: perform active connections non-blockingGokhan Sengun
1- Two new error codes are introduced. CURLE_FTP_ACCEPT_FAILED to be set whenever ACCEPTing fails because of FTP server connected. CURLE_FTP_ACCEPT_TIMEOUT to be set whenever ACCEPTing timeouts. Neither of these errors are considered fatal and control connection remains OK because it could just be a firewall blocking server to connect to the client. 2- One new setopt option was introduced. CURLOPT_ACCEPTTIMEOUT_MS It sets the maximum amount of time FTP client is going to wait for a server to connect. Internal default accept timeout is 60 seconds.
2011-12-07multi interface: fix block when CONNECT_ONLY option is usedGokhan Sengun
2011-12-02multi interface: only use non-NULL function pointer!Daniel Stenberg
If the socket callback function pointer hasn't been set, we must not attempt to use it. Commit adc88ca20 made it more likely to occur.
2011-12-02multi: handle timeouts on DNS servers by checking for new socketsJason Glasgow
If the first name server is not available, the multi interface does not invoke the socket_cb when the DNS request to the first name server timesout. Ensure that the list of sockets are always updated after calling Curl_resolver_is_resolved. This bug can be reproduced if Curl is complied with --enable_ares and your code uses the multi socket interfaces and the CURLMOPT_SOCKETFUNCTION option. To test try: iptables -I INPUT \ -s $(sed -n -e '/name/{s/.* //p;q}' /etc/resolv.conf)/32 \ -j REJECT and then run a program which uses the multi-interface.
2011-11-01multi.c: OOM handling fixYang Tse
2011-10-27multi.c: OOM handling fixYang Tse
Fix curl_multi_cleanup() segfault when using weird cleanup sequence.
2011-10-27multi: start ftp state machine when switching to DO_MOREDaniel Stenberg
This extends the fix from commit d7934b8bd491 When the multi state is changed within the multi_runsingle from DOING to DO_MORE, we didn't immediately start the FTP state machine again. That then left the FTP state in FTP_STOP. When curl_multi_fdset() was subsequently called, the ftp_domore_getsock() function would return the wrong fd info. Reported by: Gokhan Sengun
2011-10-21curl_multi_fdset: correct fdset with FTP PORT useDaniel Stenberg
After a PORT has been issued, and the multi handle would switch to the CURLM_STATE_DO_MORE state (which is unique for FTP), libcurl would return the wrong fdset to wait for when curl_multi_fdset() is called. The code would blindly assume that it was waiting for a connect of the second connection, while that isn't true immediately after the PORT command. Also, the function multi.c:domore_getsock() was highly FTP-centric and therefore ugly to keep in protocol-agnostic code. I solved this problem by introducing a new function pointer in the Curl_handler struct called domore_getsock() which is only called during the DOMORE state for protocols that set that pointer. The new ftp.c:ftp_domore_getsock() function now returns fdset info about the control connection's command/response handling while such a state is in use, and goes over to waiting for a writable second connection first once the commands are done. The original problem could be seen by running test 525 and checking the time stamps in the FTP server log. I can verify that this fix at least fixes this problem. Bug: http://curl.haxx.se/mail/lib-2011-10/0250.html Reported by: Gokhan Sengun
2011-10-21multi.c: fix segfaultYang Tse
2011-10-13multi.c: OOM handling fixesYang Tse
Prevent modification of easy handle being added with curl_multi_add_handle() unless this function actually suceeds. Run Curl_posttransfer() to allow restoring of SIGPIPE handler when Curl_connect() fails early in multi_runsingle().
2011-10-11OOM handling/cleanup slight adjustmentsYang Tse