aboutsummaryrefslogtreecommitdiff
path: root/lib/multiif.h
AgeCommit message (Collapse)Author
2019-10-02CURLMOPT_MAX_CONCURRENT_STREAMS: new setoptKunal Ekawde
Closes #4410
2019-08-06curl_multi_poll: a sister to curl_multi_wait() that waits moreDaniel Stenberg
Repeatedly we see problems where using curl_multi_wait() is difficult or just awkward because if it has no file descriptor to wait for internally, it returns immediately and leaves it to the caller to wait for a small amount of time in order to avoid occasional busy-looping. This is often missed or misunderstood, leading to underperforming applications. This change introduces curl_multi_poll() as a replacement drop-in function that accepts the exact same set of arguments. This function works identically to curl_multi_wait() - EXCEPT - for the case when there's nothing to wait for internally, as then this function will by itself wait for a "suitable" short time before it returns. This effectiely avoids all risks of busy-looping and should also make it less likely that apps "over-wait". This also changes the curl tool to use this funtion internally when doing parallel transfers and changes curl_easy_perform() to use it internally. Closes #4163
2019-08-01timediff: make it 64 bit (if possible) even with 32 bit time_tDaniel Stenberg
... to make it hold microseconds too. Fixes #4165 Closes #4168
2019-07-10headers: Remove no longer exported functionsDaniel Gustafsson
There were a leftover few prototypes of Curl_ functions that we used to export but no longer do, this removes those prototypes and cleans up any comments still referring to them. Curl_write32_le(), Curl_strcpy_url(), Curl_strlen_url(), Curl_up_free() Curl_concat_url(), Curl_detach_connnection(), Curl_http_setup_conn() were made static in 05b100aee247bb9bec8e9a1b0166496aa4248d1c. Curl_http_perhapsrewind() made static in 574aecee208f79d391f10d57520b3. For the remainder, I didn't trawl the Git logs hard enough to capture their exact time of deletion, but they were all gone: Curl_splayprint(), Curl_http2_send_request(), Curl_global_host_cache_dtor(), Curl_scan_cache_used(), Curl_hostcache_destroy(), Curl_second_connect(), Curl_http_auth_stage() and Curl_close_connections(). Closes #4096 Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2019-06-09unpause: trigger a timeout for event-based transfersDaniel Stenberg
... so that timeouts or other state machine actions get going again after a changing pause state. For example, if the last delivery was paused there's no pending socket activity. Reported-by: sstruchtrup on github Fixes #3994 Closes #4001
2019-05-02multiif.h: remove unused protosDaniel Stenberg
... for functions related to pipelining. Those functions were removed in 2f44e94efb3df. Closes #3828
2019-05-01multi: provide Curl_multiuse_state to update informationDaniel Stenberg
As soon as a TLS backend gets ALPN conformation about the specific HTTP version it can now set the multiplex situation for the "bundle" and trigger moving potentially queued up transfers to the CONNECT state.
2019-04-06pipelining: removedDaniel Stenberg
As previously planned and documented in DEPRECATE.md, all pipelining code is removed. Closes #3651
2019-01-11urldata: rename easy_conn to just connDaniel Stenberg
We use "conn" everywhere to be a pointer to the connection. Introduces two functions that "attaches" and "detaches" the connection to and from the transfer. Going forward, we should favour using "data->conn" (since a transfer always only has a single connection or none at all) to "conn->data" (since a connection can have none, one or many transfers associated with it and updating conn->data to be correct is error prone and a frequent reason for internal issues). Closes #3442
2018-11-25curl_easy_perform: fix timeout handlingDaniel Stenberg
curl_multi_wait() was erroneously used from within curl_easy_perform(). It could lead to it believing there was no socket to wait for and then instead sleep for a while instead of monitoring the socket and then miss acting on that activity as swiftly as it should (causing an up to 1000 ms delay). Reported-by: Antoni Villalonga Fixes #3305 Closes #3306 Closes #3308
2018-11-20ares: remove fd from multi fd set when ares is about to close the fdRomain Fliedel
When using c-ares for asyn dns, the dns socket fd was silently closed by c-ares without curl being aware. curl would then 'realize' the fd has been removed at next call of Curl_resolver_getsock, and only then notify the CURLMOPT_SOCKETFUNCTION to remove fd from its poll set with CURL_POLL_REMOVE. At this point the fd is already closed. By using ares socket state callback (ARES_OPT_SOCK_STATE_CB), this patch allows curl to be notified that the fd is not longer needed for neither for write nor read. At this point by calling Curl_multi_closed we are able to notify multi with CURL_POLL_REMOVE before the fd is actually closed by ares. In asyn-ares.c Curl_resolver_duphandle we can't use ares_dup anymore since it does not allow passing a different sock_state_cb_data Closes #3238
2018-03-16multi: improved pending transfers handling => improved performanceDaniel Stenberg
When a transfer is requested to get done and it is put in the pending queue when limited by number of connections, total or per-host, libcurl would previously very aggressively retry *ALL* pending transfers to get them transferring. That was very time consuming. By reducing the aggressiveness in how pending are being retried, we waste MUCH less time on putting transfers back into pending again. Some test cases got a factor 30(!) speed improvement with this change. Reported-by: Cyril B Fixes #2369 Closes #2383
2018-03-16pause: when changing pause state, update socket stateDaniel Stenberg
Especially unpausing a transfer might have to move the socket back to the "currently used sockets" hash to get monitored. Otherwise it would never get any more data and get stuck. Easily triggered with pausing using the multi_socket API. Reported-by: Philip Prindeville Bug: https://curl.haxx.se/mail/lib-2018-03/0048.html Fixes #2393 Closes #2391
2018-02-15TODO fixed: Detect when called from within callbacksBjörn Stenberg
Closes #2302
2017-06-08expire: remove Curl_expire_latest()Daniel Stenberg
With the introduction of expire IDs and the fact that existing timers can be removed now and thus never expire, the concept with adding a "latest" timer is not working anymore as it risks to not expire at all. So, to be certain the timers actually are in line and will expire, the plain Curl_expire() needs to be used. The _latest() function was added as a sort of shortcut in the past that's quite simply not necessary anymore. Follow-up to 31b39c40cf90 Reported-by: Paul Harris Closes #1555
2017-05-10multi: use a fixed array of timers instead of mallocDaniel Stenberg
... since the total amount is low this is faster, easier and reduces memory overhead. Also, Curl_expire_done() can now mark an expire timeout as done so that it never times out. Closes #1472
2017-05-10multi: assign IDs to all timers and make each timer singletonDaniel Stenberg
A) reduces the timeout lists drastically B) prevents a lot of superfluous loops for timers that expires "in vain" when it has actually already been extended to fire later on
2016-11-18lib: fix compiler warnings after de4de4e3c7cMarcel Raad
Visual C++ now complains about implicitly casting time_t (64-bit) to long (32-bit). Fix this by changing some variables from long to time_t, or explicitly casting to long where the public interface would be affected. Closes #1131
2016-08-04multi: make Curl_expire() work with 0 ms timeoutsDaniel Stenberg
Previously, passing a timeout of zero to Curl_expire() was a magic code for clearing all timeouts for the handle. That is now instead made with the new Curl_expire_clear() function and thus a 0 timeout is fine to set and will trigger a timeout ASAP. This will help removing short delays, in particular notable when doing HTTP/2.
2016-06-22internals: rename the SessionHandle struct to Curl_easyDaniel Stenberg
2016-02-03URLs: change all http:// URLs to https://Daniel Stenberg
2015-06-24http2: setup the new pushed stream properlyDaniel Stenberg
2015-05-18http2: set default concurrency, fix ConnectionExists for multiplexDaniel Stenberg
2015-05-18bundles: store no/default/pipeline/multiplexDaniel Stenberg
to allow code to act differently on the situation. Also added some more info message for the connection re-use function to make it clearer when connections are not re-used.
2015-05-18CURLMOPT_PIPELINE: bit 1 is for multiplexingDaniel Stenberg
2014-11-19multi: inform about closed sockets before they are closedJon Spencer
When the connection code decides to close a socket it informs the multi system via the Curl_multi_closed function. The multi system may, in turn, invoke the CURLMOPT_SOCKETFUNCTION function with CURL_POLL_REMOVE. This happens after the socket has already been closed. Reorder the code so that CURL_POLL_REMOVE is called before the socket is closed.
2014-11-10multi: removed Curl_multi_set_easy_connectionDaniel Stenberg
It isn't used anywhere! Reported-by: Carlo Wood
2014-08-31low-speed-limit: avoid timeout floodDaniel Stenberg
Introducing Curl_expire_latest(). To be used when we the code flow only wants to get called at a later time that is "no later than X" so that something can be checked (and another timeout be added). The low-speed logic for example could easily be made to set very many expire timeouts if it would be called faster or sooner than what it had set its own timer and this goes for a few other timers too that aren't explictiy checked for timer expiration in the code. If there's no condition the code that says if(time-passed >= TIME), then Curl_expire_latest() is preferred to Curl_expire(). If there exists such a condition, it is on the other hand important that Curl_expire() is used and not the other. Bug: http://curl.haxx.se/mail/lib-2014-06/0235.html Reported-by: Florian Weimer
2014-01-12multi: remove MULTI_TIMEOUT_INACCURACYDaniel Stenberg
With the recently added timeout "reminder" functionality, there's no reason left for us to execute timeout code before the time is ripe. Simplifies the handling too. This will make the *TIMEOUT and *CONNECTTIMEOUT options more accurate again, which probably is most important when the *_MS versions are used. In multi_socket, make sure to update 'now' after having handled activity on a socket.
2013-12-15multi: add timer inaccuracy margin to timeout/connecttimeoutDaniel Stenberg
Since all systems have inaccuracy in the timeout handling it is imperative that we add an inaccuracy margin to the general timeout and connecttimeout handling with the multi interface. This way, when the timeout fires we should be fairly sure that it has passed the timeout value and will be suitably detected. For cases where the timeout fire before the actual timeout, we would otherwise consume the timeout action and still not run the timeout code since the condition wasn't met. Reported-by: He Qin Bug: http://curl.haxx.se/bug/view.cgi?id=1298
2013-08-28multi_socket: improved 100-continue timeout handlingDaniel Stenberg
When waiting for a 100-continue response from the server, the Curl_readwrite() will refuse to run if called until the timeout has been reached. We timeout code in multi_socket() allows code to run slightly before the actual timeout time, so for test 154 it could lead to the function being executed but refused in Curl_readwrite() and then the application would just sit idling forever. This was detected with runtests.pl -e on test 154.
2013-06-20multi_socket: react on socket close immediatelyDaniel Stenberg
As a remedy to the problem when a socket gets closed and a new one is opened with the same file descriptor number and as a result multi.c:singlesocket() doesn't detect the difference, the new function Curl_multi_closed() gets told when a socket is closed so that it can be removed from the socket hash. When the old one has been removed, a new socket should be detected fine by the singlesocket() on next invoke. Bug: http://curl.haxx.se/bug/view.cgi?id=1248 Reported-by: Erik Johansson
2013-04-26curl_easy_init: use less mallocsDaniel Stenberg
By introducing an internal alternative to curl_multi_init() that accepts parameters to set the hash sizes, easy handles will now use tiny socket and connection hash tables since it will only ever add a single easy handle to that multi handle. This decreased the number mallocs in test 40 (which is a rather simple and typical easy interface use case) from 1142 to 138. The maximum amount of memory allocated used went down from 118969 to 78805.
2013-03-13Multiple pipelines and limiting the number of connections.Linus Nielsen Feltzing
Introducing a number of options to the multi interface that allows for multiple pipelines to the same host, in order to optimize the balance between the penalty for opening new connections and the potential pipelining latency. Two new options for limiting the number of connections: CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections to the same host. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished, so we can reuse the connection. CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total. When adding a handle that exceeds this limit, that handle will be put in a pending state until another handle is finished. The free connection will then be reused, if possible, or closed if the pending handle can't reuse it. Several new options for pipelining: CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a pipeline is "full" when a connection is to be reused, a new connection will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it. If not, the handle will be put in a pending state until a connection is ready (either free or a pipe got shorter). CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a transfer with a content length that is larger than this. CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not be reused if it is currently processing a chunk larger than this. CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow pipelining. CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow pipelining. See the curl_multi_setopt() man page for details.
2013-01-06Revert changes relative to lib/*.[ch] recent renamingYang Tse
This reverts renaming and usage of lib/*.h header files done 28-12-2012, reverting 2 commits: f871de0... build: make use of 76 lib/*.h renamed files ffd8e12... build: rename 76 lib/*.h files This also reverts removal of redundant include guard (redundant thanks to changes in above commits) done 2-12-2013, reverting 1 commit: c087374... curl_setup.h: remove redundant include guard This also reverts renaming and usage of lib/*.c source files done 3-12-2013, reverting 3 commits: 13606bb... build: make use of 93 lib/*.c renamed files 5b6e792... build: rename 93 lib/*.c files 7d83dff... build: commit 13606bbfde follow-up 1 Start of related discussion thread: http://curl.haxx.se/mail/lib-2013-01/0012.html Asking for confirmation on pushing this revertion commit: http://curl.haxx.se/mail/lib-2013-01/0048.html Confirmation summary: http://curl.haxx.se/mail/lib-2013-01/0079.html NOTICE: The list of 2 files that have been modified by other intermixed commits, while renamed, and also by at least one of the 6 commits this one reverts follows below. These 2 files will exhibit a hole in history unless git's '--follow' option is used when viewing logs. lib/curl_imap.h lib/curl_smtp.h
2012-12-28build: rename 76 lib/*.h filesYang Tse
76 private header files renamed to use our standard naming scheme. This commit only does the file renaming. ---------------------------------------- renamed: amigaos.h -> curl_amigaos.h renamed: arpa_telnet.h -> curl_arpa_telnet.h renamed: asyn.h -> curl_asyn.h renamed: axtls.h -> curl_axtls.h renamed: bundles.h -> curl_bundles.h renamed: conncache.h -> curl_conncache.h renamed: connect.h -> curl_connect.h renamed: content_encoding.h -> curl_content_encoding.h renamed: cookie.h -> curl_cookie.h renamed: cyassl.h -> curl_cyassl.h renamed: dict.h -> curl_dict.h renamed: easyif.h -> curl_easyif.h renamed: escape.h -> curl_escape.h renamed: file.h -> curl_file.h renamed: fileinfo.h -> curl_fileinfo.h renamed: formdata.h -> curl_formdata.h renamed: ftp.h -> curl_ftp.h renamed: ftplistparser.h -> curl_ftplistparser.h renamed: getinfo.h -> curl_getinfo.h renamed: gopher.h -> curl_gopher.h renamed: gtls.h -> curl_gtls.h renamed: hash.h -> curl_hash.h renamed: hostcheck.h -> curl_hostcheck.h renamed: hostip.h -> curl_hostip.h renamed: http.h -> curl_http.h renamed: http_chunks.h -> curl_http_chunks.h renamed: http_digest.h -> curl_http_digest.h renamed: http_negotiate.h -> curl_http_negotiate.h renamed: http_proxy.h -> curl_http_proxy.h renamed: if2ip.h -> curl_if2ip.h renamed: imap.h -> curl_imap.h renamed: inet_ntop.h -> curl_inet_ntop.h renamed: inet_pton.h -> curl_inet_pton.h renamed: krb4.h -> curl_krb4.h renamed: llist.h -> curl_llist.h renamed: memdebug.h -> curl_memdebug.h renamed: multiif.h -> curl_multiif.h renamed: netrc.h -> curl_netrc.h renamed: non-ascii.h -> curl_non-ascii.h renamed: nonblock.h -> curl_nonblock.h renamed: nssg.h -> curl_nssg.h renamed: parsedate.h -> curl_parsedate.h renamed: pingpong.h -> curl_pingpong.h renamed: polarssl.h -> curl_polarssl.h renamed: pop3.h -> curl_pop3.h renamed: progress.h -> curl_progress.h renamed: qssl.h -> curl_qssl.h renamed: rawstr.h -> curl_rawstr.h renamed: rtsp.h -> curl_rtsp.h renamed: select.h -> curl_select.h renamed: sendf.h -> curl_sendf.h renamed: setup.h -> curl_setup.h renamed: setup_once.h -> curl_setup_once.h renamed: share.h -> curl_share.h renamed: slist.h -> curl_slist.h renamed: smtp.h -> curl_smtp.h renamed: sockaddr.h -> curl_sockaddr.h renamed: socks.h -> curl_socks.h renamed: speedcheck.h -> curl_speedcheck.h renamed: splay.h -> curl_splay.h renamed: ssh.h -> curl_ssh.h renamed: sslgen.h -> curl_sslgen.h renamed: ssluse.h -> curl_ssluse.h renamed: strdup.h -> curl_strdup.h renamed: strequal.h -> curl_strequal.h renamed: strerror.h -> curl_strerror.h renamed: strtok.h -> curl_strtok.h renamed: strtoofft.h -> curl_strtoofft.h renamed: telnet.h -> curl_telnet.h renamed: tftp.h -> curl_tftp.h renamed: timeval.h -> curl_timeval.h renamed: transfer.h -> curl_transfer.h renamed: url.h -> curl_url.h renamed: urldata.h -> curl_urldata.h renamed: warnless.h -> curl_warnless.h renamed: wildcard.h -> curl_wildcard.h ----------------------------------------
2012-12-28lib/*.h: use our standard naming scheme for header inclusion guardsYang Tse
2010-08-15multi: support timeoutsDaniel Stenberg
Curl_expire() is now expanded to hold a list of timeouts for each easy handle. Only the closest in time will be the one used as the primary timeout for the handle and will be used for the splay tree (which sorts and lists all handles within the multi handle). When the main timeout has triggered/expired, the next timeout in time that is kept in the list will be moved to the main timeout position and used as the key to splay with. This way, all timeouts that are set with Curl_expire() internally will end up as a proper timeout. Previously any Curl_expire() that set a _later_ timeout than what was already set was just silently ignored and thus missed. Setting Curl_expire() with timeout 0 (zero) will cancel all previously added timeouts. Corrects known bug #62.
2010-03-24remove the CVSish $Id$ linesDaniel Stenberg
2009-06-10Adjusted to take in account that...Yang Tse
With the curl memory tracking feature decoupled from the debug build feature, CURLDEBUG and DEBUGBUILD preprocessor symbol definitions are used as follows: CURLDEBUG used for curl debug memory tracking specific code (--enable-curldebug) DEBUGBUILD used for debug enabled specific code (--enable-debug)
2008-09-13fix compiler warning: defined but not usedYang Tse
2007-08-26Fixed some minor mismatched types found by splint.Dan Fandrich
2007-04-10Ravi Pratap provided fixes for HTTP pipeliningDaniel Stenberg
2006-09-28Reported in #1561470 (http://curl.haxx.se/bug/view.cgi?id=1561470), libcurlDaniel Stenberg
would crash if a bad function sequence was used when shutting down after using the multi interface (i.e using easy_cleanup after multi_cleanup) so precautions have been added to make sure it doesn't any more - test case 529 was added to verify.
2006-09-07Major overhaul introducing http pipelining support and shared connectionDaniel Stenberg
cache within the multi handle.
2006-04-10First curl_multi_socket() commit. Should primarily be considered as an internalDaniel Stenberg
code rearrange to fit the future better.
2005-03-31Updated the copyright year since changes have been this year.Daniel Stenberg
2005-02-09FTP code turned into state machine. Not completely yet, but a good start.Daniel Stenberg
The tag 'before_ftp_statemachine' was set just before this commit in case of future need.
2005-01-11Renamed easy.h and multi.h to easyif.h and multiif.h to make sure they don'tDaniel Stenberg
shadow our public headers with the former names.