aboutsummaryrefslogtreecommitdiff
path: root/lib/multi.c
AgeCommit message (Collapse)Author
2019-01-11urldata: rename easy_conn to just connDaniel Stenberg
We use "conn" everywhere to be a pointer to the connection. Introduces two functions that "attaches" and "detaches" the connection to and from the transfer. Going forward, we should favour using "data->conn" (since a transfer always only has a single connection or none at all) to "conn->data" (since a connection can have none, one or many transfers associated with it and updating conn->data to be correct is error prone and a frequent reason for internal issues). Closes #3442
2019-01-10multi: multiplexing improvementsDaniel Stenberg
Fixes #3436 Closes #3448 Problem 1 After LOTS of scratching my head, I eventually realized that even when doing 10 uploads in parallel, sometimes the socket callback to the application that tells it what to wait for on the socket, looked like it would reflect the status of just the single transfer that just changed state. Digging into the code revealed that this was indeed the truth. When multiple transfers are using the same connection, the application did not correctly get the *combined* flags for all transfers which then could make it switch to READ (only) when in fact most transfers wanted to get told when the socket was WRITEABLE. Problem 1b A separate but related regression had also been introduced by me when I cleared connection/transfer association better a while ago, as now the logic couldn't find the connection and see if that was marked as used by more transfers and then it would also prematurely remove the socket from the socket hash table even in times other transfers were still using it! Fix 1 Make sure that each socket stored in the socket hash has a "combined" action field of what to ask the application to wait for, that is potentially the ORed action of multiple parallel transfers. And remove that socket hash entry only if there are no transfers left using it. Problem 2 The socket hash entry stored an association to a single transfer using that socket - and when curl_multi_socket_action() was called to tell libcurl about activities on that specific socket only that transfer was "handled". This was WRONG, as a single socket/connection can be used by numerous parallel transfers and not necessarily a single one. Fix 2 We now store a list of handles in the socket hashtable entry and when libcurl is told there's traffic for a particular socket, it now iterates over all known transfers using that single socket.
2019-01-07curl_multi_remove_handle() don't block terminating c-ares requestsBrad Spencer
Added Curl_resolver_kill() for all three resolver modes, which only blocks when necessary, along with test 1592 to confirm curl_multi_remove_handle() doesn't block unless it must. Closes #3428 Fixes #3371
2019-01-04printf: introduce CURL_FORMAT_TIMEDIFF_TRikard Falkeborn
2018-12-22disconnect: separate connections and easy handles betterDaniel Stenberg
Do not assume/store assocation between a given easy handle and the connection if it can be avoided. Long-term, the 'conn->data' pointer should probably be removed as it is a little too error-prone. Still used very widely though. Reported-by: masbug on github Fixes #3391 Closes #3400
2018-12-11multi: convert two timeout variables to timediff_tDaniel Stenberg
The time_t type is unsigned on some systems and these variables are used to hold return values from functions that return timediff_t already. timediff_t is always a signed type. Closes #3363
2018-12-08Upon HTTP_1_1_REQUIRED, retry the request with HTTP/1.1Johannes Schindelin
This is a companion patch to cbea2fd2c (NTLM: force the connection to HTTP/1.1, 2018-12-06): with NTLM, we can switch to HTTP/1.1 preemptively. However, with other (Negotiate) authentication it is not clear to this developer whether there is a way to make it work with HTTP/2, so let's try HTTP/2 first and fall back in case we encounter the error HTTP_1_1_REQUIRED. Note: we will still keep the NTLM workaround, as it avoids an extra round trip. Daniel Stenberg helped a lot with this patch, in particular by suggesting to introduce the Curl_h2_http_1_1_error() function. Closes #3349 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-11-25curl_easy_perform: fix timeout handlingDaniel Stenberg
curl_multi_wait() was erroneously used from within curl_easy_perform(). It could lead to it believing there was no socket to wait for and then instead sleep for a while instead of monitoring the socket and then miss acting on that activity as swiftly as it should (causing an up to 1000 ms delay). Reported-by: Antoni Villalonga Fixes #3305 Closes #3306 Closes #3308
2018-11-23snprintf: renamed and we now only use msnprintf()Daniel Stenberg
The function does not return the same value as snprintf() normally does, so readers may be mislead into thinking the code works differently than it actually does. A different function name makes this easier to detect. Reported-by: Tomas Hoger Assisted-by: Daniel Gustafsson Fixes #3296 Closes #3297
2018-11-20ares: remove fd from multi fd set when ares is about to close the fdRomain Fliedel
When using c-ares for asyn dns, the dns socket fd was silently closed by c-ares without curl being aware. curl would then 'realize' the fd has been removed at next call of Curl_resolver_getsock, and only then notify the CURLMOPT_SOCKETFUNCTION to remove fd from its poll set with CURL_POLL_REMOVE. At this point the fd is already closed. By using ares socket state callback (ARES_OPT_SOCK_STATE_CB), this patch allows curl to be notified that the fd is not longer needed for neither for write nor read. At this point by calling Curl_multi_closed we are able to notify multi with CURL_POLL_REMOVE before the fd is actually closed by ares. In asyn-ares.c Curl_resolver_duphandle we can't use ares_dup anymore since it does not allow passing a different sock_state_cb_data Closes #3238
2018-10-27multi: Fix error handling in the SENDPROTOCONNECT stateMichael Kaufmann
If Curl_protocol_connect() returns an error code, handle the error instead of switching to the next state. Closes #3170
2018-10-19multi: avoid double-freeDaniel Stenberg
Curl_follow() no longer frees the string. Make sure it happens in the caller function, like we normally handle allocations. This bug was introduced with the use of the URL API internally, it has never been in a release version Reported-by: Dario Weißer Closes #3149
2018-10-19multi: make the closure handle "inherit" CURLOPT_NOSIGNALDaniel Stenberg
Otherwise, closing that handle can still cause surprises! Reported-by: Martin Ankerl Fixes #3138 Closes #3147
2018-10-13Curl_follow: Always free the passed new URLMichael Kaufmann
Closes #3124
2018-09-29multi: fix memory leak in content encoding related error pathDaniel Stenberg
... a missing multi_done() call. Credit to OSS-Fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10728 Closes #3063
2018-09-28multi: fix location URL memleak in error pathDaniel Stenberg
Follow-up to #3044 - fix a leak OSS-Fuzz detected Closes #3057
2018-09-22url: use the URL API internally as wellDaniel Stenberg
... to make it a truly unified URL parser. Closes #3017
2018-09-18curl_multi_wait: call getsock before figuring out timeoutDaniel Stenberg
.... since getsock may update the expiry timer. Fixes #2996 Closes #3000
2018-09-06setopt: add CURLOPT_DOH_URLDaniel Stenberg
Closes #2668
2018-09-05multiplex: enable by defaultDaniel Stenberg
Starting 7.62.0, multiplexing is enabled by default in multi handles.
2018-09-05pipelining: deprecatedDaniel Stenberg
Transparently. The related curl_multi_setopt() options all still returns OK when pipelining is selected. To re-enable the support, the single line change in lib/multi.c needs to be reverted. See docs/DEPRECATE.md Closes #2705
2018-08-18upload: allocate upload buffer on-demandDaniel Stenberg
Saves 16KB on the easy handle for operations that don't need that buffer. Part 1 of #2888
2018-08-16urldata: remove unused pipe_broke struct fieldDaniel Stenberg
This struct field is never set TRUE in any existing code path. This change removes the field completely. Closes #2871
2018-07-31retry: return error if rewind was necessary but didn't happenDaniel Jelinski
Fixes #2801 Closes #2812
2018-07-11multi: always do the COMPLETED procedure/stateDaniel Stenberg
It was previously erroneously skipped in some situations. libtest/libntlmconnect.c wrongly depended on wrong behavior (that it would get a zero timeout) when no handles are "running" in a multi handle. That behavior is no longer present with this fix. Now libcurl will always return a -1 timeout when all handles are completed. Closes #2733
2018-07-11conn: remove the boolean 'inuse' fieldDaniel Stenberg
... as the usage needs to be counted.
2018-06-23multi: fix crash due to dangling entry in connect-pending listJavier Blazquez
Fixes #2677 Closes #2679
2018-06-16multi: fix memory leak when stopped during name resolveDaniel Stenberg
When the application just started the transfer and then stops it while the name resolve in the background thread hasn't completed, we need to wait for the resolve to complete and then cleanup data accordingly. Enabled test 1553 again and added test 1590 to also check when the host name resolves successfully. Detected by OSS-fuzz. Closes #1968
2018-06-11cppcheck: fix warningsMarian Klymov
- Get rid of variable that was generating false positive warning (unitialized) - Fix issues in tests - Reduce scope of several variables all over etc Closes #2631
2018-06-03spelling fixesViktor Szakats
Detected using the `codespell` tool (version 1.13.0). Also secure and fix an URL.
2018-06-01multi: remove a DEBUGF()Daniel Stenberg
... it might call infof() with a NULL first argument that isn't harmful but makes it not do anything. The infof() line is not very useful anymore, it has served it purpose. Good riddance! Fixes #2627
2018-05-31strictness: correct {infof, failf} format specifiersRikard Falkeborn
Closes #2623
2018-05-28psl: use latest psl and refresh it periodicallyPatrick Monnerat
The latest psl is cached in the multi or share handle. It is refreshed before use after 72 hours. New share lock CURL_LOCK_DATA_PSL controls the psl cache sharing. If the latest psl is not available, the builtin psl is used. Reported-by: Yaakov Selkowitz Fixes #2553 Closes #2601
2018-04-27checksrc: force indentation of lines after an elseDaniel Gustafsson
This extends the INDENTATION case to also handle 'else' statements and require proper indentation on the following line. Also fixes the offending cases found in the codebase. Closes #2532
2018-04-20http2: handle GOAWAY properlyDaniel Stenberg
When receiving REFUSED_STREAM, mark the connection for close and retry streams accordingly on another/fresh connection. Reported-by: Terry Wu Fixes #2416 Fixes #1618 Closes #2510
2018-03-17add_handle/easy_perform: clear errorbuffer on start if setDaniel Stenberg
To offer applications a more defined behavior, we clear the buffer as early as possible. Assisted-by: Jay Satiro Fixes #2190 Closes #2377
2018-03-16multi: improved pending transfers handling => improved performanceDaniel Stenberg
When a transfer is requested to get done and it is put in the pending queue when limited by number of connections, total or per-host, libcurl would previously very aggressively retry *ALL* pending transfers to get them transferring. That was very time consuming. By reducing the aggressiveness in how pending are being retried, we waste MUCH less time on putting transfers back into pending again. Some test cases got a factor 30(!) speed improvement with this change. Reported-by: Cyril B Fixes #2369 Closes #2383
2018-03-16pause: when changing pause state, update socket stateDaniel Stenberg
Especially unpausing a transfer might have to move the socket back to the "currently used sockets" hash to get monitored. Otherwise it would never get any more data and get stuck. Easily triggered with pausing using the multi_socket API. Reported-by: Philip Prindeville Bug: https://curl.haxx.se/mail/lib-2018-03/0048.html Fixes #2393 Closes #2391
2018-03-16rate-limit: use three second window to better handle high speedsDaniel Stenberg
Due to very frequent updates of the rate limit "window", it could attempt to rate limit within the same milliseconds and that then made the calculations wrong, leading to it not behaving correctly on very fast transfers. This new logic updates the rate limit "window" to be no shorter than the last three seconds and only updating the timestamps for this when switching between the states TOOFAST/PERFORM. Reported-by: 刘佩东 Fixes #2386 Closes #2388
2018-03-12http2: mark the connection for close on GOAWAYDaniel Stenberg
... don't consider it an error! Assisted-by: Jay Satiro Reported-by: Łukasz Domeradzki Fixes #2365 Closes #2375
2018-02-15TODO fixed: Detect when called from within callbacksBjörn Stenberg
Closes #2302
2018-01-18http2: don't close connection when single transfer is stoppedKartik Mahajan
Fixes #2237 Closes #2249
2017-12-10multi_done: prune DNS cacheDaniel Stenberg
Prune the DNS cache immediately after the dns entry is unlocked in multi_done. Timed out entries will then get discarded in a more orderly fashion. Test506 is updated Reported-by: Oleg Pudeyev Fixes #2169 Closes #2170
2017-12-05conncache: fix several lock issuesDaniel Stenberg
If the lock is released before the dealings with the bundle is over, it may have changed by another thread in the mean time. Fixes #2132 Fixes #2151 Closes #2139
2017-11-09share: add support for sharing the connection cacheDaniel Stenberg
2017-10-28Curl_timeleft: change return type to timediff_tDaniel Stenberg
returning 'time_t' is problematic when that type is unsigned and we return values less than zero to signal "already expired", used in several places in the code. Closes #2021
2017-10-26wildcards: don't use with non-supported protocolsMax Dymond
Fixes timeouts in the fuzzing tests for non-FTP protocols. Closes #2016
2017-10-25multi: allow table handle sizes to be overriddenMax Dymond
Allow users to specify their own hash define for CURL_CONNECTION_HASH_SIZE so that both values can be overridden. Closes #1982
2017-10-25time: rename Curl_tvnow to Curl_nowDaniel Stenberg
... since the 'tv' stood for timeval and this function does not return a timeval struct anymore. Also, cleaned up the Curl_timediff*() functions to avoid typecasts and clean up the descriptive comments. Closes #2011
2017-10-25timediff: return timediff_t from the time diff functionsDaniel Stenberg
... to cater for systems with unsigned time_t variables. - Renamed the functions to curlx_timediff and Curl_timediff_us. - Added overflow protection for both of them in either direction for both 32 bit and 64 bit time_ts - Reprefixed the curlx_time functions to use Curl_* Reported-by: Peter Piekarski Fixes #2004 Closes #2005
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-16 07:59:47 +0000