aboutsummaryrefslogtreecommitdiff
path: root/lib/multi.c
AgeCommit message (Collapse)Author
2020-05-04dynbuf: introduce internal generic dynamic buffer functionsDaniel Stenberg
A common set of functions instead of many separate implementations for creating buffers that can grow when appending data to them. Existing functionality has been ported over. In my early basic testing, the total number of allocations seem at roughly the same amount as before, possibly a few less. See docs/DYNBUF.md for a description of the API. Closes #5300
2020-04-30conncache: various concept cleanupsDaniel Stenberg
More connection cache accesses are protected by locks. CONNCACHE_* is a beter prefix for the connection cache lock macros. Curl_attach_connnection: now called as soon as there's a connection struct available and before the connection is added to the connection cache. Curl_disconnect: now assumes that the connection is already removed from the connection cache. Ref: #4915 Closes #5009
2020-03-18multi: Improve parameter check for curl_multi_remove_handleJay Satiro
- If an easy handle is owned by a multi different from the one specified then return CURLM_BAD_EASY_HANDLE. Prior to this change I assume user error could cause corruption. Closes https://github.com/curl/curl/pull/5116
2020-03-06multi: skip EINTR check on wakeup socket if it was closedJay Satiro
- Don't check errno on wakeup socket if sread returned 0 since sread doesn't set errno in that case. This is a follow-up to cf7760a from several days ago which fixed Curl_multi_wait to stop busy looping sread on the non-blocking wakeup socket if it was closed (ie sread returns 0). Due to a logic error it was still possible to busy loop in that case if errno == EINTR. Closes https://github.com/curl/curl/pull/5047
2020-03-02multi_wait: stop loop when sread() returns zeroDaniel Stenberg
It's unclear why it would ever return zero here, but this change fixes Robert's problem and it shouldn't loop forever... Reported-by: Robert Dunaj Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html Closes #5019
2020-02-17socks: make the connect phase non-blockingDaniel Stenberg
Removes two entries from KNOWN_BUGS. Closes #4907
2020-02-16multi: if Curl_readwrite sets 'comeback' use expire, not loopDaniel Stenberg
Otherwise, a very fast single transfer ricks starving out other concurrent transfers. Closes #4927
2020-02-12multi: fix outdated commentJay Satiro
- Do not say that conn->data is "cleared" by multi_done(). If the connection is in use then multi_done assigns another easy handle still using the connection to conn->data, therefore in that case it is not cleared. Closes https://github.com/curl/curl/pull/4901
2020-01-28multi_done: if multiplexed, make conn->data point to another transferDaniel Stenberg
... since the current transfer is being killed. Setting to NULL is wrong, leaving it pointing to 'data' is wrong since that handle might be about to get freed. Fixes #4845 Closes #4858 Reported-by: dmitrmax on github
2020-01-28global_init: move the IPv6 works status bool to multi handleDaniel Stenberg
Previously it was stored in a global state which contributed to curl_global_init's thread unsafety. This boolean is now instead figured out in curl_multi_init() and stored in the multi handle. Less effective, but thread safe. Closes #4851
2020-01-13ConnectionExists: respect the max_concurrent_streams limitsDaniel Stenberg
A regression made the code use 'multiplexed' as a boolean instead of the counter it is intended to be. This made curl try to "over-populate" connections with new streams. This regression came with 41fcdf71a1, shipped in curl 7.65.0. Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same check. Reported-by: Kunal Ekawde Fixes #4779 Closes #4784
2020-01-12misc: Copyright year out of date, should be 2020Daniel Stenberg
Follow-up to recent commits [skip ci]
2020-01-11multi: Change curl_multi_wait/poll to error on negative timeoutJay Satiro
- Add new error CURLM_BAD_FUNCTION_ARGUMENT and return that error when curl_multi_wait/poll is passed timeout param < 0. Prior to this change passing a negative value to curl_multi_wait/poll such as -1 could cause the function to wait forever. Reported-by: hamstergene@users.noreply.github.com Fixes https://github.com/curl/curl/issues/4763 Closes https://github.com/curl/curl/pull/4765
2019-12-27lib: remove erroneous +x file permission on some c filesXiang Xiao
Modified by commit eb9a604 accidentally. Closes https://github.com/curl/curl/pull/4756
2019-12-13multi: free sockhash on OOMDaniel Stenberg
This would otherwise leak memory in the error path. Detected by torture test 1540. Closes #4713
2019-12-09conn: always set bits.close with connclose()Daniel Stenberg
Closes #4690
2019-12-09conncache: fix multi-thread use of shared connection cacheDaniel Stenberg
It could accidentally let the connection get used by more than one thread, leading to double-free and more. Reported-by: Christopher Reid Fixes #4544 Closes #4557
2019-11-25multi: add curl_multi_wakeup()Gergely Nagy
This commit adds curl_multi_wakeup() which was previously in the TODO list under the curl_multi_unblock name. On some platforms and with some configurations this feature might not be available or can fail, in these cases a new error code (CURLM_WAKEUP_FAILURE) is returned from curl_multi_wakeup(). Fixes #4418 Closes #4608
2019-11-17multi: Fix curl_multi_poll wait when extra_fds && !extra_nfdsGergely Nagy
Prior to this change: The check if an extra wait is necessary was based not on the number of extra fds but on the pointer. If a non-null pointer was given in extra_fds, but extra_nfds was zero, then the wait was skipped even though poll was not called. Closes https://github.com/curl/curl/pull/4610
2019-11-14multi_poll: avoid busy-loop when called without easy handles attachedDaniel Stenberg
Fixes #4594 Closes #4595 Reported-by: 3dyd on github
2019-11-11remove_handle: clear expire timers after multi_done()Daniel Stenberg
Since 59041f0, a new timer might be set in multi_done() so the clearing of the timers need to happen afterwards! Reported-by: Max Kellermann Fixes #4575 Closes #4583
2019-10-02CURLMOPT_MAX_CONCURRENT_STREAMS: new setoptKunal Ekawde
Closes #4410
2019-09-20multi: value '2L' is assigned to a booleanDaniel Stenberg
Fixes warning detected by PVS-Studio Fixes #4374
2019-09-03cleanup: move functions out of url.c and make them staticDaniel Stenberg
Closes #4289
2019-08-14multi: getsock improvements for QUIC connectingDaniel Stenberg
2019-08-06curl_multi_poll: a sister to curl_multi_wait() that waits moreDaniel Stenberg
Repeatedly we see problems where using curl_multi_wait() is difficult or just awkward because if it has no file descriptor to wait for internally, it returns immediately and leaves it to the caller to wait for a small amount of time in order to avoid occasional busy-looping. This is often missed or misunderstood, leading to underperforming applications. This change introduces curl_multi_poll() as a replacement drop-in function that accepts the exact same set of arguments. This function works identically to curl_multi_wait() - EXCEPT - for the case when there's nothing to wait for internally, as then this function will by itself wait for a "suitable" short time before it returns. This effectiely avoids all risks of busy-looping and should also make it less likely that apps "over-wait". This also changes the curl tool to use this funtion internally when doing parallel transfers and changes curl_easy_perform() to use it internally. Closes #4163
2019-08-01timediff: make it 64 bit (if possible) even with 32 bit time_tDaniel Stenberg
... to make it hold microseconds too. Fixes #4165 Closes #4168
2019-07-30cleanup: remove the 'numsocks' argument used in many placesDaniel Stenberg
It was used (intended) to pass in the size of the 'socks' array that is also passed to these functions, but was rarely actually checked/used and the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries that should be used instead. Closes #4169
2019-07-30asyn-thread: create a socketpair to wait onamkatyal
Closes #4157
2019-07-25multi: call detach_connection before Curl_disconnectGergely Nagy
Curl_disconnect bails out if conn->easyq is not empty, detach_connection needs to be called first to remove the current easy from the queue. Fixes #4144 Closes #4151
2019-06-23multi: enable multiplexing by default (again)Daniel Stenberg
It was originally made default in d7c4213bd0c (7.62.0) but mistakenly reverted in commit 2f44e94efb3d (7.65.0). Now enabled again. Closes #4051
2019-06-14multi: fix the transfer hash functionDaniel Stenberg
Follow-up from 8b987cc7eb Reported-by: Tom van der Woerdt Fixes #4018 Closes #4024
2019-06-12multi: fix the transfer hashes in the socket hash entriesDaniel Stenberg
- The transfer hashes weren't using the correct keys so removing entries failed. - Simplified the iteration logic over transfers sharing the same socket and they now simply are set to expire and thus get handled in the "regular" timer loop instead. Reported-by: Tom van der Woerdt Fixes #4012 Closes #4014
2019-06-10multi: make sure 'data' can present in several sockhash entriesDaniel Stenberg
Since more than one socket can be used by each transfer at a given time, each sockhash entry how has its own hash table with transfers using that socket. In addition, the sockhash entry can now be marked 'blocked = TRUE'" which then makes the delete function just set 'removed = TRUE' instead of removing it "for real", as a way to not rip out the carpet under the feet of a parent function that iterates over the transfers of that same sockhash entry. Reported-by: Tom van der Woerdt Fixes #3961 Fixes #3986 Fixes #3995 Fixes #4004 Closes #3997
2019-06-09unpause: trigger a timeout for event-based transfersDaniel Stenberg
... so that timeouts or other state machine actions get going again after a changing pause state. For example, if the last delivery was paused there's no pending socket activity. Reported-by: sstruchtrup on github Fixes #3994 Closes #4001
2019-06-01singlesocket: use separate variable for inner loopDaniel Stenberg
An inner loop within the singlesocket() function wrongly re-used the variable for the outer loop which then could cause an infinite loop. Change to using a separate variable! Reported-by: Eric Wu Fixes #3970 Closes #3973
2019-05-28multi: track users of a socket betterDaniel Stenberg
They need to be removed from the socket hash linked list with more care. When sh_delentry() is called to remove a sockethash entry, remove all individual transfers from the list first. To enable this, each Curl_easy struct now stores a pointer to the sockethash entry to know how to remove itself. Reported-by: Tom van der Woerdt and Kunal Ekawde Fixes #3952 Fixes #3904 Closes #3953
2019-05-16cleanup: remove FIXME and TODO commentsDaniel Stenberg
They serve very little purpose and mostly just add noise. Most of them have been around for a very long time. I read them all before removing or rephrasing them. Ref: #3876 Closes #3883
2019-05-15http_negotiate: Move the Negotiate state out of the negotiatedata structureSteve Holme
Given that this member variable is not used by the SASL based protocols there is no need to have it here. Closes #3882
2019-05-15http_ntlm: Move the NTLM state out of the ntlmdata structureSteve Holme
Given that this member variable is not used by the SASL based protocols there is no need to have it here.
2019-05-13wildcard: disable from build when FTP isn't presentDaniel Stenberg
2019-05-12build: fix "clarify calculation precedence" warningsMarcel Raad
Codacy/CppCheck warns about this. Consistently use parentheses as we already do in some places to silence the warning. Closes https://github.com/curl/curl/pull/3866
2019-05-10Revert "multi: support verbose conncache closure handle"Jay Satiro
This reverts commit b0972bc. - No longer show verbose output for the conncache closure handle. The offending commit was added so that the conncache closure handle would inherit verbose mode from the user's easy handle. (Note there is no way for the user to set options for the closure handle which is why that was necessary.) Other debug settings such as the debug function were not also inherited since we determined that could lead to crashes if the user's per-handle private data was used on an unexpected handle. The reporter here says he has a debug function to capture the verbose output, and does not expect or want any output to stderr; however because the conncache closure handle does not inherit the debug function the verbose output for that handle does go to stderr. There are other plausible scenarios as well such as the user redirects stderr on their handle, which is also not inherited since it could lead to crashes when used on an unexpected handle. Short of allowing the user to set options for the conncache closure handle I don't think there's much we can safely do except no longer inherit the verbose setting. Bug: https://curl.haxx.se/mail/lib-2019-05/0021.html Reported-by: Kristoffer Gleditsch Ref: https://github.com/curl/curl/pull/3598 Ref: https://github.com/curl/curl/pull/3618 Closes https://github.com/curl/curl/pull/3856
2019-05-01multi: provide Curl_multiuse_state to update informationDaniel Stenberg
As soon as a TLS backend gets ALPN conformation about the specific HTTP version it can now set the multiplex situation for the "bundle" and trigger moving potentially queued up transfers to the CONNECT state.
2019-05-01process_pending_handles: mark queued transfers as previously pendingDaniel Stenberg
With transfers being queued up, we only move one at a a time back to the CONNECT state but now we mark moved transfers so that when a moved transfer is confirmed "successful" (it connected) it will trigger the move of another pending transfer. Previously, it would otherwise wait until the transfer was done before doing this. This makes queued up pending transfers get processed (much) faster.
2019-04-19multi: fix the statenames (follow-up fix from 2f44e94efb3df8e)Daniel Stenberg
The list of names must be in sync with the defined states in the header file!
2019-04-11CURLOPT_DNS_USE_GLOBAL_CACHE: removeDaniel Stenberg
Remove the code too. The functionality has been disabled in code since 7.62.0. Setting this option will from now on simply be ignored and have no function. Closes #3654
2019-04-07multi_runsingle(): fix use-after-freeEven Rouault
Fixes #3745 Closes #3746 The following snippet ``` int main() { CURL* hCurlHandle = curl_easy_init(); curl_easy_setopt(hCurlHandle, CURLOPT_URL, "http://example.com"); curl_easy_setopt(hCurlHandle, CURLOPT_PROXY, "1"); curl_easy_perform(hCurlHandle); curl_easy_cleanup(hCurlHandle); return 0; } ``` triggers the following Valgrind warning ``` ==4125== Invalid read of size 8 ==4125== at 0x4E7D1EE: Curl_llist_remove (llist.c:97) ==4125== by 0x4E7EF5C: detach_connnection (multi.c:798) ==4125== by 0x4E80545: multi_runsingle (multi.c:1451) ==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072) ==4125== by 0x4E766A0: easy_transfer (easy.c:625) ==4125== by 0x4E76915: easy_perform (easy.c:719) ==4125== by 0x4E7697C: curl_easy_perform (easy.c:738) ==4125== by 0x4008BE: main (in /home/even/curl/test) ==4125== Address 0x9b3d1d0 is 1,120 bytes inside a block of size 1,600 free'd ==4125== at 0x4C2ECF0: free (vg_replace_malloc.c:530) ==4125== by 0x4E62C36: conn_free (url.c:756) ==4125== by 0x4E62D34: Curl_disconnect (url.c:818) ==4125== by 0x4E48DF9: Curl_once_resolved (hostip.c:1097) ==4125== by 0x4E8052D: multi_runsingle (multi.c:1446) ==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072) ==4125== by 0x4E766A0: easy_transfer (easy.c:625) ==4125== by 0x4E76915: easy_perform (easy.c:719) ==4125== by 0x4E7697C: curl_easy_perform (easy.c:738) ==4125== by 0x4008BE: main (in /home/even/curl/test) ==4125== Block was alloc'd at ==4125== at 0x4C2F988: calloc (vg_replace_malloc.c:711) ==4125== by 0x4E6438E: allocate_conn (url.c:1654) ==4125== by 0x4E685B4: create_conn (url.c:3496) ==4125== by 0x4E6968F: Curl_connect (url.c:4023) ==4125== by 0x4E802E7: multi_runsingle (multi.c:1368) ==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072) ==4125== by 0x4E766A0: easy_transfer (easy.c:625) ==4125== by 0x4E76915: easy_perform (easy.c:719) ==4125== by 0x4E7697C: curl_easy_perform (easy.c:738) ==4125== by 0x4008BE: main (in /home/even/curl/test) ``` This has been bisected to commit 2f44e94 Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14109 Credit to OSS Fuzz
2019-04-06pipelining: removedDaniel Stenberg
As previously planned and documented in DEPRECATE.md, all pipelining code is removed. Closes #3651
2019-03-27multi: improved HTTP_1_1_REQUIRED handlingDaniel Stenberg
Make sure to downgrade to 1.1 even when we get this HTTP/2 stream error on first flight. Reported-by: niner on github Fixes #3696 Closes #3707
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-15 23:01:14 +0000