2851 Commits

Author SHA1 Message Date
HoneyryderChuck
1c880127d7 Merge branch 'gh-98' into 'master'
pool fix: when connection is acquired after waiting on it, return it...

See merge request os85/httpx!402
2025-09-01 09:16:07 +00:00
HoneyryderChuck
fb8ebd57d4 Merge branch 'issue-354' into 'master'
callbacks plugin: propagated callbacks to derived plugins

Closes #354

See merge request os85/httpx!401
2025-08-23 23:42:37 +00:00
HoneyryderChuck
09f1309dcf pool fix: when connection is acquired after waiting on it, return it immediately, instead of bookkeeping on max connections
the previous state was acquiring the connection AND reacting on the number of available connections
2025-08-22 16:57:37 +01:00
HoneyryderChuck
34237705ab callbacks plugin: propagated callbacks to derived plugins
same fix as done recently with #with, but different function
2025-08-22 16:25:21 +01:00
HoneyryderChuck
0643c696ab fixing local proxy list urls 2025-08-22 16:11:44 +01:00
HoneyryderChuck
35afcc51f6 abstract IO#wait call to workaround the jruby bug
https://github.com/jruby/jruby/issues/8917
2025-08-19 23:55:48 +01:00
HoneyryderChuck
bb6907e4ee Merge branch 'gh-94' into 'master'
Fixes and improvements for fiber scheduler integration

See merge request os85/httpx!395
2025-08-19 17:34:09 +00:00
HoneyryderChuck
14734d510e align nil inits 2025-08-19 18:14:55 +01:00
HoneyryderChuck
09342187f5 removing fiber reference when emitting request completion event 2025-08-19 18:14:55 +01:00
HoneyryderChuck
8970767954 http1: also take into account buffered requests when looking for current context
the previous patch was only considering the current inflight request and the pending requests, but in a scenario where multiple requests may have been buffered and considered in theory in-flight, this information would pass by, leading to a busy loop
2025-08-19 18:14:55 +01:00
HoneyryderChuck
6782928b99 do not allow deactivating connections which are still waiting for the ack of the PING frame
this causes a very subtle bug where, in a multifiber scenario, when the last active stream is closed and the connection is checked-in, only to be checked out later and pinged for liveness, it may get caught in the termination flow of a session, which will try to deactivateconnections from the selector; given that only inflight requests were taken into account, it'd get deactivated and checked in, which would mess with the fiber which was actively using it

this fixes it by considering the ping state in the decision on whether the connection should be deactivated; when having an inflight ping, keep it open and in the selector
2025-08-19 18:14:55 +01:00
HoneyryderChuck
2c9093f20b http1: check interests also in the case where one of the pending requests is from the current context 2025-08-19 18:14:55 +01:00
HoneyryderChuck
f003340e60 selector: skip early when io is closed
it was needlessly logging io information despite it not being taken into account for selection
2025-08-19 18:14:55 +01:00
HoneyryderChuck
8d811b133b adapted hackernews script to also test async 2025-08-19 18:14:55 +01:00
HoneyryderChuck
3e06cff2a1 adapt the hackernews pages script to be also ran in async 2025-08-19 18:14:55 +01:00
HoneyryderChuck
3cfda42878 workaround for IO#wait bug when passed a nil timeout as first argument
https://bugs.ruby-lang.org/issues/21509
2025-08-19 18:14:55 +01:00
HoneyryderChuck
aa03a3a52e set fiber context for sent request
this is used downstream to inform the selector (via IO interest calculation) whether the connection should wait on events for the current fiber (i.e. the connection should be used for requests of the current fiber, or DNS requests for the connection of such requests must be pending)
2025-08-19 18:14:55 +01:00
HoneyryderChuck
37cc138bef improve interest calculation
do not check buffer before going to the parsers, which also take that account

for http2, use the http2 connection send buffer (instead of the connection buffer) to figure out whether it's waiting on a WINDOW_UPDATE, which is more correct and efficient

on http1, the logic is the same, but the code is simplified, as the checks become the same regardless of whether request or @requests.first is picked up
2025-08-19 18:14:55 +01:00
HoneyryderChuck
ded1b90ae3 added test to reproduce the issue with the skipped query under the fiber scheduler 2025-08-19 18:14:55 +01:00
HoneyryderChuck
35c15f7fa7 selector: improve prefiltering of selectable ios
the decision to use IO.select was based on the number of selectables, however only on select_many the actual selectables would be prefiltered before being passed to it, in some cases meaning that only one descriptor would be passed to IO.select (instead of IO#wait_readable or the like)

in fiber-scheduler frameworks, this meant going a potentially more inefficient route needlessly, such as in async, where IO.wait_readable and the like are performed using epoll or kevent, whereas IO.select calls are executed in a background thread
2025-08-19 18:14:54 +01:00
HoneyryderChuck
3852e2cdf4 native dns: do not write query to buffer if waiting on a previous reply
in a normal situation this would not happen, as events on a resolver would be dealt one at a time, but in a fiber-scheduler environment like async, the first query could be buffered in the 1st fiber switch, the second could then be enqueued on 2nd, then flush buffer and fiber switch on read, and a third query would then enter a corrupt state where, due to the buffer having been flushed on the 2nd fiber, write the 3rd query before receiving the second one, and messing up the pending query bookkeeping, making the fiber then block (due to DNS query never returning back) and expose that as an empty interest registration from some other connection on the selector

this is fixed by using @timer as a proxy for knowing that a given DNS query has been flushed but is still waiting on a response (or an eventual retry)
2025-08-19 18:13:56 +01:00
HoneyryderChuck
0261449b39 fixed sig for callbacks_for 2025-08-08 17:06:03 +01:00
HoneyryderChuck
84c8126cd9 callback_for: check for ivar existence first
Closes #353
2025-08-08 16:30:17 +01:00
HoneyryderChuck
ff3f1f726f fix warning about argument potentially being ignored 2025-08-07 12:34:59 +01:00
HoneyryderChuck
b8b710470c fix sentry deprecation 2025-08-07 12:30:31 +01:00
HoneyryderChuck
0f3e3ab068 remove trailing :: from IO module usage, as there's no more internal module 2025-08-07 12:30:21 +01:00
HoneyryderChuck
095fbb3463 using local aws for the max requests tests
reduce exposure to httpbin.org even more
2025-08-07 12:12:50 +01:00
HoneyryderChuck
7790589c1f linting issue 2025-08-07 11:28:18 +01:00
HoneyryderChuck
dd8608ec3b small improv in max requests tests to make it tolerant to multi-homed networks 2025-08-07 11:22:29 +01:00
HoneyryderChuck
8205b351aa removing usage of httpbin.org peer in tests wherever possible
it has been quite unstable, 503'ing often
2025-08-07 11:21:59 +01:00
HoneyryderChuck
5992628926 update nghttp2 used in CI tests 2025-08-07 11:21:02 +01:00
HoneyryderChuck
39370b5883 Merge branch 'issue-337' into 'master'
fix for issues blocking reconnection in proxy mode

Closes #337

See merge request os85/httpx!397
2025-07-30 09:49:51 +00:00
HoneyryderChuck
1801a7815c http2 parser: fix calculation when connection closes and there's no termination handshake 2025-07-18 17:48:23 +01:00
HoneyryderChuck
0953e4f91a fix for #receive_requests bailout routing when out of selectables
the routine was using #fetch_response, which may return nil, and wasn't handling it, making it potentially return a nil instead of a response/errorresponse object. since, depending on the plugins, #fetch_response may reroute requests, one allows to keep in the loop in case there are selectables again to process as a result of it
2025-07-18 17:48:23 +01:00
HoneyryderChuck
a78a3f0b7c proxy fixes: allow proxy connection errors to be retriable
when coupled with the retries plugin, the exception is raised inside send_request, which breaks the integration; in order to protect from it, the proxy plugin will protect from proxy connection errors (socket/timeout errors happening until tunnel established) and allow them to be retried, while ignoring other proxy errors; meanwhile, the naming of errors was simplified, and now there's an HTTPX::ProxyError replacing HTTPX::HTTPProxyError (which is a breaking change).
2025-07-18 17:48:23 +01:00
HoneyryderChuck
aeb8fe5382 fix proxy ssl reconnection
when a proxied ssl connection would be lost, standard reconnection wouldn't work, as it would not pick the information from the internal tcp socket. in order to fix this, the connection retrieves the proxied io on reset/purge, which makes the establish a new proxyssl connection on reconnect
2025-07-18 17:48:23 +01:00
HoneyryderChuck
03170b6c89 promote certain transition logs to regular code (under level 3)
not really useful as telemetry metered, but would have been useful for other bugs
2025-07-18 17:48:23 +01:00
HoneyryderChuck
814d607a45 Revert "options: initialize all possible options to improve object shape"
This reverts commit f64c3ab5990b68f850d0d190535a45162929f0af.
2025-07-18 17:47:08 +01:00
HoneyryderChuck
5502332e7e logging when connections are deregistered from the selector/pool
also, logging when a response is fetched in the session
2025-07-18 17:46:43 +01:00
HoneyryderChuck
f3b68950d6 adding current fiber id to log message tags 2025-07-18 17:45:21 +01:00
HoneyryderChuck
2c4638784f Merge branch 'fix-shape' into 'master'
object shape improvements

See merge request os85/httpx!396
2025-07-14 15:38:19 +00:00
HoneyryderChuck
b0016525e3 recover from network unreachable errors when using cached IPs
while this type of error is avoided when doing HEv2, the IPs remain
in the cache; this means that, one the same host is reached, the
IPs are loaded onto the same socket, and if the issue is IPv6
connectivity, it'll break outside of the HEv2 flow.

this error is now protected inside the connect block, so that other
IPs in the list can be tried after; the IP is then evicted from the
cachee.

HEv2 related regression test is disabled in CI, as it's currently
reliable in Gitlab CI, which allows to resolve the IPv6 address,
but does not allow connecting to it
2025-07-14 15:44:47 +01:00
HoneyryderChuck
49555694fe remove check for non unique local ipv6 which is disabling HEv2
not sure anymore under which condition this was done...
2025-07-14 11:57:02 +01:00
HoneyryderChuck
93e5efa32e http2 stream header logs: initial newline to align values and make debug logs clearer 2025-07-14 11:50:22 +01:00
HoneyryderChuck
8b3c1da507 removed ivar left behind and used nowhere 2025-07-14 11:50:22 +01:00
HoneyryderChuck
d64f247e11 fix for Connection too many object shapes
some more ivars which were not initialized in the first place were leading to the warning in CI mode
2025-07-14 11:50:22 +01:00
HoneyryderChuck
f64c3ab599 options: initialize all possible options to improve object shape
Options#merge works by duping-then-filling ivars, but due to not all of them being initialized on object creation, each merge had the potential of adding more object shapes for the same class, which breaks one of the most recent ruby optimizations

this was fixed by caching all possible options names at the class level, and using that as reference in the initalize method to nilify all unreferenced options
2025-07-14 11:50:22 +01:00
HoneyryderChuck
af03ddba3b options: inlining logic from do_initialize in constructor 2025-07-14 09:10:52 +01:00
HoneyryderChuck
7012ca1f27 fixed previous commit, as the tag is not available before 1.15 2025-07-03 16:39:54 +01:00
HoneyryderChuck
d405f8905f fixed ddtrace compatibility for versions under 1.13.0 2025-07-03 16:23:27 +01:00