2863 Commits

Author SHA1 Message Date
HoneyryderChuck
b5775d73c8 adding TCP#addresses?, which drops on-the-fly expired entries, and therefore makes possible to DNS query for persistent connections 2025-09-02 10:58:06 +01:00
HoneyryderChuck
cfce76bd8b adding Resolver::Entry, using it everywhere internally
this acts as a delegator to ipaddr, and should be a drop-in replacement everywhere the ipaddress was used, with the extra information about expiration
2025-09-02 10:58:06 +01:00
HoneyryderChuck
de153bc995 decoupled form and multipart transcoder, moved check to Request::Body 2025-09-01 23:33:56 +01:00
HoneyryderChuck
6f71bd4a5b Merge branch 'fiber-concurrency-plugin' into 'master'
fiber scheduler concurrency plugin

See merge request os85/httpx!400
2025-09-01 14:20:04 +00:00
HoneyryderChuck
34a1f7ca44 Session#deactivate: fix for when connections get deregistered from selector during the loop
as the selectables array may shrink during the loop as a result
2025-09-01 15:07:13 +01:00
HoneyryderChuck
f16e7e7e97 changed the plugin name to fiber_concurrency
same as in sequel
2025-09-01 13:12:20 +01:00
HoneyryderChuck
53a3ba7037 moved logic related with supporting the fiber scheduler into its own plugin
this plugin is loaded into the persistent plugin by default, as it's already expected to work across fibers
2025-09-01 11:43:30 +01:00
HoneyryderChuck
7f34a62b82 promoting native/system/https resolver classes into options
so that those can be extended in plugins
2025-09-01 10:21:23 +01:00
HoneyryderChuck
f9ae15379a promoting http1 and http2 connection classes into options
this makes them extendable like other plugin classes, and the h2c plugin is rewritten to accomodate that requirement
2025-09-01 10:21:23 +01:00
HoneyryderChuck
d19f5150a1 move fiber selector logic from persistent plugin into its own sub-plugin 2025-09-01 10:21:23 +01:00
HoneyryderChuck
92e891919f Merge branch 'issue-355' into 'master'
plugin init fixes: enforce usage of Options or hash for the second argument;...

Closes #355

See merge request os85/httpx!403
2025-09-01 09:18:14 +00:00
HoneyryderChuck
1c880127d7 Merge branch 'gh-98' into 'master'
pool fix: when connection is acquired after waiting on it, return it...

See merge request os85/httpx!402
2025-09-01 09:16:07 +00:00
HoneyryderChuck
f0329cf653 plugin init fixes: enforce usage of Options or hash for the second argument; ensure that plugin only continues loading if it is a module or converts to one 2025-08-24 01:26:24 +01:00
HoneyryderChuck
fb8ebd57d4 Merge branch 'issue-354' into 'master'
callbacks plugin: propagated callbacks to derived plugins

Closes #354

See merge request os85/httpx!401
2025-08-23 23:42:37 +00:00
HoneyryderChuck
09f1309dcf pool fix: when connection is acquired after waiting on it, return it immediately, instead of bookkeeping on max connections
the previous state was acquiring the connection AND reacting on the number of available connections
2025-08-22 16:57:37 +01:00
HoneyryderChuck
34237705ab callbacks plugin: propagated callbacks to derived plugins
same fix as done recently with #with, but different function
2025-08-22 16:25:21 +01:00
HoneyryderChuck
0643c696ab fixing local proxy list urls 2025-08-22 16:11:44 +01:00
HoneyryderChuck
35afcc51f6 abstract IO#wait call to workaround the jruby bug
https://github.com/jruby/jruby/issues/8917
2025-08-19 23:55:48 +01:00
HoneyryderChuck
bb6907e4ee Merge branch 'gh-94' into 'master'
Fixes and improvements for fiber scheduler integration

See merge request os85/httpx!395
2025-08-19 17:34:09 +00:00
HoneyryderChuck
14734d510e align nil inits 2025-08-19 18:14:55 +01:00
HoneyryderChuck
09342187f5 removing fiber reference when emitting request completion event 2025-08-19 18:14:55 +01:00
HoneyryderChuck
8970767954 http1: also take into account buffered requests when looking for current context
the previous patch was only considering the current inflight request and the pending requests, but in a scenario where multiple requests may have been buffered and considered in theory in-flight, this information would pass by, leading to a busy loop
2025-08-19 18:14:55 +01:00
HoneyryderChuck
6782928b99 do not allow deactivating connections which are still waiting for the ack of the PING frame
this causes a very subtle bug where, in a multifiber scenario, when the last active stream is closed and the connection is checked-in, only to be checked out later and pinged for liveness, it may get caught in the termination flow of a session, which will try to deactivateconnections from the selector; given that only inflight requests were taken into account, it'd get deactivated and checked in, which would mess with the fiber which was actively using it

this fixes it by considering the ping state in the decision on whether the connection should be deactivated; when having an inflight ping, keep it open and in the selector
2025-08-19 18:14:55 +01:00
HoneyryderChuck
2c9093f20b http1: check interests also in the case where one of the pending requests is from the current context 2025-08-19 18:14:55 +01:00
HoneyryderChuck
f003340e60 selector: skip early when io is closed
it was needlessly logging io information despite it not being taken into account for selection
2025-08-19 18:14:55 +01:00
HoneyryderChuck
8d811b133b adapted hackernews script to also test async 2025-08-19 18:14:55 +01:00
HoneyryderChuck
3e06cff2a1 adapt the hackernews pages script to be also ran in async 2025-08-19 18:14:55 +01:00
HoneyryderChuck
3cfda42878 workaround for IO#wait bug when passed a nil timeout as first argument
https://bugs.ruby-lang.org/issues/21509
2025-08-19 18:14:55 +01:00
HoneyryderChuck
aa03a3a52e set fiber context for sent request
this is used downstream to inform the selector (via IO interest calculation) whether the connection should wait on events for the current fiber (i.e. the connection should be used for requests of the current fiber, or DNS requests for the connection of such requests must be pending)
2025-08-19 18:14:55 +01:00
HoneyryderChuck
37cc138bef improve interest calculation
do not check buffer before going to the parsers, which also take that account

for http2, use the http2 connection send buffer (instead of the connection buffer) to figure out whether it's waiting on a WINDOW_UPDATE, which is more correct and efficient

on http1, the logic is the same, but the code is simplified, as the checks become the same regardless of whether request or @requests.first is picked up
2025-08-19 18:14:55 +01:00
HoneyryderChuck
ded1b90ae3 added test to reproduce the issue with the skipped query under the fiber scheduler 2025-08-19 18:14:55 +01:00
HoneyryderChuck
35c15f7fa7 selector: improve prefiltering of selectable ios
the decision to use IO.select was based on the number of selectables, however only on select_many the actual selectables would be prefiltered before being passed to it, in some cases meaning that only one descriptor would be passed to IO.select (instead of IO#wait_readable or the like)

in fiber-scheduler frameworks, this meant going a potentially more inefficient route needlessly, such as in async, where IO.wait_readable and the like are performed using epoll or kevent, whereas IO.select calls are executed in a background thread
2025-08-19 18:14:54 +01:00
HoneyryderChuck
3852e2cdf4 native dns: do not write query to buffer if waiting on a previous reply
in a normal situation this would not happen, as events on a resolver would be dealt one at a time, but in a fiber-scheduler environment like async, the first query could be buffered in the 1st fiber switch, the second could then be enqueued on 2nd, then flush buffer and fiber switch on read, and a third query would then enter a corrupt state where, due to the buffer having been flushed on the 2nd fiber, write the 3rd query before receiving the second one, and messing up the pending query bookkeeping, making the fiber then block (due to DNS query never returning back) and expose that as an empty interest registration from some other connection on the selector

this is fixed by using @timer as a proxy for knowing that a given DNS query has been flushed but is still waiting on a response (or an eventual retry)
2025-08-19 18:13:56 +01:00
HoneyryderChuck
0261449b39 fixed sig for callbacks_for 2025-08-08 17:06:03 +01:00
HoneyryderChuck
84c8126cd9 callback_for: check for ivar existence first
Closes #353
2025-08-08 16:30:17 +01:00
HoneyryderChuck
ff3f1f726f fix warning about argument potentially being ignored 2025-08-07 12:34:59 +01:00
HoneyryderChuck
b8b710470c fix sentry deprecation 2025-08-07 12:30:31 +01:00
HoneyryderChuck
0f3e3ab068 remove trailing :: from IO module usage, as there's no more internal module 2025-08-07 12:30:21 +01:00
HoneyryderChuck
095fbb3463 using local aws for the max requests tests
reduce exposure to httpbin.org even more
2025-08-07 12:12:50 +01:00
HoneyryderChuck
7790589c1f linting issue 2025-08-07 11:28:18 +01:00
HoneyryderChuck
dd8608ec3b small improv in max requests tests to make it tolerant to multi-homed networks 2025-08-07 11:22:29 +01:00
HoneyryderChuck
8205b351aa removing usage of httpbin.org peer in tests wherever possible
it has been quite unstable, 503'ing often
2025-08-07 11:21:59 +01:00
HoneyryderChuck
5992628926 update nghttp2 used in CI tests 2025-08-07 11:21:02 +01:00
HoneyryderChuck
39370b5883 Merge branch 'issue-337' into 'master'
fix for issues blocking reconnection in proxy mode

Closes #337

See merge request os85/httpx!397
2025-07-30 09:49:51 +00:00
HoneyryderChuck
1801a7815c http2 parser: fix calculation when connection closes and there's no termination handshake 2025-07-18 17:48:23 +01:00
HoneyryderChuck
0953e4f91a fix for #receive_requests bailout routing when out of selectables
the routine was using #fetch_response, which may return nil, and wasn't handling it, making it potentially return a nil instead of a response/errorresponse object. since, depending on the plugins, #fetch_response may reroute requests, one allows to keep in the loop in case there are selectables again to process as a result of it
2025-07-18 17:48:23 +01:00
HoneyryderChuck
a78a3f0b7c proxy fixes: allow proxy connection errors to be retriable
when coupled with the retries plugin, the exception is raised inside send_request, which breaks the integration; in order to protect from it, the proxy plugin will protect from proxy connection errors (socket/timeout errors happening until tunnel established) and allow them to be retried, while ignoring other proxy errors; meanwhile, the naming of errors was simplified, and now there's an HTTPX::ProxyError replacing HTTPX::HTTPProxyError (which is a breaking change).
2025-07-18 17:48:23 +01:00
HoneyryderChuck
aeb8fe5382 fix proxy ssl reconnection
when a proxied ssl connection would be lost, standard reconnection wouldn't work, as it would not pick the information from the internal tcp socket. in order to fix this, the connection retrieves the proxied io on reset/purge, which makes the establish a new proxyssl connection on reconnect
2025-07-18 17:48:23 +01:00
HoneyryderChuck
03170b6c89 promote certain transition logs to regular code (under level 3)
not really useful as telemetry metered, but would have been useful for other bugs
2025-07-18 17:48:23 +01:00
HoneyryderChuck
814d607a45 Revert "options: initialize all possible options to improve object shape"
This reverts commit f64c3ab5990b68f850d0d190535a45162929f0af.
2025-07-18 17:47:08 +01:00