1147 Commits

Author SHA1 Message Date
HoneyryderChuck
7ee4c5f6d3 teaching errors how to dump 2022-08-06 22:30:46 +01:00
HoneyryderChuck
452657c805 Added the read_timeout and write_timeout timeouts
These are deadline oriented for the request and response, i.e. a write
timeout tracks the full time it takes to write the request, whereas the
read timeout does the same for receiving the response.

For back-compat, they're infinite by default. v1 may change that, and
will have to provide a safe fallback for endless "stream" requests and
responses.
2022-08-06 22:30:46 +01:00
HoneyryderChuck
f4e393af40 Response#finished? and Response.finish!
these new functions allow to mark an incomplete response as closed, such
as when a timeout happens. finishing also freezes the response headers.
2022-08-06 22:30:46 +01:00
HoneyryderChuck
25b949cf66 Added support for multiple JSON parsers
When available, httpx will either use `multi_json`, `oj`, or `yajl`,
before it falls back to default `json`.
2022-08-05 22:57:06 +01:00
HoneyryderChuck
12573a16a5 removed support for application/dns-json mime type in the DoH resolver 2022-08-05 22:57:06 +01:00
HoneyryderChuck
6336379837 added support for other json parsers 2022-08-05 22:57:06 +01:00
HoneyryderChuck
43016795f3 introducing the :no_proxy option
can be passed in the `:proxy` option hash, and receives domains, as
strings, which requests should not go through the proxy.
2022-08-05 22:37:52 +01:00
HoneyryderChuck
da0ef24b09 bumped version to 0.24.4 2022-08-01 22:46:41 +01:00
HoneyryderChuck
8359d6b007 Merge branch 'issue-200' into 'master'
response_cache: fixes and improvements

Closes #200

See merge request honeyryderchuck/httpx!216
2022-08-01 17:50:21 +00:00
HoneyryderChuck
61c36c4ef9 response cache: caching several instances for the same URL
by relying on vary header, this should have the effect of not
overflowinng, and doing what the user wants.
2022-08-01 18:40:21 +01:00
HoneyryderChuck
f48f4e414a Fixes and improvements on the response_cache plugin
The following improvements were done:

* only cacheable status codes are allowed now (200, 203, 300, 301, 410)
* only responses considered fresh are cached; fresh response means:
  * no-store directive not present in cache-control
  * response hasn’t expired (when s-maxage, max-age or expires are
      present)
2022-08-01 17:40:05 +01:00
HoneyryderChuck
b0777c61e5 fix for loop on resolution and retry on new connection
A certain behaviour was observed, when performing some tests using the
hackernews script, where after a failed request on a non-initiated
connection, a new DNS resolution would be emitted, although the
connection still had other IPs to try on. This led to a cascading
behaviour where the DNS response would fill up the connection with the
same repeated IPs and trigger coalescing, which would loop indefinitely
after emitting the resolve event.

This was fixed by not allowing DNS resolution on already resolved names,
to propagate to connections which already contain the advertised IPs.

This seems to address the github issue 5, which description matches the
observed behaviour.
2022-07-31 19:07:41 +01:00
HoneyryderChuck
32a81f2025 fix: response cache now also takes verb into account when caching
The previous strategy was working only with URLs. This strategy would
fall flat if the same url could be used with several HTTP verbs.
2022-07-31 17:03:30 +01:00
HoneyryderChuck
534b3eb91b bumped version to 0.20.3 2022-07-03 23:54:45 +01:00
HoneyryderChuck
da9993f7e0 another rubocop, anoter linting hel... 2022-07-03 23:42:45 +01:00
HoneyryderChuck
2935724a16 makinng sure the nameserver list is a list 2022-07-03 00:28:15 +01:00
HoneyryderChuck
1e2cf12f6e Doh: initialize the dns resolver, so name candidate calculation actually works for non-absolute domains 2022-07-03 00:20:28 +01:00
HoneyryderChuck
c5b3d849ce bumped version to 0.20.2 2022-06-23 08:40:40 +03:00
HoneyryderChuck
6c911768fe fixing selector timeout errors closing all connections and ignoring
resolvers

All kinds of errors happening during the select loop, will be handled as
abrupt select loop errors, and terminate all connections; this also
includes timmeout errors. This is not ideal, for some reasons:
connection timeout errors happening on the loop close all connections,
although it may be only triggered for one (or a subset of) connection
for which the timeout should trigger; second, errors on the DS channel
propagate errors to connections indirectly (the emission mentioned
above), wrongly (connections for different hostnames not yet queried,
will also fail with timeout), and won't clean the resolver state (so
subsequent queries will be done for the same hostname which failed in
the first place).

This fix is a first step to solving this problem. It does not totally
address the first, but i'll fix dealing with errors from the second
use-case.
2022-06-22 02:09:26 +03:00
HoneyryderChuck
d8e5894979 bumped version to 0.20.1 2022-06-21 16:31:52 +03:00
HoneyryderChuck
ecab7951c9 bugfix for unregistering connections timing out while resolving
A subtle bug slipped through the cracks, where if a resolve timeout
error happened, the connection would remain in the pool. Subsequent
requests to the same domain would activate it, although no requests
would go through it; the actual desired behaviour is to outright remove
it from the pool on such errors.

This was achieved by registering the "unregister_connection" callback
earlier. However, the connection accounting step would only trigger if
taking it out of the selector worked, meaning it had been registered
before.
2022-06-21 02:20:32 +03:00
HoneyryderChuck
b4a20016f2 bumped version to 0.20.0 2022-05-31 11:36:32 +01:00
HoneyryderChuck
7383347147 implementation of the sentry plugin integration 2022-05-27 16:02:52 +01:00
HoneyryderChuck
281ac03cda added tests and scaffold 2022-05-26 23:45:20 +01:00
HoneyryderChuck
4a0bfa3544 Merge branch 'issue-174' into 'master'
altsvc improvements

Closes #174

See merge request honeyryderchuck/httpx!207
2022-05-25 22:28:18 +00:00
HoneyryderChuck
38152e8dee fix vary header handling to deal with capitalized, test the path
directly in store
2022-05-25 23:15:30 +01:00
HoneyryderChuck
d39e7ba8d4 handle 305 redirects via proxy if the plugin is enabled 2022-05-25 18:58:11 +01:00
HoneyryderChuck
e740a38bad follow_redirects: halt redirection if no location header is present in response 2022-05-25 18:58:11 +01:00
HoneyryderChuck
f14016774d altsvc: ensuring origin hostname is used during TLS handshake to alternative services
this means that finding a compatible connection got a bit more
expensive, due to options equality needing to account for extra sni
hostname.
2022-05-23 23:35:01 +01:00
HoneyryderChuck
675a2aa547 do not allow downgrading from https to http during altsvc handshake 2022-05-23 00:17:49 +01:00
HoneyryderChuck
889c2f3f79 Merge branch 'issue-189' into 'master'
Support other auth schemes for proxies (digest, ntlm...)

Closes #189

See merge request honeyryderchuck/httpx!204
2022-05-16 22:15:29 +00:00
HoneyryderChuck
71bb6dc2ce Merge branch 'issue-179' into 'master'
Disable SNI when the URL host is an IP

Closes #179

See merge request honeyryderchuck/httpx!203
2022-05-16 22:15:21 +00:00
HoneyryderChuck
0d01f728aa introducing base_path option
This should complement the `:origin` option, in order to provide good
defaults to build REST SDKs around of.

Ex:

```ruby
HTTPX.with(origin: "https://api.this-product.com", base_path: "/v3.1")
```
2022-05-08 17:43:26 +01:00
HoneyryderChuck
c86f4be1a7 reworking auth APIs for a future 1.0 refactoring 2022-05-08 17:23:07 +01:00
HoneyryderChuck
be06032649 adding APIs for running other proxy auth schemes, and adapting internals to work with it 2022-05-07 18:20:07 +01:00
HoneyryderChuck
a316b31e7d fix: SNI is now disabled when the URL host is an IP 2022-05-07 18:17:47 +01:00
HoneyryderChuck
817a10a537 scoping http auth schemes out of its plugins, made them usable in proxy
In order to expose other auth schemes in proxy, the basic, digest and
ntlm modules were extracted from the plugins, these being left with the
request management. So now, an extra parameter, `:scheme`, can be
passed (it'll be "basic" for http and "socks5" for socks5 by default,
can also be "digest" or "ntlm", haven't tested those yet).
2022-05-07 13:57:10 +01:00
HoneyryderChuck
c23cc432e9 bumped version to 0.19.8 2022-05-06 17:24:12 +01:00
HoneyryderChuck
42bcfd3a93 adapting datadog plugin to datadog 1.0 release 2022-05-06 13:15:07 +01:00
HoneyryderChuck
7e54db8805 bumped version to 0.19.7 2022-04-08 09:39:30 +01:00
HoneyryderChuck
637d2f7600 fix: proxy credentials were not being used
a regression from the 0.19.x series was omitting the proxy credentials
when passed directly as proxy options (instead of being part of the URI).

Closes #188
2022-04-05 22:53:45 +01:00
HoneyryderChuck
27268ee228 bumped version to 0.19.6 2022-04-04 18:22:25 +01:00
HoneyryderChuck
1cf8c68ac6 fix: do not subclass session class to make it play along with default
class overrides

the faraday adapter was relying on subclassing the session to load
custom plugins. this doesn't play well with other integrations
monkeypatching the original session class, such as datadog or webmock,
which redefine it. from now on, we not only defer the creation of the
custom until necessary, we also use Session#plugin

Fixes #187
2022-04-04 12:54:33 +01:00
HoneyryderChuck
603fc0979a bumped version to 0.19.5 2022-03-30 20:29:44 +01:00
HoneyryderChuck
64f8ebcf51 exposing original hostname in errors when there's a dns error in a candidate name for resolution 2022-03-28 12:51:35 +01:00
HoneyryderChuck
0490175835 removing match overrides (irrelevant) 2022-03-25 06:35:15 +00:00
HoneyryderChuck
2d9d837543 fix: aligning mapping of requests to existing connections
previously, a connection could be created based on request options. The
main problem is that requests may have special headers, which would make
them assign its own connection due to the headers mismatch in headers to
the already open connection to the same origin. This, in certain
scenarios, coupled with the persistent plugin, cascades into multiple
connections to the same host which are never closed.

This fix ensures that connection initial options comes from the session.
This way, it'll never change (as connections exhaust). Alongside that,
matching headers was relaxed to only take into account headers which the
original connection knows, special request headers will then opt out
from this.
2022-03-25 06:35:15 +00:00
HoneyryderChuck
6ad59453ce handling the case where there might not be any connections running in the system resolver 2022-03-23 19:03:30 +00:00
HoneyryderChuck
30df921387 limit candidates resolution to the first resolv call, while also adapting the retries logic in the native resolver 2022-03-20 15:44:12 +00:00
HoneyryderChuck
61c4df243a https resolver: support resolv.conf search and ndots params 2022-03-20 02:29:58 +00:00