97 Commits

Author SHA1 Message Date
HoneyryderChuck
82b0a4bf28 fix: https resolver should close when no more outstanding connections are around 2022-01-16 22:54:56 +02:00
HoneyryderChuck
06b162b6ea applying a resolver manager to hold the different family type resolvers for the pool. This allows to have multiple resolvers per type, i.e. IPv6 and IPv4 2022-01-16 22:54:56 +02:00
HoneyryderChuck
71920157f4 fix: ensuring that the https resolver is using the pool it's being created in 2022-01-16 22:54:56 +02:00
HoneyryderChuck
6d33b5e59f adding support for tempfile sigs, other improvements 2022-01-16 22:54:56 +02:00
HoneyryderChuck
9bd73e5a22 removing uncaching of resolved names (not used anywhere) 2022-01-16 22:54:54 +02:00
HoneyryderChuck
921d1f6371 removing resolver_ios cache from pool (did not have accurate values, not really useful since selector avoid doubler registries already) 2022-01-16 22:53:36 +02:00
HoneyryderChuck
a8830681df changing resolver structure to rely on inheritance, which helps with typing 2022-01-16 22:53:36 +02:00
HoneyryderChuck
15a4fb83ba options: also freeze inner unfrozen vars; fixed message of resolve
timeout to include host
2022-01-15 01:38:14 +02:00
HoneyryderChuck
7da23ac89c enable connection coalescing for proxied connections 2022-01-08 15:09:11 +02:00
HoneyryderChuck
d3b36c5668 added support for HTTP/2 proxy by simplifying the overall http proxy implementation 2022-01-07 12:26:26 +02:00
HoneyryderChuck
5a61586fd5 do not ignore goaway with no errors sent by the server, let the connection go down
raise a specific error for this, which will make it easier to rescue
from.
2021-11-18 18:07:12 +00:00
HoneyryderChuck
b18b715818 some small rbs improvemments 2021-11-13 11:09:29 +00:00
HoneyryderChuck
a3ee98f410 implementation of the response cache plugin 2021-10-01 23:53:21 +01:00
HoneyryderChuck
bdcd4b31b5 aws_sdk plugin: removing S3 plugin APIs, replacing it with barebones aws-sdk-core components 2021-10-01 23:20:19 +01:00
HoneyryderChuck
efddd72caa removing persistent connections from the selector whe inactive
keeping them around was resulting in some busy loops on timer events
(i.e. retry after), making them unreliable, innacurate  and CPU
draining. they're now kept out whenever they're inactive.
2021-09-27 16:18:02 +01:00
HoneyryderChuck
52948e0f83 bugfix: prevent stream close callback from being called 2 times
an issue was observed when stream was closed from our side, that the
the request in-flight count on the connection. This was fixed by not
reacting to :stream_closed events if request has been previously deleted.
2021-09-23 12:12:48 +01:00
HoneyryderChuck
81a41d889c bugfix: remove connections from selector which have been unregistered
during interest calculation

A quirk was found whereby a connection which failed while connecting
(such as the badssl test) was properly unregistered from the pool, was
however kept in the selectables selector pool, because if this operation
happening during the interest calculation pool, and the var substitution
being performed right afterwards, leaving the pool and selector out of
sync and causing all sorts of miscalculations around timers later on.
2021-09-22 12:53:33 +01:00
HoneyryderChuck
13e865e488 hiding monotonic time funcs under the utils API 2021-09-20 16:16:20 +01:00
HoneyryderChuck
27d81f3090 introduce custom timer to replace Timers::Group
The HTTPX::Timers class mimicks the same top-level API as its
predecessors, but simplifies its implementation. Adding a timer will
resort all timers, while lookups are roughly the same complexity. The
key difference is that callbacks are now aggregated by interval, i.e.
different requests setting the same timeout, will reuse the same timer.
This is a more simple design than Timers::Group, which stores timers in
a binary search tree; the latter will perform well in any environment,
whereas the first one is more tailored for the use-case of httpx, where
most of the times no timers will be set, and when they do, the same
timer will be reused for all requests because they usually have the same
set of options (and therefore timeouts).
2021-09-20 13:19:55 +01:00
HoneyryderChuck
f768cf7a0e Improving API compatibility and error checking in responses
* `Response#error`, which, coupled with `ErrorResponse#error`, allows
  for `if response.error` kind of conditional;
* `Response#raise_for_status` now returns the response when no error is
  raise (for method chaining);

Closes #153
2021-09-20 13:02:20 +01:00
HoneyryderChuck
6e6c7848cc making jitter a retry plugin option 2021-09-20 12:37:48 +01:00
HoneyryderChuck
cdcbf14675 few sig changes (more assertive) 2021-08-31 13:50:29 +01:00
HoneyryderChuck
2a5c429dbd improvement in Session#request, one of the main APIs 2021-08-10 11:10:45 +01:00
HoneyryderChuck
8ded86cec6 improving sigs for new components 2021-08-10 11:10:45 +01:00
HoneyryderChuck
e1ee8c69dc proxy: fixing proxy resolve error filtering to also work with system resolver 2021-08-10 11:10:45 +01:00
HoneyryderChuck
6b61b8ccdb fixing signatures
also adding some checks on code, in order for steep to stop complaining
about potential nil returns.
2021-08-10 10:28:58 +01:00
HoneyryderChuck
f2d3c1f09b added Response#form (supports only x-www-urlencoded for now) 2021-08-09 15:54:25 +01:00
HoneyryderChuck
a85828d0d5 added Response#json 2021-08-09 15:54:24 +01:00
HoneyryderChuck
29105854e3 do not pass around options after requests get initialized, instead rely on rquest options 2021-08-05 15:03:49 +01:00
HoneyryderChuck
37f23ad8c3 cookie: force name to string; as for the value, only late-force it on writing the cookie 2021-07-16 09:52:21 +01:00
HoneyryderChuck
3ae18120d2 when users pass a Cookie header as option and plugin is enabled, the header will be parsed and managed by the jar 2021-07-16 09:52:21 +01:00
HoneyryderChuck
174f5f9647 allow cookie jar to merge 2021-07-15 15:53:15 +01:00
HoneyryderChuck
fbe8a16b1c small sig fixes 2021-07-13 20:04:38 +01:00
HoneyryderChuck
cab5a94d31 removing setters from Options, which weren't public API anyway 2021-07-13 19:56:59 +01:00
HoneyryderChuck
5604501d01 using the new OptionsMethods way for defining internal plugin options 2021-07-13 19:56:59 +01:00
HoneyryderChuck
b6ee019014 several signature improvements while testing with steep 2021-07-07 19:21:36 +01:00
HoneyryderChuck
f29b4a32d8 type selectables in selector 2021-07-04 23:42:35 +01:00
HoneyryderChuck
4df2363cc5 providing strategy for punycode translation, using idnx when possible, falling back otherwise 2021-06-11 18:27:40 +01:00
HoneyryderChuck
5cfb494d50 allow parsers to define a connection timeout, and channge it via callback 2021-05-30 00:57:54 +01:00
HoneyryderChuck
b569cc9984 adding the settings timeout as an option, as well as a specific error 2021-05-30 00:53:37 +01:00
HoneyryderChuck
fb3b795f29 adding missing grpc sigs (+ other missing stuff in plugin calls) 2021-05-28 12:15:09 +01:00
HoneyryderChuck
5b9793752c fixing other plugin signatures 2021-05-27 19:11:15 +01:00
HoneyryderChuck
df489676ac adding ntlm auth plugin 2021-05-27 19:11:15 +01:00
HoneyryderChuck
dda1315db4 adding tests to enure that field order is respected, even when repeated, when performing form/multipart requests (Closes #126) 2021-05-27 19:11:15 +01:00
HoneyryderChuck
c61007ba0f do not forget the filename in those multipart spoofs 2021-05-27 17:53:45 +01:00
HoneyryderChuck
8e1e6b3c6a set_protocol_headers now returns extra fields, that can be yielded in the handler, instead of merged into the request
this implicit mutation of the request was causing issue in insecure
redirects, which kept the same HTTP/2 specific headers in HTTP/1
payloads
2021-05-26 18:13:53 +01:00
HoneyryderChuck
e5a8e76d62 allow Headers#each to be passed extra headers to yield 2021-05-26 18:13:53 +01:00
HoneyryderChuck
2c3280b6d0 Merge branch 'issue-119' into 'master'
new option: origin

See merge request honeyryderchuck/httpx!131
2021-05-22 14:23:05 +00:00
HoneyryderChuck
44106325d5 bugfix: do not set new session callbacks on connections which already have them
a subtle bug surfaced when trying multiple individual requests on the
same persistent session, where the connection was being removed from the
watchable connections after each request, but kept in the pool; on the
next request, it would set the same session callbacks; this would go on
and on until connections would get exhausted, after which all of these
callbacks would have to be called.

fixed by having a new callbacks interface, #only, which discards
existing callbacks by type, thereby ensuring there's only one of the
kind.
2021-05-21 18:26:54 +01:00
HoneyryderChuck
441716a5ac fixing double-response for streams
there was a long-standing buggy workaround, whereas in stream-mode, when
there was no response yet to query from, a synchronous request would be
fired. This would break when under event streams, so we had to document
this as "make sure that...".

This fixes it by implementing a general session API convention, which
separates the step of sending the requests, from waiting for its
receival. And, given that the request knows when the response is
available, we can actually "tick until response".

This might be used in the future to refactor the way we handle the
responses, which buffer the full payload by default, instead of reading
from the connection at will.
2021-05-20 00:11:26 +01:00