resolvers
All kinds of errors happening during the select loop, will be handled as
abrupt select loop errors, and terminate all connections; this also
includes timmeout errors. This is not ideal, for some reasons:
connection timeout errors happening on the loop close all connections,
although it may be only triggered for one (or a subset of) connection
for which the timeout should trigger; second, errors on the DS channel
propagate errors to connections indirectly (the emission mentioned
above), wrongly (connections for different hostnames not yet queried,
will also fail with timeout), and won't clean the resolver state (so
subsequent queries will be done for the same hostname which failed in
the first place).
This fix is a first step to solving this problem. It does not totally
address the first, but i'll fix dealing with errors from the second
use-case.
to wait for.
Timeout calculation may trigger errors which lead to connection
unregistering, such as tital timeout errors. In such a case, we can end
up in a state where #select gets called with no timeout and no
selectable connections.
https://github.com/HoneyryderChuck/httpx/issues/3
keeping them around was resulting in some busy loops on timer events
(i.e. retry after), making them unreliable, innacurate and CPU
draining. they're now kept out whenever they're inactive.
during interest calculation
A quirk was found whereby a connection which failed while connecting
(such as the badssl test) was properly unregistered from the pool, was
however kept in the selectables selector pool, because if this operation
happening during the interest calculation pool, and the var substitution
being performed right afterwards, leaving the pool and selector out of
sync and causing all sorts of miscalculations around timers later on.
a refactoring was performed on the connection, so they could connect out
of the #to_io call; in order to proper read the interests, it was
thought that it was too late at that point, as interests before #to_io
would be different from post #to_io; this makes our selector less
"portable", as we rely on internal logic being called now
Using IO.select is costly for the one-request/one-origin case, as we
have to build array to pass, and we receive arrays, and that generates a
lot of needless garbage when we only request once. Instead, it now uses
IO#wait_readable and IO#wait_writable, which does not require extra
arrays, and uses poll under the hood.
by analyzing all of the data down to the parser, one can estimate better
whether io wants to read/write/both, thereby avoiding spurious wakeups.
This also greatly simplifies the monitor API, and solves the 100% CPU
utilization issue.
Some requests were hanging because some connection with read interests
were only on the writers selector. This issue manifested itself in high
load scenarios.
The fix is not the most performant, but it does the job: only set write
interest when connecting, otherwise read/write. This increases the
number of wakeups, but at least we have correctness.
header;
this was wrongly closing connections for HTTP/1.1 connections which
didn't send a "Connection" header; according to spec, the default is
"Keep-Alive", contrary to HTTP/1.0
