Compare commits

...

11 Commits

Author SHA1 Message Date
HoneyryderChuck
9582e17370 bumped version to 0.23.0 2023-04-29 01:18:57 +01:00
HoneyryderChuck
6937f3fbe4 fix ci job name (pages, to deploy pages) 2023-04-29 00:53:42 +01:00
HoneyryderChuck
cfac38dc62 added more typing, improved correctness of a few checks 2023-04-28 23:57:25 +01:00
HoneyryderChuck
bd233c5303 effort to increase coverage of tests 2023-04-28 23:57:25 +01:00
HoneyryderChuck
56743923f6 enable publishing of coverage 2023-04-28 15:45:18 +01:00
HoneyryderChuck
75867115b2 removing webmock cap, disabling datadog telemetry instead 2023-04-28 13:02:03 +01:00
HoneyryderChuck
4eee045b02 downgrading webmock even more 2023-04-28 12:04:06 +01:00
HoneyryderChuck
5f079f8fc0 downgrading to webmock 3.18.0
Identifying an issue when running datadog and webmock in tandem, and
running webmock 3.18.1.
2023-04-28 11:23:43 +01:00
HoneyryderChuck
ce6c1d2ce5 Merge branch 'issue-217' into 'master'
Removing HTTPX::Registry and its usage internally

Closes #217

See merge request os85/httpx!249
2023-04-28 09:16:45 +00:00
HoneyryderChuck
899b2df94f only test integrations with latest ruby, some telemetry stuff firing from the datadog sdk... 2023-04-27 23:47:55 +01:00
HoneyryderChuck
bbf257477b Removing HTTPX::Registry and its usage internally
These internnal registries were a bit magical to use, difficult to
debug, not thread-safe, and overall a nuisance when it came to type
checking. So long.
2023-04-27 22:49:20 +01:00
62 changed files with 441 additions and 355 deletions

View File

@ -131,7 +131,7 @@ coverage:
paths: paths:
- "coverage/" - "coverage/"
docs: pages:
stage: deploy stage: deploy
needs: needs:
- coverage - coverage
@ -140,12 +140,15 @@ docs:
- gem install hanna-nouveau - gem install hanna-nouveau
script: script:
- rake prepare_website - rake prepare_website
- mkdir -p public/
- cp -r coverage/ public/
artifacts: artifacts:
paths: paths:
- rdoc/ - rdoc/
- wiki/ - wiki/
- data/ - data/
- coverage/ - coverage/
- public/
only: only:
- master - master

View File

@ -20,6 +20,8 @@ group :test do
gem "spy" gem "spy"
if RUBY_VERSION < "2.3.0" if RUBY_VERSION < "2.3.0"
gem "webmock", "< 3.15.0" gem "webmock", "< 3.15.0"
elsif RUBY_VERSION < "2.4.0"
gem "webmock", "< 3.17.0"
else else
gem "webmock" gem "webmock"
end end
@ -28,11 +30,7 @@ group :test do
gem "net-ssh", "~> 4.2.0" if RUBY_VERSION < "2.2.0" gem "net-ssh", "~> 4.2.0" if RUBY_VERSION < "2.2.0"
if RUBY_VERSION >= "2.3.0" gem "ddtrace"
gem "ddtrace"
else
gem "ddtrace", "< 1.0.0"
end
platform :mri do platform :mri do
if RUBY_VERSION >= "2.3.0" if RUBY_VERSION >= "2.3.0"

View File

@ -0,0 +1,42 @@
# 0.23.0
## Features
### `:retries` plugin: resumable requests
The `:retries` plugin will now support scenarios where, if the request being retried supports the `range` header, and a partial response has been already buffered, the retry will resume from there and only download the missing data.
#### HTTPX::ErrorResponse#response
As a result, ´HTTPX::ErrorResponse#response` has also been introduced; error responses may have an actual response. This happens in cases where the request failed **after** a partial response was initiated.
#### `:buffer_size` option
A nnew option, `:buffer_size`, can be used to tweak the buffers used by the read/write socket routines (16k by default, you can lower it in memory-constrained environments).
## Improvements
### `:native` resolver falls back to TCP for truncated messages
The `:native` resolver will repeat DNS queries to a nameserver via TCP when the first attempt is marked as truncated. This behaviour is both aligned with `getaddrinfo` and the `resolv` standard library.
This introduces a new `resolver_options` option, `:socket_type`, which can now be `:tcp` if it is to remain the default.
## Chore
### HTTPX.build_request should receive upcased string (i.e. "GET")
Functions which receive an HTTP verb should be given he verb in "upcased string" format now. The usage of symbols is still possible, but a deprecation warning will be emitted, and support will be removed in v1.0.0 .
### Remove HTTPX::Registry
These internal registries were a bit magical to use, difficult to debug, not thread-safe, and overall a nuisance when it came to type checking. While there is the possibility that someone was relying on it existing, nothing had ever been publicly documented.
## Bugfixes
* fixed proxy discovery using proxy env vars (`HTTPS_PROXY`, `NO_PROXY`...) being enabled/disabled based on first host uused in the session;
* fixed `:no_proxy` option usage inn the `:proxy` plugin.
* fixed `webmock` adapter to correctly disable it when `Webmock.disable!` is called.
* fixed bug in `:digest_authentication` plugin when enabled and no credentials were passed.
* fixed several bugs in the `sentry` adapter around breadcrumb handling.
* fixed `:native` resolver candidate calculation by putting absolute domain at the bottom of the list.

View File

@ -25,6 +25,7 @@ services:
- AWS_SECRET_ACCESS_KEY=test - AWS_SECRET_ACCESS_KEY=test
- AMZ_HOST=aws:4566 - AMZ_HOST=aws:4566
- WEBDAV_HOST=webdav - WEBDAV_HOST=webdav
- DD_INSTRUMENTATION_TELEMETRY_ENABLED=false
image: ruby:alpine image: ruby:alpine
privileged: true privileged: true
depends_on: depends_on:

View File

@ -11,7 +11,6 @@ require "httpx/domain_name"
require "httpx/altsvc" require "httpx/altsvc"
require "httpx/callbacks" require "httpx/callbacks"
require "httpx/loggable" require "httpx/loggable"
require "httpx/registry"
require "httpx/transcoder" require "httpx/transcoder"
require "httpx/timers" require "httpx/timers"
require "httpx/pool" require "httpx/pool"

View File

@ -29,7 +29,6 @@ module HTTPX
# #
class Connection class Connection
extend Forwardable extend Forwardable
include Registry
include Loggable include Loggable
include Callbacks include Callbacks
@ -63,7 +62,7 @@ module HTTPX
# if there's an already open IO, get its # if there's an already open IO, get its
# peer address, and force-initiate the parser # peer address, and force-initiate the parser
transition(:already_open) transition(:already_open)
@io = IO.registry(@type).new(@origin, nil, @options) @io = build_socket
parser parser
else else
transition(:idle) transition(:idle)
@ -82,7 +81,7 @@ module HTTPX
if @io if @io
@io.add_addresses(addrs) @io.add_addresses(addrs)
else else
@io = IO.registry(@type).new(@origin, addrs, @options) @io = build_socket(addrs)
end end
end end
@ -102,7 +101,7 @@ module HTTPX
# was the result of coalescing. To prevent blind trust in the case where the # was the result of coalescing. To prevent blind trust in the case where the
# origin came from an ORIGIN frame, we're going to verify the hostname with the # origin came from an ORIGIN frame, we're going to verify the hostname with the
# SSL certificate # SSL certificate
(@origins.size == 1 || @origin == uri.origin || (@io && @io.verify_hostname(uri.host))) (@origins.size == 1 || @origin == uri.origin || (@io.is_a?(SSL) && @io.verify_hostname(uri.host)))
) && @options == options ) && @options == options
) || (match_altsvcs?(uri) && match_altsvc_options?(uri, options)) ) || (match_altsvcs?(uri) && match_altsvc_options?(uri, options))
end end
@ -116,7 +115,7 @@ module HTTPX
( (
(open? && @origin == connection.origin) || (open? && @origin == connection.origin) ||
!(@io.addresses & connection.addresses).empty? !(@io.addresses & (connection.addresses || [])).empty?
) && @options == connection.options ) && @options == connection.options
end end
@ -451,7 +450,7 @@ module HTTPX
end end
def build_parser(protocol = @io.protocol) def build_parser(protocol = @io.protocol)
parser = registry(protocol).new(@write_buffer, @options) parser = self.class.parser_type(protocol).new(@write_buffer, @options)
set_parser_callbacks(parser) set_parser_callbacks(parser)
parser parser
end end
@ -594,6 +593,17 @@ module HTTPX
remove_instance_variable(:@timeout) if defined?(@timeout) remove_instance_variable(:@timeout) if defined?(@timeout)
end end
def build_socket(addrs = nil)
transport_type = case @type
when "tcp" then TCP
when "ssl" then SSL
when "unix" then UNIX
else
raise Error, "unsupported transport (#{@type})"
end
transport_type.new(@origin, addrs, @options)
end
def on_error(error) def on_error(error)
if error.instance_of?(TimeoutError) if error.instance_of?(TimeoutError)
@ -662,5 +672,16 @@ module HTTPX
error = error_type.new(request, request.response, read_timeout) error = error_type.new(request, request.response, read_timeout)
on_error(error) on_error(error)
end end
class << self
def parser_type(protocol)
case protocol
when "h2" then HTTP2
when "http/1.1" then HTTP1
else
raise Error, "unsupported protocol (##{protocol})"
end
end
end
end end
end end

View File

@ -368,5 +368,4 @@ module HTTPX
UPCASED[field] || field.split("-").map(&:capitalize).join("-") UPCASED[field] || field.split("-").map(&:capitalize).join("-")
end end
end end
Connection.register "http/1.1", Connection::HTTP1
end end

View File

@ -412,5 +412,4 @@ module HTTPX
end end
end end
end end
Connection.register "h2", Connection::HTTP2
end end

View File

@ -5,13 +5,3 @@ require "httpx/io/udp"
require "httpx/io/tcp" require "httpx/io/tcp"
require "httpx/io/unix" require "httpx/io/unix"
require "httpx/io/ssl" require "httpx/io/ssl"
module HTTPX
module IO
extend Registry
register "udp", UDP
register "unix", HTTPX::UNIX
register "tcp", TCP
register "ssl", SSL
end
end

View File

@ -201,7 +201,7 @@ module HTTPX
def option_transport(value) def option_transport(value)
transport = value.to_s transport = value.to_s
raise TypeError, "\#{transport} is an unsupported transport type" unless IO.registry.key?(transport) raise TypeError, "#{transport} is an unsupported transport type" unless %w[unix].include?(transport)
transport transport
end end

View File

@ -20,10 +20,7 @@ module HTTPX
end end
def extra_options(options) def extra_options(options)
encodings = Module.new do options.merge(encodings: {})
extend Registry
end
options.merge(encodings: encodings)
end end
end end
@ -36,7 +33,7 @@ module HTTPX
end end
def option_encodings(value) def option_encodings(value)
raise TypeError, ":encodings must be a registry" unless value.respond_to?(:registry) raise TypeError, ":encodings must be an Hash" unless value.is_a?(Hash)
value value
end end
@ -49,7 +46,7 @@ module HTTPX
if @headers.key?("range") if @headers.key?("range")
@headers.delete("accept-encoding") @headers.delete("accept-encoding")
else else
@headers["accept-encoding"] ||= @options.encodings.registry.keys @headers["accept-encoding"] ||= @options.encodings.keys
end end
end end
end end
@ -65,7 +62,9 @@ module HTTPX
@headers.get("content-encoding").each do |encoding| @headers.get("content-encoding").each do |encoding|
next if encoding == "identity" next if encoding == "identity"
@body = Encoder.new(@body, options.encodings.registry(encoding).deflater) next unless options.encodings.key?(encoding)
@body = Encoder.new(@body, options.encodings[encoding].deflater)
end end
@headers["content-length"] = @body.bytesize unless unbounded_body? @headers["content-length"] = @body.bytesize unless unbounded_body?
end end
@ -95,7 +94,9 @@ module HTTPX
@_inflaters = @headers.get("content-encoding").filter_map do |encoding| @_inflaters = @headers.get("content-encoding").filter_map do |encoding|
next if encoding == "identity" next if encoding == "identity"
inflater = @options.encodings.registry(encoding).inflater(compressed_length) next unless @options.encodings.key?(encoding)
inflater = @options.encodings[encoding].inflater(compressed_length)
# do not uncompress if there is no decoder available. In fact, we can't reliably # do not uncompress if there is no decoder available. In fact, we can't reliably
# continue decompressing beyond that, so ignore. # continue decompressing beyond that, so ignore.
break unless inflater break unless inflater

View File

@ -5,13 +5,13 @@ module HTTPX
module Compression module Compression
module Brotli module Brotli
class << self class << self
def load_dependencies(_klass) def load_dependencies(klass)
require "brotli" require "brotli"
klass.plugin(:compression)
end end
def configure(klass) def extra_options(options)
klass.plugin(:compression) options.merge(encodings: options.encodings.merge("br" => self))
klass.default_options.encodings.register "br", self
end end
end end

View File

@ -4,14 +4,19 @@ module HTTPX
module Plugins module Plugins
module Compression module Compression
module Deflate module Deflate
def self.load_dependencies(_klass) class << self
require "stringio" def load_dependencies(_klass)
require "zlib" require "stringio"
end require "zlib"
end
def self.configure(klass) def configure(klass)
klass.plugin(:"compression/gzip") klass.plugin(:"compression/gzip")
klass.default_options.encodings.register "deflate", self end
def extra_options(options)
options.merge(encodings: options.encodings.merge("deflate" => self))
end
end end
module Deflater module Deflater

View File

@ -6,12 +6,14 @@ module HTTPX
module Plugins module Plugins
module Compression module Compression
module GZIP module GZIP
def self.load_dependencies(*) class << self
require "zlib" def load_dependencies(*)
end require "zlib"
end
def self.configure(klass) def extra_options(options)
klass.default_options.encodings.register "gzip", self options.merge(encodings: options.encodings.merge("gzip" => self))
end
end end
class Deflater class Deflater

View File

@ -233,7 +233,7 @@ module HTTPX
uri.path = rpc_method uri.path = rpc_method
headers = HEADERS.merge( headers = HEADERS.merge(
"grpc-accept-encoding" => ["identity", *@options.encodings.registry.keys] "grpc-accept-encoding" => ["identity", *@options.encodings.keys]
) )
unless deadline == Float::INFINITY unless deadline == Float::INFINITY
# convert to milliseconds # convert to milliseconds
@ -249,7 +249,7 @@ module HTTPX
if compression if compression
headers["grpc-encoding"] = compression headers["grpc-encoding"] = compression
deflater = @options.encodings.registry(compression).deflater deflater = @options.encodings[compression].deflater if @options.encodings.key?(compression)
end end
headers.merge!(@options.call_credentials.call) if @options.call_credentials headers.merge!(@options.call_credentials.call) if @options.call_credentials

View File

@ -47,7 +47,9 @@ module HTTPX
data = message.byteslice(5..size + 5 - 1) data = message.byteslice(5..size + 5 - 1)
if compressed == 1 if compressed == 1
encodings.reverse_each do |algo| encodings.reverse_each do |algo|
inflater = encoders.registry(algo).inflater(size) next unless encoders.key?(algo)
inflater = encoders[algo].inflater(size)
data = inflater.inflate(data) data = inflater.inflate(data)
size = data.bytesize size = data.bytesize
end end

View File

@ -12,13 +12,9 @@ module HTTPX
VALID_H2C_VERBS = %w[GET OPTIONS HEAD].freeze VALID_H2C_VERBS = %w[GET OPTIONS HEAD].freeze
class << self class << self
def load_dependencies(*) def load_dependencies(klass)
require "base64" require "base64"
end
def configure(klass)
klass.plugin(:upgrade) klass.plugin(:upgrade)
klass.default_options.upgrade_handlers.register "h2c", self
end end
def call(connection, request, response) def call(connection, request, response)
@ -26,7 +22,7 @@ module HTTPX
end end
def extra_options(options) def extra_options(options)
options.merge(max_concurrent_requests: 1) options.merge(max_concurrent_requests: 1, upgrade_handlers: options.upgrade_handlers.merge("h2c" => self))
end end
end end
@ -38,7 +34,7 @@ module HTTPX
connection = pool.find_connection(upgrade_request.uri, upgrade_request.options) connection = pool.find_connection(upgrade_request.uri, upgrade_request.options)
return super if connection && connection.upgrade_protocol == :h2c return super if connection && connection.upgrade_protocol == "h2c"
# build upgrade request # build upgrade request
upgrade_request.headers.add("connection", "upgrade") upgrade_request.headers.add("connection", "upgrade")
@ -83,7 +79,7 @@ module HTTPX
set_parser_callbacks(@parser) set_parser_callbacks(@parser)
@inflight += 1 @inflight += 1
@parser.upgrade(request, response) @parser.upgrade(request, response)
@upgrade_protocol = :h2c @upgrade_protocol = "h2c"
if request.options.max_concurrent_requests != @options.max_concurrent_requests if request.options.max_concurrent_requests != @options.max_concurrent_requests
@options = @options.merge(max_concurrent_requests: nil) @options = @options.merge(max_concurrent_requests: nil)

View File

@ -52,7 +52,7 @@ module HTTPX
super super
meter_elapsed_time("Session: initialized!!!") meter_elapsed_time("Session: initialized!!!")
resolver_type = @options.resolver_class resolver_type = @options.resolver_class
resolver_type = Resolver.registry(resolver_type) if resolver_type.is_a?(Symbol) resolver_type = Resolver.resolver_for(resolver_type)
return unless resolver_type <= Resolver::Native return unless resolver_type <= Resolver::Native
resolver_type.prepend TrackTimeMethods resolver_type.prepend TrackTimeMethods

View File

@ -40,9 +40,21 @@ module HTTPX
require "httpx/plugins/multipart/part" require "httpx/plugins/multipart/part"
require "httpx/plugins/multipart/mime_type_detector" require "httpx/plugins/multipart/mime_type_detector"
end end
end
def configure(*) module RequestBodyMethods
Transcoder.register("form", FormTranscoder) private
def initialize_body(options)
return FormTranscoder.encode(options.form) if options.form
super
end
end
module ResponseMethods
def form
decode(FormTranscoder)
end end
end end

View File

@ -318,7 +318,7 @@ module HTTPX
register_plugin :proxy, Proxy register_plugin :proxy, Proxy
end end
class ProxySSL < IO.registry["ssl"] class ProxySSL < SSL
def initialize(tcp, request_uri, options) def initialize(tcp, request_uri, options)
@io = tcp.to_io @io = tcp.to_io
super(request_uri, tcp.addresses, options) super(request_uri, tcp.addresses, options)

View File

@ -61,7 +61,7 @@ module HTTPX
return unless @io.connected? return unless @io.connected?
@parser || begin @parser || begin
@parser = registry(@io.protocol).new(@write_buffer, @options.merge(max_concurrent_requests: 1)) @parser = self.class.parser_type(@io.protocol).new(@write_buffer, @options.merge(max_concurrent_requests: 1))
parser = @parser parser = @parser
parser.extend(ProxyParser) parser.extend(ProxyParser)
parser.on(:response, &method(:__http_on_connect)) parser.on(:response, &method(:__http_on_connect))

View File

@ -15,16 +15,13 @@ module HTTPX
end end
def extra_options(options) def extra_options(options)
upgrade_handlers = Module.new do options.merge(upgrade_handlers: {})
extend Registry
end
options.merge(upgrade_handlers: upgrade_handlers)
end end
end end
module OptionsMethods module OptionsMethods
def option_upgrade_handlers(value) def option_upgrade_handlers(value)
raise TypeError, ":upgrade_handlers must be a registry" unless value.respond_to?(:registry) raise TypeError, ":upgrade_handlers must be a Hash" unless value.is_a?(Hash)
value value
end end
@ -41,9 +38,9 @@ module HTTPX
upgrade_protocol = response.headers["upgrade"].split(/ *, */).first upgrade_protocol = response.headers["upgrade"].split(/ *, */).first
return response unless upgrade_protocol && options.upgrade_handlers.registry.key?(upgrade_protocol) return response unless upgrade_protocol && options.upgrade_handlers.key?(upgrade_protocol)
protocol_handler = options.upgrade_handlers.registry(upgrade_protocol) protocol_handler = options.upgrade_handlers[upgrade_protocol]
return response unless protocol_handler return response unless protocol_handler

View File

@ -10,8 +10,8 @@ module HTTPX
# #
module H2 module H2
class << self class << self
def configure(klass) def extra_options(options)
klass.default_options.upgrade_handlers.register "h2", self options.merge(upgrade_handlers: options.upgrade_handlers.merge("h2" => self))
end end
def call(connection, _request, _response) def call(connection, _request, _response)
@ -32,7 +32,7 @@ module HTTPX
@parser = Connection::HTTP2.new(@write_buffer, @options) @parser = Connection::HTTP2.new(@write_buffer, @options)
set_parser_callbacks(@parser) set_parser_callbacks(@parser)
@upgrade_protocol = :h2 @upgrade_protocol = "h2"
# what's happening here: # what's happening here:
# a deviation from the state machine is done to perform the actions when a # a deviation from the state machine is done to perform the actions when a

View File

@ -244,7 +244,7 @@ module HTTPX
def find_resolver_for(connection) def find_resolver_for(connection)
connection_options = connection.options connection_options = connection.options
resolver_type = connection_options.resolver_class resolver_type = connection_options.resolver_class
resolver_type = Resolver.registry(resolver_type) if resolver_type.is_a?(Symbol) resolver_type = Resolver.resolver_for(resolver_type)
@resolvers[resolver_type] ||= begin @resolvers[resolver_type] ||= begin
resolver_manager = if resolver_type.multi? resolver_manager = if resolver_type.multi?

View File

@ -1,85 +0,0 @@
# frozen_string_literal: true
module HTTPX
# Adds a general-purpose registry API to a class. It is designed to be a
# configuration-level API, i.e. the registry is global to the class and
# should be set on **boot time**.
#
# It is used internally to associate tags with handlers.
#
# ## Register/Fetch
#
# One is strongly advised to register handlers when creating the class.
#
# There is an instance-level method to retrieve from the registry based
# on the tag:
#
# class Server
# include HTTPX::Registry
#
# register "tcp", TCPHandler
# register "ssl", SSLHandlers
# ...
#
#
# def handle(uri)
# scheme = uri.scheme
# handler = registry(scheme) #=> TCPHandler
# handler.handle
# end
# end
#
module Registry
# Base Registry Error
class Error < Error; end
def self.extended(klass)
super
klass.extend(ClassMethods)
end
def self.included(klass)
super
klass.extend(ClassMethods)
klass.__send__(:include, InstanceMethods)
end
# Class Methods
module ClassMethods
def inherited(klass)
super
klass.instance_variable_set(:@registry, @registry.dup)
end
# @param [Object] tag the handler identifier in the registry
# @return [Symbol, String, Object] the corresponding handler (if Symbol or String,
# will assume it referes to an autoloaded module, and will load-and-return it).
#
def registry(tag = nil)
@registry ||= {}
return @registry if tag.nil?
handler = @registry[tag]
raise(Error, "#{tag} is not registered in #{self}") unless handler
handler
end
# @param [Object] tag the identifier for the handler in the registry
# @return [Symbol, String, Object] the handler (if Symbol or String, it is
# assumed to be an autoloaded module, to be loaded later)
#
def register(tag, handler)
registry[tag] = handler
end
end
# Instance Methods
module InstanceMethods
# delegates to HTTPX::Registry#registry
def registry(tag)
self.class.registry(tag)
end
end
end
end

View File

@ -120,7 +120,7 @@ module HTTPX
query = [] query = []
if (q = @options.params) if (q = @options.params)
query << Transcoder.registry("form").encode(q) query << Transcoder::Form.encode(q)
end end
query << @uri.query if @uri.query query << @uri.query if @uri.query
@query = query.join("&") @query = query.join("&")
@ -160,15 +160,7 @@ module HTTPX
def initialize(headers, options) def initialize(headers, options)
@headers = headers @headers = headers
@body = if options.body @body = initialize_body(options)
Transcoder.registry("body").encode(options.body)
elsif options.form
Transcoder.registry("form").encode(options.form)
elsif options.json
Transcoder.registry("json").encode(options.json)
elsif options.xml
Transcoder.registry("xml").encode(options.xml)
end
return if @body.nil? return if @body.nil?
@headers["content-type"] ||= @body.content_type @headers["content-type"] ||= @body.content_type
@ -211,7 +203,7 @@ module HTTPX
def stream(body) def stream(body)
encoded = body encoded = body
encoded = Transcoder.registry("chunker").encode(body.enum_for(:each)) if chunked? encoded = Transcoder::Chunker.encode(body.enum_for(:each)) if chunked?
encoded encoded
end end
@ -235,6 +227,20 @@ module HTTPX
"#{unbounded_body? ? "stream" : "@bytesize=#{bytesize}"}>" "#{unbounded_body? ? "stream" : "@bytesize=#{bytesize}"}>"
end end
# :nocov: # :nocov:
private
def initialize_body(options)
if options.body
Transcoder::Body.encode(options.body)
elsif options.form
Transcoder::Form.encode(options.form)
elsif options.json
Transcoder::JSON.encode(options.json)
elsif options.xml
Transcoder::Xml.encode(options.xml)
end
end
end end
def transition(nextstate) def transition(nextstate)

View File

@ -5,8 +5,6 @@ require "ipaddr"
module HTTPX module HTTPX
module Resolver module Resolver
extend Registry
RESOLVE_TIMEOUT = 5 RESOLVE_TIMEOUT = 5
require "httpx/resolver/resolver" require "httpx/resolver/resolver"
@ -15,10 +13,6 @@ module HTTPX
require "httpx/resolver/https" require "httpx/resolver/https"
require "httpx/resolver/multi" require "httpx/resolver/multi"
register :system, System
register :native, Native
register :https, HTTPS
@lookup_mutex = Mutex.new @lookup_mutex = Mutex.new
@lookups = Hash.new { |h, k| h[k] = [] } @lookups = Hash.new { |h, k| h[k] = [] }
@ -28,6 +22,18 @@ module HTTPX
module_function module_function
def resolver_for(resolver_type)
case resolver_type
when :native then Native
when :system then System
when :https then HTTPS
else
return resolver_type if resolver_type.is_a?(Class) && resolver_type < Resolver
raise Error, "unsupported resolver type (#{resolver_type})"
end
end
def nolookup_resolve(hostname) def nolookup_resolve(hostname)
ip_resolve(hostname) || cached_lookup(hostname) || system_resolve(hostname) ip_resolve(hostname) || cached_lookup(hostname) || system_resolve(hostname)
end end

View File

@ -87,32 +87,27 @@ module HTTPX
end end
def json(*args) def json(*args)
decode("json", *args) decode(Transcoder::JSON, *args)
end end
def form def form
decode("form") decode(Transcoder::Form)
end end
def xml def xml
decode("xml") decode(Transcoder::Xml)
end end
private private
def decode(format, *args) def decode(transcoder, *args)
# TODO: check if content-type is a valid format, i.e. "application/json" for json parsing # TODO: check if content-type is a valid format, i.e. "application/json" for json parsing
transcoder = Transcoder.registry(format)
raise Error, "no decoder available for \"#{format}\"" unless transcoder.respond_to?(:decode)
decoder = transcoder.decode(self) decoder = transcoder.decode(self)
raise Error, "no decoder available for \"#{format}\"" unless decoder raise Error, "no decoder available for \"#{transcoder}\"" unless decoder
decoder.call(self, *args) decoder.call(self, *args)
rescue Registry::Error
raise Error, "no decoder available for \"#{format}\""
end end
def no_data? def no_data?
@ -203,10 +198,8 @@ module HTTPX
rescue ArgumentError # ex: unknown encoding name - utf rescue ArgumentError # ex: unknown encoding name - utf
content content
end end
when nil
"".b
else else
@buffer "".b
end end
end end
alias_method :to_str, :to_s alias_method :to_str, :to_s

View File

@ -203,6 +203,7 @@ module HTTPX
end end
def receive_requests(requests, connections) def receive_requests(requests, connections)
# @type var responses: Array[response]
responses = [] responses = []
begin begin

View File

@ -2,8 +2,6 @@
module HTTPX module HTTPX
module Transcoder module Transcoder
extend Registry
using RegexpExtensions unless Regexp.method_defined?(:match?) using RegexpExtensions unless Regexp.method_defined?(:match?)
module_function module_function

View File

@ -55,5 +55,4 @@ module HTTPX::Transcoder
Encoder.new(body) Encoder.new(body)
end end
end end
register "body", Body
end end

View File

@ -112,5 +112,4 @@ module HTTPX::Transcoder
Encoder.new(chunks) Encoder.new(chunks)
end end
end end
register "chunker", Chunker
end end

View File

@ -55,5 +55,4 @@ module HTTPX::Transcoder
Decoder Decoder
end end
end end
register "form", Form
end end

View File

@ -56,5 +56,4 @@ module HTTPX::Transcoder
end end
# rubocop:enable Style/SingleLineMethods # rubocop:enable Style/SingleLineMethods
end end
register "json", JSON
end end

View File

@ -51,5 +51,4 @@ module HTTPX::Transcoder
end end
end end
end end
register "xml", Xml
end end

View File

@ -1,5 +1,5 @@
# frozen_string_literal: true # frozen_string_literal: true
module HTTPX module HTTPX
VERSION = "0.22.5" VERSION = "0.23.0"
end end

View File

@ -17,7 +17,6 @@ module HTTPX
extend Forwardable extend Forwardable
include Loggable include Loggable
include Callbacks include Callbacks
include HTTPX::Registry[String, Class]
attr_reader type: io_type attr_reader type: io_type
@ -35,7 +34,13 @@ module HTTPX
@write_buffer: Buffer @write_buffer: Buffer
@inflight: Integer @inflight: Integer
@keep_alive_timeout: Numeric? @keep_alive_timeout: Numeric?
@timeout: Numeric?
@current_timeout: Numeric?
@total_timeout: Numeric? @total_timeout: Numeric?
@io: TCP | SSL | UNIX
@parser: HTTP1 | HTTP2 | _Parser
@connected_at: Float
@response_received_at: Float
def addresses: () -> Array[ipaddr]? def addresses: () -> Array[ipaddr]?
@ -76,6 +81,8 @@ module HTTPX
def deactivate: () -> void def deactivate: () -> void
def open?: () -> bool
def raise_timeout_error: (Numeric interval) -> void def raise_timeout_error: (Numeric interval) -> void
private private
@ -90,17 +97,20 @@ module HTTPX
def send_pending: () -> void def send_pending: () -> void
def parser: () -> _Parser def parser: () -> (HTTP1 | HTTP2 | _Parser)
def send_request_to_parser: (Request request) -> void def send_request_to_parser: (Request request) -> void
def build_parser: () -> _Parser def build_parser: (?String protocol) -> (HTTP1 | HTTP2)
| (String) -> _Parser
def set_parser_callbacks: (_Parser) -> void def set_parser_callbacks: (HTTP1 | HTTP2 parser) -> void
def transition: (Symbol) -> void def transition: (Symbol) -> void
def handle_transition: (Symbol) -> void
def build_socket: (?Array[ipaddr]? addrs) -> (TCP | SSL | UNIX)
def on_error: (HTTPX::TimeoutError | Error | StandardError) -> void def on_error: (HTTPX::TimeoutError | Error | StandardError) -> void
def handle_error: (StandardError) -> void def handle_error: (StandardError) -> void
@ -112,5 +122,7 @@ module HTTPX
def write_timeout_callback: (Request request, Numeric write_timeout) -> void def write_timeout_callback: (Request request, Numeric write_timeout) -> void
def read_timeout_callback: (Request request, Numeric read_timeout, ?singleton(RequestTimeoutError) error_type) -> void def read_timeout_callback: (Request request, Numeric read_timeout, ?singleton(RequestTimeoutError) error_type) -> void
def self.parser_type: (String protocol) -> (singleton(HTTP1) | singleton(HTTP2))
end end
end end

View File

@ -2,6 +2,12 @@ module HTTPX
class Error < StandardError class Error < StandardError
end end
class UnsupportedSchemeError < Error
end
class ConnectionError < Error
end
class TimeoutError < Error class TimeoutError < Error
attr_reader timeout: Numeric attr_reader timeout: Numeric
@ -55,4 +61,7 @@ module HTTPX
def initialize: (Connection connection, String hostname, ?String message) -> untyped def initialize: (Connection connection, String hostname, ?String message) -> untyped
end end
class MisdirectedRequestError < HTTPError
end
end end

View File

@ -1,6 +1,9 @@
module HTTPX module HTTPX
IPRegex: Regexp IPRegex: Regexp
class TLSError < OpenSSL::SSL::SSLError
end
class SSL < TCP class SSL < TCP
TLS_OPTIONS: Hash[Symbol, untyped] TLS_OPTIONS: Hash[Symbol, untyped]

View File

@ -10,7 +10,7 @@ module HTTPX
attr_reader state: Symbol attr_reader state: Symbol
attr_reader interests: Symbol attr_reader interests: io_interests
alias host ip alias host ip

View File

@ -46,7 +46,7 @@ module HTTPX
attr_reader body_threshold_size: Integer attr_reader body_threshold_size: Integer
# transport # transport
attr_reader transport: String? attr_reader transport: "unix" | nil
# transport_options # transport_options
attr_reader transport_options: Hash[untyped, untyped]? attr_reader transport_options: Hash[untyped, untyped]?

View File

@ -1,7 +1,6 @@
module HTTPX module HTTPX
module Plugins module Plugins
module Compression module Compression
type encodings_registry = Registry[Symbol, Class]
type deflatable = _Reader | _ToS type deflatable = _Reader | _ToS
@ -16,12 +15,17 @@ module HTTPX
def initialize: (Integer | Float bytesize) -> untyped def initialize: (Integer | Float bytesize) -> untyped
end end
interface _Compressor
def deflater: () -> _Deflater
def inflater: (Integer | Float bytesize) -> _Inflater
end
def self.configure: (singleton(Session)) -> void def self.configure: (singleton(Session)) -> void
interface _CompressionOptions interface _CompressionOptions
def compression_threshold_size: () -> Integer? def compression_threshold_size: () -> Integer?
def encodings: () -> encodings_registry? def encodings: () -> Hash[String, _Compressor]
end end
def self.extra_options: (Options) -> (Options & _CompressionOptions) def self.extra_options: (Options) -> (Options & _CompressionOptions)

View File

@ -13,8 +13,8 @@ module HTTPX
def []: (uri) -> Array[Cookie] def []: (uri) -> Array[Cookie]
def each: (?uri) { (Cookie) -> void } -> void def each: (?uri?) { (Cookie) -> void } -> void
| (?uri) -> Enumerable[Cookie] | (?uri?) -> Enumerable[Cookie]
def merge: (_Each[cookie] cookies) -> instance def merge: (_Each[cookie] cookies) -> instance

View File

@ -39,8 +39,8 @@ module HTTPX
def self?.encode: (String bytes, ?deflater: Compression::_Deflater?) -> String def self?.encode: (String bytes, ?deflater: Compression::_Deflater?) -> String
def self?.decode: (String message, encodings: Array[String], encoders: Compression::encodings_registry) -> String def self?.decode: (String message, encodings: Array[String], encoders: Hash[String, Compression::_Compressor]) -> String
| (String message, encodings: Array[String], encoders: Compression::encodings_registry) { (String) -> void } -> void | (String message, encodings: Array[String], encoders: Hash[String, Compression::_Compressor]) { (String) -> void } -> void
def self?.cancel: (Request) -> void def self?.cancel: (Request) -> void
@ -65,7 +65,7 @@ module HTTPX
module ResponseMethods module ResponseMethods
def merge_headers: (headers_input trailers) -> void def merge_headers: (headers_input trailers) -> void
def encoders: () -> Compression::encodings_registry def encoders: () -> Hash[String, Compression::_Compressor]
end end
module InstanceMethods module InstanceMethods

View File

@ -2,7 +2,7 @@ module HTTPX
module Plugins module Plugins
module Retries module Retries
MAX_RETRIES: Integer MAX_RETRIES: Integer
IDEMPOTENT_METHODS: Array[verb] IDEMPOTENT_METHODS: Array[String]
RETRYABLE_ERRORS: Array[singleton(StandardError)] RETRYABLE_ERRORS: Array[singleton(StandardError)]
DEFAULT_JITTER: ^(Numeric) -> Numeric DEFAULT_JITTER: ^(Numeric) -> Numeric

View File

@ -1,18 +1,20 @@
module HTTPX module HTTPX
module Plugins module Plugins
module Upgrade module Upgrade
type handlers_registry = Registry[Symbol, Class] interface _Upgrader
def call: (Connection connection, Request request, Response response) -> void
end
def self.configure: (singleton(Session)) -> void def self.configure: (singleton(Session)) -> void
interface _UpgradeOptions interface _UpgradeOptions
def upgrade_handlers: () -> handlers_registry? def upgrade_handlers: () -> Hash[String, _Upgrader]
end end
def self.extra_options: (Options) -> (Options & _UpgradeOptions) def self.extra_options: (Options) -> (Options & _UpgradeOptions)
module ConnectionMethods module ConnectionMethods
attr_reader upgrade_protocol: Symbol? attr_reader upgrade_protocol: String?
attr_reader hijacked: boolish attr_reader hijacked: boolish
def hijack_io: () -> void def hijack_io: () -> void

View File

@ -1,13 +0,0 @@
module HTTPX::Registry[unchecked out T, unchecked out V]
class Error < HTTPX::Error
end
# type registrable = Symbol | String | Class
def self.registry: [T, V] (T) -> Class
| [T, V] () -> Hash[T, V]
def self.register: [T, V] (T tag, V handler) -> void
def registry: (?T tag) -> V
end

View File

@ -7,7 +7,7 @@ module HTTPX
USER_AGENT: String USER_AGENT: String
attr_reader verb: verb attr_reader verb: verb
attr_reader uri: URI::Generic attr_reader uri: URI::HTTP | URI::HTTPS
attr_reader headers: Headers attr_reader headers: Headers
attr_reader body: Body attr_reader body: Body
attr_reader state: Symbol attr_reader state: Symbol
@ -56,6 +56,10 @@ module HTTPX
def request_timeout: () -> Numeric def request_timeout: () -> Numeric
private
def initialize_body: (Options options) -> Transcoder::_Encoder?
class Body class Body
@headers: Headers @headers: Headers
@body: body_encoder? @body: body_encoder?

View File

@ -2,8 +2,6 @@ module HTTPX
type ipaddr = IPAddr | String type ipaddr = IPAddr | String
module Resolver module Resolver
extend Registry[Symbol, Class]
RESOLVE_TIMEOUT: Integer | Float RESOLVE_TIMEOUT: Integer | Float
@lookup_mutex: Thread::Mutex @lookup_mutex: Thread::Mutex
@ -21,6 +19,11 @@ module HTTPX
def system_resolve: (String hostname) -> Array[IPAddr]? def system_resolve: (String hostname) -> Array[IPAddr]?
def self?.resolver_for: (:native resolver_type) -> singleton(Native) |
(:system resolver_type) -> singleton(System) |
(:https resolver_type) -> singleton(HTTPS) |
[U] (U resolver_type) -> U
def self?.cached_lookup: (String hostname) -> Array[IPAddr]? def self?.cached_lookup: (String hostname) -> Array[IPAddr]?
def self?.cached_lookup_set: (String hostname, ip_family family, Array[dns_result] addresses) -> void def self?.cached_lookup_set: (String hostname, ip_family family, Array[dns_result] addresses) -> void

View File

@ -41,14 +41,14 @@ module HTTPX
def initialize: (Request request, String | Integer status, String version, headers?) -> untyped def initialize: (Request request, String | Integer status, String version, headers?) -> untyped
def no_data?: () -> bool def no_data?: () -> bool
def decode:(String format, ?untyped options) -> untyped def decode:(Transcoder::_Decode transcoder, ?untyped options) -> untyped
class Body class Body
include _Reader include _Reader
include _ToS include _ToS
include _ToStr include _ToStr
attr_reader encoding: String attr_reader encoding: Encoding | String
@response: Response @response: Response
@headers: Headers @headers: Headers

View File

@ -11,8 +11,6 @@ module HTTPX
def self.plugin: (Symbol | Module plugin, ?options? options) ?{ (Class) -> void } -> singleton(Session) def self.plugin: (Symbol | Module plugin, ?options? options) ?{ (Class) -> void } -> singleton(Session)
def self.default_options: -> Options
def wrap: () { (instance) -> void } -> void def wrap: () { (instance) -> void } -> void
def close: (*untyped) -> void def close: (*untyped) -> void
@ -41,12 +39,14 @@ module HTTPX
| (verb, _Each[[uri, options]], Options) -> Array[Request] | (verb, _Each[[uri, options]], Options) -> Array[Request]
| (verb, _Each[uri], options) -> Array[Request] | (verb, _Each[uri], options) -> Array[Request]
def build_connection: (URI::Generic, Options) -> Connection def build_connection: (URI::HTTP | URI::HTTPS uri, Options options) -> Connection
def send_requests: (*Request) -> Array[response] def send_requests: (*Request) -> Array[response]
def _send_requests: (Array[Request]) -> Array[Connection] def _send_requests: (Array[Request]) -> Array[Connection]
def receive_requests: (Array[Request], Array[Connection]) -> Array[response] def receive_requests: (Array[Request], Array[Connection]) -> Array[response]
attr_reader self.default_options: Options
end end
end end

View File

@ -4,11 +4,6 @@ module HTTPX
type body_encoder = Transcoder::_Encoder | _Each[String] type body_encoder = Transcoder::_Encoder | _Each[String]
module Transcoder module Transcoder
def self?.registry: (String tag) -> _Encode
| () -> Hash[String, _Encode]
def self?.register: (String tag, _Encode handler) -> void
def self?.normalize_keys: [U] (_ToS key, _ToAry[untyped] | _ToHash[_ToS, untyped] | untyped value, ?(^(untyped value) -> bool | nil) cond) { (String, ?untyped) -> U } -> U def self?.normalize_keys: [U] (_ToS key, _ToAry[untyped] | _ToHash[_ToS, untyped] | untyped value, ?(^(untyped value) -> bool | nil) cond) { (String, ?untyped) -> U } -> U
def self?.normalize_query: (Hash[String, untyped] params, String name, String v, Integer depth) -> void def self?.normalize_query: (Hash[String, untyped] params, String name, String v, Integer depth) -> void
@ -17,6 +12,10 @@ module HTTPX
def encode: (untyped payload) -> body_encoder def encode: (untyped payload) -> body_encoder
end end
interface _Decode
def decode: (HTTPX::Response response) -> _Decoder
end
interface _Encoder interface _Encoder
def bytesize: () -> (Integer | Float) def bytesize: () -> (Integer | Float)
end end

View File

@ -1,5 +1,5 @@
module HTTPX::Transcoder module HTTPX::Transcoder
module XML module Xml
def self?.encode: (untyped xml) -> Encoder def self?.encode: (untyped xml) -> Encoder
def self?.decode: (HTTPX::Response response) -> _Decoder def self?.decode: (HTTPX::Response response) -> _Decoder

16
test/compression_test.rb Normal file
View File

@ -0,0 +1,16 @@
# frozen_string_literal: true
require_relative "test_helper"
require "httpx/plugins/compression"
class CompressionTest < Minitest::Test
include HTTPX
def test_ignore_encoding_on_range
session = HTTPX.plugin(:compression)
request = session.build_request("GET", "http://example.com")
assert request.headers.key?("accept-encoding")
range_request = session.build_request("GET", "http://example.com", headers: { "range" => "bytes=100-200" })
assert !range_request.headers.key?("accept-encoding")
end
end

115
test/cookie_jar_test.rb Normal file
View File

@ -0,0 +1,115 @@
# frozen_string_literal: true
require_relative "test_helper"
class CookieJarTest < Minitest::Test
def test_plugin_cookies_jar
HTTPX.plugin(:cookies) # force loading the modules
# Test special cases
special_jar = HTTPX::Plugins::Cookies::Jar.new
special_jar.parse(%(a="b"; Path=/, c=d; Path=/, e="f\\"; \\"g"))
cookies = special_jar[jar_cookies_uri]
assert(cookies.one? { |cookie| cookie.name == "a" && cookie.value == "b" })
assert(cookies.one? { |cookie| cookie.name == "c" && cookie.value == "d" })
assert(cookies.one? { |cookie| cookie.name == "e" && cookie.value == "f\"; \"g" })
# Test secure parameter
secure_jar = HTTPX::Plugins::Cookies::Jar.new
secure_jar.parse(%(a=b; Path=/; Secure))
assert !secure_jar[jar_cookies_uri(scheme: "https")].empty?, "cookie jar should contain the secure cookie"
assert secure_jar[jar_cookies_uri(scheme: "http")].empty?, "cookie jar should not contain the secure cookie"
# Test path parameter
path_jar = HTTPX::Plugins::Cookies::Jar.new
path_jar.parse(%(a=b; Path=/cookies))
assert path_jar[jar_cookies_uri("/")].empty?
assert !path_jar[jar_cookies_uri("/cookies")].empty?
assert !path_jar[jar_cookies_uri("/cookies/set")].empty?
# Test expires
maxage_jar = HTTPX::Plugins::Cookies::Jar.new
maxage_jar.parse(%(a=b; Path=/; Max-Age=2))
assert !maxage_jar[jar_cookies_uri].empty?
sleep 3
assert maxage_jar[jar_cookies_uri].empty?
expires_jar = HTTPX::Plugins::Cookies::Jar.new
expires_jar.parse(%(a=b; Path=/; Expires=Sat, 02 Nov 2019 15:24:00 GMT))
assert expires_jar[jar_cookies_uri].empty?
# regression test
rfc2616_expires_jar = HTTPX::Plugins::Cookies::Jar.new
rfc2616_expires_jar.parse(%(a=b; Path=/; Expires=Fri, 17-Feb-2033 12:43:41 GMT))
assert !rfc2616_expires_jar[jar_cookies_uri].empty?
# Test domain
domain_jar = HTTPX::Plugins::Cookies::Jar.new
domain_jar.parse(%(a=b; Path=/; Domain=.google.com))
assert domain_jar[jar_cookies_uri].empty?
assert !domain_jar["http://www.google.com/"].empty?
ipv4_domain_jar = HTTPX::Plugins::Cookies::Jar.new
ipv4_domain_jar.parse(%(a=b; Path=/; Domain=137.1.0.12))
assert ipv4_domain_jar["http://www.google.com/"].empty?
assert !ipv4_domain_jar["http://137.1.0.12/"].empty?
ipv6_domain_jar = HTTPX::Plugins::Cookies::Jar.new
ipv6_domain_jar.parse(%(a=b; Path=/; Domain=[fe80::1]))
assert ipv6_domain_jar["http://www.google.com/"].empty?
assert !ipv6_domain_jar["http://[fe80::1]/"].empty?
# Test duplicate
dup_jar = HTTPX::Plugins::Cookies::Jar.new
dup_jar.parse(%(a=c, a=a, a=b))
cookies = dup_jar[jar_cookies_uri]
assert cookies.size == 1, "should only have kept one of the received \"a\" cookies"
cookie = cookies.first
assert cookie.name == "a", "unexpected name"
assert cookie.value == "b", "unexpected value, should have been \"b\", instead it's \"#{cookie.value}\""
end
def test_plugin_cookies_jar_merge
HTTPX.plugin(:cookies) # force loading the modules
jar = HTTPX::Plugins::Cookies::Jar.new
assert jar.each.to_a == []
assert jar.merge("a" => "b").each.map { |c| [c.name, c.value] } == [%w[a b]]
assert jar.merge([HTTPX::Plugins::Cookies::Cookie.new("a", "b")]).each.map { |c| [c.name, c.value] } == [%w[a b]]
assert jar.merge([{ name: "a", value: "b" }]).each.map { |c| [c.name, c.value] } == [%w[a b]]
end
def test_plugins_cookies_cookie
HTTPX.plugin(:cookies) # force loading the modules
# match against uris
acc_c1 = HTTPX::Plugins::Cookies::Cookie.new("a", "b")
assert acc_c1.send(:acceptable_from_uri?, "https://www.google.com")
acc_c2 = HTTPX::Plugins::Cookies::Cookie.new("a", "b", domain: ".google.com")
assert acc_c2.send(:acceptable_from_uri?, "https://www.google.com")
assert !acc_c2.send(:acceptable_from_uri?, "https://nghttp2.org")
acc_c3 = HTTPX::Plugins::Cookies::Cookie.new("a", "b", domain: "google.com")
assert !acc_c3.send(:acceptable_from_uri?, "https://www.google.com")
# quoting funny characters
sch_cookie = HTTPX::Plugins::Cookies::Cookie.new("Bar", "value\"4")
assert sch_cookie.cookie_value == %(Bar="value\\"4")
# sorting
c1 = HTTPX::Plugins::Cookies::Cookie.new("a", "b")
c2 = HTTPX::Plugins::Cookies::Cookie.new("a", "bc")
assert [c2, c1].sort == [c1, c2]
c3 = HTTPX::Plugins::Cookies::Cookie.new("a", "b", path: "/cookies")
assert [c3, c2, c1].sort == [c3, c1, c2]
c4 = HTTPX::Plugins::Cookies::Cookie.new("a", "b", created_at: (Time.now - (60 * 60 * 24)))
assert [c4, c3, c2, c1].sort == [c3, c4, c1, c2]
end
private
def jar_cookies_uri(path = "/cookies", scheme: "http")
"#{scheme}://example.com#{path}"
end
end

View File

@ -10,6 +10,11 @@ class ErrorResponseTest < Minitest::Test
assert r1.status == "wow" assert r1.status == "wow"
end end
def test_error_response_finished?
r1 = ErrorResponse.new(request_mock, RuntimeError.new("wow"), {})
assert r1.finished?
end
def test_error_response_error def test_error_response_error
error = RuntimeError.new("wow") error = RuntimeError.new("wow")
r1 = ErrorResponse.new(request_mock, error, {}) r1 = ErrorResponse.new(request_mock, error, {})
@ -28,9 +33,18 @@ class ErrorResponseTest < Minitest::Test
assert str.match(/wow \(.*RuntimeError.*\)/), "expected \"wow (RuntimeError)\" in \"#{str}\"" assert str.match(/wow \(.*RuntimeError.*\)/), "expected \"wow (RuntimeError)\" in \"#{str}\""
end end
def test_error_response_close
response = Response.new(request_mock, 200, "1.1", {})
request_mock.response = response
r = ErrorResponse.new(request_mock, RuntimeError.new("wow"), {})
assert !response.body.closed?
r.close
assert response.body.closed?
end
private private
def request_mock def request_mock
Request.new("GET", "http://example.com/") @request_mock ||= Request.new("GET", "http://example.com/")
end end
end end

View File

@ -17,6 +17,17 @@ class ProxyTest < Minitest::Test
assert params != 1 assert params != 1
end end
%w[basic digest ntlm].each do |auth_method|
define_method :"test_proxy_factory_#{auth_method}" do
basic_proxy_opts = HTTPX.plugin(:proxy).__send__(:"with_proxy_#{auth_method}_auth", username: "user",
password: "pass").instance_variable_get(:@options)
proxy = basic_proxy_opts.proxy
assert proxy[:username] == "user"
assert proxy[:password] == "pass"
assert proxy[:scheme] == auth_method
end
end
private private
def parameters(uri: "http://proxy", **args) def parameters(uri: "http://proxy", **args)

View File

@ -29,4 +29,13 @@ class ResolverTest < Minitest::Test
ips = Resolver.cached_lookup("test.com") ips = Resolver.cached_lookup("test.com")
assert ips == %w[127.0.0.2 ::2 ::3] assert ips == %w[127.0.0.2 ::2 ::3]
end end
def test_resolver_for
assert Resolver.resolver_for(:native) == Resolver::Native
assert Resolver.resolver_for(:system) == Resolver::System
assert Resolver.resolver_for(:https) == Resolver::HTTPS
assert Resolver.resolver_for(Resolver::HTTPS) == Resolver::HTTPS
ex = assert_raises(Error) { Resolver.resolver_for(Object) }
assert(ex.message.include?("unsupported resolver type"))
end
end end

View File

@ -82,6 +82,23 @@ class ResponseTest < Minitest::Test
body5.write(payload) body5.write(payload)
assert body5 == "a" * 2048, "body messed up with file" assert body5 == "a" * 2048, "body messed up with file"
assert body5 == StringIO.new("a" * 2048), "body messed up with file" assert body5 == StringIO.new("a" * 2048), "body messed up with file"
text = ("ã" * 2048).b
body6 = Response::Body.new(Response.new(request, 200, "2.0", { "content-type" => "text/html; charset=utf" }),
Options.new(body_threshold_size: 1024))
body6.write(text)
req_text = body6.to_s
assert text == req_text, "request body must be in original encoding (#{req_text})"
end
def test_response_body_close
payload = "a" * 512
body = Response::Body.new(Response.new(request, 200, "2.0", {}), Options.new(body_threshold_size: 1024))
assert !body.closed?
body.write(payload)
assert !body.closed?
body.close
assert body.closed?
end end
def test_response_body_copy_to_memory def test_response_body_copy_to_memory
@ -205,9 +222,6 @@ class ResponseTest < Minitest::Test
form4_response = Response.new(request, 200, "2.0", { "content-type" => "application/x-www-form-urlencoded" }) form4_response = Response.new(request, 200, "2.0", { "content-type" => "application/x-www-form-urlencoded" })
form4_response << "[]" form4_response << "[]"
assert form4_response.form == {} assert form4_response.form == {}
error = assert_raises(HTTPX::Error) { form2_response.__send__(:decode, "bla") }
assert error.message.include?("no decoder available for"), "failed with unexpected error"
end end
private private

View File

@ -50,6 +50,8 @@ module Requests
verify_status(response, 401) verify_status(response, 401)
response = session.get(build_uri("/get")) response = session.get(build_uri("/get"))
verify_status(response, 200) verify_status(response, 200)
response = session.digest_auth(user, pass).get(build_uri("/get"))
verify_status(response, 200)
end end
# NTLM # NTLM
@ -68,6 +70,11 @@ module Requests
response = http.ntlm_auth("user", "password").get(uri) response = http.ntlm_auth("user", "password").get(uri)
verify_status(response, 200) verify_status(response, 200)
# bypass
response = http.get(build_uri("/get"))
verify_status(response, 200)
response = http.ntlm_auth("user", "password").get(build_uri("/get"))
verify_status(response, 200)
# invalid_response = http.ntlm_authentication("user", "fake").get(uri) # invalid_response = http.ntlm_authentication("user", "fake").get(uri)
# verify_status(invalid_response, 401) # verify_status(invalid_response, 401)
end end

View File

@ -103,104 +103,6 @@ module Requests
verify_cookies(body["cookies"], session_cookies) verify_cookies(body["cookies"], session_cookies)
end end
def test_plugin_cookies_jar
HTTPX.plugin(:cookies) # force loading the modules
# Test special cases
special_jar = HTTPX::Plugins::Cookies::Jar.new
special_jar.parse(%(a="b"; Path=/, c=d; Path=/, e="f\\"; \\"g"))
cookies = special_jar[jar_cookies_uri]
assert(cookies.one? { |cookie| cookie.name == "a" && cookie.value == "b" })
assert(cookies.one? { |cookie| cookie.name == "c" && cookie.value == "d" })
assert(cookies.one? { |cookie| cookie.name == "e" && cookie.value == "f\"; \"g" })
# Test secure parameter
secure_jar = HTTPX::Plugins::Cookies::Jar.new
secure_jar.parse(%(a=b; Path=/; Secure))
cookies = secure_jar[jar_cookies_uri]
if URI(cookies_uri).scheme == "https"
assert !cookies.empty?, "cookie jar should contain the secure cookie"
else
assert cookies.empty?, "cookie jar should not contain the secure cookie"
end
# Test path parameter
path_jar = HTTPX::Plugins::Cookies::Jar.new
path_jar.parse(%(a=b; Path=/cookies))
assert path_jar[jar_cookies_uri("/")].empty?
assert !path_jar[jar_cookies_uri("/cookies")].empty?
assert !path_jar[jar_cookies_uri("/cookies/set")].empty?
# Test expires
maxage_jar = HTTPX::Plugins::Cookies::Jar.new
maxage_jar.parse(%(a=b; Path=/; Max-Age=2))
assert !maxage_jar[jar_cookies_uri].empty?
sleep 3
assert maxage_jar[jar_cookies_uri].empty?
expires_jar = HTTPX::Plugins::Cookies::Jar.new
expires_jar.parse(%(a=b; Path=/; Expires=Sat, 02 Nov 2019 15:24:00 GMT))
assert expires_jar[jar_cookies_uri].empty?
# regression test
rfc2616_expires_jar = HTTPX::Plugins::Cookies::Jar.new
rfc2616_expires_jar.parse(%(a=b; Path=/; Expires=Fri, 17-Feb-2033 12:43:41 GMT))
assert !rfc2616_expires_jar[jar_cookies_uri].empty?
# Test domain
domain_jar = HTTPX::Plugins::Cookies::Jar.new
domain_jar.parse(%(a=b; Path=/; Domain=.google.com))
assert domain_jar[jar_cookies_uri].empty?
assert !domain_jar["http://www.google.com/"].empty?
ipv4_domain_jar = HTTPX::Plugins::Cookies::Jar.new
ipv4_domain_jar.parse(%(a=b; Path=/; Domain=137.1.0.12))
assert ipv4_domain_jar["http://www.google.com/"].empty?
assert !ipv4_domain_jar["http://137.1.0.12/"].empty?
ipv6_domain_jar = HTTPX::Plugins::Cookies::Jar.new
ipv6_domain_jar.parse(%(a=b; Path=/; Domain=[fe80::1]))
assert ipv6_domain_jar["http://www.google.com/"].empty?
assert !ipv6_domain_jar["http://[fe80::1]/"].empty?
# Test duplicate
dup_jar = HTTPX::Plugins::Cookies::Jar.new
dup_jar.parse(%(a=c, a=a, a=b))
cookies = dup_jar[jar_cookies_uri]
assert cookies.size == 1, "should only have kept one of the received \"a\" cookies"
cookie = cookies.first
assert cookie.name == "a", "unexpected name"
assert cookie.value == "b", "unexpected value, should have been \"b\", instead it's \"#{cookie.value}\""
end
def test_plugins_cookies_cookie
HTTPX.plugin(:cookies) # force loading the modules
# match against uris
acc_c1 = HTTPX::Plugins::Cookies::Cookie.new("a", "b")
assert acc_c1.send(:acceptable_from_uri?, "https://www.google.com")
acc_c2 = HTTPX::Plugins::Cookies::Cookie.new("a", "b", domain: ".google.com")
assert acc_c2.send(:acceptable_from_uri?, "https://www.google.com")
assert !acc_c2.send(:acceptable_from_uri?, "https://nghttp2.org")
acc_c3 = HTTPX::Plugins::Cookies::Cookie.new("a", "b", domain: "google.com")
assert !acc_c3.send(:acceptable_from_uri?, "https://www.google.com")
# quoting funny characters
sch_cookie = HTTPX::Plugins::Cookies::Cookie.new("Bar", "value\"4")
assert sch_cookie.cookie_value == %(Bar="value\\"4")
# sorting
c1 = HTTPX::Plugins::Cookies::Cookie.new("a", "b")
c2 = HTTPX::Plugins::Cookies::Cookie.new("a", "bc")
assert [c2, c1].sort == [c1, c2]
c3 = HTTPX::Plugins::Cookies::Cookie.new("a", "b", path: "/cookies")
assert [c3, c2, c1].sort == [c3, c1, c2]
c4 = HTTPX::Plugins::Cookies::Cookie.new("a", "b", created_at: (Time.now - (60 * 60 * 24)))
assert [c4, c3, c2, c1].sort == [c3, c4, c1, c2]
end
def test_plugin_cookies_jar_management def test_plugin_cookies_jar_management
cookie_header = lambda do |response| cookie_header = lambda do |response|
JSON.parse(response.to_s)["headers"] JSON.parse(response.to_s)["headers"]
@ -223,11 +125,6 @@ module Requests
private private
def jar_cookies_uri(path = "/cookies")
jar_origin = URI(origin).origin
build_uri(path, jar_origin)
end
def cookies_uri def cookies_uri
build_uri("/cookies") build_uri("/cookies")
end end

View File

@ -112,9 +112,8 @@ end
module WSTestPlugin module WSTestPlugin
class << self class << self
def configure(klass) def load_dependencies(klass)
klass.plugin(:upgrade) klass.plugin(:upgrade)
klass.default_options.upgrade_handlers.register("websocket", self)
end end
def call(connection, request, response) def call(connection, request, response)
@ -128,7 +127,7 @@ module WSTestPlugin
end end
def extra_options(options) def extra_options(options)
options.merge(max_concurrent_requests: 1) options.merge(max_concurrent_requests: 1, upgrade_handlers: options.upgrade_handlers.merge("websocket" => self))
end end
end end