* Introduce types to ruby
* rubocop
* forgot these files
* other request methods
* more tests, fix for raw request
* Add README entry for types
* rebase and regen
This fixes the typo of `mangers` to be `managers` in the documentation
for the `default_connection_managers` method on the `ThreadContext`
class in `lib/stripe/stripe_client.rb`.
Co-authored-by: prathmesh-stripe <165320323+prathmesh-stripe@users.noreply.github.com>
* Add response field to RequestEndEvent
* Add request body field to RequestEventEnd
* Add request headers field to RequestEventEnd
* Create a RequestContext class
* Create a ResponseContext class
* Update tests
This is just a cosmetic change that renames `Stripe.configuration` to
just `Stripe.config`. We use the shorter "config" in most other places
including `StripeClient#config`, so I feel that this is overall more
consistent.
This change is backwards compatible because the new accessor came in
with #968, and that hasn't been given a formal release yet.
I've left the class name as `StripeConfiguration` which IMO is fine. The
class uses the expanded form of the name while vars and accessors use
the shorter `config`. Also, `StripeConfiguration` has been around a
little bit longer, so renaming it is somewhat backwards incompatible
too.
Follows up #968.
As a relic from when we had global configuration, anytime any config
value is changed on any client, we still clear all connection managers
everywhere on every thread, even though this is not necessary. This
means that we lose all open connections, etc.
Here, we make changes so that if a configuration is changed, we only
clear the configuration managers pertaining to that one particular
configuration, thus conserving resources globally.
Co-authored-by: Brandur <brandur@brandur.org>
This changes allows for each instance of StripeClient to have its own
configuration object instead of relying on the global config. Each
instance can be configured to override any global config values
previously set.
Adds a new instrumentation callback called `request_begin` which, as the
name suggests, is invoked before an HTTP request is dispatched. As
outlined originally in #900, the idea is that this will enable a set of
hooks that can be used for distributed tracing.
The PR also renames the existing `request` callback to `request_end`,
although the old name is still invoked for the time being for backwards
compatibility.
A special `user_data` property is passed to `request_begin` which allows
subscribers to set custom data that will be passed through to
`request_end` for any given request. This allows, for example, a user
assigned ID to be set for the request and recognized on both ends.
I chose the naming `_begin` and `_end` (as opposed to start/finish or
any other combination) based on the naming conventions of Ruby itself.
Fixes#900.
* Add simple instrumentation callback
We used to insert Faraday::Request::Instrumentation into our Faraday middleware
stack to be able to instrument Stripe calls with StatsD. With Faraday being
removed in version 5, this required some rework. This commit implements a simple
callback system that can be used with any kind of instrumentation system.
* Add a topic to Stripe::Instrumentation notifications
... and a :request topic to subscribe to
* Use a RequestEvent value object instead of positional args in callback
This way the RequestLogContext object doesn't get exposed externally. Since the
same value object can be received by multiple subscribers it is frozen to
prevent accidental mutations across threads.
* Relocate tests for instrumentation and add more tests
Drops the use of `Time.now` in favor of using the system's monotonic
clock for various operations that calculate and use elapsed duration.
The latter is preferable because in some cases `Time.now` can be
unstable, like if it's set manually by a system administrator or an NTP
daemon.
I don't expect that the previous code would actually have caused trouble
in the vast majority of normal situations, so I'm not going to backport
anything, but this seems like good hygiene.
For better or worse I had to wrap the monotonic time calls in a new
`Util` function because (1) the normal invocation is long enough to have
caused a lot of overruns on our 80 character line lengths, and (2)
Timecop doesn't stub the monotonic clock, so the `Util` method gives us
a nice place that we can stub on where necessary.
As seen in stripe-node, adds support for the `Stripe-Should-Retry`
header which is sent by the API when it explicitly would like us to
either retry or _not_ retry.
I'll add `Retry-After` separately at some point, but I punted it on it
for now given that we're not using it yet.
See: https://github.com/stripe/stripe-node/pull/692
Introduces a system for garbage collecting connection managers in an
attempt to solve #850.
Previously, the number of connection managers (and by extension the
number of connections that they were holding) would stay stable if a
program used a stable number of threads. However, if threads were used
disposably, the number of active connection managers out there could
continue to grow unchecked, and each of those could be holding one or
more dead connections which are no longer open, but still holding a file
descriptor waiting to be unlinked in disposed of by Ruby's GC.
This PR introduces a connection manager garbage collector that runs
periodically whenever a new connection manager is created. Connection
managers get a timestamp to indicate when they were last used, and the
GC runs through each one and prunes any that haven't seen use within a
certain threshold (currently, 120 seconds). This should have the effect
of removing connection managers as they're not needed anymore, and thus
resolving the socket leakage seen in #850.
I had to make a couple implementation tweaks to get this working
correctly. Namely:
* The `StripeClient` class now tracks thread contexts instead of
connection managers. This is so that when we're disposing of a
connection manager, we can set `default_connection_manager` on its
parent thread context to `nil` so that it's not still tracking a
connection manager that we're trying to get rid of.
* `StripeClient` instances can still be instantiated as before, but no
longer internalize a reference to their own connection manager,
instead falling back to the one in the current thread context. The
rationale is that when trying to dispose of a connection manager, we'd
also have to dispose of its reference in any outstanding
`StripeClient` instances that might still be tracking it, and that
starts to get a little unwieldy. I've left `#connection_manager` in
place for backwards compatibility, but marked it as deprecated.
Tweaks the retry logic so that 429s which have the special status code
lock_timeout are retried automatically.
Similar to what was recently implemented in Go: stripe/stripe-go#935
* Convert library to use built-in `Net::HTTP`
Moves the library off of Faraday and over onto the standard library's
built-in `Net::HTTP` module. The upside of the transition is that we
break away from a few dependencies that have caused us a fair bit of
trouble in the past, the downside is that we need more of our own code
to do things (although surprisingly, not that much more).
The biggest new pieces are:
* `ConnectionManager`: A per-thread class that manages a connection to
each Stripe infrastructure URL (like `api.stripe.com`,
`connect.stripe.com`, etc.) so that we can reuse them between
requests. It's also responsible for setting up and configuring new
`Net::HTTP` connections, which is a little more heavyweight
code-wise compared to other libraries. All of this could have lived in
`StripeClient`, but I extracted it because that class has gotten so
big.
* `MultipartEncoder`: A class that does multipart form encoding for file
uploads. Unfortunately, Ruby doesn't bundle anything like this. I
built this by referencing the Go implementation because the original
RFC is not very detailed or well-written. I also made sure that it was
behaving similarly to our other custom implementations like
stripe-node, and that it can really upload a file outside the test
suite.
There's some risk here in that it's easy to miss something across one of
these big transitions. I've tried to test out various error cases
through tests, but also by leaving scripts running as I terminate my
network connection and bring it back. That said, we'd certainly release
on a major version bump because some of the interface (like setting
`Stripe.default_client`) changes.
* Drop support for old versions of Ruby
Drops support for Ruby 2.1 (EOL March 31, 2017) and 2.2 (EOL March 31,
2018). They're removed from `.travis.yml` and the gemspec and RuboCop
configuration have also been updated to the new lower bound.
Most of the diff here are minor updates to styling as required by
RuboCop:
* String literals are frozen by default, so the `.freeze` we had
everywhere is now considered redundant.
* We can now use Ruby 1.9 style hash syntax with string keys like `{
"foo": "bar" }`.
* Converted a few heredocs over to use squiggly (leading whitespace
removed) syntax.
As discussed in Slack, I didn't drop support for Ruby 2.3 (EOL March 31,
2019) as we still have quite a few users on it. As far as I know
dropping it doesn't get us access to any major syntax improvements or
anything, so it's probably not a big deal.
* Make `CardError`'s `code` parameter named instead of positional (#816)
Makes the `code` parameter on `CardError` named instead of positional.
This makes it more consistent with the rest of the constructor's
parameters and makes instantiating `CardError` from `StripeClient`
cleaner.
This is a minor breaking change so we're aiming to release it for the
next major version of stripe-ruby.
* Bump Rubocop to latest version (#818)
* Ruby minimum version increase followup (#819)
* Remove old deprecated methods (#820)
* Remove all alias for list methods (#823)
* Remove UsageRecord.create method (#826)
* Remove IssuerFraudRecord (#827)
* Add ErrorObject to StripeError exceptions (#811)
* Tweak retry logic to be a little more like stripe-node (#828)
Tweaks the retry logic to be a little more like stripe-node's. In
particular, we also retry under these conditions:
* If we receive a 500 on a non-`POST` request.
* If we receive a 503.
I made it slightly different from stripe-node which checks for a 500
with `>= 500`. I don't really like that -- if we want to retry specific
status codes we should be explicit about it.
We're actively re-examining ways on how to make it easier for clients to
figure out when to retry right now, but I figure V5 is a good time to
tweak this because the modifications change the method signature of
`should_retry?` slightly, and it's technically a public method.
* Fix inverted sign for 500 retries (#830)
I messed up in #828 by (1) accidentally flipping the comparison against
`:post` when checking whether to retry on 500, and (2) forgetting to
write new tests for the condition, which is how (1) got through.
This patch fixes both those problems.
* Remove a few more very old deprecated methods (#831)
I noticed that we had a couple of other deprecated methods on `Stripe`
and `StripeObject` that have been around for a long time. May as well
get rid of them too -- luckily they were using `Gem::Deprecate` so
they've been producing annoying deprecated warnings for quite a while
now.
* Remove extraneous slash at the end of the line
* Reset connections when connection-changing configuration changes (#829)
Adds a few basic features around connection and connection manager
management:
* `clear` on connection manager, which calls `finish` on each active
connection and then disposes of it.
* A centralized cross-thread tracking system for connection managers in
`StripeClient` and `clear_all_connection_managers` which clears all
known connection managers across all threads in a thread-safe way.
The addition of these allow us to modify the implementation of some of
our configuration on `Stripe` so that it can reset all currently open
connections when its value changes.
This fixes a currently problem with the library whereby certain
configuration must be set before the first request or it remains fixed
on any open connections. For example, if `Stripe.proxy` is set after a
request is made from the library, it has no effect because the proxy
must have been set when the connection was originally being initialized.
The impetus for getting this out is that I noticed that we will need
this internally in a few places when we're upgrading to stripe-ruby V5.
Those spots used to be able to hack around the unavailability of this
feature by just accessing the Faraday connection directly and resetting
state on it, but in V5 `StripeClient#conn` is gone, and that's no longer
possible.
* Minor cleanup in `StripeClient` (#832)
I ended up having to relax the maximum method line length in a few
previous PRs, so I wanted to try one more cleanup pass in
`execute_request` to see if I could get it back at all.
The answer was "not by much" (without reducing clarity), but I found a
few places that could be tweaked. Unfortunately, ~50 lines is probably
the "right" length for this method in that you _could_ extract it
further, but you'd end up passing huge amounts of state all over the
place in method parameters, and it really wouldn't look that good.
* Do better bookkeeping when tracking state in `Thread.current` (#833)
This is largely just another cleanup patch, but does a couple main
things:
* Hoists the `last_response` value into thread state. This is a very
minor nicety, but effectively makes `StripeClient` fully thread-safe,
which seems like a minor nicety. Two calls to `#request` to the same
`StripeObject` can now be executed on two different threads and their
results won't interfere with each other.
* Moves state off one-off `Thread.current` keys and into a single one
for the whole client which stores a new simple type of record called
`ThreadContext`. Again, this doesn't change much, but adds some minor
type safety and lets us document each field we expect to have in a
thread's context.
* Add Invoice.list_upcoming_line_items method (#834)
Add space at the end of the second line so the complete message separates the word `not` and `working`
message before
```
Stripe::APIConnectionError: Unexpected error communicating when trying to connect to Stripe. You may be seeing this message because your DNS is notworking. To check, try running `host stripe.com` from the command line.
```
message after
```
Stripe::APIConnectionError: Unexpected error communicating when trying to connect to Stripe. You may be seeing this message because your DNS is not working. To check, try running `host stripe.com` from the command line.
```
* Bump Rubocop to 0.57.2
* Style/StderrPuts: Use warn instead of .puts
* Style/ExpandPathArguments: Use expand_path('../test_helper', __dir__) instead of expand_path('../../test_helper', __FILE__)
* Style/Encoding: Unnecessary utf-8 encoding comment
* Style/StringLiterals: Prefer double-quoted strings
* Style/AccessModifierDeclarations
* Style/FormatStringToken: Prefer annotated tokens
* Naming/UncommunicativeMethodParamName
* Metrics/LineLength: set maximum line length to 100 characters
* Style/IfUnlessModifier: Favor modifier if usage when having a single-line body
* Style/ClassVars
* Metrics/LineLength: set maximum line length to 80 characters (default)
* Style/AccessModifierDeclarations: EnforcedStyle: inline
In #741 I tried to do something too clever by replacing instances of
`Faraday::UploadIO` found in parameters with a human-readable string to
improve `STRIPE_LOG` logging output.
I thought I'd tested it at the time, but apparently not (or not well
enough), and this change caused the regression detailed in #742.
My findings about how Faraday encodes multipart were apparently wrong
and it does use these parameters, so here we remove the step where we
try to nicen them for logging. The logs look a little worse, but it's
not that big of a deal.
I've tested this patch against the API and confirmed that it addresses
the problem.
Fixes#742.
Makes a few tweaks to hopefully simplify clarity things:
* `FaradayStripeEncoder` now becomes the way to encode all of form,
multipart form, and query parameters.
* Introduce a cache in it so that we don't have to encode everything
twice (once for logging, and once for the request body).
* Try to sanitize logging a bit by replacing `Faraday::UploadIO`s found
in incoming parameters with a string representation of the file (note
that all other styles of file input like `File` or `Tempfile` have
been converted to `Faraday::UploadIO` by the time they reach the
encoder).
As reported in #608, integer-indexed maps currently work when passed as
part of the body, but they are reverted to non-indexed maps when passed
in the query.
It turns out that we actually had two problems:
1. We weren't calling our `Util.encode_parameters` on our query
parameters anywhere, and it's this method will does the integer
encoding.
2. Even when I fixed (1) by calling `Util.encode_parameters`, Faraday
would still strip the integer indexes as they were transformed in
its default `NestedParamsEncoder`.
Here we fix both issues by calling `Util.encode_parameters` and sending
Faraday a custom encoder which bypasses its normal shenanigans.
Unfortunately, this has turned out to be somewhat difficult to test
because the integer-indexed maps also seem to confuse Webmock, which
strips them down to standard maps (I even tried testing against a
string, and it still got it wrong). I did use stripe-mock though to
verify that we are now sending the right payload.
Fixes#608.
Tweaks telemetry implementation slightly to be inline with the recent
implementation in stripe-php. Telemetry isn't much good if a request ID
wasn't present, so we only send telemetry if it was.
This changes the library's default connection over to use the adapter
for `Net::HTTP::Persistent`, which is a connection pooling library for
Ruby.
In the long run, I think we should probably just drop Faraday ... the
amount of value it's getting us is extremely tenuous and its API is
difficult to work with. I hate to do it at this point though because
technically people could be writing custom middleware for it.
Adds the magic `frozen_string_literal: true` comment to every file and
enables a Rubocop rule to make sure that it's always going to be there
going forward as well.
See here for more background [1], but the basic idea is that unlike many
other languages, static strings in code are mutable by default. This has
since been acknowledged as not a particularly good idea, and the
intention is to rectify the mistake when Ruby 3 comes out, where all
string literals will be frozen. The `frozen_string_literal` magic
comment was introduced in Ruby 2.3 as a way of easing the transition,
and allows libraries and projects to freeze their literals in advance.
I don't think this is breaking in any way: it's possible that users
might've been pulling out one of are literals somehow and mutating it,
but that would probably not have been useful for anything and would
certainly not be recommended, so I'm quite comfortable pushing this
change through as a minor version.
As discussed in #641.
[1] https://stackoverflow.com/a/37799399
If specifying both query parameters in a path/URL down to Faraday (e.g.,
`/v1/invoices/upcoming?coupon=25OFF`) _and_ query parameters in a hash
(e.g., `{ customer: "cus_123" }`), it will silently overwrite the ones
in the path with the ones in the hash. This can cause problems where
some critical parameters are discarded and causes an error, as seen in
issue #646.
This patch modifies `#execute_request` so that before going out to
Faraday we check whether the incoming path has query parameters. If it
does, we decode them and add them to our `query_params` hash so that
all parameters from either place are preserved.
Fixes#646.
