While these tests test our changes to pg_waldump they are quite easy to
overlook right now and where exactly should we draw the line? These
tests are not something we ever want to upstream and in the future when
we figure out how we want to make sure pg_waldump works with encrypted
WAL we likely will want to have the tests for that solution in the same
folder as our other tests anyway.
This enum was only used in one place and oscured the two dimensions of
provider types (database vs global) and principal keys (server vs
default vs database).
The purpose of the global TDE mode is to run PostgreSQL's normal test
suite but with our extension so running the pg_tde test suite when in
that mode makes no sense.
Meson supports disabling test suites with --no-suite so we only need to
do this for the Makefile.
Instead of giving them numbers we call them pg_tde_ddl_start and
pg_tde_ddl_end. Since the triggers are not on the same event the names
do not matter for the order they are executed in.
HeapTupleIsValid() is actually just a null check but PostgreSQL's
codebase almost always uses this macro and we had a confusion where we
both had a null check and called this macro so we at least should pick
just one of the two ways to write it. And here I picked the most
commonly used way in the PostgreSQL codebase.
The name pg_tde_<OID>_keyring was confusing to users due to making it
sound like it would contain keys. And the name pg_tde_<OID>_map did not
tell a user anything. The new names are <OID>_providers for the key
providers and <OID>_keys for the relation/WAL keys.
While changing the suffixes to be more descriptive I also dropped the
pg_tde_ prefix since it is just noise when they all are in the pg_tde
directory.
Tests to trigger redo routines after the server crash. It mostly checks
invariants when different redo functions might rewrite WAL keys created on the
init stage.
For PG-1539, PG-1541, PG-1468, PG-1413
We create a new WAL key during the extension init, which happens before
the redo. This means that in case of a crash,
pg_tde_save_principal_key_redo was rewriting a WAL _map file and destroying
a newly created key.
Since we emit an XLog record after the key was successfully written to
the file (the file was created), we can safely assume that we should
not change the file if it exists.
Instead of replicating relation keys we generate new ones on replay of
the XLOG_TDE_ADD_RELATION_KEY record at the replica server. This means a
replica and its master server will end up with different sets of
relation keys making a simple binary diff impossible but that is a
dubious advantage since the WAL keys will differ anyway and on on the
flip-side the new code is simpler and easier to reason about. Especially
since now WAL keys and relation keys are treated in a bit more similar
ways.
To prevent duplicate keys in the key file we skip generating and adding
a key if there already is an entry in the file for the same relation.
Make sure we can never generate relation keys on a streaming replica or
in recovery in the SMGR code. Instead the key should always have been
already generated when replaying the XLOG_TDE_ADD_RELATION_KEY record.
This way we can avoid obvious regression when refactoring the code for
replicating keys in future commits. This test can in the future be
expanded to test more interesting cases.
The old code was harder to read than necessary since it had exactly two
callers of which one had each value of the boolean flag. Breaking it up
into two functions makes the intent clearer. While at it we also clean
up the flow a bit more.
This adds some validation to make sure we can access the key provider
when it's created to make the user experience a little nicer. The actual
access validation is very rudimentary for now but can easily be
expanded.
Previously write_key_provider_info() was a bit of a "do everything"
function that had very different behavior depending on what parameters
was passed to it. This commit reworks it to a "dumb" function that just
writes the data without asking questions and have the callers take
responsibility for data validity.
This is to make it easier to validate the data in different ways
depending on the caller's needs without further complicating
write_key_provider_info().
The new name, KeyringProviderRecordInFile, describes what it is rather
than what it's used for. But the real reason is that I want to use it
for other things than the WAL in future commits.
This regression file didn't really test anything. The tests it was
supposed to do was removed here e270322f72
So now it doesn't test anything that key_provider.sql doesn't already do
for us.
Be consistent about always passing curr_pos when reading or writing the
map file. The code is easier to understand if only one variable is used
for positioning in the file.
The fucntions for lsiting keys do not look at inherit_global_providers
but even if they did it does not seem like something which would belong
in the architecture documentation.
Before this commit, we Xlogged the binary result of the _map file
content during key rotation. This led to issues:
1. Replicas would rewrite their own WAL keys with the primary's ones.
And WAL keys are different on replicas. The same would have happened
with SMGR keys since we're also planning to have them different across
replicas.
2. The crash recovery would rewrite the latest WAL key as it's being
created before redo.
This commit switches to rather Xlogging the event of rotation (to which
key should rotate) and lets redo/replicas perform the actual rotation.
Fixes PG-1468, PG-1541
There is a restart function so there is not and need to call first stop
and then start. And since by default a start, stop or restart call does
not return on error it is totally pointless to assert anything about the
return value. And since PostgreSQL's own tests also are fine with just
bailing out on error we do the same.
While at it we also always call these three functions without
parentheses to be consistent.
