Compare commits

...

17 Commits

Author SHA1 Message Date
Amit Kapila
a7db71ed27 Fix uninitialized slot array access during the upgrade.
Commit 29d0a77fa introduced fetching slot information from the old cluster
but didn't initialize the required array in all the code paths. So when
trying to access the array in verbose mode for the new cluster, it leads
to an uninitialized memory access.

Author: Vignesh C
Discussion: http://postgr.es/m/CALDaNm1tntGP5=CtMz=v+k3_PGv7kE9t6iWSgX-QiurAaFkhZw@mail.gmail.com
2023-11-09 09:26:40 +05:30
Michael Paquier
108161bcb9 pg_stat_statements: Remove duplicated tests for SET statements
This looks like a copy-paste mistake introduced in de2aca288569, that
has added checks for more patterns of SET statements while ignoring the
original test block that existed.

Backpatch down to where this has been introduced, as this shaves some
cycles.

Author: Sergei Kornilov
Discussion: https://postgr.es/m/5689421699428803@mail-sendbernar-production-main-46.myt.yp-c.yandex.net
Backpatch-through: 16
2023-11-09 10:04:31 +09:00
Heikki Linnakangas
8f4a1ab471 Fix bug in the new ResourceOwner implementation.
When the hash table is in use, ResoureOwnerSort() moves any elements
from the small fixed-size array to the hash table, and sorts it. When
the hash table is not in use, it sorts the elements in the small
fixed-size array directly. However, ResourceOwnerSort() and
ResourceOwnerReleaseAll() had different idea on when the hash table is
in use: ResourceOwnerSort() checked owner->nhash != 0, and
ResourceOwnerReleaseAll() checked owner->hash != NULL. If the hash
table was allocated but was currently empty, you hit an assertion
failure.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://www.postgresql.org/message-id/be58d565-9e95-d417-4e47-f6bd408dea4b@gmail.com
2023-11-09 01:33:14 +02:00
Bruce Momjian
322f55bdbd doc:: simplify introductory text
Reported-by: Joshua D. Drake

Discussion: https://postgr.es/m/5ac2c96d-37a6-18aa-08c4-327a6fbff24b@commandprompt.com

Author: Joshua D. Drake

Backpatch-through: master
2023-11-08 16:48:43 -05:00
Bruce Momjian
0ccb657a33 README: remove duplicate download link & mention related softw.
Reported-by: Daniel Westermann

Discussion: https://postgr.es/m/DB6PR0902MB2184F7965C9EA9070ACFCA43D2A80@DB6PR0902MB2184.eurprd09.prod.outlook.com

Backpatch-through: master
2023-11-08 16:36:18 -05:00
Bruce Momjian
3a236fc9f3 doc: change "system" to "cluster" where appropriate
Reported-by: Jeff Davis

Discussion: https://postgr.es/m/d040a1144e0127a49e335d1244a4de102a2a443b.camel@j-davis.com

Backpatch-through: master
2023-11-08 16:16:20 -05:00
Bruce Momjian
53015c8afa doc: mention that ANALYZE does block DDL
Reported-by: Aramaki Zyake

Discussion: https://postgr.es/m/156628723253.1296.7377373462603881976%40wrigleys.postgresql.org

Author: Aramaki Zyake

Backpatch-through: master
2023-11-08 16:05:02 -05:00
Alvaro Herrera
b0f7dd915b
Check stack depth in new recursive functions
Commit b0e96f311985 introduced a bunch of recursive functions, but
failed to make them check for stack depth.  This can cause the backend
to crash when operating on inheritance hierarchies several thousands
deep.  Protect the code by adding the missing stack depth checks.

Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/b2ac2392-9727-5f76-e890-721ac80c1615@gmail.com
2023-11-08 18:44:54 +01:00
Tom Lane
76db9cb636 Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements.  (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)

Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example).  Merge those counters into one.

In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types.  It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.

Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk.  This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.

Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2023-11-08 12:01:28 -05:00
Alvaro Herrera
1a5594b957
Call pqPipelineFlush from PQsendFlushRequest
When PQsendFlushRequest() was added by commit 69cf1d5429d4, we argued
against adding a PQflush() call in it[1].  This is still the right
decision: if the user wants a flush to occur, they can just call that.
However, we failed to realize that the message bytes could still be
given to the kernel for transmitting when this can be made without
blocking.  That's what pqPipelineFlush() does, and it is done for every
single other message type sent by libpq, so do that.

(When the socket is in blocking mode this may indeed block, but that's
what all the other libpq message-sending routines do, too.)

[1] https://www.postgresql.org/message-id/202106252352.5ca4byasfun5%40alvherre.pgsql

Author: Jelte Fennema-Nio <postgres@jeltef.nl>
Discussion: https://postgr.es/m/CAGECzQTxZRevRWkKodE-SnJk1Yfm4eKT+8E4Cyq3MJ9YKTnNew@mail.gmail.com
2023-11-08 16:44:08 +01:00
Heikki Linnakangas
cd694f60dc Change pgcrypto to use the new ResourceOwner mechanism.
This is a nice example of how extensions can now use ResourceOwners to
track extension-specific resource kinds

Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/d746cead-a1ef-7efe-fb47-933311e876a3%40iki.fi
2023-11-08 13:30:55 +02:00
Heikki Linnakangas
954e43564d Use a faster hash function in resource owners.
This buys back some of the performance loss that we otherwise saw from the
previous commit.

Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/d746cead-a1ef-7efe-fb47-933311e876a3%40iki.fi
2023-11-08 13:30:52 +02:00
Heikki Linnakangas
b8bff07daa Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.

The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.

All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.

The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.

Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget.  There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.

Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.

There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.

Add tests in src/test/modules/test_resowner.

Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2023-11-08 13:30:50 +02:00
Heikki Linnakangas
b70c2143bb Move a few ResourceOwnerEnlarge() calls for safety and clarity.
These are functions where a lot of things happen between the
ResourceOwnerEnlarge and ResourceOwnerRemember calls. It's important
that there are no unrelated ResourceOwnerRemember calls in the code in
between, otherwise the reserved entry might be used up by the
intervening ResourceOwnerRemember and not be available at the intended
ResourceOwnerRemember call anymore. I don't see any bugs here, but the
longer the code path between the calls is, the harder it is to verify.

In bufmgr.c, there is a function similar to ResourceOwnerEnlarge,
ReservePrivateRefCountEntry(), to ensure that the private refcount
array has enough space. The ReservePrivateRefCountEntry() calls were
made at different places than the ResourceOwnerEnlargeBuffers()
calls. Move the ResourceOwnerEnlargeBuffers() and
ReservePrivateRefCountEntry() calls together for consistency.

Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2023-11-08 13:30:46 +02:00
Peter Eisentraut
e9f075f9a1 Don't install ldap_password_func in meson
It should be handled as a test module per commit b6a0d469ca.
2023-11-08 11:27:28 +01:00
Michael Paquier
b64c8b01c2 Fix use of OPENSSL in SSL tests if command is not found
`openssl` is an optional dependency in the meson build as it may not be
installed in an environment even if SSL libraries are around.  The meson
scripts assume that, but the SSL tests thought that it was a hard
dependency, causing a meson installation to fail if `openssl` could not
be found.  Like similar tests that depend on external commands, and to
be consistent with ./configure for the SSL tests, this commit makes the
command existence optional in the tests.

Author: Tristan Partin
Discussion: https://postgr.es/m/CWSX6P5OUUM5.N7B74KQ06ZP6@neon.tech
Backpatch-through: 16
2023-11-08 17:29:02 +09:00
Michael Paquier
1b2c6b756e Enlarge assertion in bloom_init() for false_positive_rate
false_positive_rate is a parameter that can be set with the bloom
opclass in BRIN, and setting it to a value of exactly 0.25 would trigger
an assertion in the first INSERT done on the index with value set.

The assertion changed here relied on BLOOM_{MIN|MAX}_FALSE_POSITIVE_RATE
that are somewhat arbitrary values, and specifying an out-of-range value
would also trigger a failure when defining such an index.  So, as-is,
the assertion was just doubling on the min-max check of the reloption.
This is now enlarged to check that it is a correct percentage value,
instead, based on a suggestion by Tom Lane.

Author: Alexander Lakhin
Reviewed-by: Tom Lane, Shihao Zhong
Discussion: https://postgr.es/m/17969-a6c54de48026d694@postgresql.org
Backpatch-through: 14
2023-11-08 14:06:26 +09:00
53 changed files with 2738 additions and 1439 deletions

10
README
View File

@ -9,10 +9,6 @@ that supports an extended subset of the SQL standard, including
transactions, foreign keys, subqueries, triggers, user-defined types transactions, foreign keys, subqueries, triggers, user-defined types
and functions. This distribution also contains C language bindings. and functions. This distribution also contains C language bindings.
PostgreSQL has many language interfaces, many of which are listed here:
https://www.postgresql.org/download/
See the file INSTALL for instructions on how to build and install See the file INSTALL for instructions on how to build and install
PostgreSQL. That file also lists supported operating systems and PostgreSQL. That file also lists supported operating systems and
hardware platforms and contains information regarding any other hardware platforms and contains information regarding any other
@ -22,6 +18,6 @@ file COPYRIGHT. A comprehensive documentation set is included in this
distribution; it can be read as described in the installation distribution; it can be read as described in the installation
instructions. instructions.
The latest version of this software may be obtained at The latest version of this software, and related software, may be
https://www.postgresql.org/download/. For more information look at our obtained at https://www.postgresql.org/download/. For more information
web site located at https://www.postgresql.org/. look at our web site located at https://www.postgresql.org/.

View File

@ -67,6 +67,61 @@ SELECT toplevel, calls, query FROM pg_stat_statements
t | 1 | SET pg_stat_statements.track = 'all' t | 1 | SET pg_stat_statements.track = 'all'
(7 rows) (7 rows)
-- DO block - top-level tracking without utility.
SET pg_stat_statements.track = 'top';
SET pg_stat_statements.track_utility = FALSE;
SELECT pg_stat_statements_reset();
pg_stat_statements_reset
--------------------------
(1 row)
DELETE FROM stats_track_tab;
DO $$
BEGIN
DELETE FROM stats_track_tab;
END; $$;
DO LANGUAGE plpgsql $$
BEGIN
-- this is a SELECT
PERFORM 'hello world'::TEXT;
END; $$;
SELECT toplevel, calls, query FROM pg_stat_statements
ORDER BY query COLLATE "C", toplevel;
toplevel | calls | query
----------+-------+-----------------------------------
t | 1 | DELETE FROM stats_track_tab
t | 1 | SELECT pg_stat_statements_reset()
(2 rows)
-- DO block - all-level tracking without utility.
SET pg_stat_statements.track = 'all';
SELECT pg_stat_statements_reset();
pg_stat_statements_reset
--------------------------
(1 row)
DELETE FROM stats_track_tab;
DO $$
BEGIN
DELETE FROM stats_track_tab;
END; $$;
DO LANGUAGE plpgsql $$
BEGIN
-- this is a SELECT
PERFORM 'hello world'::TEXT;
END; $$;
SELECT toplevel, calls, query FROM pg_stat_statements
ORDER BY query COLLATE "C", toplevel;
toplevel | calls | query
----------+-------+-----------------------------------
f | 1 | DELETE FROM stats_track_tab
t | 1 | DELETE FROM stats_track_tab
f | 1 | SELECT $1::TEXT
t | 1 | SELECT pg_stat_statements_reset()
(4 rows)
-- PL/pgSQL function - top-level tracking. -- PL/pgSQL function - top-level tracking.
SET pg_stat_statements.track = 'top'; SET pg_stat_statements.track = 'top';
SET pg_stat_statements.track_utility = FALSE; SET pg_stat_statements.track_utility = FALSE;
@ -118,6 +173,31 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 1 | SELECT pg_stat_statements_reset() 1 | 1 | SELECT pg_stat_statements_reset()
(3 rows) (3 rows)
-- immutable SQL function --- can be executed at plan time
CREATE FUNCTION PLUS_THREE(i INTEGER) RETURNS INTEGER AS
$$ SELECT i + 3 LIMIT 1 $$ IMMUTABLE LANGUAGE SQL;
SELECT PLUS_THREE(8);
plus_three
------------
11
(1 row)
SELECT PLUS_THREE(10);
plus_three
------------
13
(1 row)
SELECT toplevel, calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
toplevel | calls | rows | query
----------+-------+------+------------------------------------------------------------------------------
t | 2 | 2 | SELECT PLUS_ONE($1)
t | 2 | 2 | SELECT PLUS_THREE($1)
t | 2 | 2 | SELECT PLUS_TWO($1)
t | 1 | 3 | SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C"
t | 1 | 1 | SELECT pg_stat_statements_reset()
(5 rows)
-- PL/pgSQL function - all-level tracking. -- PL/pgSQL function - all-level tracking.
SET pg_stat_statements.track = 'all'; SET pg_stat_statements.track = 'all';
SELECT pg_stat_statements_reset(); SELECT pg_stat_statements_reset();
@ -129,6 +209,7 @@ SELECT pg_stat_statements_reset();
-- we drop and recreate the functions to avoid any caching funnies -- we drop and recreate the functions to avoid any caching funnies
DROP FUNCTION PLUS_ONE(INTEGER); DROP FUNCTION PLUS_ONE(INTEGER);
DROP FUNCTION PLUS_TWO(INTEGER); DROP FUNCTION PLUS_TWO(INTEGER);
DROP FUNCTION PLUS_THREE(INTEGER);
-- PL/pgSQL function -- PL/pgSQL function
CREATE FUNCTION PLUS_TWO(i INTEGER) RETURNS INTEGER AS $$ CREATE FUNCTION PLUS_TWO(i INTEGER) RETURNS INTEGER AS $$
DECLARE DECLARE
@ -174,7 +255,34 @@ SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
1 | 1 | SELECT pg_stat_statements_reset() 1 | 1 | SELECT pg_stat_statements_reset()
(5 rows) (5 rows)
DROP FUNCTION PLUS_ONE(INTEGER); -- immutable SQL function --- can be executed at plan time
CREATE FUNCTION PLUS_THREE(i INTEGER) RETURNS INTEGER AS
$$ SELECT i + 3 LIMIT 1 $$ IMMUTABLE LANGUAGE SQL;
SELECT PLUS_THREE(8);
plus_three
------------
11
(1 row)
SELECT PLUS_THREE(10);
plus_three
------------
13
(1 row)
SELECT toplevel, calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
toplevel | calls | rows | query
----------+-------+------+------------------------------------------------------------------------------
f | 2 | 2 | SELECT (i + $2 + $3)::INTEGER
f | 2 | 2 | SELECT (i + $2)::INTEGER LIMIT $3
t | 2 | 2 | SELECT PLUS_ONE($1)
t | 2 | 2 | SELECT PLUS_THREE($1)
t | 2 | 2 | SELECT PLUS_TWO($1)
t | 1 | 5 | SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C"
f | 2 | 2 | SELECT i + $2 LIMIT $3
t | 1 | 1 | SELECT pg_stat_statements_reset()
(8 rows)
-- --
-- pg_stat_statements.track = none -- pg_stat_statements.track = none
-- --

View File

@ -664,30 +664,3 @@ SELECT pg_stat_statements_reset();
(1 row) (1 row)
-- SET statements.
-- These use two different strings, still they count as one entry.
SET work_mem = '1MB';
Set work_mem = '1MB';
SET work_mem = '2MB';
RESET work_mem;
SET enable_seqscan = off;
SET enable_seqscan = on;
RESET enable_seqscan;
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
calls | rows | query
-------+------+-----------------------------------
1 | 0 | RESET enable_seqscan
1 | 0 | RESET work_mem
1 | 1 | SELECT pg_stat_statements_reset()
1 | 0 | SET enable_seqscan = off
1 | 0 | SET enable_seqscan = on
2 | 0 | SET work_mem = '1MB'
1 | 0 | SET work_mem = '2MB'
(7 rows)
SELECT pg_stat_statements_reset();
pg_stat_statements_reset
--------------------------
(1 row)

View File

@ -99,13 +99,6 @@ static const uint32 PGSS_PG_MAJOR_VERSION = PG_VERSION_NUM / 100;
#define USAGE_DEALLOC_PERCENT 5 /* free this % of entries at once */ #define USAGE_DEALLOC_PERCENT 5 /* free this % of entries at once */
#define IS_STICKY(c) ((c.calls[PGSS_PLAN] + c.calls[PGSS_EXEC]) == 0) #define IS_STICKY(c) ((c.calls[PGSS_PLAN] + c.calls[PGSS_EXEC]) == 0)
/*
* Utility statements that pgss_ProcessUtility and pgss_post_parse_analyze
* ignores.
*/
#define PGSS_HANDLED_UTILITY(n) (!IsA(n, ExecuteStmt) && \
!IsA(n, PrepareStmt))
/* /*
* Extension version number, for supporting older extension versions' objects * Extension version number, for supporting older extension versions' objects
*/ */
@ -255,11 +248,8 @@ typedef struct pgssSharedState
/*---- Local variables ----*/ /*---- Local variables ----*/
/* Current nesting depth of ExecutorRun+ProcessUtility calls */ /* Current nesting depth of planner/ExecutorRun/ProcessUtility calls */
static int exec_nested_level = 0; static int nesting_level = 0;
/* Current nesting depth of planner calls */
static int plan_nested_level = 0;
/* Saved hook values in case of unload */ /* Saved hook values in case of unload */
static shmem_request_hook_type prev_shmem_request_hook = NULL; static shmem_request_hook_type prev_shmem_request_hook = NULL;
@ -836,16 +826,18 @@ pgss_post_parse_analyze(ParseState *pstate, Query *query, JumbleState *jstate)
prev_post_parse_analyze_hook(pstate, query, jstate); prev_post_parse_analyze_hook(pstate, query, jstate);
/* Safety check... */ /* Safety check... */
if (!pgss || !pgss_hash || !pgss_enabled(exec_nested_level)) if (!pgss || !pgss_hash || !pgss_enabled(nesting_level))
return; return;
/* /*
* Clear queryId for prepared statements related utility, as those will * If it's EXECUTE, clear the queryId so that stats will accumulate for
* inherit from the underlying statement's one. * the underlying PREPARE. But don't do this if we're not tracking
* utility statements, to avoid messing up another extension that might be
* tracking them.
*/ */
if (query->utilityStmt) if (query->utilityStmt)
{ {
if (pgss_track_utility && !PGSS_HANDLED_UTILITY(query->utilityStmt)) if (pgss_track_utility && IsA(query->utilityStmt, ExecuteStmt))
{ {
query->queryId = UINT64CONST(0); query->queryId = UINT64CONST(0);
return; return;
@ -897,7 +889,7 @@ pgss_planner(Query *parse,
* So testing the planner nesting level only is not enough to detect real * So testing the planner nesting level only is not enough to detect real
* top level planner call. * top level planner call.
*/ */
if (pgss_enabled(plan_nested_level + exec_nested_level) if (pgss_enabled(nesting_level)
&& pgss_track_planning && query_string && pgss_track_planning && query_string
&& parse->queryId != UINT64CONST(0)) && parse->queryId != UINT64CONST(0))
{ {
@ -918,7 +910,7 @@ pgss_planner(Query *parse,
walusage_start = pgWalUsage; walusage_start = pgWalUsage;
INSTR_TIME_SET_CURRENT(start); INSTR_TIME_SET_CURRENT(start);
plan_nested_level++; nesting_level++;
PG_TRY(); PG_TRY();
{ {
if (prev_planner_hook) if (prev_planner_hook)
@ -930,7 +922,7 @@ pgss_planner(Query *parse,
} }
PG_FINALLY(); PG_FINALLY();
{ {
plan_nested_level--; nesting_level--;
} }
PG_END_TRY(); PG_END_TRY();
@ -959,12 +951,26 @@ pgss_planner(Query *parse,
} }
else else
{ {
if (prev_planner_hook) /*
result = prev_planner_hook(parse, query_string, cursorOptions, * Even though we're not tracking plan time for this statement, we
boundParams); * must still increment the nesting level, to ensure that functions
else * evaluated during planning are not seen as top-level calls.
result = standard_planner(parse, query_string, cursorOptions, */
boundParams); nesting_level++;
PG_TRY();
{
if (prev_planner_hook)
result = prev_planner_hook(parse, query_string, cursorOptions,
boundParams);
else
result = standard_planner(parse, query_string, cursorOptions,
boundParams);
}
PG_FINALLY();
{
nesting_level--;
}
PG_END_TRY();
} }
return result; return result;
@ -986,7 +992,7 @@ pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
* counting of optimizable statements that are directly contained in * counting of optimizable statements that are directly contained in
* utility statements. * utility statements.
*/ */
if (pgss_enabled(exec_nested_level) && queryDesc->plannedstmt->queryId != UINT64CONST(0)) if (pgss_enabled(nesting_level) && queryDesc->plannedstmt->queryId != UINT64CONST(0))
{ {
/* /*
* Set up to track total elapsed time in ExecutorRun. Make sure the * Set up to track total elapsed time in ExecutorRun. Make sure the
@ -1011,7 +1017,7 @@ static void
pgss_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count, pgss_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count,
bool execute_once) bool execute_once)
{ {
exec_nested_level++; nesting_level++;
PG_TRY(); PG_TRY();
{ {
if (prev_ExecutorRun) if (prev_ExecutorRun)
@ -1021,7 +1027,7 @@ pgss_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count,
} }
PG_FINALLY(); PG_FINALLY();
{ {
exec_nested_level--; nesting_level--;
} }
PG_END_TRY(); PG_END_TRY();
} }
@ -1032,7 +1038,7 @@ pgss_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count,
static void static void
pgss_ExecutorFinish(QueryDesc *queryDesc) pgss_ExecutorFinish(QueryDesc *queryDesc)
{ {
exec_nested_level++; nesting_level++;
PG_TRY(); PG_TRY();
{ {
if (prev_ExecutorFinish) if (prev_ExecutorFinish)
@ -1042,7 +1048,7 @@ pgss_ExecutorFinish(QueryDesc *queryDesc)
} }
PG_FINALLY(); PG_FINALLY();
{ {
exec_nested_level--; nesting_level--;
} }
PG_END_TRY(); PG_END_TRY();
} }
@ -1056,7 +1062,7 @@ pgss_ExecutorEnd(QueryDesc *queryDesc)
uint64 queryId = queryDesc->plannedstmt->queryId; uint64 queryId = queryDesc->plannedstmt->queryId;
if (queryId != UINT64CONST(0) && queryDesc->totaltime && if (queryId != UINT64CONST(0) && queryDesc->totaltime &&
pgss_enabled(exec_nested_level)) pgss_enabled(nesting_level))
{ {
/* /*
* Make sure stats accumulation is done. (Note: it's okay if several * Make sure stats accumulation is done. (Note: it's okay if several
@ -1097,6 +1103,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
uint64 saved_queryId = pstmt->queryId; uint64 saved_queryId = pstmt->queryId;
int saved_stmt_location = pstmt->stmt_location; int saved_stmt_location = pstmt->stmt_location;
int saved_stmt_len = pstmt->stmt_len; int saved_stmt_len = pstmt->stmt_len;
bool enabled = pgss_track_utility && pgss_enabled(nesting_level);
/* /*
* Force utility statements to get queryId zero. We do this even in cases * Force utility statements to get queryId zero. We do this even in cases
@ -1112,7 +1119,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
* that user configured another extension to handle utility statements * that user configured another extension to handle utility statements
* only. * only.
*/ */
if (pgss_enabled(exec_nested_level) && pgss_track_utility) if (enabled)
pstmt->queryId = UINT64CONST(0); pstmt->queryId = UINT64CONST(0);
/* /*
@ -1124,11 +1131,13 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
* hash table entry for the PREPARE (with hash calculated from the query * hash table entry for the PREPARE (with hash calculated from the query
* string), and then a different one with the same query string (but hash * string), and then a different one with the same query string (but hash
* calculated from the query tree) would be used to accumulate costs of * calculated from the query tree) would be used to accumulate costs of
* ensuing EXECUTEs. This would be confusing, and inconsistent with other * ensuing EXECUTEs. This would be confusing. Since PREPARE doesn't
* cases where planning time is not included at all. * actually run the planner (only parse+rewrite), its costs are generally
* pretty negligible and it seems okay to just ignore it.
*/ */
if (pgss_track_utility && pgss_enabled(exec_nested_level) && if (enabled &&
PGSS_HANDLED_UTILITY(parsetree)) !IsA(parsetree, ExecuteStmt) &&
!IsA(parsetree, PrepareStmt))
{ {
instr_time start; instr_time start;
instr_time duration; instr_time duration;
@ -1142,7 +1151,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
walusage_start = pgWalUsage; walusage_start = pgWalUsage;
INSTR_TIME_SET_CURRENT(start); INSTR_TIME_SET_CURRENT(start);
exec_nested_level++; nesting_level++;
PG_TRY(); PG_TRY();
{ {
if (prev_ProcessUtility) if (prev_ProcessUtility)
@ -1156,7 +1165,7 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
} }
PG_FINALLY(); PG_FINALLY();
{ {
exec_nested_level--; nesting_level--;
} }
PG_END_TRY(); PG_END_TRY();
@ -1206,14 +1215,41 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
} }
else else
{ {
if (prev_ProcessUtility) /*
prev_ProcessUtility(pstmt, queryString, readOnlyTree, * Even though we're not tracking execution time for this statement,
context, params, queryEnv, * we must still increment the nesting level, to ensure that functions
dest, qc); * evaluated within it are not seen as top-level calls. But don't do
else * so for EXECUTE; that way, when control reaches pgss_planner or
standard_ProcessUtility(pstmt, queryString, readOnlyTree, * pgss_ExecutorStart, we will treat the costs as top-level if
* appropriate. Likewise, don't bump for PREPARE, so that parse
* analysis will treat the statement as top-level if appropriate.
*
* To be absolutely certain we don't mess up the nesting level,
* evaluate the bump_level condition just once.
*/
bool bump_level =
!IsA(parsetree, ExecuteStmt) &&
!IsA(parsetree, PrepareStmt);
if (bump_level)
nesting_level++;
PG_TRY();
{
if (prev_ProcessUtility)
prev_ProcessUtility(pstmt, queryString, readOnlyTree,
context, params, queryEnv, context, params, queryEnv,
dest, qc); dest, qc);
else
standard_ProcessUtility(pstmt, queryString, readOnlyTree,
context, params, queryEnv,
dest, qc);
}
PG_FINALLY();
{
if (bump_level)
nesting_level--;
}
PG_END_TRY();
} }
} }
@ -1271,7 +1307,7 @@ pgss_store(const char *query, uint64 queryId,
key.userid = GetUserId(); key.userid = GetUserId();
key.dbid = MyDatabaseId; key.dbid = MyDatabaseId;
key.queryid = queryId; key.queryid = queryId;
key.toplevel = (exec_nested_level == 0); key.toplevel = (nesting_level == 0);
/* Lookup the hash table entry with shared lock. */ /* Lookup the hash table entry with shared lock. */
LWLockAcquire(pgss->lock, LW_SHARED); LWLockAcquire(pgss->lock, LW_SHARED);

View File

@ -33,6 +33,39 @@ END; $$;
SELECT toplevel, calls, query FROM pg_stat_statements SELECT toplevel, calls, query FROM pg_stat_statements
ORDER BY query COLLATE "C", toplevel; ORDER BY query COLLATE "C", toplevel;
-- DO block - top-level tracking without utility.
SET pg_stat_statements.track = 'top';
SET pg_stat_statements.track_utility = FALSE;
SELECT pg_stat_statements_reset();
DELETE FROM stats_track_tab;
DO $$
BEGIN
DELETE FROM stats_track_tab;
END; $$;
DO LANGUAGE plpgsql $$
BEGIN
-- this is a SELECT
PERFORM 'hello world'::TEXT;
END; $$;
SELECT toplevel, calls, query FROM pg_stat_statements
ORDER BY query COLLATE "C", toplevel;
-- DO block - all-level tracking without utility.
SET pg_stat_statements.track = 'all';
SELECT pg_stat_statements_reset();
DELETE FROM stats_track_tab;
DO $$
BEGIN
DELETE FROM stats_track_tab;
END; $$;
DO LANGUAGE plpgsql $$
BEGIN
-- this is a SELECT
PERFORM 'hello world'::TEXT;
END; $$;
SELECT toplevel, calls, query FROM pg_stat_statements
ORDER BY query COLLATE "C", toplevel;
-- PL/pgSQL function - top-level tracking. -- PL/pgSQL function - top-level tracking.
SET pg_stat_statements.track = 'top'; SET pg_stat_statements.track = 'top';
SET pg_stat_statements.track_utility = FALSE; SET pg_stat_statements.track_utility = FALSE;
@ -57,6 +90,15 @@ SELECT PLUS_ONE(10);
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C"; SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
-- immutable SQL function --- can be executed at plan time
CREATE FUNCTION PLUS_THREE(i INTEGER) RETURNS INTEGER AS
$$ SELECT i + 3 LIMIT 1 $$ IMMUTABLE LANGUAGE SQL;
SELECT PLUS_THREE(8);
SELECT PLUS_THREE(10);
SELECT toplevel, calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
-- PL/pgSQL function - all-level tracking. -- PL/pgSQL function - all-level tracking.
SET pg_stat_statements.track = 'all'; SET pg_stat_statements.track = 'all';
SELECT pg_stat_statements_reset(); SELECT pg_stat_statements_reset();
@ -64,6 +106,7 @@ SELECT pg_stat_statements_reset();
-- we drop and recreate the functions to avoid any caching funnies -- we drop and recreate the functions to avoid any caching funnies
DROP FUNCTION PLUS_ONE(INTEGER); DROP FUNCTION PLUS_ONE(INTEGER);
DROP FUNCTION PLUS_TWO(INTEGER); DROP FUNCTION PLUS_TWO(INTEGER);
DROP FUNCTION PLUS_THREE(INTEGER);
-- PL/pgSQL function -- PL/pgSQL function
CREATE FUNCTION PLUS_TWO(i INTEGER) RETURNS INTEGER AS $$ CREATE FUNCTION PLUS_TWO(i INTEGER) RETURNS INTEGER AS $$
@ -85,7 +128,15 @@ SELECT PLUS_ONE(3);
SELECT PLUS_ONE(1); SELECT PLUS_ONE(1);
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C"; SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
DROP FUNCTION PLUS_ONE(INTEGER);
-- immutable SQL function --- can be executed at plan time
CREATE FUNCTION PLUS_THREE(i INTEGER) RETURNS INTEGER AS
$$ SELECT i + 3 LIMIT 1 $$ IMMUTABLE LANGUAGE SQL;
SELECT PLUS_THREE(8);
SELECT PLUS_THREE(10);
SELECT toplevel, calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
-- --
-- pg_stat_statements.track = none -- pg_stat_statements.track = none

View File

@ -329,16 +329,3 @@ DROP TABLE pgss_ctas;
DROP TABLE pgss_select_into; DROP TABLE pgss_select_into;
SELECT pg_stat_statements_reset(); SELECT pg_stat_statements_reset();
-- SET statements.
-- These use two different strings, still they count as one entry.
SET work_mem = '1MB';
Set work_mem = '1MB';
SET work_mem = '2MB';
RESET work_mem;
SET enable_seqscan = off;
SET enable_seqscan = on;
RESET enable_seqscan;
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
SELECT pg_stat_statements_reset();

View File

@ -50,9 +50,8 @@
*/ */
/* /*
* To make sure we don't leak OpenSSL handles on abort, we keep OSSLDigest * To make sure we don't leak OpenSSL handles, we use the ResourceOwner
* objects in a linked list, allocated in TopMemoryContext. We use the * mechanism to free them on abort.
* ResourceOwner mechanism to free them on abort.
*/ */
typedef struct OSSLDigest typedef struct OSSLDigest
{ {
@ -60,56 +59,41 @@ typedef struct OSSLDigest
EVP_MD_CTX *ctx; EVP_MD_CTX *ctx;
ResourceOwner owner; ResourceOwner owner;
struct OSSLDigest *next;
struct OSSLDigest *prev;
} OSSLDigest; } OSSLDigest;
static OSSLDigest *open_digests = NULL; /* ResourceOwner callbacks to hold OpenSSL digest handles */
static bool digest_resowner_callback_registered = false; static void ResOwnerReleaseOSSLDigest(Datum res);
static const ResourceOwnerDesc ossldigest_resowner_desc =
{
.name = "pgcrypto OpenSSL digest handle",
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
.release_priority = RELEASE_PRIO_FIRST,
.ReleaseResource = ResOwnerReleaseOSSLDigest,
.DebugPrint = NULL, /* default message is fine */
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberOSSLDigest(ResourceOwner owner, OSSLDigest *digest)
{
ResourceOwnerRemember(owner, PointerGetDatum(digest), &ossldigest_resowner_desc);
}
static inline void
ResourceOwnerForgetOSSLDigest(ResourceOwner owner, OSSLDigest *digest)
{
ResourceOwnerForget(owner, PointerGetDatum(digest), &ossldigest_resowner_desc);
}
static void static void
free_openssl_digest(OSSLDigest *digest) free_openssl_digest(OSSLDigest *digest)
{ {
EVP_MD_CTX_destroy(digest->ctx); EVP_MD_CTX_destroy(digest->ctx);
if (digest->prev) if (digest->owner != NULL)
digest->prev->next = digest->next; ResourceOwnerForgetOSSLDigest(digest->owner, digest);
else
open_digests = digest->next;
if (digest->next)
digest->next->prev = digest->prev;
pfree(digest); pfree(digest);
} }
/*
* Close any open OpenSSL handles on abort.
*/
static void
digest_free_callback(ResourceReleasePhase phase,
bool isCommit,
bool isTopLevel,
void *arg)
{
OSSLDigest *curr;
OSSLDigest *next;
if (phase != RESOURCE_RELEASE_AFTER_LOCKS)
return;
next = open_digests;
while (next)
{
curr = next;
next = curr->next;
if (curr->owner == CurrentResourceOwner)
{
if (isCommit)
elog(WARNING, "pgcrypto digest reference leak: digest %p still referenced", curr);
free_openssl_digest(curr);
}
}
}
static unsigned static unsigned
digest_result_size(PX_MD *h) digest_result_size(PX_MD *h)
{ {
@ -188,16 +172,12 @@ px_find_digest(const char *name, PX_MD **res)
OpenSSL_add_all_algorithms(); OpenSSL_add_all_algorithms();
} }
if (!digest_resowner_callback_registered)
{
RegisterResourceReleaseCallback(digest_free_callback, NULL);
digest_resowner_callback_registered = true;
}
md = EVP_get_digestbyname(name); md = EVP_get_digestbyname(name);
if (md == NULL) if (md == NULL)
return PXE_NO_HASH; return PXE_NO_HASH;
ResourceOwnerEnlarge(CurrentResourceOwner);
/* /*
* Create an OSSLDigest object, an OpenSSL MD object, and a PX_MD object. * Create an OSSLDigest object, an OpenSSL MD object, and a PX_MD object.
* The order is crucial, to make sure we don't leak anything on * The order is crucial, to make sure we don't leak anything on
@ -221,9 +201,7 @@ px_find_digest(const char *name, PX_MD **res)
digest->algo = md; digest->algo = md;
digest->ctx = ctx; digest->ctx = ctx;
digest->owner = CurrentResourceOwner; digest->owner = CurrentResourceOwner;
digest->next = open_digests; ResourceOwnerRememberOSSLDigest(digest->owner, digest);
digest->prev = NULL;
open_digests = digest;
/* The PX_MD object is allocated in the current memory context. */ /* The PX_MD object is allocated in the current memory context. */
h = palloc(sizeof(*h)); h = palloc(sizeof(*h));
@ -239,6 +217,17 @@ px_find_digest(const char *name, PX_MD **res)
return 0; return 0;
} }
/* ResourceOwner callbacks for OSSLDigest */
static void
ResOwnerReleaseOSSLDigest(Datum res)
{
OSSLDigest *digest = (OSSLDigest *) DatumGetPointer(res);
digest->owner = NULL;
free_openssl_digest(digest);
}
/* /*
* Ciphers * Ciphers
* *
@ -266,9 +255,8 @@ struct ossl_cipher
* OSSLCipher contains the state for using a cipher. A separate OSSLCipher * OSSLCipher contains the state for using a cipher. A separate OSSLCipher
* object is allocated in each px_find_cipher() call. * object is allocated in each px_find_cipher() call.
* *
* To make sure we don't leak OpenSSL handles on abort, we keep OSSLCipher * To make sure we don't leak OpenSSL handles, we use the ResourceOwner
* objects in a linked list, allocated in TopMemoryContext. We use the * mechanism to free them on abort.
* ResourceOwner mechanism to free them on abort.
*/ */
typedef struct OSSLCipher typedef struct OSSLCipher
{ {
@ -281,56 +269,41 @@ typedef struct OSSLCipher
const struct ossl_cipher *ciph; const struct ossl_cipher *ciph;
ResourceOwner owner; ResourceOwner owner;
struct OSSLCipher *next;
struct OSSLCipher *prev;
} OSSLCipher; } OSSLCipher;
static OSSLCipher *open_ciphers = NULL; /* ResourceOwner callbacks to hold OpenSSL cipher state */
static bool cipher_resowner_callback_registered = false; static void ResOwnerReleaseOSSLCipher(Datum res);
static const ResourceOwnerDesc osslcipher_resowner_desc =
{
.name = "pgcrypto OpenSSL cipher handle",
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
.release_priority = RELEASE_PRIO_FIRST,
.ReleaseResource = ResOwnerReleaseOSSLCipher,
.DebugPrint = NULL, /* default message is fine */
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberOSSLCipher(ResourceOwner owner, OSSLCipher *od)
{
ResourceOwnerRemember(owner, PointerGetDatum(od), &osslcipher_resowner_desc);
}
static inline void
ResourceOwnerForgetOSSLCipher(ResourceOwner owner, OSSLCipher *od)
{
ResourceOwnerForget(owner, PointerGetDatum(od), &osslcipher_resowner_desc);
}
static void static void
free_openssl_cipher(OSSLCipher *od) free_openssl_cipher(OSSLCipher *od)
{ {
EVP_CIPHER_CTX_free(od->evp_ctx); EVP_CIPHER_CTX_free(od->evp_ctx);
if (od->prev) if (od->owner != NULL)
od->prev->next = od->next; ResourceOwnerForgetOSSLCipher(od->owner, od);
else
open_ciphers = od->next;
if (od->next)
od->next->prev = od->prev;
pfree(od); pfree(od);
} }
/*
* Close any open OpenSSL cipher handles on abort.
*/
static void
cipher_free_callback(ResourceReleasePhase phase,
bool isCommit,
bool isTopLevel,
void *arg)
{
OSSLCipher *curr;
OSSLCipher *next;
if (phase != RESOURCE_RELEASE_AFTER_LOCKS)
return;
next = open_ciphers;
while (next)
{
curr = next;
next = curr->next;
if (curr->owner == CurrentResourceOwner)
{
if (isCommit)
elog(WARNING, "pgcrypto cipher reference leak: cipher %p still referenced", curr);
free_openssl_cipher(curr);
}
}
}
/* Common routines for all algorithms */ /* Common routines for all algorithms */
static unsigned static unsigned
@ -782,11 +755,7 @@ px_find_cipher(const char *name, PX_Cipher **res)
if (i->name == NULL) if (i->name == NULL)
return PXE_NO_CIPHER; return PXE_NO_CIPHER;
if (!cipher_resowner_callback_registered) ResourceOwnerEnlarge(CurrentResourceOwner);
{
RegisterResourceReleaseCallback(cipher_free_callback, NULL);
cipher_resowner_callback_registered = true;
}
/* /*
* Create an OSSLCipher object, an EVP_CIPHER_CTX object and a PX_Cipher. * Create an OSSLCipher object, an EVP_CIPHER_CTX object and a PX_Cipher.
@ -806,9 +775,7 @@ px_find_cipher(const char *name, PX_Cipher **res)
od->evp_ctx = ctx; od->evp_ctx = ctx;
od->owner = CurrentResourceOwner; od->owner = CurrentResourceOwner;
od->next = open_ciphers; ResourceOwnerRememberOSSLCipher(od->owner, od);
od->prev = NULL;
open_ciphers = od;
if (i->ciph->cipher_func) if (i->ciph->cipher_func)
od->evp_ciph = i->ciph->cipher_func(); od->evp_ciph = i->ciph->cipher_func();
@ -827,3 +794,11 @@ px_find_cipher(const char *name, PX_Cipher **res)
*res = c; *res = c;
return 0; return 0;
} }
/* ResourceOwner callbacks for OSSLCipher */
static void
ResOwnerReleaseOSSLCipher(Datum res)
{
free_openssl_cipher((OSSLCipher *) DatumGetPointer(res));
}

View File

@ -2572,7 +2572,7 @@ include_dir 'conf.d'
</term> </term>
<listitem> <listitem>
<para> <para>
Sets the maximum number of background processes that the system Sets the maximum number of background processes that the cluster
can support. This parameter can only be set at server start. The can support. This parameter can only be set at server start. The
default is 8. default is 8.
</para> </para>
@ -2680,7 +2680,7 @@ include_dir 'conf.d'
</term> </term>
<listitem> <listitem>
<para> <para>
Sets the maximum number of workers that the system can support for Sets the maximum number of workers that the cluster can support for
parallel operations. The default value is 8. When increasing or parallel operations. The default value is 8. When increasing or
decreasing this value, consider also adjusting decreasing this value, consider also adjusting
<xref linkend="guc-max-parallel-maintenance-workers"/> and <xref linkend="guc-max-parallel-maintenance-workers"/> and

View File

@ -38,24 +38,22 @@ break is not needed in a wider output rendering.
<partintro> <partintro>
<para> <para>
Welcome to the <productname>PostgreSQL</productname> Tutorial. The Welcome to the <productname>PostgreSQL</productname> Tutorial. The
following few chapters are intended to give a simple introduction tutorial is intended to give an introduction
to <productname>PostgreSQL</productname>, relational database to <productname>PostgreSQL</productname>, relational database
concepts, and the SQL language to those who are new to any one of concepts, and the SQL language. We assume some general knowledge about
these aspects. We only assume some general knowledge about how to how to use computers and no particular Unix or programming experience is
use computers. No particular Unix or programming experience is required. This tutorial is intended to provide hands-on experience with
required. This part is mainly intended to give you some hands-on important aspects of the <productname>PostgreSQL</productname> system.
experience with important aspects of the It makes no attempt to be a comprehensive treatment of the topics it covers.
<productname>PostgreSQL</productname> system. It makes no attempt
to be a complete or thorough treatment of the topics it covers.
</para> </para>
<para> <para>
After you have worked through this tutorial you might want to move After you have successfully completed this tutorial you will want to
on to reading <xref linkend="sql"/> to gain a more formal knowledge read the <xref linkend="sql"/> section to gain a better understanding
of the SQL language, or <xref linkend="client-interfaces"/> for of the SQL language, or <xref linkend="client-interfaces"/> for
information about developing applications for information about developing applications with
<productname>PostgreSQL</productname>. Those who set up and <productname>PostgreSQL</productname>. Those who provision and
manage their own server should also read <xref linkend="admin"/>. manage their own PostgreSQL installation should also read <xref linkend="admin"/>.
</para> </para>
</partintro> </partintro>
@ -73,28 +71,26 @@ break is not needed in a wider output rendering.
This part describes the use of the <acronym>SQL</acronym> language This part describes the use of the <acronym>SQL</acronym> language
in <productname>PostgreSQL</productname>. We start with in <productname>PostgreSQL</productname>. We start with
describing the general syntax of <acronym>SQL</acronym>, then describing the general syntax of <acronym>SQL</acronym>, then
explain how to create the structures to hold data, how to populate how to create tables, how to populate the database, and how to
the database, and how to query it. The middle part lists the query it. The middle part lists the available data types and
available data types and functions for use in functions for use in <acronym>SQL</acronym> commands. Lastly,
<acronym>SQL</acronym> commands. The rest treats several we address several aspects of importance for tuning a database.
aspects that are important for tuning a database for optimal
performance.
</para> </para>
<para> <para>
The information in this part is arranged so that a novice user can The information is arranged so that a novice user can
follow it start to end to gain a full understanding of the topics follow it from start to end and gain a full understanding of the topics
without having to refer forward too many times. The chapters are without having to refer forward too many times. The chapters are
intended to be self-contained, so that advanced users can read the intended to be self-contained, so that advanced users can read the
chapters individually as they choose. The information in this chapters individually as they choose. The information is presented
part is presented in a narrative fashion in topical units. in narrative form with topical units. Readers looking for a complete
Readers looking for a complete description of a particular command description of a particular command are encouraged to review
should see <xref linkend="reference"/>. the <xref linkend="reference"/>.
</para> </para>
<para> <para>
Readers of this part should know how to connect to a Readers should know how to connect to a
<productname>PostgreSQL</productname> database and issue <productname>PostgreSQL</productname> database and issue
<acronym>SQL</acronym> commands. Readers that are unfamiliar with <acronym>SQL</acronym> commands. Readers that are unfamiliar with
these issues are encouraged to read <xref linkend="tutorial"/> these issues are encouraged to read <xref linkend="tutorial"/>
first. <acronym>SQL</acronym> commands are typically entered first. <acronym>SQL</acronym> commands are typically entered
@ -125,32 +121,32 @@ break is not needed in a wider output rendering.
<partintro> <partintro>
<para> <para>
This part covers topics that are of interest to a This part covers topics that are of interest to a
<productname>PostgreSQL</productname> database administrator. This includes <productname>PostgreSQL</productname> administrator. This includes
installation of the software, set up and configuration of the installation, configuration of the server, management of users
server, management of users and databases, and maintenance tasks. and databases, and maintenance tasks. Anyone running
Anyone who runs a <productname>PostgreSQL</productname> server, even for <productname>PostgreSQL</productname> server, even for
personal use, but especially in production, should be familiar personal use, but especially in production, should be familiar
with the topics covered in this part. with these topics.
</para> </para>
<para> <para>
The information in this part is arranged approximately in the The information attempts to be in the order in which
order in which a new user should read it. But the chapters are a new user should read it. The chapters are self-contained and
self-contained and can be read individually as desired. The can be read individually as desired. The information is presented
information in this part is presented in a narrative fashion in in a narrative form in topical units. Readers looking for a complete
topical units. Readers looking for a complete description of a description of a command are encouraged to review the
particular command should see <xref linkend="reference"/>. <xref linkend="reference"/>.
</para> </para>
<para> <para>
The first few chapters are written so they can be understood The first few chapters are written so they can be understood
without prerequisite knowledge, so new users who need to set without prerequisite knowledge, so new users who need to set
up their own server can begin their exploration with this part. up their own server can begin their exploration. The rest of this
The rest of this part is about tuning and management; that material part is about tuning and management; that material
assumes that the reader is familiar with the general use of assumes that the reader is familiar with the general use of
the <productname>PostgreSQL</productname> database system. Readers are the <productname>PostgreSQL</productname> database system. Readers are
encouraged to look at <xref linkend="tutorial"/> and <xref encouraged review the <xref linkend="tutorial"/> and <xref
linkend="sql"/> for additional information. linkend="sql"/> parts for additional information.
</para> </para>
</partintro> </partintro>
@ -182,13 +178,13 @@ break is not needed in a wider output rendering.
<para> <para>
This part describes the client programming interfaces distributed This part describes the client programming interfaces distributed
with <productname>PostgreSQL</productname>. Each of these chapters can be with <productname>PostgreSQL</productname>. Each of these chapters can be
read independently. Note that there are many other programming read independently. There are many external programming
interfaces for client programs that are distributed separately and interfaces for client programs that are distributed separately. They
contain their own documentation (<xref linkend="external-projects"/> contain their own documentation (<xref linkend="external-projects"/>
lists some of the more popular ones). Readers of this part should be lists some of the more popular ones). Readers of this part should be
familiar with using <acronym>SQL</acronym> commands to manipulate familiar with using <acronym>SQL</acronym> to manipulate
and query the database (see <xref linkend="sql"/>) and of course and query the database (see <xref linkend="sql"/>) and of course
with the programming language that the interface uses. with the programming language of their choice.
</para> </para>
</partintro> </partintro>
@ -206,15 +202,15 @@ break is not needed in a wider output rendering.
<para> <para>
This part is about extending the server functionality with This part is about extending the server functionality with
user-defined functions, data types, triggers, etc. These are user-defined functions, data types, triggers, etc. These are
advanced topics which should probably be approached only after all advanced topics which should be approached only after all
the other user documentation about <productname>PostgreSQL</productname> has the other user documentation about <productname>PostgreSQL</productname> has
been understood. Later chapters in this part describe the server-side been understood. Later chapters in this part describe the server-side
programming languages available in the programming languages available in the
<productname>PostgreSQL</productname> distribution as well as <productname>PostgreSQL</productname> distribution as well as
general issues concerning server-side programming languages. It general issues concerning server-side programming. It
is essential to read at least the earlier sections of <xref is essential to read at least the earlier sections of <xref
linkend="extend"/> (covering functions) before diving into the linkend="extend"/> (covering functions) before diving into the
material about server-side programming languages. material about server-side programming.
</para> </para>
</partintro> </partintro>

View File

@ -208,7 +208,7 @@ ANALYZE [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] [ <r
<para> <para>
<command>ANALYZE</command> <command>ANALYZE</command>
requires only a read lock on the target table, so it can run in requires only a read lock on the target table, so it can run in
parallel with other activity on the table. parallel with other non-DDL activity on the table.
</para> </para>
<para> <para>

View File

@ -320,8 +320,7 @@ bloom_init(int ndistinct, double false_positive_rate)
int nhashes; /* number of hash functions */ int nhashes; /* number of hash functions */
Assert(ndistinct > 0); Assert(ndistinct > 0);
Assert((false_positive_rate >= BLOOM_MIN_FALSE_POSITIVE_RATE) && Assert(false_positive_rate > 0 && false_positive_rate < 1);
(false_positive_rate < BLOOM_MAX_FALSE_POSITIVE_RATE));
/* calculate bloom filter size / parameters */ /* calculate bloom filter size / parameters */
bloom_filter_size(ndistinct, false_positive_rate, bloom_filter_size(ndistinct, false_positive_rate,

View File

@ -27,9 +27,34 @@
#include "common/hashfn.h" #include "common/hashfn.h"
#include "utils/builtins.h" #include "utils/builtins.h"
#include "utils/datum.h" #include "utils/datum.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
#include "utils/syscache.h" #include "utils/syscache.h"
/* ResourceOwner callbacks to hold tupledesc references */
static void ResOwnerReleaseTupleDesc(Datum res);
static char *ResOwnerPrintTupleDesc(Datum res);
static const ResourceOwnerDesc tupdesc_resowner_desc =
{
.name = "tupdesc reference",
.release_phase = RESOURCE_RELEASE_AFTER_LOCKS,
.release_priority = RELEASE_PRIO_TUPDESC_REFS,
.ReleaseResource = ResOwnerReleaseTupleDesc,
.DebugPrint = ResOwnerPrintTupleDesc
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc)
{
ResourceOwnerRemember(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_desc);
}
static inline void
ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc)
{
ResourceOwnerForget(owner, PointerGetDatum(tupdesc), &tupdesc_resowner_desc);
}
/* /*
* CreateTemplateTupleDesc * CreateTemplateTupleDesc
@ -364,7 +389,7 @@ IncrTupleDescRefCount(TupleDesc tupdesc)
{ {
Assert(tupdesc->tdrefcount >= 0); Assert(tupdesc->tdrefcount >= 0);
ResourceOwnerEnlargeTupleDescs(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
tupdesc->tdrefcount++; tupdesc->tdrefcount++;
ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc); ResourceOwnerRememberTupleDesc(CurrentResourceOwner, tupdesc);
} }
@ -847,3 +872,25 @@ TupleDescGetDefault(TupleDesc tupdesc, AttrNumber attnum)
return result; return result;
} }
/* ResourceOwner callbacks */
static void
ResOwnerReleaseTupleDesc(Datum res)
{
TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res);
/* Like DecrTupleDescRefCount, but don't call ResourceOwnerForget() */
Assert(tupdesc->tdrefcount > 0);
if (--tupdesc->tdrefcount == 0)
FreeTupleDesc(tupdesc);
}
static char *
ResOwnerPrintTupleDesc(Datum res)
{
TupleDesc tupdesc = (TupleDesc) DatumGetPointer(res);
return psprintf("TupleDesc %p (%u,%d)",
tupdesc, tupdesc->tdtypeid, tupdesc->tdtypmod);
}

View File

@ -5172,9 +5172,23 @@ AbortSubTransaction(void)
ResourceOwnerRelease(s->curTransactionOwner, ResourceOwnerRelease(s->curTransactionOwner,
RESOURCE_RELEASE_BEFORE_LOCKS, RESOURCE_RELEASE_BEFORE_LOCKS,
false, false); false, false);
AtEOSubXact_RelationCache(false, s->subTransactionId, AtEOSubXact_RelationCache(false, s->subTransactionId,
s->parent->subTransactionId); s->parent->subTransactionId);
/*
* AtEOSubXact_Inval sometimes needs to temporarily bump the refcount
* on the relcache entries that it processes. We cannot use the
* subtransaction's resource owner anymore, because we've already
* started releasing it. But we can use the parent resource owner.
*/
CurrentResourceOwner = s->parent->curTransactionOwner;
AtEOSubXact_Inval(false); AtEOSubXact_Inval(false);
CurrentResourceOwner = s->curTransactionOwner;
ResourceOwnerRelease(s->curTransactionOwner, ResourceOwnerRelease(s->curTransactionOwner,
RESOURCE_RELEASE_LOCKS, RESOURCE_RELEASE_LOCKS,
false, false); false, false);

View File

@ -7630,6 +7630,9 @@ set_attnotnull(List **wqueue, Relation rel, AttrNumber attnum, bool recurse,
Form_pg_attribute attForm; Form_pg_attribute attForm;
bool retval = false; bool retval = false;
/* Guard against stack overflow due to overly deep inheritance tree. */
check_stack_depth();
tuple = SearchSysCacheCopyAttNum(RelationGetRelid(rel), attnum); tuple = SearchSysCacheCopyAttNum(RelationGetRelid(rel), attnum);
if (!HeapTupleIsValid(tuple)) if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for attribute %d of relation %u", elog(ERROR, "cache lookup failed for attribute %d of relation %u",
@ -7716,6 +7719,9 @@ ATExecSetNotNull(List **wqueue, Relation rel, char *conName, char *colName,
bool is_no_inherit = false; bool is_no_inherit = false;
List *ready = NIL; List *ready = NIL;
/* Guard against stack overflow due to overly deep inheritance tree. */
check_stack_depth();
/* /*
* In cases of multiple inheritance, we might visit the same child more * In cases of multiple inheritance, we might visit the same child more
* than once. In the topmost call, set up a list that we fill with all * than once. In the topmost call, set up a list that we fill with all
@ -9359,6 +9365,9 @@ ATAddCheckNNConstraint(List **wqueue, AlteredTableInfo *tab, Relation rel,
ListCell *child; ListCell *child;
ObjectAddress address = InvalidObjectAddress; ObjectAddress address = InvalidObjectAddress;
/* Guard against stack overflow due to overly deep inheritance tree. */
check_stack_depth();
/* At top level, permission check was done in ATPrepCmd, else do it */ /* At top level, permission check was done in ATPrepCmd, else do it */
if (recursing) if (recursing)
ATSimplePermissions(AT_AddConstraint, rel, ATT_TABLE | ATT_FOREIGN_TABLE); ATSimplePermissions(AT_AddConstraint, rel, ATT_TABLE | ATT_FOREIGN_TABLE);
@ -12428,6 +12437,9 @@ dropconstraint_internal(Relation rel, HeapTuple constraintTup, DropBehavior beha
return InvalidObjectAddress; return InvalidObjectAddress;
*readyRels = lappend_oid(*readyRels, RelationGetRelid(rel)); *readyRels = lappend_oid(*readyRels, RelationGetRelid(rel));
/* Guard against stack overflow due to overly deep inheritance tree. */
check_stack_depth();
/* At top level, permission check was done in ATPrepCmd, else do it */ /* At top level, permission check was done in ATPrepCmd, else do it */
if (recursing) if (recursing)
ATSimplePermissions(AT_DropConstraint, rel, ATT_TABLE | ATT_FOREIGN_TABLE); ATSimplePermissions(AT_DropConstraint, rel, ATT_TABLE | ATT_FOREIGN_TABLE);

View File

@ -26,7 +26,6 @@
#include "jit/jit.h" #include "jit/jit.h"
#include "miscadmin.h" #include "miscadmin.h"
#include "utils/fmgrprotos.h" #include "utils/fmgrprotos.h"
#include "utils/resowner_private.h"
/* GUCs */ /* GUCs */
bool jit_enabled = true; bool jit_enabled = true;
@ -140,7 +139,6 @@ jit_release_context(JitContext *context)
if (provider_successfully_loaded) if (provider_successfully_loaded)
provider.release_context(context); provider.release_context(context);
ResourceOwnerForgetJIT(context->resowner, PointerGetDatum(context));
pfree(context); pfree(context);
} }

View File

@ -45,7 +45,7 @@
#include "portability/instr_time.h" #include "portability/instr_time.h"
#include "storage/ipc.h" #include "storage/ipc.h"
#include "utils/memutils.h" #include "utils/memutils.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
#define LLVMJIT_LLVM_CONTEXT_REUSE_MAX 100 #define LLVMJIT_LLVM_CONTEXT_REUSE_MAX 100
@ -131,6 +131,30 @@ static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm);
static char *llvm_error_message(LLVMErrorRef error); static char *llvm_error_message(LLVMErrorRef error);
#endif /* LLVM_VERSION_MAJOR > 11 */ #endif /* LLVM_VERSION_MAJOR > 11 */
/* ResourceOwner callbacks to hold JitContexts */
static void ResOwnerReleaseJitContext(Datum res);
static const ResourceOwnerDesc jit_resowner_desc =
{
.name = "LLVM JIT context",
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
.release_priority = RELEASE_PRIO_JIT_CONTEXTS,
.ReleaseResource = ResOwnerReleaseJitContext,
.DebugPrint = NULL /* the default message is fine */
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberJIT(ResourceOwner owner, LLVMJitContext *handle)
{
ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_desc);
}
static inline void
ResourceOwnerForgetJIT(ResourceOwner owner, LLVMJitContext *handle)
{
ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_desc);
}
PG_MODULE_MAGIC; PG_MODULE_MAGIC;
@ -220,7 +244,7 @@ llvm_create_context(int jitFlags)
llvm_recreate_llvm_context(); llvm_recreate_llvm_context();
ResourceOwnerEnlargeJIT(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
context = MemoryContextAllocZero(TopMemoryContext, context = MemoryContextAllocZero(TopMemoryContext,
sizeof(LLVMJitContext)); sizeof(LLVMJitContext));
@ -228,7 +252,7 @@ llvm_create_context(int jitFlags)
/* ensure cleanup */ /* ensure cleanup */
context->base.resowner = CurrentResourceOwner; context->base.resowner = CurrentResourceOwner;
ResourceOwnerRememberJIT(CurrentResourceOwner, PointerGetDatum(context)); ResourceOwnerRememberJIT(CurrentResourceOwner, context);
llvm_jit_context_in_use_count++; llvm_jit_context_in_use_count++;
@ -300,6 +324,9 @@ llvm_release_context(JitContext *context)
llvm_jit_context->handles = NIL; llvm_jit_context->handles = NIL;
llvm_leave_fatal_on_oom(); llvm_leave_fatal_on_oom();
if (context->resowner)
ResourceOwnerForgetJIT(context->resowner, llvm_jit_context);
} }
/* /*
@ -1394,3 +1421,15 @@ llvm_error_message(LLVMErrorRef error)
} }
#endif /* LLVM_VERSION_MAJOR > 11 */ #endif /* LLVM_VERSION_MAJOR > 11 */
/*
* ResourceOwner callbacks
*/
static void
ResOwnerReleaseJitContext(Datum res)
{
JitContext *context = (JitContext *) DatumGetPointer(res);
context->resowner = NULL;
jit_release_context(context);
}

View File

@ -47,6 +47,7 @@
#include "postmaster/bgwriter.h" #include "postmaster/bgwriter.h"
#include "storage/buf_internals.h" #include "storage/buf_internals.h"
#include "storage/bufmgr.h" #include "storage/bufmgr.h"
#include "storage/fd.h"
#include "storage/ipc.h" #include "storage/ipc.h"
#include "storage/lmgr.h" #include "storage/lmgr.h"
#include "storage/proc.h" #include "storage/proc.h"
@ -55,7 +56,7 @@
#include "utils/memdebug.h" #include "utils/memdebug.h"
#include "utils/ps_status.h" #include "utils/ps_status.h"
#include "utils/rel.h" #include "utils/rel.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
#include "utils/timestamp.h" #include "utils/timestamp.h"
@ -205,6 +206,30 @@ static PrivateRefCountEntry *GetPrivateRefCountEntry(Buffer buffer, bool do_move
static inline int32 GetPrivateRefCount(Buffer buffer); static inline int32 GetPrivateRefCount(Buffer buffer);
static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref); static void ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref);
/* ResourceOwner callbacks to hold in-progress I/Os and buffer pins */
static void ResOwnerReleaseBufferIO(Datum res);
static char *ResOwnerPrintBufferIO(Datum res);
static void ResOwnerReleaseBufferPin(Datum res);
static char *ResOwnerPrintBufferPin(Datum res);
const ResourceOwnerDesc buffer_io_resowner_desc =
{
.name = "buffer io",
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
.release_priority = RELEASE_PRIO_BUFFER_IOS,
.ReleaseResource = ResOwnerReleaseBufferIO,
.DebugPrint = ResOwnerPrintBufferIO
};
const ResourceOwnerDesc buffer_pin_resowner_desc =
{
.name = "buffer pin",
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
.release_priority = RELEASE_PRIO_BUFFER_PINS,
.ReleaseResource = ResOwnerReleaseBufferPin,
.DebugPrint = ResOwnerPrintBufferPin
};
/* /*
* Ensure that the PrivateRefCountArray has sufficient space to store one more * Ensure that the PrivateRefCountArray has sufficient space to store one more
* entry. This has to be called before using NewPrivateRefCountEntry() to fill * entry. This has to be called before using NewPrivateRefCountEntry() to fill
@ -470,6 +495,7 @@ static BlockNumber ExtendBufferedRelShared(BufferManagerRelation bmr,
static bool PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy); static bool PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy);
static void PinBuffer_Locked(BufferDesc *buf); static void PinBuffer_Locked(BufferDesc *buf);
static void UnpinBuffer(BufferDesc *buf); static void UnpinBuffer(BufferDesc *buf);
static void UnpinBufferNoOwner(BufferDesc *buf);
static void BufferSync(int flags); static void BufferSync(int flags);
static uint32 WaitBufHdrUnlocked(BufferDesc *buf); static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used, static int SyncOneBuffer(int buf_id, bool skip_recently_used,
@ -477,7 +503,8 @@ static int SyncOneBuffer(int buf_id, bool skip_recently_used,
static void WaitIO(BufferDesc *buf); static void WaitIO(BufferDesc *buf);
static bool StartBufferIO(BufferDesc *buf, bool forInput); static bool StartBufferIO(BufferDesc *buf, bool forInput);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty, static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits); uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
static void shared_buffer_write_error_callback(void *arg); static void shared_buffer_write_error_callback(void *arg);
static void local_buffer_write_error_callback(void *arg); static void local_buffer_write_error_callback(void *arg);
static BufferDesc *BufferAlloc(SMgrRelation smgr, static BufferDesc *BufferAlloc(SMgrRelation smgr,
@ -639,7 +666,7 @@ ReadRecentBuffer(RelFileLocator rlocator, ForkNumber forkNum, BlockNumber blockN
Assert(BufferIsValid(recent_buffer)); Assert(BufferIsValid(recent_buffer));
ResourceOwnerEnlargeBuffers(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
ReservePrivateRefCountEntry(); ReservePrivateRefCountEntry();
InitBufferTag(&tag, &rlocator, forkNum, blockNum); InitBufferTag(&tag, &rlocator, forkNum, blockNum);
@ -1023,9 +1050,6 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
forkNum, strategy, flags); forkNum, strategy, flags);
} }
/* Make sure we will have room to remember the buffer pin */
ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum, TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
smgr->smgr_rlocator.locator.spcOid, smgr->smgr_rlocator.locator.spcOid,
smgr->smgr_rlocator.locator.dbOid, smgr->smgr_rlocator.locator.dbOid,
@ -1176,7 +1200,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
else else
{ {
/* Set BM_VALID, terminate IO, and wake up any waiters */ /* Set BM_VALID, terminate IO, and wake up any waiters */
TerminateBufferIO(bufHdr, false, BM_VALID); TerminateBufferIO(bufHdr, false, BM_VALID, true);
} }
VacuumPageMiss++; VacuumPageMiss++;
@ -1230,6 +1254,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
BufferDesc *victim_buf_hdr; BufferDesc *victim_buf_hdr;
uint32 victim_buf_state; uint32 victim_buf_state;
/* Make sure we will have room to remember the buffer pin */
ResourceOwnerEnlarge(CurrentResourceOwner);
ReservePrivateRefCountEntry();
/* create a tag so we can lookup the buffer */ /* create a tag so we can lookup the buffer */
InitBufferTag(&newTag, &smgr->smgr_rlocator.locator, forkNum, blockNum); InitBufferTag(&newTag, &smgr->smgr_rlocator.locator, forkNum, blockNum);
@ -1314,9 +1342,8 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
* use. * use.
* *
* We could do this after releasing the partition lock, but then we'd * We could do this after releasing the partition lock, but then we'd
* have to call ResourceOwnerEnlargeBuffers() & * have to call ResourceOwnerEnlarge() & ReservePrivateRefCountEntry()
* ReservePrivateRefCountEntry() before acquiring the lock, for the * before acquiring the lock, for the rare case of such a collision.
* rare case of such a collision.
*/ */
UnpinBuffer(victim_buf_hdr); UnpinBuffer(victim_buf_hdr);
@ -1591,10 +1618,10 @@ GetVictimBuffer(BufferAccessStrategy strategy, IOContext io_context)
/* /*
* Ensure, while the spinlock's not yet held, that there's a free refcount * Ensure, while the spinlock's not yet held, that there's a free refcount
* entry. * entry, and a resource owner slot for the pin.
*/ */
ReservePrivateRefCountEntry(); ReservePrivateRefCountEntry();
ResourceOwnerEnlargeBuffers(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
/* we return here if a prospective victim buffer gets used concurrently */ /* we return here if a prospective victim buffer gets used concurrently */
again: again:
@ -1859,9 +1886,6 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
MemSet((char *) buf_block, 0, BLCKSZ); MemSet((char *) buf_block, 0, BLCKSZ);
} }
/* in case we need to pin an existing buffer below */
ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
/* /*
* Lock relation against concurrent extensions, unless requested not to. * Lock relation against concurrent extensions, unless requested not to.
* *
@ -1947,6 +1971,10 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLock *partition_lock; LWLock *partition_lock;
int existing_id; int existing_id;
/* in case we need to pin an existing buffer below */
ResourceOwnerEnlarge(CurrentResourceOwner);
ReservePrivateRefCountEntry();
InitBufferTag(&tag, &bmr.smgr->smgr_rlocator.locator, fork, first_block + i); InitBufferTag(&tag, &bmr.smgr->smgr_rlocator.locator, fork, first_block + i);
hash = BufTableHashCode(&tag); hash = BufTableHashCode(&tag);
partition_lock = BufMappingPartitionLock(hash); partition_lock = BufMappingPartitionLock(hash);
@ -2088,7 +2116,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
if (lock) if (lock)
LWLockAcquire(BufferDescriptorGetContentLock(buf_hdr), LW_EXCLUSIVE); LWLockAcquire(BufferDescriptorGetContentLock(buf_hdr), LW_EXCLUSIVE);
TerminateBufferIO(buf_hdr, false, BM_VALID); TerminateBufferIO(buf_hdr, false, BM_VALID, true);
} }
pgBufferUsage.shared_blks_written += extend_by; pgBufferUsage.shared_blks_written += extend_by;
@ -2281,7 +2309,8 @@ ReleaseAndReadBuffer(Buffer buffer,
* taking the buffer header lock; instead update the state variable in loop of * taking the buffer header lock; instead update the state variable in loop of
* CAS operations. Hopefully it's just a single CAS. * CAS operations. Hopefully it's just a single CAS.
* *
* Note that ResourceOwnerEnlargeBuffers must have been done already. * Note that ResourceOwnerEnlarge() and ReservePrivateRefCountEntry()
* must have been done already.
* *
* Returns true if buffer is BM_VALID, else false. This provision allows * Returns true if buffer is BM_VALID, else false. This provision allows
* some callers to avoid an extra spinlock cycle. * some callers to avoid an extra spinlock cycle.
@ -2294,6 +2323,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
PrivateRefCountEntry *ref; PrivateRefCountEntry *ref;
Assert(!BufferIsLocal(b)); Assert(!BufferIsLocal(b));
Assert(ReservedRefCountEntry != NULL);
ref = GetPrivateRefCountEntry(b, true); ref = GetPrivateRefCountEntry(b, true);
@ -2302,7 +2332,6 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
uint32 buf_state; uint32 buf_state;
uint32 old_buf_state; uint32 old_buf_state;
ReservePrivateRefCountEntry();
ref = NewPrivateRefCountEntry(b); ref = NewPrivateRefCountEntry(b);
old_buf_state = pg_atomic_read_u32(&buf->state); old_buf_state = pg_atomic_read_u32(&buf->state);
@ -2375,7 +2404,8 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* The spinlock is released before return. * The spinlock is released before return.
* *
* As this function is called with the spinlock held, the caller has to * As this function is called with the spinlock held, the caller has to
* previously call ReservePrivateRefCountEntry(). * previously call ReservePrivateRefCountEntry() and
* ResourceOwnerEnlarge(CurrentResourceOwner);
* *
* Currently, no callers of this function want to modify the buffer's * Currently, no callers of this function want to modify the buffer's
* usage_count at all, so there's no need for a strategy parameter. * usage_count at all, so there's no need for a strategy parameter.
@ -2436,6 +2466,15 @@ PinBuffer_Locked(BufferDesc *buf)
*/ */
static void static void
UnpinBuffer(BufferDesc *buf) UnpinBuffer(BufferDesc *buf)
{
Buffer b = BufferDescriptorGetBuffer(buf);
ResourceOwnerForgetBuffer(CurrentResourceOwner, b);
UnpinBufferNoOwner(buf);
}
static void
UnpinBufferNoOwner(BufferDesc *buf)
{ {
PrivateRefCountEntry *ref; PrivateRefCountEntry *ref;
Buffer b = BufferDescriptorGetBuffer(buf); Buffer b = BufferDescriptorGetBuffer(buf);
@ -2445,9 +2484,6 @@ UnpinBuffer(BufferDesc *buf)
/* not moving as we're likely deleting it soon anyway */ /* not moving as we're likely deleting it soon anyway */
ref = GetPrivateRefCountEntry(b, false); ref = GetPrivateRefCountEntry(b, false);
Assert(ref != NULL); Assert(ref != NULL);
ResourceOwnerForgetBuffer(CurrentResourceOwner, b);
Assert(ref->refcount > 0); Assert(ref->refcount > 0);
ref->refcount--; ref->refcount--;
if (ref->refcount == 0) if (ref->refcount == 0)
@ -2550,9 +2586,6 @@ BufferSync(int flags)
int mask = BM_DIRTY; int mask = BM_DIRTY;
WritebackContext wb_context; WritebackContext wb_context;
/* Make sure we can handle the pin inside SyncOneBuffer */
ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
/* /*
* Unless this is a shutdown checkpoint or we have been explicitly told, * Unless this is a shutdown checkpoint or we have been explicitly told,
* we write only permanent, dirty buffers. But at shutdown or end of * we write only permanent, dirty buffers. But at shutdown or end of
@ -3029,9 +3062,6 @@ BgBufferSync(WritebackContext *wb_context)
* requirements, or hit the bgwriter_lru_maxpages limit. * requirements, or hit the bgwriter_lru_maxpages limit.
*/ */
/* Make sure we can handle the pin inside SyncOneBuffer */
ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
num_to_scan = bufs_to_lap; num_to_scan = bufs_to_lap;
num_written = 0; num_written = 0;
reusable_buffers = reusable_buffers_est; reusable_buffers = reusable_buffers_est;
@ -3113,8 +3143,6 @@ BgBufferSync(WritebackContext *wb_context)
* *
* (BUF_WRITTEN could be set in error if FlushBuffer finds the buffer clean * (BUF_WRITTEN could be set in error if FlushBuffer finds the buffer clean
* after locking it, but we don't care all that much.) * after locking it, but we don't care all that much.)
*
* Note: caller must have done ResourceOwnerEnlargeBuffers.
*/ */
static int static int
SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context) SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context)
@ -3124,7 +3152,9 @@ SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context)
uint32 buf_state; uint32 buf_state;
BufferTag tag; BufferTag tag;
/* Make sure we can handle the pin */
ReservePrivateRefCountEntry(); ReservePrivateRefCountEntry();
ResourceOwnerEnlarge(CurrentResourceOwner);
/* /*
* Check whether buffer needs writing. * Check whether buffer needs writing.
@ -3254,6 +3284,7 @@ CheckForBufferLeaks(void)
int RefCountErrors = 0; int RefCountErrors = 0;
PrivateRefCountEntry *res; PrivateRefCountEntry *res;
int i; int i;
char *s;
/* check the array */ /* check the array */
for (i = 0; i < REFCOUNT_ARRAY_ENTRIES; i++) for (i = 0; i < REFCOUNT_ARRAY_ENTRIES; i++)
@ -3262,7 +3293,10 @@ CheckForBufferLeaks(void)
if (res->buffer != InvalidBuffer) if (res->buffer != InvalidBuffer)
{ {
PrintBufferLeakWarning(res->buffer); s = DebugPrintBufferRefcount(res->buffer);
elog(WARNING, "buffer refcount leak: %s", s);
pfree(s);
RefCountErrors++; RefCountErrors++;
} }
} }
@ -3275,7 +3309,9 @@ CheckForBufferLeaks(void)
hash_seq_init(&hstat, PrivateRefCountHash); hash_seq_init(&hstat, PrivateRefCountHash);
while ((res = (PrivateRefCountEntry *) hash_seq_search(&hstat)) != NULL) while ((res = (PrivateRefCountEntry *) hash_seq_search(&hstat)) != NULL)
{ {
PrintBufferLeakWarning(res->buffer); s = DebugPrintBufferRefcount(res->buffer);
elog(WARNING, "buffer refcount leak: %s", s);
pfree(s);
RefCountErrors++; RefCountErrors++;
} }
} }
@ -3287,12 +3323,13 @@ CheckForBufferLeaks(void)
/* /*
* Helper routine to issue warnings when a buffer is unexpectedly pinned * Helper routine to issue warnings when a buffer is unexpectedly pinned
*/ */
void char *
PrintBufferLeakWarning(Buffer buffer) DebugPrintBufferRefcount(Buffer buffer)
{ {
BufferDesc *buf; BufferDesc *buf;
int32 loccount; int32 loccount;
char *path; char *path;
char *result;
BackendId backend; BackendId backend;
uint32 buf_state; uint32 buf_state;
@ -3314,13 +3351,13 @@ PrintBufferLeakWarning(Buffer buffer)
path = relpathbackend(BufTagGetRelFileLocator(&buf->tag), backend, path = relpathbackend(BufTagGetRelFileLocator(&buf->tag), backend,
BufTagGetForkNum(&buf->tag)); BufTagGetForkNum(&buf->tag));
buf_state = pg_atomic_read_u32(&buf->state); buf_state = pg_atomic_read_u32(&buf->state);
elog(WARNING,
"buffer refcount leak: [%03d] " result = psprintf("[%03d] (rel=%s, blockNum=%u, flags=0x%x, refcount=%u %d)",
"(rel=%s, blockNum=%u, flags=0x%x, refcount=%u %d)", buffer, path,
buffer, path, buf->tag.blockNum, buf_state & BUF_FLAG_MASK,
buf->tag.blockNum, buf_state & BUF_FLAG_MASK, BUF_STATE_GET_REFCOUNT(buf_state), loccount);
BUF_STATE_GET_REFCOUNT(buf_state), loccount);
pfree(path); pfree(path);
return result;
} }
/* /*
@ -3524,7 +3561,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* Mark the buffer as clean (unless BM_JUST_DIRTIED has become set) and * Mark the buffer as clean (unless BM_JUST_DIRTIED has become set) and
* end the BM_IO_IN_PROGRESS state. * end the BM_IO_IN_PROGRESS state.
*/ */
TerminateBufferIO(buf, true, 0); TerminateBufferIO(buf, true, 0, true);
TRACE_POSTGRESQL_BUFFER_FLUSH_DONE(BufTagGetForkNum(&buf->tag), TRACE_POSTGRESQL_BUFFER_FLUSH_DONE(BufTagGetForkNum(&buf->tag),
buf->tag.blockNum, buf->tag.blockNum,
@ -4169,9 +4206,6 @@ FlushRelationBuffers(Relation rel)
return; return;
} }
/* Make sure we can handle the pin inside the loop */
ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
for (i = 0; i < NBuffers; i++) for (i = 0; i < NBuffers; i++)
{ {
uint32 buf_state; uint32 buf_state;
@ -4185,7 +4219,9 @@ FlushRelationBuffers(Relation rel)
if (!BufTagMatchesRelFileLocator(&bufHdr->tag, &rel->rd_locator)) if (!BufTagMatchesRelFileLocator(&bufHdr->tag, &rel->rd_locator))
continue; continue;
/* Make sure we can handle the pin */
ReservePrivateRefCountEntry(); ReservePrivateRefCountEntry();
ResourceOwnerEnlarge(CurrentResourceOwner);
buf_state = LockBufHdr(bufHdr); buf_state = LockBufHdr(bufHdr);
if (BufTagMatchesRelFileLocator(&bufHdr->tag, &rel->rd_locator) && if (BufTagMatchesRelFileLocator(&bufHdr->tag, &rel->rd_locator) &&
@ -4242,9 +4278,6 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels)
if (use_bsearch) if (use_bsearch)
pg_qsort(srels, nrels, sizeof(SMgrSortArray), rlocator_comparator); pg_qsort(srels, nrels, sizeof(SMgrSortArray), rlocator_comparator);
/* Make sure we can handle the pin inside the loop */
ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
for (i = 0; i < NBuffers; i++) for (i = 0; i < NBuffers; i++)
{ {
SMgrSortArray *srelent = NULL; SMgrSortArray *srelent = NULL;
@ -4283,7 +4316,9 @@ FlushRelationsAllBuffers(SMgrRelation *smgrs, int nrels)
if (srelent == NULL) if (srelent == NULL)
continue; continue;
/* Make sure we can handle the pin */
ReservePrivateRefCountEntry(); ReservePrivateRefCountEntry();
ResourceOwnerEnlarge(CurrentResourceOwner);
buf_state = LockBufHdr(bufHdr); buf_state = LockBufHdr(bufHdr);
if (BufTagMatchesRelFileLocator(&bufHdr->tag, &srelent->rlocator) && if (BufTagMatchesRelFileLocator(&bufHdr->tag, &srelent->rlocator) &&
@ -4478,9 +4513,6 @@ FlushDatabaseBuffers(Oid dbid)
int i; int i;
BufferDesc *bufHdr; BufferDesc *bufHdr;
/* Make sure we can handle the pin inside the loop */
ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
for (i = 0; i < NBuffers; i++) for (i = 0; i < NBuffers; i++)
{ {
uint32 buf_state; uint32 buf_state;
@ -4494,7 +4526,9 @@ FlushDatabaseBuffers(Oid dbid)
if (bufHdr->tag.dbOid != dbid) if (bufHdr->tag.dbOid != dbid)
continue; continue;
/* Make sure we can handle the pin */
ReservePrivateRefCountEntry(); ReservePrivateRefCountEntry();
ResourceOwnerEnlarge(CurrentResourceOwner);
buf_state = LockBufHdr(bufHdr); buf_state = LockBufHdr(bufHdr);
if (bufHdr->tag.dbOid == dbid && if (bufHdr->tag.dbOid == dbid &&
@ -4571,7 +4605,7 @@ void
IncrBufferRefCount(Buffer buffer) IncrBufferRefCount(Buffer buffer)
{ {
Assert(BufferIsPinned(buffer)); Assert(BufferIsPinned(buffer));
ResourceOwnerEnlargeBuffers(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
if (BufferIsLocal(buffer)) if (BufferIsLocal(buffer))
LocalRefCount[-buffer - 1]++; LocalRefCount[-buffer - 1]++;
else else
@ -5169,7 +5203,7 @@ StartBufferIO(BufferDesc *buf, bool forInput)
{ {
uint32 buf_state; uint32 buf_state;
ResourceOwnerEnlargeBufferIOs(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
for (;;) for (;;)
{ {
@ -5214,9 +5248,14 @@ StartBufferIO(BufferDesc *buf, bool forInput)
* set_flag_bits gets ORed into the buffer's flags. It must include * set_flag_bits gets ORed into the buffer's flags. It must include
* BM_IO_ERROR in a failure case. For successful completion it could * BM_IO_ERROR in a failure case. For successful completion it could
* be 0, or BM_VALID if we just finished reading in the page. * be 0, or BM_VALID if we just finished reading in the page.
*
* If forget_owner is true, we release the buffer I/O from the current
* resource owner. (forget_owner=false is used when the resource owner itself
* is being released)
*/ */
static void static void
TerminateBufferIO(BufferDesc *buf, bool clear_dirty, uint32 set_flag_bits) TerminateBufferIO(BufferDesc *buf, bool clear_dirty, uint32 set_flag_bits,
bool forget_owner)
{ {
uint32 buf_state; uint32 buf_state;
@ -5231,8 +5270,9 @@ TerminateBufferIO(BufferDesc *buf, bool clear_dirty, uint32 set_flag_bits)
buf_state |= set_flag_bits; buf_state |= set_flag_bits;
UnlockBufHdr(buf, buf_state); UnlockBufHdr(buf, buf_state);
ResourceOwnerForgetBufferIO(CurrentResourceOwner, if (forget_owner)
BufferDescriptorGetBuffer(buf)); ResourceOwnerForgetBufferIO(CurrentResourceOwner,
BufferDescriptorGetBuffer(buf));
ConditionVariableBroadcast(BufferDescriptorGetIOCV(buf)); ConditionVariableBroadcast(BufferDescriptorGetIOCV(buf));
} }
@ -5245,8 +5285,12 @@ TerminateBufferIO(BufferDesc *buf, bool clear_dirty, uint32 set_flag_bits)
* *
* If I/O was in progress, we always set BM_IO_ERROR, even though it's * If I/O was in progress, we always set BM_IO_ERROR, even though it's
* possible the error condition wasn't related to the I/O. * possible the error condition wasn't related to the I/O.
*
* Note: this does not remove the buffer I/O from the resource owner.
* That's correct when we're releasing the whole resource owner, but
* beware if you use this in other contexts.
*/ */
void static void
AbortBufferIO(Buffer buffer) AbortBufferIO(Buffer buffer)
{ {
BufferDesc *buf_hdr = GetBufferDescriptor(buffer - 1); BufferDesc *buf_hdr = GetBufferDescriptor(buffer - 1);
@ -5282,7 +5326,7 @@ AbortBufferIO(Buffer buffer)
} }
} }
TerminateBufferIO(buf_hdr, false, BM_IO_ERROR); TerminateBufferIO(buf_hdr, false, BM_IO_ERROR, false);
} }
/* /*
@ -5634,3 +5678,42 @@ IssuePendingWritebacks(WritebackContext *wb_context, IOContext io_context)
wb_context->nr_pending = 0; wb_context->nr_pending = 0;
} }
/* ResourceOwner callbacks */
static void
ResOwnerReleaseBufferIO(Datum res)
{
Buffer buffer = DatumGetInt32(res);
AbortBufferIO(buffer);
}
static char *
ResOwnerPrintBufferIO(Datum res)
{
Buffer buffer = DatumGetInt32(res);
return psprintf("lost track of buffer IO on buffer %d", buffer);
}
static void
ResOwnerReleaseBufferPin(Datum res)
{
Buffer buffer = DatumGetInt32(res);
/* Like ReleaseBuffer, but don't call ResourceOwnerForgetBuffer */
if (!BufferIsValid(buffer))
elog(ERROR, "bad buffer ID: %d", buffer);
if (BufferIsLocal(buffer))
UnpinLocalBufferNoOwner(buffer);
else
UnpinBufferNoOwner(GetBufferDescriptor(buffer - 1));
}
static char *
ResOwnerPrintBufferPin(Datum res)
{
return DebugPrintBufferRefcount(DatumGetInt32(res));
}

View File

@ -21,9 +21,10 @@
#include "pgstat.h" #include "pgstat.h"
#include "storage/buf_internals.h" #include "storage/buf_internals.h"
#include "storage/bufmgr.h" #include "storage/bufmgr.h"
#include "storage/fd.h"
#include "utils/guc_hooks.h" #include "utils/guc_hooks.h"
#include "utils/memutils.h" #include "utils/memutils.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
/*#define LBDEBUG*/ /*#define LBDEBUG*/
@ -130,6 +131,8 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
if (LocalBufHash == NULL) if (LocalBufHash == NULL)
InitLocalBuffers(); InitLocalBuffers();
ResourceOwnerEnlarge(CurrentResourceOwner);
/* See if the desired buffer already exists */ /* See if the desired buffer already exists */
hresult = (LocalBufferLookupEnt *) hresult = (LocalBufferLookupEnt *)
hash_search(LocalBufHash, &newTag, HASH_FIND, NULL); hash_search(LocalBufHash, &newTag, HASH_FIND, NULL);
@ -180,7 +183,7 @@ GetLocalVictimBuffer(void)
uint32 buf_state; uint32 buf_state;
BufferDesc *bufHdr; BufferDesc *bufHdr;
ResourceOwnerEnlargeBuffers(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
/* /*
* Need to get a new buffer. We use a clock sweep algorithm (essentially * Need to get a new buffer. We use a clock sweep algorithm (essentially
@ -672,6 +675,13 @@ PinLocalBuffer(BufferDesc *buf_hdr, bool adjust_usagecount)
void void
UnpinLocalBuffer(Buffer buffer) UnpinLocalBuffer(Buffer buffer)
{
UnpinLocalBufferNoOwner(buffer);
ResourceOwnerForgetBuffer(CurrentResourceOwner, buffer);
}
void
UnpinLocalBufferNoOwner(Buffer buffer)
{ {
int buffid = -buffer - 1; int buffid = -buffer - 1;
@ -679,7 +689,6 @@ UnpinLocalBuffer(Buffer buffer)
Assert(LocalRefCount[buffid] > 0); Assert(LocalRefCount[buffid] > 0);
Assert(NLocalPinnedBuffers > 0); Assert(NLocalPinnedBuffers > 0);
ResourceOwnerForgetBuffer(CurrentResourceOwner, buffer);
if (--LocalRefCount[buffid] == 0) if (--LocalRefCount[buffid] == 0)
NLocalPinnedBuffers--; NLocalPinnedBuffers--;
} }
@ -783,8 +792,12 @@ CheckForLocalBufferLeaks(void)
if (LocalRefCount[i] != 0) if (LocalRefCount[i] != 0)
{ {
Buffer b = -i - 1; Buffer b = -i - 1;
char *s;
s = DebugPrintBufferRefcount(b);
elog(WARNING, "local buffer refcount leak: %s", s);
pfree(s);
PrintBufferLeakWarning(b);
RefCountErrors++; RefCountErrors++;
} }
} }

View File

@ -99,7 +99,7 @@
#include "storage/ipc.h" #include "storage/ipc.h"
#include "utils/guc.h" #include "utils/guc.h"
#include "utils/guc_hooks.h" #include "utils/guc_hooks.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
#include "utils/varlena.h" #include "utils/varlena.h"
/* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */ /* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */
@ -354,6 +354,31 @@ static void unlink_if_exists_fname(const char *fname, bool isdir, int elevel);
static int fsync_parent_path(const char *fname, int elevel); static int fsync_parent_path(const char *fname, int elevel);
/* ResourceOwner callbacks to hold virtual file descriptors */
static void ResOwnerReleaseFile(Datum res);
static char *ResOwnerPrintFile(Datum res);
static const ResourceOwnerDesc file_resowner_desc =
{
.name = "File",
.release_phase = RESOURCE_RELEASE_AFTER_LOCKS,
.release_priority = RELEASE_PRIO_FILES,
.ReleaseResource = ResOwnerReleaseFile,
.DebugPrint = ResOwnerPrintFile
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberFile(ResourceOwner owner, File file)
{
ResourceOwnerRemember(owner, Int32GetDatum(file), &file_resowner_desc);
}
static inline void
ResourceOwnerForgetFile(ResourceOwner owner, File file)
{
ResourceOwnerForget(owner, Int32GetDatum(file), &file_resowner_desc);
}
/* /*
* pg_fsync --- do fsync with or without writethrough * pg_fsync --- do fsync with or without writethrough
*/ */
@ -1492,7 +1517,7 @@ ReportTemporaryFileUsage(const char *path, off_t size)
/* /*
* Called to register a temporary file for automatic close. * Called to register a temporary file for automatic close.
* ResourceOwnerEnlargeFiles(CurrentResourceOwner) must have been called * ResourceOwnerEnlarge(CurrentResourceOwner) must have been called
* before the file was opened. * before the file was opened.
*/ */
static void static void
@ -1684,7 +1709,7 @@ OpenTemporaryFile(bool interXact)
* open it, if we'll be registering it below. * open it, if we'll be registering it below.
*/ */
if (!interXact) if (!interXact)
ResourceOwnerEnlargeFiles(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
/* /*
* If some temp tablespace(s) have been given to us, try to use the next * If some temp tablespace(s) have been given to us, try to use the next
@ -1816,7 +1841,7 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure)
Assert(temporary_files_allowed); /* check temp file access is up */ Assert(temporary_files_allowed); /* check temp file access is up */
ResourceOwnerEnlargeFiles(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
/* /*
* Open the file. Note: we don't use O_EXCL, in case there is an orphaned * Open the file. Note: we don't use O_EXCL, in case there is an orphaned
@ -1856,7 +1881,7 @@ PathNameOpenTemporaryFile(const char *path, int mode)
Assert(temporary_files_allowed); /* check temp file access is up */ Assert(temporary_files_allowed); /* check temp file access is up */
ResourceOwnerEnlargeFiles(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
file = PathNameOpenFile(path, mode | PG_BINARY); file = PathNameOpenFile(path, mode | PG_BINARY);
@ -3972,3 +3997,25 @@ assign_debug_io_direct(const char *newval, void *extra)
io_direct_flags = *flags; io_direct_flags = *flags;
} }
/* ResourceOwner callbacks */
static void
ResOwnerReleaseFile(Datum res)
{
File file = (File) DatumGetInt32(res);
Vfd *vfdP;
Assert(FileIsValid(file));
vfdP = &VfdCache[file];
vfdP->resowner = NULL;
FileClose(file);
}
static char *
ResOwnerPrintFile(Datum res)
{
return psprintf("File %d", DatumGetInt32(res));
}

View File

@ -38,13 +38,15 @@
#include "miscadmin.h" #include "miscadmin.h"
#include "port/pg_bitutils.h" #include "port/pg_bitutils.h"
#include "storage/dsm.h" #include "storage/dsm.h"
#include "storage/fd.h"
#include "storage/ipc.h" #include "storage/ipc.h"
#include "storage/lwlock.h" #include "storage/lwlock.h"
#include "storage/pg_shmem.h" #include "storage/pg_shmem.h"
#include "storage/shmem.h"
#include "utils/freepage.h" #include "utils/freepage.h"
#include "utils/guc.h" #include "utils/guc.h"
#include "utils/memutils.h" #include "utils/memutils.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
#define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32 #define PG_DYNSHMEM_CONTROL_MAGIC 0x9a503d32
@ -140,6 +142,32 @@ static dsm_control_header *dsm_control;
static Size dsm_control_mapped_size = 0; static Size dsm_control_mapped_size = 0;
static void *dsm_control_impl_private = NULL; static void *dsm_control_impl_private = NULL;
/* ResourceOwner callbacks to hold DSM segments */
static void ResOwnerReleaseDSM(Datum res);
static char *ResOwnerPrintDSM(Datum res);
static const ResourceOwnerDesc dsm_resowner_desc =
{
.name = "dynamic shared memory segment",
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
.release_priority = RELEASE_PRIO_DSMS,
.ReleaseResource = ResOwnerReleaseDSM,
.DebugPrint = ResOwnerPrintDSM
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg)
{
ResourceOwnerRemember(owner, PointerGetDatum(seg), &dsm_resowner_desc);
}
static inline void
ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg)
{
ResourceOwnerForget(owner, PointerGetDatum(seg), &dsm_resowner_desc);
}
/* /*
* Start up the dynamic shared memory system. * Start up the dynamic shared memory system.
* *
@ -907,7 +935,7 @@ void
dsm_unpin_mapping(dsm_segment *seg) dsm_unpin_mapping(dsm_segment *seg)
{ {
Assert(seg->resowner == NULL); Assert(seg->resowner == NULL);
ResourceOwnerEnlargeDSMs(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
seg->resowner = CurrentResourceOwner; seg->resowner = CurrentResourceOwner;
ResourceOwnerRememberDSM(seg->resowner, seg); ResourceOwnerRememberDSM(seg->resowner, seg);
} }
@ -1176,7 +1204,7 @@ dsm_create_descriptor(void)
dsm_segment *seg; dsm_segment *seg;
if (CurrentResourceOwner) if (CurrentResourceOwner)
ResourceOwnerEnlargeDSMs(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment));
dlist_push_head(&dsm_segment_list, &seg->node); dlist_push_head(&dsm_segment_list, &seg->node);
@ -1255,3 +1283,22 @@ is_main_region_dsm_handle(dsm_handle handle)
{ {
return handle & 1; return handle & 1;
} }
/* ResourceOwner callbacks */
static void
ResOwnerReleaseDSM(Datum res)
{
dsm_segment *seg = (dsm_segment *) DatumGetPointer(res);
seg->resowner = NULL;
dsm_detach(seg);
}
static char *
ResOwnerPrintDSM(Datum res)
{
dsm_segment *seg = (dsm_segment *) DatumGetPointer(res);
return psprintf("dynamic shared memory segment %u",
dsm_segment_handle(seg));
}

View File

@ -48,7 +48,7 @@
#include "storage/standby.h" #include "storage/standby.h"
#include "utils/memutils.h" #include "utils/memutils.h"
#include "utils/ps_status.h" #include "utils/ps_status.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
/* This configuration variable is used to set the lock table size */ /* This configuration variable is used to set the lock table size */

View File

@ -31,12 +31,13 @@
#endif #endif
#include "storage/lmgr.h" #include "storage/lmgr.h"
#include "utils/builtins.h" #include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/datum.h" #include "utils/datum.h"
#include "utils/fmgroids.h" #include "utils/fmgroids.h"
#include "utils/inval.h" #include "utils/inval.h"
#include "utils/memutils.h" #include "utils/memutils.h"
#include "utils/rel.h" #include "utils/rel.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
#include "utils/syscache.h" #include "utils/syscache.h"
@ -94,6 +95,8 @@ static CatCTup *CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp,
uint32 hashValue, Index hashIndex, uint32 hashValue, Index hashIndex,
bool negative); bool negative);
static void ReleaseCatCacheWithOwner(HeapTuple tuple, ResourceOwner resowner);
static void ReleaseCatCacheListWithOwner(CatCList *list, ResourceOwner resowner);
static void CatCacheFreeKeys(TupleDesc tupdesc, int nkeys, int *attnos, static void CatCacheFreeKeys(TupleDesc tupdesc, int nkeys, int *attnos,
Datum *keys); Datum *keys);
static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos, static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos,
@ -104,6 +107,56 @@ static void CatCacheCopyKeys(TupleDesc tupdesc, int nkeys, int *attnos,
* internal support functions * internal support functions
*/ */
/* ResourceOwner callbacks to hold catcache references */
static void ResOwnerReleaseCatCache(Datum res);
static char *ResOwnerPrintCatCache(Datum res);
static void ResOwnerReleaseCatCacheList(Datum res);
static char *ResOwnerPrintCatCacheList(Datum res);
static const ResourceOwnerDesc catcache_resowner_desc =
{
/* catcache references */
.name = "catcache reference",
.release_phase = RESOURCE_RELEASE_AFTER_LOCKS,
.release_priority = RELEASE_PRIO_CATCACHE_REFS,
.ReleaseResource = ResOwnerReleaseCatCache,
.DebugPrint = ResOwnerPrintCatCache
};
static const ResourceOwnerDesc catlistref_resowner_desc =
{
/* catcache-list pins */
.name = "catcache list reference",
.release_phase = RESOURCE_RELEASE_AFTER_LOCKS,
.release_priority = RELEASE_PRIO_CATCACHE_LIST_REFS,
.ReleaseResource = ResOwnerReleaseCatCacheList,
.DebugPrint = ResOwnerPrintCatCacheList
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple)
{
ResourceOwnerRemember(owner, PointerGetDatum(tuple), &catcache_resowner_desc);
}
static inline void
ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple)
{
ResourceOwnerForget(owner, PointerGetDatum(tuple), &catcache_resowner_desc);
}
static inline void
ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list)
{
ResourceOwnerRemember(owner, PointerGetDatum(list), &catlistref_resowner_desc);
}
static inline void
ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list)
{
ResourceOwnerForget(owner, PointerGetDatum(list), &catlistref_resowner_desc);
}
/* /*
* Hash and equality functions for system types that are used as cache key * Hash and equality functions for system types that are used as cache key
* fields. In some cases, we just call the regular SQL-callable functions for * fields. In some cases, we just call the regular SQL-callable functions for
@ -1268,7 +1321,7 @@ SearchCatCacheInternal(CatCache *cache,
*/ */
if (!ct->negative) if (!ct->negative)
{ {
ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
ct->refcount++; ct->refcount++;
ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple);
@ -1369,7 +1422,7 @@ SearchCatCacheMiss(CatCache *cache,
hashValue, hashIndex, hashValue, hashIndex,
false); false);
/* immediately set the refcount to 1 */ /* immediately set the refcount to 1 */
ResourceOwnerEnlargeCatCacheRefs(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
ct->refcount++; ct->refcount++;
ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple); ResourceOwnerRememberCatCacheRef(CurrentResourceOwner, &ct->tuple);
break; /* assume only one match */ break; /* assume only one match */
@ -1436,6 +1489,12 @@ SearchCatCacheMiss(CatCache *cache,
*/ */
void void
ReleaseCatCache(HeapTuple tuple) ReleaseCatCache(HeapTuple tuple)
{
ReleaseCatCacheWithOwner(tuple, CurrentResourceOwner);
}
static void
ReleaseCatCacheWithOwner(HeapTuple tuple, ResourceOwner resowner)
{ {
CatCTup *ct = (CatCTup *) (((char *) tuple) - CatCTup *ct = (CatCTup *) (((char *) tuple) -
offsetof(CatCTup, tuple)); offsetof(CatCTup, tuple));
@ -1445,7 +1504,8 @@ ReleaseCatCache(HeapTuple tuple)
Assert(ct->refcount > 0); Assert(ct->refcount > 0);
ct->refcount--; ct->refcount--;
ResourceOwnerForgetCatCacheRef(CurrentResourceOwner, &ct->tuple); if (resowner)
ResourceOwnerForgetCatCacheRef(CurrentResourceOwner, &ct->tuple);
if ( if (
#ifndef CATCACHE_FORCE_RELEASE #ifndef CATCACHE_FORCE_RELEASE
@ -1581,7 +1641,7 @@ SearchCatCacheList(CatCache *cache,
dlist_move_head(&cache->cc_lists, &cl->cache_elem); dlist_move_head(&cache->cc_lists, &cl->cache_elem);
/* Bump the list's refcount and return it */ /* Bump the list's refcount and return it */
ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
cl->refcount++; cl->refcount++;
ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl); ResourceOwnerRememberCatCacheListRef(CurrentResourceOwner, cl);
@ -1605,8 +1665,6 @@ SearchCatCacheList(CatCache *cache,
* block to ensure we can undo those refcounts if we get an error before * block to ensure we can undo those refcounts if we get an error before
* we finish constructing the CatCList. * we finish constructing the CatCList.
*/ */
ResourceOwnerEnlargeCatCacheListRefs(CurrentResourceOwner);
ctlist = NIL; ctlist = NIL;
PG_TRY(); PG_TRY();
@ -1694,6 +1752,9 @@ SearchCatCacheList(CatCache *cache,
table_close(relation, AccessShareLock); table_close(relation, AccessShareLock);
/* Make sure the resource owner has room to remember this entry. */
ResourceOwnerEnlarge(CurrentResourceOwner);
/* Now we can build the CatCList entry. */ /* Now we can build the CatCList entry. */
oldcxt = MemoryContextSwitchTo(CacheMemoryContext); oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
nmembers = list_length(ctlist); nmembers = list_length(ctlist);
@ -1777,12 +1838,19 @@ SearchCatCacheList(CatCache *cache,
*/ */
void void
ReleaseCatCacheList(CatCList *list) ReleaseCatCacheList(CatCList *list)
{
ReleaseCatCacheListWithOwner(list, CurrentResourceOwner);
}
static void
ReleaseCatCacheListWithOwner(CatCList *list, ResourceOwner resowner)
{ {
/* Safety checks to ensure we were handed a cache entry */ /* Safety checks to ensure we were handed a cache entry */
Assert(list->cl_magic == CL_MAGIC); Assert(list->cl_magic == CL_MAGIC);
Assert(list->refcount > 0); Assert(list->refcount > 0);
list->refcount--; list->refcount--;
ResourceOwnerForgetCatCacheListRef(CurrentResourceOwner, list); if (resowner)
ResourceOwnerForgetCatCacheListRef(CurrentResourceOwner, list);
if ( if (
#ifndef CATCACHE_FORCE_RELEASE #ifndef CATCACHE_FORCE_RELEASE
@ -2058,31 +2126,43 @@ PrepareToInvalidateCacheTuple(Relation relation,
} }
} }
/* ResourceOwner callbacks */
/* static void
* Subroutines for warning about reference leaks. These are exported so ResOwnerReleaseCatCache(Datum res)
* that resowner.c can call them.
*/
void
PrintCatCacheLeakWarning(HeapTuple tuple)
{ {
ReleaseCatCacheWithOwner((HeapTuple) DatumGetPointer(res), NULL);
}
static char *
ResOwnerPrintCatCache(Datum res)
{
HeapTuple tuple = (HeapTuple) DatumGetPointer(res);
CatCTup *ct = (CatCTup *) (((char *) tuple) - CatCTup *ct = (CatCTup *) (((char *) tuple) -
offsetof(CatCTup, tuple)); offsetof(CatCTup, tuple));
/* Safety check to ensure we were handed a cache entry */ /* Safety check to ensure we were handed a cache entry */
Assert(ct->ct_magic == CT_MAGIC); Assert(ct->ct_magic == CT_MAGIC);
elog(WARNING, "cache reference leak: cache %s (%d), tuple %u/%u has count %d", return psprintf("cache %s (%d), tuple %u/%u has count %d",
ct->my_cache->cc_relname, ct->my_cache->id, ct->my_cache->cc_relname, ct->my_cache->id,
ItemPointerGetBlockNumber(&(tuple->t_self)), ItemPointerGetBlockNumber(&(tuple->t_self)),
ItemPointerGetOffsetNumber(&(tuple->t_self)), ItemPointerGetOffsetNumber(&(tuple->t_self)),
ct->refcount); ct->refcount);
} }
void static void
PrintCatCacheListLeakWarning(CatCList *list) ResOwnerReleaseCatCacheList(Datum res)
{ {
elog(WARNING, "cache reference leak: cache %s (%d), list %p has count %d", ReleaseCatCacheListWithOwner((CatCList *) DatumGetPointer(res), NULL);
list->my_cache->cc_relname, list->my_cache->id, }
list, list->refcount);
static char *
ResOwnerPrintCatCacheList(Datum res)
{
CatCList *list = (CatCList *) DatumGetPointer(res);
return psprintf("cache %s (%d), list %p has count %d",
list->my_cache->cc_relname, list->my_cache->id,
list, list->refcount);
} }

View File

@ -69,7 +69,7 @@
#include "tcop/utility.h" #include "tcop/utility.h"
#include "utils/inval.h" #include "utils/inval.h"
#include "utils/memutils.h" #include "utils/memutils.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
#include "utils/rls.h" #include "utils/rls.h"
#include "utils/snapmgr.h" #include "utils/snapmgr.h"
#include "utils/syscache.h" #include "utils/syscache.h"
@ -119,6 +119,31 @@ static void PlanCacheRelCallback(Datum arg, Oid relid);
static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheObjectCallback(Datum arg, int cacheid, uint32 hashvalue);
static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue); static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue);
/* ResourceOwner callbacks to track plancache references */
static void ResOwnerReleaseCachedPlan(Datum res);
static const ResourceOwnerDesc planref_resowner_desc =
{
.name = "plancache reference",
.release_phase = RESOURCE_RELEASE_AFTER_LOCKS,
.release_priority = RELEASE_PRIO_PLANCACHE_REFS,
.ReleaseResource = ResOwnerReleaseCachedPlan,
.DebugPrint = NULL /* the default message is fine */
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan)
{
ResourceOwnerRemember(owner, PointerGetDatum(plan), &planref_resowner_desc);
}
static inline void
ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan)
{
ResourceOwnerForget(owner, PointerGetDatum(plan), &planref_resowner_desc);
}
/* GUC parameter */ /* GUC parameter */
int plan_cache_mode = PLAN_CACHE_MODE_AUTO; int plan_cache_mode = PLAN_CACHE_MODE_AUTO;
@ -1233,7 +1258,7 @@ GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
/* Flag the plan as in use by caller */ /* Flag the plan as in use by caller */
if (owner) if (owner)
ResourceOwnerEnlargePlanCacheRefs(owner); ResourceOwnerEnlarge(owner);
plan->refcount++; plan->refcount++;
if (owner) if (owner)
ResourceOwnerRememberPlanCacheRef(owner, plan); ResourceOwnerRememberPlanCacheRef(owner, plan);
@ -1396,7 +1421,7 @@ CachedPlanAllowsSimpleValidityCheck(CachedPlanSource *plansource,
/* Bump refcount if requested. */ /* Bump refcount if requested. */
if (owner) if (owner)
{ {
ResourceOwnerEnlargePlanCacheRefs(owner); ResourceOwnerEnlarge(owner);
plan->refcount++; plan->refcount++;
ResourceOwnerRememberPlanCacheRef(owner, plan); ResourceOwnerRememberPlanCacheRef(owner, plan);
} }
@ -1457,7 +1482,7 @@ CachedPlanIsSimplyValid(CachedPlanSource *plansource, CachedPlan *plan,
/* It's still good. Bump refcount if requested. */ /* It's still good. Bump refcount if requested. */
if (owner) if (owner)
{ {
ResourceOwnerEnlargePlanCacheRefs(owner); ResourceOwnerEnlarge(owner);
plan->refcount++; plan->refcount++;
ResourceOwnerRememberPlanCacheRef(owner, plan); ResourceOwnerRememberPlanCacheRef(owner, plan);
} }
@ -2203,3 +2228,20 @@ ResetPlanCache(void)
cexpr->is_valid = false; cexpr->is_valid = false;
} }
} }
/*
* Release all CachedPlans remembered by 'owner'
*/
void
ReleaseAllPlanCacheRefsInOwner(ResourceOwner owner)
{
ResourceOwnerReleaseAllOfKind(owner, &planref_resowner_desc);
}
/* ResourceOwner callbacks */
static void
ResOwnerReleaseCachedPlan(Datum res)
{
ReleaseCachedPlan((CachedPlan *) DatumGetPointer(res), NULL);
}

View File

@ -80,13 +80,14 @@
#include "storage/smgr.h" #include "storage/smgr.h"
#include "utils/array.h" #include "utils/array.h"
#include "utils/builtins.h" #include "utils/builtins.h"
#include "utils/catcache.h"
#include "utils/datum.h" #include "utils/datum.h"
#include "utils/fmgroids.h" #include "utils/fmgroids.h"
#include "utils/inval.h" #include "utils/inval.h"
#include "utils/lsyscache.h" #include "utils/lsyscache.h"
#include "utils/memutils.h" #include "utils/memutils.h"
#include "utils/relmapper.h" #include "utils/relmapper.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
#include "utils/snapmgr.h" #include "utils/snapmgr.h"
#include "utils/syscache.h" #include "utils/syscache.h"
@ -273,6 +274,7 @@ static HTAB *OpClassCache = NULL;
/* non-export function prototypes */ /* non-export function prototypes */
static void RelationCloseCleanup(Relation relation);
static void RelationDestroyRelation(Relation relation, bool remember_tupdesc); static void RelationDestroyRelation(Relation relation, bool remember_tupdesc);
static void RelationClearRelation(Relation relation, bool rebuild); static void RelationClearRelation(Relation relation, bool rebuild);
@ -2115,6 +2117,31 @@ RelationIdGetRelation(Oid relationId)
* ---------------------------------------------------------------- * ----------------------------------------------------------------
*/ */
/* ResourceOwner callbacks to track relcache references */
static void ResOwnerReleaseRelation(Datum res);
static char *ResOwnerPrintRelCache(Datum res);
static const ResourceOwnerDesc relref_resowner_desc =
{
.name = "relcache reference",
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
.release_priority = RELEASE_PRIO_RELCACHE_REFS,
.ReleaseResource = ResOwnerReleaseRelation,
.DebugPrint = ResOwnerPrintRelCache
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel)
{
ResourceOwnerRemember(owner, PointerGetDatum(rel), &relref_resowner_desc);
}
static inline void
ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel)
{
ResourceOwnerForget(owner, PointerGetDatum(rel), &relref_resowner_desc);
}
/* /*
* RelationIncrementReferenceCount * RelationIncrementReferenceCount
* Increments relation reference count. * Increments relation reference count.
@ -2126,7 +2153,7 @@ RelationIdGetRelation(Oid relationId)
void void
RelationIncrementReferenceCount(Relation rel) RelationIncrementReferenceCount(Relation rel)
{ {
ResourceOwnerEnlargeRelationRefs(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
rel->rd_refcnt += 1; rel->rd_refcnt += 1;
if (!IsBootstrapProcessingMode()) if (!IsBootstrapProcessingMode())
ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel); ResourceOwnerRememberRelationRef(CurrentResourceOwner, rel);
@ -2162,6 +2189,12 @@ RelationClose(Relation relation)
/* Note: no locking manipulations needed */ /* Note: no locking manipulations needed */
RelationDecrementReferenceCount(relation); RelationDecrementReferenceCount(relation);
RelationCloseCleanup(relation);
}
static void
RelationCloseCleanup(Relation relation)
{
/* /*
* If the relation is no longer open in this session, we can clean up any * If the relation is no longer open in this session, we can clean up any
* stale partition descriptors it has. This is unlikely, so check to see * stale partition descriptors it has. This is unlikely, so check to see
@ -6813,3 +6846,30 @@ unlink_initfile(const char *initfilename, int elevel)
initfilename))); initfilename)));
} }
} }
/*
* ResourceOwner callbacks
*/
static char *
ResOwnerPrintRelCache(Datum res)
{
Relation rel = (Relation) DatumGetPointer(res);
return psprintf("relation \"%s\"", RelationGetRelationName(rel));
}
static void
ResOwnerReleaseRelation(Datum res)
{
Relation rel = (Relation) DatumGetPointer(res);
/*
* This reference has already been removed from the resource owner, so
* just decrement reference count without calling
* ResourceOwnerForgetRelationRef.
*/
Assert(rel->rd_refcnt > 0);
rel->rd_refcnt -= 1;
RelationCloseCleanup((Relation) res);
}

View File

@ -39,8 +39,8 @@ because transactions may initiate operations that require resources (such
as query parsing) when no associated Portal exists yet. as query parsing) when no associated Portal exists yet.
API Overview Usage
------------ -----
The basic operations on a ResourceOwner are: The basic operations on a ResourceOwner are:
@ -54,13 +54,6 @@ The basic operations on a ResourceOwner are:
* delete a ResourceOwner (including child owner objects); all resources * delete a ResourceOwner (including child owner objects); all resources
must have been released beforehand must have been released beforehand
This API directly supports the resource types listed in the definition of
ResourceOwnerData struct in src/backend/utils/resowner/resowner.c.
Other objects can be associated with a ResourceOwner by recording the address
of the owning ResourceOwner in such an object. There is an API for other
modules to get control during ResourceOwner release, so that they can scan
their own data structures to find the objects that need to be deleted.
Locks are handled specially because in non-error situations a lock should Locks are handled specially because in non-error situations a lock should
be held until end of transaction, even if it was originally taken by a be held until end of transaction, even if it was originally taken by a
subtransaction or portal. Therefore, the "release" operation on a child subtransaction or portal. Therefore, the "release" operation on a child
@ -79,3 +72,106 @@ CurrentResourceOwner must point to the same resource owner that was current
when the buffer, lock, or cache reference was acquired. It would be possible when the buffer, lock, or cache reference was acquired. It would be possible
to relax this restriction given additional bookkeeping effort, but at present to relax this restriction given additional bookkeeping effort, but at present
there seems no need. there seems no need.
Adding a new resource type
--------------------------
ResourceOwner can track ownership of many different kinds of resources. In
core PostgreSQL it is used for buffer pins, lmgr locks, and catalog cache
references, to name a few examples.
To add a new kind of resource, define a ResourceOwnerDesc to describe it.
For example:
static const ResourceOwnerDesc myresource_desc = {
.name = "My fancy resource",
.release_phase = RESOURCE_RELEASE_AFTER_LOCKS,
.release_priority = RELEASE_PRIO_FIRST,
.ReleaseResource = ReleaseMyResource,
.DebugPrint = PrintMyResource
};
ResourceOwnerRemember() and ResourceOwnerForget() functions take a pointer
to that struct, along with a Datum to represent the resource. The meaning
of the Datum depends on the resource type. Most resource types use it to
store a pointer to some struct, but it can also be a file descriptor or
library handle, for example.
The ReleaseResource callback is called when a resource owner is released or
deleted. It should release any resources (e.g. close files, free memory)
associated with the resource. Because the callback is called during
transaction abort, it must perform only low-level cleanup with no user
visible effects. The callback should not perform operations that could
fail, like allocate memory.
The optional DebugPrint callback is used in the warning at transaction
commit, if any resources are leaked. If not specified, a generic
implementation that prints the resource name and the resource as a pointer
is used.
There is another API for other modules to get control during ResourceOwner
release, so that they can scan their own data structures to find the objects
that need to be deleted. See RegisterResourceReleaseCallback function.
This used to be the only way for extensions to use the resource owner
mechanism with new kinds of objects; nowadays it easier to define a custom
ResourceOwnerDesc struct.
Releasing
---------
Releasing the resources of a ResourceOwner happens in three phases:
1. "Before-locks" resources
2. Locks
3. "After-locks" resources
Each resource type specifies whether it needs to be released before or after
locks. Each resource type also has a priority, which determines the order
that the resources are released in. Note that the phases are performed fully
for the whole tree of resource owners, before moving to the next phase, but
the priority within each phase only determines the order within that
ResourceOwner. Child resource owners are always handled before the parent,
within each phase.
For example, imagine that you have two ResourceOwners, parent and child,
as follows:
Parent
parent resource BEFORE_LOCKS priority 1
parent resource BEFORE_LOCKS priority 2
parent resource AFTER_LOCKS priority 10001
parent resource AFTER_LOCKS priority 10002
Child
child resource BEFORE_LOCKS priority 1
child resource BEFORE_LOCKS priority 2
child resource AFTER_LOCKS priority 10001
child resource AFTER_LOCKS priority 10002
These resources would be released in the following order:
child resource BEFORE_LOCKS priority 1
child resource BEFORE_LOCKS priority 2
parent resource BEFORE_LOCKS priority 1
parent resource BEFORE_LOCKS priority 2
(locks)
child resource AFTER_LOCKS priority 10001
child resource AFTER_LOCKS priority 10002
parent resource AFTER_LOCKS priority 10001
parent resource AFTER_LOCKS priority 10002
To release all the resources, you need to call ResourceOwnerRelease() three
times, once for each phase. You may perform additional tasks between the
phases, but after the first call to ResourceOwnerRelease(), you cannot use
the ResourceOwner to remember any more resources. You also cannot call
ResourceOwnerForget on the resource owner to release any previously
remembered resources "in retail", after you have started the release process.
Normally, you are expected to call ResourceOwnerForget on every resource so
that at commit, the ResourceOwner is empty (locks are an exception). If there
are any resources still held at commit, ResourceOwnerRelease will print a
WARNING on each such resource. At abort, however, we truly rely on the
ResourceOwner mechanism and it is normal that there are resources to be
released.

File diff suppressed because it is too large Load Diff

View File

@ -57,6 +57,7 @@
#include "lib/pairingheap.h" #include "lib/pairingheap.h"
#include "miscadmin.h" #include "miscadmin.h"
#include "port/pg_lfind.h" #include "port/pg_lfind.h"
#include "storage/fd.h"
#include "storage/predicate.h" #include "storage/predicate.h"
#include "storage/proc.h" #include "storage/proc.h"
#include "storage/procarray.h" #include "storage/procarray.h"
@ -66,7 +67,7 @@
#include "utils/builtins.h" #include "utils/builtins.h"
#include "utils/memutils.h" #include "utils/memutils.h"
#include "utils/rel.h" #include "utils/rel.h"
#include "utils/resowner_private.h" #include "utils/resowner.h"
#include "utils/snapmgr.h" #include "utils/snapmgr.h"
#include "utils/syscache.h" #include "utils/syscache.h"
#include "utils/timestamp.h" #include "utils/timestamp.h"
@ -162,9 +163,34 @@ static List *exportedSnapshots = NIL;
/* Prototypes for local functions */ /* Prototypes for local functions */
static Snapshot CopySnapshot(Snapshot snapshot); static Snapshot CopySnapshot(Snapshot snapshot);
static void UnregisterSnapshotNoOwner(Snapshot snapshot);
static void FreeSnapshot(Snapshot snapshot); static void FreeSnapshot(Snapshot snapshot);
static void SnapshotResetXmin(void); static void SnapshotResetXmin(void);
/* ResourceOwner callbacks to track snapshot references */
static void ResOwnerReleaseSnapshot(Datum res);
static const ResourceOwnerDesc snapshot_resowner_desc =
{
.name = "snapshot reference",
.release_phase = RESOURCE_RELEASE_AFTER_LOCKS,
.release_priority = RELEASE_PRIO_SNAPSHOT_REFS,
.ReleaseResource = ResOwnerReleaseSnapshot,
.DebugPrint = NULL /* the default message is fine */
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snap)
{
ResourceOwnerRemember(owner, PointerGetDatum(snap), &snapshot_resowner_desc);
}
static inline void
ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snap)
{
ResourceOwnerForget(owner, PointerGetDatum(snap), &snapshot_resowner_desc);
}
/* /*
* Snapshot fields to be serialized. * Snapshot fields to be serialized.
* *
@ -796,7 +822,7 @@ RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner)
snap = snapshot->copied ? snapshot : CopySnapshot(snapshot); snap = snapshot->copied ? snapshot : CopySnapshot(snapshot);
/* and tell resowner.c about it */ /* and tell resowner.c about it */
ResourceOwnerEnlargeSnapshots(owner); ResourceOwnerEnlarge(owner);
snap->regd_count++; snap->regd_count++;
ResourceOwnerRememberSnapshot(owner, snap); ResourceOwnerRememberSnapshot(owner, snap);
@ -832,11 +858,16 @@ UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner)
if (snapshot == NULL) if (snapshot == NULL)
return; return;
ResourceOwnerForgetSnapshot(owner, snapshot);
UnregisterSnapshotNoOwner(snapshot);
}
static void
UnregisterSnapshotNoOwner(Snapshot snapshot)
{
Assert(snapshot->regd_count > 0); Assert(snapshot->regd_count > 0);
Assert(!pairingheap_is_empty(&RegisteredSnapshots)); Assert(!pairingheap_is_empty(&RegisteredSnapshots));
ResourceOwnerForgetSnapshot(owner, snapshot);
snapshot->regd_count--; snapshot->regd_count--;
if (snapshot->regd_count == 0) if (snapshot->regd_count == 0)
pairingheap_remove(&RegisteredSnapshots, &snapshot->ph_node); pairingheap_remove(&RegisteredSnapshots, &snapshot->ph_node);
@ -1923,3 +1954,11 @@ XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot)
return false; return false;
} }
/* ResourceOwner callbacks */
static void
ResOwnerReleaseSnapshot(Datum res)
{
UnregisterSnapshotNoOwner((Snapshot) DatumGetPointer(res));
}

View File

@ -408,7 +408,7 @@ get_db_infos(ClusterInfo *cluster)
i_spclocation = PQfnumber(res, "spclocation"); i_spclocation = PQfnumber(res, "spclocation");
ntups = PQntuples(res); ntups = PQntuples(res);
dbinfos = (DbInfo *) pg_malloc(sizeof(DbInfo) * ntups); dbinfos = (DbInfo *) pg_malloc0(sizeof(DbInfo) * ntups);
for (tupnum = 0; tupnum < ntups; tupnum++) for (tupnum = 0; tupnum < ntups; tupnum++)
{ {
@ -636,15 +636,11 @@ get_old_cluster_logical_slot_infos(DbInfo *dbinfo, bool live_check)
PGconn *conn; PGconn *conn;
PGresult *res; PGresult *res;
LogicalSlotInfo *slotinfos = NULL; LogicalSlotInfo *slotinfos = NULL;
int num_slots = 0; int num_slots;
/* Logical slots can be migrated since PG17. */ /* Logical slots can be migrated since PG17. */
if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1600) if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1600)
{
dbinfo->slot_arr.slots = slotinfos;
dbinfo->slot_arr.nslots = num_slots;
return; return;
}
conn = connectToServer(&old_cluster, dbinfo->db_name); conn = connectToServer(&old_cluster, dbinfo->db_name);

View File

@ -31,7 +31,6 @@
#ifndef FRONTEND #ifndef FRONTEND
#include "utils/memutils.h" #include "utils/memutils.h"
#include "utils/resowner.h" #include "utils/resowner.h"
#include "utils/resowner_private.h"
#endif #endif
/* /*
@ -74,6 +73,32 @@ struct pg_cryptohash_ctx
#endif #endif
}; };
/* ResourceOwner callbacks to hold cryptohash contexts */
#ifndef FRONTEND
static void ResOwnerReleaseCryptoHash(Datum res);
static const ResourceOwnerDesc cryptohash_resowner_desc =
{
.name = "OpenSSL cryptohash context",
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
.release_priority = RELEASE_PRIO_CRYPTOHASH_CONTEXTS,
.ReleaseResource = ResOwnerReleaseCryptoHash,
.DebugPrint = NULL /* the default message is fine */
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberCryptoHash(ResourceOwner owner, pg_cryptohash_ctx *ctx)
{
ResourceOwnerRemember(owner, PointerGetDatum(ctx), &cryptohash_resowner_desc);
}
static inline void
ResourceOwnerForgetCryptoHash(ResourceOwner owner, pg_cryptohash_ctx *ctx)
{
ResourceOwnerForget(owner, PointerGetDatum(ctx), &cryptohash_resowner_desc);
}
#endif
static const char * static const char *
SSLerrmessage(unsigned long ecode) SSLerrmessage(unsigned long ecode)
{ {
@ -104,7 +129,7 @@ pg_cryptohash_create(pg_cryptohash_type type)
* allocation to avoid leaking. * allocation to avoid leaking.
*/ */
#ifndef FRONTEND #ifndef FRONTEND
ResourceOwnerEnlargeCryptoHash(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
#endif #endif
ctx = ALLOC(sizeof(pg_cryptohash_ctx)); ctx = ALLOC(sizeof(pg_cryptohash_ctx));
@ -138,8 +163,7 @@ pg_cryptohash_create(pg_cryptohash_type type)
#ifndef FRONTEND #ifndef FRONTEND
ctx->resowner = CurrentResourceOwner; ctx->resowner = CurrentResourceOwner;
ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ResourceOwnerRememberCryptoHash(CurrentResourceOwner, ctx);
PointerGetDatum(ctx));
#endif #endif
return ctx; return ctx;
@ -307,8 +331,8 @@ pg_cryptohash_free(pg_cryptohash_ctx *ctx)
EVP_MD_CTX_destroy(ctx->evpctx); EVP_MD_CTX_destroy(ctx->evpctx);
#ifndef FRONTEND #ifndef FRONTEND
ResourceOwnerForgetCryptoHash(ctx->resowner, if (ctx->resowner)
PointerGetDatum(ctx)); ResourceOwnerForgetCryptoHash(ctx->resowner, ctx);
#endif #endif
explicit_bzero(ctx, sizeof(pg_cryptohash_ctx)); explicit_bzero(ctx, sizeof(pg_cryptohash_ctx));
@ -351,3 +375,16 @@ pg_cryptohash_error(pg_cryptohash_ctx *ctx)
Assert(false); /* cannot be reached */ Assert(false); /* cannot be reached */
return _("success"); return _("success");
} }
/* ResourceOwner callbacks */
#ifndef FRONTEND
static void
ResOwnerReleaseCryptoHash(Datum res)
{
pg_cryptohash_ctx *ctx = (pg_cryptohash_ctx *) DatumGetPointer(res);
ctx->resowner = NULL;
pg_cryptohash_free(ctx);
}
#endif

View File

@ -31,7 +31,6 @@
#ifndef FRONTEND #ifndef FRONTEND
#include "utils/memutils.h" #include "utils/memutils.h"
#include "utils/resowner.h" #include "utils/resowner.h"
#include "utils/resowner_private.h"
#endif #endif
/* /*
@ -73,6 +72,32 @@ struct pg_hmac_ctx
#endif #endif
}; };
/* ResourceOwner callbacks to hold HMAC contexts */
#ifndef FRONTEND
static void ResOwnerReleaseHMAC(Datum res);
static const ResourceOwnerDesc hmac_resowner_desc =
{
.name = "OpenSSL HMAC context",
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
.release_priority = RELEASE_PRIO_HMAC_CONTEXTS,
.ReleaseResource = ResOwnerReleaseHMAC,
.DebugPrint = NULL /* the default message is fine */
};
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberHMAC(ResourceOwner owner, pg_hmac_ctx *ctx)
{
ResourceOwnerRemember(owner, PointerGetDatum(ctx), &hmac_resowner_desc);
}
static inline void
ResourceOwnerForgetHMAC(ResourceOwner owner, pg_hmac_ctx *ctx)
{
ResourceOwnerForget(owner, PointerGetDatum(ctx), &hmac_resowner_desc);
}
#endif
static const char * static const char *
SSLerrmessage(unsigned long ecode) SSLerrmessage(unsigned long ecode)
{ {
@ -115,7 +140,7 @@ pg_hmac_create(pg_cryptohash_type type)
ERR_clear_error(); ERR_clear_error();
#ifdef HAVE_HMAC_CTX_NEW #ifdef HAVE_HMAC_CTX_NEW
#ifndef FRONTEND #ifndef FRONTEND
ResourceOwnerEnlargeHMAC(CurrentResourceOwner); ResourceOwnerEnlarge(CurrentResourceOwner);
#endif #endif
ctx->hmacctx = HMAC_CTX_new(); ctx->hmacctx = HMAC_CTX_new();
#else #else
@ -137,7 +162,7 @@ pg_hmac_create(pg_cryptohash_type type)
#ifdef HAVE_HMAC_CTX_NEW #ifdef HAVE_HMAC_CTX_NEW
#ifndef FRONTEND #ifndef FRONTEND
ctx->resowner = CurrentResourceOwner; ctx->resowner = CurrentResourceOwner;
ResourceOwnerRememberHMAC(CurrentResourceOwner, PointerGetDatum(ctx)); ResourceOwnerRememberHMAC(CurrentResourceOwner, ctx);
#endif #endif
#else #else
memset(ctx->hmacctx, 0, sizeof(HMAC_CTX)); memset(ctx->hmacctx, 0, sizeof(HMAC_CTX));
@ -303,7 +328,8 @@ pg_hmac_free(pg_hmac_ctx *ctx)
#ifdef HAVE_HMAC_CTX_FREE #ifdef HAVE_HMAC_CTX_FREE
HMAC_CTX_free(ctx->hmacctx); HMAC_CTX_free(ctx->hmacctx);
#ifndef FRONTEND #ifndef FRONTEND
ResourceOwnerForgetHMAC(ctx->resowner, PointerGetDatum(ctx)); if (ctx->resowner)
ResourceOwnerForgetHMAC(ctx->resowner, ctx);
#endif #endif
#else #else
explicit_bzero(ctx->hmacctx, sizeof(HMAC_CTX)); explicit_bzero(ctx->hmacctx, sizeof(HMAC_CTX));
@ -346,3 +372,16 @@ pg_hmac_error(pg_hmac_ctx *ctx)
Assert(false); /* cannot be reached */ Assert(false); /* cannot be reached */
return _("success"); return _("success");
} }
/* ResourceOwner callbacks */
#ifndef FRONTEND
static void
ResOwnerReleaseHMAC(Datum res)
{
pg_hmac_ctx *ctx = (pg_hmac_ctx *) DatumGetPointer(res);
ctx->resowner = NULL;
pg_hmac_free(ctx);
}
#endif

View File

@ -101,4 +101,19 @@ murmurhash32(uint32 data)
return h; return h;
} }
/* 64-bit variant */
static inline uint64
murmurhash64(uint64 data)
{
uint64 h = data;
h ^= h >> 33;
h *= 0xff51afd7ed558ccd;
h ^= h >> 33;
h *= 0xc4ceb9fe1a85ec53;
h ^= h >> 33;
return h;
}
#endif /* HASHFN_H */ #endif /* HASHFN_H */

View File

@ -26,6 +26,7 @@
#include "storage/smgr.h" #include "storage/smgr.h"
#include "storage/spin.h" #include "storage/spin.h"
#include "utils/relcache.h" #include "utils/relcache.h"
#include "utils/resowner.h"
/* /*
* Buffer state is a single 32-bit variable where following data is combined. * Buffer state is a single 32-bit variable where following data is combined.
@ -383,6 +384,32 @@ typedef struct CkptSortItem
extern PGDLLIMPORT CkptSortItem *CkptBufferIds; extern PGDLLIMPORT CkptSortItem *CkptBufferIds;
/* ResourceOwner callbacks to hold buffer I/Os and pins */
extern const ResourceOwnerDesc buffer_io_resowner_desc;
extern const ResourceOwnerDesc buffer_pin_resowner_desc;
/* Convenience wrappers over ResourceOwnerRemember/Forget */
static inline void
ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer)
{
ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_pin_resowner_desc);
}
static inline void
ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer)
{
ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_pin_resowner_desc);
}
static inline void
ResourceOwnerRememberBufferIO(ResourceOwner owner, Buffer buffer)
{
ResourceOwnerRemember(owner, Int32GetDatum(buffer), &buffer_io_resowner_desc);
}
static inline void
ResourceOwnerForgetBufferIO(ResourceOwner owner, Buffer buffer)
{
ResourceOwnerForget(owner, Int32GetDatum(buffer), &buffer_io_resowner_desc);
}
/* /*
* Internal buffer management routines * Internal buffer management routines
*/ */
@ -418,6 +445,7 @@ extern void BufTableDelete(BufferTag *tagPtr, uint32 hashcode);
/* localbuf.c */ /* localbuf.c */
extern bool PinLocalBuffer(BufferDesc *buf_hdr, bool adjust_usagecount); extern bool PinLocalBuffer(BufferDesc *buf_hdr, bool adjust_usagecount);
extern void UnpinLocalBuffer(Buffer buffer); extern void UnpinLocalBuffer(Buffer buffer);
extern void UnpinLocalBufferNoOwner(Buffer buffer);
extern PrefetchBufferResult PrefetchLocalBuffer(SMgrRelation smgr, extern PrefetchBufferResult PrefetchLocalBuffer(SMgrRelation smgr,
ForkNumber forkNum, ForkNumber forkNum,
BlockNumber blockNum); BlockNumber blockNum);

View File

@ -207,7 +207,7 @@ extern Buffer ExtendBufferedRelTo(BufferManagerRelation bmr,
extern void InitBufferPoolAccess(void); extern void InitBufferPoolAccess(void);
extern void AtEOXact_Buffers(bool isCommit); extern void AtEOXact_Buffers(bool isCommit);
extern void PrintBufferLeakWarning(Buffer buffer); extern char *DebugPrintBufferRefcount(Buffer buffer);
extern void CheckPointBuffers(int flags); extern void CheckPointBuffers(int flags);
extern BlockNumber BufferGetBlockNumber(Buffer buffer); extern BlockNumber BufferGetBlockNumber(Buffer buffer);
extern BlockNumber RelationGetNumberOfBlocksInFork(Relation relation, extern BlockNumber RelationGetNumberOfBlocksInFork(Relation relation,
@ -248,8 +248,6 @@ extern bool ConditionalLockBufferForCleanup(Buffer buffer);
extern bool IsBufferCleanupOK(Buffer buffer); extern bool IsBufferCleanupOK(Buffer buffer);
extern bool HoldingBufferPinThatDelaysRecovery(void); extern bool HoldingBufferPinThatDelaysRecovery(void);
extern void AbortBufferIO(Buffer buffer);
extern bool BgBufferSync(struct WritebackContext *wb_context); extern bool BgBufferSync(struct WritebackContext *wb_context);
/* in buf_init.c */ /* in buf_init.c */

View File

@ -225,7 +225,4 @@ extern void PrepareToInvalidateCacheTuple(Relation relation,
HeapTuple newtuple, HeapTuple newtuple,
void (*function) (int, uint32, Oid)); void (*function) (int, uint32, Oid));
extern void PrintCatCacheLeakWarning(HeapTuple tuple);
extern void PrintCatCacheListLeakWarning(CatCList *list);
#endif /* CATCACHE_H */ #endif /* CATCACHE_H */

View File

@ -188,6 +188,8 @@ typedef struct CachedExpression
extern void InitPlanCache(void); extern void InitPlanCache(void);
extern void ResetPlanCache(void); extern void ResetPlanCache(void);
extern void ReleaseAllPlanCacheRefsInOwner(ResourceOwner owner);
extern CachedPlanSource *CreateCachedPlan(struct RawStmt *raw_parse_tree, extern CachedPlanSource *CreateCachedPlan(struct RawStmt *raw_parse_tree,
const char *query_string, const char *query_string,
CommandTag commandTag); CommandTag commandTag);

View File

@ -37,19 +37,87 @@ extern PGDLLIMPORT ResourceOwner AuxProcessResourceOwner;
/* /*
* Resource releasing is done in three phases: pre-locks, locks, and * Resource releasing is done in three phases: pre-locks, locks, and
* post-locks. The pre-lock phase must release any resources that are * post-locks. The pre-lock phase must release any resources that are visible
* visible to other backends (such as pinned buffers); this ensures that * to other backends (such as pinned buffers); this ensures that when we
* when we release a lock that another backend may be waiting on, it will * release a lock that another backend may be waiting on, it will see us as
* see us as being fully out of our transaction. The post-lock phase * being fully out of our transaction. The post-lock phase should be used for
* should be used for backend-internal cleanup. * backend-internal cleanup.
*
* Within each phase, resources are released in priority order. Priority is
* just an integer specified in ResourceOwnerDesc. The priorities of built-in
* resource types are given below, extensions may use any priority relative to
* those or RELEASE_PRIO_FIRST/LAST. RELEASE_PRIO_FIRST is a fine choice if
* your resource doesn't depend on any other resources.
*/ */
typedef enum typedef enum
{ {
RESOURCE_RELEASE_BEFORE_LOCKS, RESOURCE_RELEASE_BEFORE_LOCKS = 1,
RESOURCE_RELEASE_LOCKS, RESOURCE_RELEASE_LOCKS,
RESOURCE_RELEASE_AFTER_LOCKS, RESOURCE_RELEASE_AFTER_LOCKS,
} ResourceReleasePhase; } ResourceReleasePhase;
typedef uint32 ResourceReleasePriority;
/* priorities of built-in BEFORE_LOCKS resources */
#define RELEASE_PRIO_BUFFER_IOS 100
#define RELEASE_PRIO_BUFFER_PINS 200
#define RELEASE_PRIO_RELCACHE_REFS 300
#define RELEASE_PRIO_DSMS 400
#define RELEASE_PRIO_JIT_CONTEXTS 500
#define RELEASE_PRIO_CRYPTOHASH_CONTEXTS 600
#define RELEASE_PRIO_HMAC_CONTEXTS 700
/* priorities of built-in AFTER_LOCKS resources */
#define RELEASE_PRIO_CATCACHE_REFS 100
#define RELEASE_PRIO_CATCACHE_LIST_REFS 200
#define RELEASE_PRIO_PLANCACHE_REFS 300
#define RELEASE_PRIO_TUPDESC_REFS 400
#define RELEASE_PRIO_SNAPSHOT_REFS 500
#define RELEASE_PRIO_FILES 600
/* 0 is considered invalid */
#define RELEASE_PRIO_FIRST 1
#define RELEASE_PRIO_LAST UINT32_MAX
/*
* In order to track an object, resowner.c needs a few callbacks for it.
* The callbacks for resources of a specific kind are encapsulated in
* ResourceOwnerDesc.
*
* Note that the callbacks occur post-commit or post-abort, so the callback
* functions can only do noncritical cleanup and must not fail.
*/
typedef struct ResourceOwnerDesc
{
const char *name; /* name for the object kind, for debugging */
/* when are these objects released? */
ResourceReleasePhase release_phase;
ResourceReleasePriority release_priority;
/*
* Release resource.
*
* This is called for each resource in the resource owner, in the order
* specified by 'release_phase' and 'release_priority' when the whole
* resource owner is been released or when ResourceOwnerReleaseAllOfKind()
* is called. The resource is implicitly removed from the owner, the
* callback function doesn't need to call ResourceOwnerForget.
*/
void (*ReleaseResource) (Datum res);
/*
* Format a string describing the resource, for debugging purposes. If a
* resource has not been properly released before commit, this is used to
* print a WARNING.
*
* This can be left to NULL, in which case a generic "[resource name]: %p"
* format is used.
*/
char *(*DebugPrint) (Datum res);
} ResourceOwnerDesc;
/* /*
* Dynamically loaded modules can get control during ResourceOwnerRelease * Dynamically loaded modules can get control during ResourceOwnerRelease
* by providing a callback of this form. * by providing a callback of this form.
@ -71,16 +139,28 @@ extern void ResourceOwnerRelease(ResourceOwner owner,
ResourceReleasePhase phase, ResourceReleasePhase phase,
bool isCommit, bool isCommit,
bool isTopLevel); bool isTopLevel);
extern void ResourceOwnerReleaseAllPlanCacheRefs(ResourceOwner owner);
extern void ResourceOwnerDelete(ResourceOwner owner); extern void ResourceOwnerDelete(ResourceOwner owner);
extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner); extern ResourceOwner ResourceOwnerGetParent(ResourceOwner owner);
extern void ResourceOwnerNewParent(ResourceOwner owner, extern void ResourceOwnerNewParent(ResourceOwner owner,
ResourceOwner newparent); ResourceOwner newparent);
extern void ResourceOwnerEnlarge(ResourceOwner owner);
extern void ResourceOwnerRemember(ResourceOwner owner, Datum res, const ResourceOwnerDesc *kind);
extern void ResourceOwnerForget(ResourceOwner owner, Datum res, const ResourceOwnerDesc *kind);
extern void ResourceOwnerReleaseAllOfKind(ResourceOwner owner, const ResourceOwnerDesc *kind);
extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback, extern void RegisterResourceReleaseCallback(ResourceReleaseCallback callback,
void *arg); void *arg);
extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, extern void UnregisterResourceReleaseCallback(ResourceReleaseCallback callback,
void *arg); void *arg);
extern void CreateAuxProcessResourceOwner(void); extern void CreateAuxProcessResourceOwner(void);
extern void ReleaseAuxProcessResources(bool isCommit); extern void ReleaseAuxProcessResources(bool isCommit);
/* special support for local lock management */
struct LOCALLOCK;
extern void ResourceOwnerRememberLock(ResourceOwner owner, struct LOCALLOCK *locallock);
extern void ResourceOwnerForgetLock(ResourceOwner owner, struct LOCALLOCK *locallock);
#endif /* RESOWNER_H */ #endif /* RESOWNER_H */

View File

@ -3316,6 +3316,14 @@ PQsendFlushRequest(PGconn *conn)
return 0; return 0;
} }
/*
* Give the data a push (in pipeline mode, only if we're past the size
* threshold). In nonblock mode, don't complain if we're unable to send
* it all; PQgetResult() will do any additional flushing needed.
*/
if (pqPipelineFlush(conn) < 0)
return 0;
return 1; return 1;
} }

View File

@ -8470,7 +8470,7 @@ plpgsql_xact_cb(XactEvent event, void *arg)
FreeExecutorState(shared_simple_eval_estate); FreeExecutorState(shared_simple_eval_estate);
shared_simple_eval_estate = NULL; shared_simple_eval_estate = NULL;
if (shared_simple_eval_resowner) if (shared_simple_eval_resowner)
ResourceOwnerReleaseAllPlanCacheRefs(shared_simple_eval_resowner); ReleaseAllPlanCacheRefsInOwner(shared_simple_eval_resowner);
shared_simple_eval_resowner = NULL; shared_simple_eval_resowner = NULL;
} }
else if (event == XACT_EVENT_ABORT || else if (event == XACT_EVENT_ABORT ||

View File

@ -288,7 +288,7 @@ plpgsql_call_handler(PG_FUNCTION_ARGS)
/* Be sure to release the procedure resowner if any */ /* Be sure to release the procedure resowner if any */
if (procedure_resowner) if (procedure_resowner)
{ {
ResourceOwnerReleaseAllPlanCacheRefs(procedure_resowner); ReleaseAllPlanCacheRefsInOwner(procedure_resowner);
ResourceOwnerDelete(procedure_resowner); ResourceOwnerDelete(procedure_resowner);
} }
} }
@ -393,7 +393,7 @@ plpgsql_inline_handler(PG_FUNCTION_ARGS)
/* Clean up the private EState and resowner */ /* Clean up the private EState and resowner */
FreeExecutorState(simple_eval_estate); FreeExecutorState(simple_eval_estate);
ResourceOwnerReleaseAllPlanCacheRefs(simple_eval_resowner); ReleaseAllPlanCacheRefsInOwner(simple_eval_resowner);
ResourceOwnerDelete(simple_eval_resowner); ResourceOwnerDelete(simple_eval_resowner);
/* Function should now have no remaining use-counts ... */ /* Function should now have no remaining use-counts ... */
@ -410,7 +410,7 @@ plpgsql_inline_handler(PG_FUNCTION_ARGS)
/* Clean up the private EState and resowner */ /* Clean up the private EState and resowner */
FreeExecutorState(simple_eval_estate); FreeExecutorState(simple_eval_estate);
ResourceOwnerReleaseAllPlanCacheRefs(simple_eval_resowner); ReleaseAllPlanCacheRefsInOwner(simple_eval_resowner);
ResourceOwnerDelete(simple_eval_resowner); ResourceOwnerDelete(simple_eval_resowner);
/* Function should now have no remaining use-counts ... */ /* Function should now have no remaining use-counts ... */

View File

@ -28,6 +28,7 @@ SUBDIRS = \
test_predtest \ test_predtest \
test_rbtree \ test_rbtree \
test_regex \ test_regex \
test_resowner \
test_rls_hooks \ test_rls_hooks \
test_shm_mq \ test_shm_mq \
test_slru \ test_slru \

View File

@ -14,7 +14,7 @@ endif
ldap_password_func = shared_module('ldap_password_func', ldap_password_func = shared_module('ldap_password_func',
ldap_password_func_sources, ldap_password_func_sources,
kwargs: pg_mod_args + { kwargs: pg_test_mod_args + {
'dependencies': [ldap, pg_mod_args['dependencies']], 'dependencies': [ldap, pg_mod_args['dependencies']],
}, },
) )

View File

@ -25,6 +25,7 @@ subdir('test_pg_dump')
subdir('test_predtest') subdir('test_predtest')
subdir('test_rbtree') subdir('test_rbtree')
subdir('test_regex') subdir('test_regex')
subdir('test_resowner')
subdir('test_rls_hooks') subdir('test_rls_hooks')
subdir('test_shm_mq') subdir('test_shm_mq')
subdir('test_slru') subdir('test_slru')

View File

@ -0,0 +1,4 @@
# Generated subdirectories
/log/
/results/
/tmp_check/

View File

@ -0,0 +1,24 @@
# src/test/modules/test_resowner/Makefile
MODULE_big = test_resowner
OBJS = \
$(WIN32RES) \
test_resowner_basic.o \
test_resowner_many.o
PGFILEDESC = "test_resowner - test code for ResourceOwners"
EXTENSION = test_resowner
DATA = test_resowner--1.0.sql
REGRESS = test_resowner
ifdef USE_PGXS
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
else
subdir = src/test/modules/test_resowner
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
include $(top_srcdir)/contrib/contrib-global.mk
endif

View File

@ -0,0 +1,197 @@
CREATE EXTENSION test_resowner;
-- This is small enough that everything fits in the small array
SELECT test_resowner_priorities(2, 3);
NOTICE: releasing resources before locks
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing locks
NOTICE: releasing resources after locks
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 2
test_resowner_priorities
--------------------------
(1 row)
-- Same test with more resources, to exercise the hash table
SELECT test_resowner_priorities(2, 32);
NOTICE: releasing resources before locks
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 1
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: child before locks priority 2
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 1
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing string: parent before locks priority 2
NOTICE: releasing locks
NOTICE: releasing resources after locks
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 1
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: child after locks priority 2
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 1
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
NOTICE: releasing string: parent after locks priority 2
test_resowner_priorities
--------------------------
(1 row)
-- Basic test with lots more resources, to test extending the hash table
SELECT test_resowner_many(
3, -- # of different resource kinds
100000, -- before-locks resources to remember
500, -- before-locks resources to forget
100000, -- after-locks resources to remember
500 -- after-locks resources to forget
);
NOTICE: remembering 100000 before-locks resources
NOTICE: remembering 100000 after-locks resources
NOTICE: forgetting 500 before-locks resources
NOTICE: forgetting 500 after-locks resources
NOTICE: releasing resources before locks
NOTICE: releasing locks
NOTICE: releasing resources after locks
test_resowner_many
--------------------
(1 row)
-- Test resource leak warning
SELECT test_resowner_leak();
WARNING: resource was not closed: test string "my string"
NOTICE: releasing string: my string
test_resowner_leak
--------------------
(1 row)
-- Negative tests, using a resource owner after release-phase has started.
set client_min_messages='warning'; -- order between ERROR and NOTICE varies
SELECT test_resowner_remember_between_phases();
ERROR: ResourceOwnerEnlarge called after release started
SELECT test_resowner_forget_between_phases();
ERROR: ResourceOwnerForget called for test resource after release started
reset client_min_messages;

View File

@ -0,0 +1,34 @@
# Copyright (c) 2022-2023, PostgreSQL Global Development Group
test_resowner_sources = files(
'test_resowner_basic.c',
'test_resowner_many.c',
)
if host_system == 'windows'
test_resowner_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
'--NAME', 'test_resowner',
'--FILEDESC', 'test_resowner - test code for ResourceOwners',])
endif
test_resowner = shared_module('test_resowner',
test_resowner_sources,
kwargs: pg_test_mod_args,
)
test_install_libs += test_resowner
test_install_data += files(
'test_resowner.control',
'test_resowner--1.0.sql',
)
tests += {
'name': 'test_resowner',
'sd': meson.current_source_dir(),
'bd': meson.current_build_dir(),
'regress': {
'sql': [
'test_resowner',
],
},
}

View File

@ -0,0 +1,25 @@
CREATE EXTENSION test_resowner;
-- This is small enough that everything fits in the small array
SELECT test_resowner_priorities(2, 3);
-- Same test with more resources, to exercise the hash table
SELECT test_resowner_priorities(2, 32);
-- Basic test with lots more resources, to test extending the hash table
SELECT test_resowner_many(
3, -- # of different resource kinds
100000, -- before-locks resources to remember
500, -- before-locks resources to forget
100000, -- after-locks resources to remember
500 -- after-locks resources to forget
);
-- Test resource leak warning
SELECT test_resowner_leak();
-- Negative tests, using a resource owner after release-phase has started.
set client_min_messages='warning'; -- order between ERROR and NOTICE varies
SELECT test_resowner_remember_between_phases();
SELECT test_resowner_forget_between_phases();
reset client_min_messages;

View File

@ -0,0 +1,30 @@
/* src/test/modules/test_resowner/test_resowner--1.0.sql */
-- complain if script is sourced in psql, rather than via CREATE EXTENSION
\echo Use "CREATE EXTENSION test_resowner" to load this file. \quit
CREATE FUNCTION test_resowner_priorities(nkinds pg_catalog.int4, nresources pg_catalog.int4)
RETURNS pg_catalog.void
AS 'MODULE_PATHNAME' LANGUAGE C;
CREATE FUNCTION test_resowner_leak()
RETURNS pg_catalog.void
AS 'MODULE_PATHNAME' LANGUAGE C;
CREATE FUNCTION test_resowner_remember_between_phases()
RETURNS pg_catalog.void
AS 'MODULE_PATHNAME' LANGUAGE C;
CREATE FUNCTION test_resowner_forget_between_phases()
RETURNS pg_catalog.void
AS 'MODULE_PATHNAME' LANGUAGE C;
CREATE FUNCTION test_resowner_many(
nkinds pg_catalog.int4,
nremember_bl pg_catalog.int4,
nforget_bl pg_catalog.int4,
nremember_al pg_catalog.int4,
nforget_al pg_catalog.int4
)
RETURNS pg_catalog.void
AS 'MODULE_PATHNAME' LANGUAGE C;

View File

@ -0,0 +1,4 @@
comment = 'Test code for ResourceOwners'
default_version = '1.0'
module_pathname = '$libdir/test_resowner'
relocatable = true

View File

@ -0,0 +1,211 @@
/*--------------------------------------------------------------------------
*
* test_resowner_basic.c
* Test basic ResourceOwner functionality
*
* Copyright (c) 2022-2023, PostgreSQL Global Development Group
*
* IDENTIFICATION
* src/test/modules/test_resowner/test_resowner_basic.c
*
* -------------------------------------------------------------------------
*/
#include "postgres.h"
#include "fmgr.h"
#include "lib/ilist.h"
#include "utils/memutils.h"
#include "utils/resowner.h"
PG_MODULE_MAGIC;
static void ReleaseString(Datum res);
static char *PrintString(Datum res);
/*
* A resource that tracks strings and prints the string when it's released.
* This makes the order that the resources are released visible.
*/
static const ResourceOwnerDesc string_desc = {
.name = "test resource",
.release_phase = RESOURCE_RELEASE_AFTER_LOCKS,
.release_priority = RELEASE_PRIO_FIRST,
.ReleaseResource = ReleaseString,
.DebugPrint = PrintString
};
static void
ReleaseString(Datum res)
{
elog(NOTICE, "releasing string: %s", DatumGetPointer(res));
}
static char *
PrintString(Datum res)
{
return psprintf("test string \"%s\"", DatumGetPointer(res));
}
/* demonstrates phases and priorities between a parent and child context */
PG_FUNCTION_INFO_V1(test_resowner_priorities);
Datum
test_resowner_priorities(PG_FUNCTION_ARGS)
{
int32 nkinds = PG_GETARG_INT32(0);
int32 nresources = PG_GETARG_INT32(1);
ResourceOwner parent,
child;
ResourceOwnerDesc *before_desc;
ResourceOwnerDesc *after_desc;
if (nkinds <= 0)
elog(ERROR, "nkinds must be greater than zero");
if (nresources <= 0)
elog(ERROR, "nresources must be greater than zero");
parent = ResourceOwnerCreate(CurrentResourceOwner, "test parent");
child = ResourceOwnerCreate(parent, "test child");
before_desc = palloc(nkinds * sizeof(ResourceOwnerDesc));
for (int i = 0; i < nkinds; i++)
{
before_desc[i].name = psprintf("test resource before locks %d", i);
before_desc[i].release_phase = RESOURCE_RELEASE_BEFORE_LOCKS;
before_desc[i].release_priority = RELEASE_PRIO_FIRST + i;
before_desc[i].ReleaseResource = ReleaseString;
before_desc[i].DebugPrint = PrintString;
}
after_desc = palloc(nkinds * sizeof(ResourceOwnerDesc));
for (int i = 0; i < nkinds; i++)
{
after_desc[i].name = psprintf("test resource after locks %d", i);
after_desc[i].release_phase = RESOURCE_RELEASE_AFTER_LOCKS;
after_desc[i].release_priority = RELEASE_PRIO_FIRST + i;
after_desc[i].ReleaseResource = ReleaseString;
after_desc[i].DebugPrint = PrintString;
}
/* Add a bunch of resources to child, with different priorities */
for (int i = 0; i < nresources; i++)
{
ResourceOwnerDesc *kind = &before_desc[i % nkinds];
ResourceOwnerEnlarge(child);
ResourceOwnerRemember(child,
CStringGetDatum(psprintf("child before locks priority %d", kind->release_priority)),
kind);
}
for (int i = 0; i < nresources; i++)
{
ResourceOwnerDesc *kind = &after_desc[i % nkinds];
ResourceOwnerEnlarge(child);
ResourceOwnerRemember(child,
CStringGetDatum(psprintf("child after locks priority %d", kind->release_priority)),
kind);
}
/* And also to the parent */
for (int i = 0; i < nresources; i++)
{
ResourceOwnerDesc *kind = &after_desc[i % nkinds];
ResourceOwnerEnlarge(parent);
ResourceOwnerRemember(parent,
CStringGetDatum(psprintf("parent after locks priority %d", kind->release_priority)),
kind);
}
for (int i = 0; i < nresources; i++)
{
ResourceOwnerDesc *kind = &before_desc[i % nkinds];
ResourceOwnerEnlarge(parent);
ResourceOwnerRemember(parent,
CStringGetDatum(psprintf("parent before locks priority %d", kind->release_priority)),
kind);
}
elog(NOTICE, "releasing resources before locks");
ResourceOwnerRelease(parent, RESOURCE_RELEASE_BEFORE_LOCKS, false, false);
elog(NOTICE, "releasing locks");
ResourceOwnerRelease(parent, RESOURCE_RELEASE_LOCKS, false, false);
elog(NOTICE, "releasing resources after locks");
ResourceOwnerRelease(parent, RESOURCE_RELEASE_AFTER_LOCKS, false, false);
ResourceOwnerDelete(parent);
PG_RETURN_VOID();
}
PG_FUNCTION_INFO_V1(test_resowner_leak);
Datum
test_resowner_leak(PG_FUNCTION_ARGS)
{
ResourceOwner resowner;
resowner = ResourceOwnerCreate(CurrentResourceOwner, "TestOwner");
ResourceOwnerEnlarge(resowner);
ResourceOwnerRemember(resowner, CStringGetDatum("my string"), &string_desc);
/* don't call ResourceOwnerForget, so that it is leaked */
ResourceOwnerRelease(resowner, RESOURCE_RELEASE_BEFORE_LOCKS, true, false);
ResourceOwnerRelease(resowner, RESOURCE_RELEASE_LOCKS, true, false);
ResourceOwnerRelease(resowner, RESOURCE_RELEASE_AFTER_LOCKS, true, false);
ResourceOwnerDelete(resowner);
PG_RETURN_VOID();
}
PG_FUNCTION_INFO_V1(test_resowner_remember_between_phases);
Datum
test_resowner_remember_between_phases(PG_FUNCTION_ARGS)
{
ResourceOwner resowner;
resowner = ResourceOwnerCreate(CurrentResourceOwner, "TestOwner");
ResourceOwnerRelease(resowner, RESOURCE_RELEASE_BEFORE_LOCKS, true, false);
/*
* Try to remember a new resource. Fails because we already called
* ResourceOwnerRelease.
*/
ResourceOwnerEnlarge(resowner);
ResourceOwnerRemember(resowner, CStringGetDatum("my string"), &string_desc);
/* unreachable */
elog(ERROR, "ResourceOwnerEnlarge should have errored out");
PG_RETURN_VOID();
}
PG_FUNCTION_INFO_V1(test_resowner_forget_between_phases);
Datum
test_resowner_forget_between_phases(PG_FUNCTION_ARGS)
{
ResourceOwner resowner;
Datum str_resource;
resowner = ResourceOwnerCreate(CurrentResourceOwner, "TestOwner");
ResourceOwnerEnlarge(resowner);
str_resource = CStringGetDatum("my string");
ResourceOwnerRemember(resowner, str_resource, &string_desc);
ResourceOwnerRelease(resowner, RESOURCE_RELEASE_BEFORE_LOCKS, true, false);
/*
* Try to forget the resource that was remembered earlier. Fails because
* we already called ResourceOwnerRelease.
*/
ResourceOwnerForget(resowner, str_resource, &string_desc);
/* unreachable */
elog(ERROR, "ResourceOwnerForget should have errored out");
PG_RETURN_VOID();
}

View File

@ -0,0 +1,296 @@
/*--------------------------------------------------------------------------
*
* test_resowner_many.c
* Test ResourceOwner functionality with lots of resources
*
* Copyright (c) 2022-2023, PostgreSQL Global Development Group
*
* IDENTIFICATION
* src/test/modules/test_resowner/test_resowner_many.c
*
* -------------------------------------------------------------------------
*/
#include "postgres.h"
#include "fmgr.h"
#include "lib/ilist.h"
#include "utils/memutils.h"
#include "utils/resowner.h"
/*
* Define a custom resource type to use in the test. The resource being
* tracked is a palloc'd ManyTestResource struct.
*
* To cross-check that the ResourceOwner calls the callback functions
* correctly, we keep track of the remembered resources ourselves in a linked
* list, and also keep counters of how many times the callback functions have
* been called.
*/
typedef struct
{
ResourceOwnerDesc desc;
int nremembered;
int nforgotten;
int nreleased;
int nleaked;
dlist_head current_resources;
} ManyTestResourceKind;
typedef struct
{
ManyTestResourceKind *kind;
dlist_node node;
} ManyTestResource;
/*
* Current release phase, and priority of last call to the release callback.
* This is used to check that the resources are released in correct order.
*/
static ResourceReleasePhase current_release_phase;
static uint32 last_release_priority = 0;
/* prototypes for local functions */
static void ReleaseManyTestResource(Datum res);
static char *PrintManyTest(Datum res);
static void InitManyTestResourceKind(ManyTestResourceKind *kind, char *name,
ResourceReleasePhase phase, uint32 priority);
static void RememberManyTestResources(ResourceOwner owner,
ManyTestResourceKind *kinds, int nkinds,
int nresources);
static void ForgetManyTestResources(ResourceOwner owner,
ManyTestResourceKind *kinds, int nkinds,
int nresources);
static int GetTotalResourceCount(ManyTestResourceKind *kinds, int nkinds);
/* ResourceOwner callback */
static void
ReleaseManyTestResource(Datum res)
{
ManyTestResource *mres = (ManyTestResource *) DatumGetPointer(res);
elog(DEBUG1, "releasing resource %p from %s", mres, mres->kind->desc.name);
Assert(last_release_priority <= mres->kind->desc.release_priority);
dlist_delete(&mres->node);
mres->kind->nreleased++;
last_release_priority = mres->kind->desc.release_priority;
pfree(mres);
}
/* ResourceOwner callback */
static char *
PrintManyTest(Datum res)
{
ManyTestResource *mres = (ManyTestResource *) DatumGetPointer(res);
/*
* XXX: we assume that the DebugPrint function is called once for each
* leaked resource, and that there are no other callers.
*/
mres->kind->nleaked++;
return psprintf("many-test resource from %s", mres->kind->desc.name);
}
static void
InitManyTestResourceKind(ManyTestResourceKind *kind, char *name,
ResourceReleasePhase phase, uint32 priority)
{
kind->desc.name = name;
kind->desc.release_phase = phase;
kind->desc.release_priority = priority;
kind->desc.ReleaseResource = ReleaseManyTestResource;
kind->desc.DebugPrint = PrintManyTest;
kind->nremembered = 0;
kind->nforgotten = 0;
kind->nreleased = 0;
kind->nleaked = 0;
dlist_init(&kind->current_resources);
}
/*
* Remember 'nresources' resources. The resources are remembered in round
* robin fashion with the kinds from 'kinds' array.
*/
static void
RememberManyTestResources(ResourceOwner owner,
ManyTestResourceKind *kinds, int nkinds,
int nresources)
{
int kind_idx = 0;
for (int i = 0; i < nresources; i++)
{
ManyTestResource *mres = palloc(sizeof(ManyTestResource));
mres->kind = &kinds[kind_idx];
dlist_node_init(&mres->node);
ResourceOwnerEnlarge(owner);
ResourceOwnerRemember(owner, PointerGetDatum(mres), &kinds[kind_idx].desc);
kinds[kind_idx].nremembered++;
dlist_push_tail(&kinds[kind_idx].current_resources, &mres->node);
elog(DEBUG1, "remembered resource %p from %s", mres, mres->kind->desc.name);
kind_idx = (kind_idx + 1) % nkinds;
}
}
/*
* Forget 'nresources' resources, in round robin fashion from 'kinds'.
*/
static void
ForgetManyTestResources(ResourceOwner owner,
ManyTestResourceKind *kinds, int nkinds,
int nresources)
{
int kind_idx = 0;
int ntotal;
ntotal = GetTotalResourceCount(kinds, nkinds);
if (ntotal < nresources)
elog(PANIC, "cannot free %d resources, only %d remembered", nresources, ntotal);
for (int i = 0; i < nresources; i++)
{
bool found = false;
for (int j = 0; j < nkinds; j++)
{
kind_idx = (kind_idx + 1) % nkinds;
if (!dlist_is_empty(&kinds[kind_idx].current_resources))
{
ManyTestResource *mres = dlist_head_element(ManyTestResource, node, &kinds[kind_idx].current_resources);
ResourceOwnerForget(owner, PointerGetDatum(mres), &kinds[kind_idx].desc);
kinds[kind_idx].nforgotten++;
dlist_delete(&mres->node);
pfree(mres);
found = true;
break;
}
}
if (!found)
elog(ERROR, "could not find a test resource to forget");
}
}
/*
* Get total number of currently active resources among 'kinds'.
*/
static int
GetTotalResourceCount(ManyTestResourceKind *kinds, int nkinds)
{
int ntotal = 0;
for (int i = 0; i < nkinds; i++)
ntotal += kinds[i].nremembered - kinds[i].nforgotten - kinds[i].nreleased;
return ntotal;
}
/*
* Remember lots of resources, belonging to 'nkinds' different resource types
* with different priorities. Then forget some of them, and finally, release
* the resource owner. We use a custom resource type that performs various
* sanity checks to verify that all the the resources are released, and in the
* correct order.
*/
PG_FUNCTION_INFO_V1(test_resowner_many);
Datum
test_resowner_many(PG_FUNCTION_ARGS)
{
int32 nkinds = PG_GETARG_INT32(0);
int32 nremember_bl = PG_GETARG_INT32(1);
int32 nforget_bl = PG_GETARG_INT32(2);
int32 nremember_al = PG_GETARG_INT32(3);
int32 nforget_al = PG_GETARG_INT32(4);
ResourceOwner resowner;
ManyTestResourceKind *before_kinds;
ManyTestResourceKind *after_kinds;
/* Sanity check the arguments */
if (nkinds < 0)
elog(ERROR, "nkinds must be >= 0");
if (nremember_bl < 0)
elog(ERROR, "nremember_bl must be >= 0");
if (nforget_bl < 0 || nforget_bl > nremember_bl)
elog(ERROR, "nforget_bl must between 0 and 'nremember_bl'");
if (nremember_al < 0)
elog(ERROR, "nremember_al must be greater than zero");
if (nforget_al < 0 || nforget_al > nremember_al)
elog(ERROR, "nforget_al must between 0 and 'nremember_al'");
/* Initialize all the different resource kinds to use */
before_kinds = palloc(nkinds * sizeof(ManyTestResourceKind));
for (int i = 0; i < nkinds; i++)
{
InitManyTestResourceKind(&before_kinds[i],
psprintf("resource before locks %d", i),
RESOURCE_RELEASE_BEFORE_LOCKS,
RELEASE_PRIO_FIRST + i);
}
after_kinds = palloc(nkinds * sizeof(ManyTestResourceKind));
for (int i = 0; i < nkinds; i++)
{
InitManyTestResourceKind(&after_kinds[i],
psprintf("resource after locks %d", i),
RESOURCE_RELEASE_AFTER_LOCKS,
RELEASE_PRIO_FIRST + i);
}
resowner = ResourceOwnerCreate(CurrentResourceOwner, "TestOwner");
/* Remember a bunch of resources */
if (nremember_bl > 0)
{
elog(NOTICE, "remembering %d before-locks resources", nremember_bl);
RememberManyTestResources(resowner, before_kinds, nkinds, nremember_bl);
}
if (nremember_al > 0)
{
elog(NOTICE, "remembering %d after-locks resources", nremember_al);
RememberManyTestResources(resowner, after_kinds, nkinds, nremember_al);
}
/* Forget what was remembered */
if (nforget_bl > 0)
{
elog(NOTICE, "forgetting %d before-locks resources", nforget_bl);
ForgetManyTestResources(resowner, before_kinds, nkinds, nforget_bl);
}
if (nforget_al > 0)
{
elog(NOTICE, "forgetting %d after-locks resources", nforget_al);
ForgetManyTestResources(resowner, after_kinds, nkinds, nforget_al);
}
/* Start releasing */
elog(NOTICE, "releasing resources before locks");
current_release_phase = RESOURCE_RELEASE_BEFORE_LOCKS;
last_release_priority = 0;
ResourceOwnerRelease(resowner, RESOURCE_RELEASE_BEFORE_LOCKS, false, false);
Assert(GetTotalResourceCount(before_kinds, nkinds) == 0);
elog(NOTICE, "releasing locks");
current_release_phase = RESOURCE_RELEASE_LOCKS;
last_release_priority = 0;
ResourceOwnerRelease(resowner, RESOURCE_RELEASE_LOCKS, false, false);
elog(NOTICE, "releasing resources after locks");
current_release_phase = RESOURCE_RELEASE_AFTER_LOCKS;
last_release_priority = 0;
ResourceOwnerRelease(resowner, RESOURCE_RELEASE_AFTER_LOCKS, false, false);
Assert(GetTotalResourceCount(before_kinds, nkinds) == 0);
Assert(GetTotalResourceCount(after_kinds, nkinds) == 0);
ResourceOwnerDelete(resowner);
PG_RETURN_VOID();
}

View File

@ -7,7 +7,7 @@ tests += {
'tap': { 'tap': {
'env': { 'env': {
'with_ssl': ssl_library, 'with_ssl': ssl_library,
'OPENSSL': openssl.path(), 'OPENSSL': openssl.found() ? openssl.path() : '',
}, },
'tests': [ 'tests': [
't/001_ssltests.pl', 't/001_ssltests.pl',

View File

@ -1527,6 +1527,8 @@ MVDependencies
MVDependency MVDependency
MVNDistinct MVNDistinct
MVNDistinctItem MVNDistinctItem
ManyTestResource
ManyTestResourceKind
Material Material
MaterialPath MaterialPath
MaterialState MaterialState
@ -2359,8 +2361,10 @@ ReplicationStateOnDisk
ResTarget ResTarget
ReservoirState ReservoirState
ReservoirStateData ReservoirStateData
ResourceArray ResourceElem
ResourceOwner ResourceOwner
ResourceOwnerData
ResourceOwnerDesc
ResourceReleaseCallback ResourceReleaseCallback
ResourceReleaseCallbackItem ResourceReleaseCallbackItem
ResourceReleasePhase ResourceReleasePhase