Document new statement reset behavior #388
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,67 @@ | ||||||
| --- | ||||||
| title: Query text unavailable and reducing memory usage | ||||||
| backlink_href: /docs/install/troubleshooting | ||||||
| backlink_title: 'Installation Troubleshooting' | ||||||
| --- | ||||||
|
|
||||||
| If the collector is using too much memory or `<query text unavailable>` is showing up in pganalyze, | ||||||
| you may want to set up an automatic reset of `pg_stat_statements`. | ||||||
|
|
||||||
| ## <query text unavailable> | ||||||
|
|
||||||
| Earlier versions of Postgres aren't able to group certain queries together (e.g. those using `IN` | ||||||
| lists of variable lengths), leading to many duplicate entries that often have very low call counts. | ||||||
| When `pg_stat_statements.max` is reached, Postgres will deallocate (remove) entries based on their | ||||||
| usage, which is based on the call count. In periods of high database activity this can happen so | ||||||
| quickly that the collector isn't able to capture the query text, causing `<query text unavailable>` | ||||||
| to show up in pganalyze. | ||||||
|
|
||||||
| This can be addressed by enabling the "Maximum Query Stats Reset Frequency" setting in pganalyze. | ||||||
| We recommend a setting of "Every 1 hour" unless you're specifically concerned about lock contention | ||||||
| (`LWLock:pg_stat_statements`) on a very busy database. This enables automatic pg_stat_statments | ||||||
| resets once the statement space is 90% full (so a deallocation would occur soon), but resets will | ||||||
| not occur more often than the frequency setting you choose. | ||||||
|
|
||||||
| ## High memory usage | ||||||
|
|
||||||
| Every 10 minutes, the collector loads the query text from pg_stat_statements into memory in order | ||||||
| to normalize and fingerprint it. This is a very expensive step and can result in the collector | ||||||
| crashing from running out of memory. To address that, when resets are enabled with the "Maximum | ||||||
| Query Stats Reset Frequency" setting, the collector will also perform a reset if the aggregate | ||||||
| size of the query text returned from pg_stat_statements grows beyond 250 MB. You can optionally | ||||||
| increase that limit using the "Maximum Query Size Before Next Reset" setting, which can be | ||||||
| advisable when you have very large query texts. | ||||||
|
|
||||||
| ## Setup | ||||||
|
|
||||||
| [Collector version 0.70.0](https://github.com/pganalyze/collector/blob/main/CHANGELOG.md) | ||||||
| or later is required. | ||||||
|
|
||||||
| We recommend defining this function to avoid permissions errors: | ||||||
|
|
||||||
| <CodeBlock language="sql"> | ||||||
| {`CREATE OR REPLACE FUNCTION pganalyze.reset_stat_statements() RETURNS SETOF void AS | ||||||
| $$ | ||||||
| /* pganalyze-collector */ SELECT * FROM public.pg_stat_statements_reset(); | ||||||
| $$ LANGUAGE sql VOLATILE SECURITY DEFINER;`} | ||||||
| </CodeBlock> | ||||||
|
|
||||||
| After enabling the "Maximum Query Stats Reset Frequency" setting on the server settings page | ||||||
| and optionally tuning "Maximum Query Size Before Next Reset", the collector will automatically | ||||||
| perform resets when needed. You should see evidence of that in the collector logs. | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
| ## Sources of pg_stat_statements churn | ||||||
|
|
||||||
| These issues cause queries to not be grouped properly in pg_stat_statements. Enabling the | ||||||
| collector's automatic statement reset ensures that pganalyze tracks the queries despite these | ||||||
| issues, but you may want to address their root causes so other Postgres monitoring tools can more | ||||||
| reliably depend on pg_stat_statements. | ||||||
|
|
||||||
| - Queries with a variable number of bind params like `id IN ($1, $2, ...)` which can be addressed by | ||||||
| upgrading to Postgres 18 or by rewriting the query to use `id = ANY($1::bigint[])` | ||||||
| - Utility statements including a random comment (like a request ID), which can be addressed by | ||||||
| upgrading to Postgres 16 or by setting `pg_stat_statements.track_utility = off` | ||||||
| - Dynamically-generated queries with varying column lists or where clauses, which can be addressed by | ||||||
| increasing `pg_stat_statements.max` | ||||||
|
Contributor
There was a problem hiding this comment. Should we say something like
Suggested change
? I think whenever there's a setting we recommend changing, we should note the trade-offs unless they're obvious. I guess they're kind of obvious here? |
||||||
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we should put a section after this that talks about reducing pg_stat_statements churn, and specifically references how different Postgres versions have addressed different problems that can occur (and explain when upgrading helps, and to which version). That would also be a good place to talk about
pg_stat_statements.max, which I think is still relevant as a fix for this kind of problem.