-
Notifications
You must be signed in to change notification settings - Fork 6
Document new statement reset behavior #388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
cbbe9e3
c000fa6
a198066
922c37a
1b59336
abddf78
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,46 @@ | ||||||||||
| --- | ||||||||||
| title: Reducing memory usage and query text unavailable | ||||||||||
| backlink_href: /docs/install/troubleshooting | ||||||||||
| backlink_title: 'Installation Troubleshooting' | ||||||||||
| --- | ||||||||||
|
|
||||||||||
| If the collector is using too much memory or `<query text unavailable>` is showing up in pganalyze, | ||||||||||
| you may want to set up an automatic reset of pg_stat_statements. | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
|
|
||||||||||
| ## <query text unavailable> | ||||||||||
|
|
||||||||||
| Some versions of Postgres aren't able to accurately normalize queries (e.g. those with variable bind params), | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| leading to many duplicate entries that often have very low call counts. When `pg_stat_statements.max` is reached, | ||||||||||
| Postgres will deallocate (remove) the entries with the lowest call count. In periods of high database activity | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| this can happen so so quickly that the collector isn't able to capture the query text, causing | ||||||||||
| `<query text unavailable>` to show up in pganalyze. | ||||||||||
|
|
||||||||||
| This can be addressed by enabling the Maximum Query Stats Reset Frequency setting. We recommend a setting of | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| "Every 1 hour" unless you're specifically concerned about lock contention on a very busy database. This enables | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| automatic pg_stat_statments resets to be done once the statement space is 90% full (so a deallocation would occur soon), | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| but resets will not occur more often than the frequency setting you choose. | ||||||||||
|
|
||||||||||
| ## High memory usage | ||||||||||
|
|
||||||||||
| Every 10 minutes, the collector loads the query text from pg_stat_statements into memory in order to normalize and | ||||||||||
| fingerprint it. This is a very expensive step and can result in the collector crashing from running out of memory. | ||||||||||
| To address that, when resets are enabled with the Maximum Query Stats Reset Frequency setting, the collector will | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| also perform a reset if the size of the query text grows beyond 250 MB. You can optionally increase that limit | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| using the Maximum Query Size Before Next Reset setting. | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
|
|
||||||||||
| ## Setup | ||||||||||
|
|
||||||||||
| Collector version 0.70.0 or later is required. | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We could link this to the collector release changelog. |
||||||||||
|
|
||||||||||
| After enabling the Maximum Query Stats Reset Frequency setting on the server settings page (and optionally tuning | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it'd be good to keep a server settings page screenshot here, possibly with the relevant two settings highlighted with a red border or similar to make it very clear what needs to be changed. |
||||||||||
| Maximum Query Size Before Next Reset), the collector will automatically start performing resets when needed. | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||
| You should see evidence of that in the collector logs. | ||||||||||
|
|
||||||||||
| If the collector fails to reset with permissions errors, you may need to define this function: | ||||||||||
|
|
||||||||||
| <CodeBlock language="sql"> | ||||||||||
| {`CREATE OR REPLACE FUNCTION pganalyze.reset_stat_statements() RETURNS SETOF void AS | ||||||||||
| $$ | ||||||||||
| /* pganalyze-collector */ SELECT * FROM public.pg_stat_statements_reset(); | ||||||||||
| $$ LANGUAGE sql VOLATILE SECURITY DEFINER;`} | ||||||||||
| </CodeBlock> | ||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wonder if we should put a section after this that talks about reducing pg_stat_statements churn, and specifically references how different Postgres versions have addressed different problems that can occur (and explain when upgrading helps, and to which version). That would also be a good place to talk about |
||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.