Document new statement reset behavior - #388
Conversation
lfittl
left a comment
There was a problem hiding this comment.
Overall looks good - I'll take one more look after wording adjustments are done.
| @@ -1,54 +0,0 @@ | |||
| --- | |||
There was a problem hiding this comment.
We should make sure to add a redirect via the nginx config in the web repo.
| --- | ||
|
|
||
| If the collector is using too much memory or `<query text unavailable>` is showing up in pganalyze, | ||
| you may want to set up an automatic reset of pg_stat_statements. |
There was a problem hiding this comment.
| you may want to set up an automatic reset of pg_stat_statements. | |
| you may want to set up an automatic reset of `pg_stat_statements`. |
|
|
||
| ## <query text unavailable> | ||
|
|
||
| Some versions of Postgres aren't able to accurately normalize queries (e.g. those with variable bind params), |
There was a problem hiding this comment.
| Some versions of Postgres aren't able to accurately normalize queries (e.g. those with variable bind params), | |
| Earlier versions of Postgres aren't able to group certain queries together (e.g. those using `IN` lists of variable lengths), |
|
|
||
| Some versions of Postgres aren't able to accurately normalize queries (e.g. those with variable bind params), | ||
| leading to many duplicate entries that often have very low call counts. When `pg_stat_statements.max` is reached, | ||
| Postgres will deallocate (remove) the entries with the lowest call count. In periods of high database activity |
There was a problem hiding this comment.
| Postgres will deallocate (remove) the entries with the lowest call count. In periods of high database activity | |
| Postgres will deallocate (remove) entries based on their usage, which is based on the call count. In periods of high database activity |
| this can happen so so quickly that the collector isn't able to capture the query text, causing | ||
| `<query text unavailable>` to show up in pganalyze. | ||
|
|
||
| This can be addressed by enabling the Maximum Query Stats Reset Frequency setting. We recommend a setting of |
There was a problem hiding this comment.
| This can be addressed by enabling the Maximum Query Stats Reset Frequency setting. We recommend a setting of | |
| This can be addressed by enabling the "Maximum Query Stats Reset Frequency" setting in pganalyze. We recommend a setting of |
|
|
||
| ## Setup | ||
|
|
||
| Collector version 0.70.0 or later is required. |
There was a problem hiding this comment.
We could link this to the collector release changelog.
| After enabling the Maximum Query Stats Reset Frequency setting on the server settings page (and optionally tuning | ||
| Maximum Query Size Before Next Reset), the collector will automatically start performing resets when needed. |
There was a problem hiding this comment.
| After enabling the Maximum Query Stats Reset Frequency setting on the server settings page (and optionally tuning | |
| Maximum Query Size Before Next Reset), the collector will automatically start performing resets when needed. | |
| After enabling the "Maximum Query Stats Reset Frequency" setting on the server settings page (and optionally tuning | |
| "Maximum Query Size Before Next Reset"), the collector will automatically start performing resets when needed. |
|
|
||
| Collector version 0.70.0 or later is required. | ||
|
|
||
| After enabling the Maximum Query Stats Reset Frequency setting on the server settings page (and optionally tuning |
There was a problem hiding this comment.
I think it'd be good to keep a server settings page screenshot here, possibly with the relevant two settings highlighted with a red border or similar to make it very clear what needs to be changed.
| $$ | ||
| /* pganalyze-collector */ SELECT * FROM public.pg_stat_statements_reset(); | ||
| $$ LANGUAGE sql VOLATILE SECURITY DEFINER;`} | ||
| </CodeBlock> |
There was a problem hiding this comment.
I wonder if we should put a section after this that talks about reducing pg_stat_statements churn, and specifically references how different Postgres versions have addressed different problems that can occur (and explain when upgrading helps, and to which version). That would also be a good place to talk about pg_stat_statements.max, which I think is still relevant as a fix for this kind of problem.
| @@ -0,0 +1,46 @@ | |||
| --- | |||
| title: Reducing memory usage and query text unavailable | |||
There was a problem hiding this comment.
| title: Reducing memory usage and query text unavailable | |
| title: Query text unavailable and reducing memory usage |
| - Utility statements including a random comment (like a request ID), which can be addressed by | ||
| upgrading to Postgres 16 or by setting `pg_stat_statements.track_utility = off` | ||
| - Dynamically-generated queries with varying column lists or where clauses, which can be addressed by | ||
| increasing `pg_stat_statements.max` |
There was a problem hiding this comment.
Should we say something like
| increasing `pg_stat_statements.max` | |
| increasing `pg_stat_statements.max` (the trade-off is higher memory usage in Postgres) |
? I think whenever there's a setting we recommend changing, we should note the trade-offs unless they're obvious. I guess they're kind of obvious here?
* mcp: add api authentication docs, and note about rate limiting * update hierarchy * rewrites * Keiko review feedback Co-authored-by: Keiko Oda <keiko713@gmail.com> * Document new statement reset behavior (#388) --------- Co-authored-by: Keiko Oda <keiko713@gmail.com>
This documents the new pg_stat_statements reset behavior in pganalyze/collector#775
I decided to remove the "Collector times out" page because statement text collection timeouts should be largely resolved ever since we delayed pg_query calls until after the SELECT query was complete.