Skip known query texts from pg_stat_statements - #776
Conversation
| SELECT dbid, userid, queryid, %s, query | ||
| FROM %s` | ||
| FROM %s | ||
| WHERE queryid = ANY($1)` |
There was a problem hiding this comment.
As discussed, doing this filtering has three main benefits that I can see:
- We avoid sending query text over the network (from the database to the collector) that we've already seen
- We avoid running normalize on query texts we've already seen
- We avoid sending query text to pganalyze that pganalyze already got
Just for clarity, because of how pg_stat_statements works today, this unfortunately won't reduce the effort done to read the query text file and put the query text and statistics into the tuplestore, since the WHERE condition won't be pushed down into pg_stat_statements_internal.
As discussed, for For IgnoreIoTiming, it does seem best to keep this on the collector side (and we can make that conditional on Aurora), and keep a mapping from query ID to a utility statement boolean. Overall I'm not sure on this yet (and need to spend more time reviewing), specifically:
|
Building on top of #768, this avoids loading query text from pg_stat_statements for known query IDs as a performance optimization.
Todo:
QueryStatisticuse fingerprint instead of query index?<pganalyze-collector>grouping andIgnoreIoTimingbe implemented on the server side?