Repository navigation
Conversation
Every API, bg-jobs, stats and cron pod opens its own pools against Cloud SQL, so connections grow with pod count. Managed connection pooling needs Cloud SQL Enterprise Plus, so the pooler runs in the cluster instead: PgBouncer 1.26 in transaction mode in front of cloudsql-proxy-console-api, off by default, with its userlist read from a PGBOUNCER_USERLIST key in the release's Doppler secret. Refs CON-1354
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Console API database connections grow with pod count, because every API, bg-jobs, stats and cron pod opens its own pools against Cloud SQL. Cloud SQL Managed Connection Pooling needs Enterprise Plus and our instance is on Enterprise, so the pooler runs in the cluster.
Ref CON-1354
What
pgbouncerblock in the console-api chart, off by default. When enabled, each release getsconsole-api-<chain>-pgbouncer: a Deployment (2 replicas, preferred anti-affinity), a Service on 5432, a PodDisruptionBudget and a ConfigMap holdingpgbouncer.ini.edoburu/pgbouncer:v1.26.1-p0) runs in transaction mode in front ofcloudsql-proxy-console-api. A wildcard database entry serves both the user DB and the chain DB.max_prepared_statements = 200keeps postgres.js prepared statements working.PGBOUNCER_USERLISTkey in the release's Doppler secret (one"user" "password"line per database user), mounted as a file. The container startspgbouncerdirectly instead of the image entrypoint, because the entrypoint truncates passwords at a:and doesn't decode URL-encoded characters.defaultPoolSize(20) per database and user on each replica, so a release with both databases opens at most 80.Rollout, staging first:
PGBOUNCER_USERLISTand setPOSTGRES_MIGRATION_DB_URIto the current direct URI.pgbouncer.enabled: truein the console repo's.helm/console-api-staging-<chain>-values.yamland deploy.POSTGRES_DB_URIandCHAIN_INDEXER_POSTGRES_DB_URIatconsole-api-<chain>-pgbouncer:5432. This step needs fix(config): let the api run behind a transaction-mode connection pooler console#4646 deployed first, so migrations keep a direct connection.Testing:
ct lintpasses. Withpgbouncer.enabledunset, the chart renders exactly as before.pgbouncer.iniand ran it with the pinned image and the same command against a local Postgres that requires SCRAM. Both databases were reachable through the wildcard entry, a wrong password was rejected, postgres.js with prepared statements on completed 450/450 queries and transactions, and node-postgres (pg-boss's driver) connected.