Skip to content

feat(doqa): bump to 4.3.1-box with new queue topology - #110

Merged
batonogov merged 2 commits into
mainfrom
doqa-4.3.1
Sep 28, 2026
Merged

batonogov merged 2 commits into
mainfrom
doqa-4.3.1

Conversation

@batonogov

@batonogov batonogov commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

Vendor 4_3_1 changes (configs_4_2_2 → configs_4_3_1)

Structural update, not a tag bump. nginx configs and the Dockerfile are unchanged.

Image pins (.env.install):

  • backend 4.2.12-box → 4.3.1-box
  • frontend 4.2.9-box → 4.3.0-box
  • autotest-parser 4.2.1-box → 4.3.0-box
  • statistic 3.0.0-box → 4.3.0-box
  • notify 4.2.1-box → 4.3.0-box
  • telegram-bot 1.0.1-box → 4.3.0-box
  • llm 4.2.1-box → 4.3.0-box
  • doqa-autotest-result-parser removed upstream (no image pin, no compose service)

New .env.install variables:

  • MINIO_USE_PATH_STYLE=true, QUEUE_BATCHING_CONNECTION=pgsql → <release>-env ConfigMap
  • AUTOTEST_PARSER_SERVICE_URL → ConfigMap, computed from the autotest-parser Service name (same pattern as PUSHER_APP_HOST)
  • INTERNAL_SYSTEM_KEY=GENERATE_SECRET_KEY → new generated key internal-system-key in the <release>-app-secrets Secret (lookup-stable like the others). Vendor's env_file delivers it to backend/queue/cron and the new workers — the chart reaches all of them via doqa.backendSecretEnv — plus an explicit env entry on autotest-parser, matching vendor's explicit list.

Compose topology:

  • doqa_queue now consumes default,search,requirements,autotests-domain,outbound-webhooks,import (llm-bulk moved out)
  • New doqa_autotests → templates/autotests-worker.yaml: declares the 8 RabbitMQ queues, then runs autotestsWorker.workerProcesses (default 3, vendor AUTOTESTS_WORKER_PROCESSES) parallel autotests:work consumers with autotestsWorker.prefetch (default 10, vendor AUTOTESTS_PREFETCH); replicas default 3 (vendor scales with QUEUE_WORKERS)
  • New doqa_llm_bulk_queue → templates/llm-bulk-queue.yaml: single replica, queue:work rabbitmq --queue=llm-bulk --sleep=3 --tries=1 --timeout=3660
  • doqa_autotest_parser gains explicit MINIO_BUCKET/ENDPOINT/KEY/REGION/SECRET (MinIO creds from the minio Secret) and INTERNAL_SYSTEM_KEY
  • Vendor switched depends_on to a RabbitMQ healthcheck; both new workers get the same wait-for-rabbitmq initContainer the chart already runs for queue/migrations (disable via autotestsWorker.waitForRabbitmq / llmBulkQueue.waitForRabbitmq). autotest-parser keeps relying on restart policy, as before.

Breaking / upgrade notes

  • The autotestResultParser values section and the autotest-result-parser Deployment are gone. Remove autotestResultParser from your values files. (values.schema.json does not set additionalProperties: false, so leftover keys still validate — they are simply ignored.)
  • With secrets.create=false or an external secrets.app, the Secret must now also contain internal-system-key (render fails otherwise with a required error / pods CrashLoop on the missing key). Chart-managed secrets generate it automatically on upgrade.
  • Chart renders 16 Deployments by default now (was 15).

Image contract audit (scripts/audit-doqa-images.sh)

Image Verdict
doqa-backend 4.2.12→4.3.1 ENV diff is only the PHP base bump (8.2.26→8.4.23, GPG keys, tarball URLs). No new required env, Cmd/Entrypoint unchanged. OK
doqa-frontend 4.2.9→4.3.0 Cmd switched from shell form (/bin/sh -c "nginx -g 'daemon off;'") to exec form (nginx -g 'daemon off;'). The chart does not override the frontend command, so this is compatible. OK
doqa-parsing-autotests 4.2.1→4.3.0 no env/cmd/entrypoint changes
doqa-statistic 3.0.0→4.3.0 no env/cmd/entrypoint changes
doqa-notify 4.2.1→4.3.0 no env/cmd/entrypoint changes
doqa-llm 4.2.1→4.3.0 no env/cmd/entrypoint changes
doqa-telegram-bot 1.0.1→4.3.0 new ENV TG_PROXY= with empty default; vendor .env.install does not set it, so the chart does not wire it. OK
doqa-autotest-result-parser skipped — component removed upstream

Validation

  • scripts/audit-doqa-images.sh — reviewed above; benign diffs only
  • helm template test-doqa charts/doqa -f charts/doqa/ci/test-values.yaml — renders, 16 Deployments; also rendered with networkPolicy.enabled=true + telegramBot.enabled=true and with ci/external-values.yaml
  • scripts/check-helm-docs.sh, scripts/helm-lint-all.sh, scripts/helm-template-all.sh — pass
  • pre-commit run --all-files — all hooks pass (incl. doqa portability)
  • ct lint --config .github/ct.yaml --target-branch main — pass (version increment ok, both ci values files lint clean)

xray-health-exporter untouched.

Vendor box release 4_3_1 (configs_4_3_1.zip):

- Image pins: backend 4.3.1-box; frontend, autotest-parser, statistic,
  notify, telegram-bot, llm 4.3.0-box
- autotest-result-parser removed upstream; the chart drops the
  autotest-result-parser Deployment and the autotestResultParser values
  section
- New backend-image workers: autotests-worker (declares the eight
  RabbitMQ queues, runs AUTOTESTS_WORKER_PROCESSES parallel
  autotests:work consumers) and single-replica llm-bulk-queue
  (queue:work rabbitmq --queue=llm-bulk --timeout=3660)
- queue command now consumes
  default,search,requirements,autotests-domain,outbound-webhooks,import
- New ConfigMap keys: MINIO_USE_PATH_STYLE=true,
  QUEUE_BATCHING_CONNECTION=pgsql, AUTOTEST_PARSER_SERVICE_URL computed
  from the autotest-parser Service name
- New generated secret INTERNAL_SYSTEM_KEY in the app Secret, wired into
  backend/queue/cron/workers via backendSecretEnv and explicitly into
  autotest-parser, which also gains explicit MINIO_* credentials
- NetworkPolicy: result-parser rules replaced by the new workers;
  autotest-parser and autotests-worker allowed to reach MinIO

Chart version 0.6.3 -> 0.7.0 (autotestResultParser removal is
breaking-ish for user values).
Review feedback on the 4.3.1 bump:

- allow-nginx-to-autotest-parser renamed to allow-app-to-autotest-parser;
  backend (AUTOTEST_PARSER_SERVICE_URL), queue, and autotests-worker can
  now reach autotest-parser:8000 alongside nginx
- allow-app-to-llm: llm-bulk-queue added (it consumes llm-bulk jobs that
  call the LLM API)
- allow-app-to-websocket / -notification / -statistic: autotests-worker
  and llm-bulk-queue added alongside queue, which carried this traffic
  on 0.6.x
- minio ingress rule: llm-bulk-queue added (receives MINIO_* env via
  backendSecretEnv)
- scripts/audit-doqa-images.sh: drop the removed
  doqa-autotest-result-parser repository
- README: note that autotests-worker redeclares the 8 queues with
  durable/no-args params (PRECONDITION_FAILED only for hand-redeclared
  legacy queues)
@batonogov
batonogov merged commit 190e51a into main Sep 28, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant