[aw-failures] Fleet-wide smoke-test outage: cli-proxy image tag mismatch (:0.27.44 tagged, :latest expected) blocks 9 workflows #50681

Problem

Pin the awf compose stack to the versioned image tag, not :latest — the download-and-tag step and the compose-up step disagree and it just killed 9 smoke workflows at once. Between 2026-08-05 16:08–16:18 UTC, docker compose up -d --pull never failed with Error response from daemon: No such image: ghcr.io/github/gh-aw-firewall/cli-proxy:latest for every affected run, immediately after download_docker_images.sh successfully pulled and retagged the digest-pinned image as ghcr.io/github/gh-aw-firewall/cli-proxy:0.27.44 (not :latest). --pull never then can't find :latest locally and the whole job aborts before the agent CLI ever starts. audit-diff on §31023914914 vs the last success confirms this: the failed run made zero requests to any allowed domain (Copilot API, Sentry, Grafana OTLP) — the container stack never came up, so no agent execution happened at all.

Affected Workflows and Run IDs (all 2026-08-05, ~16:08–16:18 UTC, same PR/branch bump-mcpg-awf-versions which bumps AWF to v0.27.44)

• Smoke OTEL — §31023914914
• Smoke Claude — §31023852205
• Smoke Gemini — §31023852182
• Smoke Codex — §31023852370
• Smoke Pi — §31023852242
• Smoke Copilot Auto — §31023852408
• Smoke Copilot - AOAI (Entra) — §31023908831
• Smoke Copilot - AOAI (apikey) — §31023907002
• Smoke Copilot — §31023905263

Comparator (same workflow, last success, before this window): §30716156183 (Smoke OTEL, 2026-08-01)

Probable Root Cause

The version-bump PR (Bump AWF to v0.27.44 and MCP Gateway to v0.4.8) changed download_docker_images.sh to pull and tag digest-pinned images as <image>:0.27.44, but the docker-compose.yml/awf launcher still references the container image as <image>:latest when starting awf-cli-proxy under --pull never. The tag written by the download step and the tag read by compose never match, so every run in this window hit it simultaneously — this is a hard regression in the version-bump PR, not an intermittent registry/network blip (the pull itself succeeded every time).

Specific Proposed Remediation

1. Make the compose file / awf launcher reference the same versioned tag (0.27.44) that download_docker_images.sh writes, or have the download script also tag/alias the image as :latest so --pull never resolves either way.
2. Add a smoke-level pre-flight check that fails fast with a clear "image tag mismatch" error instead of the opaque docker compose up exit 1.
3. Re-run the full smoke fleet once fixed to confirm all engines (not just Copilot) come up clean.

Success Criteria / Verification

• Re-running any of the 9 affected smoke workflows on the current main (post-fix) succeeds past container startup.
• docker compose up -d --pull never no longer errors with No such image ... :latest for awf-cli-proxy (or any gh-aw-firewall/* image) in the next 6h window.
• audit-diff on the next run of Smoke OTEL shows non-zero allowed-domain traffic (Copilot/OTLP), confirming the agent container actually started.
  Related to [aw-failures] [aw] Failure Investigator Report — 2026-08-05 (6h) #50518

Generated by 🔍 [aw] Failure Investigator (6h) · agent · 148.2 AIC · ⌖ 49.3 AIC · ⊞ 5.2K · ◷

• expires on Aug 12, 2026, 11:24 AM UTC-08:00