fix(infra): boot-diag empty-bucket guidance and boot.log selection - #1049
Merged
Conversation
Adopted from raak (cellajs/raak#125), which debugged a production incident where every deploy failed at cutover and diag gave no signal. - aws s3 ls on a zero-object prefix exits 1 with no output; treat that as a finding (nothing was ever uploaded) instead of a fatal listing error, and print actionable guidance naming the two known causes (boot runner never ran, or uploads denied by a pre-v2 boot principal). Real failures still abort: they carry stderr. - selectDiagnostics pinned latestFull to -boot.log keys; the sibling -events.jsonl sorted after it and won .at(-1), replacing the readable transcript with raw OTLP records. - renderDiagnostics prints a single no-diagnostics line when a service owns zero objects instead of misleading per-section noise. One delta from the raak original: its guidance pointed at the infra CLI "Migrate IAM model" flow, which was deleted in #1024 (IAM v2 only); the guidance now points at the "Stack setup" apply, which provisions the current boot principal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adopts raak's boot-diag improvements upstream (cellajs/raak#125, merged there), from a production incident where every deploy failed at cutover and
pnpm --filter infra diaggave no useful signal.What
aws s3 lson a zero-object prefix exits 1 with no stdout and no stderr;createAwsReader.list()treated that as a fatal listing failure. NewisEmptyPrefixLs()classifies it (exit 1 + empty stdout + empty stderr only, so AccessDenied / bad endpoint / bad bucket still abort),list()returns''for it, andemptyBootDiagGuidance()prints the two known causes: the boot runner never ran (check serial console for::<slug>::markers and "BOOT FAILED"), or uploads are denied because the boot principal predates the IAM v2 model (Scaleway does not honor bucket-policy-only grants). Every boot uploads a transcript, success or failure (boot.tsfinally), so an empty prefix is a finding, not an error.selectDiagnosticspicks the actual transcript. The sibling<svc>-<stamp>-events.jsonlsorts after<svc>-<stamp>-boot.log, so.at(-1)returned raw OTLP records instead of the readable transcript.latestFullis now pinned to-boot.logkeys.renderDiagnosticsprints a single "nothing was ever uploaded" line when a service owns zero objects, instead of misleading per-section noise.diag.tsthreadsappConfig.sluginto the guidance and prints it whenparseKeysyields nothing (except under--replay).Delta from the raak original
raak's guidance pointed at the infra CLI "Migrate IAM model" flow, which no longer exists: the IAM migration machinery was deleted in #1024 (IAM v2 only). The guidance (and its test) now points at the "Stack setup" apply, which is what provisions the current boot principal (
ObjectStorageObjectsWriteviaBOOT_PROJECT_PERMISSION_SETS).Not adopted: raak's
cella/migrations/manifest.jsonroots tweak for20260730T0858-frontend-module-placements. That migration precedes20260730T1258-cella-config-into-cella-folderin replay order, socella.config.tsis still at the repo root at that point in the sequence; the existing value is correct.Verification
pnpm --filter infra exec vitest run: 78 files, 665 passed (includes new coverage forisEmptyPrefixLs, empty-prefix listing, boot.log-over-events selection, and the single no-diagnostics line)pnpm check: cleanpnpm cella sync. projectcampus's copies are byte-identical to current main, so it inherits conflict-free.🤖 Generated with Claude Code