ci: onboard Blacksmith Testbox #10110
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,298 @@ | ||
| name: cmux-tui Rust Testbox setup | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| testbox_id: | ||
| description: "Testbox session ID supplied by blacksmith testbox warmup" | ||
| required: true | ||
| type: string | ||
| source_sha: | ||
| description: "Optional full source SHA; if set, it must equal the pushed branch head" | ||
| required: false | ||
| default: "" | ||
| type: string | ||
|
|
||
| # The Blacksmith CLI supplies testbox_id and dispatches the selected pushed | ||
| # branch. It does not expose arbitrary workflow inputs, so source_sha is an | ||
| # optional manual-dispatch assertion; the benchmark's remote guard is required | ||
| # for the CLI path below. | ||
| permissions: {} | ||
|
|
||
| concurrency: | ||
| # A Testbox is a mutable shared workspace. Queue every request that names the | ||
| # same box/source instead of allowing two syncs or stage commands to race. | ||
| group: cmux-tui-testbox-${{ inputs.testbox_id }}-${{ inputs.source_sha || github.sha }} | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| cmux-tui-rust: | ||
| name: cmux-tui Rust setup | ||
| runs-on: blacksmith-32vcpu-ubuntu-2404 | ||
| environment: | ||
| # Configure this environment with required reviewers and no secrets. | ||
| # Approval must happen before begin-testbox exposes its auth token. | ||
| name: blacksmith-testbox-trusted | ||
| permissions: | ||
| contents: read | ||
| timeout-minutes: 45 | ||
| steps: | ||
| # begin-testbox must be the first step. It attaches the VM requested by | ||
| # `blacksmith testbox warmup` and leaves it alive after this setup job. | ||
| - name: Begin Testbox | ||
| uses: useblacksmith/begin-testbox@233448af4bfdc6fca509a7f0974411ac6d8a8043 # v2 | ||
|
Comment on lines
+110
to
+111
There was a problem hiding this comment.
When this workflow is dispatched from another repository on its Useful? React with 👍 / 👎. |
||
| with: | ||
| testbox_id: ${{ inputs.testbox_id }} | ||
|
|
||
| - name: Validate trusted branch dispatch and source identity | ||
| env: | ||
| EXPECTED_INPUT_SHA: ${{ inputs.source_sha }} | ||
| DISPATCH_SHA: ${{ github.sha }} | ||
| DISPATCH_REF: ${{ github.ref }} | ||
| TESTBOX_ID: ${{ inputs.testbox_id }} | ||
| REPOSITORY: ${{ github.repository }} | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| [[ "$REPOSITORY" == "manaflow-ai/cmux" ]] || { | ||
| echo "::error::this Testbox lane is only valid for manaflow-ai/cmux" >&2 | ||
| exit 1 | ||
| } | ||
| [[ "$TESTBOX_ID" =~ ^tbx_[A-Za-z0-9_-]+$ ]] || { | ||
| echo "::error::malformed Testbox ID" >&2 | ||
| exit 1 | ||
| } | ||
| [[ "$DISPATCH_SHA" =~ ^[0-9a-f]{40}$ ]] || { | ||
| echo "::error::github.sha must be a lowercase full commit SHA" >&2 | ||
| exit 1 | ||
| } | ||
| if [[ ! "$DISPATCH_REF" =~ ^refs/heads/[A-Za-z0-9._/-]+$ || "$DISPATCH_REF" == *..* || "$DISPATCH_REF" == */ || "$DISPATCH_REF" == *//* ]]; then | ||
| echo "::error::Testbox warmup must dispatch a pushed branch ref, not a raw SHA, tag, or malformed ref" >&2 | ||
| exit 1 | ||
| fi | ||
| if [[ -n "$EXPECTED_INPUT_SHA" ]]; then | ||
| [[ "$EXPECTED_INPUT_SHA" =~ ^[0-9a-f]{40}$ ]] || { | ||
| echo "::error::source_sha must be a lowercase full commit SHA" >&2 | ||
| exit 1 | ||
| } | ||
| [[ "$EXPECTED_INPUT_SHA" == "$DISPATCH_SHA" ]] || { | ||
| echo "::error::source_sha does not equal the workflow dispatch SHA" >&2 | ||
| exit 1 | ||
| } | ||
| fi | ||
| remote_sha="$(git ls-remote --exit-code "https://github.com/${REPOSITORY}.git" "$DISPATCH_REF" | awk 'NR == 1 { print $1 }')" | ||
| [[ "$remote_sha" == "$DISPATCH_SHA" ]] || { | ||
| echo "::error::the pushed branch moved during dispatch: remote=$remote_sha workflow=$DISPATCH_SHA" >&2 | ||
| exit 1 | ||
| } | ||
| printf 'trusted source ref: %s\ntrusted source SHA: %s\n' "$DISPATCH_REF" "$DISPATCH_SHA" | ||
|
|
||
| - name: Checkout exact dispatch commit | ||
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
| with: | ||
| fetch-depth: 0 | ||
| persist-credentials: false | ||
| ref: ${{ github.sha }} | ||
|
|
||
| - name: Require exact checkout and clean source | ||
| env: | ||
| EXPECTED_SHA: ${{ github.sha }} | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| actual_sha="$(git rev-parse HEAD)" | ||
| [[ "$actual_sha" == "$EXPECTED_SHA" ]] || { | ||
| echo "::error::checked out $actual_sha, expected dispatch SHA $EXPECTED_SHA" >&2 | ||
| exit 1 | ||
| } | ||
| [[ -z "$(git status --porcelain=v1 --untracked-files=normal)" ]] || { | ||
| echo "::error::source checkout is dirty before hydration" >&2 | ||
| git status --short >&2 | ||
| exit 1 | ||
| } | ||
| source_tree_sha="$(git rev-parse 'HEAD^{tree}')" | ||
| ghostty_gitlink_sha="$(git rev-parse 'HEAD:ghostty')" | ||
| [[ "$ghostty_gitlink_sha" =~ ^[0-9a-f]{40}$ ]] || { | ||
| echo "::error::HEAD:ghostty is not a gitlink SHA" >&2 | ||
| exit 1 | ||
| } | ||
| printf 'source_sha=%s\nsource_tree_sha=%s\nghostty_gitlink_sha=%s\n' \ | ||
| "$actual_sha" "$source_tree_sha" "$ghostty_gitlink_sha" | ||
|
|
||
| - name: Initialize Ghostty source submodule | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| git submodule update --init --depth 1 ghostty | ||
| [[ "$(git -C ghostty rev-parse --show-toplevel)" == "$GITHUB_WORKSPACE/ghostty" ]] || { | ||
| echo "::error::ghostty did not initialize as its own submodule checkout" >&2 | ||
| exit 1 | ||
| } | ||
| expected_ghostty_sha="$(git rev-parse HEAD:ghostty)" | ||
| actual_ghostty_sha="$(git -C ghostty rev-parse HEAD)" | ||
| [[ "$actual_ghostty_sha" == "$expected_ghostty_sha" ]] || { | ||
| echo "::error::Ghostty checkout $actual_ghostty_sha does not match gitlink $expected_ghostty_sha" >&2 | ||
| exit 1 | ||
| } | ||
| [[ -z "$(git -C ghostty status --porcelain=v1 --untracked-files=normal)" ]] || { | ||
| echo "::error::Ghostty submodule is dirty after initialization" >&2 | ||
| git -C ghostty status --short >&2 | ||
| exit 1 | ||
| } | ||
| test -f ghostty/build.zig.zon | ||
|
|
||
| - name: Install Linux build dependencies | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| sudo apt-get update | ||
| sudo apt-get install -y clang libclang-dev pkg-config | ||
|
|
||
| - name: Cache Zig package downloads | ||
| uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 | ||
| with: | ||
| path: ~/.cache/zig | ||
| key: cmux-tui-zig-${{ hashFiles('ghostty/build.zig.zon', 'ghostty/build.zig.zon.json') }} | ||
| restore-keys: | | ||
| cmux-tui-zig- | ||
|
|
||
| - name: Install repository-pinned Zig | ||
| shell: bash | ||
| run: ./scripts/install-zig-ci.sh | ||
|
|
||
| - name: Fetch Ghostty Zig dependencies without compiling | ||
| working-directory: ghostty | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| # `--fetch` hydrates the package cache and exits before a build. | ||
| "$CMUX_ZIG" build --fetch | ||
|
|
||
| - name: Cache Cargo registry and git dependencies | ||
| uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 | ||
| with: | ||
| path: | | ||
| ~/.cargo/registry | ||
| ~/.cargo/git | ||
| key: cmux-tui-cargo-${{ runner.os }}-${{ hashFiles('cmux-tui/Cargo.lock', 'cmux-tui/rust-toolchain.toml') }} | ||
| restore-keys: | | ||
| cmux-tui-cargo-${{ runner.os }}- | ||
|
|
||
| - name: Set up repository-pinned cmux-tui Rust | ||
| uses: ./.github/actions/setup-cmux-tui-rust | ||
|
|
||
| - name: Fetch Cargo dependencies without compiling | ||
| working-directory: cmux-tui | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| cargo fetch --locked | ||
|
|
||
| - name: Record runner, toolchain, and Ghostty identity | ||
| env: | ||
| SOURCE_SHA: ${{ github.sha }} | ||
| SOURCE_REF: ${{ github.ref }} | ||
| TESTBOX_ID: ${{ inputs.testbox_id }} | ||
| RUNNER_LABEL: blacksmith-32vcpu-ubuntu-2404 | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| source_tree_sha="$(git rev-parse 'HEAD^{tree}')" | ||
| ghostty_gitlink_sha="$(git rev-parse 'HEAD:ghostty')" | ||
| ghostty_head_sha="$(git -C ghostty rev-parse HEAD)" | ||
| [[ "$SOURCE_SHA" == "$(git rev-parse HEAD)" ]] || exit 1 | ||
| [[ "$ghostty_gitlink_sha" == "$ghostty_head_sha" ]] || exit 1 | ||
| [[ -z "$(git status --porcelain=v1 --untracked-files=normal)" ]] || exit 1 | ||
| [[ -z "$(git -C ghostty status --porcelain=v1 --untracked-files=normal)" ]] || exit 1 | ||
| pushd cmux-tui >/dev/null | ||
| cargo metadata --locked --no-deps --format-version 1 > "$RUNNER_TEMP/cmux-tui-cargo-metadata.json" | ||
| test -s "$RUNNER_TEMP/cmux-tui-cargo-metadata.json" | ||
| RUST_TOOLCHAIN="$(rustup show active-toolchain)" | ||
| RUSTC_VERSION="$(rustc --version)" | ||
| CARGO_VERSION="$(cargo --version)" | ||
| popd >/dev/null | ||
| mkdir -p testbox-benchmark | ||
| SOURCE_TREE_SHA="$source_tree_sha" | ||
| GHOSTTY_GITLINK_SHA="$ghostty_gitlink_sha" | ||
| GHOSTTY_HEAD_SHA="$ghostty_head_sha" | ||
| ZIG_VERSION="$("$CMUX_ZIG" version)" | ||
| ZIG_PATH="$CMUX_ZIG" | ||
| RUNNER_UNAME="$(uname -a)" | ||
| RUNNER_CPU_COUNT="$(nproc)" | ||
| CARGO_METADATA_SHA256="$(sha256sum "$RUNNER_TEMP/cmux-tui-cargo-metadata.json" | cut -d ' ' -f 1)" | ||
| RUST_TOOLCHAIN_FILE_SHA256="$(sha256sum cmux-tui/rust-toolchain.toml | cut -d ' ' -f 1)" | ||
| CARGO_LOCK_SHA256="$(sha256sum cmux-tui/Cargo.lock | cut -d ' ' -f 1)" | ||
| GHOSTTY_ZON_SHA256="$(sha256sum ghostty/build.zig.zon | cut -d ' ' -f 1)" | ||
| export SOURCE_TREE_SHA GHOSTTY_GITLINK_SHA GHOSTTY_HEAD_SHA RUST_TOOLCHAIN RUSTC_VERSION CARGO_VERSION ZIG_VERSION ZIG_PATH RUNNER_UNAME RUNNER_CPU_COUNT CARGO_METADATA_SHA256 RUST_TOOLCHAIN_FILE_SHA256 CARGO_LOCK_SHA256 GHOSTTY_ZON_SHA256 | ||
| python3 - <<'PY' > testbox-benchmark/setup-identity.json | ||
| import json | ||
| import os | ||
| import platform | ||
|
|
||
| print(json.dumps({ | ||
| "schema": 2, | ||
| "source": { | ||
| "ref": os.environ["SOURCE_REF"], | ||
| "commit_sha": os.environ["SOURCE_SHA"], | ||
| "tree_sha": os.environ["SOURCE_TREE_SHA"], | ||
| "ghostty_gitlink_sha": os.environ["GHOSTTY_GITLINK_SHA"], | ||
| "ghostty_head_sha": os.environ["GHOSTTY_HEAD_SHA"], | ||
| }, | ||
| "testbox": { | ||
| "id": os.environ["TESTBOX_ID"], | ||
| "setup_workflow_run_id": os.environ["GITHUB_RUN_ID"], | ||
| }, | ||
| "runner": { | ||
| "label": os.environ["RUNNER_LABEL"], | ||
| "name": os.environ.get("RUNNER_NAME"), | ||
| "os": os.environ.get("RUNNER_OS"), | ||
| "arch": os.environ.get("RUNNER_ARCH"), | ||
| "hostname": platform.node(), | ||
| "uname": os.environ["RUNNER_UNAME"], | ||
| "cpu_count": int(os.environ["RUNNER_CPU_COUNT"]), | ||
| }, | ||
| "toolchain": { | ||
| "rust_toolchain": os.environ["RUST_TOOLCHAIN"], | ||
| "rustc": os.environ["RUSTC_VERSION"], | ||
| "cargo": os.environ["CARGO_VERSION"], | ||
| "rust_toolchain_file_sha256": os.environ["RUST_TOOLCHAIN_FILE_SHA256"], | ||
| "cargo_lock_sha256": os.environ["CARGO_LOCK_SHA256"], | ||
| "cargo_metadata_sha256": os.environ["CARGO_METADATA_SHA256"], | ||
| "zig_path": os.environ["ZIG_PATH"], | ||
| "zig": os.environ["ZIG_VERSION"], | ||
| "ghostty_build_zig_zon_sha256": os.environ["GHOSTTY_ZON_SHA256"], | ||
| }, | ||
| }, sort_keys=True, indent=2)) | ||
| PY | ||
| test -s testbox-benchmark/setup-identity.json | ||
| cat testbox-benchmark/setup-identity.json | ||
|
|
||
| - name: Require clean hydrated source | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| [[ -z "$(git status --porcelain=v1 --untracked-files=normal)" ]] || { | ||
| echo "::error::source became dirty during hydration" >&2 | ||
| git status --short >&2 | ||
| exit 1 | ||
| } | ||
| [[ -z "$(git -C ghostty status --porcelain=v1 --untracked-files=normal)" ]] || { | ||
| echo "::error::Ghostty became dirty during hydration" >&2 | ||
| git -C ghostty status --short >&2 | ||
| exit 1 | ||
| } | ||
|
|
||
| - name: Upload setup identity JSON | ||
| if: always() | ||
| uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 | ||
| with: | ||
| name: cmux-tui-testbox-setup-${{ github.run_id }} | ||
| path: testbox-benchmark/setup-identity.json | ||
| if-no-files-found: warn | ||
| retention-days: 14 | ||
|
|
||
| # Always hand control back to Testbox. Warmup is setup-only: all actual | ||
| # Rust builds are issued later with `blacksmith testbox run`. | ||
| - name: Run Testbox | ||
| uses: useblacksmith/run-testbox@5ca05834db1d3813554d1dd109e5f2087a8d7cbc # v2 | ||
| if: always() | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the same Testbox ID is dispatched for two different revisions—for example, a retry after the source branch advances—the source component makes the concurrency groups different, so both jobs may attach to and hydrate the same mutable VM concurrently. This can race checkouts and dependency state despite the comment's serialization intent; key the group solely by
inputs.testbox_id.Useful? React with 👍 / 👎.