Skip to content

dispatch-e2e validation + agent time discipline rules - #10024

Open
azooz2003-bit wants to merge 7 commits into
mainfrom
feat-agent-time-discipline
Open

dispatch-e2e validation + agent time discipline rules#10024
azooz2003-bit wants to merge 7 commits into
mainfrom
feat-agent-time-discipline

Conversation

@azooz2003-bit

@azooz2003-bit azooz2003-bit commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

An audit of 23 agent sessions found hosted test-e2e dispatches whose test_filter matched zero tests, each wasting a full dispatch+watch cycle before re-dispatch, plus hours lost to poll-slice waiting, re-dispatched builds, per-one-line-fix rebuilds, and iPhone unlock watchers.

scripts/dispatch-e2e.sh is a validated dispatcher for test-e2e.yml. Before dispatching it requires each filter item's class to exist under cmuxUITests/ (or cmuxTests/ when target-qualified) and, for Class/method, the method to exist in that class's files. On a miss it refuses to dispatch, exits 2, and prints the nearest candidate names. Comma-separated filters dispatch one run each. --dry-run stops after validation and prints the gh command; --watch blocks on gh run watch --exit-status; --runner, --record-video, --timeout, and --job-timeout pass through to the workflow inputs. Shellcheck clean.

CLAUDE.md now documents the wrapper as the E2E entrypoint (raw gh workflow run kept as fallback, also updated in the cmux-testing skill reference) and adds an agent time discipline section: park on one blocking command per wait instead of poll slices, one build dispatch per need, batch a dogfood round's fixes into one tagged rebuild, and for a locked or offline iPhone probe once, enqueue, notify, and end the turn.

Docs and CI tooling only, no runtime change, so no tagged build.

Validation transcripts:

$ ./scripts/dispatch-e2e.sh --ref main --filter "FeedSidebarUITests" --dry-run
ok: FeedSidebarUITests (class FeedSidebarUITests in cmuxUITests/)
gh workflow run test-e2e.yml --repo manaflow-ai/cmux -f ref=main -f test_filter=FeedSidebarUITests
(exit 0)

$ ./scripts/dispatch-e2e.sh --ref main --filter "FeedSidebarUITest" --dry-run
error: no 'class FeedSidebarUITest' found under cmuxUITests/ ...
nearest candidates: FeedSidebarUITests, SidebarResizeUITests, ...
(exit 2)

$ ./scripts/dispatch-e2e.sh --ref main --filter "FeedSidebarUITests/testDoesNotExistAnywhere" --dry-run
error: no 'func testDoesNotExistAnywhere' found in class FeedSidebarUITests ...
nearest candidates: testDockTerminalRerendersAfterRightSidebarHideShow, testFeedReceivesAndResolvesPermissionRequest
(exit 2)

🤖 Generated with Claude Code


Summary by cubic

Validates hosted E2E dispatches and enforces time‑discipline to prevent zero‑test runs and silent watch passes. Previously a test_filter with no matches still dispatched and --watch could succeed after omitting unresolved runs; now the wrapper validates selectors scoped to the class or its extensions, accepts only parameterless instance test* methods, resolves runs exactly via a per-dispatch dispatch_id, and fails --watch if any run id cannot be resolved. No runtime changes.

  • Use: ./scripts/dispatch-e2e.sh --ref <branch-or-sha> --filter "<Class or Class/method>[,more]" [--watch] [--dry-run] [--runner] [--record-video] [--timeout] [--job-timeout]; run it from a checkout of the same ref you dispatch.
  • Exact run resolution: test-e2e.yml adds optional dispatch_id; the wrapper passes a nonce and, if the workflow on main rejects unknown inputs, retries once without it and falls back to a pre‑existing‑ids heuristic.
  • Each comma-separated item dispatches its own run; cap to one dispatch per dogfood round.
  • Raw fallback: gh workflow run test-e2e.yml --repo manaflow-ai/cmux -f ref=<branch-or-sha> -f test_filter="<Class or Class/method>" (required, non-empty).
  • Do not dispatch‑fix‑redispatch; root‑cause red runs locally or on a fleet simulator. PR checks are advisory only; merge validation happens via the merge gate.

Written for commit b91b7ee. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • New Features

    • Added guided dispatch for hosted end-to-end tests with selectable test filters.
    • Added validation for invalid or empty test selections, including suggested close matches.
    • Added dry-run previews, optional run monitoring, clear failure reporting, and support for individual test groups.
    • Retained a raw workflow fallback for hosted test execution.
  • Documentation

    • Updated testing guidance for hosted runs, virtual machine execution, and fallback workflows.
    • Clarified dispatch limits, wait practices, asynchronous checks, and merge-gate handoff procedures.

scripts/dispatch-e2e.sh validates each test_filter item against local
test sources (class under cmuxUITests/ or cmuxTests/, method in that
class's files) before dispatching test-e2e.yml, exiting 2 with nearest
candidates on a miss so a typo no longer costs a full dispatch+watch
cycle. Supports comma-separated filters (one run each), --dry-run,
--watch, and passthrough for runner, record_video, test_timeout, and
job_timeout.

CLAUDE.md documents the wrapper as the E2E entrypoint (raw gh command
as fallback) and adds an agent time discipline section: one blocking
command per wait, one build dispatch per need, batched fixes per tagged
rebuild, and probe-once-then-queue for a locked or offline iPhone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The change adds a validated hosted E2E dispatcher. It validates Swift test filters, supports dry runs and run monitoring, and updates testing and agent guidance for hosted verification.

Changes

Hosted E2E dispatch

Layer / File(s) Summary
Filter validation and command interface
scripts/dispatch-e2e.sh
The script parses options, validates values, discovers Swift tests, and reports candidate matches for invalid filters.
Workflow dispatch and run monitoring
scripts/dispatch-e2e.sh
The script supports dry runs, per-filter dispatches, run-ID resolution, result URLs, sequential watching, and failure propagation.
Testing guidance and agent operating rules
CLAUDE.md, skills/cmux-testing/references/local-vs-ci-validation.md
The guidance recommends the validated dispatcher, documents the raw workflow fallback, and adds rules for waits, rebuilds, device availability, asynchronous checks, and merge-gate status.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Mergeability Score: 🟡 Moderate · up to 8d939

The new E2E dispatcher can misread XCTest methods when comments, strings, or nested functions contain similar syntax, allowing invalid filters through or rejecting valid ones and potentially causing wasted hosted runs. The documented one-dispatch limit also remains inconsistent with the implementation, so these issues should be fixed or explicitly accepted before merge.

Sequence Diagram(s)

sequenceDiagram
  participant Engineer
  participant dispatch_e2e_sh
  participant GitHubActions
  Engineer->>dispatch_e2e_sh: Provide validated E2E filters and options
  dispatch_e2e_sh->>GitHubActions: Dispatch one workflow per filter
  GitHubActions-->>dispatch_e2e_sh: Return workflow run information
  dispatch_e2e_sh->>GitHubActions: Watch runs when requested
  GitHubActions-->>Engineer: Report run URLs and failures
Loading

Possibly related PRs


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (2 errors)

Check name Status Explanation Resolution
Cmux No Hacky Sleeps ❌ Error New scripts/dispatch-e2e.sh adds sleep 2 inside a 15-attempt gh run list loop to wait for dispatched workflow registration, matching the rule's fixed polling delay. Replace the fixed sleep/poll loop with a readiness signal or cancellation-aware bounded mechanism that identifies the dispatched workflow run without elapsed-time synchronization.
Cmux Algorithmic Complexity ❌ Error New scripts/dispatch-e2e.sh loops over comma filters at line 348 and recursively scans each target at lines 125/131 per item, making validation O(F·S) over Swift files. Build one target-wide class/extension/method index before validating CLEAN_ITEMS, then reuse it so validation is O(S+F) instead of rescanning source files per filter.
✅ Passed checks (23 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Cmux Swift Actor Isolation ✅ Passed The diff changes only CLAUDE.md, a shell dispatcher, and a testing reference; it contains no changed Swift files or production Swift declarations.
Cmux Swift Blocking Runtime ✅ Passed The full PR diff contains only documentation and scripts; git diff --name-only -- '*.swift' is empty, so the production Swift blocking-runtime check is inapplicable.
Cmux Browser Automation Off-Main ✅ Passed The PR changes only documentation and scripts/dispatch-e2e.sh; browser automation sources, worker policy, and policy tests are unchanged, with no added browser/WebKit routing code.
Cmux Expensive Synchronous Load ✅ Passed The PR diff contains only CLAUDE.md, one shell script, and a Markdown reference; it adds no Swift files or production Swift load paths.
Cmux Cache Substitution Correctness ✅ Passed The diff changes only CLAUDE.md, a shell dispatcher, and a skill reference; no production Swift, TypeScript, or JavaScript persistence/history/undo/snapshot read is changed.
Cmux Swift Concurrency ✅ Passed The PR range changes only CLAUDE.md, a Bash dispatcher, and a Markdown reference; it adds no cmux-owned Swift code or legacy Swift concurrency patterns.
Cmux Swift @Concurrent ✅ Passed The PR range changes only CLAUDE.md, dispatch-e2e.sh, and the testing reference; git diff reports zero changed .swift files, so the Swift @concurrent check is inapplicable.
Cmux Swift Package Boundaries ✅ Passed The merge-base diff contains only CLAUDE.md, scripts/dispatch-e2e.sh, and a Markdown reference; it introduces no production Swift changes, so the boundary rule is inapplicable.
Cmux Swiftpm Lockfiles ✅ Passed PR diff changes only CLAUDE.md, dispatch-e2e.sh, and a skill reference; it changes no Package.swift, Package.resolved, .gitignore, workflow, or Xcode project files.
Cmux Swift Logging ✅ Passed The complete PR diff changes only CLAUDE.md, a shell dispatcher, and a skill reference; it adds no Swift files or production Swift logging.
Cmux User-Facing Error Privacy ✅ Passed The diff changes only a developer/CI dispatcher and operational documentation; it does not add a production UI/API error path or expose prohibited secrets, tokens, headers, or payloads.
Cmux Full Internationalization ✅ Passed The diff changes only an operational Bash dispatcher and developer/agent documentation; it adds no production Swift, app catalog, web UI, API, metadata, or rendered user-facing copy.
Cmux Swiftui State Layout ✅ Passed The PR diff changes only two Markdown files and one shell script; it introduces no Swift/SwiftUI files or SwiftUI state, layout, list-row, or render-time mutation patterns.
Cmux Architecture Rethink ✅ Passed The diff changes only CLAUDE.md, a Bash E2E dispatcher, and one Markdown reference; it introduces no Swift architecture or UI lifecycle changes covered by the rule.
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed The PR diff from the common base changes only CLAUDE.md, the E2E shell script, and a Markdown reference; it contains no Swift window changes.
Cmux Source Artifacts ✅ Passed The diff adds one intentional executable dispatcher and edits two durable Markdown docs; no logs, screenshots, recordings, caches, temp directories, build output, or artifact files are added.
Cmux No Test Or Debug Seam In Production Source ✅ Passed The cumulative PR diff changes only CLAUDE.md, scripts/dispatch-e2e.sh, and a Markdown reference; it adds or modifies no Swift file under any Sources path.
Cmux No Ambient Global State ✅ Passed The PR range changes only Markdown files and scripts/dispatch-e2e.sh; it contains no .swift paths, so the production Swift ambient-global-state rule is inapplicable.
Title check ✅ Passed The title clearly identifies the validated E2E dispatcher and the agent time-discipline documentation changes.
Description check ✅ Passed The description explains the changes and testing, but it omits the template's Review Trigger and Checklist sections.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat-agent-time-discipline

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CLAUDE.md`:
- Around line 65-74: Update the wrapper guidance in CLAUDE.md lines 65-74 and
beside the wrapper command in
skills/cmux-testing/references/local-vs-ci-validation.md line 20 to state that
the command must run from a worktree checked out at the same branch or SHA
passed via --ref; make no other changes.

In `@scripts/dispatch-e2e.sh`:
- Around line 272-288: Replace the displayTitle-based matching in resolve_run_id
with an authoritative dispatch-to-run correlation mechanism. Do not select runs
by title prefix or snapshot ordering; when no reliable run ID is returned, fail
closed and require an explicit run ID for --watch, ensuring the script never
watches an unrelated run.
- Around line 305-307: Update the dispatch validation flow around validate_item
and the CLEAN_ITEMS loop to build a per-target class and method index from
class_files once before iterating filters. Pass or otherwise reuse that index
for each validate_item call, replacing repeated recursive scans while preserving
the existing validation results and filtering behavior.
- Around line 275-291: Replace the fixed 15-attempt loop around the `gh run
list` lookup with an explicit completion or correlation signal that confirms the
target workflow run is available. Remove the two-second `sleep` and avoid
retrying `gh run list` on a wall-clock schedule; preserve filtering by `pre_ids`
and the `"$filter on "*` title match when resolving the run.
- Around line 207-239: Update the selector validation around method discovery to
build a class-qualified index of test methods, including class extensions,
rather than matching any func in a file containing the class. Ensure only test
methods belonging to $cls validate, while helper and lifecycle methods are
rejected; use this same index for candidate suggestions. Add a regression
fixture covering multiple classes or a helper method in one file.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 5f297077-809d-4b77-af01-dec70d5f587f

📥 Commits

Reviewing files that changed from the base of the PR and between c8b9afc and c1b52cd.

📒 Files selected for processing (3)
  • CLAUDE.md
  • scripts/dispatch-e2e.sh
  • skills/cmux-testing/references/local-vs-ci-validation.md

Comment thread CLAUDE.md
Comment on lines +65 to +74
Dispatch hosted UI/E2E runs through the validated wrapper. It checks every filter item against the local test sources and refuses to dispatch a filter that matches zero tests, which otherwise wastes a full dispatch+watch cycle before the run fails:

```bash
./scripts/dispatch-e2e.sh --ref <branch-or-sha> --filter "<Class or Class/method>[,more]" --watch
```

`--dry-run` validates and prints the `gh` command without dispatching; `--runner`, `--record-video`, `--timeout` (per-test seconds), and `--job-timeout` (minutes) pass through. Raw fallback:

```bash
gh workflow run test-e2e.yml --repo manaflow-ai/cmux -f ref=<branch-or-sha> -f test_filter="<Class or Class/method>"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

State that the checkout must match the dispatched ref.

The wrapper validates sources from the current checkout, not from --ref. If these differ, validation can reject a selector that exists in the dispatched ref or approve one that does not.

  • CLAUDE.md#L65-L74: State that the command must run from a worktree checked out at <branch-or-sha>.
  • skills/cmux-testing/references/local-vs-ci-validation.md#L20-L20: Add the same checkout/ref precondition beside the wrapper command.
📍 Affects 2 files
  • CLAUDE.md#L65-L74 (this comment)
  • skills/cmux-testing/references/local-vs-ci-validation.md#L20-L20
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CLAUDE.md` around lines 65 - 74, Update the wrapper guidance in CLAUDE.md
lines 65-74 and beside the wrapper command in
skills/cmux-testing/references/local-vs-ci-validation.md line 20 to state that
the command must run from a worktree checked out at the same branch or SHA
passed via --ref; make no other changes.

Comment thread scripts/dispatch-e2e.sh
Comment thread scripts/dispatch-e2e.sh
Comment thread scripts/dispatch-e2e.sh
Comment thread scripts/dispatch-e2e.sh
Comment on lines +305 to +307
for item in "${CLEAN_ITEMS[@]}"; do
validate_item "$item"
done

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 Performance & Scalability | 🟠 Major | 🏗️ Heavy lift

Build the test-source index once for the filter batch.

Each validate_item call performs recursive target scans through class_files. For F filters and S Swift files, this is O(F × S) file scanning. A batch of 100 filters across 1,000 test files can cause about 100,000 file inspections.

Build one per-target class and method index before this loop. Query that index for each filter. As per coding guidelines, avoid repeated batch rescans over scalable collections. As per path instructions, apply .github/review-bot-rules/algorithmic-complexity.md.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/dispatch-e2e.sh` around lines 305 - 307, Update the dispatch
validation flow around validate_item and the CLEAN_ITEMS loop to build a
per-target class and method index from class_files once before iterating
filters. Pass or otherwise reuse that index for each validate_item call,
replacing repeated recursive scans while preserving the existing validation
results and filtering behavior.

Sources: Coding guidelines, Path instructions

The org removed required status checks from manaflow-ai/cmux; all PR
checks are advisory review bots (verified: zero required_status_checks
rules on main). The first-pass paragraph now says merge validation
happens via the merge gate, not by watching PR checks. Hosted E2E
dispatch gains a once-per-dogfood-round cap and a no
dispatch-fix-redispatch rule (root-cause red runs locally or on a fleet
simulator; only a new XCUITest needs one green hosted run), mirrored in
the cmux-testing skill reference and as an agent time discipline rule.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CLAUDE.md`:
- Line 77: Update the dispatch-cap guidance near “At most one dispatch per
dogfood round” to define the limit in terms of hosted runs, and explicitly
require a single comma-separated filter item for this workflow. Ensure the
wording accounts for scripts/dispatch-e2e.sh creating one hosted run per filter
item.

In `@skills/cmux-testing/references/local-vs-ci-validation.md`:
- Line 20: Update the raw GitHub Actions fallback command in the local-vs-ci
validation guidance to include both the target branch/ref and the selected
test_filter, matching the complete invocation documented in CLAUDE.md. Keep the
existing dispatch and filter guidance unchanged.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: cbbc7d24-8c5a-4230-9182-3aa1f8ece946

📥 Commits

Reviewing files that changed from the base of the PR and between c1b52cd and 96181f6.

📒 Files selected for processing (2)
  • CLAUDE.md
  • skills/cmux-testing/references/local-vs-ci-validation.md

Comment thread CLAUDE.md Outdated
gh workflow run test-e2e.yml --repo manaflow-ai/cmux -f ref=<branch-or-sha> -f test_filter="<Class or Class/method>"
```

At most one dispatch per dogfood round. A red run means root-cause locally or on a fleet simulator first; never loop dispatch-fix-redispatch. A new XCUITest needs one green hosted run before the task is done; nothing else does.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Define the dispatch cap in terms of hosted runs.

scripts/dispatch-e2e.sh dispatches one hosted run for each comma-separated filter item. Therefore, --filter "ClassA,ClassB" creates two hosted runs although Line 77 says “At most one dispatch per dogfood round.” If the limit is one hosted run, reject multiple filter items for this workflow or document that only one filter item is allowed per round.

Proposed wording
-At most one dispatch per dogfood round.
+Use at most one filter item per dogfood round. The wrapper dispatches one hosted run per comma-separated filter item.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
At most one dispatch per dogfood round. A red run means root-cause locally or on a fleet simulator first; never loop dispatch-fix-redispatch. A new XCUITest needs one green hosted run before the task is done; nothing else does.
Use at most one filter item per dogfood round. The wrapper dispatches one hosted run per comma-separated filter item. A red run means root-cause locally or on a fleet simulator first; never loop dispatch-fix-redispatch. A new XCUITest needs one green hosted run before the task is done; nothing else does.
🧰 Tools
🪛 LanguageTool

[grammar] ~77-~77: Ensure spelling is correct
Context: .../method>" ``` At most one dispatch per dogfood round. A red run means root-cause local...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CLAUDE.md` at line 77, Update the dispatch-cap guidance near “At most one
dispatch per dogfood round” to define the limit in terms of hosted runs, and
explicitly require a single comma-separated filter item for this workflow.
Ensure the wording accounts for scripts/dispatch-e2e.sh creating one hosted run
per filter item.

Comment thread skills/cmux-testing/references/local-vs-ci-validation.md Outdated
azooz2003-bit and others added 4 commits August 12, 2026 19:43
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…un dispatch cap, executable raw fallback)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ved runs

Method filters now validate inside class/extension blocks of the target class
(brace-depth awk scope), so a same-file helper or sibling class cannot satisfy
a selector XCTest would resolve to zero tests. --watch exits nonzero when a
dispatched run id could not be resolved instead of silently omitting it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/dispatch-e2e.sh`:
- Around line 141-160: Replace the raw awk brace-counting and grep logic in
class_scoped_lines and the XCTest indexing block with token-aware parsing that
ignores comments and string literals, tracks the target class body, and matches
only direct parameterless func test*() declarations. Ensure nested functions and
sibling classes are excluded, including when braces appear in comments or
strings, and add fixtures covering nested functions, comments, and
brace-containing strings.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 515a0e0f-9662-4493-999b-e98eb60342f8

📥 Commits

Reviewing files that changed from the base of the PR and between 03e3a26 and 8d93900.

📒 Files selected for processing (1)
  • scripts/dispatch-e2e.sh

Comment thread scripts/dispatch-e2e.sh
Comment on lines +141 to +160
awk -v cls="$cls" '
FNR == 1 { inside = 0; depth = 0; seen_open = 0 }
{
if (!inside) {
if ($0 ~ ("(^|[^A-Za-z0-9_])(class|extension)[[:space:]]+" cls "([^A-Za-z0-9_]|$)")) {
inside = 1; depth = 0; seen_open = 0
} else {
next
}
}
print
line = $0
o = gsub(/{/, "", line)
c = gsub(/}/, "", line)
depth += o - c
if (o > 0) seen_open = 1
if (seen_open && depth <= 0) inside = 0
}
' "$@"
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Audit test-method declarations and their surrounding source before adding
# regression fixtures for nested functions and brace-containing strings.
fd -e swift . cmuxUITests cmuxTests -x \
  rg -n -C 5 'func[[:space:]]+test[A-Za-z0-9_]*[[:space:]]*\(\)'

Repository: manaflow-ai/cmux

Length of output: 50373


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- scripts/dispatch-e2e.sh: relevant definitions and callers ---'
sed -n '100,175p;230,285p;340,420p' scripts/dispatch-e2e.sh

printf '%s\n' '--- selector-related references ---'
rg -n -C 4 'class_scoped_lines|test[A-Za-z0-9_]*|selector|awk' scripts/dispatch-e2e.sh

printf '%s\n' '--- Swift test declarations in likely test roots, without context expansion ---'
for root in cmuxUITests cmuxTests Packages; do
  if [ -d "$root" ]; then
    rg -n --glob '*.swift' '(^|[[:space:]])func[[:space:]]+test[A-Za-z0-9_]*[[:space:]]*\(' "$root" | head -n 120
  fi
done

Repository: manaflow-ai/cmux

Length of output: 49876


🏁 Script executed:

#!/bin/bash
set -euo pipefail

tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT

cat >"$tmpdir/fixture.swift" <<'SWIFT'
final class OuterTests: XCTestCase {
    func testDirect() {}
    func helper() {
        func testNested() {}
        let text = "a { brace }"
    }
}

extension OuterTests {
    func testExtensionDirect() {}
}

final class NextTests: XCTestCase {
    func testAdjacent() {}
}
SWIFT

# Extract the exact awk block from the reviewed script and apply it to the fixture.
awk '
  /awk -v cls="\$cls"/ { capture = 1 }
  capture { print }
  capture && /^  '\'' "\$@"/ { exit }
' scripts/dispatch-e2e.sh >"$tmpdir/extractor.awk"

printf '%s\n' '--- extracted awk implementation ---'
cat "$tmpdir/extractor.awk"

printf '%s\n' '--- class-scoped output for OuterTests ---'
awk -v cls='OuterTests' '
  FNR == 1 { inside = 0; depth = 0; seen_open = 0 }
  {
    if (!inside) {
      if ($0 ~ ("(^|[^A-Za-z0-9_])(class|extension)[[:space:]]+" cls "([^A-Za-z0-9_]|$)")) {
        inside = 1; depth = 0; seen_open = 0
      } else {
        next
      }
    }
    print
    line = $0
    o = gsub(/{/, "", line)
    c = gsub(/}/, "", line)
    depth += o - c
    if (o > 0) seen_open = 1
    if (seen_open && depth <= 0) inside = 0
  }
' "$tmpdir/fixture.swift"

printf '%s\n' '--- direct declaration candidates from extracted output ---'
awk -v cls='OuterTests' '
  FNR == 1 { inside = 0; depth = 0; seen_open = 0 }
  {
    if (!inside) {
      if ($0 ~ ("(^|[^A-Za-z0-9_])(class|extension)[[:space:]]+" cls "([^A-Za-z0-9_]|$)")) {
        inside = 1; depth = 0; seen_open = 0
      } else {
        next
      }
    }
    if ($0 ~ /func[[:space:]]+test[A-Za-z0-9_]*/) print FNR ":" $0
    line = $0
    o = gsub(/{/, "", line)
    c = gsub(/}/, "", line)
    depth += o - c
    if (o > 0) seen_open = 1
    if (seen_open && depth <= 0) inside = 0
  }
' "$tmpdir/fixture.swift"

Repository: manaflow-ai/cmux

Length of output: 1096


🏁 Script executed:

#!/bin/bash
set -euo pipefail

tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT

cat >"$tmpdir/fixture.swift" <<'SWIFT'
final class EarlyEndTests: XCTestCase {
    let text = "}"
    func testMissedAfterString() {}
}

final class LateEndTests: XCTestCase {
    let text = "{"
    func testIncludedBeforeSibling() {}
}
final class SiblingTests: XCTestCase {
    func testSibling() {}
}

final class CommentEndTests: XCTestCase {
    // }
    func testMissedAfterComment() {}
}

final class FalsePositiveTests: XCTestCase {
    // func testCommentedOut() {}
    let text = "func testQuoted() {}"
    func helper() {
        func testNested() {}
    }
}
SWIFT

extract='
  FNR == 1 { inside = 0; depth = 0; seen_open = 0 }
  {
    if (!inside) {
      if ($0 ~ ("(^|[^A-Za-z0-9_])(class|extension)[[:space:]]+" cls "([^A-Za-z0-9_]|$)")) {
        inside = 1; depth = 0; seen_open = 0
      } else {
        next
      }
    }
    print
    line = $0
    o = gsub(/{/, "", line)
    c = gsub(/}/, "", line)
    depth += o - c
    if (o > 0) seen_open = 1
    if (seen_open && depth <= 0) inside = 0
  }
'

for cls in EarlyEndTests LateEndTests CommentEndTests FalsePositiveTests; do
  printf '%s\n' "--- scoped lines for $cls ---"
  awk -v cls="$cls" "$extract" "$tmpdir/fixture.swift"
done

printf '%s\n' '--- exact method-validation pattern for FalsePositiveTests ---'
awk -v cls='FalsePositiveTests' "$extract" "$tmpdir/fixture.swift" |
  grep -E 'func[[:space:]]+test[A-Za-z0-9_]*[[:space:]]*\(\)' || true

Repository: manaflow-ai/cmux

Length of output: 1022


Index only direct XCTest method declarations.

class_scoped_lines and the grep at lines 260-271 accept nested functions, comments, and string literals as test methods. Raw brace counting also misses methods after } in a string or includes sibling classes after { in a string or comment. Use token-aware parsing to ignore comments and strings and match only direct parameterless func test*() members. Add fixtures for nested functions, comments, and brace-containing strings.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/dispatch-e2e.sh` around lines 141 - 160, Replace the raw awk
brace-counting and grep logic in class_scoped_lines and the XCTest indexing
block with token-aware parsing that ignores comments and string literals, tracks
the target class body, and matches only direct parameterless func test*()
declarations. Ensure nested functions and sibling classes are excluded,
including when braces appear in comments or strings, and add fixtures covering
nested functions, comments, and brace-containing strings.

…ly discoverable test methods

test-e2e.yml gains an optional dispatch_id input echoed into the run name;
the dispatcher passes a per-dispatch nonce and resolves its exact run by it,
falling back to the pre-existing-ids heuristic (with one no-nonce retry) while
main's workflow predates the input. Method validation now only accepts
depth-1 instance members of the class or its extensions, rejecting static,
class, private, and fileprivate funcs plus nested functions and nested types'
methods, matching XCTest discovery rules.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant