Skip to content

docs: PR #387 Kiro×Claude evaluation & reconciliation plan (review before code) - #15

Closed
plauzy wants to merge 1 commit into
feat/agentic-protocols-generative-uifrom
docs/pr387-reconciliation
Closed

plauzy wants to merge 1 commit into
feat/agentic-protocols-generative-uifrom
docs/pr387-reconciliation

Conversation

@plauzy

@plauzy plauzy commented Jul 6, 2026

Copy link
Copy Markdown
Owner

What

Plan-only deliverable for the head-to-head evaluation of the Kiro and Claude remediations of upstream PR awslabs#387, plus the reconciliation plan to produce reconcile/pr387-agui-core and reconcile/pr387-a2a-hardened.

No reconciliation code is written yet — this PR exists so the plan is reviewable first.

Contents (docs/reviews/pr387-reconciliation-plan.md)

  • Phase 1 scorecard (empirical): gates executed on all four branches — exact pytest/mypy/black/isort + PWA numbers, every failure triaged against the diff.
  • Phase 2 reconciliation plan: per-contrast decision + one-line rationale tied to the priority-ordered goals; unions complementary test suites; carries the PWA cursor-loss fix regardless of source.
  • What both missed: emit_ui rate-limiting; STATE_DELTA debounce (both verified absent).

Headline empirical findings

  • kiro/pr387-agui-core: 3487 passed / 0 failed, all gates green.
  • claude/pr387-agui-core: 3 failed — 2 self-authored telemetry tests broken on Python 3.12 (find_module removed) + 1 pre-existing macOS fifo flake.
  • claude/pr387-a2a-hardened: 3595 passed / 0 failed.
  • kiro/pr387-a2a-hardened: 3599 passed, 1 unrelated env flake.
  • Cursor-loss fix: Claude fixes cao_pwa/src/api.ts (native EventSource drops the ?since= cursor on reconnect); Kiro's api.ts is byte-identical to base (ships the bug) — corroborated by Kiro's own commit 660e3f4.
  • PR-B: Claude authenticates-before-parse + fixes the reset_jwks_cache latent break + real-JWT roundtrip; Kiro has the broader 28-case (monkeypatched) matrix + cleaner extracted mount guard.

Constraints honored

No pushes to feat/*, kiro/*, or claude/*. Reconciled branches will be reconcile/*, opened as drafts based on feat/agentic-protocols-generative-ui, PR-A before PR-B (per @fanhongy).

Verified empirically on macOS arm64; live Playwright specs were environment-blocked (only chrome-headless-shell cached, no full Chromium) — documented in §0, reconnect correctness verified via source + green PWA unit suites instead.

…tion plan

Phase-1 head-to-head scorecard (empirical: gates run on all four branches),
Phase-2 reconciliation plan (per-contrast decisions + rationale, stacked on
f40933d as reconcile/*), and a 'what both missed' list. Plan only — no
reconciliation code until approved.

Co-authored-by: Kiro Agent <244629292+kiro-agent@users.noreply.github.com>
@plauzy

plauzy commented Sep 28, 2026

Copy link
Copy Markdown
Owner Author

Closing: the plan this PR proposes has been executed and merged upstream (awslabs#436 and awslabs#485), so there is no action left to take on the PR. The file itself is worth keeping and has been archived to disk — it is the macOS/arm64 half of a cross-platform gate pair with #16, and the two runs independently identify the same real defect (test/telemetry/test_optional_extra.py using the legacy find_module/load_module finder API that Python 3.12 no longer calls) while disagreeing only on platform-specific flakes. Branch preserved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant