Repository navigation
feat(agents): emit decision.answered and decision.defaulted telemetry (spec 136) - #929
Merged
Merged
Conversation
Claude-Session: https://claude.ai/code/session_014Zwb23kb4xTBxJtqc2hKbU Co-Authored-By: Shep Bot <shep-agent@users.noreply.github.com>
…ion.defaulted decision.answered carries DecisionKind and the new DecisionSurface enum instead of raw strings. DecisionOutcome gains Disabled for an ask made while agent questions are off; stored questions keep the narrower RecordedDecisionOutcome. Claude-Session: https://claude.ai/code/session_014Zwb23kb4xTBxJtqc2hKbU Co-Authored-By: Shep Bot <shep-agent@users.noreply.github.com>
…endation pick AnswerAgentQuestionUseCase records one decision.answered after its settle write wins, with the decision kind, the surface its caller names (web, cli, chat, supervisor), a latency bucket and whether the recommended option was picked. A lost race or a refused answer records nothing. Claude-Session: https://claude.ai/code/session_014Zwb23kb4xTBxJtqc2hKbU Co-Authored-By: Shep Bot <shep-agent@users.noreply.github.com>
…eadline AskAgentDecisionUseCase records decision.defaulted with the decision kind and the deadline bucket when its expired settle wins; a person answering at the deadline records nothing. Its outcome is now the DecisionOutcome enum instead of a string union, with identical wire values. Claude-Session: https://claude.ai/code/session_014Zwb23kb4xTBxJtqc2hKbU Co-Authored-By: Shep Bot <shep-agent@users.noreply.github.com>
Claude-Session: https://claude.ai/code/session_014Zwb23kb4xTBxJtqc2hKbU Co-Authored-By: Shep Bot <shep-agent@users.noreply.github.com>
135 belongs to the review-cleanup spec in #925. Claude-Session: https://claude.ai/code/session_014Zwb23kb4xTBxJtqc2hKbU Co-Authored-By: Shep Bot <shep-agent@users.noreply.github.com>
…emetry Co-Authored-By: Shep Bot <shep-agent@users.noreply.github.com>
arielshad
pushed a commit
that referenced
this pull request
Oct 11, 2026
<p align="center"> <a href="https://github.com/shep-ai/shep"> <img src="https://raw.githubusercontent.com/shep-ai/shep/main/docs/screenshots/shep-card.jpg" alt="Shep — run multiple AI agents in parallel" width="720" /> </a> </p> # 🚀 Shep [v1.241.0](/compare/v1.240.0...v1.241.0) · _2026-10-11_ > Your organization does not have access to Claude. Please login again or contact your administrator. ### ✨ Features * **agents:** emit decision.answered and decision.defaulted telemetry (spec 136) ([#929](#929)) ([1e1f859](1e1f859)), closes [#922](#922) [#925](#925)     * **agents:** git checkpoints after each feature-agent step (spec 138) ([#931](#931)) ([26265ed](26265ed)), closes [#925](#925)     * **web:** put the SDLC board behind a feature flag, off by default ([#932](#932)) ([e71f138](e71f138)), closes [#925](#925) ### 🐛 Bug Fixes * **cli:** wait for the conpty host to exit when a terminal closes ([#928](#928)) ([94a846d](94a846d)), closes [#925](#925) [#925](#925) ## 📦 Install or update ```bash # upgrade an existing install npm i -g @shepai/cli@1.241.0 # or run instantly without installing npx @shepai/cli@latest ``` ## 💬 Join the community [💬 **Discord**](https://discord.gg/ES6tdVFfur) · [📖 **Docs**](https://github.com/shep-ai/shep#readme) · [⭐ **Star on GitHub**](https://github.com/shep-ai/shep) · [🐛 **Report an issue**](https://github.com/shep-ai/shep/issues) --- <sub>🤖 Released autonomously by Shep — built by parallel AI agents working in isolated git worktrees. Try it: `npx @shepai/cli`</sub> Co-Authored-By: Shep Bot <shep-agent@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This is the telemetry follow-up noted in #922 and #924. Every recorded answer to an agent decision now emits
decision.answered, and a background agent's question that reaches its deadline emits the newdecision.defaultedevent. Both go through theITelemetryport from #923.decision.answeredkind(DecisionKind),surface(newDecisionSurface: web / cli / chat / supervisor / other),latency(bucket from asked to answered),pickedRecommendedAnswerAgentQuestionUseCase, once per answer it actually recordsdecision.defaultedkind,timeout(bucket of the deadline the agent set)AskAgentDecisionUseCase, when its deadline settle winsWhy
Spec 133 defined
decision.answeredbut did not emit it, and itskindandsurfacewere raw strings. The T3 Code review asks three things that need data: whether people answer agent questions, where they answer them, and whether they take the recommendation. It also asks how often agents proceed on their own at the deadline.settlePendingsucceeds. Each decision is therefore counted exactly once without anonceKey. A lost race (CLI vs web), a refused answer, or agent questions being off records nothing. A person answering at the deadline records an answer, not a default.web, which overrides anything the client sends), the CLI (cli), the chat bridge (chat) and the supervisor router (supervisor). Nothing parses the free-formansweredByactor.kindandsurfaceare now enums. That makes the compiler reject a content property at every emit site.AskAgentDecisionOutcomestring union is now theDecisionOutcomeenum, with a newDisabledmember. The MCP tool's wire values are unchanged. Stored questions useRecordedDecisionOutcome, which excludesDisabled, so the activity log needs no UI for a state it can never show.docs/telemetry.mdlists both events. Spec:specs/136-decision-telemetry/. It was renumbered from 135, which belongs to feat(web): review cleanup — per-area feature flags, ASPM off by default #925; the branch name still says 135.Screenshots / Recording
No UI changes.
Testing
answer-agent-question.use-case.test.ts:pickedRecommended: true.Legacy, withOtheras the default surface.ask-agent-decision.use-case.test.ts:decision.defaultedwith the timeout bucket.answer(with--answerand interactive), chat bridge, supervisor router.Ran locally:
pnpm lint,format:check,check:stories,typecheck,test:unit(14864 passed),test:int(2132 passed),build,build:web,build:storybook, andgeneratewith a clean diff. After the renumbering: lint, format:check, typecheck and the affected test files.Checklist
pnpm lintpassespnpm format:checkpassespnpm typecheckpassespnpm test:unitandpnpm test:intpasspnpm buildsucceedspnpm tsp:compileran andpackages/core/src/domain/generated/output.tsis committeddomain/orapplication/file imports anything frominfrastructure/🤖 Generated with Claude Code
https://claude.ai/code/session_014Zwb23kb4xTBxJtqc2hKbU