fix(uipath-feedback): relevance-filtered prompts/retrospective, optional Expected/Actual framing

[purnimaviswanath]

Summary

• Fixes UV-16327: Step 2e ("Capture sample prompts") selected prompts by session intent rather than relevance to the reported issue, so agent-discovered bugs got padded with 3-5 topical-but-irrelevant prompts to fill a quota. Reframes the criterion around a relevance test ("would a triager learn something new from this prompt + ## Error?"), makes 5 an explicit ceiling not a target, and states the zero-relevant-prompts case as normal (per UV-16327's suggested fix).
• Applies the same relevance-and-omission treatment to Step 2f's Session retrospective: each of the 5 questions, and the section as a whole, is now optional and omitted by default when it would otherwise be filled with generic/obvious content, instead of always emitting all 5 in full. This was a second source of boilerplate-y tickets beyond Step 2e.
• Adds new optional Step 2h: before building the description, offer the user one skippable chance (via AskUserQuestion, Skip (Recommended) on both) to state Expected outcome / What actually happened in their own words. If answered, it supplements ## What happened (quoted verbatim, sanitized) instead of the agent's own paraphrase; if skipped, behavior is unchanged from today.
• Environment, Troubleshooting, Error, --description-file mechanism, and the Area/CLI-Skill title tagging are untouched.

All content edits are to the canonical skills/uipath-feedback/SKILL.md only — no flavor-marker passages were touched, so the Studio Web override needs no change.

Testing

Structural / mechanical (passing):

• npm run skills:validate — Studio Web flavor composes cleanly, 0 broken flavor markers.
• npm run skills:check-links — all 6597 relative links across skills/flavors resolve.
• Verified --description-file (used by the untouched Step 4) exists on the latest published @uipath/cli (1.202.0-dev.8521) — ruling out a false alarm from a stale local CLI install.

Cold dry-run testing (the main verification for this PR): two independent fresh agents — no knowledge of this PR's intent, given only the edited SKILL.md and a fabricated transcript, explicitly forbidden from touching Bash/uip — traced Steps 1-3 by hand and stopped before any real send:

• Scenario A (agent-discovered bug, no relevant prompts): correctly omitted ## Sample prompts (0/3 candidates passed relevance) and included only the 3/5 ## Session retrospective bullets that actually cleared the bar.
• Scenario B (session-intent bug, user states Expected/Actual in plain conversation): correctly reused the user's own words under ## What happened via Step 2h without re-asking — but surfaced a real bug: the same sentence was also being duplicated into ## Sample prompts, since Step 2e's relevance test only checked against ## Error and had no awareness of what Step 2h already placed under ## What happened. Fixed in a follow-up commit (dedupe rule added to Step 2e), along with two smaller clarifications the dry runs flagged (splitting a compound "expected X but got Y" sentence across the two Step 2h labels; an inline ceiling-not-target reminder on the "Top 3 Improvements" template line, since the bare template alone still nudged toward padding).

Test coverage / regression scaffold (partial — flagging explicitly):

• Generated tests/reports/uipath-feedback.md via /test-coverage (not committed — tests/reports/ is gitignored by repo convention). Confirmed the skill has zero existing coder_eval tasks; inventoried 24 components / 5 workflow steps / 8 critical rules / 12 anti-patterns and recommended 4 starter tests.
• Added tests/tasks/uipath-feedback/smoke/omit_when_irrelevant.yaml — the skill's first coder_eval task, targeting exactly the Scenario-A behavior above (Sample Prompts omitted, not padded, when the issue is agent-discovered). Self-linted per .claude/commands/lint-task.md: verdict Low (2 Low issues, both documented trade-offs — see the task's description field and the commit message; no Critical/High/Medium findings). scripts/check-cli-verbs.py exits clean.
• Design note specific to this skill: CI's smoke workflow authenticates against a real tenant (real UIPATH_ROPC_CLIENT_ID/SECRET secrets), so a task that actually reaches uip feedback send risks filing a real ticket in production Jira on every PR run — unlike most skills where the mutated state is a disposable test-tenant resource. The task is built so it never invokes the real send command regardless of CI auth state (persists the built description to a local file instead, with a hard command_not_executed guard on uip feedback send as a backstop), rather than relying on the sandbox happening to lack credentials.
• NOT YET RUN with coder-eval. Installing the harness (cd tests && make install) and executing a live sandboxed agent session was intentionally deferred — real infra/token cost that needs a separate go-ahead. This is an unverified scaffold per the repo's own /generate-task convention. .github/workflows/lint-tasks.yml will raise a High finding on this PR for the missing passing-run claim — that's expected; running skill-feedback-omit-when-irrelevant and confirming a pass is called out here as required follow-up before merge, either by a reviewer or in a follow-up commit.

Test plan

• npm run skills:validate
• npm run skills:check-links
• Two independent cold dry-runs (documented above), one bug found and fixed
• tests/tasks/uipath-feedback/smoke/omit_when_irrelevant.yaml authored + self-linted (Low)
• Run skill-feedback-omit-when-irrelevant with coder-eval and confirm a pass (deferred — see note above)
• Manual run of /uipath-feedback on a live agent-discovered issue and a live session-intent bug, to sanity-check beyond the dry runs

🤖 Generated with Claude Code

[github-actions]

Claude finished @purnimaviswanath's task in 2m 51s —— View job

---

PR Review: fix(uipath-feedback): relevance-filtered prompts/retrospective, optional Expected/Actual framing

• Read review criteria and project conventions
• Read full diff and changed file
• Check skill structure and conventions
• Verify links and cross-skill references
• Post review findings

---

Summary

This PR makes three targeted improvements to skills/uipath-feedback/SKILL.md: (1) reframes Step 2e's sample-prompt selection from a session-intent quota ("3-5") to a relevance test with 5 as a ceiling, (2) applies the same relevance-and-omission treatment to Step 2f's retrospective so each of the 5 bullets (and the section as a whole) is now individually optional, and (3) adds a new Step 2h offering users an optional, always-skippable chance to state Expected/Actual in their own words. The old Steps 2h/2i are renumbered to 2i/2j accordingly.

Change-by-Change Review

1. skills/uipath-feedback/SKILL.md — Step 2e (Capture sample prompts)

Severity: OK

The rewrite is well-structured. The relevance test ("would a triager who reads only this prompt and the ## Error section learn something they didn't already know?") is concrete and agent-actionable — much better than the prior "pick 3-5 prompts showing what the user was trying to do." The "5 is a ceiling, not a target" language and explicit zero-relevant-prompts case are clear. No issues.

2. skills/uipath-feedback/SKILL.md — Step 2f (Session retrospective)

Severity: OK

Each of the five retrospective bullets now has an explicit omission criterion, and the section-level threshold ("fewer than two bullets → omit entirely") is a sound heuristic. The per-bullet guidance is specific enough to be actionable (e.g., "a bare 'Full, as expected' is not worth a bullet"). No issues.

3. skills/uipath-feedback/SKILL.md — Step 2h (Expected/Actual framing)

Severity: OK

The new step is well-integrated: it checks whether Expected/Actual is already stated in conversation before asking, uses AskUserQuestion with Skip (Recommended) defaults, and has clear rules for how answers flow into ## What happened. The non-blocking semantics ("does not trigger 2j's fallback by itself") are explicitly stated. The sanitization rule #8 covers the new user-provided text. No issues.

4. skills/uipath-feedback/SKILL.md — Step renumbering (2h→2i, 2i→2j)

Severity: OK

All cross-references to the old step numbers have been updated consistently. The Step 2 heading (silent by default -- optional/fallback interaction only in 2h/2j) correctly references the new numbering. Critical Rules #1, #7, #8, the design principle callout, formatting rule #9, sanitization rule #8, and "What NOT to Do" #11 and #12 all reference the correct new step numbers. No stale references found.

5. skills/uipath-feedback/SKILL.md — Examples

Severity: OK

Both examples are excellent additions. The "session-intent report" example demonstrates the Expected/Actual framing, a partial retrospective (3 of 5 bullets), and fewer than 5 sample prompts. The "agent-discovered issue" example demonstrates the zero-prompts, zero-retrospective case with an explanatory note. Both examples include clear "Note what's missing on purpose" annotations that reinforce the omission-is-normal philosophy. The first example's retrospective was also trimmed (removed Intent/Outcome and one vague Improvement), which models the guidance.

6. skills/uipath-feedback/SKILL.md — Critical Rules #7 and #8

Severity: OK

New rules #7 ("Omission beats padding") and #8 ("Step 2h offer is opt-in, not required") are clear, numbered, and consistent with the detailed guidance in Steps 2e/2f/2h. Appropriate placement at the end of the Critical Rules section.

7. skills/uipath-feedback/SKILL.md — "What NOT to Do" #11 and #12

Severity: OK

These reinforce the new behavior from the anti-pattern angle. #11 covers the padding anti-pattern for both sections, #12 covers the re-asking anti-pattern for Step 2h. Both are specific and actionable.

8. skills/uipath-feedback/SKILL.md — Flavor markers

Severity: OK

No flavor-marked passages were modified. The studioweb override only covers the description-budget, attachments, preview, send-command, and cleanup blocks — all untouched by this PR. Composition is unaffected.

What's Missing

• No test tasks for uipath-feedback. There is no tests/tasks/uipath-feedback/ directory. This is a pre-existing gap (flagged by the repo's CI bot in another comment on this PR), not introduced by this PR — but worth noting as a Medium issue for future work. The behavioral changes here (relevance filtering, optional Expected/Actual) would benefit from at least a smoke task validating the omission path and an e2e task validating the full flow with Step 2h answers.

Area Ratings

Area	Status	Notes
Frontmatter	OK	Unchanged; name matches folder, description under 1024 chars
E2E Tests	Pre-existing gap	No tests/tasks/uipath-feedback/ — not introduced by this PR
Skill Body	OK	Changes are well-structured, internally consistent, and prescriptive
References & Assets	OK	No references modified; no broken links; internal anchor resolves
Repo Hygiene	OK	CODEOWNERS has wildcard coverage; skill-status.json and skills.sh.json have entries; no secrets or personal paths; scoped to one file

Issues for Manual Review

• Behavioral validation. The two unchecked manual test-plan items (agent-discovered issue producing omitted sections; Step 2h appearing once and being skippable) cannot be verified by static review. These should be confirmed before merge.
• AskUserQuestion interaction. Step 2h specifies Skip (Recommended) as a button label on both questions. Verify that AskUserQuestion supports this UX pattern (offering a skip option alongside a free-text input) in the environments where this skill runs.

Conclusion

Clean, well-scoped PR. The changes are internally consistent, the step renumbering has no stale references, no flavor markers are affected, and the two new examples effectively model both the omission and inclusion paths. The logic is sound — relevance-test criteria are concrete and actionable by an agent. No blocking issues found. Approve.

---

[github-actions]

⚠️ Smoke test coverage gap — the following changed skills have no tests under tests/tasks/:

• uipath-feedback

Consider adding smoke tests before merging.