Skip to content

feat(skills): add change-risk model reference for code-review depth selection - #2550

Open
pratik wayase (PratikWayase) wants to merge 8 commits into
microsoft:mainfrom
PratikWayase:feat/change-risk-model-reference-for-code-review
Open

feat(skills): add change-risk model reference for code-review depth selection#2550
pratik wayase (PratikWayase) wants to merge 8 commits into
microsoft:mainfrom
PratikWayase:feat/change-risk-model-reference-for-code-review

Conversation

@PratikWayase

Copy link
Copy Markdown
Contributor

Description

Implements Phase 1 of the quantitative change-risk modelling proposal for the code-review skill. This PR adds a documentation-only reference (change-risk-model.md) that defines a deterministic, git-computable signal taxonomy (Likelihood, Severity, Detectability, Recoverability) alongside agentic-era modifiers.

It replaces heuristic "gut-feel" risk assessment with an evidence-based Change-Risk Profile. The profile is computed during the Tier 0 context bootstrap and directly informs human-confirmed scoping, hotspot ranking, and depth-tier selection without acting as a hard gate.

Related Issue(s)

Resolves #2501

Type of Change

  • Documentation update

(Note: Updates reference files within the existing code-review skill. Does not alter SKILL.md or add executable scripts.)

Sample Prompts

N/A — Documentation-only reference update. No new executable AI artifacts, prompts, or scripts were added.

Testing

Documentation only. Verified markdown formatting, heading hierarchy, and internal link resolution.

Local validation commands successfully run against changed files:

  • npm run spell-check
  • npm run validate:skills
  • npm run lint:md-links ✅ (Validated links in all 4 modified files)

Checklist

Required Checks

  • Documentation is updated
  • Files follow existing naming conventions
  • Changes are backwards compatible
  • Tests added for new functionality (N/A - Documentation only)

AI Artifact Contributions

  • Used hve-builder review mode to review contribution (N/A - Documentation only)
  • Addressed all actionable findings from the hve-builder review (N/A)
  • Verified contribution follows common standards and type-specific requirements

Required Local Checks

  • Local validation aggregate: npm run validate:local (Skipped - targeted checks run instead)
  • Documentation validation: npm run validate:docs (Skipped - targeted checks run instead)
  • Spell checking: npm run spell-check
  • Link validation: npm run lint:md-links
  • Skill structure validation: npm run validate:skills

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues (N/A)
  • Security-related scripts follow the principle of least privilege (N/A)

Additional Notes

This PR strictly implements the "Definition of Done" for Phase 1 (knowledge-level model) as requested in #2501.

Strict Scope Constraints Met:

  • No executable scripts added.
  • No skill collection manifest (SKILL.md) changes.
  • No session/transcript capture mechanisms introduced.
  • No compliance or regulatory claims (e.g., CPS 230, APRA) are made in the repository files.

Out of Scope (Follow-up Sub-issues):
If this foundational reference is accepted, subsequent issues will be filed for:

  1. Phase 2: Script-backed change-risk skill (bash + PowerShell parity, Pester tests, 80% coverage) emitting the profile as JSON.
  2. Phase 3: Calibration guidance and eval cases against repo history.
  3. Phase 4: Schema'd evidence emission via the persisted emission-record contract.

@PratikWayase
pratik wayase (PratikWayase) requested a review from a team as a code owner July 29, 2026 10:09
## Tier model

Review depth is a verification-rigor dial, not a lane-selection mechanism. The selected perspectives determine which review lanes run; the selected depth tier determines how deeply each lane verifies the confirmed change scope.
Review depth is a verification-rigor dial, not a lane-selection mechanism. The selected perspectives determine which review lanes run; the selected depth tier determines how deeply each lane verifies the confirmed change scope. Depth-tier recommendations are driven by the evidence provided in the **Change-Risk Profile** (see [Change-Risk Model](change-risk-model.md)), mapping deterministic, git-computable signals to verification rigor rather than relying on heuristic "gut feel".

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the ms.date frontmatter field was not updated in any of the three modified files. Repo convention requires updating this field to the edit date whenever a file that carries it is changed. Applies to depth-tiers.md (2026-06-18), context-bootstrap.md (2026-06-26), and severity-taxonomy.md (2026-06-18); all should be updated to 2026-07-29.

@rezatnoMsirhC

Copy link
Copy Markdown
Contributor

The new change-risk-model.md reference is cross-linked from three existing reference documents but is not listed in the SKILL.md discovery index (.github/skills/coding-standards/code-review/SKILL.md). Agents load the skill via that numbered list; a missing entry means the document won't be surfaced during skill load unless a secondary link is followed. Consider appending it as item 10:

10. [Change-Risk Model](references/change-risk-model.md) — deterministic signal taxonomy and scoring rubric for evidence-based change-risk profiling.

@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.47%. Comparing base (207ced2) to head (24d884d).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2550      +/-   ##
==========================================
- Coverage   82.53%   82.47%   -0.06%     
==========================================
  Files         155      143      -12     
  Lines       21123    21033      -90     
  Branches       13        0      -13     
==========================================
- Hits        17434    17348      -86     
+ Misses       3687     3685       -2     
+ Partials        2        0       -2     
Flag Coverage Δ
docusaurus ?
pester 85.62% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 13 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@PratikWayase
pratik wayase (PratikWayase) force-pushed the feat/change-risk-model-reference-for-code-review branch from 1210811 to 8f57304 Compare July 31, 2026 06:33
4. Present the emerging brief and hotspot candidates to the human for confirmation and correction.
5. Invite the human to add or remove hotspots and to mark out-of-scope areas before review lanes dispatch.
6. Persist the confirmed brief, the scoped hotspot list, the tagged specialist concerns, and out-of-scope areas as the review context for later aggregation.
2. Draft a **Change-Risk Profile** alongside the change brief using the deterministic signals defined in [Change-Risk Model](change-risk-model.md). Evaluate Likelihood (size, diffusion, entropy), Severity (path criticality), Detectability (test presence), and Recoverability (rollback markers) using `git log` and `git diff` heuristics.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Heuristics" in step 2 contradicts the "deterministic" framing established in the change-risk-model.md document that this step directly cites. The model opens with: "All signals are computable deterministically using git log and git diff." Please replace "heuristics" with "signals":

...Evaluate Likelihood (size, diffusion, entropy), Severity (path criticality),
Detectability (test presence), and Recoverability (rollback markers) using
`git log` and `git diff` signals.

Comment thread .github/skills/coding-standards/code-review/references/severity-taxonomy.md Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(skills): change-risk model reference for code-review depth selection

4 participants