-
Notifications
You must be signed in to change notification settings - Fork 251
feat(skills): add change-risk model reference for code-review depth selection #2550
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
pratik wayase (PratikWayase)
wants to merge
8
commits into
microsoft:main
Choose a base branch
from
PratikWayase:feat/change-risk-model-reference-for-code-review
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
e67e776
feat(skills): add change-risk model reference for code-review depth s…
PratikWayase ba1b8ab
Merge branch 'main' into feat/change-risk-model-reference-for-code-re…
rezatnoMsirhC 3c10cfc
Merge branch 'main' into feat/change-risk-model-reference-for-code-re…
WilliamBerryiii c9e819f
chore: address review feedback for change-risk-model reference
PratikWayase 8f57304
feat(skills): add change-risk model reference for code-review depth t…
PratikWayase 24d884d
Merge branch 'main' into feat/change-risk-model-reference-for-code-re…
rezatnoMsirhC 3fad5fb
Merge branch 'main' into feat/change-risk-model-reference-for-code-re…
WilliamBerryiii 0fa46b0
fix: address reviewer suggestions and merge main into feat/change-ris…
PratikWayase File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
64 changes: 64 additions & 0 deletions
64
.github/skills/coding-standards/code-review/references/change-risk-model.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| --- | ||
| title: Change-Risk Model | ||
| description: Signal taxonomy and scoring rubric for deterministic, evidence-based change-risk profiling. | ||
| ms.date: 2026-07-29 | ||
| --- | ||
|
|
||
| ## The Change-Risk Profile | ||
|
|
||
| The model produces an advisory **Change-Risk Profile**: a vector of named, evidenced signals, never a single opaque score. To build the profile, evaluate each signal below as High, Medium, or Low based on git history, then combine them into a named-signal profile citing the specific evidence (e.g., "High Diffusion: 12 files across 4 subsystems"). The profile informs the human-scoping step, drives depth-tier selection, and ranks hotspot candidates using deterministic, git-computable signals. | ||
|
|
||
|
|
||
| ## Risk factors | ||
|
|
||
| Evaluate every change across four distinct factors. All signals are computable deterministically using `git log` and `git diff`. | ||
|
|
||
| ### Likelihood | ||
| How likely the change introduces a defect. Measure using: | ||
|
|
||
| * `Size` — lines added or modified. | ||
| * `Diffusion` — number of files, directories, and subsystems touched. | ||
| * `Entropy` — scatter of the change across the codebase. | ||
| * `Hotspot Overlap` — intersection with trailing 90-day high-churn or high-complexity files. | ||
| * `Missed Co-change` — touching one file of a historically coupled cluster while missing its siblings. | ||
|
|
||
| ### Severity | ||
| The blast radius if the change fails. Measure using: | ||
|
|
||
| * `Path Criticality` — classification of touched paths per [Severity Taxonomy](severity-taxonomy.md). | ||
| * *Note: Dependency fan-in and call-graph analysis are deferred to v2 to maintain language-agnostic determinism.* | ||
|
|
||
| ### Detectability | ||
| Whether a defect would be caught before impact. Measure using: | ||
|
|
||
| * `Test Presence` — test files included in the diff. | ||
| * `Coverage` — touched-file test coverage (where CI data is available). | ||
|
|
||
| ### Recoverability | ||
| How quickly the change can be backed out. Measure using: | ||
|
|
||
| * `Reversibility Markers` — presence of feature flags. | ||
| * `Hard-to-Rollback` — schema migrations, database changes, or irreversible config changes versus standard code. | ||
|
|
||
| ## Agentic-era modifiers | ||
|
|
||
| Agent-authored code requires specific adjustments to classic risk models. Apply the following modifiers to the profile: | ||
|
|
||
| * `Provenance-Aware Weighting` — detect agent authorship (e.g., `Co-authored-by:` trailers). Neutralize traditional "author experience" signals, as agents have perfect recall of immediate context but zero memory of historical design intent. | ||
| * `Comprehension Ratio` — evaluate the semantic weight of the delta. A 4,000-line mechanical rename scores lower on Likelihood than 40 lines changing core retry semantics. | ||
| * `Amplification Ratio` — compare the RPI plan artefact delta to the actual code delta. Flag high-risk mismatches where a small plan produces an enormous code delta, or a large plan produces a suspiciously small diff. | ||
|
|
||
| ## Cold starts and confidence | ||
|
|
||
| The model relies on git history. For repositories with shallow history, signal confidence degrades. | ||
|
|
||
| * If trailing 90-day history is insufficient to calculate Hotspot Overlap or Missed Co-change, explicitly state: *"Confidence: Low (shallow history). Relying on Size and Diffusion signals."* | ||
| * Widen confidence intervals and default to standard depth tiers unless Size or Diffusion signals are extreme. Never artificially inflate a risk rating due to missing data. | ||
|
|
||
| ## Gaming and Goodhart's Law | ||
|
|
||
| Agents aware of the scoring model may attempt to game it (e.g., pathologically splitting changes to lower Diffusion scores). | ||
|
|
||
| * The profile is advisory input to a human-confirmed scoping step, not a gate. | ||
| * Batch-splitting that genuinely reduces per-change blast radius is acceptable engineering behavior. | ||
| * The human-scoping protocol evaluates the *intent* of the change, serving as the ultimate safeguard against metric manipulation. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: the
ms.datefrontmatter field was not updated in any of the three modified files. Repo convention requires updating this field to the edit date whenever a file that carries it is changed. Applies todepth-tiers.md(2026-06-18),context-bootstrap.md(2026-06-26), andseverity-taxonomy.md(2026-06-18); all should be updated to2026-07-29.