fix: harden rewrite replacement and leak metrics

[binaryaaron]

Summary

• apply replacement-map entries to validated source spans before rewrite generation and carry that baseline through repair
• emit schema-v2 metrics that distinguish map entries, targeted/applied/skipped spans, unique leaked values, and associated source occurrences
• separate post-validation detection artifacts from materialized final entities and keep legacy schema-v1 ingestion isolated
• consolidate local and generated-map DataFrame orchestration around one span-application helper, without iterrows() or per-row Series construction

Why

The schema-v1 leak metric used raw substring matching for longer values and counted every same-valued source entity when one match appeared in output. This produced false positives such as lawyer in lawyers. Adjacent replacement metrics also described map membership rather than actual span execution, and detection sidecars exported post-validation detections under final_* names.

The rewrite path additionally relied on the model to apply deterministic replacement-map entries. This change uses the existing offset-based replacement boundary, fails closed when required mappings cannot be applied, and prevents the repair loop from returning to the original sensitive baseline. It supersedes the global-regex runtime approach proposed in #208.

Both local replacement and LLM-generated replacement maps now use the same DataFrame orchestration helper. The helper iterates aligned columns directly, preserves row order and custom column names, and records replacement text and application telemetry together.

Compatibility

• new measurements emit schema version 2 and explicit metric names
• strict ingress continues to accept schema-v1 records
• mixed schema snapshots and cross-version metric fields are rejected
• benchmark groups include measurement schema version, so v1 and v2 series are not aggregated together
• legacy artifact sidecars are interpreted as detected metrics; final metrics remain unavailable unless actual final entities are present

Validation

• make check
• make test (1,219 passed, one existing W&B deprecation warning)
• uv run pytest tests/engine/test_replace_strategies.py tests/engine/test_replace_runner.py -q (21 passed)
• git diff --check
• Astnav syntax query confirms zero iterrows() calls in strategies.py

[greptile-apps]

Greptile Summary

The PR hardens rewrite processing by applying deterministic replacements to validated spans before generation and retaining that sanitized baseline through repair. It also introduces schema-v2 replacement and leak metrics and updates benchmark and W&B tooling to keep schema versions isolated.

• Adds label-aware, offset-based replacement application with execution telemetry.
• Prevents unavailable required replacements from being accepted or repaired from sensitive source text.
• Distinguishes replacement-map membership, applied spans, unique leaked values, and leaked source occurrences.
• Updates measurement ingestion, analysis, completion, reporting, and documentation for schema v2.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
src/anonymizer/engine/replace/strategies.py	Introduces the shared offset-based replacement primitive, ambiguity handling, and per-row replacement-application telemetry.
src/anonymizer/engine/rewrite/rewrite_generation.py	Applies required replacement-map entries before rewrite generation and fails closed when required spans cannot be replaced.
src/anonymizer/engine/rewrite/rewrite_workflow.py	Preserves replacement-unavailable state through evaluation and repair and routes affected rows to human review.
src/anonymizer/engine/rewrite/repair.py	Uses the pre-replaced baseline rather than the original sensitive source when constructing repair prompts.
src/anonymizer/measurement/metrics/replacements.py	Separates replacement-map entry metrics from targeted, applied, and skipped span metrics.
src/anonymizer/measurement/metrics/rewrite.py	Replaces ambiguous leak counts with explicit unique-value and source-occurrence cardinalities using boundary-aware matching.
tools/measurement/measurement_tools/wandb_ingress.py	Enforces schema-specific metric fields and isolates legacy schema-v1 ingestion from schema-v2 records.
tools/measurement/analyze_benchmark_output.py	Updates benchmark analysis for schema-version isolation and separates detected artifacts from materialized final entities.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Validated source entities] --> B[Replacement map and rewrite dispositions]
  B --> C[Apply label-aware replacements to source spans]
  C --> D{All required spans applied?}
  D -- No --> E[Mark rewrite unavailable]
  E --> F[Require human review]
  D -- Yes --> G[Build sanitized rewrite baseline and tagged text]
  G --> H[Generate rewrite]
  H --> I[Evaluate rewrite]
  I --> J{Repair needed?}
  J -- Yes --> K[Repair from sanitized baseline]
  K --> I
  J -- No --> L[Materialize final rewrite]
  C --> M[Emit replacement application metrics]
  L --> N[Emit schema-v2 leak metrics]

Loading

_{Reviews (3): Last reviewed commit: "fix: address schema and rewrite review f..." | Re-trigger Greptile}

[lipikaramaswamy]

General note: this PR correctly supersedes #208 by using validated entity spans instead of global regex replacement, preventing embedded-substring corruption such as Ann inside Anna. It verifies that every required span was replaced and fails closed when a map is absent, partial, ambiguous, or cannot be applied. It also preserves the sanitized baseline through repair and reports only safe labels and counts in diagnostics. cc @asteier2026