feat: biometric-engine benchmark harness + accuracy protocol

[munisp]

What

WP7: a benchmark harness + published accuracy protocol for services/biometric-engine, as a partial answer to the Onfido/Entrust maturity gap.

• services/biometric-engine/benchmarks/run_benchmarks.py — deterministic, fully offline decision-layer benchmark:
  • Synthetic cohort generator, seeded (numpy.default_rng(seed), default 20260114): N=200 identities (configurable) × K=10 samples of L2-normalised 512-d embeddings. Inter-class centres uniform on the unit sphere (impostor cosine ~ N(0, 1/512), std ≈ 0.044); intra-class additive Gaussian noise with σ_intra = sqrt((1/target − 1)/dim) derived from --target-genuine-cosine (default 0.65). Genuine pairs = all within-identity pairs (9,000); impostor pairs = equal-count cross-identity sample.
  • Metrics against the engine's actual thresholds: LIVENESS_THRESHOLD / MATCH_THRESHOLD / ANTISPOOFING_THRESHOLD are parsed from main.py (or the same env vars the engine reads) — no hardcoded duplicates; provenance is recorded in the report. FAR/FRR at the operating point, full ROC sweep, EER (interpolated crossing), trapezoidal AUC.
  • Latency distribution (p50/p95/p99) of the 512-d cosine-similarity path via perf_counter (18,000 timed calls).
  • Anti-spoofing decision-layer summary: seeded Beta(8,2)/Beta(2,8) score models pushed through the ANTISPOOFING_THRESHOLD rule (classifier weights deliberately NOT loaded offline — labelled as such).
  • Outputs benchmarks/report.json (seed, params, thresholds+provenance, metrics, latency, environment; label SYNTHETIC-COHORT) + markdown summary on stdout.
  • Runs with no network and no dataset downloads. MediaPipe/InsightFace/ONNX/MiniFASNetV2 weights are not loaded — the harness benchmarks the pure similarity/threshold decision layer, which is where the thresholds live. If main.py is importable it uses the engine's own _cosine_similarity; otherwise byte-identical local math is used and the report records which ran.
• services/biometric-engine/test_benchmarks.py — 18 tests: generator determinism (same seed → bit-identical cohort), unit-norm + intra<inter variance invariants, hand-computed FAR/FRR/EER/AUC on toy confusion sets, threshold sourcing (env override wins; parsed default equals the literal in main.py), report schema validation, CLI smoke.
• docs/biometric-benchmarks.md — protocol, parameters, thresholds, real-run results table, LIMITATIONS, and the exact CI job YAML (see deviation below).
• services/biometric-engine/benchmarks/report.json — committed snapshot of a real run.

Why

Vendor-maturity gap (Onfido/Entrust): we had no reproducible, honest measurement of the biometric decision layer. This provides deterministic validation of the threshold/similarity logic and a published protocol — without inflating accuracy claims.

How tested (real output)

$ cd services/biometric-engine && python -m unittest test_benchmarks -v
...
Ran 18 tests in 0.263s
OK

(Existing suite still green: python -m unittest test_main → Ran 21 tests ... OK.)

Real benchmark run (python benchmarks/run_benchmarks.py, seed 20260114, 200×10, 9,000 genuine / 9,000 impostor pairs, thresholds read from engine: MATCH=0.40, LIVENESS=0.72, ANTISPOOF=0.60):

Metric	Value
FAR @ MATCH_THRESHOLD=0.40	0.000000
FRR @ MATCH_THRESHOLD=0.40	0.000000
EER	0.000000 (crossing threshold 0.5445)
ROC AUC	1.000000
Genuine cosine (mean ± std)	0.6493 ± 0.0215
Impostor cosine (mean ± std)	−0.0008 ± 0.0438
Similarity latency p50 / p95 / p99	8.7 / 13.7 / 20.4 µs
Anti-spoof genuine pass rate @ 0.60	0.9314 (synthetic score model)
Anti-spoof spoof pass rate @ 0.60	0.0030 (synthetic score model)

Zero errors are expected — the synthetic cohort is well separated by construction. These numbers validate the decision layer (threshold logic, ROC/EER math, env config path, similarity speed); they say nothing about real-world accuracy and are labelled SYNTHETIC-COHORT everywhere.

Limitations

• Synthetic-cohort results validate the decision layer only — production FAR/FRR depend on ArcFace embedding quality on real imagery, not measured here.
• iBeta PAD Level 1/2 certification is NOT claimed — it requires an accredited lab + real attack-instrument datasets (ISO/IEC 30107-3). Roadmap item, stated plainly in the docs.
• Anti-spoofing classifier itself is not executed offline (weights unavailable); only its threshold decision rule is exercised on synthetic scores.

Closes-gap

Partial close of the Onfido/Entrust maturity gap: reproducible decision-layer validation + published protocol now exist. Remaining gap (real-dataset FAR/FRR, demographic breakdown, iBeta PAD L1/L2) is external/accredited-lab roadmap, documented in docs/biometric-benchmarks.md.

Deviation

The biometric-benchmarks CI job (non-blocking, continue-on-error: true, uploads report.json artifact) was written and YAML-validated but could not be pushed: the automation token lacks the workflow scope required to modify .github/workflows/ (HTTP 403 insufficient scopes). The exact, ready-to-apply job block and run command are documented in docs/biometric-benchmarks.md → "CI wiring". Everything else is in this branch.