Skip to content

Preserve mailroom metrics privacy - #1754

Draft
DanGould wants to merge 3 commits into
payjoin:masterfrom
DanGould:mailroom-metrics-privacy
Draft

Preserve mailroom metrics privacy#1754
DanGould wants to merge 3 commits into
payjoin:masterfrom
DanGould:mailroom-metrics-privacy

Conversation

@DanGould

@DanGould DanGould commented Jul 21, 2026

Copy link
Copy Markdown
Member

Make the mailroom's exported metrics coarse enough that a public dashboard, or anyone who sees it, cannot track an individual operator's or user's traffic. This is the prerequisite for onboarding third-party mailroom operators. We will not put anyone on a public mailroom list until the metrics they push to us are safe to aggregate and publish.

Today every node pushes precise per-request counters and live gauges to Grafana Cloud, labelled with the operator's domain. The README also claims metrics stay local, which is not true once a [telemetry] section is configured.

What the branch does

The privacy model is in the README diff. Two things it does not say:

  • Exactly five weekly gauges leave the process: requests, requests started, DB entries,
    tunnel sheds, and unique short IDs (a HyperLogLog estimate). Everything else stays local.
  • The suppression threshold (10) and quantization bin (5) do nothing at payjo.in scale,
    about 3M requests a week. They exist for a new or low-traffic operator, where a weekly
    count of 1 or 2 could be tied to a known event, and precise small integers let someone
    who sent known traffic infer the remainder by subtraction. The protection that works at
    every scale is the settled-window rule plus the absence of attributes, not these knobs.

Done when

  1. The five exported gauges, the settled-window rule, suppression, quantization, and the
    attribute allowlist are each pinned by a test. (Present; reviewer confirms coverage.)
  2. Weekly buckets survive a restart. ExportWindows and the HLL sketches are in memory, so
    a redeploy mid-week zeroes the count, suppression drops it, and that week silently never
    reports. Counters are eight integers; the sketches need hyperloglogplus's serde
    feature plus a deterministic hasher, because RandomState is seeded per process and a
    reloaded sketch would hash new IDs under a different seed. Precedent for the on-disk
    file: storage_dir and db/files.rs's xor.dat. The PR body must say that a
    short-ID-derived sketch now lands on the operator's disk where it was memory-only. It is
    not reversible to IDs and the mailroom already stores mailbox payloads on disk, but a
    privacy PR says so rather than letting a reviewer find it. (Not done. If the sketch half
    is too much for this PR, persist the counters here and file the sketch as a follow-up;
    say which in the body.)
  3. The README describes what actually happens. With export configured, the precise
    instruments are built on a provider with no reader, so they are neither exported nor
    observable locally. Either say that plainly or add a local sink. "Metrics stay local"
    as a blanket claim goes. (Partly done; the current text is accurate on export and silent
    on local observability.)
  4. Two processes under one reporter.id are handled. Each holds half the week and writes
    the same series, so the export is one process's share, not the sum. Resolve as a
    documented one-process-per-reporter constraint in README and config.example.toml, or
    by aggregating at a collector. Constraint is acceptable for this PR. (Not done.)
  5. nix fmt -- --ci, nix develop -c ./contrib/lint.sh, and the mailroom clippy pass with
    both --all-features and --no-default-features are green, plus the telemetry tests.
    test_local.sh has an unrelated pre-existing failure in test_bad_ohttp_keys; note it,
    don't chase it here.
  6. Commit series stays three commits (document, wire, coarsen) or grows by one for
    persistence. No fixups in history.

Out of scope

  • Separating test infrastructure from production metrics for a tighter sessions-per-day
    bound. Separate item, no branch yet.
  • The service.instance.id branch (mailroom-otel-instance-id). This PR pins the
    resource to two attributes, which is the opposite design; that branch is parked, not
    rebased, and the multi-writer problem it addressed is item 4 above.

Disclosure: co-authored by Claude Code.

Pull Request Checklist

Please confirm the following before requesting review:

DanGould added 3 commits July 20, 2026 15:52
Precise counters remain local. Exporting only suppressed and quantized
counts from completed fixed UTC weeks prevents live probing and daily
differencing.
Use a Foundation-issued opaque reporter ID, not a domain.
Metric points carry no attributes. Retry each frozen weekly aggregate
daily to improve delivery resilience.
Explain the export boundary, configuration, and separation of Grafana
access control from future collector aggregation so operators can assess
the privacy tradeoff before opting in.
@coveralls

Copy link
Copy Markdown
Collaborator

Coverage Report for CI Build 29818970076

Coverage increased (+0.07%) to 86.371%

Details

  • Coverage increased (+0.07%) from the base build.
  • Patch coverage: 35 uncovered changes across 4 files (260 of 295 lines covered, 88.14%).
  • No coverage regressions found.

Uncovered Changes

File Changed Covered %
payjoin-mailroom/src/lib.rs 14 0 0.0%
payjoin-mailroom/src/main.rs 11 0 0.0%
payjoin-mailroom/src/config.rs 9 0 0.0%
payjoin-mailroom/src/metrics.rs 242 241 99.59%
Total (5 files) 295 260 88.14%

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 16281
Covered Lines: 14062
Line Coverage: 86.37%
Coverage Strength: 339.34 hits per line

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants