Preserve mailroom metrics privacy - #1754
Draft
DanGould wants to merge 3 commits into
Draft
Conversation
Precise counters remain local. Exporting only suppressed and quantized counts from completed fixed UTC weeks prevents live probing and daily differencing.
Use a Foundation-issued opaque reporter ID, not a domain. Metric points carry no attributes. Retry each frozen weekly aggregate daily to improve delivery resilience.
Explain the export boundary, configuration, and separation of Grafana access control from future collector aggregation so operators can assess the privacy tradeoff before opting in.
Collaborator
Coverage Report for CI Build 29818970076Coverage increased (+0.07%) to 86.371%Details
Uncovered Changes
Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make the mailroom's exported metrics coarse enough that a public dashboard, or anyone who sees it, cannot track an individual operator's or user's traffic. This is the prerequisite for onboarding third-party mailroom operators. We will not put anyone on a public mailroom list until the metrics they push to us are safe to aggregate and publish.
Today every node pushes precise per-request counters and live gauges to Grafana Cloud, labelled with the operator's domain. The README also claims metrics stay local, which is not true once a
[telemetry]section is configured.What the branch does
The privacy model is in the README diff. Two things it does not say:
tunnel sheds, and unique short IDs (a HyperLogLog estimate). Everything else stays local.
about 3M requests a week. They exist for a new or low-traffic operator, where a weekly
count of 1 or 2 could be tied to a known event, and precise small integers let someone
who sent known traffic infer the remainder by subtraction. The protection that works at
every scale is the settled-window rule plus the absence of attributes, not these knobs.
Done when
attribute allowlist are each pinned by a test. (Present; reviewer confirms coverage.)
ExportWindowsand the HLL sketches are in memory, soa redeploy mid-week zeroes the count, suppression drops it, and that week silently never
reports. Counters are eight integers; the sketches need
hyperloglogplus'sserdefeature plus a deterministic hasher, because
RandomStateis seeded per process and areloaded sketch would hash new IDs under a different seed. Precedent for the on-disk
file:
storage_diranddb/files.rs'sxor.dat. The PR body must say that ashort-ID-derived sketch now lands on the operator's disk where it was memory-only. It is
not reversible to IDs and the mailroom already stores mailbox payloads on disk, but a
privacy PR says so rather than letting a reviewer find it. (Not done. If the sketch half
is too much for this PR, persist the counters here and file the sketch as a follow-up;
say which in the body.)
instruments are built on a provider with no reader, so they are neither exported nor
observable locally. Either say that plainly or add a local sink. "Metrics stay local"
as a blanket claim goes. (Partly done; the current text is accurate on export and silent
on local observability.)
reporter.idare handled. Each holds half the week and writesthe same series, so the export is one process's share, not the sum. Resolve as a
documented one-process-per-reporter constraint in README and
config.example.toml, orby aggregating at a collector. Constraint is acceptable for this PR. (Not done.)
nix fmt -- --ci,nix develop -c ./contrib/lint.sh, and the mailroom clippy pass withboth
--all-featuresand--no-default-featuresare green, plus the telemetry tests.test_local.shhas an unrelated pre-existing failure intest_bad_ohttp_keys; note it,don't chase it here.
persistence. No fixups in history.
Out of scope
bound. Separate item, no branch yet.
service.instance.idbranch (mailroom-otel-instance-id). This PR pins theresource to two attributes, which is the opposite design; that branch is parked, not
rebased, and the multi-writer problem it addressed is item 4 above.
Disclosure: co-authored by Claude Code.
Pull Request Checklist
Please confirm the following before requesting review:
AI
in the body of this PR.