Skip to content

badger, lmdb: match tag values exactly, and check every tag of a filter - #75

Open
sunmoonron wants to merge 1 commit into
fiatjaf:masterfrom
sunmoonron:tag-exact-match
Open

sunmoonron wants to merge 1 commit into
fiatjaf:masterfrom
sunmoonron:tag-exact-match

Conversation

@sunmoonron

@sunmoonron sunmoonron commented Sep 8, 2026 •

Copy link
Copy Markdown

This is the bug behind fiatjaf/khatru#52 (the "silent rejection of certain replaceable events"), plus a smaller lmdb gap found while writing the test.

badger. For plain string tag values, and for the d part of a-style values, the tag index key is the value itself with nothing after it, so a prefix iteration for d=a also walks the keys of a-b, abc and so on. The planner then dropped the indexed tag from the post-fetch extraFilter (CopyMapWithoutKey), trusting the index to be exact. So with a-b stored first and newer, publishing an addressable event with d=a was silently not stored: the replace query for d=a returned the a-b event, IsOlder said the new one was older, and shouldStore went false with no error. Plain tag queries had the same false positives. Fix: the indexed tag stays in extraFilter.Tags, so filterMatchesTags checks the fetched event for the exact value. No index format change, no migration.

lmdb. Its keys are already exact for these cases, so it did not have the d=a bug. But when a filter has authors or kinds and two tags the planner has no index preference for (the default branch of ChooseNarrowestTag), prepareQueries returned only one extra tag and the other was never enforced. It now returns the whole TagMap, and the query and both count paths check every tag against the fetched event (tagsMatch).

test. tagPrefixTest runs on every backend from test/db_test.go: d=a next to d=a-b through RelayWrapper.Publish, a tags pointing at those addresses, plain t tags, a two-condition filter, and a two-unknown-tags filter. Without the change it fails on badger at the d=a query and on lmdb at the two-unknown-tags query; with it, TestSliceStore, TestBadger and TestLMDB pass.

Cost: one tag scan per fetched candidate on tag queries, which were already decoding the event.

A length-prefixed or terminated badger index key would remove the false positives at the iterator level instead; that needs a migration, so it is left out of this PR, happy to follow up if you prefer that shape.

🤖 Generated with Claude Code

@sunmoonron sunmoonron changed the title badger, lmdb: match tag values exactly, not as prefixes of longer values badger, lmdb: match tag values exactly, and check every tag of a filter Sep 8, 2026
For plain string tag values, and for the d part of a-style values, the
badger tag index key is the value itself with nothing after it, so a
query for d=a also walks the keys of a-b, abc and so on, and badger
dropped the indexed tag from the post-fetch check, trusting the index.
The visible symptom is khatru#52: with a-b stored first and newer,
publishing an addressable event with d=a was silently not stored,
because the replace query for d=a matched a-b.

badger keeps the indexed tag in the extra filter, so the fetched event
is checked for the exact value.

lmdb's keys are exact for these cases, but when a filter has authors
or kinds and two tags the planner has no index preference for, only
one of the tags was enforced. It now checks every tag of the filter
against the fetched event, in queries and in both count paths.

A shared regression test covers d, a, t and unknown tags; without the
change it fails on badger at the d=a query and on lmdb at the two
unknown tags query.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant