Skip to content

Every paid crawl now reaches the people whose niche was crawled - #14

Merged
ralyodio merged 1 commit into
mainfrom
worktree-x402-attribution
Sep 6, 2026
Merged

Every paid crawl now reaches the people whose niche was crawled#14
ralyodio merged 1 commit into
mainfrom
worktree-x402-attribution

Conversation

@ralyodio

@ralyodio ralyodio commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Phase 4: the gateway's onSale hook now books into the revenue ledger, so a paid crawl reaches the people operating the niches it read.

What was actually missing

Worth stating plainly, because it is easy to assume this already worked. The gateway has been selling crawl passes since launch, and onSale already did two things: recorded the sale in crawl_sales, and split it to partners. It never touched the revenue ledger — so a Knowledge Influencer operating a niche earned nothing when a crawler paid to read it. That is the gap this closes.

The hard part

A pass buys the whole index for a day, not one niche. There is no single niche to attribute it to.

The only measurable answer to "whose data did this pay for" is how much of the index each niche holds — the same rule the partner split already uses. So one sale becomes several revenue events:

  • one per niche somebody actually operates, sized pro-rata against the whole index
  • one unattributed event for the part nobody operates

Against the whole index deliberately: a niche holding one percent of the rows must not collect the whole dollar just because it is the only one with an operator.

index: 100 items (games 90, research 10), only games operated
$1 sale ->  x402:<ref>:games   90c   attributed, then split by the ladder
            x402:<ref>         10c   platform, unattributed
                              ----
                              100c

Every cent lands somewhere

The largest-remainder split that already kept basis points summing to 8000 is now a general apportion, used by both. The unoperated remainder is a claimant in that division rather than a leftover, so the parts sum to exactly the sale rather than to 99 cents. Each event records the item counts its share was computed from, so the number can be argued with rather than believed.

Idempotent per event (x402:<ref>:<niche> and x402:<ref>) — a settlement delivered twice books once even though it lands as several rows. Attribution is never allowed to fail the sale; the money has already moved.

One structural change

splitSaleAcrossNiches lives in packages/knowledge with the rest of the pure domain, not beside the database call. That is also what lets it be tested without a DATABASE_URL — the first version imported the db module and broke the suite's "needs no server" property.

Verification

  • bun test177 pass, 0 fail (12 new). Needs no server.
  • bun run lint — clean; the 4 CSS warnings and biome.json deprecation are pre-existing on main.
  • Production image builds.
  • Driven by calling the real onSale hook, not a mock: a $1 sale over an index of 100 items with games (90) operated and research (10) not booked 90c to games and 10c unattributed; the operator was owed 18c at the contributor rate; redelivering the identical sale left the ledger unchanged; and once research had an operator too, the next sale split 90/10 with no remainder. crawl_sales still records exactly as before.

Worth knowing before this matters

There have been zero crawl sales on nichedb.dev so far — the paywall is live and returns 402, but nothing has bought a pass yet, and there are no partner accounts or niche operators either. This wires the pipe before anything flows through it, which is the right order: retroactively attributing sales that were never recorded against a niche is much harder than attributing them as they arrive.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W9NGGLDvNayi6uWheSXDGF

Phase 4. The gateway has been selling crawl passes since the site launched and
the money went two places: crawl_sales, and the partner split. It never reached
the revenue ledger, so a Knowledge Influencer operating a niche earned nothing
from a crawler paying to read it.

The hard part is that a pass buys the whole index for a day, not one niche, so
there is no single niche to hand it to. The only measurable answer to whose data
it paid for is how much of the index each niche holds, which is the rule the
partner split already uses. One sale therefore becomes several revenue events:
one per operated niche, sized pro-rata against the WHOLE index, plus an
unattributed one for the part nobody operates.

Against the whole index deliberately. A niche holding one percent of the rows
must not collect the whole dollar because it happens to be the only one with an
operator.

Every cent lands somewhere. The largest-remainder split that already kept
basis points summing to 8000 is now a general `apportion`, used by both, so the
parts sum to exactly the sale rather than to 99 cents. The unoperated remainder
is a claimant in that division rather than a leftover. Each event records the
item counts its share was computed from, so the number can be argued with.

Idempotent per event (`x402:<ref>:<niche>` and `x402:<ref>`), so a settlement
delivered twice books once even though it lands as several rows. Never allowed
to fail the sale: the money has already moved.

The splitting arithmetic lives in packages/knowledge with the rest of the pure
domain rather than beside the database call, which is also what lets it be
tested without a DATABASE_URL.

Verified by calling the real onSale hook: a $1 sale over an index of 100 items
with games (90) operated and research (10) not booked 90c to games and 10c
unattributed, the operator was owed 18c at the contributor rate, redelivery
changed nothing, and once research had an operator too the next sale split
90/10 with no remainder.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W9NGGLDvNayi6uWheSXDGF
@ralyodio
ralyodio merged commit 12bd572 into main Sep 6, 2026
3 checks passed
@ralyodio
ralyodio deleted the worktree-x402-attribution branch September 6, 2026 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant