Repository navigation
Manage a cursor for worker automatically - #11
Merged
Merged
Conversation
The work query may now return a `cursor` alongside its batch, and the component stores it and hands it back as `args.cursor` on the next call. The worker mutation can return one too, which wins over the query's — it's the one that knows how far the work actually got. The cursor commits with the batch: the worker mutation shares the loop's transaction, so a batch that throws leaves the cursor where it was. It rides along on the `workerState` patch the loop already does every iteration, so persisting it costs no extra write. Deliberately no cursor on the `idle` branch: idle means the loop may never run again until something pings it. Skipping past a scanned stretch is `work` with an empty batch instead. Additive: `vBatchQueryArgs` gains an optional `cursor`, so upgrading picks it up. The component only sends a cursor once a batch has returned one, so a worker that doesn't use them never sees the new arg. Adds `lib.cursor` / `lib.setCursor` (and `getCursor` / `setCursor` client helpers) to read it from your own queries and to reset it.
`vBatchQueryArgs` / `vBatchResult` only cover the default commit-timestamp
cursor. `batchValidators({ batch, cursor })` builds all four validators from
one declaration, so a worker keyed on something else — a paginator cursor,
say — stays type-safe, and the batch shape is pinned in one place instead of
being repeated in the query's `returns` and the mutation's `args`.
`ping` takes the cursor type from the work query's `cursor` arg and treats
the two return positions as non-inference sites, so a mismatch points at one
place rather than producing an unresolvable-inference error.
Drops the `cursors` table and cursor.ts helpers. Each work query now reads `args.cursor` and returns how far it got; the worker mutations lose the cursor from their args and stop writing it, so they only do their own work and cleanup. Queries that count pending work read the cursor back with `getCursor`, and the e2e harness resets it with `setCursor`. No `as bigint` casts left: the cursor is typed `v.commitTs()`, which is exactly the type of the row fields being handed back. `setCursor` now throws when the worker doesn't exist rather than silently dropping the write — the cursor lives on the worker doc, so there's nowhere to put it until `ping` has created one.
Reading the cursor belongs in the work query's args, not in a helper. For the rarer cases — a query counting pending work, or resetting the cursor — call `lib.cursor` / `lib.setCursor` on the component directly, the same way `status`/`start`/`stop` are already used.
The main usage example now carries a cursor, so the README no longer opens with the version it tells you not to write. The step-by-step "track the cursor yourself" section collapses into a description of the contract, plus the custom-cursor-type and mutation-override cases.
`vBatchQueryArgs` / `vBatchResult` / `vWorkerResult` are each fixed to the default commit-timestamp cursor, and are declared separately enough that the query and mutation can drift apart. Deprecate all three in favour of the factory, which derives all four from one declaration. `vBatchResult` loses the cursor-type parameter it briefly had — overriding the cursor in the return validator but not the args validator was the inconsistency that prompted this. They're now built from shared internal helpers, so the deprecated exports behave exactly as before.
Each example declares its batch shape once instead of repeating it across the query's returns and the mutation's args, and none of them use the deprecated validators any more. Also clears out the last references to the deleted cursor.ts.
commit: |
ianmacartney
changed the title
ianmacartney
marked this pull request as ready for review
August 7, 2026 03:58
reeceyang
approved these changes
Aug 7, 2026
reeceyang
left a comment
reeceyang
left a comment
Contributor
There was a problem hiding this comment.
looks good, this will be super useful!
I just had a few small comments about docs
Comment on lines
+150
to
+151
| * alongside the other three validators from one declaration — so the query and | ||
| * the mutation can't drift apart, and the cursor can be any type. This one is |
Contributor
There was a problem hiding this comment.
maybe so the batch shape matches between the query and the mutation? ditto for the other occurrences of this text
ianmacartney
commented
Aug 8, 2026
Comment on lines
72
to
76
| * | ||
| * Typing it as `v.commitTs()` rather than `v.int64()` means a row's timestamp | ||
| * field can be handed straight back as the cursor without a cast, since that's | ||
| * exactly the type such a field has. | ||
| */ |
Member
Author
There was a problem hiding this comment.
Suggested change
| * | |
| * Typing it as `v.commitTs()` rather than `v.int64()` means a row's timestamp | |
| * field can be handed straight back as the cursor without a cast, since that's | |
| * exactly the type such a field has. | |
| */ | |
| */ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The work query may now return a
cursoralongside its batch, and thecomponent stores it and hands it back as
args.cursoron the next call.The worker mutation can return one too, which wins over the query's — it's
the one that knows how far the work actually got.
The cursor commits with the batch: the worker mutation shares the loop's
transaction, so a batch that throws leaves the cursor where it was. It
rides along on the
workerStatepatch the loop already does everyiteration, so persisting it costs no extra write.
Deliberately no cursor on the
idlebranch: idle means the loop may neverrun again until something pings it. Skipping past a scanned stretch is
workwith an empty batch instead.vBatchQueryArgs/vBatchResultonly cover the default commit-timestampcursor.
defineBatchWorkerValidators({ batch, cursor })builds all four validators fromone declaration, so a worker keyed on something else — a paginator cursor,
say — stays type-safe, and the batch shape is pinned in one place instead of
being repeated in the query's
returnsand the mutation'sargs.The main usage example now carries a cursor, so the README no longer opens
with the version it tells you not to write. The step-by-step "track the
cursor yourself" section collapses into a description of the contract, plus
the custom-cursor-type and mutation-override cases.
vBatchQueryArgs/vBatchResult/vWorkerResultare each fixed to thedefault commit-timestamp cursor, and are declared separately enough that the
query and mutation can drift apart. Deprecate all three in favor of the
factory, which derives all four from one declaration.