[moremeds]

What was wrong

capture_dark_lit_for asked UW for one page (limit=500) per ticker, day and source. UW serves prints newest first, so busy names kept only the after-hours tail. NVDA 2026-09-15: about 13,500 dark prints at UW, 500 stored. September cut rate (argon-8b, prod read-only): dark 45.6% of ticker-days, lit 20.9%.

What changes

• sources/uw.py: fetch_darkpool_prints / fetch_lit_flow take an optional older_than (new _print_page_fetcher; the pinned surface fixture is regenerated on purpose, +2 request cases that pin the param).
• uw_alpha_capture._fetch_print_day pages each source until:
  • a page shorter than limit, or
  • a page reaching a print whose ET date is before the market date, or
  • no progress, or
  • the 60-page cap per source (WARN + page_cap_hits). Worst case is 2 × 60 = 120 UW calls per ticker-day.
• Prints are kept only when executed_at's ET date equals market_date. Every stored row already satisfies this (prod: 2,031,461 rows with market_date ≥ 2026-09-01, 100% same ET date).

Two UW behaviours measured live (2026-10-05, NVDA 2026-09-15, 5 calls)

Probe	Result
older_than strictness	strictly older: page 2 had 0 rows at page 1's last second
rows sharing one second	up to 17 on page 1
page 1 boundary second 19:58:11Z	page 1 held 3 prints; re-asking with older_than=19:58:12Z returned 12
lit-flow honours older_than	yes, page 2 all older

So a strict cursor drops prints at every page boundary. The cursor is last-second + 1 s; the overlap is re-read and the PK (source, tracking_id, executed_at, price, size, volume) + ON CONFLICT DO NOTHING drops repeats. If a full page sits inside the overlap second (no progress), it steps strictly once.

Callers

• Nightly dark_lit_capture: summary now carries pages and page_cap_hits. Its UW cost rises from 2 calls per ticker to the real page count; it stays unbudget-gated as before (durable data).
• Data-gap healer (uw_alpha_dark_lit): the healer charges a fixed est_per_item for every dataset, not real calls. Raised 2 → 10 (typical day; sampled dark CRM 3, PLTR 7, DELL 13 pages; lit 1–2). Not an upper bound. The healer dispatches only missing ticker-days, so already-cut days are not re-healed here; that is the history backfill (follow-up PR).
• scripts/backfill/uw_alpha_catchup.py: estimate 10 for the pre-check and dry run; a live run charges the pages actually fetched, including pages fetched before a failure.

Tests

• New tests/unit/worker/test_dark_lit_paging.py: a stub with UW's measured semantics (newest first, strict older_than, 3 prints per second, limit=7 so every boundary splits a second, prior-session tail).
  • Covers: multi-page stitching, prior-day filter, one call on a short page, overlap cursor value, and the page cap (count + stop).
  • Before-proof: all 5 fail against main's capture module. A strict-cursor mutant fails the 2 boundary tests.
• Unit 3,165 passed (3,160 + 5). Related unit + integration (test_uw_alpha_capture, test_uw_historical_alpha_repository, catch-up, surface): 433 passed. ruff check src/ tests/ scripts/ clean.

🤖 Generated with Claude Code