Skip to content

refactor(worker): scheduler jobs split into vrp/surface/ops/market_data families (Wave 7a) - #540

Merged
moremeds merged 7 commits into
mainfrom
chore/scheduler-families
Oct 5, 2026
Merged

moremeds merged 7 commits into
mainfrom
chore/scheduler-families

Conversation

@moremeds

@moremeds moremeds commented Oct 5, 2026

Copy link
Copy Markdown
Owner

Summary

Wave 7a of the architecture remediation plan: the last 38 inline add_job sites (42 jobs) leave worker/scheduler.py for four new job families under worker/schedule/. All four use the existing family shape, register(sched, settings).

Family add_job sites Predicates moved with it
vrp.py (216 lines) 11: markout, research, candidates, paper open/mark, backtest, macro entry ×3 + grid, macro signal _should_schedule_vrp_macro_entry
surface.py (265) 8: vol + skew rollups, skew markout, skew swing greeks, option-surface capture, research capture + catch-up, IV canary _should_schedule_option_surface_capture, _should_schedule_skew_swing_greeks
ops.py (137) 5: freshness monitor, pipeline benchmark, record-health snapshot, mcp_event retention, data gap healer _should_schedule_pipeline_benchmark, _should_schedule_data_gap_healer, _should_schedule_mcp_event_retention
market_data.py (450) 14: SPY OHLC, technicals daily + live, corporate actions, greek exposure, cockpit, UW-alpha loop (5 jobs), sector RS, release calendar, volatility backfill tick, SPX density, Theta Harvester ×2, trade-insight outcome backfill _should_schedule_uw_alpha_capture, _should_schedule_sector_rs_daily

scheduler.py goes from 1115 to 191 lines. main() now holds only settings, the schema gate, the listener, the heartbeat and start.

How the move was done: a script copied each block's lines verbatim, with the same indentation and the same nesting of role guards. It also kept wrapper names and kept lazy imports inside the wrappers. Only unused imports were pruned (ruff F401 --fix). The commits are one per family (pure moves), plus one format-only commit (blank lines in scheduler.py) and one docs commit (the CLAUDE.md "Scheduled jobs" row and CHANGELOG [Unreleased], both bullets kept after rebasing onto #539).

Acceptance

  1. tests/unit/worker/test_scheduler_jobs_golden.py and its fixture are byte-identical to main (git diff --quiet origin/main on both), and every role passes.
  2. test_source_telemetry_wiring.py passes. uv run pytest tests/unit/: 3160 passed. uv run ruff check src/ tests/ scripts/: clean.
  3. Each family is its own commit. The commits carry no AI or Co-Authored-By trailer.

Test retargets (import and monkeypatch targets only)

  • test_source_telemetry_wiring.py: the release-calendar and technical-live cases now patch _external_api_recorder, _repo, _uw_client and datetime on schedule.market_data. Before this change those patches would have missed silently.
  • test_scheduler.py: three predicates are now imported from their new modules (ops, surface, roles).
  • test_vrp_macro_entry_schedule.py: the predicate is imported from schedule.vrp.
  • tests/integration/worker/test_vrp_research_jobs.py: _repo is patched on schedule.vrp. Without this, the patch would miss and the job would open its own connection.

Notes

  • Re-exports in scheduler.py: two compatibility re-exports stay, for importers outside this wave's files. _should_schedule_data_gap_healer is imported by tests/integration/worker/test_data_gap_healer_scheduler.py, a data_gap_* file that belongs to another lane. _should_schedule_option_surface_capture is imported by tests/unit/test_scheduler_option_surface_gate.py. _repo also stays, because the heartbeat, 3 research scripts and test_macro_state_jobs use it.
  • Logger names: the moved wrappers now log under their module name, the same as the existing families. Before, they logged under uw_scan.worker.
  • Not updated: src/uw_scan/worker/CLAUDE.md still lists only the old six families. That file was outside this wave's files.
  • Integration tests not run locally: the database option_wizard_test_sched7a does not exist locally, so CI covers the integration tests.

🤖 Generated with Claude Code

chenxi added 7 commits October 5, 2026 11:55
Pure move of the 11 VRP add_job blocks, their main()-local wrappers and
_should_schedule_vrp_macro_entry out of scheduler.main(). Each block keeps its
guard (massive-0 primary, the entry-capture pin, the global daily owner).
Tests that imported or patched the moved names now point at schedule.vrp.
… (Wave 7a)

Pure move of the 8 surface add_job blocks (vol + skew rollups, skew markout,
skew swing greeks, option-surface capture, research capture + catch-up, IV
canary), their wrappers and the option-surface / skew-swing predicates. Each
block keeps its guard. _should_schedule_option_surface_capture stays importable
from scheduler for tests/unit/test_scheduler_option_surface_gate.py.
Pure move of the 5 ops add_job blocks (data freshness monitor, pipeline
benchmark, record-health snapshot, mcp_event retention, data gap healer),
their wrappers and the benchmark / healer / mcp-retention predicates. Each
block keeps its guard. _should_schedule_data_gap_healer stays importable from
scheduler for tests/integration/worker/test_data_gap_healer_scheduler.py.
…y family (Wave 7a)

Pure move of the last 14 add_job blocks (SPY OHLC, technicals daily + live,
corporate actions, greek exposure, cockpit snapshot, UW-alpha captures, sector
RS, release calendar, volatility backfill tick, SPX density, Theta Harvester
scan + markout, trade-insight outcome backfill), their wrappers and the
uw-alpha / sector-RS predicates. Each block keeps its guard. main() now holds
only settings, the schema gate, the listener, the heartbeat and start. The
telemetry test patches the two moved wrappers in their new module.
Blank lines only: collapses the gaps the moved blocks left in main().
…rom their family

Both tests only call the predicates, so importing from schedule.surface /
schedule.ops is the same check. worker/CLAUDE.md lists the four new families.
@moremeds
moremeds merged commit d8c331c into main Oct 5, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant