Repository navigation
fix: compute pcr_oi from open interest, not UW's volume ratio - #570
Merged
Merged
Conversation
added 4 commits
November 8, 2026 10:24
The scan stored UW's screener put_call_ratio as MarketAggregates.pcr_oi, but UW's spec defines that field as put volume / call volume. pcr_oi has been a copy of pcr_vol in every pcr_history row since 2026-01-02, and pcr_delta_30d, the watchlist "PCR (OI)" tile and the Trade Insights AI prompt all read the volume ratio under an OI label. - pipeline.aggregates_from_screener derives both ratios from the screener's own OI and volume counts (zero puts is 0, zero calls is no ratio). - Repository.repair_pcr_history_oi + scripts/backfill/pcr_history_oi_repair.py rewrite pre-fix rows from options_volume_daily (latest session within --max-lag-days of snapshot_date, else NULL). Dry run by default, idempotent, bounded by --before.
- Cutoff guidance: pass the day after the deploy as --before; the deploy day can still hold old-code rows, and the repair recomputes the same value for rows the fixed scan wrote. - Reject a negative --max-lag-days, which would NULL every in-scope row. - Docstring: scheduled scans stamp the ET session date; only ad hoc evening or weekend rescans land on a non-session date. - Tests: zero call OI sets NULL, a second ticker's session does not leak through the correlated subquery, negative lag is rejected.
# Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
MarketAggregates.pcr_oiwas set from UW screenerput_call_ratio. UW's spec (docs/uw-samples/unusual_whales_api_spec.yaml,/api/screener/stocks) defines that field as put volume / call volume. Sopcr_oihas been a copy ofpcr_vol:pcr_history: 38,376 rows (2026-01-02 → 2026-11-06); 38,342 havepcr_oi == pcr_vol, 0 differ, 34 have NULLpcr_vol.pcr_delta_30d,market_overview.positioning, the Trade Insights AI prompt input.Fix
pipeline.aggregates_from_screener()computespcr_oi = put_oi / call_oiandpcr_vol = put_vol / call_volfrom the screener's own counts. Zero puts → 0; missing/zero calls → NULL.run_single_stockuses it.Repository.repair_pcr_history_oi()(in_MarketDataMixin) +scripts/backfill/pcr_history_oi_repair.pyrewrite pre-fix rows fromoptions_volume_daily. The match is the latest session<= snapshot_datewithin--max-lag-days(default 4), because the UTC-stampedsnapshot_datecan be a day ahead of the ET session (weekend rows). No match → NULL, never the old volume ratio. Dry run by default; idempotent; bounded by--before.Post-deploy (manual, needs operator confirmation)
Pass the day after the deploy as
--before, and run only after the deploy day's 18:15 ET flow refresh has writtenoptions_volume_dailyfor that day.Prod check (read-only, 2026-11-08): of 17,203 rows that map to their last-writing scan run, the repair reproduces that scan's own OI ratio on 16,661; 275 match an earlier session (morning-only last scans saw pre-publish OI); 267 have no exact match (248 from notes-tagged, non-full-scan runs).
Until it runs,
pcr_delta_30dmixes the old volume ratio with the new OI ratio.Tests
tests/unit/test_pipeline_aggregates_from_screener.py— UW's AAPL spec example (OI ratio ≠ volume ratio), zero/missing counts.tests/integration/storage/test_pcr_history_oi_repair.py— dry run rolls back, apply values, weekend row takes Friday, no-source → NULL, post-cutover row untouched, rerun changes 0.option_wizard_local: 34,657 in scope, 34,652 with OI source, 5 → NULL.Review
Tribunal: Opus 5.5 lead + Fable 5.1 + GPT-6-Astra (herdr panes). Fixed: cutoff guidance (both), negative
--max-lag-daysnow raises (Astra), stamping docstring (Fable, Astra), tests for zero call OI + per-ticker isolation (Fable; mutation-checked). Astra's "repair picks a later session" CRITICAL was withdrawn after the prod evidence above. All findings verified resolved by the raising seat.Not in scope
snapshot_dateusesdate.today()in a UTC container, so an ad hoc rescan after ~20:00 ET stamps the next day.scan_runs.aggregatesJSON history keeps the oldpcr_oivalue; nothing reads old runs for PCR history. Separate issues.🤖 Generated with Claude Code