Skip to content

ups - #6991

Closed
joseedson18jc wants to merge 27 commits into
Tracer-Cloud:mainfrom
joseedson18jc:main
Closed

ups#6991
joseedson18jc wants to merge 27 commits into
Tracer-Cloud:mainfrom
joseedson18jc:main

Conversation

@joseedson18jc

@joseedson18jc joseedson18jc commented Nov 9, 2026 •

Copy link
Copy Markdown

Fixes #

Describe the changes you have made in this PR -

Demo/Screenshot for feature changes and bug fixes -


Code Understanding and AI Usage

Did you use AI assistance (ChatGPT, Claude, Copilot, etc.) to write any part of this code?

  • No, I wrote all the code myself
  • Yes, I used AI assistance (continue below)

If you used AI assistance:

  • I have reviewed every single line of the AI-generated code
  • I can explain the purpose and logic of each function/component I added
  • I have tested edge cases and understand how the code handles them
  • I have modified the AI output to follow this project's coding standards and conventions

Explain your implementation approach:


Checklist before requesting a review

  • I have added proper PR title and linked to the issue
  • I have performed a self-review of my code
  • I can explain the purpose of every function, class, and logic block I added
  • I understand why my changes work and have tested them thoroughly
  • I have considered potential edge cases and how my code handles them
  • If it is a core feature, I have added thorough tests
  • My code follows the project's style guidelines and conventions

Note: Please check Allow edits from maintainers if you would like us to assist in the PR.

claude and others added 16 commits November 8, 2026 18:58
Self-contained quant_desk/ folder (own venv, not packaged with opensre):
- fetch 30 days of BTC/USDT 1m bars from Binance public data, resample
  to 15m (stamped at bar close) and store both in a local ArcticDB (LMDB)
- EMA, Bollinger and RSI via gs_quant.timeseries plus a volume profile
- csp replays the bars as a stream; one graph runs all 212 EMA long/short
  combos (fast 4-24 x slow 12-96, slow >= 1.5 x fast) with 2 bps fees
- Sharpe ranking, 4-window split, plateau-vs-peak labels, walk-forward
- charts mark every LONG/SHORT on the exact bar; agent.log fires from the
  same csp fill stream
- tests pin lookahead, fee accounting, csp-vs-gs-quant EMA parity and
  log/marker alignment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
- fetch BTC/USDT 15m history since 2022 into ArcticDB; resample to 1h/4h
  (close-stamped bars, closed="right")
- ema_filtered csp node: stays flat outside an ER or ADX trend regime,
  optional min EMA gap (x ATR) and cooldown between entries
- Bollinger fade (mean reversion) csp node, optionally only in chop
- 7 variants x 3 bar sizes swept over full grids; 8 windows, plateau
  needs >= 6/8 positive windows; fee stress at 10 bps; 7-step
  walk-forward vs buy & hold incl. drawdown
- nodes emit fills only; returns rebuilt from fills (same accounting,
  30-day sweep reproduces exactly)
- tests: causality of every regime/band input at many cuts, lookahead per
  variant, fee re-pricing vs real run, 1h resample boundaries, flat and
  cooldown behaviour, Bollinger exit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
- long/flat option on the filtered EMA node; three long/flat variants
- ensemble books trade the average position of the whole 212-combo grid
  (no parameter picked), fees on net exposure change
- volatility targeting (40%, 30d, daily, no leverage) applied to both the
  ensembles and buy & hold so the benchmark is like-for-like
- pass rule fixed up front: W2-W8 at 10 bps, Sharpe above and max DD
  shallower than matching buy & hold; robust only if it passes on two
  adjacent bar sizes
- add 8h and 1d bars; run cells in 4 spawn processes (2.5 min for 10
  variants x 5 bar sizes)
- tests: long/flat never short, exposure vs fill accounting, ensemble
  average, vol weight causal/capped/daily, adjacency rule
- re-run on fresh data: report, charts, candidates.csv

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…s, paper trader

- execution model: decide at bar close, fill at next bar's open; headline
  cost 15 bps (10 taker + 5 slippage); close and 1-bar-late as stress,
  2 bps only as optimistic reference
- walk-forward only picks combos that traded in the training window
- meta walk-forward picks variant x sizing x bar size from past data only;
  Deflated Sharpe Ratio (absolute and vs matched buy & hold)
- sensitivity grid over vol target, gap, cooldown and ADX level
- BTC history from Aug 2017; same untuned long/flat books on 9 coins and
  an equal-weight portfolio
- append-only paper trader (run_paper.py, --loop) for the pre-registered
  BTC 4h book; late catch-up decisions never trade in the past
- one ArcticDB handle per store/library (LMDB + version-map caching)
- memory: int8 positions, fill-based trade counts, 2 workers on 15m
- tests for execution timing, WF eligibility, DSR, paper ledger, portfolio

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…rison

The cross-sectional Sharpe spread was dominated by 15m books losing to
costs, which made every DSR 0.00. Use the null dispersion 1/(T-1), and test
the actual claim (higher Sharpe than B&H) on book minus vol-matched B&H.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
The DSR section in this ROBUSTNESS_REPORT.md uses the inflated
cross-sectional dispersion (all 0.00); superseded by the next output commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…, other venues

- venues: Bitstamp BTC/USD (2013-) and Coinbase BTC-USD (2015-) fetchers
- universe: every Binance USDT pair ever listed (incl. 199 delisted),
  point-in-time monthly top-10 by trailing volume; tickers split at halts
  so the old and new LUNA are never stitched
- universe_portfolio: same untuned books per member sleeve, forced exit
  when a coin's data stops, vs equal-weight buy & hold of the same
  members; survivors-only universe to measure the bias
- long_short: each long/short book vs its long/flat twin and the short
  side's own contribution, scored only on data the decision never saw
- cross_venue: same BTC books per venue, correlations, Bitstamp
  2013-2017 hold-out
- tests: Coinbase column order, halt splitting, risk-asset filter,
  point-in-time membership, forced exit, additive short side

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…oinbase

Fresh data from Binance (incl. 199 delisted pairs), Bitstamp and Coinbase.
ROBUSTNESS_REPORT.md sections 8-10 and the updated "still open" list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…d history

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…ort-from)

history_report.py and variants.py imported history/strategy both as a module
and by name; reference history.Study and strategy.ComboRun instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Greptile code review

This repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md.

Run a review — add a PR comment with:

@greptile review

Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5.

Optional: automate with the greploop skill.

@greptile-apps

greptile-apps Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 0/5

[Medium impact] The PR adds local crypto research and simulated paper-trading workflows.

Do not merge yet: paper trading and universe calculations can report results that do not match the trades or members described.

Findings

  1. P1 Paper fills happen too early ▶
  2. P1 Old paper decisions return ▶
  3. P1 Unfinished bars stay saved ▶
  4. P1 Relisted coin history disappears ▶
  5. P1 Portfolio returns are wrong ▶
  6. P1 Dead coins count as survivors ▶
  7. P1 Books hold different coins ▶
  8. P1 Unqualified strategies still win ▶
  9. P1 Missing venue stops the report ▶
  10. P1 Offline reports still need network ▶
  11. P1 No casualties stops the report ▶
  12. P1 Walk-forward skips trading fees ▶
  13. P2 Charts show the wrong dates ▶
  14. P2 Background trader never starts ▶

Summary

Adds multi-year strategy studies, whole-grid books, cost and execution comparisons, cross-venue checks, a monthly coin universe, and paper trading.

  • Researchers can run a BTC strategy sweep and get charts and a report.
  • The history runner compares strategy books across years and reports their results.
  • Historical strategy comparisons now cover more coins, exchanges, and delisted pairs.
  • A paper trader records new strategy decisions in a saved ledger.

Previous unnumbered threads: joseedson18jc acknowledged the incomplete-bar finding with “ok.” The no-trade chart and shared-constants threads were manually resolved without explanation. The documented Python minimum is now corrected.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Exchanges[Exchange bars] --> Store[ArcticDB]
  Store --> Strategies[Strategy grids]
  Strategies --> Books[Combined exposures]
  Books --> Returns[Execution and costs]
  Returns --> Studies[Historical comparisons]
  Studies --> Reports[Reports and charts]
  Strategies --> Decisions[Paper decisions]
  Decisions --> Ledger[Saved ledger]
  Ledger --> PaperReturns[Paper returns]
Loading

Reviews (2) · Last reviewed commit: "Merge pull request #3 from joseedson18jc..." · Reviewed by Greptile

Comment thread quant_desk/desk/sweep.py Outdated
Comment on lines +143 to +144
test_r = full[best].loc[test]
oos_parts.append(test_r)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Walk-forward skips trading fees

walk_forward copies returns from an already-running strategy without paying to enter its position. This skips the initial entry fee and any fee needed when a later pick changes the held position. The report and chart therefore overstate returns after fees. Track the position actually held across windows and charge for changes when each pick takes over.

Comment thread quant_desk/desk/data.py Resolved
Comment thread quant_desk/desk/chart.py Resolved
Comment thread quant_desk/README.md Resolved
Comment thread quant_desk/desk/config.py Resolved

@joseedson18jc joseedson18jc left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gggg

@joseedson18jc joseedson18jc left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apptoved

Comment thread quant_desk/desk/config.py Resolved
joseedson18jc and others added 2 commits November 9, 2026 05:12
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

@joseedson18jc joseedson18jc left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved

Comment thread quant_desk/desk/data.py Resolved
Comment thread quant_desk/desk/data.py Resolved
claude and others added 5 commits November 9, 2026 08:16
…r a cross

ema_cross only traded on a sign flip, so after warm-up it sat flat until the
first EMA cross, although the variant is documented as always long or short.
It also put always_in out of step with its long_flat twin (ema_filtered
opens as soon as it is warmed up), which skewed the short-side measurement
(always_in minus long_flat) at the start of every series.

Now the first warmed-up bar opens at the EMAs' sign and every later cross
reverses; the fee stays fee_bps x |position change| (1 unit to open, 2 to
reverse). Tests pin the opening trade on a trend that never crosses, and that
long_flat equals the long half of always_in on every bar.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
Reports, charts, CSVs and logs under quant_desk/output/ are regenerated by
run_desk.py, run_history.py and run_paper.py; ignore the folder and remove
the committed snapshots so the PR only carries source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
main applied a fix for the first EMA entry and the output/ ignore directly to
the old 30-day strategy.py and .gitignore. Keep this branch's versions of both:
they already carry the same fix (always_in opens after warm-up, every later
cross reverses, fee = |position change|) with regression tests, and the
rewritten multi-year strategy module. Takes main's README Python 3.11+ change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
quant_desk: multi-year study, robustness checks, survivorship-free universe, other venues, paper trader
claude and others added 2 commits November 9, 2026 08:47
Re-ran run_history.py with the corrected always_in entry. None of the figures
the README quotes moved (only a few always_in rows of the report's long/short
tables, mostly daily bars, changed). Corrects the top-10 buy & hold worst
drawdown to -95% (8h; -94% was the 4h row) and notes that the full report is
generated by run_history.py, not committed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
quant_desk README: results re-checked after the always_in entry fix
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Greptile code review

This repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md.

Run a review — add a PR comment with:

@greptile review

Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5.

Optional: automate with the greploop skill.

…ensre

A fork's own runs (pushes to its main included) have no OPENAI_API_KEY, so
the live tool-selection contracts fail the shard with "requires configured,
current credentials" and the CI Gate goes red on every push to a fork's main.
Fork PRs already drop live_llm tests; extend that to the cli-live-agent shard
whenever the repository is not Tracer-Cloud/opensre. The shard's offline
tests and coverage upload still run, and upstream behaviour is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the right correction is clear:

In quant_desk/desk/strategy.py, the entry logic should allow the first valid non-zero EMA signal to open a position when the strategy is flat.
The condition should not require a prior non-zero s_prev_sign before the first trade.
Also, generated output under quant_desk/output/ should be ignored, not committed.
The corrected logic should be equivalent to:

If s_position == 0 and sign != 0 after warmup, open at sign
Else if already in a position and the sign flips, close/reverse with fee
Keep the fee logic tied to the position change magnitude

Comment thread quant_desk/desk/paper.py
"""Each decision applies from the first bar closed at or after the moment it was made."""
decided = pd.to_datetime(ledger["decided_at"], utc=True)
when = pd.Series(ledger.index, index=ledger.index)
effective = when.where(~ledger["late"].astype(bool), decided)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Paper fills happen too early

effective_exposure backdates ordinary paper decisions to the bar close. The loop normally runs 90 seconds afterward, but late=False makes next_open pricing use the opening price from 90 seconds earlier. A run made hours into a 4h bar gets the same treatment. Paper returns therefore include moves before the bot made its decision.

Use the actual decision time and a price available afterward.

Comment thread quant_desk/desk/paper.py
Comment on lines +108 to +112
pos = index.searchsorted(pd.DatetimeIndex(effective), side="left")
keep = pos < len(index)
exposure = pd.Series(ledger["exposure"].to_numpy()[keep], index=index[pos[keep]])
exposure = exposure[~exposure.index.duplicated(keep="last")]
return exposure.reindex(index).ffill().fillna(0.0)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Old paper decisions return

Catch-up rows can replace a newer paper decision later. After reconnecting at 16:02, older late rows map to 20:00 while the newest 16:00 row maps to 16:00. If the next run is missed, rebuilding performance makes the old catch-up exposure take over at 20:00, even though the bot was offline.

Discard superseded catch-up decisions or ensure older bars cannot replace newer decisions.

Comment thread quant_desk/desk/paper.py
start = (now - timedelta(days=config.PAPER_LOOKBACK_DAYS)).strftime("%Y-%m-%d")
bars = data.fetch_history(start, book.symbol, interval, end=now)
native = 240 if interval == "4h" else 15
return bars if book.minutes == native else data.resample_closed(bars, book.minutes)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Unfinished bars stay saved

For supported 1h, 8h, and 1d books, fetch_recent can return an unfinished bar. An 8h book fetched at 04:02, for example, gets a partial bar stamped 08:00. record saves that decision, excludes the same timestamp on later runs, and keeps the partial prices through combine_first. The completed bar never replaces it.

Drop unfinished resampled bars before deciding or storing them.

Comment on lines +156 to +159
segments = split_segments(raw, pd.Timedelta(days=config.UNIVERSE_SPLIT_GAP_DAYS))
if segment >= len(segments):
return pd.DataFrame()
bars = segments[segment]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Relisted coin history disappears

_segment_4h applies a segment number from the full daily history to a shortened 4h download. If a ticker's second incarnation first joins the universe more than 200 days after the first incarnation ended, the download contains only one segment. The _2 lookup then returns empty and drops that member from the study.

Preserve each instrument's start and end dates instead of renumbering segments after shortening history.

Comment on lines +63 to +65
parts.append(to_daily(sleeve_returns(bars, exposure, mask, config.UNIVERSE_SIZE)))
frame = pd.concat(parts, axis=1).fillna(0.0)
daily = frame.sum(axis=1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Portfolio returns are wrong

portfolio_daily compounds each already-weighted coin separately and then adds the daily returns. That is not the return of the combined portfolio. With two equally weighted coins each gaining 10% on two bars, this gives 20.5% instead of 21%. The report then calculates growth and drawdowns from the wrong series.

Add the weighted returns at each bar first, then compound the combined series into days.

)

ax = axes[1]
bars, exp = expos[(cross_venue.venue_symbol("bitstamp"), "4h")]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Missing venue stops the report

holdouts_chart requires Bitstamp even though venue ingestion explicitly skips failed downloads. If Bitstamp fails or is absent from the cached data, this lookup raises KeyError after the other studies finish, before the final report is written.

Skip the unavailable panel or check required venues before starting the study.

Comment thread quant_desk/run_history.py
)

log.info("paper trading: one step for %s", [b.key for b in paper.paper_books()])
status = paper.run_once()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Offline reports still need network

--no-fetch still downloads missing universe members and always runs live paper fetching. Even with every study dataset cached, an offline run reaches paper.run_once() and fails before writing the final report.

Keep network work behind the fetch flag and make the paper step separately optional.

f"{book}_while_member": float((1 + book_r).prod() - 1),
}
)
return pd.DataFrame(rows).sort_values("buy_and_hold_while_member")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 No casualties stops the report

casualties crashes when no eligible member has stopped trading. With no rows, the returned DataFrame has no buy_and_hold_while_member column, so sorting raises KeyError and prevents the report from finishing.

Return an empty frame with the expected columns, matching the report's existing support for an empty casualties table.

"EMA trend variants (fast × slow)" if family == "trend" else "Bollinger fade (window × k)"
)
fig.suptitle(
f"{label}: Sharpe over the whole grid, Jan 2022 – Nov 2026.\n"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Charts show the wrong dates

The heatmap title says January 2022 through November 2026, but the plotted Sharpe values cover the whole history starting in August 2017. The fixed end date also becomes wrong on later runs. Readers therefore see the wrong period for these results.

Build the label from the study's actual dates.

Comment thread quant_desk/README.md
valem a partir de quando foram tomadas. Status em `output/paper/STATUS.md`.

```bash
nohup .venv/bin/python run_paper.py --loop >> output/paper/paper.log 2>&1 & # 24/7

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Background trader never starts

The documented background command fails on a fresh checkout because output/paper/ does not exist. The shell opens the redirected log before Python starts, so run_once never gets a chance to create the directory.

Add mkdir -p output/paper before this command so a new user can start the trader.

ci: run cli-live-agent without live_llm tests outside Tracer-Cloud/opensre
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants