Repository navigation
ups - #6991
ups#6991joseedson18jc wants to merge 27 commits into
Conversation
Self-contained quant_desk/ folder (own venv, not packaged with opensre): - fetch 30 days of BTC/USDT 1m bars from Binance public data, resample to 15m (stamped at bar close) and store both in a local ArcticDB (LMDB) - EMA, Bollinger and RSI via gs_quant.timeseries plus a volume profile - csp replays the bars as a stream; one graph runs all 212 EMA long/short combos (fast 4-24 x slow 12-96, slow >= 1.5 x fast) with 2 bps fees - Sharpe ranking, 4-window split, plateau-vs-peak labels, walk-forward - charts mark every LONG/SHORT on the exact bar; agent.log fires from the same csp fill stream - tests pin lookahead, fee accounting, csp-vs-gs-quant EMA parity and log/marker alignment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
- fetch BTC/USDT 15m history since 2022 into ArcticDB; resample to 1h/4h (close-stamped bars, closed="right") - ema_filtered csp node: stays flat outside an ER or ADX trend regime, optional min EMA gap (x ATR) and cooldown between entries - Bollinger fade (mean reversion) csp node, optionally only in chop - 7 variants x 3 bar sizes swept over full grids; 8 windows, plateau needs >= 6/8 positive windows; fee stress at 10 bps; 7-step walk-forward vs buy & hold incl. drawdown - nodes emit fills only; returns rebuilt from fills (same accounting, 30-day sweep reproduces exactly) - tests: causality of every regime/band input at many cuts, lookahead per variant, fee re-pricing vs real run, 1h resample boundaries, flat and cooldown behaviour, Bollinger exit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
- long/flat option on the filtered EMA node; three long/flat variants - ensemble books trade the average position of the whole 212-combo grid (no parameter picked), fees on net exposure change - volatility targeting (40%, 30d, daily, no leverage) applied to both the ensembles and buy & hold so the benchmark is like-for-like - pass rule fixed up front: W2-W8 at 10 bps, Sharpe above and max DD shallower than matching buy & hold; robust only if it passes on two adjacent bar sizes - add 8h and 1d bars; run cells in 4 spawn processes (2.5 min for 10 variants x 5 bar sizes) - tests: long/flat never short, exposure vs fill accounting, ensemble average, vol weight causal/capped/daily, adjacency rule - re-run on fresh data: report, charts, candidates.csv Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…s, paper trader - execution model: decide at bar close, fill at next bar's open; headline cost 15 bps (10 taker + 5 slippage); close and 1-bar-late as stress, 2 bps only as optimistic reference - walk-forward only picks combos that traded in the training window - meta walk-forward picks variant x sizing x bar size from past data only; Deflated Sharpe Ratio (absolute and vs matched buy & hold) - sensitivity grid over vol target, gap, cooldown and ADX level - BTC history from Aug 2017; same untuned long/flat books on 9 coins and an equal-weight portfolio - append-only paper trader (run_paper.py, --loop) for the pre-registered BTC 4h book; late catch-up decisions never trade in the past - one ArcticDB handle per store/library (LMDB + version-map caching) - memory: int8 positions, fill-based trade counts, 2 workers on 15m - tests for execution timing, WF eligibility, DSR, paper ledger, portfolio Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…rison The cross-sectional Sharpe spread was dominated by 15m books losing to costs, which made every DSR 0.00. Use the null dispersion 1/(T-1), and test the actual claim (higher Sharpe than B&H) on book minus vol-matched B&H. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
The DSR section in this ROBUSTNESS_REPORT.md uses the inflated cross-sectional dispersion (all 0.00); superseded by the next output commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…status Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…, other venues - venues: Bitstamp BTC/USD (2013-) and Coinbase BTC-USD (2015-) fetchers - universe: every Binance USDT pair ever listed (incl. 199 delisted), point-in-time monthly top-10 by trailing volume; tickers split at halts so the old and new LUNA are never stitched - universe_portfolio: same untuned books per member sleeve, forced exit when a coin's data stops, vs equal-weight buy & hold of the same members; survivors-only universe to measure the bias - long_short: each long/short book vs its long/flat twin and the short side's own contribution, scored only on data the decision never saw - cross_venue: same BTC books per venue, correlations, Bitstamp 2013-2017 hold-out - tests: Coinbase column order, halt splitting, risk-asset filter, point-in-time membership, forced exit, additive short side Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…nding) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…oinbase Fresh data from Binance (incl. 199 delisted pairs), Bitstamp and Coinbase. ROBUSTNESS_REPORT.md sections 8-10 and the updated "still open" list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…d history Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
…ort-from) history_report.py and variants.py imported history/strategy both as a module and by name; reference history.Study and strategy.ComboRun instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
Greptile code reviewThis repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md. Run a review — add a PR comment with: Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5. Optional: automate with the greploop skill. |
[Medium impact] The PR adds local crypto research and simulated paper-trading workflows. Do not merge yet: paper trading and universe calculations can report results that do not match the trades or members described. Findings
SummaryAdds multi-year strategy studies, whole-grid books, cost and execution comparisons, cross-venue checks, a monthly coin universe, and paper trading.
Previous unnumbered threads:
|
| test_r = full[best].loc[test] | ||
| oos_parts.append(test_r) |
There was a problem hiding this comment.
Walk-forward skips trading fees
walk_forward copies returns from an already-running strategy without paying to enter its position. This skips the initial entry fee and any fee needed when a later pick changes the held position. The report and chart therefore overstate returns after fees. Track the position actually held across windows and charge for changes when each pick takes over.
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…r a cross ema_cross only traded on a sign flip, so after warm-up it sat flat until the first EMA cross, although the variant is documented as always long or short. It also put always_in out of step with its long_flat twin (ema_filtered opens as soon as it is warmed up), which skewed the short-side measurement (always_in minus long_flat) at the start of every series. Now the first warmed-up bar opens at the EMAs' sign and every later cross reverses; the fee stays fee_bps x |position change| (1 unit to open, 2 to reverse). Tests pin the opening trade on a trend that never crosses, and that long_flat equals the long half of always_in on every bar. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
Reports, charts, CSVs and logs under quant_desk/output/ are regenerated by run_desk.py, run_history.py and run_paper.py; ignore the folder and remove the committed snapshots so the PR only carries source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
main applied a fix for the first EMA entry and the output/ ignore directly to the old 30-day strategy.py and .gitignore. Keep this branch's versions of both: they already carry the same fix (always_in opens after warm-up, every later cross reverses, fee = |position change|) with regression tests, and the rewritten multi-year strategy module. Takes main's README Python 3.11+ change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
quant_desk: multi-year study, robustness checks, survivorship-free universe, other venues, paper trader
Re-ran run_history.py with the corrected always_in entry. None of the figures the README quotes moved (only a few always_in rows of the report's long/short tables, mostly daily bars, changed). Corrects the top-10 buy & hold worst drawdown to -95% (8h; -94% was the 4h row) and notes that the full report is generated by run_history.py, not committed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
quant_desk README: results re-checked after the always_in entry fix
Greptile code reviewThis repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md. Run a review — add a PR comment with: Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5. Optional: automate with the greploop skill. |
…ensre A fork's own runs (pushes to its main included) have no OPENAI_API_KEY, so the live tool-selection contracts fail the shard with "requires configured, current credentials" and the CI Gate goes red on every push to a fork's main. Fork PRs already drop live_llm tests; extend that to the cli-live-agent shard whenever the repository is not Tracer-Cloud/opensre. The shard's offline tests and coverage upload still run, and upstream behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014a3BAbue5mtQrZTtukw8LU
There was a problem hiding this comment.
But the right correction is clear:
In quant_desk/desk/strategy.py, the entry logic should allow the first valid non-zero EMA signal to open a position when the strategy is flat.
The condition should not require a prior non-zero s_prev_sign before the first trade.
Also, generated output under quant_desk/output/ should be ignored, not committed.
The corrected logic should be equivalent to:
If s_position == 0 and sign != 0 after warmup, open at sign
Else if already in a position and the sign flips, close/reverse with fee
Keep the fee logic tied to the position change magnitude
| """Each decision applies from the first bar closed at or after the moment it was made.""" | ||
| decided = pd.to_datetime(ledger["decided_at"], utc=True) | ||
| when = pd.Series(ledger.index, index=ledger.index) | ||
| effective = when.where(~ledger["late"].astype(bool), decided) |
There was a problem hiding this comment.
effective_exposure backdates ordinary paper decisions to the bar close. The loop normally runs 90 seconds afterward, but late=False makes next_open pricing use the opening price from 90 seconds earlier. A run made hours into a 4h bar gets the same treatment. Paper returns therefore include moves before the bot made its decision.
Use the actual decision time and a price available afterward.
| pos = index.searchsorted(pd.DatetimeIndex(effective), side="left") | ||
| keep = pos < len(index) | ||
| exposure = pd.Series(ledger["exposure"].to_numpy()[keep], index=index[pos[keep]]) | ||
| exposure = exposure[~exposure.index.duplicated(keep="last")] | ||
| return exposure.reindex(index).ffill().fillna(0.0) |
There was a problem hiding this comment.
Catch-up rows can replace a newer paper decision later. After reconnecting at 16:02, older late rows map to 20:00 while the newest 16:00 row maps to 16:00. If the next run is missed, rebuilding performance makes the old catch-up exposure take over at 20:00, even though the bot was offline.
Discard superseded catch-up decisions or ensure older bars cannot replace newer decisions.
| start = (now - timedelta(days=config.PAPER_LOOKBACK_DAYS)).strftime("%Y-%m-%d") | ||
| bars = data.fetch_history(start, book.symbol, interval, end=now) | ||
| native = 240 if interval == "4h" else 15 | ||
| return bars if book.minutes == native else data.resample_closed(bars, book.minutes) |
There was a problem hiding this comment.
For supported 1h, 8h, and 1d books, fetch_recent can return an unfinished bar. An 8h book fetched at 04:02, for example, gets a partial bar stamped 08:00. record saves that decision, excludes the same timestamp on later runs, and keeps the partial prices through combine_first. The completed bar never replaces it.
Drop unfinished resampled bars before deciding or storing them.
| segments = split_segments(raw, pd.Timedelta(days=config.UNIVERSE_SPLIT_GAP_DAYS)) | ||
| if segment >= len(segments): | ||
| return pd.DataFrame() | ||
| bars = segments[segment] |
There was a problem hiding this comment.
Relisted coin history disappears
_segment_4h applies a segment number from the full daily history to a shortened 4h download. If a ticker's second incarnation first joins the universe more than 200 days after the first incarnation ended, the download contains only one segment. The _2 lookup then returns empty and drops that member from the study.
Preserve each instrument's start and end dates instead of renumbering segments after shortening history.
| parts.append(to_daily(sleeve_returns(bars, exposure, mask, config.UNIVERSE_SIZE))) | ||
| frame = pd.concat(parts, axis=1).fillna(0.0) | ||
| daily = frame.sum(axis=1) |
There was a problem hiding this comment.
portfolio_daily compounds each already-weighted coin separately and then adds the daily returns. That is not the return of the combined portfolio. With two equally weighted coins each gaining 10% on two bars, this gives 20.5% instead of 21%. The report then calculates growth and drawdowns from the wrong series.
Add the weighted returns at each bar first, then compound the combined series into days.
| ) | ||
|
|
||
| ax = axes[1] | ||
| bars, exp = expos[(cross_venue.venue_symbol("bitstamp"), "4h")] |
There was a problem hiding this comment.
Missing venue stops the report
holdouts_chart requires Bitstamp even though venue ingestion explicitly skips failed downloads. If Bitstamp fails or is absent from the cached data, this lookup raises KeyError after the other studies finish, before the final report is written.
Skip the unavailable panel or check required venues before starting the study.
| ) | ||
|
|
||
| log.info("paper trading: one step for %s", [b.key for b in paper.paper_books()]) | ||
| status = paper.run_once() |
There was a problem hiding this comment.
Offline reports still need network
--no-fetch still downloads missing universe members and always runs live paper fetching. Even with every study dataset cached, an offline run reaches paper.run_once() and fails before writing the final report.
Keep network work behind the fetch flag and make the paper step separately optional.
| f"{book}_while_member": float((1 + book_r).prod() - 1), | ||
| } | ||
| ) | ||
| return pd.DataFrame(rows).sort_values("buy_and_hold_while_member") |
There was a problem hiding this comment.
No casualties stops the report
casualties crashes when no eligible member has stopped trading. With no rows, the returned DataFrame has no buy_and_hold_while_member column, so sorting raises KeyError and prevents the report from finishing.
Return an empty frame with the expected columns, matching the report's existing support for an empty casualties table.
| "EMA trend variants (fast × slow)" if family == "trend" else "Bollinger fade (window × k)" | ||
| ) | ||
| fig.suptitle( | ||
| f"{label}: Sharpe over the whole grid, Jan 2022 – Nov 2026.\n" |
There was a problem hiding this comment.
The heatmap title says January 2022 through November 2026, but the plotted Sharpe values cover the whole history starting in August 2017. The fixed end date also becomes wrong on later runs. Readers therefore see the wrong period for these results.
Build the label from the study's actual dates.
| valem a partir de quando foram tomadas. Status em `output/paper/STATUS.md`. | ||
|
|
||
| ```bash | ||
| nohup .venv/bin/python run_paper.py --loop >> output/paper/paper.log 2>&1 & # 24/7 |
There was a problem hiding this comment.
Background trader never starts
The documented background command fails on a fresh checkout because output/paper/ does not exist. The shell opens the redirected log before Python starts, so run_once never gets a chance to create the directory.
Add mkdir -p output/paper before this command so a new user can start the trader.
ci: run cli-live-agent without live_llm tests outside Tracer-Cloud/opensre
Fixes #
Describe the changes you have made in this PR -
Demo/Screenshot for feature changes and bug fixes -
Code Understanding and AI Usage
Did you use AI assistance (ChatGPT, Claude, Copilot, etc.) to write any part of this code?
If you used AI assistance:
Explain your implementation approach:
Checklist before requesting a review
Note: Please check Allow edits from maintainers if you would like us to assist in the PR.