Skip to content

ENH: lib: Add deflated_sharpe_ratio() to judge optimize() results - #1388

Open
ipezygj wants to merge 1 commit into
kernc:masterfrom
ipezygj:add-deflated-sharpe-ratio
Open

ENH: lib: Add deflated_sharpe_ratio() to judge optimize() results#1388
ipezygj wants to merge 1 commit into
kernc:masterfrom
ipezygj:add-deflated-sharpe-ratio

Conversation

@ipezygj

@ipezygj ipezygj commented Aug 7, 2026

Copy link
Copy Markdown

What this adds

lib.deflated_sharpe_ratio(stats, trial_sharpe_ratios) — the probability (0–1) that the best run returned by Backtest.optimize() has a Sharpe ratio genuinely greater than zero, after correcting for the multiple testing the optimization itself performs.

Why

optimize() returns the maximum over all tried parameter combinations. The expected best Sharpe of N skill-less trials is already well above zero and grows with N, so the winning run's Sharpe is inflated by selection — the more combinations tried, the more the "best" result reflects luck rather than edge. The library currently reports the winner's Sharpe with no way to ask whether it clears that bar. This is the standard correction: Bailey & López de Prado (2014), The Deflated Sharpe Ratio (doi:10.3905/jpm.2014.40.5.094).

The trial count and the trial Sharpe dispersion are taken from the search itself (the heatmap), not from an assumed default — optimize() already produces exactly the data the estimator needs:

stats, heatmap = bt.optimize(fast=range(5, 30, 5), slow=range(10, 70, 10),
                             maximize='Sharpe Ratio', return_heatmap=True)
deflated_sharpe_ratio(stats, heatmap)

Concrete example with the test SmaCross on GOOG (28 combinations): the winner's Sharpe looks significant as a single test (PSR ≈ 0.98) but does not clear the hurdle its own 28-trial search sets by chance (DSR ≈ 0.18). That flip is the information the number adds.

Implementation notes

  • No new dependencies — normal CDF/PPF come from stdlib statistics.NormalDist; the rest is numpy/pandas already in use.
  • The periodic-returns resampling was extracted from compute_stats() into _stats.periodic_returns() and reused, so the winner's returns are computed by the exact same code path as the reported Sharpe — no duplicated freq logic.
  • Warns if the passed heatmap's name isn't 'Sharpe Ratio' (i.e. optimization maximized something else, so the values wouldn't be trial Sharpes).
  • Verified against an independent implementation of the same estimator (agreement to 3+ decimals on the example above and on the single-trial degenerate case, where DSR reduces to the probabilistic Sharpe ratio).
  • Unit test included; flake8 clean; full python -m backtesting.test suite passes.

The best run of Backtest.optimize() is the maximum over all tried
parameter combinations, so its Sharpe ratio is inflated by multiple
testing: the expected best Sharpe of N skill-less trials grows with N.
Add lib.deflated_sharpe_ratio(stats, trial_sharpe_ratios), computing
the probability the winning Sharpe ratio exceeds zero after correcting
for the number and dispersion of trials actually made
(Bailey & Lopez de Prado 2014, https://doi.org/10.3905/jpm.2014.40.5.094).

Uses only stdlib statistics.NormalDist — no new dependencies.
The periodic-returns resampling is extracted from compute_stats()
into _stats.periodic_returns() and reused, not duplicated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant