Ship the reference benchmark pipelines in the package, with an accessor

[bkowshik]

The 26 .yml + 2 .py configs in the repo-root pipelines/ directory ship in no artifact — not the 1.5.0 wheel, not the sdist, not pip install git+…@<sha> (PEP 517 builds the wheel). MANIFEST.in alone would not fix it: it governs the sdist, and include-package-data reaches the wheel only for files inside a package — recursive-include docs … ships nothing, include moabb/datasets/summary_*.csv ships all five.

So pipelines="./pipelines/" resolves only from a checkout, and the documented route to the reference pipelines is the GitHub link in plot_benchmark.py.

Downstream. Four cases, none of which produced an issue about the shared cause:

• Reproducing results for PhysionetMI? #640 — hand-written CSP+LDA missed the published number; resolved with a link.
• How to save a pipeline to yml? #639 — answered by hand-transcribing CSP.yml into Python in a comment.
• Keras_DeepConvNet.yml and Keras_ShallowConvNet.yml pipelines skipping datasets silently #1072 (open) — reproducibility thesis on pipelines="../pipelines", datasets skipped silently.
• facebookresearch/neuroai — cov_ts_lr.yaml:10 cites TSLR.yml, then diverges in three steps.

People who want the reference approximate it, and the result is plausible enough to publish.

In-repo. Nothing exercises these configs — CI uses the FakePipeline fixtures. from:/name: resolve at runtime, so an upstream rename goes undetected until a user hits it; #1123 renamed imports one layer beneath them. All 28 still resolve against current deps, but nothing asserts it.

Proposal

1. Ship the configs as moabb/pipelines/configs/ — MANIFEST.in plus an explicit [tool.setuptools.package-data] entry; no __init__.py, so packages.find still ignores it.
2. moabb.pipelines.get_benchmark_pipelines(paradigm=None, dir_name=None) — parse_pipelines_from_directory over that directory, returning the list-of-dicts benchmark() has accepted since Easier handling of pipelines for benchmark() #826. Mirrors moabb.datasets.base.get_summary_table.
3. benchmark(pipelines=None) and run.py --pipelines default to it, with a DeprecationWarning when ./pipelines/ exists and nothing was passed — rather than CWD-dependent resolution. Partly addresses Add moabb.run as installed script #175.
4. Warn when parse_pipelines_from_directory gets a valid directory with zero configs; it currently returns [] silently, which is what Keras_DeepConvNet.yml and Keras_ShallowConvNet.yml pipelines skipping datasets silently #1072 hit. Its .py branch also does an unguarded foo.PIPELINE.
5. Parametrized test constructing every packaged config, so a rename is a red build rather than a user report.

In use:

from moabb import benchmark
from moabb.pipelines import get_benchmark_pipelines

pipelines = get_benchmark_pipelines(paradigm="LeftRightImagery")
results = benchmark(pipelines=pipelines, evaluations=["WithinSession"])

Move or mirror? git mv pipelines moabb/pipelines/configs is simpler and removes the CWD question in (3), but breaks external blob/develop/pipelines/<file> links. The alternative keeps pipelines/ canonical plus a synced copy, costing a pre-commit hook and an identity test. I lean move; your call.

Happy to implement.