Quantile regression

[MaxHalford]

See scikit-learn

We already have losses.Quantile.

[Siege-Cobby]

Hi @MaxHalford, I've been playing around this issue for sometime. but i held off mentioning it here since I've been pretty pressed for time lately, and didn't want to "I'll take this up!" then go quiet for months. So I just went ahead with it when i had some leasure time on my had.

I did QuantileRegressor in linear_model, wiring the existing losses.Quantile into the standard GLM machinery (similar structure to LinearRegression/PARegressor).

Includes:

quantile parameter (default 0.5 = median regression) with validation
Full docstring with examples and reference to scikit-learn's QuantileRegressor
Tests covering invalid quantile handling and convergence to known quantiles on a synthetic skewed distribution
All existing tests pass, and it's lint/format clean. I'll open a PR shortly — happy to adjust based on any design feedback first if you'd prefer to steer the API before I finalize it.

[MaxHalford]

Hey @Siege-Cobby! Thanks for your contribution. I have to be honest, I am not very much a fan, for two reasons:

• I am not in favour of adding estimators that do not bring anything new to the table. Here you're just inheriting from LinearRegression and overriding a parameter. Typically, I would add a recipe to the documentation, not an estimator.
• We should provide users with a quantile regression solution that works out of the box. Here, I don't think it works. We want a quantile regression that is conditioned on x and guarantees that the 95% interval is valid. Like LightGBM for instance. My current belief is that the conformal prediction field is the direction to take.

[Siege-Cobby]

Fair enough @MaxHalford! both points make sense.

• On the first, I do agree. this seemed like just a config of what’s already there, not worth its own estimator. a recipe in the docs is the right home for that.

• On the second, a straight linear fit doesn’t give you conditioned, calibrated quantiles the way LightGBM does either, so you’re right there too.

I’m just taking a second look at this and thinking something tree-based.

What do you make of Adaptive Random Forest style, where each leaf tracks the actual distribution, not just a mean, so it can give proper conditioned quantiles and intervals? I’ll also add the recipe for the simple linear case while I’m at it.

[MaxHalford]

Yes the leaf approach can/will work, but it will be specific to that kind of model. I'm in favour of documenting that + the linear model approach into a dedicated recipe. What I'm also saying is that we should additionally look for a generic solution that works for any kind estimator.

[Siege-Cobby]

Thanks @MaxHalford. I’ve reworked #1971 based on this discussion.

The standalone QuantileRegressor has been removed and the PR is now a dedicated recipe covering both approaches we discussed:

linear quantile regression using the existing LinearRegression + optim.losses.Quantile
the tree/leaf-based quantile approach, documented conceptually as a model-specific way of estimating quantiles from the target distribution within a leaf

The recipe also demonstrates conf.RegressionJackknife as River’s existing estimator-agnostic interval approach, while noting that its intervals are marginal rather than feature-conditional.

I’ve deliberately kept the PR documentation-only, with no new estimator or public API.

The remaining piece I’d like to explore separately is the generic feature-conditional solution that could work across arbitrary estimators.

Updated PR: #1971