[scikit-learn] Efficient forward stepwise regression

Alexandre Gramfort alexandre.gramfort at inria.fr
Mon Apr 26 02:23:03 EDT 2021

hi Matt,

I'd like to implement a forward stepwise regression algorithm using the
> efficient procedure described in the first problem here
> <http://stat.rutgers.edu/home/hxiao/stat588_2011/hw1.pdf>. It does not
> seem that such a model exists anywhere in Python. Would it be useful for me
> to write this model up for sklearn?

to be considered I would first ask you to evaluate and discuss what you
think it will bring
over existing estimators. Typically do you foresee a clear benefit compared
to Lars or LassoLars ?

For more see

> If you're interested, here's a high-level view of how I think it would
> work:
> - The model would have sklearn.linear_model.LinearRegression as its base
> class.
> - The additional model parameters would include
>    - An array of the indices (or column names) of the features in X1
>    - The Q and R matrices
> - The additional methods would include
>    - An add_features() method that adds a specified number of features to
>    the model. Updates all model parameters
>    - A fit() method that requires a specification of the number of
>    parameters to fit and optional sample weight. It calls the add_features
>    method once on a model with no features.
the API of scikit-learn estimator is quite strict. See


I invite you to read
if you are willing to help the team.


> I would do this for OLS first, but supposedly it could be adapted for
> regularized models as well.
> How does this sound?
> Thanks,
> Matt S.
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scikit-learn/attachments/20210426/62608e26/attachment.html>

More information about the scikit-learn mailing list