[scikit-learn] Efficient forward stepwise regression

[Alexandre Gramfort]

hi Matt,

I'd like to implement a forward stepwise regression algorithm using the
> efficient procedure described in the first problem here
> <http://stat.rutgers.edu/home/hxiao/stat588_2011/hw1.pdf>. It does not
> seem that such a model exists anywhere in Python. Would it be useful for me
> to write this model up for sklearn?
>

to be considered I would first ask you to evaluate and discuss what you
think it will bring
over existing estimators. Typically do you foresee a clear benefit compared
to Lars or LassoLars ?

For more see
https://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms


> If you're interested, here's a high-level view of how I think it would
> work:
>
> - The model would have sklearn.linear_model.LinearRegression as its base
> class.
> - The additional model parameters would include
>
>    - An array of the indices (or column names) of the features in X1
>    - The Q and R matrices
>
> - The additional methods would include
>
>    - An add_features() method that adds a specified number of features to
>    the model. Updates all model parameters
>    - A fit() method that requires a specification of the number of
>    parameters to fit and optional sample weight. It calls the add_features
>    method once on a model with no features.
>
>
the API of scikit-learn estimator is quite strict. See

https://scikit-learn.org/stable/developers/develop.html?highlight=check_estimator

I invite you to read
https://scikit-learn.org/stable/developers/contributing.html?highlight=contribut
if you are willing to help the team.

Alex


> I would do this for OLS first, but supposedly it could be adapted for
> regularized models as well.
>
> How does this sound?
>
> Thanks,
>
> Matt S.
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scikit-learn/attachments/20210426/62608e26/attachment.html>