[scikit-learn] Combine already fitted models

[Sebastian Raschka]

Hi, Rares,

> I am looking at VotingClassifier but it seems that it is expected that the estimators are fitted when VotingClassifier.fit() is called. I don't see how I can have already fitted classifiers combined under a VotingClassifier.

I think the opposite is true: The classifiers provided via an `estimators` argument upon initialization will be cloned and fitted if you call VotingClassifier's  fit(). Based on your follow-up question, I think you meant "it is expected that the estimators are *not* fitted when VotingClassifier.fit() is called," right?!

>  I don't see how I can have already fitted classifiers combined under a VotingClassifier.


The VotingClassifier in scikit-learn is based on the EnsembleVoteClassifier I had implemented in mlxtend (http://rasbt.github.io/mlxtend/user_guide/classifier/EnsembleVoteClassifier/#api). While I generally recommend using the VotingClassifier in scikit-learn, the code base of EnsembleVoteClassifier should be quite similar, and I have added a `refit` param which can be set to True or False. If refit=True, it's the same behavior as in sklearn. If refit=False, however, it will not refit the estimators and will allow you to use pre-fit classifiers, which is what you are asking for, I think?

@scikit-learn devs:
Not sure if such a parameter should be added to scikit-learn's VotingClassifier as it may cause some weird behavior in GridSearch etc? Otherwise, I am happy to add an issue or submit a PR to discuss/work on this further :)

Best,
Sebastian


> On Oct 1, 2017, at 6:53 PM, Rares Vernica <rvernica at gmail.com> wrote:
> 
> Hello,
> 
> I have a distributed setup where subsets of the data is available at different hosts. I plan to have each host fit a model with the subset of the data it owns. Once these individual models are fitted, how can I go about and combine them under one model.
> 
> I don't have a preference on a specific algorithm, but I am looking into a classification problem.
> 
> I am looking at VotingClassifier but it seems that it is expected that the estimators are fitted when VotingClassifier.fit() is called. I don't see how I can have already fitted classifiers combined under a VotingClassifier.
> 
> Thanks!
> Rares
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn