Estimator.predict() thread safety
Hi all, I notice when I train a model and expose the predict function through a web API, predict takes longer to run in a multi-threaded environment than a single-threaded one. I'm guessing the root cause has something to do with thread collisions but I must be doing something incorrectly within the code (I set n_jobs=-1 for both FeatureUnion and estimators/gridsearchers there) has someone else ran into a similar issue? I can provide more details if this Q is rather opaque still best, Philip
That’s interesting. I believe n_jobs just wraps multiprocessing through joblib. So, setting it to -1 can essentially spawn as many processes as you have processors available, which may come with an undesired overhead in your environment? I am not sure if this is (still) and issue, but maybe have a look at this one here: https://twiki.cern.ch/twiki/bin/view/Main/PythonLoggingThreadingMultiprocess...
Mixing Python modules multiprocessing and threading along with logging error/debug messages with module logging is a very bad idea which leads to unexpected process stalling. Indeed, having concurrent entities based on both multiprocessing.Process and threading.Thread within a single application causes right the multiprocessing.Process to stall and never the threading.Thread, as the attached examples demonstrate. The problematic behaviour demonstrates:
Best, Sebastian
On Jun 17, 2016, at 11:01 AM, Philip Tully <tully@csc.kth.se> wrote:
Hi all,
I notice when I train a model and expose the predict function through a web API, predict takes longer to run in a multi-threaded environment than a single-threaded one. I'm guessing the root cause has something to do with thread collisions but I must be doing something incorrectly within the code (I set n_jobs=-1 for both FeatureUnion and estimators/gridsearchers there)
has someone else ran into a similar issue? I can provide more details if this Q is rather opaque still
best, Philip _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
I set n_jobs=-1 for both FeatureUnion and estimators/gridsearchers there
I am typically careful with this, e.g., if my machine has 16 cores, I’d set feature union to n_jobs=3 and the gridsearch_cv to n_jobs=4 or so. Curious to hear what the scikit devs think about nesting calls n_jobs=-1; am I too conservative? Best, Sebastian
On Jun 17, 2016, at 11:01 AM, Philip Tully <tully@csc.kth.se> wrote:
Hi all,
I notice when I train a model and expose the predict function through a web API, predict takes longer to run in a multi-threaded environment than a single-threaded one. I'm guessing the root cause has something to do with thread collisions but I must be doing something incorrectly within the code (I set n_jobs=-1 for both FeatureUnion and estimators/gridsearchers there)
has someone else ran into a similar issue? I can provide more details if this Q is rather opaque still
best, Philip _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
I am typically careful with this, e.g., if my machine has 16 cores, I’d set feature union to n_jobs=3 and the gridsearch_cv to n_jobs=4 or so. Curious to hear what the scikit devs think about nesting calls n_jobs=-1; am I too conservative?
Nested parallelism doesn't work. It's a limitation of multiprocessing.
Gotcha - so perhaps I should ensure FeatureUnion[n_jobs] + GirdSearch[n_jobs] < # cores? On Fri, Jun 17, 2016 at 11:21 AM, Gael Varoquaux < gael.varoquaux@normalesup.org> wrote:
I am typically careful with this, e.g., if my machine has 16 cores, I’d set feature union to n_jobs=3 and the gridsearch_cv to n_jobs=4 or so. Curious to hear what the scikit devs think about nesting calls n_jobs=-1; am I too conservative?
Nested parallelism doesn't work. It's a limitation of multiprocessing. _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
No, the inner loop won't be run parallel. G On Fri, Jun 17, 2016 at 11:46:59AM -0400, Philip Tully wrote:
Gotcha - so perhaps I should ensure FeatureUnion[n_jobs] + GirdSearch[n_jobs] < # cores?
On Fri, Jun 17, 2016 at 11:21 AM, Gael Varoquaux <gael.varoquaux@normalesup.org
wrote:
> I am typically careful with this, e.g., if my machine has 16 cores, I’d > set feature union to n_jobs=3 and the gridsearch_cv to n_jobs=4 or so. > Curious to hear what the scikit devs think about nesting calls > n_jobs=-1; am I too conservative?
Nested parallelism doesn't work. It's a limitation of multiprocessing. _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
-- Gael Varoquaux Researcher, INRIA Parietal NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France Phone: ++ 33-1-69-08-79-68 http://gael-varoquaux.info http://twitter.com/GaelVaroquaux
I think
FeatureUnion[n_jobs=1] + GirdSearch[n_jobs <= cores]
would be better regarding the nested parallelism limitation
On Jun 17, 2016, at 11:46 AM, Philip Tully <tully@csc.kth.se> wrote:
Gotcha - so perhaps I should ensure FeatureUnion[n_jobs] + GirdSearch[n_jobs] < # cores?
On Fri, Jun 17, 2016 at 11:21 AM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
I am typically careful with this, e.g., if my machine has 16 cores, I’d set feature union to n_jobs=3 and the gridsearch_cv to n_jobs=4 or so. Curious to hear what the scikit devs think about nesting calls n_jobs=-1; am I too conservative?
Nested parallelism doesn't work. It's a limitation of multiprocessing. _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
Can confirm FeatureUnion[n_jobs=1] solved my issue, fwiw Thanks for prompt replies On Fri, Jun 17, 2016 at 11:52 AM, Sebastian Raschka < mail@sebastianraschka.com> wrote:
I think
FeatureUnion[n_jobs=1] + GirdSearch[n_jobs <= cores]
would be better regarding the nested parallelism limitation
On Jun 17, 2016, at 11:46 AM, Philip Tully <tully@csc.kth.se> wrote:
Gotcha - so perhaps I should ensure FeatureUnion[n_jobs] + GirdSearch[n_jobs] < # cores?
On Fri, Jun 17, 2016 at 11:21 AM, Gael Varoquaux < gael.varoquaux@normalesup.org> wrote:
I am typically careful with this, e.g., if my machine has 16 cores, I’d set feature union to n_jobs=3 and the gridsearch_cv to n_jobs=4 or so. Curious to hear what the scikit devs think about nesting calls n_jobs=-1; am I too conservative?
Nested parallelism doesn't work. It's a limitation of multiprocessing. _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
participants (3)
-
Gael Varoquaux -
Philip Tully -
Sebastian Raschka