[Neuroimaging] [nilearn] Running nilearn on kubernetes cluster.

[bthirion]

Indeed this seems only indirectly related to Nilearn. Let's wait for 
@ogrisel's reponse.
Best,
B

On 16/02/2019 11:45, Roberto Guidotti wrote:
> Hi,
>
> I'm writing since I performed some other attempts to figure out the 
> issue, using scikit-learn and it turned out that may be an issue with 
> cross_* functions since I had also a similar problem with a 
> scikit-learn script without any nilearn's line of code as reported 
> here https://github.com/dask/distributed/issues/2532.
>
> Bests,
> R
>
> On Fri, 15 Feb 2019 at 13:11, Roberto Guidotti <robbenson18 at gmail.com 
> <mailto:robbenson18 at gmail.com>> wrote:
>
>     Hi all,
>
>     I tried to run nilearn on a kubernetes cluster with dask distributed.
>     Here's the snippet.
>
>     from dask_kubernetes import KubeCluster
>     from dask.distributed import Client
>     cluster = KubeCluster.from_yaml('pods.yaml')
>     cluster.scale(3)
>     client = Client(cluster.scheduler_address)
>
>     Then I ran the plot_haxby_searchlight.py example, using following
>     line for fitting.
>
>     from sklearn.externals import joblib
>     with joblib.parallel_backend('dask'):
>         searchlight.fit(fmri_img, y)
>
>     [Parallel(n_jobs=-1)]: Using backend DaskDistributedBackend with 6
>     concurrent workers.
>     tornado.application - ERROR - Exception in callback
>     functools.partial(<function wrap.<locals>.null_wrapper at
>     0x7f886a7cd730>, <Future finished
>     exception=CancelledError(['_group_iter_search_light-batch-18a44db2f247402da4955d757c513610'],)>)
>     ...
>
>     CancelledError: _fit_and_score-batch-cb239abe1e714386b5a5607c78d927e8
>
>     I can run sklearn examples with dask without any problem, but with
>     nilearn I had this.
>
>     On the kubernetes cluster, I use a docker image that is based on
>     this repo:
>     https://github.com/robbisg/docker-dask-k8s
>
>     Basically I need to run some nilearn analyses using HPC, is this
>     the less painful way to do this or you suggest something else.
>
>     Thank you,
>     Roberto
>
>
>
>     -- 
>     Ing. Roberto Guidotti, PhD.
>     PostDoc Fellow
>     Institute for Advanced Biomedical Technologies - ITAB
>     Department of Neuroscience and Imaging
>     University of Chieti "G. D'Annunzio"
>     Via dei Vestini, 33
>     66013 Chieti, Italy
>     tel: +39 0871 3556919
>     e-mail: r.guidotti at unich.it <mailto:r.guidotti at unich.it>;
>     rguidotti at acm.org <mailto:rguidotti at acm.org>
>     linkedin: http://it.linkedin.com/in/robertogui/
>     twitter: @robbisg
>     github: https://github.com/robbisg
>
>
>
> -- 
> Ing. Roberto Guidotti, PhD.
> PostDoc Fellow
> Institute for Advanced Biomedical Technologies - ITAB
> Department of Neuroscience and Imaging
> University of Chieti "G. D'Annunzio"
> Via dei Vestini, 33
> 66013 Chieti, Italy
> tel: +39 0871 3556919
> e-mail: r.guidotti at unich.it <mailto:r.guidotti at unich.it>; 
> rguidotti at acm.org <mailto:rguidotti at acm.org>
> linkedin: http://it.linkedin.com/in/robertogui/
> twitter: @robbisg
> github: https://github.com/robbisg
>
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/neuroimaging/attachments/20190217/6eba3a14/attachment.html>