[Neuroimaging] [nilearn] Running nilearn on kubernetes cluster.

[Roberto Guidotti]

Hi all,

I tried to run nilearn on a kubernetes cluster with dask distributed.
Here's the snippet.

from dask_kubernetes import KubeCluster
from dask.distributed import Client
cluster = KubeCluster.from_yaml('pods.yaml')
cluster.scale(3)
client = Client(cluster.scheduler_address)

Then I ran the plot_haxby_searchlight.py example, using following line for
fitting.

from sklearn.externals import joblib
with joblib.parallel_backend('dask'):
    searchlight.fit(fmri_img, y)

[Parallel(n_jobs=-1)]: Using backend DaskDistributedBackend with 6
concurrent workers.
tornado.application - ERROR - Exception in callback
functools.partial(<function wrap.<locals>.null_wrapper at 0x7f886a7cd730>,
<Future finished
exception=CancelledError(['_group_iter_search_light-batch-18a44db2f247402da4955d757c513610'],)>)
...

CancelledError: _fit_and_score-batch-cb239abe1e714386b5a5607c78d927e8

I can run sklearn examples with dask without any problem, but with nilearn
I had this.

On the kubernetes cluster, I use a docker image that is based on this repo:
https://github.com/robbisg/docker-dask-k8s

Basically I need to run some nilearn analyses using HPC, is this the less
painful way to do this or you suggest something else.

Thank you,
Roberto



-- 
Ing. Roberto Guidotti, PhD.
PostDoc Fellow
Institute for Advanced Biomedical Technologies - ITAB
Department of Neuroscience and Imaging
University of Chieti "G. D'Annunzio"
Via dei Vestini, 33
66013 Chieti, Italy
tel: +39 0871 3556919
e-mail: r.guidotti at unich.it; rguidotti at acm.org
linkedin: http://it.linkedin.com/in/robertogui/
twitter: @robbisg
github: https://github.com/robbisg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/neuroimaging/attachments/20190215/06a9bbac/attachment.html>