[IPython-dev] ipcluster (LSF) timing (check if all engines are running)
Florian M. Wagner
wagnerfl at student.ethz.ch
Mon Aug 19 09:34:48 EDT 2013
Hey all,
I am using IPython.parallel on a large cluster, where controller and
engines are launched via LSF. My current workflow is as follows:
#!/bin/bash
python pre_processing.py
ipcluster start --profile=cluster --n=128 > ipcluster.log 2>&1
sleep 120
python main_computation.py
python post_processing.py
I am not entirely happy with this, since the 2 minutes are not always
enough depending on the load of the cluster. I believe that there is a
much more elegant way to launch the cluster and check if all the eninges
are running, before proceeding with the main computation. I would highly
appreciate any help.
Best regards
Florian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130819/681be7be/attachment.html>
More information about the IPython-dev
mailing list