[IPython-dev] Justin, Satra, everyone: wrapping up 0.10.1?

[Satrajit Ghosh]

hi fernando,

the way justin set it up is that PBS/SGE/LSF obviates the need for writing a
special script. however, there is an option to provide your own script if
you want. we can augment the docs with a mini-example for your own script.

recently i realized that i would love if the ssh option for ipcluster had an
option to use my current environment similar to the way SGE/PBS does. it's
not useful when you are talking cross-platform but very useful if you are on
the same platform. part of this is motivated by the fact that my virtual
environment does not get activated over ssh (the bashrc doesn't get
executed). what do you guys think? i might have some time tonight - should i
take a crack at it?

cheers,

satra


On Mon, Sep 20, 2010 at 8:17 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hey guys,
>
> On Fri, Sep 17, 2010 at 5:52 PM, Satrajit Ghosh <satra at mit.edu> wrote:
> >
> > you can merge the changes from justin's 0.10.1-sge branch. that contains
> the
> > sge+lsf support fixes. other than that i think you already have min's
> taskid
> > fix for the controller (i.e the clear function now accepts a task id).
> >
>
> Great, thanks for the info.  One quick question, in the diff I see
> this towards the end:
>
> -The PBS mode uses the Portable Batch System [PBS]_ to start the engines.
>  To us
> e this mode, you first need to create a PBS script template that will be
> used to
>  start the engines.  Here is a sample PBS script template:
> +The PBS mode uses the Portable Batch System [PBS]_ to start the engines.
>
> -.. sourcecode:: bash
> +To start an ipcluster using the Portable Batch System::
> +
> +    $ ipcluster pbs -n 12
>
> -    #PBS -N ipython
> -    #PBS -j oe
> -    #PBS -l walltime=00:10:00
> -    #PBS -l nodes=${n/4}:ppn=4
> -    #PBS -q parallel
> +The above command will launch a PBS job array with 12 tasks using the
> default q
> ueue. If you would like to submit the job to a different queue use the -q
> option
> :
>
> -    cd $$PBS_O_WORKDIR
> -    export PATH=$$HOME/usr/local/bin
> -    export PYTHONPATH=$$HOME/usr/local/lib/python2.4/site-packages
> -    /usr/local/bin/mpiexec -n ${n} ipengine
> --logfile=$$PBS_O_WORKDIR/ipengine
> +    $ ipcluster pbs -n 12 -q hpcqueue
>
>
> Is it correct to remove all references to the #PBS lines?  Do  your
> improvements to PBS obsolete completely the need for the manual script
> writing we had before?
>
> I just want to make sure that I understand that correctly before I
> merge it, once we clarify this I'll merge and will cut an RC so we can
> get it tested.
>
> Cheers,
>
> f
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100921/0f2b05c2/attachment.html>