[IPython-dev] roadmap for IPython.zmq.parallel

Satrajit Ghosh satra at mit.edu
Sat Jan 8 18:43:57 EST 2011


hi barry,

thanks. sorry for the delay, i'm just getting my head back into things after
a nice long break. it will be awesome to work together and improve the
framework. after all any workflow engine slowly becomes a meta-language!
also, ipython, will have some nice features that should reduce the nipype
code and make for a cleaner implementation. also chris at enthought is
converting traits to pure python and that should clean up the engine code as
well. we currently have some work-arounds for a couple of dynamic traits
bugs.

do send me some more details of what you want to do. (based on what you have
below, you should find our base classes rather appropriate).

cheers,

satra


On Fri, Jan 7, 2011 at 9:55 AM, Barry Wark <barrywark at gmail.com> wrote:

> On Tue, Dec 21, 2010 at 2:04 AM, Fernando Perez <fperez.net at gmail.com>
> wrote:
> > Hey Barry,
> >
> > On Fri, Dec 17, 2010 at 9:15 PM, Barry Wark <barrywark at gmail.com> wrote:
> >> It's been too long since I've been able to hang out in IPython land.
> >> Given my previous interests, it's really exciting to see the work in
> >> frontends accelerating with the new refactoring.
> >
> > Glad to have you around again!
> >
> >> I'm very excited to have a new opportunity to get back to IPython work
> >> on a client project. The contract is to build a scientific data
> >> processing and analysis framework. The analyses are expressed as a
> >> DAG, with computation at the nodes done by exectuables that take a
> >> standardized set of arguments and return a contracted output format.
> >> Some of the executables are C, some Matlab, some Python, etc--standard
> >> fare in academia. Our job is to build the engine to execute these
> >> workflows, monitor results, etc. Jobs will initially execute on a
> >> single machine (thus multiprocessing or a higher-level framework like
> >> Rufus, http://www.ruffus.org.uk/) make sense, but the user may
> >> eventually want to expand onto a local cluster.
> >
> > I'm still in India and will be offline as of tomorrow (traveling
> > back)...   But I'd suggest you have at least a look at:
> >
> > http://nipy.sourceforge.net/nipype/
> >
> > Nipype is Satra's brainchild (the same Satra who has committed the
> > recent work on ipython) and it already has support for IPython's
> > parallel execution using the 0.10.x code.  It was also the motivation
> > behind some of the new DAG support, as we hope in the future to have
> > even better integration between nipype and ipython.
>
> Yes, Nipype FTW! Awesome framework Satra!
>
> >
> > Satra is also in India right now (we were at the same conference) but
> > he's on holiday for a fe days with his family and likely offline, so I
> > expect him to pitch in only a little bit later.  But I hope that in a
> > few days when people's travel schedules normalize, we can see what can
> > be done to benefit from common goals so that we reuse as much of the
> > effort and manpower as possible.
> >
> > All the best,
> >
> > f
> >
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110108/179f06df/attachment.html>


More information about the IPython-dev mailing list