[IPython-dev] roadmap for IPython.zmq.parallel

Barry Wark barrywark at gmail.com
Fri Jan 7 09:55:21 EST 2011


On Tue, Dec 21, 2010 at 2:04 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hey Barry,
>
> On Fri, Dec 17, 2010 at 9:15 PM, Barry Wark <barrywark at gmail.com> wrote:
>> It's been too long since I've been able to hang out in IPython land.
>> Given my previous interests, it's really exciting to see the work in
>> frontends accelerating with the new refactoring.
>
> Glad to have you around again!
>
>> I'm very excited to have a new opportunity to get back to IPython work
>> on a client project. The contract is to build a scientific data
>> processing and analysis framework. The analyses are expressed as a
>> DAG, with computation at the nodes done by exectuables that take a
>> standardized set of arguments and return a contracted output format.
>> Some of the executables are C, some Matlab, some Python, etc--standard
>> fare in academia. Our job is to build the engine to execute these
>> workflows, monitor results, etc. Jobs will initially execute on a
>> single machine (thus multiprocessing or a higher-level framework like
>> Rufus, http://www.ruffus.org.uk/) make sense, but the user may
>> eventually want to expand onto a local cluster.
>
> I'm still in India and will be offline as of tomorrow (traveling
> back)...   But I'd suggest you have at least a look at:
>
> http://nipy.sourceforge.net/nipype/
>
> Nipype is Satra's brainchild (the same Satra who has committed the
> recent work on ipython) and it already has support for IPython's
> parallel execution using the 0.10.x code.  It was also the motivation
> behind some of the new DAG support, as we hope in the future to have
> even better integration between nipype and ipython.

Yes, Nipype FTW! Awesome framework Satra!

>
> Satra is also in India right now (we were at the same conference) but
> he's on holiday for a fe days with his family and likely offline, so I
> expect him to pitch in only a little bit later.  But I hope that in a
> few days when people's travel schedules normalize, we can see what can
> be done to benefit from common goals so that we reuse as much of the
> effort and manpower as possible.
>
> All the best,
>
> f
>



More information about the IPython-dev mailing list