[IPython-dev] newparallel

[Fernando Perez]

On Wed, Feb 9, 2011 at 2:13 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> On Wed, Feb 09, 2011 at 12:15:04PM -0800, Fernando Perez wrote:
>> In situation 1, it's probably not of much value to have ipython
>> around, except for the case where you might want to debug a
>> problematic execution.
>
> IMHO that's a hugely important point. The number one reason why I don't
> do much parallel work is that a significant amount of my time on research
> code is spent debuging. So I would say: please don't underestimate the
> fantastic contribution that you have been making in this direction with
> IPython.

Thanks for the comments, Gael.  Indeed that was very much my take on
it, but I know you and I have similar workstyles and we really value
interactive access to 'hands-on' code and data.  I was just trying to
be somewhat neutral towards the more classic grid computing approaches
that toss interactivity overboard.

But you are right that we shouldn't shy away from highlighting the
value of this.  And it's clear to me that with a bit of api help on
our side, we can make it trivial and zero-overhead for batch-oriented
systems to offer the capability to do:

try:
  normal_execution()
except:
  open_ipython_client()

This would not even invoke ipython if things go OK, but offer the
ability to debug a problematic node interactively whenever there's a
problem (which is more often than the grid computing literature wants
you to believe ;)

So thanks for coming back to that point, it's more important than I
sometimes give it credit in a misguided attempt at moderating my
ipython/interactive bias :)

Best,

f