[IPython-dev] ipython1 and synchronous printing of stdout

[Fernando Perez]

I realize that Barry and Brian will be going offline, and I'll be
traveling tomorrow back home as well, so this is mostly for the
conversation to continue later, but while the ideas are in my head.

I greatly appreciate that everyone is trying to wrap their heads
around a rather abstract and difficult question, but I do worry a bit
that it's getting too far for me to make a solid judgment on which
road will really work better.  We all agree on the basics we want
(loose coupling, acceptable performance, etc), but we're trying to
figure out a flexible architecture that lets us implement things we
need:

- across gui toolkits

- out of process (which more or less implies also in a
network-independent way, since something that can be communicated out
of process can be sent over the network,  given that we're not using
shared memory tricks).

The question is how to get from here to there with the least amount of
pain and keeping us all one happy family :)  I'd really prefer it if
we didn't end up with a fork just because the Wx/traits frontend gets
bogged down in a heavy and over-abstracted notification API, so I hope
we can figure out a viable solution for the lot.


On Thu, Jul 24, 2008 at 1:23 PM, Barry Wark <barrywark at gmail.com> wrote:

> As Brian points out below, delegation can lead to some unexpected
> dependencies. Let me give one other example: suppose we use subclasses
> of parameterized objects to add behavior to the interpreter.
[...snip careful description of inheritance issues...]

Note that the problems you point out here are simply the generic
issues that exist with subclassing with Python: any time you use a
subclass in python, you have to worry about the parent class in the
future growing new attribute names that may collide with your own.
That's actually a flaw of the Python object model that is rarely
discussed, and which languages like C++ in fact do not have.

But this particular issue is a general OO/Python one, that has nothing
to do with a notification pattern or event handling.

[Brian]

>> The real benefit of the observer/notification model is that everything
>> can be completely transient.  For example, you might have a notifier
>> that observes the user's namespace.  But you might only want it to
>> operate when the user opens a specific window.  Another example is
>> when an observer is hooked up to an unreliable network connection.  So
>> I guess I would ad another design constraint that I have until now
>> kept in the back of my mind:  we need loose coupling that is also
>> dynamic and transient.  And for these situations, I still think the
>> observer pattern is the best solution.
>
> Just to reiterate and paraphrase Brian, the observer pattern lets the
> Interpreter not know *anything* about the frontend (even whether it
> exists or not) that is observing its behavior and it lets the frontend
> not know *anything* about the Interpreter's implementation except that
> it will fire notifications for the events defined in its interface.
> So, when a new event is added to the Interpreter, no observer code
> needs to be modified (it doesn't even need to know that the new
> notification is fired). If the Interpreter's implementation is changes
> so that, e.g. notification of writing to stdout happens immediately
> after writing or after some short delay, none of the observing code
> needs to be reviewed as long as the original Interpreter API didn't
> specify the exact time. In other words, using an intermediary
> notification center enforces a loose coupling; the only dependency
> between Interpreter and frontend is the API that defines the
> event/notificaiton itself.

The part that worries me is building YAEHA (Yet Another Event Handling
API)... Matplotlib did exactly that, and while it does allow you to
wire events using pure matplotlib code, it adds real, non-trivial
complexity to the code and makes it harder to fit into a specific
event handling API 'from the front'.  All GUI systems have their own
event models, so does Twisted, and I'm really not convinced that
IPython should grow its own (next to matplotlib's, and Traits', and
WX's, ...)

On the other hand, I think this is a discussion that is beginning to
get far enough into design abstractions that it's escaping my
(admittedly very limited) ability to gauge fruitfully the real world
outcomes.  So here's a concrete proposal:  Gael is already hard at
work (and far along) on a WX frontend that Enthought is funding for
use in Envisage apps.  Since this is real, working code, let's see
where it goes.  In the meantime, Barry can play with the design for a
lightweight notification center idea, and actual implementation will
likely be very enlightening.  I've got my hands more than full trying
to get at least a first cut of all the tests that appeared (once I
completed the testing plugin, all sorts of little things got picked up
as tests that now need to actually work).  Since we're trying to push
a release before scipy, this is really more than we can do all at the
same time.

With Gael's and Barry's implementation on hand,  I think it will be
easier to see if the notification center can be made lightweight
enough for adapting it to Traits/Twisted/Cocoa/whatever without undue
pain,  or if an object-by-object approach ends up being better.  I can
easily see it being both: object parametrization for certain kinds of
architectural customization with a little message center for
lightweight notifications.

But honestly, I'm finding it hard to see far enough into the fog with
pure 'whiteboard' thinking.  We always brag about how nice it is to
use python to actually implement ideas rather than only think about
them in the abstract, I think in this case this may be the wisest
approach.


> [1] Re: performance
> I believe that Fernando's "no-op" parameter must incur *some*
> overhead; either the Interpreter must test for None, or the
> Interpreter must call a method in Fernando's object which does
> nothing. In other words, there's no free lunch. I can't think of a way
> to add the ability to notify observers and/or modify the behavior of
> Interpreter via delegates with truely zero overhead while avoiding the
> parallel implementation problem that I describe above.

No, the None check is only done at isntantiation time of the main
objects.  Afterwards, it's the responsibility of the  subclasses to
produce/manage events for specific behavior.  The default object will
for example have a .write() method that simply prints, while an event
handling subclass would have its .write() be Traited so that a GUI
could listen in, or a Twisted one would handle it over the network.
So it really is zero-overhead, simply because in the default, there is
no event production/consumption at all.

So from a performance perspective, that approach really produces
exactly zero cost.  The question that I think will be best answered
with a bit of prototyping, is whether the architectural costs become
greater than those of a message center.

Gael just emphasized that in the code he's doing,  if it turns out the
message center approach works well, there's little cost in adding it
later on.  So it's really not an either/or issue, just one of getting
functioning prototypes  working  quickly so we can judge based on the
actual code rather than pure abstractions.

Does this sound like a reasonable plan?

Cheers,

f