[IPython-dev] frontend plans

Brian Granger ellisonbg.net at gmail.com
Thu Jun 26 23:31:19 EDT 2008

On Thu, Jun 26, 2008 at 9:18 PM, Barry Wark <barrywark at gmail.com> wrote:
> On Thu, Jun 26, 2008 at 4:03 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
>> Barry,
>>>> I think that we could have a single base class that implements the
>>>> actual frontend logic and then have multiple subclasses that handle
>>>> the calls to the various types of backends (kernel.core or
>>>> kernel.engineservice).  I need to look more at how the frontend is
>>>> implemented, but Barry, do you think this would work.
>>> Yes, I think this would work. I thought zope.interface was pure
>>> python. Since we can't require zope.interface in the stripped-down
>>> ipython, I now think option 1 is the best approach. I'll work on that.
>> Great, I think that will be best, even though it has its own downsides.
> We're in agreement then. I'll try option 1 and see how that shakes out.
>>>> Also, the Synchronous deferred does not present the exact same
>>>> interface as a true deferred or make the same promises about when
>>>> things will happen.
>>> What is your understanding of the differences?
>> For Deferreds, I think the interface is not a complete specification
>> of the objects behavior.  For Deferreds, there are subtle cases that
>> arise when you chain them together.  The behavior exhibited in complex
>> Deferred chaining _can_ depend on whether or a given Deferred has
>> fired or not when the chaining is setup.  We have in various places
>> that handles these odd edge cases.  Also, we sometimes access the
>> semi-private attributes of Deferreds.  Another thing is that if you
>> have multiple unfired Deferreds, the order in which they will fire is
>> not deterministic.  For the synchronous deferreds, I think the
>> ordering has to be deterministic.  But in my mind, this constitues a
>> different behavior and thus "interface" in the board sense of the
>> word.
>> We might be able to get the synchronous deferreds to work, but I am a
>> little hesitant to go down that route simply because all of these
>> things are _super_ subtle and nearly impossible to debug.
> I see the issues. Too bad; the Twisted universe seems to want to
> swallow its victims whole. I should have really seen this coming--I've
> worked on a Cocoa framework that uses OS X's NSOperationQueue to
> emulate the Twisted reactor/Deferred system. You're exactly right that
> the Deferred behavior is _very_ complicated when you dig into it. Oh,
> well. As long as we're willing to skip the engineservice layer for
> synchronous interface to core, it's not an issue and we can forget
> about Synchronous Deferreds.
>>> Thus, I my mind, the new IEngineCore
>>>> implementation _couldn't_ possible really implement the interface.  In
>>>> fact, if I remember correctly, we have a test that can be run on any
>>>> IEngineCore implementer.  It actually calls all the methods of the
>>>> interface and checks:
>>>> self.assert_(isinstance(return_value, t.i.Deferred)
>>>> A fake deferred won't pass such tests.
>>> I don't think this is an appropriate test then. The interface should
>>> specify the behavior of the returned object, not its implementation.
>>> If the Synchronous Deferred behaves the same as a t.i.defer.Deferred,
>>> then it should pass the test. The fact that it may not behave the same
>>> is a bigger issue.
>> True, I could write a better test for this.  The current test is only
>> a super weak test behavior wise, but given the fact that there are no
>> other objects that truly act like a Deferred, the test suffices.
>>>> So, in summary, I think more discussion is needed before commiting to
>>>> #2.  I am crazy busy, but I will try to have a look at IFrontEnd
>>>> tonight.
>>>>> Since the IEngine* interface is much more stable than the IFrontEnd
>>>>> interface, I propose going with solution #2. Although it adds an
>>>>> additional python file as external dependency, it seems conceptually
>>>>> cleaner (all frontends go through the engine interface) and allows
>>>>> frontends to decide between synchronous and asynchronous behavior
>>>>> without any code changes.
>>>> I think it is conceptually convoluted because the only significant way
>>>> that IEngireCore different from the underlying core is that its
>>>> methods return deferreds (and Failures).  Thus to use IEngineCore, but
>>>> then try to get rid of the (true) Deferreds by using fake ones seems
>>>> like double work (why not just call the class whose methods don't
>>>> return deferreds in the first place).
>>> It feels weird that there are two ways to interact with ipython—engine
>>> and core directly. It seemed that making a common interface was
>>> cleaner in that it allowed users of that interface to be able to use
>>> engine or core without modification of the frontend. Since that's not
>>> really an option anymore (because of zope.interface), I'm happy to
>>> give up that opinion and move on.
>> I do sort of agree with this, but I think having two interfaces is
>> appropriate because they have vastly different behaviors (synchronous
>> vs asynchronous).
> Fair enough. I'm convinced. I had it wrong in my head-- I.kernel.core
> is still the 'real' IPython. I.kernel is just a Twised wrapper around
> that core. Naturally anything that doesn't want what Twisted has to
> offer should talk directly with I.kernel.core and the price they pay
> is that they loose all the goodies that come with the I.kernel layer.
> Now that I just wrote this out, it seems wrong that I.kernel.core is a
> subpackage of I.kernel. I know that it's there to isolate the ipython1
> stuff from ipython0 stuff, but before too many people start writing
> code using I.kernel.core, is it worth discussing if there's a better
> spot for it in the IPython tree?

Yes, probably.  I had originally thoughts about moving it to
IPython.core.  But the problem with that is I am afraid that it
suggests that it is a complete and working core.  My plan originally
was thus:

1.  Move the old core IPython.*.py -> IPython.core.*.py

2.  Refactor that stuff until it looks more like IPython.kernel.core

3.  At that point, get rid of IPython.kernel.core

But maybe the better approach is:

1.  Just move IPython.kernel.core -> IPython.core

2.  Also move IPython.*.py -< IPython.core

3.  Refactor/combine the two inplace

What do you think?  This probably needs more disucssion in a separate
thread on the list.



>>>> Actually, I think we are in agreement that tab completion (the part
>>>> that actually looks up things in the users namespace) needs to be done
>>>> in the engine/core through its complete method.  There is just no way
>>>> it is reasonable to mirror the user_ns in the frontend for the reasons
>>>> you mention.  Sorry if we have said anything confusion on this front.
>>>> So I guess the complete method of the frontend should just call the
>>>> complete method of the engine/core?
>>> Yes. I agree. frontend.complete() calls engine/core.complete().
>> Cool,
>> Cheers,
>> Brian
>>>> Cheers,
>>>> Brian
>>>>> Barry
>>>>> [1] Removing the Twisted dependence completely will require moving the
>>>>> IEngine* interfaces to a separate module from the implementations. I
>>>>> propose moving them to IPython.kernel.engineinterface.py.
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev

More information about the IPython-dev mailing list