From barrywark at gmail.com  Tue Jul  1 01:11:58 2008
From: barrywark at gmail.com (Barry Wark)
Date: Mon, 30 Jun 2008 22:11:58 -0700
Subject: [IPython-dev] frontend plans
In-Reply-To: <db6b5ecc0806301737k7d5c0cafo5395a9fc4c933935@mail.gmail.com>
References: <cd7634ce0806251133i305d0b1dsccf083e3c64fcf73@mail.gmail.com>
	<6ce0ac130806251559s7350cc6csd1ce6ab65b7510f6@mail.gmail.com>
	<cd7634ce0806261354y7d6b228id4f17c04e9921e36@mail.gmail.com>
	<6ce0ac130806261603m24868dedh26cd9243ff389b20@mail.gmail.com>
	<cd7634ce0806262018g2deae150w9f39d7db82e55eb1@mail.gmail.com>
	<6ce0ac130806262031l191304d9xbd49100ab09def8@mail.gmail.com>
	<db6b5ecc0806301737k7d5c0cafo5395a9fc4c933935@mail.gmail.com>
Message-ID: <cd7634ce0806302211k3896931fk90eedf09e5d7304a@mail.gmail.com>

On Mon, Jun 30, 2008 at 5:37 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hey folks,
>
> I'm sorry that I failed to respond during this thread, but as it
> happened I was at a conference with only micro-breaks for 'easy'
> emails, but not to digest all of this :)
>
> Having said that, here are a few thoughts on  what has been said so far:
>
> - Gael: as Brian asked, what are your design parameters regarding GUI
> blocking?  If you put the exec calls in a thread (like ipython
> -Xthread does) you have some hope of interface responsiveness, if you
> put them in a process you get full GUI responsiveness at the cost of
> other complexities.  Are you OK with a non-responsive GUI while exec()
> is busy?
>
> - Zope interfaces: what do they exactly bring us?  Nose uses
> interfaces in a non-enforcing way by making them pure python classes
> that are meant to document behavior but *not* to be subclassed.
> Perhaps we could have something similar for the pure python version
> and then a ZI version for the rest of the twisted layer:
>
> class BasicInterface(object):
>  def foo(self,x,y):
>    """does z with x and y"""
>    raise NotImplementedError()
>
> class RealInterface(BasicInterface,zope.interfaces.whatever):
>  pass
>
> Code NOT using twisted would inherit from the first type of functions,
> and all twisted-based code would inherit from the second.

What about something like the following::

try:
    raise ImportError()
    from zope.interface import Interface, Attribute, implements, classProvides
except ImportError:
    def Attribute(name,doc): pass
    def implements(interface): pass
    def classProvides(interface): pass

class IExample(Interface):
    Attribute('name', 'Attribute name __doc__')

    def example_method():
        """example_method"""

        pass


class Implementor(object):
    """Implements  the IExample interface."""

    implements(IExample)

    def example_method(self):
        """example_method"""

        print self

The only thing you don't get is being able to check that Implementor
implements IExample if zope.interface isn't present.

>
> - basic observation: exec() is fundamentally a blocking primitive.  At
> the end of the day, you have to wait for that particular chunk of code
> to finish, and that's the python interpreter itself you're waiting
> for.  For this reason, it makes sense that the lowest level
> abstractions we have should be blocking, and asynchronous interfaces
> are wrapped around those for systems that are 'removed' from this
> execution core (by being out of process, in another computer, etc).
> Yes, I know that exec() could be calling threaded code, but that
> simply means that exec(x) can finish quickly and some of the results
> will be ready later.  But the overall operation of completing the
> execution of 'x' is still a blocking one.
>
> - Because of the above, I'm not crazy about the whole "synchronous
> deferred" use.  In my mind, the logical containment chain is:
>
> (0 - python VM - fundamentally synchronous object) < (1 - Ipython
> layer that manages this) < (2 - Asynchronous wrappers for systems that
> will work out of process, over the network, etc).

I completely agree. The previous discussion convinced me this is the
right way to go.

>
>
> - Tab completion: this is just one case of the more generic case of
> how to represent out-of-process information.  We have to assume that
> in general, only the core has true access to the real in-memory
> objects of the user's namespace.  Clients may request information
> about this for display purposes, and some communication of reduced
> data may happen with such clients, but we can't expect to  copy the
> user's namespace across the wire in a general sense.  So yes, tab
> completion and similar introspection will always happen by clients
> requesting the operation on the core, and the core sending back
> something reasonable back (a list of strings, for example).

Agreed. I think the statement "clients may request information about
this for display purposes" is a somewhat complicated idea that will
require some more work.

>
>
> In summary:  it seems from what Barry said in the end, as well as
> Gael, that we're all happy with the notion of a core, blocking system
> that ultimately is just a bells-and-whistles version of python's "exec
> code in namespace" statement.  It lets you manage that namespace,
> introspect it, it offers extensions, history, etc, but ultimately it's
> just wrapping that one single statement.  Because that statement
> *fundamentally* blocks, this system blocks.  You can embed it in a GUI
> or use it in a terminal, but it still blocks.
>
> Beyond that, it makes sense to wrap this in a Twisted layer that turns
> the result of exec() into a deferred.  This makes complete sense when
> the process doing exec() is different than your own (gui, network,
> etc) and you actively want to move on with your life, while having a
> notification mechanism for handling results/errors arising from the
> exec call.  At this point, the Twisted callback/errback system seems
> well tailored for this.
>
> And I certainly want to ensure that Gael finds the code that's in
> there sufficient for him to use in his WX project.   It seems to me
> that's the case, but have we left any question unanswered on that
> front?

There aren't any immediate questions that I'm aware of.

-Barry

>
> Cheers,
>
> f
>


From gael.varoquaux at normalesup.org  Tue Jul  1 01:27:59 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 1 Jul 2008 07:27:59 +0200
Subject: [IPython-dev] frontend plans
In-Reply-To: <db6b5ecc0806301737k7d5c0cafo5395a9fc4c933935@mail.gmail.com>
References: <cd7634ce0806251133i305d0b1dsccf083e3c64fcf73@mail.gmail.com>
	<6ce0ac130806251559s7350cc6csd1ce6ab65b7510f6@mail.gmail.com>
	<cd7634ce0806261354y7d6b228id4f17c04e9921e36@mail.gmail.com>
	<6ce0ac130806261603m24868dedh26cd9243ff389b20@mail.gmail.com>
	<cd7634ce0806262018g2deae150w9f39d7db82e55eb1@mail.gmail.com>
	<6ce0ac130806262031l191304d9xbd49100ab09def8@mail.gmail.com>
	<db6b5ecc0806301737k7d5c0cafo5395a9fc4c933935@mail.gmail.com>
Message-ID: <20080701052759.GA4509@phare.normalesup.org>

On Mon, Jun 30, 2008 at 05:37:44PM -0700, Fernando Perez wrote:
> - Gael: as Brian asked, what are your design parameters regarding GUI
> blocking?  If you put the exec calls in a thread (like ipython
> -Xthread does) you have some hope of interface responsiveness, if you
> put them in a process you get full GUI responsiveness at the cost of
> other complexities.  Are you OK with a non-responsive GUI while exec()
> is busy?

I thought I had laid this out clearly: yes I am OK with that.

> - Zope interfaces: what do they exactly bring us?  Nose uses
> interfaces in a non-enforcing way by making them pure python classes
> that are meant to document behavior but *not* to be subclassed.
> Perhaps we could have something similar for the pure python version
> and then a ZI version for the rest of the twisted layer:

I am fine with Zope interfaces, we can imbed the files.

> In summary:  it seems from what Barry said in the end, as well as
> Gael, that we're all happy with the notion of a core, blocking system
> that ultimately is just a bells-and-whistles version of python's "exec
> code in namespace" statement.  It lets you manage that namespace,
> introspect it, it offers extensions, history, etc, but ultimately it's
> just wrapping that one single statement.  Because that statement
> *fundamentally* blocks, this system blocks.  You can embed it in a GUI
> or use it in a terminal, but it still blocks.

> Beyond that, it makes sense to wrap this in a Twisted layer that turns
> the result of exec() into a deferred.  This makes complete sense when
> the process doing exec() is different than your own (gui, network,
> etc) and you actively want to move on with your life, while having a
> notification mechanism for handling results/errors arising from the
> exec call.  At this point, the Twisted callback/errback system seems
> well tailored for this.

> And I certainly want to ensure that Gael finds the code that's in
> there sufficient for him to use in his WX project.   It seems to me
> that's the case, but have we left any question unanswered on that
> front?

I am fine with all you said up here. Your summary of the situation is
good, and I am happy with the way things turn out. I'll be on Mayavi
sprint this week, so pretty much off line for this discussin, sorry, but
I like the way everything sounds.

Thanks guys,

Ga?l


From fperez.net at gmail.com  Tue Jul  1 03:12:29 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 1 Jul 2008 00:12:29 -0700
Subject: [IPython-dev] frontend plans
In-Reply-To: <20080701052759.GA4509@phare.normalesup.org>
References: <cd7634ce0806251133i305d0b1dsccf083e3c64fcf73@mail.gmail.com>
	<6ce0ac130806251559s7350cc6csd1ce6ab65b7510f6@mail.gmail.com>
	<cd7634ce0806261354y7d6b228id4f17c04e9921e36@mail.gmail.com>
	<6ce0ac130806261603m24868dedh26cd9243ff389b20@mail.gmail.com>
	<cd7634ce0806262018g2deae150w9f39d7db82e55eb1@mail.gmail.com>
	<6ce0ac130806262031l191304d9xbd49100ab09def8@mail.gmail.com>
	<db6b5ecc0806301737k7d5c0cafo5395a9fc4c933935@mail.gmail.com>
	<20080701052759.GA4509@phare.normalesup.org>
Message-ID: <db6b5ecc0807010012j211ffd8bx818ab7792c1dbdc@mail.gmail.com>

On Mon, Jun 30, 2008 at 10:27 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> On Mon, Jun 30, 2008 at 05:37:44PM -0700, Fernando Perez wrote:
>> - Gael: as Brian asked, what are your design parameters regarding GUI
>> blocking?  If you put the exec calls in a thread (like ipython
>> -Xthread does) you have some hope of interface responsiveness, if you
>> put them in a process you get full GUI responsiveness at the cost of
>> other complexities.  Are you OK with a non-responsive GUI while exec()
>> is busy?
>
> I thought I had laid this out clearly: yes I am OK with that.

Thanks.  I was pretty much sure that's what you meant, but with all
the back and forth, there was a tiny bit of confusion left in my
brain, thanks for the clarification.

>> - Zope interfaces: what do they exactly bring us?  Nose uses
>> interfaces in a non-enforcing way by making them pure python classes
>> that are meant to document behavior but *not* to be subclassed.
>> Perhaps we could have something similar for the pure python version
>> and then a ZI version for the rest of the twisted layer:
>
> I am fine with Zope interfaces, we can imbed the files.

As Brian pointed out, they bring in a C dependency we'd rather avoid
if we can (and if it can be avoided with minimal cost).  Barry
outlined the solution with more detail than my flaky sketch, and I
don't see any problem with that approach, so I think we can use that.


> I am fine with all you said up here. Your summary of the situation is
> good, and I am happy with the way things turn out. I'll be on Mayavi
> sprint this week, so pretty much off line for this discussin, sorry, but
> I like the way everything sounds.
>
> Thanks guys,

Well, thanks to you for having the patience to sort all of this!  Good
luck with the Mayavi week, let us know how it all goes!  In the
meantime, we've got plenty to stay busy here :)

Cheers,

f


From jorgen.stenarson at bostream.nu  Tue Jul  1 16:26:13 2008
From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Tue, 01 Jul 2008 22:26:13 +0200
Subject: [IPython-dev] pyreadline refactor progess
Message-ID: <486A92E5.5010701@bostream.nu>

Hi,

the refactoring of pyreadline is progressing nicely. I have now moved 
all stuff that depends on the console out of the emacs mode. The vi mode 
still uses some console stuff and should not be used in non-console mode.

There is a very simplistic tk_gui.py example in the examples directory. 
The translation of tk key events to the event type used in pyreadline is 
not complete. Dead keys, keys using alt gr, as well as many 
non-alphanumeric keys are not translated. The example also has a very 
simple python prompt.
There also needs to be work on getting the cursor visible.

The code is available, using bzr, from: 
lp:~jorgen-stenarson/pyreadline/pyreadline-refactor

If you want to try it out note that an install will install a 
readline.py which may interfere with your regular readline.

/J?rgen




From ondrej at certik.cz  Wed Jul  2 09:38:37 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 2 Jul 2008 15:38:37 +0200
Subject: [IPython-dev] how to filter launchpad bugs
Message-ID: <85b5c3130807020638j3e73008fkd0435aed52d13de3@mail.gmail.com>

Hi,

is there a way to filter launchpad bugs? They are filling up my inbox,
but I would rather put them to the ipython folder or something. They
usually look like this:

[Bug 244854] [NEW] ipython.el doesn't work with optparse

so I wonder how to filter them. Some mailinglists have a header like this:

mailing list:	ipython-dev.scipy.org

or at least some footer, but these bug reports have nothing. Any ideas?

Ondrej


From fperez.net at gmail.com  Wed Jul  2 13:21:16 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 2 Jul 2008 10:21:16 -0700
Subject: [IPython-dev] how to filter launchpad bugs
In-Reply-To: <85b5c3130807020638j3e73008fkd0435aed52d13de3@mail.gmail.com>
References: <85b5c3130807020638j3e73008fkd0435aed52d13de3@mail.gmail.com>
Message-ID: <db6b5ecc0807021021l13f3f44by9069b81432c9f8f5@mail.gmail.com>

On Wed, Jul 2, 2008 at 6:38 AM, Ondrej Certik <ondrej at certik.cz> wrote:
> Hi,
>
> is there a way to filter launchpad bugs? They are filling up my inbox,
> but I would rather put them to the ipython folder or something. They
> usually look like this:
>
> [Bug 244854] [NEW] ipython.el doesn't work with optparse
>
> so I wonder how to filter them. Some mailinglists have a header like this:
>
> mailing list:   ipython-dev.scipy.org
>
> or at least some footer, but these bug reports have nothing. Any ideas?

The best I could do was to make a filter (in gmail) for "has the words
bugs.launchpad.net ipython", because they don't mark the headers with
list ids or anything sensible.  But that's working reasonably well for
me so far (though it could easily produce a false positive,
obviously).

Cheers,

f


From ondrej at certik.cz  Wed Jul  2 13:30:01 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 2 Jul 2008 19:30:01 +0200
Subject: [IPython-dev] how to filter launchpad bugs
In-Reply-To: <db6b5ecc0807021021l13f3f44by9069b81432c9f8f5@mail.gmail.com>
References: <85b5c3130807020638j3e73008fkd0435aed52d13de3@mail.gmail.com>
	<db6b5ecc0807021021l13f3f44by9069b81432c9f8f5@mail.gmail.com>
Message-ID: <85b5c3130807021030q4d0d2e14h5f63fa94d1997670@mail.gmail.com>

On Wed, Jul 2, 2008 at 7:21 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Wed, Jul 2, 2008 at 6:38 AM, Ondrej Certik <ondrej at certik.cz> wrote:
>> Hi,
>>
>> is there a way to filter launchpad bugs? They are filling up my inbox,
>> but I would rather put them to the ipython folder or something. They
>> usually look like this:
>>
>> [Bug 244854] [NEW] ipython.el doesn't work with optparse
>>
>> so I wonder how to filter them. Some mailinglists have a header like this:
>>
>> mailing list:   ipython-dev.scipy.org
>>
>> or at least some footer, but these bug reports have nothing. Any ideas?
>
> The best I could do was to make a filter (in gmail) for "has the words
> bugs.launchpad.net ipython", because they don't mark the headers with
> list ids or anything sensible.  But that's working reasonably well for
> me so far (though it could easily produce a false positive,
> obviously).

Yes, for example this email. :) But I marked all of them with ipython
tag, so it doesn't matter. Thanks for the tip.

Ondrej


From fperez.net at gmail.com  Wed Jul  2 13:44:40 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 2 Jul 2008 10:44:40 -0700
Subject: [IPython-dev] how to filter launchpad bugs
In-Reply-To: <85b5c3130807021030q4d0d2e14h5f63fa94d1997670@mail.gmail.com>
References: <85b5c3130807020638j3e73008fkd0435aed52d13de3@mail.gmail.com>
	<db6b5ecc0807021021l13f3f44by9069b81432c9f8f5@mail.gmail.com>
	<85b5c3130807021030q4d0d2e14h5f63fa94d1997670@mail.gmail.com>
Message-ID: <db6b5ecc0807021044l75144d2pe3cc80b660641989@mail.gmail.com>

On Wed, Jul 2, 2008 at 10:30 AM, Ondrej Certik <ondrej at certik.cz> wrote:

> Yes, for example this email. :) But I marked all of them with ipython

GMailRuntimeError: maximum recursion depth exceeded in filter.


;)

cheers,

f


From fperez.net at gmail.com  Wed Jul  2 16:55:47 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 2 Jul 2008 13:55:47 -0700
Subject: [IPython-dev] Tentative release plans...
Message-ID: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>

Hi all,

how does a quick release next week sound for everyone?  Rationale: we
all seem to agree that making more frequent releases to checkpoint
development is good for the project, and recently we've had a large
merge of all the previous ip1 code and docs, as well as all the
frontend scaffolding.  I'd like to push for a release next week
because we're almost done with the two things that in my mind were
necessary:

- process control, so it's easier to start/stop engines/controllers
from within a python standalone script.  Min is hard at work on this.

- testing: many of our recent headaches are due to our lack of testing
for the core code in ip0, and with the upcoming frondend refactoring
not having this fixed would simply be catastrophic.  I've thus
'enjoyed' for the last few days a knee-deep excursion into the
intertwined jungle of unittest/doctest/nose and I'm almost done (will
be by the weekend) with  a nose plugin that recognizes ipython
docstrings, standalone ipython sessions and extension code with
docstrings.  This will give us a way to quickly turn interactive
sessions into doctests so anyone who's about to refactor some big
piece of code can simply first paste a working session and then go
hog-wild with the refactoring, safely knowing that any breakage will
be automatically caught by the tests.

The suggested timing is also due to the fact that Brian and I will
both be together for a few days at a conference in San Diego, so we
can find lunch/evening times to work and push the testing and
logistics of a release.

How does this sound to everyone?

Keep in mind that we DON'T need to have everything in place!  We're
trying to make releases reasonably frequently, so as long as we
clearly mark what's ready and what's not yet, it's OK.  I just don't
want to catch anyone in mid-stride with any specific changes.

Cheers,

f


From barrywark at gmail.com  Wed Jul  2 17:35:56 2008
From: barrywark at gmail.com (Barry Wark)
Date: Wed, 2 Jul 2008 14:35:56 -0700
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
Message-ID: <cd7634ce0807021435x1d9a4a03h51ccec8acfa1148e@mail.gmail.com>

+1 I'm really looking forward to the process control improvements for
engines/controllers.

On Wed, Jul 2, 2008 at 1:55 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi all,
>
> how does a quick release next week sound for everyone?  Rationale: we
> all seem to agree that making more frequent releases to checkpoint
> development is good for the project, and recently we've had a large
> merge of all the previous ip1 code and docs, as well as all the
> frontend scaffolding.  I'd like to push for a release next week
> because we're almost done with the two things that in my mind were
> necessary:
>
> - process control, so it's easier to start/stop engines/controllers
> from within a python standalone script.  Min is hard at work on this.
>
> - testing: many of our recent headaches are due to our lack of testing
> for the core code in ip0, and with the upcoming frondend refactoring
> not having this fixed would simply be catastrophic.  I've thus
> 'enjoyed' for the last few days a knee-deep excursion into the
> intertwined jungle of unittest/doctest/nose and I'm almost done (will
> be by the weekend) with  a nose plugin that recognizes ipython
> docstrings, standalone ipython sessions and extension code with
> docstrings.  This will give us a way to quickly turn interactive
> sessions into doctests so anyone who's about to refactor some big
> piece of code can simply first paste a working session and then go
> hog-wild with the refactoring, safely knowing that any breakage will
> be automatically caught by the tests.
>
> The suggested timing is also due to the fact that Brian and I will
> both be together for a few days at a conference in San Diego, so we
> can find lunch/evening times to work and push the testing and
> logistics of a release.
>
> How does this sound to everyone?
>
> Keep in mind that we DON'T need to have everything in place!  We're
> trying to make releases reasonably frequently, so as long as we
> clearly mark what's ready and what's not yet, it's OK.  I just don't
> want to catch anyone in mid-stride with any specific changes.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From ellisonbg.net at gmail.com  Wed Jul  2 18:25:59 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Wed, 2 Jul 2008 16:25:59 -0600
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
Message-ID: <6ce0ac130807021525j68c2b54ar1fcb3d9a79c3e2c9@mail.gmail.com>

I am all for this.  There are a few other things that must be done for
this release:

1.  The docs need to be updated to reflect the merge.  The most
important things are the install docs and some of the parallel
computing stuff.

2.  Some bugs with setup.py and friends.  The data files are broken
right now and last night I found some bugs with .txt files in the
testing dirs not being included in the package_data

3.  Dependencies.  We now have lots of code that has optional
dependencies (like Twisted, zope.interface, pyobjc, etc.).  We need to
really make sure these are truly optional and don't cause problems for
users who don't have these things installed.

Sounds like I should file some tickets :)

Cheers,

Brian

On Wed, Jul 2, 2008 at 2:55 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi all,
>
> how does a quick release next week sound for everyone?  Rationale: we
> all seem to agree that making more frequent releases to checkpoint
> development is good for the project, and recently we've had a large
> merge of all the previous ip1 code and docs, as well as all the
> frontend scaffolding.  I'd like to push for a release next week
> because we're almost done with the two things that in my mind were
> necessary:
>
> - process control, so it's easier to start/stop engines/controllers
> from within a python standalone script.  Min is hard at work on this.
>
> - testing: many of our recent headaches are due to our lack of testing
> for the core code in ip0, and with the upcoming frondend refactoring
> not having this fixed would simply be catastrophic.  I've thus
> 'enjoyed' for the last few days a knee-deep excursion into the
> intertwined jungle of unittest/doctest/nose and I'm almost done (will
> be by the weekend) with  a nose plugin that recognizes ipython
> docstrings, standalone ipython sessions and extension code with
> docstrings.  This will give us a way to quickly turn interactive
> sessions into doctests so anyone who's about to refactor some big
> piece of code can simply first paste a working session and then go
> hog-wild with the refactoring, safely knowing that any breakage will
> be automatically caught by the tests.
>
> The suggested timing is also due to the fact that Brian and I will
> both be together for a few days at a conference in San Diego, so we
> can find lunch/evening times to work and push the testing and
> logistics of a release.
>
> How does this sound to everyone?
>
> Keep in mind that we DON'T need to have everything in place!  We're
> trying to make releases reasonably frequently, so as long as we
> clearly mark what's ready and what's not yet, it's OK.  I just don't
> want to catch anyone in mid-stride with any specific changes.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From fperez.net at gmail.com  Wed Jul  2 18:28:40 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 2 Jul 2008 15:28:40 -0700
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <6ce0ac130807021525j68c2b54ar1fcb3d9a79c3e2c9@mail.gmail.com>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
	<6ce0ac130807021525j68c2b54ar1fcb3d9a79c3e2c9@mail.gmail.com>
Message-ID: <db6b5ecc0807021528h359ffd28wb22c73cc8dfe5e0c@mail.gmail.com>

On Wed, Jul 2, 2008 at 3:25 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
> I am all for this.  There are a few other things that must be done for
> this release:
>
> 1.  The docs need to be updated to reflect the merge.  The most
> important things are the install docs and some of the parallel
> computing stuff.
>
> 2.  Some bugs with setup.py and friends.  The data files are broken
> right now and last night I found some bugs with .txt files in the
> testing dirs not being included in the package_data
>
> 3.  Dependencies.  We now have lots of code that has optional
> dependencies (like Twisted, zope.interface, pyobjc, etc.).  We need to
> really make sure these are truly optional and don't cause problems for
> users who don't have these things installed.
>
> Sounds like I should file some tickets :)

I'm up to my neck in the testing stuff (but I just a *second* ago got
a key piece to work, so happy here :), so if anyone else can help with
some of this, that would be great.  If not, I guess Brian and I know
what we'll be doing Sunday evening :)

Cheers,

f


From anand.prabhakar.patil at gmail.com  Thu Jul  3 04:35:48 2008
From: anand.prabhakar.patil at gmail.com (Anand Patil)
Date: Thu, 3 Jul 2008 09:35:48 +0100
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
Message-ID: <934A35FD-E0FE-4D1E-A10B-5488BDCC4447@gmail.com>

On 2 Jul 2008, at 21:55, Fernando Perez wrote:

> recently we've had a large
> merge of all the previous ip1 code and docs, as well as all the
> frontend scaffolding.

Does this mean the ipython1 Bazaar branch will be no more?

Cheers,
Anand


From ellisonbg.net at gmail.com  Thu Jul  3 10:23:37 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 3 Jul 2008 08:23:37 -0600
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <934A35FD-E0FE-4D1E-A10B-5488BDCC4447@gmail.com>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
	<934A35FD-E0FE-4D1E-A10B-5488BDCC4447@gmail.com>
Message-ID: <6ce0ac130807030723q2becc5e6n2bd905de9e8d63a7@mail.gmail.com>

On Thu, Jul 3, 2008 at 2:35 AM, Anand Patil
<anand.prabhakar.patil at gmail.com> wrote:
> On 2 Jul 2008, at 21:55, Fernando Perez wrote:
>
>> recently we've had a large
>> merge of all the previous ip1 code and docs, as well as all the
>> frontend scaffolding.
>
> Does this mean the ipython1 Bazaar branch will be no more?

Yep, all of the stuff in ipython1 has been merged into IPython trunk.
We will probably leave ipython1-dev branch around for a while to be
safe, but people should not be using ipython1-dev anymore.

Cheers,

Brian

> Cheers,
> Anand
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From hans_meine at gmx.net  Thu Jul  3 10:38:37 2008
From: hans_meine at gmx.net (Hans Meine)
Date: Thu, 3 Jul 2008 16:38:37 +0200
Subject: [IPython-dev] frontend plans
In-Reply-To: <db6b5ecc0806301737k7d5c0cafo5395a9fc4c933935@mail.gmail.com>
References: <cd7634ce0806251133i305d0b1dsccf083e3c64fcf73@mail.gmail.com>
	<6ce0ac130806262031l191304d9xbd49100ab09def8@mail.gmail.com>
	<db6b5ecc0806301737k7d5c0cafo5395a9fc4c933935@mail.gmail.com>
Message-ID: <200807031638.38346.hans_meine@gmx.net>

Am Dienstag, 01. Juli 2008 02:37:44 schrieb Fernando Perez:
> - Tab completion: this is just one case of the more generic case of
> how to represent out-of-process information.  We have to assume that
> in general, only the core has true access to the real in-memory
> objects of the user's namespace.

Right.  In our PyQt-based pyterm, tab-completion checks whether the backend 
(TCP server running the python commands) is busy and does nothing then.
If you think about it, it's not too different from a regular (ipython / 
operating system) shell, and it has not bothered me a lot at all.  One may 
think about possible out-of-process completions for some corner cases later 
(e.g. for filenames, it may be nontricky to implement something that's useful 
at least in many cases).

> Clients may request information 
> about this for display purposes, and some communication of reduced
> data may happen with such clients, but we can't expect to  copy the
> user's namespace across the wire in a general sense.

Definitely not.

I am particularly happy that the idea of having an optional IPC/nonblocking 
layer between the front- and backend has been discussed; I have myself 
refactored our pyterm in the past in order to separate the communication 
between the two (and make it optional) such that I can run the backend 
in-process (all in one thread).  As IIRC Gael wrote, this can be a common, 
desired mode of operation in GUI programs; at least you can add scripting to 
your GUI very easily then, without thinking about threading and/or 
synchronization (which may be overkill e.g. for your own mini-GUI for 
scientific visualization).

Have a nice day,
  Hans


From ellisonbg.net at gmail.com  Thu Jul  3 11:49:01 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 3 Jul 2008 09:49:01 -0600
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <db6b5ecc0807021528h359ffd28wb22c73cc8dfe5e0c@mail.gmail.com>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
	<6ce0ac130807021525j68c2b54ar1fcb3d9a79c3e2c9@mail.gmail.com>
	<db6b5ecc0807021528h359ffd28wb22c73cc8dfe5e0c@mail.gmail.com>
Message-ID: <6ce0ac130807030849h60a99665mf0cf95a48e704ada@mail.gmail.com>

> I'm up to my neck in the testing stuff (but I just a *second* ago got
> a key piece to work, so happy here :), so if anyone else can help with
> some of this, that would be great.  If not, I guess Brian and I know
> what we'll be doing Sunday evening :)

I didn't mean for this to sound like more things that _you_ should be
doing.  It was more of a reminder list to myself.  I would much rather
have you continue on the testing stuff.  I can start to work on this
other stuff later next week.

Cheers,

Brian

> Cheers,
>
> f
>


From fperez.net at gmail.com  Thu Jul  3 12:09:10 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 3 Jul 2008 09:09:10 -0700
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <6ce0ac130807030849h60a99665mf0cf95a48e704ada@mail.gmail.com>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
	<6ce0ac130807021525j68c2b54ar1fcb3d9a79c3e2c9@mail.gmail.com>
	<db6b5ecc0807021528h359ffd28wb22c73cc8dfe5e0c@mail.gmail.com>
	<6ce0ac130807030849h60a99665mf0cf95a48e704ada@mail.gmail.com>
Message-ID: <db6b5ecc0807030909t3498aa62x4c9fed55accdccf9@mail.gmail.com>

On Thu, Jul 3, 2008 at 8:49 AM, Brian Granger <ellisonbg.net at gmail.com> wrote:
>> I'm up to my neck in the testing stuff (but I just a *second* ago got
>> a key piece to work, so happy here :), so if anyone else can help with
>> some of this, that would be great.  If not, I guess Brian and I know
>> what we'll be doing Sunday evening :)
>
> I didn't mean for this to sound like more things that _you_ should be
> doing.  It was more of a reminder list to myself.  I would much rather
> have you continue on the testing stuff.  I can start to work on this
> other stuff later next week.

No worries, I didn't take it that  way :)

I did get most of the pieces working right, so I'll package things up
and push it upstream probably in a day or so.  There are a few more
things  I want to make sure work, but I think we're in pretty good
shape now.  It was surprisingly hard to get this code working even
though it's a fairly small amount, but I ended up chasing/fixing bugs
in all of Cython, doctest and nose.  Having to work over bugs in 3
separate codebases that aren't your own while trying to integrate them
felt like a root canal...

In any case, I'm reasonably happy with the outcome, even if it's not
perfect.  I'll clear my remaining todo items on that code and put it
in for others to review/comment.

Cheers,

f


From ellisonbg.net at gmail.com  Thu Jul  3 12:22:16 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 3 Jul 2008 10:22:16 -0600
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <db6b5ecc0807030909t3498aa62x4c9fed55accdccf9@mail.gmail.com>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
	<6ce0ac130807021525j68c2b54ar1fcb3d9a79c3e2c9@mail.gmail.com>
	<db6b5ecc0807021528h359ffd28wb22c73cc8dfe5e0c@mail.gmail.com>
	<6ce0ac130807030849h60a99665mf0cf95a48e704ada@mail.gmail.com>
	<db6b5ecc0807030909t3498aa62x4c9fed55accdccf9@mail.gmail.com>
Message-ID: <6ce0ac130807030922w41444441ka4789da678ef4179@mail.gmail.com>

> I did get most of the pieces working right, so I'll package things up
> and push it upstream probably in a day or so.  There are a few more
> things  I want to make sure work, but I think we're in pretty good
> shape now.  It was surprisingly hard to get this code working even
> though it's a fairly small amount, but I ended up chasing/fixing bugs
> in all of Cython, doctest and nose.  Having to work over bugs in 3
> separate codebases that aren't your own while trying to integrate them
> felt like a root canal...

Well, this will be a huge step forward in helping us to get testing
into IPython code.  Look forward to seeing it.

> In any case, I'm reasonably happy with the outcome, even if it's not
> perfect.  I'll clear my remaining todo items on that code and put it
> in for others to review/comment.

Great.

Brian

> Cheers,
>
> f
>


From barrywark at gmail.com  Thu Jul  3 14:18:21 2008
From: barrywark at gmail.com (Barry Wark)
Date: Thu, 3 Jul 2008 11:18:21 -0700
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <db6b5ecc0807030909t3498aa62x4c9fed55accdccf9@mail.gmail.com>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
	<6ce0ac130807021525j68c2b54ar1fcb3d9a79c3e2c9@mail.gmail.com>
	<db6b5ecc0807021528h359ffd28wb22c73cc8dfe5e0c@mail.gmail.com>
	<6ce0ac130807030849h60a99665mf0cf95a48e704ada@mail.gmail.com>
	<db6b5ecc0807030909t3498aa62x4c9fed55accdccf9@mail.gmail.com>
Message-ID: <cd7634ce0807031118p1caa45caq6a4a57da3b8d5b78@mail.gmail.com>

On Thu, Jul 3, 2008 at 9:09 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Thu, Jul 3, 2008 at 8:49 AM, Brian Granger <ellisonbg.net at gmail.com> wrote:
>>> I'm up to my neck in the testing stuff (but I just a *second* ago got
>>> a key piece to work, so happy here :), so if anyone else can help with
>>> some of this, that would be great.  If not, I guess Brian and I know
>>> what we'll be doing Sunday evening :)
>>
>> I didn't mean for this to sound like more things that _you_ should be
>> doing.  It was more of a reminder list to myself.  I would much rather
>> have you continue on the testing stuff.  I can start to work on this
>> other stuff later next week.
>
> No worries, I didn't take it that  way :)
>
> I did get most of the pieces working right, so I'll package things up
> and push it upstream probably in a day or so.  There are a few more
> things  I want to make sure work, but I think we're in pretty good
> shape now.  It was surprisingly hard to get this code working even
> though it's a fairly small amount, but I ended up chasing/fixing bugs
> in all of Cython, doctest and nose.  Having to work over bugs in 3
> separate codebases that aren't your own while trying to integrate them
> felt like a root canal...
>
> In any case, I'm reasonably happy with the outcome, even if it's not
> perfect.  I'll clear my remaining todo items on that code and put it
> in for others to review/comment.

Thanks for taking on this not-so-fun task! It's obviously an important
step in moving forward with confidence and we'll all benefit from your
misery :)

>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From stefan at sun.ac.za  Fri Jul  4 03:55:53 2008
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Fri, 4 Jul 2008 09:55:53 +0200
Subject: [IPython-dev] how to filter launchpad bugs
In-Reply-To: <85b5c3130807020638j3e73008fkd0435aed52d13de3@mail.gmail.com>
References: <85b5c3130807020638j3e73008fkd0435aed52d13de3@mail.gmail.com>
Message-ID: <9457e7c80807040055u518b0c5cyed5e3e15d5e6a052@mail.gmail.com>

2008/7/2 Ondrej Certik <ondrej at certik.cz>:
> is there a way to filter launchpad bugs? They are filling up my inbox,
> but I would rather put them to the ipython folder or something. They
> usually look like this:

Use the following header:

X-Launchpad-Message-Rationale: Subscriber (IPython) @ipython

You can add a Google Mail filter by adding the part after the colon in
the "Must contain" field.

Cheers
St?fan


From vivainio at gmail.com  Fri Jul  4 11:57:11 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 4 Jul 2008 18:57:11 +0300
Subject: [IPython-dev] Getting rid of "print" in ipython
Message-ID: <46cb515a0807040857w6252c7a4l71977155b2a668ed@mail.gmail.com>

IPython source code is riddled with direct calls to print statement.
Obviously, we should get rid of them.

What's the exact plan for this? For the time being, we could get rid
of them by introducing a global function for this (that initially just
does normal "print", and add proper output handling later.

My most urgent interest in this lies in the current inability to
easily capture magic command output. All magic command output should
be accessible in output history as string list, which we could easily
add if printing (inside magics) happened in controlled fashion.

My current suggestion: add global callable "pr" in ipapi, that is
connected to _ip.pr for the currently active IPApi object. Then, all
extensions could do "from IPython.ipapi import pr" and print to that
object to their heart's content.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Fri Jul  4 15:35:24 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 4 Jul 2008 12:35:24 -0700
Subject: [IPython-dev] Getting rid of "print" in ipython
In-Reply-To: <46cb515a0807040857w6252c7a4l71977155b2a668ed@mail.gmail.com>
References: <46cb515a0807040857w6252c7a4l71977155b2a668ed@mail.gmail.com>
Message-ID: <db6b5ecc0807041235s64543e7i23a5ba6b352f1021@mail.gmail.com>

Howdy,

On Fri, Jul 4, 2008 at 8:57 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> IPython source code is riddled with direct calls to print statement.
> Obviously, we should get rid of them.

Yup.

>
> What's the exact plan for this? For the time being, we could get rid
> of them by introducing a global function for this (that initially just
> does normal "print", and add proper output handling later.
>
> My most urgent interest in this lies in the current inability to
> easily capture magic command output. All magic command output should
> be accessible in output history as string list, which we could easily
> add if printing (inside magics) happened in controlled fashion.
>
> My current suggestion: add global callable "pr" in ipapi, that is
> connected to _ip.pr for the currently active IPApi object. Then, all
> extensions could do "from IPython.ipapi import pr" and print to that
> object to their heart's content.

We already have a mechanism in place, we just don't use it
consistenly: self.write/write_err.  Those redirect to the Term object
that can then be provided by a GUI, or replaced to log output, etc.
Rather than a module global called pr, let's simply expose the
write/write_err methods in ipapi and have users explicitly access the
instance.  I don't want more module-level globals, and especially not
in ipapi.

Something like

self.write = self.IP.write # similar for _err

should do.

Cheers,

f


From fperez.net at gmail.com  Fri Jul  4 15:40:59 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 4 Jul 2008 12:40:59 -0700
Subject: [IPython-dev] pyreadline refactor progess
In-Reply-To: <486A92E5.5010701@bostream.nu>
References: <486A92E5.5010701@bostream.nu>
Message-ID: <db6b5ecc0807041240s32748572s9f121b590e5f7c96@mail.gmail.com>

Hey Jorgen,

On Tue, Jul 1, 2008 at 1:26 PM, J?rgen Stenarson
<jorgen.stenarson at bostream.nu> wrote:
> Hi,
>
> the refactoring of pyreadline is progressing nicely. I have now moved
> all stuff that depends on the console out of the emacs mode. The vi mode
> still uses some console stuff and should not be used in non-console mode.


What are your thoughts on this being workable as a full replacement
for python's readline in text terminals?  There are platforms where
readline continues to be a problem (until recently OSX, and John
Hunter tells me his Solaris boxes never work quite right, for
example).  It would be nice if ipython could use its own
self-contained readline everywhere, with a uniform API for text
consoles or GUIs.  But I don't know how to go about the work of
trapping keyboard events in a text terminal (precisely what GNU
readline does), and I don't feel like learning :)

Cheers,

f


From jorgen.stenarson at bostream.nu  Fri Jul  4 16:52:05 2008
From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Fri, 04 Jul 2008 22:52:05 +0200
Subject: [IPython-dev] pyreadline refactor progess
In-Reply-To: <db6b5ecc0807041240s32748572s9f121b590e5f7c96@mail.gmail.com>
References: <486A92E5.5010701@bostream.nu>
	<db6b5ecc0807041240s32748572s9f121b590e5f7c96@mail.gmail.com>
Message-ID: <486E8D75.6020002@bostream.nu>

Fernando Perez skrev:
> Hey Jorgen,
> 
> What are your thoughts on this being workable as a full replacement
> for python's readline in text terminals?  There are platforms where
> readline continues to be a problem (until recently OSX, and John
> Hunter tells me his Solaris boxes never work quite right, for
> example).  It would be nice if ipython could use its own
> self-contained readline everywhere, with a uniform API for text
> consoles or GUIs.  But I don't know how to go about the work of
> trapping keyboard events in a text terminal (precisely what GNU
> readline does), and I don't feel like learning :)
> 

Fernando,

I think it should be possible, but I'm not at a position to do the 
terminal backend development myself (I'm only on windows currently). 
There is an ironpython backend in there but I haven't tried it for a 
long time so I'm not sure how well (if at all) it works now, there were 
some workarounds I had to do because the .NET console class does not 
implement the full windows console functionality.

As a longer term goal of the refactoring I should isolate the methods 
that are necessary for pyreadline to work from those that are included 
but not really necessary. This should make it easier to develop new 
backends by calrifying which methods are necessary for its functionality.

Perhaps it is possible to use the curses library on platforms where that 
is available but I have no idea how well that transmits the keystrokes 
and their interpretation for different locales etc.

/J?rgen


From Alexander_Brown at uml.edu  Fri Jul  4 18:13:55 2008
From: Alexander_Brown at uml.edu (Alex Brown)
Date: Fri, 04 Jul 2008 18:13:55 -0400
Subject: [IPython-dev] ipython-0.9.0.bzr.r1016-py2.5.win32-setup.exe?? [was:
 Re: Win32 IPython1?]
In-Reply-To: <cd7634ce0806281153h5ac76143ie825a0bfa2a41eca@mail.gmail.com>
References: <48457029.8020401@uml.edu>
	<db6b5ecc0806031222x672c6524m64043723da45afa5@mail.gmail.com>
	<48459D9F.9020900@uml.edu>
	<db6b5ecc0806031317y6c790992yfed719ed0e8b4c47@mail.gmail.com>
	<4864549D.2010407@uml.edu>
	<db6b5ecc0806262219w424ae80eh39c54321c789a91c@mail.gmail.com>
	<48659443.4050109@uml.edu>
	<db6b5ecc0806271842l208ebc0cwc554ff7756d6f0c4@mail.gmail.com>
	<4865AB18.4040000@uml.edu>
	<db6b5ecc0806281138p4a3f2c83i16f3f6453abca055@mail.gmail.com>
	<cd7634ce0806281153h5ac76143ie825a0bfa2a41eca@mail.gmail.com>
Message-ID: <486EA0A3.9020307@uml.edu>

Following your recommendations I'm working with your (merged) trunk in 
WinXP.  However I am having continuing difficulty getting 
ipython-0.9.0.bzr.r1016-py2.5 to work with either Python 2.4 or 2.5 
(both Enthought distributions).  It may be that I'm not familiar enough 
with the setup and installer tools, or with Windows touchup tricks.  Is 
there any chance of getting an exe installer, perhaps 
ipython-0.9.0.bzr.r1016-py2.5.win32-setup.exe? 

Alex Brown <Alexander_Brown at uml.edu>
http://gis.uml.edu/abrown2



http://ipython.scipy.org/moin/IpythonOnWindows
> Running IPython on Win32
>
> IPython works well with Windows, even as a replacement for that clunky 
> old cmd.exe command prompt. There are some considerations, though. ...
>
> ... Sometimes when installing a new version of IPython, you need to 
> remove old IPython files to get a clean slate.
>
> To delete all traces of old IPython installation(s), delete the 
> following files (adjust for your python installation):
>
>     * c:\Python25\Scripts\ipython*
>     * C:\Python25\Lib\site-packages\ipython* (also the directories).
>
> Delete start menu shortcuts:
>
>    *
>
>       Start menu -> All programs -> right click on IPython, choose
>       delete.
>
> Install new IPython, launch it, run %upgrade.
Unfortunately this does not catch the version numbers in the 
"ipcontroller", "ipengine", and "ipcluster" scripts.  Are there other 
scripts which have embedded version numbers?

http://ipython.scipy.org/dist/
>
>
>   Index of /dist
>
> Icon  Name <http://ipython.scipy.org/dist/?C=N;O=D>                                      Last modified <http://ipython.scipy.org/dist/?C=M;O=A>      Size <http://ipython.scipy.org/dist/?C=S;O=A>  Description <http://ipython.scipy.org/dist/?C=D;O=A>
> ------------------------------------------------------------------------
> [DIR] Parent Directory <http://ipython.scipy.org/>                                               -   
> ...
>
> [   ] ipython-0.8.2.win32-setup.exe <http://ipython.scipy.org/dist/ipython-0.8.2.win32-setup.exe>             29-Nov-2007 13:14  1.7M  





Barry Wark wrote:
> On Sat, Jun 28, 2008 at 11:38 AM, Fernando Perez <fperez.net at gmail.com> wrote:
>   
>> Hi Alex,
>>
>> again, please send these messages on-list, so they get properly
>> archived and others may also help you if I'm not available (I'm at a
>> conference right now).
>>
>> On Fri, Jun 27, 2008 at 8:08 PM, Alex Brown <Alexander_Brown at uml.edu> wrote:
>>
>>     
>>> I could not find a way to uninstall a module either with setup.py or
>>> easy_install
>>> (http://mail.python.org/pipermail/python-list/2004-December/294789.html) -
>>>       
>> easy_install and distutils have no automated uninstallation, unfortunately.
>>     
>
> I believe you can do easy_install -m to effectively uninstall an app
> that was installed using setuptools (not one that was installed via
> distutils). The -m (or --multi-version) flag removes the package from
> the setuptools .pth file, but doesn't remove it from the
> site-directory. This way, anyone that wants to use it has to
> explicitly require it from setuptools but everything else will not
> know it's there. This is one way to have multiple versions of
> setuptools-installed packages available or to "uninstall" a package
> without breaking any other packages that depend on that package.
>
>
>   
>>> I've simply moved ipython1* out of the tree, OK?   I can't find anything
>>> other than comment and docstring references to ipython1 after removal.
>>>       
>> That is OK.
>>
>> Cheers,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>>
>>     
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080704/2151e7ad/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: blank.gif
Type: image/gif
Size: 148 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080704/2151e7ad/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: back.gif
Type: image/gif
Size: 216 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080704/2151e7ad/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: binary.gif
Type: image/gif
Size: 246 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080704/2151e7ad/attachment-0002.gif>

From fperez.net at gmail.com  Fri Jul  4 18:36:32 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 4 Jul 2008 15:36:32 -0700
Subject: [IPython-dev] Doc guidelines
Message-ID: <db6b5ecc0807041536j3c7c07d0hcd6a075c65c7df01@mail.gmail.com>

Hi all,

I'll be committing soon the testing stuff, and because of this I've
been looking again at the docs directories.  We now have merged docs
from various sources (hand-written reST from ipython1, converted reST
from the old lyx sources, by various people, etc) and I realized that
we have a mix of lots of styles.  Fortunately reST is fairly flexible
so we don't need to go on a major cleanup expedition, but I think that
moving forward we should try to also keep a consistent formatting
style in reST for the same reasons that consistent style in source
code is a good idea when working in a team.

Rather than coming up with any ad-hoc rules of our own, we might as
well use what everyone else related to us is using.  The sphinx
project itself has some information on the basic use of reST with
sphinx here:

http://sphinx.pocoo.org/contents.html

and I'd suggest that we also follow the advice of what matplotlib is doing:

http://matplotlib.sourceforge.net/doc/html/devel/documenting_mpl.html

MPL has recently gone on a *major* documentation effort, and we might
as well share the tools they've added on top of basic sphinx.  We
obviously don't need the math rendering, but things like
ipython-specific highlighting might come in handy :)

Finally, a few things I noticed that I'd like to propose as standard
practice, that aren't mentioned above.  They basically boil down to
re-using PEP-8 conventions in reST when applicable:

- No hard tabs, 4 spaces for indenting code blocks and the like.
While having a mix of tabs and spaces won't cause bugs like it can in
python so easily, it's still likely to produce in the long run
messy-to-edit sources since different people will  do different
things.  It seems best to just stick to one solution consistent with
the rest, else we'll be forever dealing with mixups due to how
everyone had their editor set on any given day.

- Lines to 78 chars or so.  There are lots of unwrapped paragraphs,
these tend to wrap differently depending on the editor used, so it's
probably best to stick to regular line-broken paragraphs as for python
source.

- Any code block meant to represent a command line should have '$'
markers.  I think I'll be able to build support for all reST in the
testing stuff, and this might allow us to mix commands meant for the
shell with python sources without confusing the testing code nor
making the parsing code a complete nightmare.  (the sources are
already pretty good in doing this, though I've seen a few spots where
we've missed it).

I can't think of anything else for now.  But if you all agree on this,
we'll put it in the guidelines and we can update things as we go.

Cheers,

f


From gael.varoquaux at normalesup.org  Sun Jul  6 17:07:31 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 6 Jul 2008 23:07:31 +0200
Subject: [IPython-dev] Schedule for the SciPy08 conferencez
Message-ID: <20080706210731.GC25810@phare.normalesup.org>

We have received a large number of excellent contributions for papers for
the SciPy 2008 conference. The program committee has had to make a
difficult selection and we are happy to bring to you a preliminary
schedule:

Thursday
=========

**8:00** Registration/Breakfast

**8:55** Welcome (Travis Vaught)

**9:10** Keynote (Alex Martelli)

**10:00** State of SciPy (Travis Vaught, Jarrod Millman)

**10:40** -- Break --

**11:00** Sympy - Python library for symbolic mathematics: introduction
and applications (Ond?ej ?ertik)

**11:40** Interval arithmetic: Python implementation and applications
(Stefano Taschini)

**12:00** Experiences Using Scipy for Computer Vision Research (Damian
Eads)

**12:20** -- Lunch --

**1:40** The new NumPy documentation framework (St?fan Van der Walt)

**2:00** Matplotlib solves the riddle of the sphinx (Michael Droettboom)

**2:40** The SciPy documentation project (Joe Harrington)

**3:00** -- Break --

**3:40** Sage: creating a viable free Python-based open source
alternatice to Magma, Maple, Mathematica and Matlab (William Stein)

**4:20** Open space for lightning talks

Friday
========

**8:30** Breakfast

**9:00** Pysynphot: A Python Re-Implementation of a Legacy App in
Astronomy
(Perry Greenfield)

**9:40**  How the Large Synoptic Survey Telescope (LSST) is using
Python (Robert Lupton)

**10:00** Real-time Astronomical Time-series Classification and Broadcast
Pipeline (Dan Starr)

**10:20** Analysis and Visualization of Multi-Scale Astrophysical
Simulations using Python and NumPy (Matthew Turk)

**10:40** -- Break --

**11:00** Exploring network structure, dynamics, and function using
NetworkX (Aric Hagberg)

**11:40** Mayavi: Making 3D data visualization reusable (Prabhu
Ramachandran, Ga?l Varoquaux)

**12:00** Finite Element Modeling of Contact and Impact Problems Using
Python (Ryan Krauss)

**12:20** -- Lunch --

**2:00** PyCircuitScape: A Tool for Landscape Ecology (Viral Shah)

**2:20** Summarizing Complexity in High Dimensional Spaces (Karl Young)

**2:40** UFuncs: A generic function mechanism in Python (Travis Oliphant)

**3:20** -- Break --

**3:40** NumPy Optimization: Manual tuning and automated approaches (Evan
Patterson)

**4:00** Converting Python functions to dynamically-compiled C (Ilan
Schnell)

**4:20** unPython: Converting Python numerical programs into C
(Rahul Garg)

**4:40** Implementing the Grammar of Graphics for Python (Robert Kern)

**5:00** Ask the experts session.

A more detailled booklet including the abstract text will be available
soon.
We are looking forward to seeing you in Caltech,

Ga?l Varoquaux, on behalf of the program committee.

--
SciPy2008 conference.
Program committee

Anne Archibald, McGill University
Matthew Brett
Perry Greenfield, Space Telescope Science Institute
Charles Harris
Ryan Krauss, Southern Illinois University
Ga?l Varoquaux
St?fan van der Walt, University of Stellenbosch


From fperez.net at gmail.com  Sun Jul  6 21:34:38 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 6 Jul 2008 18:34:38 -0700
Subject: [IPython-dev] Wiki edit policy change
Message-ID: <db6b5ecc0807061834k45ef02b7rc63b5625d834924e@mail.gmail.com>

Hi all,

for a very long time, Ryan James has tirelessly been cleaning up spam
on the wiki, but that's neither fun nor fair nor a wise use of
resources (his time in this case).  We recently realized, with Jarrod
Millman's help, what  the problem was: all the recommended techniques
for disabling Moin account creation (at least the publicly listed
ones) fail hopelessly,  because from any existing logged in user, it
is trivial to trick Moin into auto-generating new user accounts that
it  recognizes.

So instead of the ineffective account creation block, I've changed
things to openly allow new accounts, but without write privileges.
Only users listed here:

http://ipython.scipy.org/moin/WritersGroup

can edit pages.   I went through all the logs I could find to add
everyone I could find that seemed like a valid user, please just let
us know and we'll add you if I missed you.  Please note that anyone in

http://ipython.scipy.org/moin/EditorsGroup

can edit the WritersGroup page and add new names, so it should be
trivial to get added if you want to contribute, just ask here on the
list or email an editor (or me) and we'll add you right away.

Hopefully this will now work, since ACLs in Moin seem to be more
robust than their Gruy?re cheese security model for user creation.

And many, many thanks again to Ryan for an endless stream of
spam-cleaning edits!!!

Cheers,

f


From fperez.net at gmail.com  Sun Jul  6 22:10:36 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 6 Jul 2008 19:10:36 -0700
Subject: [IPython-dev] Testing cleanup and support committed
Message-ID: <db6b5ecc0807061910x5844eccepc9a172f6adb287eb@mail.gmail.com>

Hi all,

I've just added

http://bazaar.launchpad.net/~ipython/ipython/trunk/revision/1038

the nose support for ipython doctests and doctests in extension code.
I also cleaned up a  bunch of things from old testing stuff that we
had.  It's NOT finished, there's one piece of  the puzzle that I still
want to have done, which is the ability to run regular reST files with
doctests AND code blocks, where the code blocks are executed and
considered to succeed as long as they don't raise,  but the doctests
are still executed as such.  This last piece would let us run all our
examples in the new documentation, even if they have significant bits
of code,  as part of the test suite.

The plugin directory has a little makefile for temporary testing, it's
just a handy way to keep a note of the command you need to run  while
debugging.  Eventually the (unfinished) iptest script that's there
will provide this functionality, and it will not be necessary to
actually install the plugin.

But the nice thing is that now it is trivial to have entire ipython
sessions copy/pasted from a terminal into either a docstring or a reST
file as a codeblock, and it's automatically a test.  Even though I'm
now busy for a few days at a a conference (with the python talk yet to
be written :) I wanted this out so others could play with it.

It is NOT meant to cleanly test all of ipython yet,  but we'll get
there soon.  And once we're happy with the functionality, I hope we'll
all use this to properly butress any refactoring BEFORE breaking
anything with good tests.

For years the main (terminal) ipython had suffered  tremendously from
the lack of solid testing.  I hope this is a first step to remedy that
problem.

Cheers,

f


From fperez.net at gmail.com  Mon Jul  7 14:10:43 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 7 Jul 2008 11:10:43 -0700
Subject: [IPython-dev] [IPython-user] Wiki edit policy change
In-Reply-To: <18546.21598.436480.380370@montanaro-dyndns-org.local>
References: <db6b5ecc0807061834k45ef02b7rc63b5625d834924e@mail.gmail.com>
	<18546.21598.436480.380370@montanaro-dyndns-org.local>
Message-ID: <db6b5ecc0807071110p5f5ab46encdd2beb43e544c6a@mail.gmail.com>

On Mon, Jul 7, 2008 at 10:37 AM,  <skip at pobox.com> wrote:
>
>    Fernando> Please note that anyone in
>
>    Fernando> http://ipython.scipy.org/moin/EditorsGroup
>
>    Fernando> can edit the WritersGroup page and add new names, so it should
>    Fernando> be trivial to get added if you want to contribute, just ask
>    Fernando> here on the list or email an editor (or me) and we'll add you
>    Fernando> right away.
>
> FWIW, that page appears to be unreadable, even when logged in, so it won't
> work as a way to discover an editor to contact for write privileges.  (No,
> I'm not asking for write privileges.  Just checking the setup.)

Thanks, fixed.  It turns out ACLs in Moin are super-picky about whitespace:

#acl EditorsGroup:read, write, revert, admin All:read

turned out to be (silently) invalid, it should be:

#acl EditorsGroup:read,write,revert,admin All:read

(no spaces after the commas).

Cheers,

f


From stefan at sun.ac.za  Tue Jul  8 05:31:33 2008
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Tue, 8 Jul 2008 11:31:33 +0200
Subject: [IPython-dev] Using pyreadline by default
Message-ID: <9457e7c80807080231i5260403cvd65b13cd61b897c3@mail.gmail.com>

Hi,

As I mentioned earlier, I am not fond of the pager in readline, and
would like to migrate towards pyreadline, in which I can implement my
own.

It looks like the choice between pyreadline and readline is hard-coded
on a platform-by-platform basis in rlineimpl.py.  It would be useful
if we had a configuration variable which allowed the user to make that
choice.  If you are amenable to this idea, I'll write a patch.

Regards
St?fan


From fperez.net at gmail.com  Tue Jul  8 05:39:55 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 8 Jul 2008 02:39:55 -0700
Subject: [IPython-dev] Using pyreadline by default
In-Reply-To: <9457e7c80807080231i5260403cvd65b13cd61b897c3@mail.gmail.com>
References: <9457e7c80807080231i5260403cvd65b13cd61b897c3@mail.gmail.com>
Message-ID: <db6b5ecc0807080239l29c6b374m5894d04eb32c5267@mail.gmail.com>

On Tue, Jul 8, 2008 at 2:31 AM, St?fan van der Walt <stefan at sun.ac.za> wrote:
> Hi,
>
> As I mentioned earlier, I am not fond of the pager in readline, and
> would like to migrate towards pyreadline, in which I can implement my
> own.
>
> It looks like the choice between pyreadline and readline is hard-coded
> on a platform-by-platform basis in rlineimpl.py.  It would be useful
> if we had a configuration variable which allowed the user to make that
> choice.  If you are amenable to this idea, I'll write a patch.

As Jorgen mentioned, the problem is that pyreadline currently only
implements the full line-handling functionality you'd want under win32
(by accessing the console APIs).  Under osx/linux, you'd be stuck
without any actual line handling support, unless I'm not understanding
what you are trying to do.

By the way, isn't it easier for what you want to just define your own
$PAGER environment variable?  You can even do it for ipython only by
making the change in your config file.

We'd certainly love to see pyreadline grow further, as we've been
recently discussing.  But I have the impression that what you're
talking about is a lot of work, not just a small patch (and likely to
require extensive testing on multiple platforms/terminals).  Or else I
totally misunderstood what you're talking about.

Cheers,

f


From stefan at sun.ac.za  Tue Jul  8 06:32:37 2008
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Tue, 8 Jul 2008 12:32:37 +0200
Subject: [IPython-dev] Using pyreadline by default
In-Reply-To: <db6b5ecc0807080239l29c6b374m5894d04eb32c5267@mail.gmail.com>
References: <9457e7c80807080231i5260403cvd65b13cd61b897c3@mail.gmail.com>
	<db6b5ecc0807080239l29c6b374m5894d04eb32c5267@mail.gmail.com>
Message-ID: <9457e7c80807080332s6c090d9btf5005d71c5b61d6f@mail.gmail.com>

2008/7/8 Fernando Perez <fperez.net at gmail.com>:
> As Jorgen mentioned, the problem is that pyreadline currently only
> implements the full line-handling functionality you'd want under win32
> (by accessing the console APIs).  Under osx/linux, you'd be stuck
> without any actual line handling support, unless I'm not understanding
> what you are trying to do.

Oops, I missed that part.

> By the way, isn't it easier for what you want to just define your own
> $PAGER environment variable?  You can even do it for ipython only by
> making the change in your config file.

Unfortunately, that doesn't help: the pager is not used for tab completion.

Regards
St?fan


From dsdale24 at gmail.com  Tue Jul  8 10:20:25 2008
From: dsdale24 at gmail.com (Darren Dale)
Date: Tue, 8 Jul 2008 10:20:25 -0400
Subject: [IPython-dev] question about decision to use twisted
Message-ID: <200807081020.27330.dsdale24@gmail.com>

I have a data acquisition and analysis project that currently uses Parallel 
Python for distributed processing and PyQt4's for some threading and event 
handling. I am concidering the processing module scheduled for inclusion in 
the python standard library starting with version 2.6, and also I am trying 
to get my head around twisted.

I would like to know what the ipython developers concerns were at the time the 
decision was made to use twisted in ipython1, was there some discussion on 
the ipython mailing lists? Any advice or comments would be greatly 
appreciated.

Darren


From jorgen.stenarson at bostream.nu  Tue Jul  8 13:14:41 2008
From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Tue, 08 Jul 2008 19:14:41 +0200
Subject: [IPython-dev] Using pyreadline by default
In-Reply-To: <9457e7c80807080332s6c090d9btf5005d71c5b61d6f@mail.gmail.com>
References: <9457e7c80807080231i5260403cvd65b13cd61b897c3@mail.gmail.com>	<db6b5ecc0807080239l29c6b374m5894d04eb32c5267@mail.gmail.com>
	<9457e7c80807080332s6c090d9btf5005d71c5b61d6f@mail.gmail.com>
Message-ID: <4873A081.7090708@bostream.nu>

St?fan van der Walt skrev:
> 2008/7/8 Fernando Perez <fperez.net at gmail.com>:
>> As Jorgen mentioned, the problem is that pyreadline currently only
>> implements the full line-handling functionality you'd want under win32
>> (by accessing the console APIs).  Under osx/linux, you'd be stuck
>> without any actual line handling support, unless I'm not understanding
>> what you are trying to do.
> 
> Oops, I missed that part.
> 
>> By the way, isn't it easier for what you want to just define your own
>> $PAGER environment variable?  You can even do it for ipython only by
>> making the change in your config file.
> 
> Unfortunately, that doesn't help: the pager is not used for tab completion.
>

You are right right, now in pyreadline there is a _display_completions 
method which just dumps all the matches to the screen. Which means there 
is no check of the $pager variable. I'm not even sure it is a good idea 
if we want to use the code for guis.

The problem with the current approach is that it is at the moment 
hardcoded to use the console. Which means nothing will be shown if you 
try to use it in a gui. There are at least one more function 
(clear_screen) that talks directly to the console and thus does not work 
in a gui. Ideally the completions as well as the request for a 
clear_screen should be propagated to the calling function that 
implements the drawing. I have not decided yet how this is done best.

/J?rgen


From ellisonbg.net at gmail.com  Tue Jul  8 15:57:39 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 8 Jul 2008 12:57:39 -0700
Subject: [IPython-dev] question about decision to use twisted
In-Reply-To: <200807081020.27330.dsdale24@gmail.com>
References: <200807081020.27330.dsdale24@gmail.com>
Message-ID: <6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>

> I have a data acquisition and analysis project that currently uses Parallel
> Python for distributed processing and PyQt4's for some threading and event
> handling. I am concidering the processing module scheduled for inclusion in
> the python standard library starting with version 2.6, and also I am trying
> to get my head around twisted.

Twisted is a much lower level solution than parallel python or
processing.  Are you thinking about using Twisted directly or just
using it through IPython (ipython1 is not in trunk IPython).  Unless
you really want the treading API that processing provides, I think
IPython is by far the best solution [I fully acknowledge my bias here]
for high-level parallelism.

> I would like to know what the ipython developers concerns were at the time the
> decision was made to use twisted in ipython1, was there some discussion on
> the ipython mailing lists? Any advice or comments would be greatly
> appreciated.

I don't think we really discussed this on the mailing list, it was
probably private discussion between Fernando and myself.  We went with
Twisted as:

1) we didn't want to reinvent Twisted

2) we really need what Twisted provides, so the temptation to reinvent
it was great.

3) Twisted has the right abstractions that force you to write correct
an robust networking code.  Of course you can do this without Twisted,
but you are going to work much harder and probably get it wrong in
subtle ways that are hard to figure out.

We have been extremely happy about going with Twisted - it is our secret sauce.

Are there any specific questions you have about Twisted?

Cheers,

Brian

> Darren
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From robert.kern at gmail.com  Tue Jul  8 16:21:17 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 08 Jul 2008 15:21:17 -0500
Subject: [IPython-dev] question about decision to use twisted
In-Reply-To: <6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>
References: <200807081020.27330.dsdale24@gmail.com>
	<6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>
Message-ID: <g50i7t$6kv$1@ger.gmane.org>

Brian Granger wrote:
>> I have a data acquisition and analysis project that currently uses Parallel
>> Python for distributed processing and PyQt4's for some threading and event
>> handling. I am concidering the processing module scheduled for inclusion in
>> the python standard library starting with version 2.6, and also I am trying
>> to get my head around twisted.
> 
> Twisted is a much lower level solution than parallel python or
> processing.  Are you thinking about using Twisted directly or just
> using it through IPython (ipython1 is not in trunk IPython).

s/not/now/

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From barrywark at gmail.com  Tue Jul  8 16:23:33 2008
From: barrywark at gmail.com (Barry Wark)
Date: Tue, 8 Jul 2008 13:23:33 -0700
Subject: [IPython-dev] question about decision to use twisted
In-Reply-To: <6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>
References: <200807081020.27330.dsdale24@gmail.com>
	<6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>
Message-ID: <cd7634ce0807081323k543d2407lb89a924581102616@mail.gmail.com>

On Tue, Jul 8, 2008 at 12:57 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
>> I have a data acquisition and analysis project that currently uses Parallel
>> Python for distributed processing and PyQt4's for some threading and event
>> handling. I am concidering the processing module scheduled for inclusion in
>> the python standard library starting with version 2.6, and also I am trying
>> to get my head around twisted.
>
> Twisted is a much lower level solution than parallel python or
> processing.  Are you thinking about using Twisted directly or just
> using it through IPython (ipython1 is not in trunk IPython).  Unless
> you really want the treading API that processing provides, I think
> IPython is by far the best solution [I fully acknowledge my bias here]
> for high-level parallelism.

Just wanted to clarify Brian's comment... ipython1 is _now_ in trunk
IPython. Brian's done a ton of work to make this happen and I just
didn't want a typo to get in they way ;)

>
>> I would like to know what the ipython developers concerns were at the time the
>> decision was made to use twisted in ipython1, was there some discussion on
>> the ipython mailing lists? Any advice or comments would be greatly
>> appreciated.
>
> I don't think we really discussed this on the mailing list, it was
> probably private discussion between Fernando and myself.  We went with
> Twisted as:
>
> 1) we didn't want to reinvent Twisted
>
> 2) we really need what Twisted provides, so the temptation to reinvent
> it was great.
>
> 3) Twisted has the right abstractions that force you to write correct
> an robust networking code.  Of course you can do this without Twisted,
> but you are going to work much harder and probably get it wrong in
> subtle ways that are hard to figure out.
>
> We have been extremely happy about going with Twisted - it is our secret sauce.
>
> Are there any specific questions you have about Twisted?
>
> Cheers,
>
> Brian
>
>> Darren
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From gael.varoquaux at normalesup.org  Tue Jul  8 16:35:52 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 8 Jul 2008 22:35:52 +0200
Subject: [IPython-dev] question about decision to use twisted
In-Reply-To: <cd7634ce0807081323k543d2407lb89a924581102616@mail.gmail.com>
References: <200807081020.27330.dsdale24@gmail.com>
	<6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>
	<cd7634ce0807081323k543d2407lb89a924581102616@mail.gmail.com>
Message-ID: <20080708203552.GB12533@phare.normalesup.org>

On Tue, Jul 08, 2008 at 01:23:33PM -0700, Barry Wark wrote:
> Just wanted to clarify Brian's comment... ipython1 is _now_ in trunk
> IPython. Brian's done a ton of work to make this happen and I just
> didn't want a typo to get in they way ;)

Yes, this is a huge deal. Thanks a lot to Brian, Fernando, and everybody
else who helped make this happen.

Ga?l


From dsdale24 at gmail.com  Tue Jul  8 16:59:45 2008
From: dsdale24 at gmail.com (Darren Dale)
Date: Tue, 8 Jul 2008 16:59:45 -0400
Subject: [IPython-dev] question about decision to use twisted
In-Reply-To: <6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>
References: <200807081020.27330.dsdale24@gmail.com>
	<6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>
Message-ID: <200807081659.46721.dsdale24@gmail.com>

Hi Brian,

On Tuesday 08 July 2008 03:57:39 pm Brian Granger wrote:
> > I have a data acquisition and analysis project that currently uses
> > Parallel Python for distributed processing and PyQt4's for some threading
> > and event handling. I am concidering the processing module scheduled for
> > inclusion in the python standard library starting with version 2.6, and
> > also I am trying to get my head around twisted.
>
> Twisted is a much lower level solution than parallel python or
> processing.  Are you thinking about using Twisted directly or just
> using it through IPython (ipython1 is now in trunk IPython). 
(typo corrected)

I was thinking of using twisted directly, but I guess I really need to dig 
into ipython1. I don't have a sense of how I would use ipython1 in a 
non-interactive way, is there some discussion in the docs or some examples of 
scripting with ipython1? Is there a callback mechanism in ipython1 to respond 
to non-blocking events?

> Unless 
> you really want the treading API that processing provides, I think
> IPython is by far the best solution [I fully acknowledge my bias here]
> for high-level parallelism.
>
> > I would like to know what the ipython developers concerns were at the
> > time the decision was made to use twisted in ipython1, was there some
> > discussion on the ipython mailing lists? Any advice or comments would be
> > greatly appreciated.
>
> I don't think we really discussed this on the mailing list, it was
> probably private discussion between Fernando and myself.  We went with
> Twisted as:
>
> 1) we didn't want to reinvent Twisted
>
> 2) we really need what Twisted provides, so the temptation to reinvent
> it was great.
>
> 3) Twisted has the right abstractions that force you to write correct
> an robust networking code.  Of course you can do this without Twisted,
> but you are going to work much harder and probably get it wrong in
> subtle ways that are hard to figure out.
>
> We have been extremely happy about going with Twisted - it is our secret
> sauce.
>
> Are there any specific questions you have about Twisted?

After reading some positive things about twisted, I went looking for a couple 
negative reviews. I've read that twisted is a bit of a moving target, and the 
twisted jargon is pretty unfamiliar to me. So finally I was hoping to temper 
all this information with the informed opinion from a group I know and 
respect. This is just the kind of feedback I was hoping to get.

Thank you,
Darren


From robert.kern at gmail.com  Tue Jul  8 18:02:05 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 08 Jul 2008 17:02:05 -0500
Subject: [IPython-dev] question about decision to use twisted
In-Reply-To: <200807081659.46721.dsdale24@gmail.com>
References: <200807081020.27330.dsdale24@gmail.com>	<6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>
	<200807081659.46721.dsdale24@gmail.com>
Message-ID: <g50o4t$sas$1@ger.gmane.org>

Darren Dale wrote:
> After reading some positive things about twisted, I went looking for a couple 
> negative reviews. I've read that twisted is a bit of a moving target,

There are parts that are a moving target, but the team is fairly rigorous about 
labeling those parts as unstable.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ellisonbg.net at gmail.com  Tue Jul  8 19:36:33 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 8 Jul 2008 16:36:33 -0700
Subject: [IPython-dev] question about decision to use twisted
In-Reply-To: <cd7634ce0807081323k543d2407lb89a924581102616@mail.gmail.com>
References: <200807081020.27330.dsdale24@gmail.com>
	<6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>
	<cd7634ce0807081323k543d2407lb89a924581102616@mail.gmail.com>
Message-ID: <6ce0ac130807081636h5e2bc676xd344f1b0aed972e2@mail.gmail.com>

> Just wanted to clarify Brian's comment... ipython1 is _now_ in trunk
> IPython. Brian's done a ton of work to make this happen and I just
> didn't want a typo to get in they way ;)

Thanks Barry for catching this.  Makes me think of Homer:

I am so smart, SMRT.  I am so smart, SMRT.


From ellisonbg.net at gmail.com  Tue Jul  8 19:55:06 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 8 Jul 2008 16:55:06 -0700
Subject: [IPython-dev] question about decision to use twisted
In-Reply-To: <200807081659.46721.dsdale24@gmail.com>
References: <200807081020.27330.dsdale24@gmail.com>
	<6ce0ac130807081257w17730d32y57c85f10eef48610@mail.gmail.com>
	<200807081659.46721.dsdale24@gmail.com>
Message-ID: <6ce0ac130807081655g72f528cehe329e3e8d15709b3@mail.gmail.com>

> I was thinking of using twisted directly, but I guess I really need to dig
> into ipython1. I don't have a sense of how I would use ipython1 in a
> non-interactive way, is there some discussion in the docs or some examples of
> scripting with ipython1? Is there a callback mechanism in ipython1 to respond
> to non-blocking events?

This is one aspect of our marketing that has been bad.  We always talk
about "interactive parallel computing", but IPython (because ipython1
has been merged into IPython, I will just talk about IPython from here
on out) fully supports non-interactive usage.

We have a full set of non-interactive examples that can be found here:


Warning: these examples have not been merged from ipython1->IPython.
Fernando and I are about to do that this week.

As far as callbacks, we have two interfaces for everything:

1.  A fully asynchronous version that uses twisted and all method
return deferreds.  This doesn't work interactively, but it what you
want to use if you need to respond to non-blocking events.

2.  A regular blocking interface that can be used interactively or
not.  This interface doesn't really block - it just lets you poll for
asynchronous results.

But, we fully supports both approaches and option 1 will fully plug
into a regular Twisted based app.

>> Unless
>> you really want the treading API that processing provides, I think
>> IPython is by far the best solution [I fully acknowledge my bias here]
>> for high-level parallelism.
>>
>> > I would like to know what the ipython developers concerns were at the
>> > time the decision was made to use twisted in ipython1, was there some
>> > discussion on the ipython mailing lists? Any advice or comments would be
>> > greatly appreciated.
>>
>> I don't think we really discussed this on the mailing list, it was
>> probably private discussion between Fernando and myself.  We went with
>> Twisted as:
>>
>> 1) we didn't want to reinvent Twisted
>>
>> 2) we really need what Twisted provides, so the temptation to reinvent
>> it was great.
>>
>> 3) Twisted has the right abstractions that force you to write correct
>> an robust networking code.  Of course you can do this without Twisted,
>> but you are going to work much harder and probably get it wrong in
>> subtle ways that are hard to figure out.
>>
>> We have been extremely happy about going with Twisted - it is our secret
>> sauce.
>>
>> Are there any specific questions you have about Twisted?
>
> After reading some positive things about twisted, I went looking for a couple
> negative reviews. I've read that twisted is a bit of a moving target, and the
> twisted jargon is pretty unfamiliar to me. So finally I was hoping to temper
> all this information with the informed opinion from a group I know and
> respect. This is just the kind of feedback I was hoping to get.

The Twisted dev team is borderline OCD about their code.  Everything
is tested or else removed, they do code review and have a great policy
about API changes.  In summary, while there are parts that are
unstable, most of Twisted is rock solid and really robust.  There are
not many other open source projects that I would recommend as highly
as Twisted in this respect.  I am not a Twisted dev.

Some people complain about the documentation, but I have never really
found it to be a problem.  With that said, it does take a while to
really understand the model.  But it is time well spent.

Cheers,

Brian

> Thank you,
> Darren
>


From edreamleo at gmail.com  Wed Jul  9 06:58:34 2008
From: edreamleo at gmail.com (Edward K. Ream)
Date: Wed, 9 Jul 2008 05:58:34 -0500
Subject: [IPython-dev] Getting rid of "print" in ipython
In-Reply-To: <db6b5ecc0807041235s64543e7i23a5ba6b352f1021@mail.gmail.com>
References: <46cb515a0807040857w6252c7a4l71977155b2a668ed@mail.gmail.com>
	<db6b5ecc0807041235s64543e7i23a5ba6b352f1021@mail.gmail.com>
Message-ID: <ffb592890807090358w35bc1029i3ac505708e867e26@mail.gmail.com>

On Fri, Jul 4, 2008 at 2:35 PM, Fernando Perez <fperez.net at gmail.com> wrote:

> Howdy,
>
> On Fri, Jul 4, 2008 at 8:57 AM, Ville M. Vainio <vivainio at gmail.com>
> wrote:
> > IPython source code is riddled with direct calls to print statement.
> > Obviously, we should get rid of them.
>
> Yup.
>
> > What's the exact plan for this? For the time being, we could get rid
> > of them by introducing a global function for this (that initially just
> > does normal "print", and add proper output handling later.
>

Does anyone know of a way to define the global function in a straightforward
way?  There appears to be no way to use 'print' in that function that will
work on both Python 2.x and Python 3.x.  My working assumption is that in
order to preserve a common code base the function will have to use something
like sys.stdout.

Edward
--------------------------------------------------------------------
Edward K. Ream email: edreamleo at gmail.com
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080709/e77a4d30/attachment.html>

From dsdale24 at gmail.com  Wed Jul  9 09:26:08 2008
From: dsdale24 at gmail.com (Darren Dale)
Date: Wed, 9 Jul 2008 09:26:08 -0400
Subject: [IPython-dev] furls
Message-ID: <200807090926.10183.dsdale24@gmail.com>

I am getting into ipython's parallel execution this morning, but I've run into 
a problem with missing furl files. The docs say:

> As a user, you don?t need to know anything about the details of these
> FURLs, other than that when the controller starts, it saves a set of FURLs
> to files named something.furl. The default location of these files is your
> ~./ipython directory.

but when I run examples/kernel/helloworld.py, I get:

ValueError: not a furl or a file containing a 
furl: /home/darren/.ipython/ipcontroller-tc.furl

Probably it is something simple that broke during the transition from the 
ipython1 branch, and devs who already have furl files wouldnt have run into 
it.

Darren


From hans_meine at gmx.net  Wed Jul  9 10:59:34 2008
From: hans_meine at gmx.net (Hans Meine)
Date: Wed, 9 Jul 2008 16:59:34 +0200
Subject: [IPython-dev] Getting rid of "print" in ipython
In-Reply-To: <ffb592890807090520y4575fe4bic49162602a7af213@mail.gmail.com>
References: <46cb515a0807040857w6252c7a4l71977155b2a668ed@mail.gmail.com>
	<200807091337.04454.hans_meine@gmx.net>
	<ffb592890807090520y4575fe4bic49162602a7af213@mail.gmail.com>
Message-ID: <200807091659.35233.hans_meine@gmx.net>

Am Mittwoch, 09. Juli 2008 14:20:40 schrieb Edward K. Ream:
> > [in reply to my private question what's wrong with sys.stdout.write]
> Good question.  There is nothing, per se, wrong with sys.std.write, but it
> is not equivalent to print.  In Python 3.x, the print *function* has
> several keyword arguments, and the semantics of those arguments must be
> handled by, say, IPython's pr function.

What are those arguments?  Where would they be needed in IPython?
So far, the usual string interpolation techniques in connection with 
sys.stdout.write() have always been sufficient for me.

Asking the Python folks for a transition method could make sense (although I 
am sure that the name sys.print_function will *not* be chosen ;-p ), but 
obviously, that is not relevant for IPython.

Greetings,
  Hans


From ellisonbg.net at gmail.com  Wed Jul  9 12:45:28 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Wed, 9 Jul 2008 09:45:28 -0700
Subject: [IPython-dev] furls
In-Reply-To: <200807090926.10183.dsdale24@gmail.com>
References: <200807090926.10183.dsdale24@gmail.com>
Message-ID: <6ce0ac130807090945t23f1fc0fq9cded513a6f4c3d3@mail.gmail.com>

Here is a bit more about how this works.  The FURL stuff is new and we
are working on the docs for it.  As you can see, you have caught us
right _before_ a stable release ;-)

How are you staring the controller and engines?

When the controller is started (either through ipcontroller or
ipcluster, which calls ipcontroller on localhost), it should write the
furl files to the ~./ipython dir on _that_ machine.  If the engines or
clint is running on a different machine, you will need to move the
furl files to the ~./ipython dir on the machine that the client or
engine is running on.

The error you are getting makes it look like the controller is running
on a different machine that the client and that the furl file for the
client has not been moved to the ~./ipython dir of the clients host.

Just to make sure everything is working, can you try to stat the
controller and engines on localhost using:

ipcluster -n 4

And then try to run helloworld.py on that same host?

If that doesn't work, something else is going on and we can help you
figure it out.  The other thing to look at is the log files in
~./ipython/log.  These should show if there are other odd things going
on.

Cheers,

Brian


On Wed, Jul 9, 2008 at 6:26 AM, Darren Dale <dsdale24 at gmail.com> wrote:
> I am getting into ipython's parallel execution this morning, but I've run into
> a problem with missing furl files. The docs say:
>
>> As a user, you don't need to know anything about the details of these
>> FURLs, other than that when the controller starts, it saves a set of FURLs
>> to files named something.furl. The default location of these files is your
>> ~./ipython directory.
>
> but when I run examples/kernel/helloworld.py, I get:
>
> ValueError: not a furl or a file containing a
> furl: /home/darren/.ipython/ipcontroller-tc.furl
>
> Probably it is something simple that broke during the transition from the
> ipython1 branch, and devs who already have furl files wouldnt have run into
> it.
>
> Darren
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From hans_meine at gmx.net  Wed Jul  9 13:00:51 2008
From: hans_meine at gmx.net (Hans Meine)
Date: Wed, 9 Jul 2008 19:00:51 +0200
Subject: [IPython-dev] Getting rid of "print" in ipython
In-Reply-To: <ffb592890807090912v4918d8b8nf7dc352bc23d3e04@mail.gmail.com>
References: <46cb515a0807040857w6252c7a4l71977155b2a668ed@mail.gmail.com>
	<200807091659.35233.hans_meine@gmx.net>
	<ffb592890807090912v4918d8b8nf7dc352bc23d3e04@mail.gmail.com>
Message-ID: <200807091900.58132.hans_meine@gmx.net>

On Mittwoch 09 Juli 2008, Edward K. Ream wrote:
> On Wed, Jul 9, 2008 at 9:59 AM, Hans Meine <hans_meine at gmx.net> wrote:
> > What are those arguments?
>
> http://www.python.org/dev/peps/pep-3105/
>
> def print(*args, sep=' ', end='\n', file=None)

OK, so this is not really surprising, but I don't see which of the arguments 
could be needed in IPython.  Actually, I like string interpolation even more 
than passing separate args, and one could even discuss the \n (e.g. whether 
to append it automatically or be explicit about it), but file is certainly 
not needed (it's up to the frontend IMO).

So I would really propose to just use a minimum wrapper around 
sys.stdout/err.write, which is exactly what iplib already offers. :-)

-- 
Ciao, /  /                                                    .o.
     /--/                                                     ..o
    /  / ANS                                                  ooo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080709/3c67d9cc/attachment.sig>

From dsdale24 at gmail.com  Wed Jul  9 13:37:36 2008
From: dsdale24 at gmail.com (Darren Dale)
Date: Wed, 9 Jul 2008 13:37:36 -0400
Subject: [IPython-dev] furls
In-Reply-To: <6ce0ac130807090945t23f1fc0fq9cded513a6f4c3d3@mail.gmail.com>
References: <200807090926.10183.dsdale24@gmail.com>
	<6ce0ac130807090945t23f1fc0fq9cded513a6f4c3d3@mail.gmail.com>
Message-ID: <200807091337.37727.dsdale24@gmail.com>

On Wednesday 09 July 2008 12:45:28 pm you wrote:
> Here is a bit more about how this works.  The FURL stuff is new and we
> are working on the docs for it.  As you can see, you have caught us
> right _before_ a stable release ;-)
>
> How are you staring the controller and engines?

I was just running the examples in the doc/examples/kernel directory, like:

python helloworld.py

> When the controller is started (either through ipcontroller or
> ipcluster, which calls ipcontroller on localhost), it should write the
> furl files to the ~./ipython dir on _that_ machine.  If the engines or
> clint is running on a different machine, you will need to move the
> furl files to the ~./ipython dir on the machine that the client or
> engine is running on.
>
> The error you are getting makes it look like the controller is running
> on a different machine that the client and that the furl file for the
> client has not been moved to the ~./ipython dir of the clients host.
>
> Just to make sure everything is working, can you try to stat the
> controller and engines on localhost using:
>
> ipcluster -n 4
>
> And then try to run helloworld.py on that same host?

Ok. I didn't realize I had to run ipcluster first. Sorry for the trouble. 
Maybe a readme in examples/kernel would help provide a gentle introduction to 
new users.

Darren


From fperez.net at gmail.com  Wed Jul  9 17:44:12 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 9 Jul 2008 14:44:12 -0700
Subject: [IPython-dev] Getting rid of "print" in ipython
In-Reply-To: <200807091900.58132.hans_meine@gmx.net>
References: <46cb515a0807040857w6252c7a4l71977155b2a668ed@mail.gmail.com>
	<200807091659.35233.hans_meine@gmx.net>
	<ffb592890807090912v4918d8b8nf7dc352bc23d3e04@mail.gmail.com>
	<200807091900.58132.hans_meine@gmx.net>
Message-ID: <db6b5ecc0807091444n45e4b4f5lc1aa68a080dea97e@mail.gmail.com>

On Wed, Jul 9, 2008 at 10:00 AM, Hans Meine <hans_meine at gmx.net> wrote:
> On Mittwoch 09 Juli 2008, Edward K. Ream wrote:
>> On Wed, Jul 9, 2008 at 9:59 AM, Hans Meine <hans_meine at gmx.net> wrote:
>> > What are those arguments?
>>
>> http://www.python.org/dev/peps/pep-3105/
>>
>> def print(*args, sep=' ', end='\n', file=None)
>
> OK, so this is not really surprising, but I don't see which of the arguments
> could be needed in IPython.  Actually, I like string interpolation even more
> than passing separate args, and one could even discuss the \n (e.g. whether
> to append it automatically or be explicit about it), but file is certainly
> not needed (it's up to the frontend IMO).
>
> So I would really propose to just use a minimum wrapper around
> sys.stdout/err.write, which is exactly what iplib already offers. :-)

Fortunately the current interface is almost a subset of the above,
except for the file option and the fact that we don't append the \n.
So if we decide to extend to a 3k-compatible print function, no code
working today would break.  We might want to add at least the 'end'
option soon to keep the newline-handing interface future-proof.

I still do like separate out/err functions rather than a keyword in this case.

Cheers,

f


From gael.varoquaux at normalesup.org  Thu Jul 10 23:48:49 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 11 Jul 2008 05:48:49 +0200
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
Message-ID: <20080711034849.GC10447@phare.normalesup.org>

Hi guys,

Regarding the release, Eric Jones just mentionned to me that he wanted to
start teaching ipython1 for parallelisme in his classes. He bases his
teaching on EPD (Enthought Python Distribution) and there should be a new
release of EPD soon (early August). He would be very happy to include a
new version of ipython, while it is still warm :).

Ga?l


From ellisonbg.net at gmail.com  Fri Jul 11 13:24:19 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Fri, 11 Jul 2008 10:24:19 -0700
Subject: [IPython-dev] Tentative release plans...
In-Reply-To: <20080711034849.GC10447@phare.normalesup.org>
References: <db6b5ecc0807021355h189c3ar2e96ceaec28ca662@mail.gmail.com>
	<20080711034849.GC10447@phare.normalesup.org>
Message-ID: <6ce0ac130807111024m1186abbat79bce090290834a0@mail.gmail.com>

Hi,

We would like to see if we can get a release out by a few days before
August 1st.  This should allow us to get this into the next EPD.

I have just updated the tickets on launchpad and have marked the
released blockers that I know of as critical.  Can everyone look
through the other tickets and mark as critical anything that must be
done before the release?  Also, if people know of un-ticketed items
that need to be done as well, please file critical tickets soon as
well.

Also, this release will be 0.9.

Cheers,

Brian

On Thu, Jul 10, 2008 at 8:48 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> Hi guys,
>
> Regarding the release, Eric Jones just mentionned to me that he wanted to
> start teaching ipython1 for parallelisme in his classes. He bases his
> teaching on EPD (Enthought Python Distribution) and there should be a new
> release of EPD soon (early August). He would be very happy to include a
> new version of ipython, while it is still warm :).
>
> Ga?l
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From gael.varoquaux at normalesup.org  Sat Jul 12 20:28:59 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 13 Jul 2008 02:28:59 +0200
Subject: [IPython-dev] Unused imports
Message-ID: <20080713002859.GB5193@phare.normalesup.org>

I am currently exploring the IPython codebase (I am looking at how to
extract the line processing of IPython0 to port it in IPython1).

We all know that the IPython codebase is a web of entangled object hard
to break down. One thing that strikes me as a low hanging fruit to
improve the situation, though, is cleaning up the imports. Almost every
file I have looked at has a large amount of unused imports.

If you use a powerful IDE like eclipse, you can configure it to show the
unused imports, but if you like emacs or vim like me you can use pyflakes
to check for these. Inspired by the mighty Robert Kern, I have binded one
of my keys in vim to running the file in pyflakes. This is easy and
allows to spot the unused imports in a split second.

Ga?l


From fperez.net at gmail.com  Sun Jul 13 01:31:07 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 12 Jul 2008 22:31:07 -0700
Subject: [IPython-dev] Unused imports
In-Reply-To: <20080713002859.GB5193@phare.normalesup.org>
References: <20080713002859.GB5193@phare.normalesup.org>
Message-ID: <db6b5ecc0807122231q195a8ed6mbac70cb1f5b9809@mail.gmail.com>

Hey,

On Sat, Jul 12, 2008 at 5:28 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> I am currently exploring the IPython codebase (I am looking at how to
> extract the line processing of IPython0 to port it in IPython1).
>
> We all know that the IPython codebase is a web of entangled object hard
> to break down. One thing that strikes me as a low hanging fruit to
> improve the situation, though, is cleaning up the imports. Almost every
> file I have looked at has a large amount of unused imports.

Historical cruft left over after something gets split into two files,
old imports stay behind.

> If you use a powerful IDE like eclipse, you can configure it to show the
> unused imports, but if you like emacs or vim like me you can use pyflakes
> to check for these. Inspired by the mighty Robert Kern, I have binded one
> of my keys in vim to running the file in pyflakes. This is easy and
> allows to spot the unused imports in a split second.

Go for it, by all means!

Cheers,

f


From gael.varoquaux at normalesup.org  Sun Jul 13 01:35:37 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 13 Jul 2008 07:35:37 +0200
Subject: [IPython-dev] Unused imports
In-Reply-To: <db6b5ecc0807122231q195a8ed6mbac70cb1f5b9809@mail.gmail.com>
References: <20080713002859.GB5193@phare.normalesup.org>
	<db6b5ecc0807122231q195a8ed6mbac70cb1f5b9809@mail.gmail.com>
Message-ID: <20080713053537.GE5193@phare.normalesup.org>

On Sat, Jul 12, 2008 at 10:31:07PM -0700, Fernando Perez wrote:
> Go for it, by all means!

Will do. Right now I am working in a branch, so the changes won't
propagate to trunk immediately, but I have started cleaning up the output
of pyflakes.

See you,

Ga?l


From gael.varoquaux at normalesup.org  Sun Jul 13 22:04:30 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Mon, 14 Jul 2008 04:04:30 +0200
Subject: [IPython-dev] Traceback trap in Interpreter
Message-ID: <20080714020430.GF22362@phare.normalesup.org>

Hi,

I must be very dumb. I don't get how I am supposed to use the traceback
trap in interpreter? Right now when I send to the interpreter something
that generates a traceback, I get an exception, and the interpreter does
not finish its execution procedure. As a result I don't get the traceback
trapped in the return message. Maybe I am doing something stupid. My test
code for this is:

from IPython.kernel.core.interpreter import Interpreter
i = Interpreter()
i.execute('print i')

Any clues?

Cheers,

Ga?l


From mhansen at gmail.com  Mon Jul 14 15:30:11 2008
From: mhansen at gmail.com (Mike Hansen)
Date: Mon, 14 Jul 2008 14:30:11 -0500
Subject: [IPython-dev] IPython issues in Sage
Message-ID: <f67335450807141230h46a09fds2f115c8fbc5dfd81@mail.gmail.com>

tHello,

I have two issues that I'd like to report to see if someone knows
what's going wrong since after looking at them for awhile, I'm stuck.

1) When Sage upgraded to IPython 0.8.2, the following trace command broke:

import pdb
import preprocessor
def trace(code, preparse=True):
    code = preparser.preparse(code)
    return pdb.runeval(code)

Before, it would have all the the global variables in the namespace
that code was being run in.  After the swtich to 0.8.2, none of the
globals where present.  I tried explicitly including ip.user_ns in the
environment, but only sort of worked:  see
http://trac.sagemath.org/sage_trac/ticket/3345


2) I tried updating Sage to use IPython 0.8.4, and code in
sage.misc.interpreter which refers to "_ip" now gives

NameError: global name '_ip' is not defined

Was this variable magically inserted into the namespace before?
Anyways, if I add

import IPython.ipapi
_ip = IPython.ipapi.get()

then I don't get the error about "_ip", but Sage quits without running
the mainloop after starting up.  This seems similar to what was
happening here: http://trac.sagemath.org/sage_trac/ticket/1264 .  I
tried clearing out all old IPython modules from site-packages and then
install 0.8.4 cleanly, but that didn't seem to help.

Thanks,
Mike


From robert.kern at gmail.com  Mon Jul 14 15:58:24 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 14 Jul 2008 14:58:24 -0500
Subject: [IPython-dev] Traceback trap in Interpreter
In-Reply-To: <20080714020430.GF22362@phare.normalesup.org>
References: <20080714020430.GF22362@phare.normalesup.org>
Message-ID: <g5gb50$4i6$1@ger.gmane.org>

Gael Varoquaux wrote:
> Hi,
> 
> I must be very dumb. I don't get how I am supposed to use the traceback
> trap in interpreter? Right now when I send to the interpreter something
> that generates a traceback, I get an exception, and the interpreter does
> not finish its execution procedure. As a result I don't get the traceback
> trapped in the return message. Maybe I am doing something stupid. My test
> code for this is:
> 
> from IPython.kernel.core.interpreter import Interpreter
> i = Interpreter()
> i.execute('print i')

i.execute('print i', raiseException=False)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From gael.varoquaux at normalesup.org  Mon Jul 14 19:38:44 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 15 Jul 2008 01:38:44 +0200
Subject: [IPython-dev] Traceback trap in Interpreter
In-Reply-To: <g5gb50$4i6$1@ger.gmane.org>
References: <20080714020430.GF22362@phare.normalesup.org>
	<g5gb50$4i6$1@ger.gmane.org>
Message-ID: <20080714233844.GA14606@phare.normalesup.org>

On Mon, Jul 14, 2008 at 02:58:24PM -0500, Robert Kern wrote:
> i.execute('print i', raiseException=False)

Hey Robert, thanks a lot. I can't believe I managed to miss this. 

Ga?l


From vivainio at gmail.com  Tue Jul 15 03:40:15 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 15 Jul 2008 10:40:15 +0300
Subject: [IPython-dev] IPython issues in Sage
In-Reply-To: <f67335450807141230h46a09fds2f115c8fbc5dfd81@mail.gmail.com>
References: <f67335450807141230h46a09fds2f115c8fbc5dfd81@mail.gmail.com>
Message-ID: <46cb515a0807150040k44cdf434he63ab3cce9d1e940@mail.gmail.com>

On Mon, Jul 14, 2008 at 10:30 PM, Mike Hansen <mhansen at gmail.com> wrote:

Just a quick answer:

> 2) I tried updating Sage to use IPython 0.8.4, and code in
> sage.misc.interpreter which refers to "_ip" now gives
>
> NameError: global name '_ip' is not defined
>
> Was this variable magically inserted into the namespace before?

Yes, and still is. IPython and depends on it to run at all. Perhaps
you are looking in a wrong namespace dict?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From mhansen at gmail.com  Tue Jul 15 03:48:49 2008
From: mhansen at gmail.com (Mike Hansen)
Date: Tue, 15 Jul 2008 02:48:49 -0500
Subject: [IPython-dev] IPython issues in Sage
In-Reply-To: <46cb515a0807150040k44cdf434he63ab3cce9d1e940@mail.gmail.com>
References: <f67335450807141230h46a09fds2f115c8fbc5dfd81@mail.gmail.com>
	<46cb515a0807150040k44cdf434he63ab3cce9d1e940@mail.gmail.com>
Message-ID: <f67335450807150048k5b99f83ay8987a789e4ffc0ad@mail.gmail.com>

> Yes, and still is. IPython and depends on it to run at all. Perhaps
> you are looking in a wrong namespace dict?

I'm not sure what you mean by "looking in a wrong namespace dict".
There was code that used the _ip variable that was working fine with
IPython 0.8.2, but when I upgraded to 0.8.4, it gives me a NameError.

--Mike


From vivainio at gmail.com  Tue Jul 15 04:21:26 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 15 Jul 2008 11:21:26 +0300
Subject: [IPython-dev] IPython issues in Sage
In-Reply-To: <f67335450807150048k5b99f83ay8987a789e4ffc0ad@mail.gmail.com>
References: <f67335450807141230h46a09fds2f115c8fbc5dfd81@mail.gmail.com>
	<46cb515a0807150040k44cdf434he63ab3cce9d1e940@mail.gmail.com>
	<f67335450807150048k5b99f83ay8987a789e4ffc0ad@mail.gmail.com>
Message-ID: <46cb515a0807150121r6e4f21fah1c27cda7112677f7@mail.gmail.com>

On Tue, Jul 15, 2008 at 10:48 AM, Mike Hansen <mhansen at gmail.com> wrote:

>> Yes, and still is. IPython and depends on it to run at all. Perhaps
>> you are looking in a wrong namespace dict?
>
> I'm not sure what you mean by "looking in a wrong namespace dict".
> There was code that used the _ip variable that was working fine with
> IPython 0.8.2, but when I upgraded to 0.8.4, it gives me a NameError.

Yeah, but perhaps it didn't look for it in the right place? It
certainly is in the IPython user namespace. However, it's not in
__builtin__ namespace anymore (it caused various problems when used
from there, can't remember the exact bugs that were fixed by that).
You need to acquire the _ip object by using IPython.ipapi.get() if you
are not operating within ipython user namespace.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From mhansen at gmail.com  Tue Jul 15 04:26:10 2008
From: mhansen at gmail.com (Mike Hansen)
Date: Tue, 15 Jul 2008 03:26:10 -0500
Subject: [IPython-dev] IPython issues in Sage
In-Reply-To: <46cb515a0807150121r6e4f21fah1c27cda7112677f7@mail.gmail.com>
References: <f67335450807141230h46a09fds2f115c8fbc5dfd81@mail.gmail.com>
	<46cb515a0807150040k44cdf434he63ab3cce9d1e940@mail.gmail.com>
	<f67335450807150048k5b99f83ay8987a789e4ffc0ad@mail.gmail.com>
	<46cb515a0807150121r6e4f21fah1c27cda7112677f7@mail.gmail.com>
Message-ID: <f67335450807150126h252b0bbcg18c8d081dcf0ab68@mail.gmail.com>

> Yeah, but perhaps it didn't look for it in the right place? It
> certainly is in the IPython user namespace. However, it's not in
> __builtin__ namespace anymore (it caused various problems when used
> from there, can't remember the exact bugs that were fixed by that).
> You need to acquire the _ip object by using IPython.ipapi.get() if you
> are not operating within ipython user namespace.

In my original message, I mentioned that I tried that and it
eliminated the NameError, but then I get the issue where the main loop
is never run.  Which is similar to what is happening here:
http://trac.sagemath.org/sage_trac/ticket/1264 .  I wasn't able to
figure out what Fernando did the last time to get rid of the problem.

--Mike


From vivainio at gmail.com  Tue Jul 15 15:37:50 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 15 Jul 2008 22:37:50 +0300
Subject: [IPython-dev] IPython issues in Sage
In-Reply-To: <f67335450807150126h252b0bbcg18c8d081dcf0ab68@mail.gmail.com>
References: <f67335450807141230h46a09fds2f115c8fbc5dfd81@mail.gmail.com>
	<46cb515a0807150040k44cdf434he63ab3cce9d1e940@mail.gmail.com>
	<f67335450807150048k5b99f83ay8987a789e4ffc0ad@mail.gmail.com>
	<46cb515a0807150121r6e4f21fah1c27cda7112677f7@mail.gmail.com>
	<f67335450807150126h252b0bbcg18c8d081dcf0ab68@mail.gmail.com>
Message-ID: <46cb515a0807151237ufed29bcl3ca4229ec2c03155@mail.gmail.com>

On Tue, Jul 15, 2008 at 11:26 AM, Mike Hansen <mhansen at gmail.com> wrote:

> In my original message, I mentioned that I tried that and it
> eliminated the NameError, but then I get the issue where the main loop
> is never run.  Which is similar to what is happening here:

These problems are probably unrelated.

> http://trac.sagemath.org/sage_trac/ticket/1264 .  I wasn't able to
> figure out what Fernando did the last time to get rid of the problem.

That report seems to indicate that it happen because sage launches
ipython so that it enters "batch" mode and exits ipython. See the end
of "post_config_initialization" in iplib.py.

Your problem probably is that you are launching ipython the wrong way,
so that "rc.interact" option is not set. The old behaviour was broken
for the documented functionality of command line args -i and -c
(possibly the functionality was broken exactly for the benefit of
sage). How do you launch IPython from sage, exactly? Is sys.argv
clean? Does it have -c without -i?

Also, have you considered making a "sage profile"
(ipy_profile_sage.py) that would launch ipython first and only bring
in sage later on?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Tue Jul 15 15:57:36 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 15 Jul 2008 22:57:36 +0300
Subject: [IPython-dev] Unused imports
In-Reply-To: <20080713002859.GB5193@phare.normalesup.org>
References: <20080713002859.GB5193@phare.normalesup.org>
Message-ID: <46cb515a0807151257x68740df5x21c63b8f0f04fb7b@mail.gmail.com>

On Sun, Jul 13, 2008 at 3:28 AM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:

> We all know that the IPython codebase is a web of entangled object hard
> to break down. One thing that strikes me as a low hanging fruit to
> improve the situation, though, is cleaning up the imports. Almost every
> file I have looked at has a large amount of unused imports.

Other low-hanging fruit in related vein is splitting up genutils to
smaller parts. Most of the circular dependency problems I've had were
because of genutils.

When working with input history, be careful to observe that raw
history and "native history" stay in sync. I would be in favor of
ditching the native history completely, since it would appear to be
quite unnecessary apart from debugging why certain statements do not
work (i.e. it's more interesting to developers than end users).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From strawman at astraw.com  Tue Jul 15 20:01:03 2008
From: strawman at astraw.com (Andrew Straw)
Date: Tue, 15 Jul 2008 17:01:03 -0700
Subject: [IPython-dev] pushing python class definition to engines
Message-ID: <487D3A3F.4000304@astraw.com>

After seeing Brian Granger's great demo of the parallel abilities in
IPython last week at the SIAM conference, I'm playing with ipython trunk
which I just pulled out of launchpad (for this reason, I'm posting on
the -dev list).

Anyhow, if I start a client in the usual way:

from IPython.kernel import client
mec = client.MultiEngineClient()

I can easily push Python integers:

a=5
mec['a'] = a
print mec.gather('a')

But pushing classes doesn't work:

class MyObj(object):
    def __init__(self,x):
        print 'PID',os.getpid()
        self.x=x

mec['MyObj'] = MyObj
print mec.gather('MyObj')

That gives the following error. How can I push a class definition to my
engines?

Traceback (most recent call last):
  File "flydra/test_ipython.py", line 14, in <module>
    mec.push_function(dict(MyObj= MyObj,))
  File
"/home/astraw/PY_ipython/lib/python2.5/site-packages/IPython/kernel/multiengineclient.py",
line 606, in push_function
    return self._blockFromThread(self.smultiengine.push_function,
namespace, targets=targets, block=block)
  File
"/home/astraw/PY_ipython/lib/python2.5/site-packages/IPython/kernel/multiengineclient.py",
line 456, in _blockFromThread
    result = blockingCallFromThread(function, *args, **kwargs)
  File
"/home/astraw/PY_ipython/lib/python2.5/site-packages/IPython/kernel/twistedutil.py",
line 69, in blockingCallFromThread
    return twisted.internet.threads.blockingCallFromThread(reactor, f,
*a, **kw)
  File
"/home/astraw/PY_ipython/lib/python2.5/site-packages/Twisted-8.1.0-py2.5-linux-x86_64.egg/twisted/internet/threads.py",
line 83, in blockingCallFromThread
    result.raiseException()
  File
"/home/astraw/PY_ipython/lib/python2.5/site-packages/Twisted-8.1.0-py2.5-linux-x86_64.egg/twisted/python/failure.py",
line 319, in raiseException
    raise self.type, self.value, self.tb
AttributeError: 'module' object has no attribute 'MyObj'



From robert.kern at gmail.com  Tue Jul 15 20:16:39 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 15 Jul 2008 19:16:39 -0500
Subject: [IPython-dev] pushing python class definition to engines
In-Reply-To: <487D3A3F.4000304@astraw.com>
References: <487D3A3F.4000304@astraw.com>
Message-ID: <g5jel8$1ud$1@ger.gmane.org>

Andrew Straw wrote:
> After seeing Brian Granger's great demo of the parallel abilities in
> IPython last week at the SIAM conference, I'm playing with ipython trunk
> which I just pulled out of launchpad (for this reason, I'm posting on
> the -dev list).
> 
> Anyhow, if I start a client in the usual way:
> 
> from IPython.kernel import client
> mec = client.MultiEngineClient()
> 
> I can easily push Python integers:
> 
> a=5
> mec['a'] = a
> print mec.gather('a')
> 
> But pushing classes doesn't work:
> 
> class MyObj(object):
>     def __init__(self,x):
>         print 'PID',os.getpid()
>         self.x=x
> 
> mec['MyObj'] = MyObj
> print mec.gather('MyObj')
> 
> That gives the following error. How can I push a class definition to my
> engines?

It's really tricky since class objects normally don't really serialize. Pickles 
just contain a reference to the fully-qualified name of the class. In this case, 
it's __main__.MyObj. Since there is no MyObj in __main__ in your remote 
namespace, you get the error you see. You will have to push actual code to your 
remote kernels.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From strawman at astraw.com  Tue Jul 15 20:30:35 2008
From: strawman at astraw.com (Andrew Straw)
Date: Tue, 15 Jul 2008 17:30:35 -0700
Subject: [IPython-dev] pushing python class definition to engines
In-Reply-To: <g5jel8$1ud$1@ger.gmane.org>
References: <487D3A3F.4000304@astraw.com> <g5jel8$1ud$1@ger.gmane.org>
Message-ID: <487D412B.9040709@astraw.com>

Robert Kern wrote:
> Andrew Straw wrote:
>   
>> After seeing Brian Granger's great demo of the parallel abilities in
>> IPython last week at the SIAM conference, I'm playing with ipython trunk
>> which I just pulled out of launchpad (for this reason, I'm posting on
>> the -dev list).
>>
>> Anyhow, if I start a client in the usual way:
>>
>> from IPython.kernel import client
>> mec = client.MultiEngineClient()
>>
>> I can easily push Python integers:
>>
>> a=5
>> mec['a'] = a
>> print mec.gather('a')
>>
>> But pushing classes doesn't work:
>>
>> class MyObj(object):
>>     def __init__(self,x):
>>         print 'PID',os.getpid()
>>         self.x=x
>>
>> mec['MyObj'] = MyObj
>> print mec.gather('MyObj')
>>
>> That gives the following error. How can I push a class definition to my
>> engines?
>>     
>
> It's really tricky since class objects normally don't really serialize. Pickles 
> just contain a reference to the fully-qualified name of the class. In this case, 
> it's __main__.MyObj. Since there is no MyObj in __main__ in your remote 
> namespace, you get the error you see. You will have to push actual code to your 
> remote kernels.
>   
OK. Thanks for the info. How do I do that/what do you mean? (Do I simply
make a string with my class definition and push that across?)

-Andrew


From robert.kern at gmail.com  Tue Jul 15 20:39:05 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 15 Jul 2008 19:39:05 -0500
Subject: [IPython-dev] pushing python class definition to engines
In-Reply-To: <487D412B.9040709@astraw.com>
References: <487D3A3F.4000304@astraw.com> <g5jel8$1ud$1@ger.gmane.org>
	<487D412B.9040709@astraw.com>
Message-ID: <g5jfv9$4jb$1@ger.gmane.org>

Andrew Straw wrote:
> Robert Kern wrote:
>> Andrew Straw wrote:
>>   
>>> After seeing Brian Granger's great demo of the parallel abilities in
>>> IPython last week at the SIAM conference, I'm playing with ipython trunk
>>> which I just pulled out of launchpad (for this reason, I'm posting on
>>> the -dev list).
>>>
>>> Anyhow, if I start a client in the usual way:
>>>
>>> from IPython.kernel import client
>>> mec = client.MultiEngineClient()
>>>
>>> I can easily push Python integers:
>>>
>>> a=5
>>> mec['a'] = a
>>> print mec.gather('a')
>>>
>>> But pushing classes doesn't work:
>>>
>>> class MyObj(object):
>>>     def __init__(self,x):
>>>         print 'PID',os.getpid()
>>>         self.x=x
>>>
>>> mec['MyObj'] = MyObj
>>> print mec.gather('MyObj')
>>>
>>> That gives the following error. How can I push a class definition to my
>>> engines?
>>>     
>> It's really tricky since class objects normally don't really serialize. Pickles 
>> just contain a reference to the fully-qualified name of the class. In this case, 
>> it's __main__.MyObj. Since there is no MyObj in __main__ in your remote 
>> namespace, you get the error you see. You will have to push actual code to your 
>> remote kernels.
>>   
> OK. Thanks for the info. How do I do that/what do you mean? (Do I simply
> make a string with my class definition and push that across?)

mec.execute('class MyObj(object):\n    pass') for example.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ellisonbg.net at gmail.com  Tue Jul 15 22:36:45 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 15 Jul 2008 20:36:45 -0600
Subject: [IPython-dev] pushing python class definition to engines
In-Reply-To: <487D3A3F.4000304@astraw.com>
References: <487D3A3F.4000304@astraw.com>
Message-ID: <6ce0ac130807151936o2b8fe47do7ee7900d8df62e4f@mail.gmail.com>

Andrew,

As Robert mentioned, this is really a limitation in how Python
serializes classes and instances.  I have dealt with this in a couple
of ways:

1.  Don't use classes :)  More, seriously though, functions, even
those defined interactively can be pushed and pulled using
push_function and pull_function.  There are only a few limitations,
like no closures.

2.  Define your classes in a standalone module that can be imported by
the engines, controller and client.  Then, you can push and pull
instances just fine using push and pull.  This is mostly how I work -
I develop classes in serial first, put them in a module and then
simply import.

Ironically, functions work better for this than classes.  Until Python
has a different way of serializing classes and instances, we are
stuck.

Cheers,

Brian


On Tue, Jul 15, 2008 at 6:01 PM, Andrew Straw <strawman at astraw.com> wrote:
> After seeing Brian Granger's great demo of the parallel abilities in
> IPython last week at the SIAM conference, I'm playing with ipython trunk
> which I just pulled out of launchpad (for this reason, I'm posting on
> the -dev list).
>
> Anyhow, if I start a client in the usual way:
>
> from IPython.kernel import client
> mec = client.MultiEngineClient()
>
> I can easily push Python integers:
>
> a=5
> mec['a'] = a
> print mec.gather('a')
>
> But pushing classes doesn't work:
>
> class MyObj(object):
>    def __init__(self,x):
>        print 'PID',os.getpid()
>        self.x=x
>
> mec['MyObj'] = MyObj
> print mec.gather('MyObj')
>
> That gives the following error. How can I push a class definition to my
> engines?
>
> Traceback (most recent call last):
>  File "flydra/test_ipython.py", line 14, in <module>
>    mec.push_function(dict(MyObj= MyObj,))
>  File
> "/home/astraw/PY_ipython/lib/python2.5/site-packages/IPython/kernel/multiengineclient.py",
> line 606, in push_function
>    return self._blockFromThread(self.smultiengine.push_function,
> namespace, targets=targets, block=block)
>  File
> "/home/astraw/PY_ipython/lib/python2.5/site-packages/IPython/kernel/multiengineclient.py",
> line 456, in _blockFromThread
>    result = blockingCallFromThread(function, *args, **kwargs)
>  File
> "/home/astraw/PY_ipython/lib/python2.5/site-packages/IPython/kernel/twistedutil.py",
> line 69, in blockingCallFromThread
>    return twisted.internet.threads.blockingCallFromThread(reactor, f,
> *a, **kw)
>  File
> "/home/astraw/PY_ipython/lib/python2.5/site-packages/Twisted-8.1.0-py2.5-linux-x86_64.egg/twisted/internet/threads.py",
> line 83, in blockingCallFromThread
>    result.raiseException()
>  File
> "/home/astraw/PY_ipython/lib/python2.5/site-packages/Twisted-8.1.0-py2.5-linux-x86_64.egg/twisted/python/failure.py",
> line 319, in raiseException
>    raise self.type, self.value, self.tb
> AttributeError: 'module' object has no attribute 'MyObj'
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From fperez.net at gmail.com  Wed Jul 16 02:23:54 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 15 Jul 2008 23:23:54 -0700
Subject: [IPython-dev] Unused imports
In-Reply-To: <46cb515a0807151257x68740df5x21c63b8f0f04fb7b@mail.gmail.com>
References: <20080713002859.GB5193@phare.normalesup.org>
	<46cb515a0807151257x68740df5x21c63b8f0f04fb7b@mail.gmail.com>
Message-ID: <db6b5ecc0807152323h5feb811cx91104ff6be8fb4c4@mail.gmail.com>

On Tue, Jul 15, 2008 at 12:57 PM, Ville M. Vainio <vivainio at gmail.com> wrote:

> When working with input history, be careful to observe that raw
> history and "native history" stay in sync. I would be in favor of
> ditching the native history completely, since it would appear to be
> quite unnecessary apart from debugging why certain statements do not
> work (i.e. it's more interesting to developers than end users).

I seem to recall Sage needing it, but I could be wrong.  It's been a
while, but let's not pull it out without confirmation that they don't
need it.

Cheers,

f


From fperez.net at gmail.com  Wed Jul 16 02:31:26 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 15 Jul 2008 23:31:26 -0700
Subject: [IPython-dev] IPython issues in Sage
In-Reply-To: <f67335450807150048k5b99f83ay8987a789e4ffc0ad@mail.gmail.com>
References: <f67335450807141230h46a09fds2f115c8fbc5dfd81@mail.gmail.com>
	<46cb515a0807150040k44cdf434he63ab3cce9d1e940@mail.gmail.com>
	<f67335450807150048k5b99f83ay8987a789e4ffc0ad@mail.gmail.com>
Message-ID: <db6b5ecc0807152331x1181d2a3k894b2e6eb09cefe9@mail.gmail.com>

On Tue, Jul 15, 2008 at 12:48 AM, Mike Hansen <mhansen at gmail.com> wrote:
>> Yes, and still is. IPython and depends on it to run at all. Perhaps
>> you are looking in a wrong namespace dict?
>
> I'm not sure what you mean by "looking in a wrong namespace dict".
> There was code that used the _ip variable that was working fine with
> IPython 0.8.2, but when I upgraded to 0.8.4, it gives me a NameError.

Ville's explanation is correct, I'll just give you a tiny bit of
detail to clarify things further: ipython executes all user code in a
special dict called user_ns.  It *used* to also populate the builtin
namespace with a few things, but we're trying to move away as much as
possible from this, because having system-wide globals causes all
kinds of other problems.

Sorry if this one bit you, since it was an API change, but it was for the best.

Let us know if Ville's other suggestions don't fully solve your problem.

Cheers,

f


From mhansen at gmail.com  Wed Jul 16 02:38:09 2008
From: mhansen at gmail.com (Mike Hansen)
Date: Wed, 16 Jul 2008 01:38:09 -0500
Subject: [IPython-dev] IPython issues in Sage
In-Reply-To: <db6b5ecc0807152331x1181d2a3k894b2e6eb09cefe9@mail.gmail.com>
References: <f67335450807141230h46a09fds2f115c8fbc5dfd81@mail.gmail.com>
	<46cb515a0807150040k44cdf434he63ab3cce9d1e940@mail.gmail.com>
	<f67335450807150048k5b99f83ay8987a789e4ffc0ad@mail.gmail.com>
	<db6b5ecc0807152331x1181d2a3k894b2e6eb09cefe9@mail.gmail.com>
Message-ID: <f67335450807152338k2078998fn4341d62d3c21e147@mail.gmail.com>

> Ville's explanation is correct, I'll just give you a tiny bit of
> detail to clarify things further: ipython executes all user code in a
> special dict called user_ns.  It *used* to also populate the builtin
> namespace with a few things, but we're trying to move away as much as
> possible from this, because having system-wide globals causes all
> kinds of other problems.
>
> Sorry if this one bit you, since it was an API change, but it was for the best.
>
> Let us know if Ville's other suggestions don't fully solve your problem.

Yep, that worked.  Thanks to both of you for the explanation.

--Mike


From vivainio at gmail.com  Wed Jul 16 15:09:06 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 16 Jul 2008 22:09:06 +0300
Subject: [IPython-dev] IPython issues in Sage
In-Reply-To: <f67335450807152338k2078998fn4341d62d3c21e147@mail.gmail.com>
References: <f67335450807141230h46a09fds2f115c8fbc5dfd81@mail.gmail.com>
	<46cb515a0807150040k44cdf434he63ab3cce9d1e940@mail.gmail.com>
	<f67335450807150048k5b99f83ay8987a789e4ffc0ad@mail.gmail.com>
	<db6b5ecc0807152331x1181d2a3k894b2e6eb09cefe9@mail.gmail.com>
	<f67335450807152338k2078998fn4341d62d3c21e147@mail.gmail.com>
Message-ID: <46cb515a0807161209y2a89d071q3762e8b68f8f1965@mail.gmail.com>

On Wed, Jul 16, 2008 at 9:38 AM, Mike Hansen <mhansen at gmail.com> wrote:

>> Ville's explanation is correct, I'll just give you a tiny bit of
>> detail to clarify things further: ipython executes all user code in a
>> special dict called user_ns.  It *used* to also populate the builtin
>> namespace with a few things, but we're trying to move away as much as
>> possible from this, because having system-wide globals causes all
>> kinds of other problems.
>>
>> Sorry if this one bit you, since it was an API change, but it was for the best.
>>
>> Let us know if Ville's other suggestions don't fully solve your problem.
>
> Yep, that worked.  Thanks to both of you for the explanation.

Please explain the answer you found here as well, for the benefit of
search engines and others who could bump on the same problem.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From gael.varoquaux at normalesup.org  Sun Jul 20 12:46:15 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 20 Jul 2008 18:46:15 +0200
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
Message-ID: <20080720164615.GB23971@phare.normalesup.org>

Hi,

I am making progress on my graphical fronend for ipython1, but I have hit
an interesting small difficulty: currently the way the interreter is set,
it traps all output, and returns it at the end of the execution. This is
quite bad if a users is (in a synchronous, local workflow) running a long
job, and printing information on the advancment of his job, say for
instance:

for i in range(10):
    print i
    sleep(1)

I can off course monkey patch the output_trap with my own write method,
but this seems a bit ugly. What do the ipython1 developpers think the
right way of doing this (and maybe modifying output_trap) is?

Cheers,

Ga?l


From gael.varoquaux at normalesup.org  Sun Jul 20 15:27:02 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 20 Jul 2008 21:27:02 +0200
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <20080720164615.GB23971@phare.normalesup.org>
References: <20080720164615.GB23971@phare.normalesup.org>
Message-ID: <20080720192702.GH23971@phare.normalesup.org>

> I can off course monkey patch the output_trap with my own write method,
> but this seems a bit ugly. What do the ipython1 developpers think the
> right way of doing this (and maybe modifying output_trap) is?

I went ahead experimenting, and created a subclass of output_trap gear
towards printing via a callback, as can be seen on
http://bazaar.launchpad.net/~gael-varoquaux/ipython/ipython-sync-frontend/annotate/1041?file_id=sync_output_trap.py-20080720174256-b1fz1ebbxl2ka4t9-1

I hope this approach is OK. By passing this this output_trap to the
interpreter I get the desired result, and I sync this approach is
reusable for any synchronous frontend, such as the one into which
ipython0 will, hopefully, morph one day.

Ga?l


From ellisonbg.net at gmail.com  Sun Jul 20 15:50:01 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Sun, 20 Jul 2008 13:50:01 -0600
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <20080720192702.GH23971@phare.normalesup.org>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
Message-ID: <6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>

I don't have any time to look at this now, but we should have an
interface in the interpreter that allow the registration of a callback
that gets called or these things.

Cheers,

Brian

On Sun, Jul 20, 2008 at 1:27 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
>> I can off course monkey patch the output_trap with my own write method,
>> but this seems a bit ugly. What do the ipython1 developpers think the
>> right way of doing this (and maybe modifying output_trap) is?
>
> I went ahead experimenting, and created a subclass of output_trap gear
> towards printing via a callback, as can be seen on
> http://bazaar.launchpad.net/~gael-varoquaux/ipython/ipython-sync-frontend/annotate/1041?file_id=sync_output_trap.py-20080720174256-b1fz1ebbxl2ka4t9-1
>
> I hope this approach is OK. By passing this this output_trap to the
> interpreter I get the desired result, and I sync this approach is
> reusable for any synchronous frontend, such as the one into which
> ipython0 will, hopefully, morph one day.
>
> Ga?l
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From laurent.dufrechou at gmail.com  Sun Jul 20 15:56:23 2008
From: laurent.dufrechou at gmail.com (Laurent Dufrechou)
Date: Sun, 20 Jul 2008 21:56:23 +0200
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <20080720192702.GH23971@phare.normalesup.org>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
Message-ID: <7ced318f0807201256i7fb5826ah21455c5faf9a92b@mail.gmail.com>

Hi gael,
I've checked out your branch and done `python setup.py build_ext --inplace`
Launching your app:

laurent at Pep:~/ipython-sync-frontend/IPython$ python
./frontend/wx/wx_frontend.py
Traceback (most recent call last):
  File "./frontend/wx/wx_frontend.py", line 30, in <module>
    from IPython.frontend.prefilterfrontend import PrefilterFrontEnd
ImportError: No module named frontend.prefilterfrontend

Am I missing something?

laurent at Pep:~/ipython-sync-frontend/IPython/frontend$ ls
asyncfrontendbase.py  cocoa  frontendbase.py  __init__.py
linefrontendbase.py  tests  wx

Laurent

2008/7/20 Gael Varoquaux <gael.varoquaux at normalesup.org>:
>> I can off course monkey patch the output_trap with my own write method,
>> but this seems a bit ugly. What do the ipython1 developpers think the
>> right way of doing this (and maybe modifying output_trap) is?
>
> I went ahead experimenting, and created a subclass of output_trap gear
> towards printing via a callback, as can be seen on
> http://bazaar.launchpad.net/~gael-varoquaux/ipython/ipython-sync-frontend/annotate/1041?file_id=sync_output_trap.py-20080720174256-b1fz1ebbxl2ka4t9-1
>
> I hope this approach is OK. By passing this this output_trap to the
> interpreter I get the desired result, and I sync this approach is
> reusable for any synchronous frontend, such as the one into which
> ipython0 will, hopefully, morph one day.
>
> Ga?l
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From gael.varoquaux at normalesup.org  Sun Jul 20 16:03:50 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 20 Jul 2008 22:03:50 +0200
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
Message-ID: <20080720200350.GA31836@phare.normalesup.org>

On Sun, Jul 20, 2008 at 01:50:01PM -0600, Brian Granger wrote:
> I don't have any time to look at this now, but we should have an
> interface in the interpreter that allow the registration of a callback
> that gets called or these things.

OK. Well, there is no hurry, but when you have time it would be nice to
agree on an interface. For now my branch is not merge into trunk.

Ga?l


From gael.varoquaux at normalesup.org  Sun Jul 20 16:07:12 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 20 Jul 2008 22:07:12 +0200
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <7ced318f0807201256i7fb5826ah21455c5faf9a92b@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<7ced318f0807201256i7fb5826ah21455c5faf9a92b@mail.gmail.com>
Message-ID: <20080720200712.GB31836@phare.normalesup.org>

On Sun, Jul 20, 2008 at 09:56:23PM +0200, Laurent Dufrechou wrote:
> Hi gael,
> I've checked out your branch and done `python setup.py build_ext --inplace`
> Launching your app:

> laurent at Pep:~/ipython-sync-frontend/IPython$ python
> ./frontend/wx/wx_frontend.py
> Traceback (most recent call last):
>   File "./frontend/wx/wx_frontend.py", line 30, in <module>
>     from IPython.frontend.prefilterfrontend import PrefilterFrontEnd
> ImportError: No module named frontend.prefilterfrontend

Hey Laurent,

Thanks a lot for taking time to check this out. I had forgetten to add a
file in bzr. I have just corrected that and pushed a new version.

The prefilterfrontend.py file is the file where are concentred all the
really hacky tricks I have to play to get ipython1 working with magics,
tab completion, and all that. This is what Fernando and I are going to
use as a guide to drive our refactoring of ipython0 and ipython1 to get
the magics and co cleanely in ipython1.

Ga?l


From gael.varoquaux at normalesup.org  Sun Jul 20 16:14:14 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 20 Jul 2008 22:14:14 +0200
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <7ced318f0807201256i7fb5826ah21455c5faf9a92b@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<7ced318f0807201256i7fb5826ah21455c5faf9a92b@mail.gmail.com>
Message-ID: <20080720201414.GC31836@phare.normalesup.org>

On Sun, Jul 20, 2008 at 09:56:23PM +0200, Laurent Dufrechou wrote:
> I've checked out your branch and done `python setup.py build_ext --inplace`
> Launching your app:

Oh, by the way, Laurent, be careful, the frontend is currently not at all
windows friendly. When executing external commands that might ask for
terminal input there is a severe gotcha that I haven't managed to work
around (namely: how to you redirect stdin, stout, and stderr simultanous
to a process). As a result I falled back on starting an xterm. obvious
this works only on Linux. I am on conference right now, and do not have
windows or Macs to get a proper solution. I'll work on that when I get
back in Austin.

Cheers,

Ga?l


From laurent.dufrechou at gmail.com  Sun Jul 20 16:16:52 2008
From: laurent.dufrechou at gmail.com (Laurent Dufrechou)
Date: Sun, 20 Jul 2008 22:16:52 +0200
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <20080720201414.GC31836@phare.normalesup.org>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<7ced318f0807201256i7fb5826ah21455c5faf9a92b@mail.gmail.com>
	<20080720201414.GC31836@phare.normalesup.org>
Message-ID: <7ced318f0807201316v6faf9d47j9237bfb9252455f4@mail.gmail.com>

No problem, I'm under linux rigth now ;)

2008/7/20 Gael Varoquaux <gael.varoquaux at normalesup.org>:
> On Sun, Jul 20, 2008 at 09:56:23PM +0200, Laurent Dufrechou wrote:
>> I've checked out your br anch and done `python setup.py build_ext --inplace`
>> Launching your app:
>
> Oh, by the way, Laurent, be careful, the frontend is currently not at all
> windows friendly. When executing external commands that might ask for
> terminal input there is a severe gotcha that I haven't managed to work
> around (namely: how to you redirect stdin, stout, and stderr simultanous
> to a process). As a result I falled back on starting an xterm. obvious
> this works only on Linux. I am on conference right now, and do not have
> windows or Macs to get a proper solution. I'll work on that when I get
> back in Austin.
>
> Cheers,
>
> Ga?l
>


From gael.varoquaux at normalesup.org  Sun Jul 20 16:54:29 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 20 Jul 2008 22:54:29 +0200
Subject: [IPython-dev] Nicer display of exceptions with ipython1
Message-ID: <20080720205429.GD31836@phare.normalesup.org>

Currently, with the ipython1 interpreter, there are two ways of dealing
with exception: either you let the interpreter deal with them, and you
get the traceback in the return message, or you ask the interpreter to
raise them, and you deal with them yourself.

I have been doing the second, in my frontend, because I want my frontend
to be able to display more information about the traceback that there is
in the message. The problem with this is that I have to catch the
exception outside of the interpreter, thus the callstack contains
irrelevant information, for instance this is the kind of session I might
have:

In [1]: print i
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call
last)

/home/varoquau/dev/ipython/ipython-frontend/IPython/kernel/core/interpreter.pyc
in execute_block(self, code)
    373         outflag = 1 # start by assuming error, success will reset
it
    374         try:
--> 375             exec code in self.user_ns
    376             outflag = 00    377
except         SystemExit
:/home/varoquau/dev/ipython/ipython-frontend/IPython/frontend/wx/<string>

<module>() in 
----> 1 
      2
      3
      4
      5
 NameError

It seems to me that an elegant way to resolve this problem would be to
provide a callback to the interpreter to deal with traceback. I am
looking at the traceback_trap and trying to figure out if what I want
fits in this model or not, and I am not too sure it does. I think I want
something more flexible.

Should I go ahead and implement a callback somewhere? And if so, where?
In the traceback trap?

Cheers,

Ga?l


From ellisonbg.net at gmail.com  Sun Jul 20 17:52:05 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Sun, 20 Jul 2008 15:52:05 -0600
Subject: [IPython-dev] Nicer display of exceptions with ipython1
In-Reply-To: <20080720205429.GD31836@phare.normalesup.org>
References: <20080720205429.GD31836@phare.normalesup.org>
Message-ID: <6ce0ac130807201452u7fdff84agbd409ed14f326951@mail.gmail.com>

Yes, this is a generic problem that we have in IPython.kernel - all of
the user's exceptions get polluted with information from IPython
itself.

> It seems to me that an elegant way to resolve this problem would be to
> provide a callback to the interpreter to deal with traceback. I am
> looking at the traceback_trap and trying to figure out if what I want
> fits in this model or not, and I am not too sure it does. I think I want
> something more flexible.

In this case, I am not sure that a callback will really do what we
want.  One reason we have the option to allow the interpreter actually
raise the exception is that is how Python handles errors.  If we move
things to a callback, we sort of get stuck with our own non-exception
based error handling.  That would be very non-ideal.

I am not sure if it is possible, but I think the best solution is if
the traceback_trap can be smart and dynamically modify the traceback
to remove the unwanted information.  This is also needed even if the
interpreter is not raising the exception.  In that case, the
interpreter returns a dict that has the traceback as a string - with
the unwanted info as well.  Conclusion:  we need to figure out a way
of remove the unwanted info in the first place.

Also, isn't this what the callback would have to do anyway?  If so,
that logic should just be in the traceback_trap itself.

Cheers,

Brian

> Should I go ahead and implement a callback somewhere? And if so, where?
> In the traceback trap?



> Cheers,
>
> Ga?l
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From gael.varoquaux at normalesup.org  Sun Jul 20 18:13:53 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Mon, 21 Jul 2008 00:13:53 +0200
Subject: [IPython-dev] Nicer display of exceptions with ipython1
In-Reply-To: <6ce0ac130807201452u7fdff84agbd409ed14f326951@mail.gmail.com>
References: <20080720205429.GD31836@phare.normalesup.org>
	<6ce0ac130807201452u7fdff84agbd409ed14f326951@mail.gmail.com>
Message-ID: <20080720221353.GH31836@phare.normalesup.org>

On Sun, Jul 20, 2008 at 03:52:05PM -0600, Brian Granger wrote:
> In this case, I am not sure that a callback will really do what we
> want.  One reason we have the option to allow the interpreter actually
> raise the exception is that is how Python handles errors.  If we move
> things to a callback, we sort of get stuck with our own non-exception
> based error handling.  That would be very non-ideal.

Well, I think this needs to be thought of carefully, and we probably need
the option to reraise the exception anyhow.

> I am not sure if it is possible, but I think the best solution is if
> the traceback_trap can be smart and dynamically modify the traceback
> to remove the unwanted information.  This is also needed even if the
> interpreter is not raising the exception.  In that case, the
> interpreter returns a dict that has the traceback as a string - with
> the unwanted info as well.  Conclusion:  we need to figure out a way
> of remove the unwanted info in the first place.

Hum, I thought that if you catched the exception in the very function
where the exec call lies, like it is done in ipython0 (iplib.py, line
2083), this give you a cleaner traceback with no postprocessing required.
However, there must be an additional trick that ipython0 plays. It seems
that the code is executed in a virtual module. As a result the traceback
is very clean:

In [7]: print i
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call
last)

/home/varoquau/<ipython console> in <module>()

NameError: name 'i' is not defined

We need to figure out how this is done. I'll ask Fernando tomorrow.

> Also, isn't this what the callback would have to do anyway?  If so,
> that logic should just be in the traceback_trap itself.

Well, the callback could do more. In an IDE, for instance, you could have
an option to open the relevant file in an editor, or open a debugger, or
a stack inspector. This is why I am thinking of a callback. I can hardly
see how to account for all these possibilities without either resort to
some obfuscated code, or using a callback. Maybe I am missing the
obvious.

Cheers,

Ga?l


From ellisonbg.net at gmail.com  Sun Jul 20 19:47:35 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Sun, 20 Jul 2008 17:47:35 -0600
Subject: [IPython-dev] Nicer display of exceptions with ipython1
In-Reply-To: <20080720221353.GH31836@phare.normalesup.org>
References: <20080720205429.GD31836@phare.normalesup.org>
	<6ce0ac130807201452u7fdff84agbd409ed14f326951@mail.gmail.com>
	<20080720221353.GH31836@phare.normalesup.org>
Message-ID: <6ce0ac130807201647r77018438x1fa0839e4a55e245@mail.gmail.com>

>> I am not sure if it is possible, but I think the best solution is if
>> the traceback_trap can be smart and dynamically modify the traceback
>> to remove the unwanted information.  This is also needed even if the
>> interpreter is not raising the exception.  In that case, the
>> interpreter returns a dict that has the traceback as a string - with
>> the unwanted info as well.  Conclusion:  we need to figure out a way
>> of remove the unwanted info in the first place.
>
> Hum, I thought that if you catched the exception in the very function
> where the exec call lies, like it is done in ipython0 (iplib.py, line
> 2083), this give you a cleaner traceback with no postprocessing required.
> However, there must be an additional trick that ipython0 plays. It seems
> that the code is executed in a virtual module. As a result the traceback
> is very clean:
>
> In [7]: print i
> ---------------------------------------------------------------------------
> NameError                                 Traceback (most recent call
> last)
>
> /home/varoquau/<ipython console> in <module>()
>
> NameError: name 'i' is not defined
>
> We need to figure out how this is done. I'll ask Fernando tomorrow.

Sounds good, I think he knows that code much better than I do.

>> Also, isn't this what the callback would have to do anyway?  If so,
>> that logic should just be in the traceback_trap itself.
>
> Well, the callback could do more. In an IDE, for instance, you could have
> an option to open the relevant file in an editor, or open a debugger, or
> a stack inspector. This is why I am thinking of a callback. I can hardly
> see how to account for all these possibilities without either resort to
> some obfuscated code, or using a callback. Maybe I am missing the
> obvious.

Ah, yes, now I see why something like a callback would be nice.  But I
don't think we should do that only instead of figuring out how to get
a nicer traceback.  We should probably do both.

Brian

> Cheers,
>
> Ga?l
>


From gael.varoquaux at normalesup.org  Sun Jul 20 19:49:43 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Mon, 21 Jul 2008 01:49:43 +0200
Subject: [IPython-dev] Nicer display of exceptions with ipython1
In-Reply-To: <6ce0ac130807201647r77018438x1fa0839e4a55e245@mail.gmail.com>
References: <20080720205429.GD31836@phare.normalesup.org>
	<6ce0ac130807201452u7fdff84agbd409ed14f326951@mail.gmail.com>
	<20080720221353.GH31836@phare.normalesup.org>
	<6ce0ac130807201647r77018438x1fa0839e4a55e245@mail.gmail.com>
Message-ID: <20080720234943.GJ31836@phare.normalesup.org>

On Sun, Jul 20, 2008 at 05:47:35PM -0600, Brian Granger wrote:
> Ah, yes, now I see why something like a callback would be nice.  But I
> don't think we should do that only instead of figuring out how to get
> a nicer traceback.  We should probably do both.

Agreed. I'll work on that.

Ga?l


From gael.varoquaux at normalesup.org  Mon Jul 21 00:00:11 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Mon, 21 Jul 2008 06:00:11 +0200
Subject: [IPython-dev] History in ipython1
Message-ID: <20080721040011.GO31836@phare.normalesup.org>

I am about to implement saving history between session for my frontend.	I
cannot reuse the existing code, as it heavily relies on readline. I had a
look at the pyreadline code, and I don't think I am going to ruese it.
First of all, I would have to extract it from pylreadline, as pyreadline
cannot be installed on a non-windows OS (not only because of its
dependencies, but also because it overrides readline when it is
installed), second there is a lot of stuff I don't need in there.

Before doing this, I want to clean up the history code. I have the
impression that the current History objects where derived from the
history implementation I had sketched almost a year ago. In this
implementation the history cursor was 0 to indicate the last line, and
went up as you would go up the history. In the current version, althought
the docstrings haven't been modified, it is the other way around.

I prefer the way in which last=0, the reason been that with the
exception, and the various code path that can lead to an execution or a
new prompt, or the loading of history, it is easy to loose track of the
number of past executions. Of course, we can always fallback on our feet
by checking the length of the history, and resetting the cursor to that,
but it seems simpler to count from the back. How to people feel about
that?

In addition, I would like to add history loading in the history object,
throught a seperate method. No controverse?

Ga?l


From barrywark at gmail.com  Mon Jul 21 00:36:43 2008
From: barrywark at gmail.com (Barry Wark)
Date: Sun, 20 Jul 2008 21:36:43 -0700
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
Message-ID: <cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>

On Sun, Jul 20, 2008 at 12:50 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
> I don't have any time to look at this now, but we should have an
> interface in the interpreter that allow the registration of a callback
> that gets called or these things.
>
> Cheers,
>
> Brian

There are, I think, several "events" for which a frontend might want a
callback. Opening files, stdout/stderr output, and modifications to
the user_ns are some that come immediately to mind. Opening files
might be of interest if the frontend wants to notify the user that
there is file output to be retrieved from the engine, stdout/stderr
for the resason Gael outlines, and modifications to the user_ns to
allow the frontend to keep an up-to-date view of the user_ns (ala
Matlab's Workspace browser) without querying the the entire engine
user_ns after every executed block.

Apple's NSDistributedNotificationCenter API
(http://developer.apple.com/documentation/Cocoa/Reference/Foundation/Classes/NSDistributedNotificationCenter_Class/Reference/Reference.html)
seems like a good pattern for this task (I'm sure there are equivalent
APIs in other frameworks). Basically, the frontend can register for
notifications by name. The interpreter then fires notifications for
all events and they are passed on, by the notification center, to all
registered observers.

Just my 2c to start the discussion.

Barry

>
> On Sun, Jul 20, 2008 at 1:27 PM, Gael Varoquaux
> <gael.varoquaux at normalesup.org> wrote:
>>> I can off course monkey patch the output_trap with my own write method,
>>> but this seems a bit ugly. What do the ipython1 developpers think the
>>> right way of doing this (and maybe modifying output_trap) is?
>>
>> I went ahead experimenting, and created a subclass of output_trap gear
>> towards printing via a callback, as can be seen on
>> http://bazaar.launchpad.net/~gael-varoquaux/ipython/ipython-sync-frontend/annotate/1041?file_id=sync_output_trap.py-20080720174256-b1fz1ebbxl2ka4t9-1
>>
>> I hope this approach is OK. By passing this this output_trap to the
>> interpreter I get the desired result, and I sync this approach is
>> reusable for any synchronous frontend, such as the one into which
>> ipython0 will, hopefully, morph one day.
>>
>> Ga?l
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From gael.varoquaux at normalesup.org  Mon Jul 21 00:50:21 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Mon, 21 Jul 2008 06:50:21 +0200
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
	<cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
Message-ID: <20080721045021.GP31836@phare.normalesup.org>

On Sun, Jul 20, 2008 at 09:36:43PM -0700, Barry Wark wrote:
> There are, I think, several "events" for which a frontend might want a
> callback. Opening files,

Hum, in what sens? What would trigger this event?

> stdout/stderr output,

Sure, this is why I came up with SyncOutputTrap, that allows me to pass a
callback to write. I add a wx.Yield() in this calback and my UI can
refresh each time you write something to stdout/stdin.

> and modifications to the user_ns are some that come immediately to
> mind. Opening files might be of interest if the frontend wants to
> notify the user that there is file output to be retrieved from the
> engine, stdout/stderr for the resason Gael outlines, and modifications
> to the user_ns to allow the frontend to keep an up-to-date view of the
> user_ns (ala Matlab's Workspace browser) without querying the the
> entire engine user_ns after every executed block.

Hum, I am not too excited about firing events when the namespace gets
updated. This could slow things down a lot. I general I am not too
excited about having events fired all over the place. This can lead to
tedious code with getters and setters and a udge performance loss. I am
much more interested in using objects with a standard interface (such as
a file, for instance) and then, if you want getters and setters, or
the possibility to add callabcks, you do it in these objects. 

> Apple's NSDistributedNotificationCenter API
> (http://developer.apple.com/documentation/Cocoa/Reference/Foundation/Classes/NSDistributedNotificationCenter_Class/Reference/Reference.html)
> seems like a good pattern for this task (I'm sure there are equivalent
> APIs in other frameworks). Basically, the frontend can register for
> notifications by name. The interpreter then fires notifications for all
> events and they are passed on, by the notification center, to all
> registered observers.

Well, I don't really like this kind of code. I find it clunky, and you
end up have notification registrations all over the place (reminds me of
wx). I actually prefer a lot the traits model, or the VTK model, AFAIK,
where objects can be there own observers. As a result you do not have
notifications blending in objects where they do not belong. For the core
frontends, obviously, we do not want to add a dependence to traits, but I
must imagine I cannot imagine going much further without it. To take your
example of updating the namespace view, all you have to do is to pass a
traited dict, instead of a dict, to the interpreter as a namespace. Then
you can have your namespace update automatically (AFAIK Robert is
actually working on a branch where he does that).

If we start adding callbacks whereever we need them, I am worried we'll
end up with callbacks everywhere. I think we need to add callback only
where they are absolutely necessary.

My two cents,

Ga?l


From vivainio at gmail.com  Mon Jul 21 11:32:30 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 21 Jul 2008 18:32:30 +0300
Subject: [IPython-dev] History in ipython1
In-Reply-To: <20080721040011.GO31836@phare.normalesup.org>
References: <20080721040011.GO31836@phare.normalesup.org>
Message-ID: <46cb515a0807210832g3322a398kddba126ba7ecaa2a@mail.gmail.com>

On Mon, Jul 21, 2008 at 7:00 AM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:

> number of past executions. Of course, we can always fallback on our feet
> by checking the length of the history, and resetting the cursor to that,
> but it seems simpler to count from the back. How to people feel about
> that?

It would seem simpler to have 0 as the first element, mostly to ensure
that item number 3 will always be the same item (i.e. stable
numbering).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From gael.varoquaux at normalesup.org  Mon Jul 21 23:47:31 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 22 Jul 2008 05:47:31 +0200
Subject: [IPython-dev] History in ipython1
In-Reply-To: <46cb515a0807210832g3322a398kddba126ba7ecaa2a@mail.gmail.com>
References: <20080721040011.GO31836@phare.normalesup.org>
	<46cb515a0807210832g3322a398kddba126ba7ecaa2a@mail.gmail.com>
Message-ID: <20080722034731.GF5258@phare.normalesup.org>

On Mon, Jul 21, 2008 at 06:32:30PM +0300, Ville M. Vainio wrote:
> On Mon, Jul 21, 2008 at 7:00 AM, Gael Varoquaux
> <gael.varoquaux at normalesup.org> wrote:

> > number of past executions. Of course, we can always fallback on our feet
> > by checking the length of the history, and resetting the cursor to that,
> > but it seems simpler to count from the back. How to people feel about
> > that?

> It would seem simpler to have 0 as the first element, mostly to ensure
> that item number 3 will always be the same item (i.e. stable
> numbering).

Hum, as you load a history, you prepend things to the beginning of the
list, so item number 3 will stop being item number 3. I think of history
as "item 3 lines ago".

Ga?l


From fperez.net at gmail.com  Tue Jul 22 04:20:43 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 22 Jul 2008 01:20:43 -0700
Subject: [IPython-dev] Nicer display of exceptions with ipython1
In-Reply-To: <6ce0ac130807201647r77018438x1fa0839e4a55e245@mail.gmail.com>
References: <20080720205429.GD31836@phare.normalesup.org>
	<6ce0ac130807201452u7fdff84agbd409ed14f326951@mail.gmail.com>
	<20080720221353.GH31836@phare.normalesup.org>
	<6ce0ac130807201647r77018438x1fa0839e4a55e245@mail.gmail.com>
Message-ID: <db6b5ecc0807220120u30b42cd3obb831edc66afe781@mail.gmail.com>

On Sun, Jul 20, 2008 at 4:47 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:

>> We need to figure out how this is done. I'll ask Fernando tomorrow.
>
> Sounds good, I think he knows that code much better than I do.

Done.  There's a parameter in the traceback handlers for the number of
stack frames to skip.

>>> Also, isn't this what the callback would have to do anyway?  If so,
>>> that logic should just be in the traceback_trap itself.
>>
>> Well, the callback could do more. In an IDE, for instance, you could have
>> an option to open the relevant file in an editor, or open a debugger, or
>> a stack inspector. This is why I am thinking of a callback. I can hardly
>> see how to account for all these possibilities without either resort to
>> some obfuscated code, or using a callback. Maybe I am missing the
>> obvious.
>
> Ah, yes, now I see why something like a callback would be nice.  But I
> don't think we should do that only instead of figuring out how to get
> a nicer traceback.  We should probably do both.

Extending the exception handling API to  optionally take a callback is
certainly possible, and I can see valid uses for it.

Cheers,

f


From barrywark at gmail.com  Tue Jul 22 10:36:14 2008
From: barrywark at gmail.com (Barry Wark)
Date: Tue, 22 Jul 2008 09:36:14 -0500
Subject: [IPython-dev] History in ipython1
In-Reply-To: <20080722034731.GF5258@phare.normalesup.org>
References: <20080721040011.GO31836@phare.normalesup.org>
	<46cb515a0807210832g3322a398kddba126ba7ecaa2a@mail.gmail.com>
	<20080722034731.GF5258@phare.normalesup.org>
Message-ID: <cd7634ce0807220736w1d25aae0l829f788060ba1f88@mail.gmail.com>

On Mon, Jul 21, 2008 at 10:47 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> On Mon, Jul 21, 2008 at 06:32:30PM +0300, Ville M. Vainio wrote:
>> On Mon, Jul 21, 2008 at 7:00 AM, Gael Varoquaux
>> <gael.varoquaux at normalesup.org> wrote:
>
>> > number of past executions. Of course, we can always fallback on our feet
>> > by checking the length of the history, and resetting the cursor to that,
>> > but it seems simpler to count from the back. How to people feel about
>> > that?
>
>> It would seem simpler to have 0 as the first element, mostly to ensure
>> that item number 3 will always be the same item (i.e. stable
>> numbering).
>
> Hum, as you load a history, you prepend things to the beginning of the
> list, so item number 3 will stop being item number 3. I think of history
> as "item 3 lines ago".

It seems this is a frontend-specific issue. A notebook interface would
want to think of history counting forwards in time from 0 whereas a
terminal interface might want to think of history going backwards in
time from 0.

>
> Ga?l
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From ellisonbg.net at gmail.com  Tue Jul 22 12:00:32 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 22 Jul 2008 10:00:32 -0600
Subject: [IPython-dev] History in ipython1
In-Reply-To: <cd7634ce0807220736w1d25aae0l829f788060ba1f88@mail.gmail.com>
References: <20080721040011.GO31836@phare.normalesup.org>
	<46cb515a0807210832g3322a398kddba126ba7ecaa2a@mail.gmail.com>
	<20080722034731.GF5258@phare.normalesup.org>
	<cd7634ce0807220736w1d25aae0l829f788060ba1f88@mail.gmail.com>
Message-ID: <6ce0ac130807220900o4f87f8day20bb9b7571d06ea0@mail.gmail.com>

Also, I think the history of history is important:

In [1]: a = 10

In [2]: b = 20

In [3]: history
1: a = 10
2: b = 20
3: _ip.magic("history ")

Any departure from this will probably confuse developers and users.

Brian

On Tue, Jul 22, 2008 at 8:36 AM, Barry Wark <barrywark at gmail.com> wrote:
> On Mon, Jul 21, 2008 at 10:47 PM, Gael Varoquaux
> <gael.varoquaux at normalesup.org> wrote:
>> On Mon, Jul 21, 2008 at 06:32:30PM +0300, Ville M. Vainio wrote:
>>> On Mon, Jul 21, 2008 at 7:00 AM, Gael Varoquaux
>>> <gael.varoquaux at normalesup.org> wrote:
>>
>>> > number of past executions. Of course, we can always fallback on our feet
>>> > by checking the length of the history, and resetting the cursor to that,
>>> > but it seems simpler to count from the back. How to people feel about
>>> > that?
>>
>>> It would seem simpler to have 0 as the first element, mostly to ensure
>>> that item number 3 will always be the same item (i.e. stable
>>> numbering).
>>
>> Hum, as you load a history, you prepend things to the beginning of the
>> list, so item number 3 will stop being item number 3. I think of history
>> as "item 3 lines ago".
>
> It seems this is a frontend-specific issue. A notebook interface would
> want to think of history counting forwards in time from 0 whereas a
> terminal interface might want to think of history going backwards in
> time from 0.
>
>>
>> Ga?l
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From gael.varoquaux at normalesup.org  Tue Jul 22 12:50:43 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 22 Jul 2008 18:50:43 +0200
Subject: [IPython-dev] History in ipython1
In-Reply-To: <6ce0ac130807220900o4f87f8day20bb9b7571d06ea0@mail.gmail.com>
References: <20080721040011.GO31836@phare.normalesup.org>
	<46cb515a0807210832g3322a398kddba126ba7ecaa2a@mail.gmail.com>
	<20080722034731.GF5258@phare.normalesup.org>
	<cd7634ce0807220736w1d25aae0l829f788060ba1f88@mail.gmail.com>
	<6ce0ac130807220900o4f87f8day20bb9b7571d06ea0@mail.gmail.com>
Message-ID: <20080722165043.GA9152@phare.normalesup.org>

On Tue, Jul 22, 2008 at 10:00:32AM -0600, Brian Granger wrote:
> In [3]: history
> 1: a = 10
> 2: b = 20
> 3: _ip.magic("history ")

> Any departure from this will probably confuse developers and users.

OK, so lets stick to the way it is currenlty implemented. I just wanted
to have this discussion rather early than late.

Ga?l


From barrywark at gmail.com  Tue Jul 22 14:05:00 2008
From: barrywark at gmail.com (Barry Wark)
Date: Tue, 22 Jul 2008 13:05:00 -0500
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <20080721045021.GP31836@phare.normalesup.org>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
	<cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
	<20080721045021.GP31836@phare.normalesup.org>
Message-ID: <cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>

As always, I think working code is better than good theory, so please
read the following with that in mind. I think that the opinion you
express below may put us on the road to an architectural mistake in
the redesign of ipython. My understanding of the historical context is
that the current ipython0 interpreter is direclty tied to the frontend
in a way that makes new frontends or stand-alone interpreters
difficult. The solution, as embodied by the the ipython1 codebase and
the evolving IPython.frontend package is to establish a separation
between interpreter and frontend to prevent the type of dependency
that exists in the current code base.

Now, back to the issue at hand. In developing frontends using the new
architecture, you have noticed that there are a lot of things that the
new system is missing. In general, these fall into the category of the
frontend wanting to monitor and/or modify execution in the interpreter
at a more fine-grained level than entire code blocks (the only option
currently). There is a standard pattern for handling this type of
notification, the Observer pattern, and this type of fine-grained
modification, the Delegate pattern. Propper implementation of either
produces minimal overhead (and it's worth remembering that by writing
IPython in python instead of a lower level language, we've all already
stated our preference for abstraction over raw efficiency) [1]. The
alternative, that I believe you are proposing below and in the current
thread on adding a callback to the exception handling code in the
interpreter, is to add callback hooks to the interpreter so that one
(or more?) callbacks can be attached to specific events in the
interpreter's execution flow.

In fact, I think these are not really different solutions, just
different implementations of a similar idea. You propose handling
observer notification from within the shell, whereas I propose
handling it in an intermediary entity (the notification center). The
callback approach is more explicit -- the interpreter must explicitly
expose a callback hook and the frontend must explicitly register *with
the interpreter* for that callback -- but less general. Unless we
create a callback registry etc. in the shell, essentially duplicating
the notification center idea, then when we want to add a new
notification (e.g. for exceptions as in the currently running thread),
we will need to duplicate the callback calling code in the shell
rather than just marking that event for notification by the
notification center. The observer/delegate combination allows the
interpreter to remain agnostic wrt the interface of the frontend and
vice-versa (and huge win, in my opinion). By adding an intermediary --
the notification center and/or delegate dispatcher -- we avoid the
issue of frontends and interpreters being closely tied by contract and
make it easier to provide the same interface directly (between
frontend and shell as in your Wx frontend) or locally or remotely via
a twisted service (as between and engine and the frontend in my Cocoa
frontend).

I know that you're on a deadline and need to write code, but I would
urge a little bit of restraint and planning before you commit to the
callback-only approach. I think avoiding this type of dependency
between shell and frontend will benefit us in the long run.

Barry

[1] For example, firing a notification when no observers are
registered for that notification should not incur significant overhead
beyond a couple of method calls, and perhaps a dictionary lookup, even
in a naive implementation.

On Sun, Jul 20, 2008 at 9:50 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> On Sun, Jul 20, 2008 at 09:36:43PM -0700, Barry Wark wrote:
>> There are, I think, several "events" for which a frontend might want a
>> callback. Opening files,
>
> Hum, in what sens? What would trigger this event?
>
>> stdout/stderr output,
>
> Sure, this is why I came up with SyncOutputTrap, that allows me to pass a
> callback to write. I add a wx.Yield() in this calback and my UI can
> refresh each time you write something to stdout/stdin.
>
>> and modifications to the user_ns are some that come immediately to
>> mind. Opening files might be of interest if the frontend wants to
>> notify the user that there is file output to be retrieved from the
>> engine, stdout/stderr for the resason Gael outlines, and modifications
>> to the user_ns to allow the frontend to keep an up-to-date view of the
>> user_ns (ala Matlab's Workspace browser) without querying the the
>> entire engine user_ns after every executed block.
>
> Hum, I am not too excited about firing events when the namespace gets
> updated. This could slow things down a lot. I general I am not too
> excited about having events fired all over the place. This can lead to
> tedious code with getters and setters and a udge performance loss. I am
> much more interested in using objects with a standard interface (such as
> a file, for instance) and then, if you want getters and setters, or
> the possibility to add callabcks, you do it in these objects.
>
>> Apple's NSDistributedNotificationCenter API
>> (http://developer.apple.com/documentation/Cocoa/Reference/Foundation/Classes/NSDistributedNotificationCenter_Class/Reference/Reference.html)
>> seems like a good pattern for this task (I'm sure there are equivalent
>> APIs in other frameworks). Basically, the frontend can register for
>> notifications by name. The interpreter then fires notifications for all
>> events and they are passed on, by the notification center, to all
>> registered observers.
>
> Well, I don't really like this kind of code. I find it clunky, and you
> end up have notification registrations all over the place (reminds me of
> wx). I actually prefer a lot the traits model, or the VTK model, AFAIK,
> where objects can be there own observers. As a result you do not have
> notifications blending in objects where they do not belong. For the core
> frontends, obviously, we do not want to add a dependence to traits, but I
> must imagine I cannot imagine going much further without it. To take your
> example of updating the namespace view, all you have to do is to pass a
> traited dict, instead of a dict, to the interpreter as a namespace. Then
> you can have your namespace update automatically (AFAIK Robert is
> actually working on a branch where he does that).
>
> If we start adding callbacks whereever we need them, I am worried we'll
> end up with callbacks everywhere. I think we need to add callback only
> where they are absolutely necessary.
> My two cents,
>
> Ga?l
>


From fperez.net at gmail.com  Tue Jul 22 14:24:26 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 22 Jul 2008 11:24:26 -0700
Subject: [IPython-dev] History in ipython1
In-Reply-To: <6ce0ac130807220900o4f87f8day20bb9b7571d06ea0@mail.gmail.com>
References: <20080721040011.GO31836@phare.normalesup.org>
	<46cb515a0807210832g3322a398kddba126ba7ecaa2a@mail.gmail.com>
	<20080722034731.GF5258@phare.normalesup.org>
	<cd7634ce0807220736w1d25aae0l829f788060ba1f88@mail.gmail.com>
	<6ce0ac130807220900o4f87f8day20bb9b7571d06ea0@mail.gmail.com>
Message-ID: <db6b5ecc0807221124k2ca11229t6ea5f7645ab75c35@mail.gmail.com>

On Tue, Jul 22, 2008 at 9:00 AM, Brian Granger <ellisonbg.net at gmail.com> wrote:
> Also, I think the history of history is important:
>
> In [1]: a = 10
>
> In [2]: b = 20
>
> In [3]: history
> 1: a = 10
> 2: b = 20
> 3: _ip.magic("history ")
>
> Any departure from this will probably confuse developers and users.

Yes, esp. because there are things like %macro and %edit that use
those numbers, as well as the fact that the In[] list-like object is
also fully accessible and can be used at run-time, and it relies on
the same indexing mode:

In [1]: x = 1

In [2]: x+=1

In [3]: print x
2

In [4]: exec In[2:4]
3

In [6]: exec In[2:4]
4

In [7]: exec In[2:4]
5


Cheers,

f


From ellisonbg.net at gmail.com  Tue Jul 22 16:15:59 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 22 Jul 2008 14:15:59 -0600
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <20080721045021.GP31836@phare.normalesup.org>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
	<cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
	<20080721045021.GP31836@phare.normalesup.org>
Message-ID: <6ce0ac130807221315p386a629avb87d6cd5feba57f4@mail.gmail.com>

> Well, I don't really like this kind of code. I find it clunky, and you
> end up have notification registrations all over the place (reminds me of
> wx). I actually prefer a lot the traits model, or the VTK model, AFAIK,
> where objects can be there own observers. As a result you do not have
> notifications blending in objects where they do not belong. For the core
> frontends, obviously, we do not want to add a dependence to traits, but I
> must imagine I cannot imagine going much further without it. To take your
> example of updating the namespace view, all you have to do is to pass a
> traited dict, instead of a dict, to the interpreter as a namespace. Then
> you can have your namespace update automatically (AFAIK Robert is
> actually working on a branch where he does that).

I understand the benefits of traits, but we have already made a firm
decision to keep Traits (or any other compiled code for that matter)
out of the core of IPython.  IPython has to be able to run (at least
in principal) on non CPython implementation of Python.  Obviously, a
GUI frontend can use traits, wx, qt, etc., and the kernel relies on
Twisted which has compiled code.  But I it almost sounds like you are
proposing using traits in the interpreter class itself.

I am all for clever uses of Traits like Robert Kern has been thinking
about.  But, we do need a more general approach that doesn't rely on
Traits.

> If we start adding callbacks whereever we need them, I am worried we'll
> end up with callbacks everywhere. I think we need to add callback only
> where they are absolutely necessary.

I tend to agree with this.  A bunch of random disconnected callbacks
makes an interface very disjoint.  See my upcoming reply to Barry to
the rest of my thoughts...

Cheers,

Brian

> My two cents,
>
> Ga?l
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From ellisonbg.net at gmail.com  Tue Jul 22 16:25:45 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 22 Jul 2008 14:25:45 -0600
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
	<cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
	<20080721045021.GP31836@phare.normalesup.org>
	<cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>
Message-ID: <6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>

> As always, I think working code is better than good theory, so please
> read the following with that in mind. I think that the opinion you
> express below may put us on the road to an architectural mistake in
> the redesign of ipython. My understanding of the historical context is
> that the current ipython0 interpreter is direclty tied to the frontend
> in a way that makes new frontends or stand-alone interpreters
> difficult. The solution, as embodied by the the ipython1 codebase and
> the evolving IPython.frontend package is to establish a separation
> between interpreter and frontend to prevent the type of dependency
> that exists in the current code base.

One thing that is absolutely critical is loose coupling.  All the
parts of IPython need to depend only very weakly on each other, and
interact through clean, standardized interfaces (I don't mean this
word in the script zope.interface sense though).  The big problem with
the traditional IPython codebase is that everything is welded
together.  The question is what types of design patterns will make
this possible.

>From this thread, I see a few options:

1.  Traits.  While I really like the Traits model and I think it is
highly appropriate for GUI programming, I think it would lead to too
tightly coupled code.  Also, we can't use Traits in the core of
ipython because it has compiled code.

2.  Plain callbacks.  At least a couple of us seem to think that an
interface that has lots of callbacks becomes very cumbersome.

3.  The Observer/Delegate patterns as are typical is Cocoa.  I know
that Gael is not fund of this, but I do think this pattern is much
better than plain callbacks, as it give a nice formal structure to
everything.  Also, I agree with Barry that this pattern is pretty much
what you get when you try to do plain callbacks properly.

Thus, my own conclusion is that the Observer/Delegate pattern is
probably the best option for us and will lead to the loose coupling
that we need.

Barry, what would it take to put together an example of this pattern?
Will we need some amount of infrastructure to make this possible?

Cheers,

Brian

> Now, back to the issue at hand. In developing frontends using the new
> architecture, you have noticed that there are a lot of things that the
> new system is missing. In general, these fall into the category of the
> frontend wanting to monitor and/or modify execution in the interpreter
> at a more fine-grained level than entire code blocks (the only option
> currently). There is a standard pattern for handling this type of
> notification, the Observer pattern, and this type of fine-grained
> modification, the Delegate pattern. Propper implementation of either
> produces minimal overhead (and it's worth remembering that by writing
> IPython in python instead of a lower level language, we've all already
> stated our preference for abstraction over raw efficiency) [1]. The
> alternative, that I believe you are proposing below and in the current
> thread on adding a callback to the exception handling code in the
> interpreter, is to add callback hooks to the interpreter so that one
> (or more?) callbacks can be attached to specific events in the
> interpreter's execution flow.
>
> In fact, I think these are not really different solutions, just
> different implementations of a similar idea. You propose handling
> observer notification from within the shell, whereas I propose
> handling it in an intermediary entity (the notification center). The
> callback approach is more explicit -- the interpreter must explicitly
> expose a callback hook and the frontend must explicitly register *with
> the interpreter* for that callback -- but less general. Unless we
> create a callback registry etc. in the shell, essentially duplicating
> the notification center idea, then when we want to add a new
> notification (e.g. for exceptions as in the currently running thread),
> we will need to duplicate the callback calling code in the shell
> rather than just marking that event for notification by the
> notification center. The observer/delegate combination allows the
> interpreter to remain agnostic wrt the interface of the frontend and
> vice-versa (and huge win, in my opinion). By adding an intermediary --
> the notification center and/or delegate dispatcher -- we avoid the
> issue of frontends and interpreters being closely tied by contract and
> make it easier to provide the same interface directly (between
> frontend and shell as in your Wx frontend) or locally or remotely via
> a twisted service (as between and engine and the frontend in my Cocoa
> frontend).
>
> I know that you're on a deadline and need to write code, but I would
> urge a little bit of restraint and planning before you commit to the
> callback-only approach. I think avoiding this type of dependency
> between shell and frontend will benefit us in the long run.
>
> Barry
>
> [1] For example, firing a notification when no observers are
> registered for that notification should not incur significant overhead
> beyond a couple of method calls, and perhaps a dictionary lookup, even
> in a naive implementation.
>
> On Sun, Jul 20, 2008 at 9:50 PM, Gael Varoquaux
> <gael.varoquaux at normalesup.org> wrote:
>> On Sun, Jul 20, 2008 at 09:36:43PM -0700, Barry Wark wrote:
>>> There are, I think, several "events" for which a frontend might want a
>>> callback. Opening files,
>>
>> Hum, in what sens? What would trigger this event?
>>
>>> stdout/stderr output,
>>
>> Sure, this is why I came up with SyncOutputTrap, that allows me to pass a
>> callback to write. I add a wx.Yield() in this calback and my UI can
>> refresh each time you write something to stdout/stdin.
>>
>>> and modifications to the user_ns are some that come immediately to
>>> mind. Opening files might be of interest if the frontend wants to
>>> notify the user that there is file output to be retrieved from the
>>> engine, stdout/stderr for the resason Gael outlines, and modifications
>>> to the user_ns to allow the frontend to keep an up-to-date view of the
>>> user_ns (ala Matlab's Workspace browser) without querying the the
>>> entire engine user_ns after every executed block.
>>
>> Hum, I am not too excited about firing events when the namespace gets
>> updated. This could slow things down a lot. I general I am not too
>> excited about having events fired all over the place. This can lead to
>> tedious code with getters and setters and a udge performance loss. I am
>> much more interested in using objects with a standard interface (such as
>> a file, for instance) and then, if you want getters and setters, or
>> the possibility to add callabcks, you do it in these objects.
>>
>>> Apple's NSDistributedNotificationCenter API
>>> (http://developer.apple.com/documentation/Cocoa/Reference/Foundation/Classes/NSDistributedNotificationCenter_Class/Reference/Reference.html)
>>> seems like a good pattern for this task (I'm sure there are equivalent
>>> APIs in other frameworks). Basically, the frontend can register for
>>> notifications by name. The interpreter then fires notifications for all
>>> events and they are passed on, by the notification center, to all
>>> registered observers.
>>
>> Well, I don't really like this kind of code. I find it clunky, and you
>> end up have notification registrations all over the place (reminds me of
>> wx). I actually prefer a lot the traits model, or the VTK model, AFAIK,
>> where objects can be there own observers. As a result you do not have
>> notifications blending in objects where they do not belong. For the core
>> frontends, obviously, we do not want to add a dependence to traits, but I
>> must imagine I cannot imagine going much further without it. To take your
>> example of updating the namespace view, all you have to do is to pass a
>> traited dict, instead of a dict, to the interpreter as a namespace. Then
>> you can have your namespace update automatically (AFAIK Robert is
>> actually working on a branch where he does that).
>>
>> If we start adding callbacks whereever we need them, I am worried we'll
>> end up with callbacks everywhere. I think we need to add callback only
>> where they are absolutely necessary.
>> My two cents,
>>
>> Ga?l
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From barrywark at gmail.com  Wed Jul 23 12:17:46 2008
From: barrywark at gmail.com (Barry Wark)
Date: Wed, 23 Jul 2008 11:17:46 -0500
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
	<cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
	<20080721045021.GP31836@phare.normalesup.org>
	<cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>
	<6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
Message-ID: <8C8A5DF0-F0A1-4294-90F0-BA2F4CE277BB@gmail.com>



On Jul 22, 2008, at 3:25 PM, "Brian Granger" <ellisonbg.net at gmail.com>  
wrote:

>> As always, I think working code is better than good theory, so please
>> read the following with that in mind. I think that the opinion you
>> express below may put us on the road to an architectural mistake in
>> the redesign of ipython. My understanding of the historical context  
>> is
>> that the current ipython0 interpreter is direclty tied to the  
>> frontend
>> in a way that makes new frontends or stand-alone interpreters
>> difficult. The solution, as embodied by the the ipython1 codebase and
>> the evolving IPython.frontend package is to establish a separation
>> between interpreter and frontend to prevent the type of dependency
>> that exists in the current code base.
>
> One thing that is absolutely critical is loose coupling.  All the
> parts of IPython need to depend only very weakly on each other, and
> interact through clean, standardized interfaces (I don't mean this
> word in the script zope.interface sense though).  The big problem with
> the traditional IPython codebase is that everything is welded
> together.  The question is what types of design patterns will make
> this possible.
>
> From this thread, I see a few options:
>
> 1.  Traits.  While I really like the Traits model and I think it is
> highly appropriate for GUI programming, I think it would lead to too
> tightly coupled code.  Also, we can't use Traits in the core of
> ipython because it has compiled code.
>
> 2.  Plain callbacks.  At least a couple of us seem to think that an
> interface that has lots of callbacks becomes very cumbersome.
>
> 3.  The Observer/Delegate patterns as are typical is Cocoa.  I know
> that Gael is not fund of this, but I do think this pattern is much
> better than plain callbacks, as it give a nice formal structure to
> everything.  Also, I agree with Barry that this pattern is pretty much
> what you get when you try to do plain callbacks properly.
>
> Thus, my own conclusion is that the Observer/Delegate pattern is
> probably the best option for us and will lead to the loose coupling
> that we need.
>
> Barry, what would it take to put together an example of this pattern?
> Will we need some amount of infrastructure to make this possible?

I'd be happy to write up an example and a draft implementation.  I'm  
going to be mostly offline, traveling, until the weekend. I'm most  
familiar with the Cocoa implementation so I'd like to do a bit of  
research of alternatives before I write the spec. Therefore next week  
is the realistic timeline.

Off the top of my head, I think a synchronous notification center  
implementation in I.k.core and an addition to the engineservice API  
along with an asynchronous notification center impl in I.kernel would  
cover the use cases. I will write up a functional/technical spec and  
solicit feedback before I make any decisions though.

Besides the notificatipn center implementation(s), I don't think any  
more infrastructure is required beyond modifying the interpreter to  
actually fire the appropriate notifications (support for obvious  
callbacks such as Gael's use cases should be easy; the notifying  
worskspace ala Robert's work will obviously require more effort). As  
you pointed out, this is really just carefully implemented callbacks  
with a loose coupling.

I'm glad we're dealing with this issue early. I think getting it right  
will pay big dividends in the future.

Cheers,
Barry

>
>
> Cheers,
>
> Brian
>
>> Now, back to the issue at hand. In developing frontends using the new
>> architecture, you have noticed that there are a lot of things that  
>> the
>> new system is missing. In general, these fall into the category of  
>> the
>> frontend wanting to monitor and/or modify execution in the  
>> interpreter
>> at a more fine-grained level than entire code blocks (the only option
>> currently). There is a standard pattern for handling this type of
>> notification, the Observer pattern, and this type of fine-grained
>> modification, the Delegate pattern. Propper implementation of either
>> produces minimal overhead (and it's worth remembering that by writing
>> IPython in python instead of a lower level language, we've all  
>> already
>> stated our preference for abstraction over raw efficiency) [1]. The
>> alternative, that I believe you are proposing below and in the  
>> current
>> thread on adding a callback to the exception handling code in the
>> interpreter, is to add callback hooks to the interpreter so that one
>> (or more?) callbacks can be attached to specific events in the
>> interpreter's execution flow.
>>
>> In fact, I think these are not really different solutions, just
>> different implementations of a similar idea. You propose handling
>> observer notification from within the shell, whereas I propose
>> handling it in an intermediary entity (the notification center). The
>> callback approach is more explicit -- the interpreter must explicitly
>> expose a callback hook and the frontend must explicitly register  
>> *with
>> the interpreter* for that callback -- but less general. Unless we
>> create a callback registry etc. in the shell, essentially duplicating
>> the notification center idea, then when we want to add a new
>> notification (e.g. for exceptions as in the currently running  
>> thread),
>> we will need to duplicate the callback calling code in the shell
>> rather than just marking that event for notification by the
>> notification center. The observer/delegate combination allows the
>> interpreter to remain agnostic wrt the interface of the frontend and
>> vice-versa (and huge win, in my opinion). By adding an intermediary  
>> --
>> the notification center and/or delegate dispatcher -- we avoid the
>> issue of frontends and interpreters being closely tied by contract  
>> and
>> make it easier to provide the same interface directly (between
>> frontend and shell as in your Wx frontend) or locally or remotely via
>> a twisted service (as between and engine and the frontend in my Cocoa
>> frontend).
>>
>> I know that you're on a deadline and need to write code, but I would
>> urge a little bit of restraint and planning before you commit to the
>> callback-only approach. I think avoiding this type of dependency
>> between shell and frontend will benefit us in the long run.
>>
>> Barry
>>
>> [1] For example, firing a notification when no observers are
>> registered for that notification should not incur significant  
>> overhead
>> beyond a couple of method calls, and perhaps a dictionary lookup,  
>> even
>> in a naive implementation.
>>
>> On Sun, Jul 20, 2008 at 9:50 PM, Gael Varoquaux
>> <gael.varoquaux at normalesup.org> wrote:
>>> On Sun, Jul 20, 2008 at 09:36:43PM -0700, Barry Wark wrote:
>>>> There are, I think, several "events" for which a frontend might  
>>>> want a
>>>> callback. Opening files,
>>>
>>> Hum, in what sens? What would trigger this event?
>>>
>>>> stdout/stderr output,
>>>
>>> Sure, this is why I came up with SyncOutputTrap, that allows me to  
>>> pass a
>>> callback to write. I add a wx.Yield() in this calback and my UI can
>>> refresh each time you write something to stdout/stdin.
>>>
>>>> and modifications to the user_ns are some that come immediately to
>>>> mind. Opening files might be of interest if the frontend wants to
>>>> notify the user that there is file output to be retrieved from the
>>>> engine, stdout/stderr for the resason Gael outlines, and  
>>>> modifications
>>>> to the user_ns to allow the frontend to keep an up-to-date view  
>>>> of the
>>>> user_ns (ala Matlab's Workspace browser) without querying the the
>>>> entire engine user_ns after every executed block.
>>>
>>> Hum, I am not too excited about firing events when the namespace  
>>> gets
>>> updated. This could slow things down a lot. I general I am not too
>>> excited about having events fired all over the place. This can  
>>> lead to
>>> tedious code with getters and setters and a udge performance loss.  
>>> I am
>>> much more interested in using objects with a standard interface  
>>> (such as
>>> a file, for instance) and then, if you want getters and setters, or
>>> the possibility to add callabcks, you do it in these objects.
>>>
>>>> Apple's NSDistributedNotificationCenter API
>>>> (http://developer.apple.com/documentation/Cocoa/Reference/Foundation/Classes/NSDistributedNotificationCenter_Class/Reference/Reference.html 
>>>> )
>>>> seems like a good pattern for this task (I'm sure there are  
>>>> equivalent
>>>> APIs in other frameworks). Basically, the frontend can register for
>>>> notifications by name. The interpreter then fires notifications  
>>>> for all
>>>> events and they are passed on, by the notification center, to all
>>>> registered observers.
>>>
>>> Well, I don't really like this kind of code. I find it clunky, and  
>>> you
>>> end up have notification registrations all over the place (reminds  
>>> me of
>>> wx). I actually prefer a lot the traits model, or the VTK model,  
>>> AFAIK,
>>> where objects can be there own observers. As a result you do not  
>>> have
>>> notifications blending in objects where they do not belong. For  
>>> the core
>>> frontends, obviously, we do not want to add a dependence to  
>>> traits, but I
>>> must imagine I cannot imagine going much further without it. To  
>>> take your
>>> example of updating the namespace view, all you have to do is to  
>>> pass a
>>> traited dict, instead of a dict, to the interpreter as a  
>>> namespace. Then
>>> you can have your namespace update automatically (AFAIK Robert is
>>> actually working on a branch where he does that).
>>>
>>> If we start adding callbacks whereever we need them, I am worried  
>>> we'll
>>> end up with callbacks everywhere. I think we need to add callback  
>>> only
>>> where they are absolutely necessary.
>>> My two cents,
>>>
>>> Ga?l
>>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>>


From vivainio at gmail.com  Wed Jul 23 12:35:38 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 23 Jul 2008 19:35:38 +0300
Subject: [IPython-dev] History in ipython1
In-Reply-To: <20080722165043.GA9152@phare.normalesup.org>
References: <20080721040011.GO31836@phare.normalesup.org>
	<46cb515a0807210832g3322a398kddba126ba7ecaa2a@mail.gmail.com>
	<20080722034731.GF5258@phare.normalesup.org>
	<cd7634ce0807220736w1d25aae0l829f788060ba1f88@mail.gmail.com>
	<6ce0ac130807220900o4f87f8day20bb9b7571d06ea0@mail.gmail.com>
	<20080722165043.GA9152@phare.normalesup.org>
Message-ID: <46cb515a0807230935k4d6a2ca0r76052ac8b3458ad0@mail.gmail.com>

On Tue, Jul 22, 2008 at 7:50 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:

>> Any departure from this will probably confuse developers and users.
>
> OK, so lets stick to the way it is currenlty implemented. I just wanted
> to have this discussion rather early than late.

Still, there is nothing stopping you from using indices like -1, -3 to
start counting from the end.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From ondrej at certik.cz  Wed Jul 23 16:11:04 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 23 Jul 2008 22:11:04 +0200
Subject: [IPython-dev] let ipython return parent's class method docstring
	automatically
Message-ID: <85b5c3130807231311g4d1382cr58831b124f2f25f6@mail.gmail.com>

Hi,

we are with Mateusz Paprocki at my place doing some hacking and we'd
like to know what is the best way to handle the following situation in
sympy:

 In [1]: e = x*y

In [2]: e.nseries?
Type:		instancemethod
Base Class:	<type 'instancemethod'>
String Form:	<bound method Mul.nseries of x*y>
Namespace:	Interactive
File:		/home/ondra/sympy/sympy/core/mul.py
Definition:	e.nseries(self, x, x0, n)
Docstring:
    <no docstring>


So the Mul.nseries() is missing a docstring, but:

In [4]: print e.__class__.__base__.nseries.__doc__

        Calculates a generalized series expansion.

        The difference between oseries and nseries is that nseries calculates
        "n" terms in the innermost expressions and then builds up the final
        series just by "cross-mutliplying" everything out.

        Advantage -- it's fast, because we don't have to determine how many
        terms we need to calculate in advance.

        Disadvantage -- you may endup with less terms than you may have
        expected, but the O(x**n) term appended will always be correct, so the
        result is correct, but maybe shorter.



The partent class, in this case Basic, has a nice docstring. The thing
is, that the Basic class has the docstring, but the child classes
don't (obviously, because it'd be the same).
What is the best way to handle this?

We can see two possibilities:

1) patch ipython to return parent's (or parent's parent's) docstring.
I checked that and the patch would be just a few lines of code in
OInspect.py
2) have just Basic.nseries() method and use _eval_nseries() in all
subclasses. That way, if the user types e.nseries(), the
Basic.nseries() with a nice docstring will be invoked.

So I'd like to ask the ipython guys if 1) is the way to go, or rather
2)? Or is there also some other way to fix this.

Ondrej & Mateusz


From fperez.net at gmail.com  Wed Jul 23 17:59:57 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 23 Jul 2008 14:59:57 -0700
Subject: [IPython-dev] let ipython return parent's class method
	docstring automatically
In-Reply-To: <85b5c3130807231311g4d1382cr58831b124f2f25f6@mail.gmail.com>
References: <85b5c3130807231311g4d1382cr58831b124f2f25f6@mail.gmail.com>
Message-ID: <db6b5ecc0807231459l1bf5d8fuc10c2181f4eee0ff@mail.gmail.com>

On Wed, Jul 23, 2008 at 1:11 PM, Ondrej Certik <ondrej at certik.cz> wrote:

> The partent class, in this case Basic, has a nice docstring. The thing
> is, that the Basic class has the docstring, but the child classes
> don't (obviously, because it'd be the same).
> What is the best way to handle this?
>
> We can see two possibilities:
>
> 1) patch ipython to return parent's (or parent's parent's) docstring.
> I checked that and the patch would be just a few lines of code in
> OInspect.py

I think this is a fairly generic problem, that of children who
override parent  methods but don't rewrite the docstrings.  So I'd be
happy with ipython making the user's life easier out of the box,
though I think in this case we should say something like:

Docstring : [extracted from parent class foo.bar.baz]
blah...

so users are aware that they're reading the parent's docstring, just
in case there are small inconsistencies the developer forgot to
document.

How does this sound?

Cheers,

f


From ondrej at certik.cz  Wed Jul 23 18:25:28 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Thu, 24 Jul 2008 00:25:28 +0200
Subject: [IPython-dev] let ipython return parent's class method
	docstring automatically
In-Reply-To: <db6b5ecc0807231459l1bf5d8fuc10c2181f4eee0ff@mail.gmail.com>
References: <85b5c3130807231311g4d1382cr58831b124f2f25f6@mail.gmail.com>
	<db6b5ecc0807231459l1bf5d8fuc10c2181f4eee0ff@mail.gmail.com>
Message-ID: <85b5c3130807231525w3064eef9xa0f53777fc67a479@mail.gmail.com>

On Wed, Jul 23, 2008 at 11:59 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Wed, Jul 23, 2008 at 1:11 PM, Ondrej Certik <ondrej at certik.cz> wrote:
>
>> The partent class, in this case Basic, has a nice docstring. The thing
>> is, that the Basic class has the docstring, but the child classes
>> don't (obviously, because it'd be the same).
>> What is the best way to handle this?
>>
>> We can see two possibilities:
>>
>> 1) patch ipython to return parent's (or parent's parent's) docstring.
>> I checked that and the patch would be just a few lines of code in
>> OInspect.py
>
> I think this is a fairly generic problem, that of children who
> override parent  methods but don't rewrite the docstrings.  So I'd be
> happy with ipython making the user's life easier out of the box,
> though I think in this case we should say something like:
>
> Docstring : [extracted from parent class foo.bar.baz]
> blah...
>
> so users are aware that they're reading the parent's docstring, just
> in case there are small inconsistencies the developer forgot to
> document.
>
> How does this sound?

I'll send a patch.

Ondrej


From fperez.net at gmail.com  Wed Jul 23 23:22:53 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 23 Jul 2008 20:22:53 -0700
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
	<cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
	<20080721045021.GP31836@phare.normalesup.org>
	<cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>
	<6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
Message-ID: <db6b5ecc0807232022r550c5639j1035568269f9f910@mail.gmail.com>

Hi all,

I think we all agree on the fact that we want the combination of loose
coupling, ease of code reuse across actual frontend implementations,
and a reasonable implementation in terms of performance.  The latter
is actually important: while it's true that by using python we've gone
for flexibility over speed, we still need to be careful.  It's not
that hard to write python code that runs slow enough to be a problem
in practice (Traits 1.0 was implemented in pure python, for example,
and it had serious performance issues).

On Tue, Jul 22, 2008 at 1:25 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:

Now, on to what to do next:

> >From this thread, I see a few options:
>
> 1.  Traits.  While I really like the Traits model and I think it is
> highly appropriate for GUI programming, I think it would lead to too
> tightly coupled code.  Also, we can't use Traits in the core of
> ipython because it has compiled code.
>
> 2.  Plain callbacks.  At least a couple of us seem to think that an
> interface that has lots of callbacks becomes very cumbersome.
>
> 3.  The Observer/Delegate pNSDistributedNotificationCenteratterns as are typical is Cocoa.  I know
> that Gael is not fund of this, but I do think this pattern is much
> better than plain callbacks, as it give a nice formal structure to
> everything.  Also, I agree with Barry that this pattern is pretty much
> what you get when you try to do plain callbacks properly.

I think there's a fourth, that Gael mentioned in passing and which
we've been discussing here face to face for a while: using objects
whose basic behavior defines the core API, but which can be given at
construction time instead of being hardcoded in our various __init__
methods.  This allows subclasses (or even non-subclassing application
writers who use the code as a library to build their own tools) to
cleanly provide only the level of behavior modification they need.

I've just gone through the exercise of building the nose plugin for
ipython testing, and I precisely ran into the problem that in many
places, I was forced to copy nose code simply to override a specific
internal object in one place because it hadn't been parametrized.  But
if the nose plugins had more of their components be parametric, it
would have been much simpler to write my code, since all I was doing
was replacing these hard-coded objects with subclasses of the same
that customized one small specific behavior.

This seems to provide a simple solution to our design question: all we
need to do is to  clarify which objects are our core publicly
modifiable components in the API, and then users can tweak only the
parts they need.  Said objects can themselves provide
observer/delegation behavior if they so desire, but they can do it in
the most appropriate way for a given problem.  For example, someone
writing a Traits app (who's already committed to using Traits for
their own reasons) can simply stick in there traited versions of the
same things, and with essentially zero extra code they get the Traits
event model everyhwere.

In a Cocoa environment, these objects could be lightly wrapped
versions of the originals that register (via a modified __getattr__
for example) the necessary observation/update calls into the
NSDistributedNotificationCenter that Cocoa provides, if that's the
best way to do it in such an environment.

And someone who just needs a callback here and there can simply stick
their callback-enhanced object where they want.  I could even see this
being very handy for occasional debugging, by quickly activating
tracing versions of certain objects as needed.

One thing that I like about this approach is that it doesn't
necessarily close the door on a *future* implementation of an
ipython-wide notification center, if we end up finding in practice
that it's really needed as the code grows.  But right now, it seems to
me that this simpler approach solves all of our problems, with the
advantage of introducing exactly *zero* (not small, but truly zero)
overhead for the normal use that has no customizations.

Does this sound reasonable to others, or did I totally miss this
particular boat?  I really want us all to be happy with the solution
we find to this so that we move forward reasonably convinced of it
being viable.

Cheers,

f


From ellisonbg.net at gmail.com  Thu Jul 24 01:22:58 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Wed, 23 Jul 2008 23:22:58 -0600
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <db6b5ecc0807232022r550c5639j1035568269f9f910@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
	<cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
	<20080721045021.GP31836@phare.normalesup.org>
	<cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>
	<6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
	<db6b5ecc0807232022r550c5639j1035568269f9f910@mail.gmail.com>
Message-ID: <6ce0ac130807232222j295701denfa660731fc729f07@mail.gmail.com>

> I think there's a fourth, that Gael mentioned in passing and which
> we've been discussing here face to face for a while: using objects
> whose basic behavior defines the core API, but which can be given at
> construction time instead of being hardcoded in our various __init__
> methods.  This allows subclasses (or even non-subclassing application
> writers who use the code as a library to build their own tools) to
> cleanly provide only the level of behavior modification they need.

Yes, this is a design pattern that I very much like and that is well
suited for certain things in ipython.  We actually use this in a few
places in the ipython1 code:

1.  An IPython engine takes an interpreter class as a parameter in its
init method.  Thus a user can pass the engine a custom interpreter
subclass to get custom behavior.

2.  We don't do it yet, but the interpreter could take a Traited dict
like object to use as the users namespace.

We definitely should do more of this.  But, I think this design
pattern addresses a slightly different need than the
callback/observer/notification stuff.  Here is why:

The real benefit of the observer/notification model is that everything
can be completely transient.  For example, you might have a notifier
that observes the user's namespace.  But you might only want it to
operate when the user opens a specific window.  Another example is
when an observer is hooked up to an unreliable network connection.  So
I guess I would ad another design constraint that I have until now
kept in the back of my mind:  we need loose coupling that is also
dynamic and transient.  And for these situations, I still think the
observer pattern is the best solution.

> This seems to provide a simple solution to our design question: all we
> need to do is to  clarify which objects are our core publicly
> modifiable components in the API, and then users can tweak only the
> parts they need.  Said objects can themselves provide
> observer/delegation behavior if they so desire, but they can do it in
> the most appropriate way for a given problem.  For example, someone
> writing a Traits app (who's already committed to using Traits for
> their own reasons) can simply stick in there traited versions of the
> same things, and with essentially zero extra code they get the Traits
> event model everyhwere.

I worry that such interfaces _can_ at times be a bit too implicit and
thus hide subtle behaviors and introduce code coupling that isn't
obvious.  One example of a subtle behavior is the following.  Traits
is based on a model that says this "an interface consists of a set of
attributes that you simply get and set as atributes"  But, the second
you go to propagate an interface over a network connection, you
discover that it is really difficult.  Interfaces that "network
friendly" tend to be i) be methods/functions that ii) have arguments
with very basic types.

We do have some examples in IPython.kernel where we propagate
interfaces that are attribute based, but a good amount of extra work
and care is required.

Thus, while I think the idea of passing a Traited dict to the
interpreter to observe the user's namespace is a beautiful idea, the
reality is much more painful:  there is no straightforward way of
propagating that interface over a network connection.

With a standardize observer/notifier pattern that is based on methods,
it is very straightforward to:

* Propagate observation/notification events over a network
* Unregister such events when network connections fail
* Integrate such events with Twisted.
* Integrate with other event loops like wx, qt, cocoa

One thing that I hadn't realized is that the observer/notifier pattern
has come up for Min and I in IPython.kernel a number of times, but we
have never taken the time to really abstract it properly.

.....a few minutes later...

I just reread this email and I think I am talking in circles and a bit
tired.  But, I do think we need to have a solution that is "network
friendly".

Cheers,

Brian

PS - starting tomorrow, I will be offline for about a week.


> In a Cocoa environment, these objects could be lightly wrapped
> versions of the originals that register (via a modified __getattr__
> for example) the necessary observation/update calls into the
> NSDistributedNotificationCenter that Cocoa provides, if that's the
> best way to do it in such an environment.
>
> And someone who just needs a callback here and there can simply stick
> their callback-enhanced object where they want.  I could even see this
> being very handy for occasional debugging, by quickly activating
> tracing versions of certain objects as needed.
>
> One thing that I like about this approach is that it doesn't
> necessarily close the door on a *future* implementation of an
> ipython-wide notification center, if we end up finding in practice
> that it's really needed as the code grows.  But right now, it seems to
> me that this simpler approach solves all of our problems, with the
> advantage of introducing exactly *zero* (not small, but truly zero)
> overhead for the normal use that has no customizations.
>
> Does this sound reasonable to others, or did I totally miss this
> particular boat?  I really want us all to be happy with the solution
> we find to this so that we move forward reasonably convinced of it
> being viable.
>
> Cheers,
>
> f
>


From hans_meine at gmx.net  Thu Jul 24 04:44:45 2008
From: hans_meine at gmx.net (Hans Meine)
Date: Thu, 24 Jul 2008 10:44:45 +0200
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>
	<6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
Message-ID: <200807241044.46716.hans_meine@gmx.net>

Am Dienstag, 22. Juli 2008 22:25:45 schrieb Brian Granger:
> 3.  The Observer/Delegate patterns as are typical is Cocoa.  I know
> that Gael is not fund of this, but I do think this pattern is much
> better than plain callbacks, as it give a nice formal structure to
> everything.  Also, I agree with Barry that this pattern is pretty much
> what you get when you try to do plain callbacks properly.
>
> Thus, my own conclusion is that the Observer/Delegate pattern is
> probably the best option for us and will lead to the loose coupling
> that we need.
>
> Barry, what would it take to put together an example of this pattern?
> Will we need some amount of infrastructure to make this possible?

My experience is that a callback mechanism is easily implemented in Python as 
a (standard python) list of callables that are called in turn.  Optionally, 
you may allow to specify priorities when adding callbacks to get the proper 
order, but that looks like overkill here.  This relies on Python's automatic 
member function binding and duck typing (i.e. you can pass objects with a 
__call__ method, normal functions, or bound member functions) and works very 
well.

AFAICS, there is no big difference between supporting callbacks and the 
Observer pattern; if you provide an "addOutputCallback" method that 
internally appends its argument to a list of outputCallbacks, an Observer 
would simply call that method and pass self (if it has a specific __call__ 
method) or e.g. self._watchOutput.  The latter IMHO makes it so powerful 
because an Observer may easily watch several different Subjects without being 
forced to follow a certain interface (i.e. method names) for that.

Just my 2 cents,
  Hans

PS: Forgot to press send; I actually wrote this about 24 hours ago.  Looks 
nearly off-topic to me now, but anyhow..


From barrywark at gmail.com  Thu Jul 24 16:23:53 2008
From: barrywark at gmail.com (Barry Wark)
Date: Thu, 24 Jul 2008 15:23:53 -0500
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <cd7634ce0807241322w760e90ackf676b6a21958f40d@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<20080720192702.GH23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
	<cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
	<20080721045021.GP31836@phare.normalesup.org>
	<cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>
	<6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
	<db6b5ecc0807232022r550c5639j1035568269f9f910@mail.gmail.com>
	<6ce0ac130807232222j295701denfa660731fc729f07@mail.gmail.com>
	<cd7634ce0807241322w760e90ackf676b6a21958f40d@mail.gmail.com>
Message-ID: <cd7634ce0807241323k2fe098fay8b16986f4d312642@mail.gmail.com>

I fear that I may have muddied the waters in talking about
notification and delegation all together. They really solve two
different problems. I'll try to lay that out below, but the short
version is that notification/observation seems to be the most
appropriate when the flow of information is unidirectional (e.g. a
frontend wants to know that something was printed to stdout), whereas
delegation is more appropriate when the flow is bidirectional (e.g.
the Shell wants to give a delegate the opportunity to modify its
internal behavior). I think that Brian and I are really talking about
the former, while Gael and Fernando are really talking mostly about
the later. I'll try to make the case below that delegation is not the
right solution for the observer problem, but that it may also be very
useful.

I will be truly offline until Sunday, so I'll say my piece here, and
let the discussion run its course.

On Thu, Jul 24, 2008 at 12:22 AM, Brian Granger <ellisonbg.net at gmail.com> wrote:
>> I think there's a fourth, that Gael mentioned in passing and which
>> we've been discussing here face to face for a while: using objects
>> whose basic behavior defines the core API, but which can be given at
>> construction time instead of being hardcoded in our various __init__
>> methods.  This allows subclasses (or even non-subclassing application
>> writers who use the code as a library to build their own tools) to
>> cleanly provide only the level of behavior modification they need.
>
> Yes, this is a design pattern that I very much like and that is well
> suited for certain things in ipython.  We actually use this in a few
> places in the ipython1 code:
>
> 1.  An IPython engine takes an interpreter class as a parameter in its
> init method.  Thus a user can pass the engine a custom interpreter
> subclass to get custom behavior.
>
> 2.  We don't do it yet, but the interpreter could take a Traited dict
> like object to use as the users namespace.

I agree that parameterizing many of the iteractions is useful
(especially for testing). When we want to be able to modify the
Interperter's behavior without subclassing Interpreter, this is
probably the right approach. However, if we believe that the
intersection of notifications that the set of delegates (e.g. the
frontends) may want to recieve is non-empty, the observer pattern may
be more appropriate *for event notification*...

As Brian points out below, delegation can lead to some unexpected
dependencies. Let me give one other example: suppose we use subclasses
of parameterized objects to add behavior to the interpreter. In this
scenario, the Interpreter constructor gets a parameter to which it
delegates responsibility for handling event notification (by callback,
observer pattern, etc.). Suppose Gael implements this object to use
callbacks for the events he needs in his Wx frontend. I also implement
a subclass to provide an observer-pattern based implementation of
event handling. Fernando wants to avoid the overhead of
events/callbacks and so passes None as this paramteer (or provides a
subclass that has no-op methods) [1]. Now imagine that we want to add
new functionality to the Interpreter class that changes when or how
the Interpreter will call this parameterized object (such as calling
the delegate's wrote_to_stdout method when the Interpreter writes to
stdout). Even if the change does not directly affect the use-case for
which Gael and I designed our subclasses, all our code needs to be
reviewed to see if it is affected by this change in the Interpreter's
internal implementation (in the example case, all of the possible
delegates need to be checked to make sure they didn't use
wrote_to_stdout as a method name for a different purpose *and* the
common base class needs to be updated to respond to the
wrote_to_stdout method with a no-op). In other words, because there is
a direct contractual relationship between the Interpreter and these
subclasses, dependencies are likely. In the observer pattern, the
Interpreter could fire a new event, but it would be ignored by any
observer that didnt' explicitly register for that event type. No
changes needed outside of the Interpreter class. A real world example,
I believe, is the matplotlib backend architecture. The matplotlib
library uses a plugable backend module to render plots. Each backend
thus needs to be maintained in parallel, a difficult and growing
problem for the Matplotlib team.

However, as Brian points out, delegation is *very* useful and is
probably the best way to handle the user_ns use case.


>
> We definitely should do more of this.  But, I think this design
> pattern addresses a slightly different need than the
> callback/observer/notification stuff.  Here is why:
>
> The real benefit of the observer/notification model is that everything
> can be completely transient.  For example, you might have a notifier
> that observes the user's namespace.  But you might only want it to
> operate when the user opens a specific window.  Another example is
> when an observer is hooked up to an unreliable network connection.  So
> I guess I would ad another design constraint that I have until now
> kept in the back of my mind:  we need loose coupling that is also
> dynamic and transient.  And for these situations, I still think the
> observer pattern is the best solution.

Just to reiterate and paraphrase Brian, the observer pattern lets the
Interpreter not know *anything* about the frontend (even whether it
exists or not) that is observing its behavior and it lets the frontend
not know *anything* about the Interpreter's implementation except that
it will fire notifications for the events defined in its interface.
So, when a new event is added to the Interpreter, no observer code
needs to be modified (it doesn't even need to know that the new
notification is fired). If the Interpreter's implementation is changes
so that, e.g. notification of writing to stdout happens immediately
after writing or after some short delay, none of the observing code
needs to be reviewed as long as the original Interpreter API didn't
specify the exact time. In other words, using an intermediary
notification center enforces a loose coupling; the only dependency
between Interpreter and frontend is the API that defines the
event/notificaiton itself.

Using the observer pattern makes adding new event types to the shell
easy too. For example, we could write function decorators to fire
events before/after the function. So, assuming all stdout goes to a
print_stdout() method in shell or some such, then

print_stdout(...):
   blah...

becomes

@notify_after(STDOUT_EVENT_TYPE)
print_stdout(...):
   blah...

In other words, no code in the function that fires the notification
needs to be modified if the notification can be fired before or after
the entire methods completes. We benefit from code reuse in the notify
method and don't have to rewrite similar functionality for each
individual callback use (this was Brian's point that a properly
implemented callback system is, essentially, a notification center).
In the case of Fernando's use-case above, notify_after() can call
through directly to the print_stdout function. The only performance
penalty is one additional method call. Is that acceptable?



>
>> This seems to provide a simple solution to our design question: all we
>> need to do is to  clarify which objects are our core publicly
>> modifiable components in the API, and then users can tweak only the
>> parts they need.  Said objects can themselves provide
>> observer/delegation behavior if they so desire, but they can do it in
>> the most appropriate way for a given problem.  For example, someone
>> writing a Traits app (who's already committed to using Traits for
>> their own reasons) can simply stick in there traited versions of the
>> same things, and with essentially zero extra code they get the Traits
>> event model everyhwere.
>
> I worry that such interfaces _can_ at times be a bit too implicit and
> thus hide subtle behaviors and introduce code coupling that isn't
> obvious.  One example of a subtle behavior is the following.  Traits
> is based on a model that says this "an interface consists of a set of
> attributes that you simply get and set as atributes"  But, the second
> you go to propagate an interface over a network connection, you
> discover that it is really difficult.  Interfaces that "network
> friendly" tend to be i) be methods/functions that ii) have arguments
> with very basic types.
>
> We do have some examples in IPython.kernel where we propagate
> interfaces that are attribute based, but a good amount of extra work
> and care is required.
>
> Thus, while I think the idea of passing a Traited dict to the
> interpreter to observe the user's namespace is a beautiful idea, the
> reality is much more painful:  there is no straightforward way of
> propagating that interface over a network connection.
>
> With a standardize observer/notifier pattern that is based on methods,
> it is very straightforward to:
>
> * Propagate observation/notification events over a network
> * Unregister such events when network connections fail
> * Integrate such events with Twisted.
> * Integrate with other event loops like wx, qt, cocoa
>
> One thing that I hadn't realized is that the observer/notifier pattern
> has come up for Min and I in IPython.kernel a number of times, but we
> have never taken the time to really abstract it properly.
>
> .....a few minutes later...
>
> I just reread this email and I think I am talking in circles and a bit
> tired.  But, I do think we need to have a solution that is "network
> friendly".
>
> Cheers,
>
> Brian
>
> PS - starting tomorrow, I will be offline for about a week.
>
>
>> In a Cocoa environment, these objects could be lightly wrapped
>> versions of the originals that register (via a modified __getattr__
>> for example) the necessary observation/update calls into the
>> NSDistributedNotificationCenter that Cocoa provides, if that's the
>> best way to do it in such an environment.
>>
>> And someone who just needs a callback here and there can simply stick
>> their callback-enhanced object where they want.  I could even see this
>> being very handy for occasional debugging, by quickly activating
>> tracing versions of certain objects as needed.
>>
>> One thing that I like about this approach is that it doesn't
>> necessarily close the door on a *future* implementation of an
>> ipython-wide notification center, if we end up finding in practice
>> that it's really needed as the code grows.  But right now, it seems to
>> me that this simpler approach solves all of our problems, with the
>> advantage of introducing exactly *zero* (not small, but truly zero)
>> overhead for the normal use that has no customizations.

[1] Re: performance
I believe that Fernando's "no-op" parameter must incur *some*
overhead; either the Interpreter must test for None, or the
Interpreter must call a method in Fernando's object which does
nothing. In other words, there's no free lunch. I can't think of a way
to add the ability to notify observers and/or modify the behavior of
Interpreter via delegates with truely zero overhead while avoiding the
parallel implementation problem that I describe above.

>>
>> Does this sound reasonable to others, or did I totally miss this
>> particular boat?  I really want us all to be happy with the solution
>> we find to this so that we move forward reasonably convinced of it
>> being viable.
>>
>> Cheers,
>>
>> f
>>
>


From fperez.net at gmail.com  Thu Jul 24 22:45:38 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 24 Jul 2008 19:45:38 -0700
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <cd7634ce0807241323k2fe098fay8b16986f4d312642@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<6ce0ac130807201250m26fb1a5flcbc9bdb62844cda8@mail.gmail.com>
	<cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
	<20080721045021.GP31836@phare.normalesup.org>
	<cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>
	<6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
	<db6b5ecc0807232022r550c5639j1035568269f9f910@mail.gmail.com>
	<6ce0ac130807232222j295701denfa660731fc729f07@mail.gmail.com>
	<cd7634ce0807241322w760e90ackf676b6a21958f40d@mail.gmail.com>
	<cd7634ce0807241323k2fe098fay8b16986f4d312642@mail.gmail.com>
Message-ID: <db6b5ecc0807241945g3d1dc610r5ae0162a95558bde@mail.gmail.com>

I realize that Barry and Brian will be going offline, and I'll be
traveling tomorrow back home as well, so this is mostly for the
conversation to continue later, but while the ideas are in my head.

I greatly appreciate that everyone is trying to wrap their heads
around a rather abstract and difficult question, but I do worry a bit
that it's getting too far for me to make a solid judgment on which
road will really work better.  We all agree on the basics we want
(loose coupling, acceptable performance, etc), but we're trying to
figure out a flexible architecture that lets us implement things we
need:

- across gui toolkits

- out of process (which more or less implies also in a
network-independent way, since something that can be communicated out
of process can be sent over the network,  given that we're not using
shared memory tricks).

The question is how to get from here to there with the least amount of
pain and keeping us all one happy family :)  I'd really prefer it if
we didn't end up with a fork just because the Wx/traits frontend gets
bogged down in a heavy and over-abstracted notification API, so I hope
we can figure out a viable solution for the lot.


On Thu, Jul 24, 2008 at 1:23 PM, Barry Wark <barrywark at gmail.com> wrote:

> As Brian points out below, delegation can lead to some unexpected
> dependencies. Let me give one other example: suppose we use subclasses
> of parameterized objects to add behavior to the interpreter.
[...snip careful description of inheritance issues...]

Note that the problems you point out here are simply the generic
issues that exist with subclassing with Python: any time you use a
subclass in python, you have to worry about the parent class in the
future growing new attribute names that may collide with your own.
That's actually a flaw of the Python object model that is rarely
discussed, and which languages like C++ in fact do not have.

But this particular issue is a general OO/Python one, that has nothing
to do with a notification pattern or event handling.

[Brian]

>> The real benefit of the observer/notification model is that everything
>> can be completely transient.  For example, you might have a notifier
>> that observes the user's namespace.  But you might only want it to
>> operate when the user opens a specific window.  Another example is
>> when an observer is hooked up to an unreliable network connection.  So
>> I guess I would ad another design constraint that I have until now
>> kept in the back of my mind:  we need loose coupling that is also
>> dynamic and transient.  And for these situations, I still think the
>> observer pattern is the best solution.
>
> Just to reiterate and paraphrase Brian, the observer pattern lets the
> Interpreter not know *anything* about the frontend (even whether it
> exists or not) that is observing its behavior and it lets the frontend
> not know *anything* about the Interpreter's implementation except that
> it will fire notifications for the events defined in its interface.
> So, when a new event is added to the Interpreter, no observer code
> needs to be modified (it doesn't even need to know that the new
> notification is fired). If the Interpreter's implementation is changes
> so that, e.g. notification of writing to stdout happens immediately
> after writing or after some short delay, none of the observing code
> needs to be reviewed as long as the original Interpreter API didn't
> specify the exact time. In other words, using an intermediary
> notification center enforces a loose coupling; the only dependency
> between Interpreter and frontend is the API that defines the
> event/notificaiton itself.

The part that worries me is building YAEHA (Yet Another Event Handling
API)... Matplotlib did exactly that, and while it does allow you to
wire events using pure matplotlib code, it adds real, non-trivial
complexity to the code and makes it harder to fit into a specific
event handling API 'from the front'.  All GUI systems have their own
event models, so does Twisted, and I'm really not convinced that
IPython should grow its own (next to matplotlib's, and Traits', and
WX's, ...)

On the other hand, I think this is a discussion that is beginning to
get far enough into design abstractions that it's escaping my
(admittedly very limited) ability to gauge fruitfully the real world
outcomes.  So here's a concrete proposal:  Gael is already hard at
work (and far along) on a WX frontend that Enthought is funding for
use in Envisage apps.  Since this is real, working code, let's see
where it goes.  In the meantime, Barry can play with the design for a
lightweight notification center idea, and actual implementation will
likely be very enlightening.  I've got my hands more than full trying
to get at least a first cut of all the tests that appeared (once I
completed the testing plugin, all sorts of little things got picked up
as tests that now need to actually work).  Since we're trying to push
a release before scipy, this is really more than we can do all at the
same time.

With Gael's and Barry's implementation on hand,  I think it will be
easier to see if the notification center can be made lightweight
enough for adapting it to Traits/Twisted/Cocoa/whatever without undue
pain,  or if an object-by-object approach ends up being better.  I can
easily see it being both: object parametrization for certain kinds of
architectural customization with a little message center for
lightweight notifications.

But honestly, I'm finding it hard to see far enough into the fog with
pure 'whiteboard' thinking.  We always brag about how nice it is to
use python to actually implement ideas rather than only think about
them in the abstract, I think in this case this may be the wisest
approach.


> [1] Re: performance
> I believe that Fernando's "no-op" parameter must incur *some*
> overhead; either the Interpreter must test for None, or the
> Interpreter must call a method in Fernando's object which does
> nothing. In other words, there's no free lunch. I can't think of a way
> to add the ability to notify observers and/or modify the behavior of
> Interpreter via delegates with truely zero overhead while avoiding the
> parallel implementation problem that I describe above.

No, the None check is only done at isntantiation time of the main
objects.  Afterwards, it's the responsibility of the  subclasses to
produce/manage events for specific behavior.  The default object will
for example have a .write() method that simply prints, while an event
handling subclass would have its .write() be Traited so that a GUI
could listen in, or a Twisted one would handle it over the network.
So it really is zero-overhead, simply because in the default, there is
no event production/consumption at all.

So from a performance perspective, that approach really produces
exactly zero cost.  The question that I think will be best answered
with a bit of prototyping, is whether the architectural costs become
greater than those of a message center.

Gael just emphasized that in the code he's doing,  if it turns out the
message center approach works well, there's little cost in adding it
later on.  So it's really not an either/or issue, just one of getting
functioning prototypes  working  quickly so we can judge based on the
actual code rather than pure abstractions.

Does this sound like a reasonable plan?

Cheers,

f


From barrywark at gmail.com  Sun Jul 27 22:02:08 2008
From: barrywark at gmail.com (Barry Wark)
Date: Sun, 27 Jul 2008 22:02:08 -0400
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <db6b5ecc0807241945g3d1dc610r5ae0162a95558bde@mail.gmail.com>
References: <20080720164615.GB23971@phare.normalesup.org>
	<cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
	<20080721045021.GP31836@phare.normalesup.org>
	<cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>
	<6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
	<db6b5ecc0807232022r550c5639j1035568269f9f910@mail.gmail.com>
	<6ce0ac130807232222j295701denfa660731fc729f07@mail.gmail.com>
	<cd7634ce0807241322w760e90ackf676b6a21958f40d@mail.gmail.com>
	<cd7634ce0807241323k2fe098fay8b16986f4d312642@mail.gmail.com>
	<db6b5ecc0807241945g3d1dc610r5ae0162a95558bde@mail.gmail.com>
Message-ID: <cd7634ce0807271902n2d0fa7bbp49ccb6ec8f2192c8@mail.gmail.com>

Taking a bit of my own medicine?that working code is better than
awesome theory?I've checked in the ipython-notification branch and
proposed it for merging into trunk. It includes a functional spec in
/blueprints (it's a bit tounge-in-cheek; I hope no one is offended) as
well as an implementation of the NotificationCenter I've been
discussin in IPython.kernel.core.notification.

Hopefully code will answer more questions than the "fog" we've created
so far. I've added a couple of comments below, but I'd guess a quick
look at the code is more valuable.

B

On Thu, Jul 24, 2008 at 10:45 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> I realize that Barry and Brian will be going offline, and I'll be
> traveling tomorrow back home as well, so this is mostly for the
> conversation to continue later, but while the ideas are in my head.
>
> I greatly appreciate that everyone is trying to wrap their heads
> around a rather abstract and difficult question, but I do worry a bit
> that it's getting too far for me to make a solid judgment on which
> road will really work better.  We all agree on the basics we want
> (loose coupling, acceptable performance, etc), but we're trying to
> figure out a flexible architecture that lets us implement things we
> need:
>
> - across gui toolkits
>
> - out of process (which more or less implies also in a
> network-independent way, since something that can be communicated out
> of process can be sent over the network,  given that we're not using
> shared memory tricks).
>
> The question is how to get from here to there with the least amount of
> pain and keeping us all one happy family :)  I'd really prefer it if
> we didn't end up with a fork just because the Wx/traits frontend gets
> bogged down in a heavy and over-abstracted notification API, so I hope
> we can figure out a viable solution for the lot.

I hope you'll see that this is a pretty light-weight implementation...

>
>
> On Thu, Jul 24, 2008 at 1:23 PM, Barry Wark <barrywark at gmail.com> wrote:
>
>> As Brian points out below, delegation can lead to some unexpected
>> dependencies. Let me give one other example: suppose we use subclasses
>> of parameterized objects to add behavior to the interpreter.
> [...snip careful description of inheritance issues...]
>
> Note that the problems you point out here are simply the generic
> issues that exist with subclassing with Python: any time you use a
> subclass in python, you have to worry about the parent class in the
> future growing new attribute names that may collide with your own.
> That's actually a flaw of the Python object model that is rarely
> discussed, and which languages like C++ in fact do not have.
>
> But this particular issue is a general OO/Python one, that has nothing
> to do with a notification pattern or event handling.

I wasn't trying to suggest that this is an observer pattern-specific
problem, just that this particular problem is likely to crop up in the
notification/callback context if we're not careful. Probably not a
great reason to go with one solution over the other, however.

>
> [Brian]
>
>>> The real benefit of the observer/notification model is that everything
>>> can be completely transient.  For example, you might have a notifier
>>> that observes the user's namespace.  But you might only want it to
>>> operate when the user opens a specific window.  Another example is
>>> when an observer is hooked up to an unreliable network connection.  So
>>> I guess I would ad another design constraint that I have until now
>>> kept in the back of my mind:  we need loose coupling that is also
>>> dynamic and transient.  And for these situations, I still think the
>>> observer pattern is the best solution.
>>
>> Just to reiterate and paraphrase Brian, the observer pattern lets the
>> Interpreter not know *anything* about the frontend (even whether it
>> exists or not) that is observing its behavior and it lets the frontend
>> not know *anything* about the Interpreter's implementation except that
>> it will fire notifications for the events defined in its interface.
>> So, when a new event is added to the Interpreter, no observer code
>> needs to be modified (it doesn't even need to know that the new
>> notification is fired). If the Interpreter's implementation is changes
>> so that, e.g. notification of writing to stdout happens immediately
>> after writing or after some short delay, none of the observing code
>> needs to be reviewed as long as the original Interpreter API didn't
>> specify the exact time. In other words, using an intermediary
>> notification center enforces a loose coupling; the only dependency
>> between Interpreter and frontend is the API that defines the
>> event/notificaiton itself.
>
> The part that worries me is building YAEHA (Yet Another Event Handling
> API)... Matplotlib did exactly that, and while it does allow you to
> wire events using pure matplotlib code, it adds real, non-trivial
> complexity to the code and makes it harder to fit into a specific
> event handling API 'from the front'.  All GUI systems have their own
> event models, so does Twisted, and I'm really not convinced that
> IPython should grow its own (next to matplotlib's, and Traits', and
> WX's, ...)

I appologize if I gave the impression I was trying to create YAEHA.
This is about non-frontend events. There shouldn't be much, if any
overlap between GUI events and the notifications we're talking about.
It really is just a clean implementation of callbacks, nothing more,
hopefully not too much less :)

>
> On the other hand, I think this is a discussion that is beginning to
> get far enough into design abstractions that it's escaping my
> (admittedly very limited) ability to gauge fruitfully the real world
> outcomes.  So here's a concrete proposal:  Gael is already hard at
> work (and far along) on a WX frontend that Enthought is funding for
> use in Envisage apps.  Since this is real, working code, let's see
> where it goes.  In the meantime, Barry can play with the design for a
> lightweight notification center idea, and actual implementation will
> likely be very enlightening.  I've got my hands more than full trying
> to get at least a first cut of all the tests that appeared (once I
> completed the testing plugin, all sorts of little things got picked up
> as tests that now need to actually work).  Since we're trying to push
> a release before scipy, this is really more than we can do all at the
> same time.
>
> With Gael's and Barry's implementation on hand,  I think it will be
> easier to see if the notification center can be made lightweight
> enough for adapting it to Traits/Twisted/Cocoa/whatever without undue
> pain,  or if an object-by-object approach ends up being better.  I can
> easily see it being both: object parametrization for certain kinds of
> architectural customization with a little message center for
> lightweight notifications.

As always, the best solution is likely the middle road between two good options.

>
> But honestly, I'm finding it hard to see far enough into the fog with
> pure 'whiteboard' thinking.  We always brag about how nice it is to
> use python to actually implement ideas rather than only think about
> them in the abstract, I think in this case this may be the wisest
> approach.
>
>
>> [1] Re: performance
>> I believe that Fernando's "no-op" parameter must incur *some*
>> overhead; either the Interpreter must test for None, or the
>> Interpreter must call a method in Fernando's object which does
>> nothing. In other words, there's no free lunch. I can't think of a way
>> to add the ability to notify observers and/or modify the behavior of
>> Interpreter via delegates with truely zero overhead while avoiding the
>> parallel implementation problem that I describe above.
>
> No, the None check is only done at isntantiation time of the main
> objects.  Afterwards, it's the responsibility of the  subclasses to
> produce/manage events for specific behavior.  The default object will
> for example have a .write() method that simply prints, while an event
> handling subclass would have its .write() be Traited so that a GUI
> could listen in, or a Twisted one would handle it over the network.
> So it really is zero-overhead, simply because in the default, there is
> no event production/consumption at all.

Ah, I see. You're proposing moving _all_ the event generating code
into the parametrized objects. Now I get it. I've written into the
spec O(1) performance of my implementation, but it currently depends
on the (undocumented) complexity of the builtin set type. We'll have
to see if performance is acceptable.

>
> So from a performance perspective, that approach really produces
> exactly zero cost.  The question that I think will be best answered
> with a bit of prototyping, is whether the architectural costs become
> greater than those of a message center.

And I suppose whether the added number of objects and method calls
ends up being the same cost as the notification center implementation.

>
> Gael just emphasized that in the code he's doing,  if it turns out the
> message center approach works well, there's little cost in adding it
> later on.  So it's really not an either/or issue, just one of getting
> functioning prototypes  working  quickly so we can judge based on the
> actual code rather than pure abstractions.
>
> Does this sound like a reasonable plan?

Absolutely. I look forward to hearing you (and others') opinions on
the ipython-notification branch.

Cheers,

Barry


From edreamleo at gmail.com  Mon Jul 28 11:27:40 2008
From: edreamleo at gmail.com (Edward K. Ream)
Date: Mon, 28 Jul 2008 10:27:40 -0500
Subject: [IPython-dev] Thoughts re Leo and Ipython notebooks.
Message-ID: <ffb592890807280827m6a00e0cfx68ed0f4e2cb865bb@mail.gmail.com>

I've just posted some thoughts re Leo as a compound document:
http://groups.google.com/group/leo-editor/browse_thread/thread/759d20f81119e5ca

Among other things, the post discusses adding spreadsheet-like features to
Leo in a fairly easy way.  These are just ideas, intended to encourage
further invention.

My intuition tells me that IPython folk might find these ideas useful or
provocative:

- Leo could be considered an IPython notebook.

- Generalized spreadsheets might be useful in recording scientific
experiments.

- Scientific experiments might want to generate notebook/spreadsheet entries
using scripts.

- The post shows how to declare the "type" of scripts/nodes and show how to
embed scripts of arbitrary types into notebook entries.

Imo, we have only just scratched the surface of what can be done with
scripts and notebooks.

Edward
--------------------------------------------------------------------
Edward K. Ream email: edreamleo at gmail.com
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080728/fae6eeac/attachment.html>

From gael.varoquaux at normalesup.org  Wed Jul 30 20:55:28 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 31 Jul 2008 02:55:28 +0200
Subject: [IPython-dev] ipython1 and synchronous printing of stdout
In-Reply-To: <cd7634ce0807271902n2d0fa7bbp49ccb6ec8f2192c8@mail.gmail.com>
References: <cd7634ce0807202136k397006bqcc7bbe23069581e@mail.gmail.com>
	<20080721045021.GP31836@phare.normalesup.org>
	<cd7634ce0807221105q78ba3f89h65e9f777e2f17d78@mail.gmail.com>
	<6ce0ac130807221325y19ffd8dbtafc27d1c6ca5f4ac@mail.gmail.com>
	<db6b5ecc0807232022r550c5639j1035568269f9f910@mail.gmail.com>
	<6ce0ac130807232222j295701denfa660731fc729f07@mail.gmail.com>
	<cd7634ce0807241322w760e90ackf676b6a21958f40d@mail.gmail.com>
	<cd7634ce0807241323k2fe098fay8b16986f4d312642@mail.gmail.com>
	<db6b5ecc0807241945g3d1dc610r5ae0162a95558bde@mail.gmail.com>
	<cd7634ce0807271902n2d0fa7bbp49ccb6ec8f2192c8@mail.gmail.com>
Message-ID: <20080731005528.GA13565@phare.normalesup.org>

Hi,

Sorry for the silence, I have been working my ass off on a release of
Mayavi. Now I can go back to working on ipython.

On Sun, Jul 27, 2008 at 10:02:08PM -0400, Barry Wark wrote:
> Taking a bit of my own medicine?that working code is better than
> awesome theory?I've checked in the ipython-notification branch and
> proposed it for merging into trunk. It includes a functional spec in
> /blueprints (it's a bit tounge-in-cheek; I hope no one is offended) as
> well as an implementation of the NotificationCenter I've been
> discussin in IPython.kernel.core.notification.

I had a look at that. I seems to me like a (good) textbook implementation
of the patterns we are discussing. I still fail to see the point of
having this in Ipython, at least at the state we are in right now. My
point is that there are many implementation of such frameworks. The one
you have developed is a simple one, but when you start using it a lot,
you will need to add features, like delegations, or ordering the
callbacks. Also, something that is very important is how does this blend
in the code people write? How do you make it easy for people to write
notifications without thinking too much about it, and registering
handlers (hint decorators are cool). But once again, this looks a lot to
me like traits 1. I am not trying to sell traits here, I don't think
IPyhton should use traits, I am just flashing a warning: this smells like
weel reinvention, and on top of that developing a framework, which is
something you want to limit as much as possible: we already have too many
frameworks. I have the feeling every single person coming to develop an
app using Ipython will already have his notification framework (traits,
cocoa, does Qt have one?). So now he will to write adapting code to use
your framework.

Now there is value in having an iptyhon-specific notification layer, and
it is not having uniform code across backends; matplotlib tried that, and
it doesn't bring much value, amongst other things because the backend are
anyhow riddled with toolkit-specific code. The value is that we could
think of network-transparent notifications. That is much harder
(obviously, the simple problems don't need solutions), and I would like
to know if twisted doesn't provide this kind of service.

I guess my point is not that we shouldn't do this, but that we should do
this when the need arises, and not now, for no benefit. Doing this too
early is a good way of getting it wrong. In addition this will just slow
our progress towards getting frontends and we absolutely do not need this
to get the simple frontends. I am not going to spend anytime on this, and
spending too much time on this is a good way of not having any useable
code for a long time, I believe.

Sorry, not only am I not paid to worry about distant future, or
network-transparency problems, I am also having difficulties being
interested in something that I cannot specify properly because it doesn't
answer a problem I have. I am not in the business of making a framework,
I am in the business of making a widget. The whole notification framework
stuff, for me, will be done in traits, because this is what we use all
over our apps, because we have a lot of code that works well with it, and
because is it a mature and powerful library. I develop my apps using
traits, and I want an ipython widget to plug in my apps. If the objects
used internally in the ipython widget are standard object like dicts or
lists, I can use traits without any problem. If there are part of a
notification framework that I have to adapt to, I probably have to double
the size of my codebase. Let us see where we really need this framework.
Maybe it can also be developed in a subclass of the standard objects we
use in ipython, say a different frontend base class?

I hope I am not being to rough or criticizing too much your software
engineering efforts. I am just trying to be practical and figure out how
to ship software that is useful to users (and yes this includes
open-source software) in a reasonable amount of time. And the basis of
agile methods is to get it working, ship it to your users, see the
limitations in your implementation, refactor, ship it again, always
focusing on what benefits to your users, rather than on the architecture
or the software engineering, which is a tool, not a goal.

As for your code, if you want to merge it in, it is fine by me, but I am
afraid it won't be used for a while, and when we get to use it, we might
want to change it because we have more insight on our problems.

Thanks for bringing up on interesting discussion,

Ga?l