From takowl at gmail.com  Tue Feb  1 07:02:10 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 1 Feb 2011 12:02:10 +0000
Subject: [IPython-dev] Unicode in tests
Message-ID: <AANLkTini4eTaAFfD3u6d6aQJikqULy52JV15Y6qafFCW@mail.gmail.com>

While testing the use of non-ascii characters in 0.11 recently, I found that
there are things which will fail while actually using ipython, but cannot be
made to fail in the test suite. Investigating, I found that for the test
suite, the default encoding is somehow set to utf-8 (ascii is the default
default encoding for Python 2). I don't know quite what's doing this, but it
did the same with my system version of nose (0.11.1 in Ubuntu) and with nose
1.0.0 installed in the virtualenv.

At present, I've worked round that by adding an entirely separate
unicode_tests.py file, which has to be run separately from the main iptest
test suite.

https://github.com/ipython/ipython/pull/252

Does anyone have any insight on whether this is something our test suite is
doing, something nose is doing, something that can be fixed, or something
that we have to work around?

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110201/3f20adaa/attachment.html>

From Nicolas.rougier at inria.fr  Tue Feb  1 12:56:03 2011
From: Nicolas.rougier at inria.fr (Nicolas Rougier)
Date: Tue, 1 Feb 2011 18:56:03 +0100
Subject: [IPython-dev] Ascii imshow
Message-ID: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>



Hi all,

I've made some time ago a kind of 'ascii imshow' function to render a numpy array directly within the console by exploiting the 256 colors capability of most modern terminal. I don't know how much interesting it can be for IPython but I found the code useful for myself.

Code is located at: http://www.loria.fr/~rougier/coding/software/numpy_imshow.py
(GPL license can be replaced by the BSD one if necessary)


Nicolas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110201/f3f391df/attachment.html>

From fperez.net at gmail.com  Tue Feb  1 13:36:07 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 1 Feb 2011 13:36:07 -0500
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
Message-ID: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>

Hi folks,

I was just thinking, before the new display infrastructure starts
getting used too much, should we adjust our convention to use
__repr_html__, __repr_svg__, __repr_*__ as a naming pattern instead of
__html__, __svg__, etc?  My concern is two-fold:

- the more generic names may be already in use by something else that
doesn't quite match our 'protocol', or python may in the future
(though this is probably needless paranoia) choose to add them.

- a more regular pattern makes it easier to collect all __repr_FOO__
methods from an object as part of our display machinery rather than
matching against a hard-coded list.

Thoughts?

f


From ellisonbg at gmail.com  Tue Feb  1 14:05:52 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 1 Feb 2011 11:05:52 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
Message-ID: <AANLkTin6UkyhcsrMEOrLy8f--eNgkpspJnhVyOcotEfT@mail.gmail.com>

Fernando,

On Tue, Feb 1, 2011 at 10:36 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi folks,
>
> I was just thinking, before the new display infrastructure starts
> getting used too much, should we adjust our convention to use
> __repr_html__, __repr_svg__, __repr_*__ as a naming pattern instead of
> __html__, __svg__, etc? ?My concern is two-fold:
>
> - the more generic names may be already in use by something else that
> doesn't quite match our 'protocol', or python may in the future
> (though this is probably needless paranoia) choose to add them.

Yes, this is important and I have spent some time thinking about this.
 The collision with other project is something to think about.  I can
say why I made the choice I did:

* Simplicity.  __html__ is easier to remember that __repr_html__.  I
want writing these methods to be like eating candy.
* There are projects that use this convention already and our usage in
compatible with theirs.  The biggest example is the __html__ method
which Pylons, mako, Jinja2, Genshi and Django supposedly support.  I
thought - wow, these guys would _love_ us if there objects were
immediately representable in IPython.

I am definitely open to the __repr_html__ style names though.  What do
other people think?

> - a more regular pattern makes it easier to collect all __repr_FOO__
> methods from an object as part of our display machinery rather than
> matching against a hard-coded list.

Yes, that is true, but it would be difficult to really use such
dynamic display logic because each FOO has to be associated with a
unique MIME type and formatter.

I was expecting this question to come up in the review of the code.
It didn't, but I am glad it is coming up now.

Cheers,

Brian

> Thoughts?
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From robert.kern at gmail.com  Tue Feb  1 14:28:11 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 01 Feb 2011 11:28:11 -0800
Subject: [IPython-dev] Unicode in tests
In-Reply-To: <AANLkTini4eTaAFfD3u6d6aQJikqULy52JV15Y6qafFCW@mail.gmail.com>
References: <AANLkTini4eTaAFfD3u6d6aQJikqULy52JV15Y6qafFCW@mail.gmail.com>
Message-ID: <ii9msc$32g$1@dough.gmane.org>

On 2/1/11 4:02 AM, Thomas Kluyver wrote:
> While testing the use of non-ascii characters in 0.11 recently, I found that
> there are things which will fail while actually using ipython, but cannot be
> made to fail in the test suite. Investigating, I found that for the test suite,
> the default encoding is somehow set to utf-8 (ascii is the default default
> encoding for Python 2). I don't know quite what's doing this, but it did the
> same with my system version of nose (0.11.1 in Ubuntu) and with nose 1.0.0
> installed in the virtualenv.
>
> At present, I've worked round that by adding an entirely separate
> unicode_tests.py file, which has to be run separately from the main iptest test
> suite.
>
> https://github.com/ipython/ipython/pull/252
>
> Does anyone have any insight on whether this is something our test suite is
> doing, something nose is doing, something that can be fixed, or something that
> we have to work around?

There is no code in IPython or nose itself that does this. Can you make sure 
that nose is not picking up any extra plugins (print out sys.modules inside of a 
test suite)?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From takowl at gmail.com  Tue Feb  1 14:33:05 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 1 Feb 2011 19:33:05 +0000
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <AANLkTin6UkyhcsrMEOrLy8f--eNgkpspJnhVyOcotEfT@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<AANLkTin6UkyhcsrMEOrLy8f--eNgkpspJnhVyOcotEfT@mail.gmail.com>
Message-ID: <AANLkTi=Tx3BGt4ajETZtJyN8ZMgo+fdAG11QFRk==DO3@mail.gmail.com>

On 1 February 2011 19:05, Brian Granger <ellisonbg at gmail.com> wrote:

> * Simplicity.  __html__ is easier to remember that __repr_html__.  I
> want writing these methods to be like eating candy.
>

I suppose the key question is: is it reasonable to expect that none of these
names will have another use? I can't off the top of my head think of any
reason that __html__ would produce anything but an HTML representation, and
likewise for __svg__ and __png__, but it's worth thinking about.

Hmm, actually... a tool which is building, say, an SVG document may have
methods on the components which produce fragments (SVG which is valid within
an SVG document, but isn't a document by itself). I've seen this with an
svg() method, although not with an __svg__() method. Can the display code
handle such fragments? How does it fall over if it attempts to display
invalid svg/html/etc.?

For collecting all possible repr-like methods, there could be an informal
standard to put them into something like __repr_methods__. But unless it has
a real world role, library writers will simply ignore it.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110201/3bba18ba/attachment.html>

From robert.kern at gmail.com  Tue Feb  1 14:49:44 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 01 Feb 2011 11:49:44 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
Message-ID: <ii9o4q$abd$1@dough.gmane.org>

On 2/1/11 10:36 AM, Fernando Perez wrote:
> Hi folks,
>
> I was just thinking, before the new display infrastructure starts
> getting used too much, should we adjust our convention to use
> __repr_html__, __repr_svg__, __repr_*__ as a naming pattern instead of
> __html__, __svg__, etc?  My concern is two-fold:
>
> - the more generic names may be already in use by something else that
> doesn't quite match our 'protocol',

This is a pretty big problem particularly since there is currently no error 
recovery around the method call. There is a lot of room for error here. For 
example, the PlainTextFormatter, which uses pretty.py, specifies a 
print_method='__pretty__'. However, the __pretty__ method has a distinctly 
different signature that takes arguments. Fortunately, the __call__ was 
overridden to ignore the print_method and just use the pretty.py machinery 
directly. These kinds of mistakes are easy to make, and it bakes in a certain 
amount of brittleness into the system.

> or python may in the future
> (though this is probably needless paranoia) choose to add them.

Python reserves __foo__ names for itself and requests that third party packages 
do not make their own. Third party packages violate this pretty frequently, but 
that doesn't mean it's kosher. SAGE does something sensible and uses single 
underscores on either side.

Personally, I don't want to ever change one of my classes to get it printed 
nicely for IPython. That's an inversion of responsibilities that I find 
difficult to swallow (especially for HTML and JSON that are best done using 
particular libraries). Instead, I want the to write formatter functions and 
register them for the appropriate class. Maybe a conventional print_method could 
be a default.

In any case, the priority as implemented is inverted; we should check for the 
print_method *after* the registered functions are checked. I, the IPython user, 
should always be able to override what the class author thought was a good idea 
at the time. Or who implemented an __html__ method for a different system and a 
different calling convention.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ellisonbg at gmail.com  Tue Feb  1 16:31:02 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 1 Feb 2011 13:31:02 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <ii9o4q$abd$1@dough.gmane.org>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
Message-ID: <AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>

Robert,

On Tue, Feb 1, 2011 at 11:49 AM, Robert Kern <robert.kern at gmail.com> wrote:
> On 2/1/11 10:36 AM, Fernando Perez wrote:
>> Hi folks,
>>
>> I was just thinking, before the new display infrastructure starts
>> getting used too much, should we adjust our convention to use
>> __repr_html__, __repr_svg__, __repr_*__ as a naming pattern instead of
>> __html__, __svg__, etc? ?My concern is two-fold:
>>
>> - the more generic names may be already in use by something else that
>> doesn't quite match our 'protocol',
>
> This is a pretty big problem particularly since there is currently no error
> recovery around the method call. There is a lot of room for error here. For
> example, the PlainTextFormatter, which uses pretty.py, specifies a
> print_method='__pretty__'. However, the __pretty__ method has a distinctly
> different signature that takes arguments. Fortunately, the __call__ was
> overridden to ignore the print_method and just use the pretty.py machinery
> directly. These kinds of mistakes are easy to make, and it bakes in a certain
> amount of brittleness into the system.

Yes, there is definitely some brittleness right now in the design.  We
should definitely add additional errors checking in appropriate
places.

>> or python may in the future
>> (though this is probably needless paranoia) choose to add them.
>
> Python reserves __foo__ names for itself and requests that third party packages
> do not make their own. Third party packages violate this pretty frequently, but
> that doesn't mean it's kosher. SAGE does something sensible and uses single
> underscores on either side.

I am aware of the official status of the __foo__ names, but it is
tempting as i) the chances of Python using these names in minimal and
ii) other products do this in a way that is consistent with our usage
(the __html__ examples I gave earlier).  But in terms of alternate
names that don't use the __foo__ notation:

_repr_html_
_html_ - problem is that SAGE uses this format with possibly different meaning.
_html   - problem is that sympy uses this format with possibly
different meaning.
???

> Personally, I don't want to ever change one of my classes to get it printed
> nicely for IPython. That's an inversion of responsibilities that I find
> difficult to swallow (especially for HTML and JSON that are best done using
> particular libraries). Instead, I want the to write formatter functions and
> register them for the appropriate class. Maybe a conventional print_method could
> be a default.

> In any case, the priority as implemented is inverted; we should check for the
> print_method *after* the registered functions are checked. I, the IPython user,
> should always be able to override what the class author thought was a good idea
> at the time. Or who implemented an __html__ method for a different system and a
> different calling convention.

I got the idea to look for the special print methods first and then
the print function from IPython.externals.pretty itself. The first
thing the pretty printer does is this:

            if hasattr(obj_class, '__pretty__'):
                return obj_class.__pretty__(obj, self, cycle)

When I implemented the other printers, I wanted to make them as
similar to pretty as possible, so I stuck with this ordering of
priorities.  I don't mind using the other priority order as long as we
are consistent and modify pretty to use the same order.  Is this what
you would recommend Robert?

Cheers,

Brian


> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
> ?that is made terrible by our own mad attempt to interpret it as though it had
> ?an underlying truth."
> ? -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From alex.gaynor at gmail.com  Tue Feb  1 16:32:53 2011
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Tue, 1 Feb 2011 16:32:53 -0500
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
Message-ID: <AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>

On Tue, Feb 1, 2011 at 4:31 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Robert,
>
> On Tue, Feb 1, 2011 at 11:49 AM, Robert Kern <robert.kern at gmail.com>
> wrote:
> > On 2/1/11 10:36 AM, Fernando Perez wrote:
> >> Hi folks,
> >>
> >> I was just thinking, before the new display infrastructure starts
> >> getting used too much, should we adjust our convention to use
> >> __repr_html__, __repr_svg__, __repr_*__ as a naming pattern instead of
> >> __html__, __svg__, etc?  My concern is two-fold:
> >>
> >> - the more generic names may be already in use by something else that
> >> doesn't quite match our 'protocol',
> >
> > This is a pretty big problem particularly since there is currently no
> error
> > recovery around the method call. There is a lot of room for error here.
> For
> > example, the PlainTextFormatter, which uses pretty.py, specifies a
> > print_method='__pretty__'. However, the __pretty__ method has a
> distinctly
> > different signature that takes arguments. Fortunately, the __call__ was
> > overridden to ignore the print_method and just use the pretty.py
> machinery
> > directly. These kinds of mistakes are easy to make, and it bakes in a
> certain
> > amount of brittleness into the system.
>
> Yes, there is definitely some brittleness right now in the design.  We
> should definitely add additional errors checking in appropriate
> places.
>
> >> or python may in the future
> >> (though this is probably needless paranoia) choose to add them.
> >
> > Python reserves __foo__ names for itself and requests that third party
> packages
> > do not make their own. Third party packages violate this pretty
> frequently, but
> > that doesn't mean it's kosher. SAGE does something sensible and uses
> single
> > underscores on either side.
>
> I am aware of the official status of the __foo__ names, but it is
> tempting as i) the chances of Python using these names in minimal and
> ii) other products do this in a way that is consistent with our usage
> (the __html__ examples I gave earlier).  But in terms of alternate
> names that don't use the __foo__ notation:
>
> _repr_html_
> _html_ - problem is that SAGE uses this format with possibly different
> meaning.
> _html   - problem is that sympy uses this format with possibly
> different meaning.
> ???
>
> > Personally, I don't want to ever change one of my classes to get it
> printed
> > nicely for IPython. That's an inversion of responsibilities that I find
> > difficult to swallow (especially for HTML and JSON that are best done
> using
> > particular libraries). Instead, I want the to write formatter functions
> and
> > register them for the appropriate class. Maybe a conventional
> print_method could
> > be a default.
>
> > In any case, the priority as implemented is inverted; we should check for
> the
> > print_method *after* the registered functions are checked. I, the IPython
> user,
> > should always be able to override what the class author thought was a
> good idea
> > at the time. Or who implemented an __html__ method for a different system
> and a
> > different calling convention.
>
> I got the idea to look for the special print methods first and then
> the print function from IPython.externals.pretty itself. The first
> thing the pretty printer does is this:
>
>            if hasattr(obj_class, '__pretty__'):
>                return obj_class.__pretty__(obj, self, cycle)
>
> When I implemented the other printers, I wanted to make them as
> similar to pretty as possible, so I stuck with this ordering of
> priorities.  I don't mind using the other priority order as long as we
> are consistent and modify pretty to use the same order.  Is this what
> you would recommend Robert?
>
> Cheers,
>
> Brian
>
>
> > --
> > Robert Kern
> >
> > "I have come to believe that the whole world is an enigma, a harmless
> enigma
> >  that is made terrible by our own mad attempt to interpret it as though
> it had
> >  an underlying truth."
> >   -- Umberto Eco
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>

Be careful with __html__ (Django doesn't support it, I think all the others
you named do), it's not really intended to handle HTML formatting of an
object, but really just manage whether or not it's supposed to be escaped or
not.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110201/4902b2f3/attachment.html>

From takowl at gmail.com  Tue Feb  1 16:36:42 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 1 Feb 2011 21:36:42 +0000
Subject: [IPython-dev] Unicode in tests
In-Reply-To: <AANLkTik0zYQgGVitAOP6aomh9_2mVCHSg3soFUh+29er@mail.gmail.com>
References: <AANLkTini4eTaAFfD3u6d6aQJikqULy52JV15Y6qafFCW@mail.gmail.com>
	<ii9msc$32g$1@dough.gmane.org>
	<AANLkTikQZ2R=nF_4A1kDf81wiVZzCeisBeAR8LPRJwv2@mail.gmail.com>
	<AANLkTik0zYQgGVitAOP6aomh9_2mVCHSg3soFUh+29er@mail.gmail.com>
Message-ID: <AANLkTi=w6Dt7M+zXR12RS1XSxDqh=Tsd2QV2Vp7xrjDY@mail.gmail.com>

(Apologies, I have a bad habit of hitting reply instead of reply-all, so the
last two messages didn't go to the list)

If GTK is changing the default encoding, presumably there isn't an easy fix
from our point of view? So somehow, we need to compartmentalise the tests
which use GTK from the tests which use non-ASCII characters. Is my current
strategy (main test suite and unicode tests) sensible, or would it make more
sense to include unicode in the main test suite, and have any tests which
imported gtk separate? I think the latter could be hard to enforce as new
tests are added.

Or am I missing some way that the tests can be combined properly?

Thomas

On 1 February 2011 20:20, Robert Kern <robert.kern at gmail.com> wrote:

> GTK is the culprit.
>
> Robert Kern
> On Feb 1, 2011 11:54 AM, "Thomas Kluyver" <takowl at gmail.com> wrote:
> > It has what I assume to be the standard plugins shipped with nose. The
> > following are listed in sys.modules:
> >
> > allmodules, attrib, base, builtin, capture, collect, cover, cPickle,
> > cStringIO, debug, deprecated, doctest, doctests, errorclass,
> failuredetail,
> > hotshot, inspect, isolate, logcapture, logging, manager, multiprocess,
> nose,
> > optparse, os, pdb, pickle, pkg_resources, plugintest, prof, Queue, re,
> skip,
> > StringIO, sys, tempfile, testid, textwrap, threading, time, traceback,
> > unittest, warnings, xml, xunit
> >
> > They are all either showing a path within the virtualenv, or the module
> > itself is None.
> >
> > However, the various imports in the test suite have brought in over 1000
> > modules. Could something like Zope or GTK be the offending party (just
> > scanning down the list)?
> >
> > Thomas
> >
> > On 1 February 2011 19:28, Robert Kern <robert.kern at gmail.com> wrote:
> >
> >> On 2/1/11 4:02 AM, Thomas Kluyver wrote:
> >> > While testing the use of non-ascii characters in 0.11 recently, I
> found
> >> that
> >> > there are things which will fail while actually using ipython, but
> cannot
> >> be
> >> > made to fail in the test suite. Investigating, I found that for the
> test
> >> suite,
> >> > the default encoding is somehow set to utf-8 (ascii is the default
> >> default
> >> > encoding for Python 2). I don't know quite what's doing this, but it
> did
> >> the
> >> > same with my system version of nose (0.11.1 in Ubuntu) and with nose
> >> 1.0.0
> >> > installed in the virtualenv.
> >> >
> >> > At present, I've worked round that by adding an entirely separate
> >> > unicode_tests.py file, which has to be run separately from the main
> >> iptest test
> >> > suite.
> >> >
> >> > https://github.com/ipython/ipython/pull/252
> >> >
> >> > Does anyone have any insight on whether this is something our test
> suite
> >> is
> >> > doing, something nose is doing, something that can be fixed, or
> something
> >> that
> >> > we have to work around?
> >>
> >> There is no code in IPython or nose itself that does this. Can you make
> >> sure
> >> that nose is not picking up any extra plugins (print out sys.modules
> inside
> >> of a
> >> test suite)?
> >>
> >> --
> >> Robert Kern
> >>
> >> "I have come to believe that the whole world is an enigma, a harmless
> >> enigma
> >> that is made terrible by our own mad attempt to interpret it as though
> it
> >> had
> >> an underlying truth."
> >> -- Umberto Eco
> >>
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110201/57986d31/attachment.html>

From robert.kern at gmail.com  Tue Feb  1 17:24:40 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 01 Feb 2011 14:24:40 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>	<ii9o4q$abd$1@dough.gmane.org>	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
Message-ID: <iia179$u6s$1@dough.gmane.org>

On 2/1/11 1:32 PM, Alex Gaynor wrote:

> Be careful with __html__ (Django doesn't support it, I think all the others you
> named do), it's not really intended to handle HTML formatting of an object, but
> really just manage whether or not it's supposed to be escaped or not.

Right. It's a tool for frameworks/template libraries to implement internal 
SafeString/Markup/whatever objects, not for application/library authors to make 
HTML representations of their model objects. If a Jinja2 user is viewing a 
Markup object in IPython, they want to see "Markup(u'<p>foo</p>')", not an 
HTML-formatted "foo". The HTMLFormatter's behavior is undesirable, and I think 
this is a good reason to avoid using __html__, at least.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From robert.kern at gmail.com  Tue Feb  1 17:33:27 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 01 Feb 2011 14:33:27 -0800
Subject: [IPython-dev] Unicode in tests
In-Reply-To: <AANLkTi=w6Dt7M+zXR12RS1XSxDqh=Tsd2QV2Vp7xrjDY@mail.gmail.com>
References: <AANLkTini4eTaAFfD3u6d6aQJikqULy52JV15Y6qafFCW@mail.gmail.com>	<ii9msc$32g$1@dough.gmane.org>	<AANLkTikQZ2R=nF_4A1kDf81wiVZzCeisBeAR8LPRJwv2@mail.gmail.com>	<AANLkTik0zYQgGVitAOP6aomh9_2mVCHSg3soFUh+29er@mail.gmail.com>
	<AANLkTi=w6Dt7M+zXR12RS1XSxDqh=Tsd2QV2Vp7xrjDY@mail.gmail.com>
Message-ID: <iia1no$133$1@dough.gmane.org>

On 2/1/11 1:36 PM, Thomas Kluyver wrote:
> (Apologies, I have a bad habit of hitting reply instead of reply-all, so the
> last two messages didn't go to the list)
>
> If GTK is changing the default encoding, presumably there isn't an easy fix from
> our point of view? So somehow, we need to compartmentalise the tests which use
> GTK from the tests which use non-ASCII characters. Is my current strategy (main
> test suite and unicode tests) sensible, or would it make more sense to include
> unicode in the main test suite, and have any tests which imported gtk separate?
> I think the latter could be hard to enforce as new tests are added.
>
> Or am I missing some way that the tests can be combined properly?

The GTK tests may need to be separated. At startup, iptest currently will 
attempt to import gtk just to test if it exists. Perhaps it ought to exclude the 
GTK tests unless if they are explicitly requested.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ellisonbg at gmail.com  Tue Feb  1 19:21:18 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 1 Feb 2011 16:21:18 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <iia179$u6s$1@dough.gmane.org>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
	<iia179$u6s$1@dough.gmane.org>
Message-ID: <AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>

On Tue, Feb 1, 2011 at 2:24 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On 2/1/11 1:32 PM, Alex Gaynor wrote:
>
>> Be careful with __html__ (Django doesn't support it, I think all the others you
>> named do), it's not really intended to handle HTML formatting of an object, but
>> really just manage whether or not it's supposed to be escaped or not.
>
> Right. It's a tool for frameworks/template libraries to implement internal
> SafeString/Markup/whatever objects, not for application/library authors to make
> HTML representations of their model objects. If a Jinja2 user is viewing a
> Markup object in IPython, they want to see "Markup(u'<p>foo</p>')", not an
> HTML-formatted "foo". The HTMLFormatter's behavior is undesirable, and I think
> this is a good reason to avoid using __html__, at least.

OK, this is very useful information.  Let's find a different naming
convention for these.  What do people prefer?

Also, Robert, do you think we need to change the priority of the
special method/print function logic?

Brian

> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
> ?that is made terrible by our own mad attempt to interpret it as though it had
> ?an underlying truth."
> ? -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From benjaminrk at gmail.com  Tue Feb  1 20:13:16 2011
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 1 Feb 2011 17:13:16 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
	<iia179$u6s$1@dough.gmane.org>
	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
Message-ID: <AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>

On Tue, Feb 1, 2011 at 16:21, Brian Granger <ellisonbg at gmail.com> wrote:

> On Tue, Feb 1, 2011 at 2:24 PM, Robert Kern <robert.kern at gmail.com> wrote:
> > On 2/1/11 1:32 PM, Alex Gaynor wrote:
> >
> >> Be careful with __html__ (Django doesn't support it, I think all the
> others you
> >> named do), it's not really intended to handle HTML formatting of an
> object, but
> >> really just manage whether or not it's supposed to be escaped or not.
> >
> > Right. It's a tool for frameworks/template libraries to implement
> internal
> > SafeString/Markup/whatever objects, not for application/library authors
> to make
> > HTML representations of their model objects. If a Jinja2 user is viewing
> a
> > Markup object in IPython, they want to see "Markup(u'<p>foo</p>')", not
> an
> > HTML-formatted "foo". The HTMLFormatter's behavior is undesirable, and I
> think
> > this is a good reason to avoid using __html__, at least.
>
> OK, this is very useful information.  Let's find a different naming
> convention for these.  What do people prefer?
>

My vote might be for '__htmlrepr__',
I like 'htmlrepr' more than 'repr_html' because it's 'html representation'
not 'representation html', despite the tab-completion friendliness of the
latter pattern.

or we could use a mimetypes method that lets developers pick their own
methods by returning a dict:

def __mimetypes__(self):
      return { 'text/html' : self.htmlify,
          'image/png' : lambda self: self.as_image(type='png'),
      }



>
> Also, Robert, do you think we need to change the priority of the
> special method/print function logic?
>
> Brian
>
> > --
> > Robert Kern
> >
> > "I have come to believe that the whole world is an enigma, a harmless
> enigma
> >  that is made terrible by our own mad attempt to interpret it as though
> it had
> >  an underlying truth."
> >   -- Umberto Eco
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
>
>
>
> --
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110201/aec6da02/attachment.html>

From pivanov314 at gmail.com  Tue Feb  1 21:05:31 2011
From: pivanov314 at gmail.com (Paul Ivanov)
Date: Tue, 1 Feb 2011 18:05:31 -0800
Subject: [IPython-dev] New display methods,
 should we fine-tune our naming decision?
In-Reply-To: <AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
	<iia179$u6s$1@dough.gmane.org>
	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>
Message-ID: <20110202020531.GC423@ykcyc>

MinRK, on 2011-02-01 17:13,  wrote:
> On Tue, Feb 1, 2011 at 16:21, Brian Granger <ellisonbg at gmail.com> wrote:
> > Let's find a different naming convention for these.  What do
> > people prefer?
> 
> My vote might be for '__htmlrepr__',
> I like 'htmlrepr' more than 'repr_html' because it's 'html representation'
> not 'representation html', despite the tab-completion friendliness of the
> latter pattern.

I'd vote the other way, since obj.__repr_<tab> is a lot nicer to use
than writing a one-liner to get everything with a 'repr'
substring to see in what manner you can represent a given object.
Also, I don't think I'm alone in poking around an object I don't
know much about using just obj.<tab>, and the __repr_html__
convention would keep all of the reprs listed next to each other,
instead of sprinkled about. 

I read __repr_html__ as "representation: html", or "represented
as html", which makes sense from a course-to-fine /
general-to-specific type of convention.  Consider how annoying it
would be if get_foo and set_foo methods of some object swapped
the order of the verb and the noun and you couldn't easily figure
out what all the things you could get and set were. The same goes
for IPython.iplib.InteractiveShell.magic_*, which is easier to
wrap one's head around than .*_magic 

best,
-- 
Paul Ivanov
314 address only used for lists,  off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110201/2e58e893/attachment.sig>

From pivanov314 at gmail.com  Tue Feb  1 21:05:31 2011
From: pivanov314 at gmail.com (Paul Ivanov)
Date: Tue, 1 Feb 2011 18:05:31 -0800
Subject: [IPython-dev] New display methods,
 should we fine-tune our naming decision?
In-Reply-To: <AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
	<iia179$u6s$1@dough.gmane.org>
	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>
Message-ID: <20110202020531.GC423@ykcyc>

MinRK, on 2011-02-01 17:13,  wrote:
> On Tue, Feb 1, 2011 at 16:21, Brian Granger <ellisonbg at gmail.com> wrote:
> > Let's find a different naming convention for these.  What do
> > people prefer?
> 
> My vote might be for '__htmlrepr__',
> I like 'htmlrepr' more than 'repr_html' because it's 'html representation'
> not 'representation html', despite the tab-completion friendliness of the
> latter pattern.

I'd vote the other way, since obj.__repr_<tab> is a lot nicer to use
than writing a one-liner to get everything with a 'repr'
substring to see in what manner you can represent a given object.
Also, I don't think I'm alone in poking around an object I don't
know much about using just obj.<tab>, and the __repr_html__
convention would keep all of the reprs listed next to each other,
instead of sprinkled about. 

I read __repr_html__ as "representation: html", or "represented
as html", which makes sense from a course-to-fine /
general-to-specific type of convention.  Consider how annoying it
would be if get_foo and set_foo methods of some object swapped
the order of the verb and the noun and you couldn't easily figure
out what all the things you could get and set were. The same goes
for IPython.iplib.InteractiveShell.magic_*, which is easier to
wrap one's head around than .*_magic 

best,
-- 
Paul Ivanov
314 address only used for lists,  off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110201/2e58e893/attachment-0001.sig>

From benjaminrk at gmail.com  Tue Feb  1 21:16:52 2011
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 1 Feb 2011 18:16:52 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <20110202020531.GC423@ykcyc>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
	<iia179$u6s$1@dough.gmane.org>
	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>
	<20110202020531.GC423@ykcyc>
Message-ID: <AANLkTim7TfnZwvFkvJ+Y8_C9vqEc0YKM6RSmOnjZwTGy@mail.gmail.com>

On Tue, Feb 1, 2011 at 18:05, Paul Ivanov <pivanov314 at gmail.com> wrote:

> MinRK, on 2011-02-01 17:13,  wrote:
> > On Tue, Feb 1, 2011 at 16:21, Brian Granger <ellisonbg at gmail.com> wrote:
> > > Let's find a different naming convention for these.  What do
> > > people prefer?
> >
> > My vote might be for '__htmlrepr__',
> > I like 'htmlrepr' more than 'repr_html' because it's 'html
> representation'
> > not 'representation html', despite the tab-completion friendliness of the
> > latter pattern.
>
> I'd vote the other way, since obj.__repr_<tab> is a lot nicer to use
> than writing a one-liner to get everything with a 'repr'
> substring to see in what manner you can represent a given object.
> Also, I don't think I'm alone in poking around an object I don't
> know much about using just obj.<tab>, and the __repr_html__
> convention would keep all of the reprs listed next to each other,
> instead of sprinkled about.
>
> I read __repr_html__ as "representation: html", or "represented
> as html", which makes sense from a course-to-fine /
> general-to-specific type of convention.  Consider how annoying it
> would be if get_foo and set_foo methods of some object swapped
> the order of the verb and the noun and you couldn't easily figure
> out what all the things you could get and set were. The same goes
> for IPython.iplib.InteractiveShell.magic_*, which is easier to
> wrap one's head around than .*_magic
>

Well put, I'm convinced.

I still don't like having underscores in the middle of __x_y__ method names.
I think it's ugly, but I also think PEP8 is terrible, so maybe ignore me on
that point.

An alternative to developers writing methods in their classes would be for
the Display system to use its own lookup table for representation methods,
and provide an extension API for modules, IPython, third-parties, or users
to register additional methods for classes.  This would be more elaborate,
but would make the display system a better Python citizen, and would
eliminate the possiblity of collision with other projects' private
conventions.

-MinRK


>
> best,
> --
> Paul Ivanov
> 314 address only used for lists,  off-list direct email at:
> http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEARECAAYFAk1Iu9kACgkQe+cmRQ8+KPeVVQCfdgts2XwnsnKosaW1J3Aae7v+
> URMAn2qm+98LxbGpnV+8Bvmr58X3lWrL
> =QE5n
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110201/6283923c/attachment.html>

From benjaminrk at gmail.com  Tue Feb  1 21:16:52 2011
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 1 Feb 2011 18:16:52 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <20110202020531.GC423@ykcyc>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
	<iia179$u6s$1@dough.gmane.org>
	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>
	<20110202020531.GC423@ykcyc>
Message-ID: <AANLkTim7TfnZwvFkvJ+Y8_C9vqEc0YKM6RSmOnjZwTGy@mail.gmail.com>

On Tue, Feb 1, 2011 at 18:05, Paul Ivanov <pivanov314 at gmail.com> wrote:

> MinRK, on 2011-02-01 17:13,  wrote:
> > On Tue, Feb 1, 2011 at 16:21, Brian Granger <ellisonbg at gmail.com> wrote:
> > > Let's find a different naming convention for these.  What do
> > > people prefer?
> >
> > My vote might be for '__htmlrepr__',
> > I like 'htmlrepr' more than 'repr_html' because it's 'html
> representation'
> > not 'representation html', despite the tab-completion friendliness of the
> > latter pattern.
>
> I'd vote the other way, since obj.__repr_<tab> is a lot nicer to use
> than writing a one-liner to get everything with a 'repr'
> substring to see in what manner you can represent a given object.
> Also, I don't think I'm alone in poking around an object I don't
> know much about using just obj.<tab>, and the __repr_html__
> convention would keep all of the reprs listed next to each other,
> instead of sprinkled about.
>
> I read __repr_html__ as "representation: html", or "represented
> as html", which makes sense from a course-to-fine /
> general-to-specific type of convention.  Consider how annoying it
> would be if get_foo and set_foo methods of some object swapped
> the order of the verb and the noun and you couldn't easily figure
> out what all the things you could get and set were. The same goes
> for IPython.iplib.InteractiveShell.magic_*, which is easier to
> wrap one's head around than .*_magic
>

Well put, I'm convinced.

I still don't like having underscores in the middle of __x_y__ method names.
I think it's ugly, but I also think PEP8 is terrible, so maybe ignore me on
that point.

An alternative to developers writing methods in their classes would be for
the Display system to use its own lookup table for representation methods,
and provide an extension API for modules, IPython, third-parties, or users
to register additional methods for classes.  This would be more elaborate,
but would make the display system a better Python citizen, and would
eliminate the possiblity of collision with other projects' private
conventions.

-MinRK


>
> best,
> --
> Paul Ivanov
> 314 address only used for lists,  off-list direct email at:
> http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEARECAAYFAk1Iu9kACgkQe+cmRQ8+KPeVVQCfdgts2XwnsnKosaW1J3Aae7v+
> URMAn2qm+98LxbGpnV+8Bvmr58X3lWrL
> =QE5n
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110201/6283923c/attachment-0001.html>

From ellisonbg at gmail.com  Tue Feb  1 21:21:43 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 1 Feb 2011 18:21:43 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <20110202020531.GC423@ykcyc>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
	<iia179$u6s$1@dough.gmane.org>
	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>
	<20110202020531.GC423@ykcyc>
Message-ID: <AANLkTimrOXX1KpsXQO9nxViDoCv9okMNyNhXNLF+N1_u@mail.gmail.com>

On Tue, Feb 1, 2011 at 6:05 PM, Paul Ivanov <pivanov314 at gmail.com> wrote:
> MinRK, on 2011-02-01 17:13, ?wrote:
>> On Tue, Feb 1, 2011 at 16:21, Brian Granger <ellisonbg at gmail.com> wrote:
>> > Let's find a different naming convention for these. ?What do
>> > people prefer?
>>
>> My vote might be for '__htmlrepr__',
>> I like 'htmlrepr' more than 'repr_html' because it's 'html representation'
>> not 'representation html', despite the tab-completion friendliness of the
>> latter pattern.
>
> I'd vote the other way, since obj.__repr_<tab> is a lot nicer to use
> than writing a one-liner to get everything with a 'repr'
> substring to see in what manner you can represent a given object.
> Also, I don't think I'm alone in poking around an object I don't
> know much about using just obj.<tab>, and the __repr_html__
> convention would keep all of the reprs listed next to each other,
> instead of sprinkled about.

I agree, but do we want to avoid the double underscores, so it would be:

_repr_html_
_repr_svg_
...

> I read __repr_html__ as "representation: html", or "represented
> as html", which makes sense from a course-to-fine /
> general-to-specific type of convention. ?Consider how annoying it
> would be if get_foo and set_foo methods of some object swapped
> the order of the verb and the noun and you couldn't easily figure
> out what all the things you could get and set were. The same goes
> for IPython.iplib.InteractiveShell.magic_*, which is easier to
> wrap one's head around than .*_magic
>
> best,
> --
> Paul Ivanov
> 314 address only used for lists, ?off-list direct email at:
> http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEARECAAYFAk1Iu9kACgkQe+cmRQ8+KPeVVQCfdgts2XwnsnKosaW1J3Aae7v+
> URMAn2qm+98LxbGpnV+8Bvmr58X3lWrL
> =QE5n
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ellisonbg at gmail.com  Tue Feb  1 21:21:43 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 1 Feb 2011 18:21:43 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <20110202020531.GC423@ykcyc>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
	<iia179$u6s$1@dough.gmane.org>
	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>
	<20110202020531.GC423@ykcyc>
Message-ID: <AANLkTimrOXX1KpsXQO9nxViDoCv9okMNyNhXNLF+N1_u@mail.gmail.com>

On Tue, Feb 1, 2011 at 6:05 PM, Paul Ivanov <pivanov314 at gmail.com> wrote:
> MinRK, on 2011-02-01 17:13, ?wrote:
>> On Tue, Feb 1, 2011 at 16:21, Brian Granger <ellisonbg at gmail.com> wrote:
>> > Let's find a different naming convention for these. ?What do
>> > people prefer?
>>
>> My vote might be for '__htmlrepr__',
>> I like 'htmlrepr' more than 'repr_html' because it's 'html representation'
>> not 'representation html', despite the tab-completion friendliness of the
>> latter pattern.
>
> I'd vote the other way, since obj.__repr_<tab> is a lot nicer to use
> than writing a one-liner to get everything with a 'repr'
> substring to see in what manner you can represent a given object.
> Also, I don't think I'm alone in poking around an object I don't
> know much about using just obj.<tab>, and the __repr_html__
> convention would keep all of the reprs listed next to each other,
> instead of sprinkled about.

I agree, but do we want to avoid the double underscores, so it would be:

_repr_html_
_repr_svg_
...

> I read __repr_html__ as "representation: html", or "represented
> as html", which makes sense from a course-to-fine /
> general-to-specific type of convention. ?Consider how annoying it
> would be if get_foo and set_foo methods of some object swapped
> the order of the verb and the noun and you couldn't easily figure
> out what all the things you could get and set were. The same goes
> for IPython.iplib.InteractiveShell.magic_*, which is easier to
> wrap one's head around than .*_magic
>
> best,
> --
> Paul Ivanov
> 314 address only used for lists, ?off-list direct email at:
> http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEARECAAYFAk1Iu9kACgkQe+cmRQ8+KPeVVQCfdgts2XwnsnKosaW1J3Aae7v+
> URMAn2qm+98LxbGpnV+8Bvmr58X3lWrL
> =QE5n
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From robert.kern at gmail.com  Wed Feb  2 01:13:31 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 01 Feb 2011 22:13:31 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>	<ii9o4q$abd$1@dough.gmane.org>	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>	<iia179$u6s$1@dough.gmane.org>
	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
Message-ID: <iiasmb$gpq$1@dough.gmane.org>

On 2/1/11 4:21 PM, Brian Granger wrote:

> Also, Robert, do you think we need to change the priority of the
> special method/print function logic?

Yes.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From robert.kern at gmail.com  Wed Feb  2 01:18:30 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 01 Feb 2011 22:18:30 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <AANLkTim7TfnZwvFkvJ+Y8_C9vqEc0YKM6RSmOnjZwTGy@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>	<ii9o4q$abd$1@dough.gmane.org>	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>	<iia179$u6s$1@dough.gmane.org>	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>	<20110202020531.GC423@ykcyc>
	<AANLkTim7TfnZwvFkvJ+Y8_C9vqEc0YKM6RSmOnjZwTGy@mail.gmail.com>
Message-ID: <iiasvn$i0b$1@dough.gmane.org>

On 2/1/11 6:16 PM, MinRK wrote:

> An alternative to developers writing methods in their classes would be for the
> Display system to use its own lookup table for representation methods, and
> provide an extension API for modules, IPython, third-parties, or users to
> register additional methods for classes.  This would be more elaborate, but
> would make the display system a better Python citizen, and would eliminate the
> possiblity of collision with other projects' private conventions.

The code already provides this mechanism in addition to the special method, and 
I think this is the one that should be promoted as the recommended way to do it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From takowl at gmail.com  Wed Feb  2 05:08:53 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 2 Feb 2011 10:08:53 +0000
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>
	<ii9o4q$abd$1@dough.gmane.org>
	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>
	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>
	<iia179$u6s$1@dough.gmane.org>
	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>
	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>
Message-ID: <AANLkTikmnVzBWQzOo8CScS47JG-FscZDBvuz+5EFaOcf@mail.gmail.com>

On 2 February 2011 01:13, MinRK <benjaminrk at gmail.com> wrote:

> or we could use a mimetypes method that lets developers pick their own
> methods by returning a dict:


Thinking about it, I quite like this sort of approach, but I think it would
make more sense just to have a dict at the attribute, rather than a method
returning a dict. It makes it much easier to change on the fly.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110202/5037c472/attachment.html>

From walter at livinglogic.de  Wed Feb  2 12:09:45 2011
From: walter at livinglogic.de (=?UTF-8?B?V2FsdGVyIETDtnJ3YWxk?=)
Date: Wed, 02 Feb 2011 18:09:45 +0100
Subject: [IPython-dev] New display methods,
 should we fine-tune our naming decision?
In-Reply-To: <AANLkTim7TfnZwvFkvJ+Y8_C9vqEc0YKM6RSmOnjZwTGy@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>	<ii9o4q$abd$1@dough.gmane.org>	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>	<iia179$u6s$1@dough.gmane.org>	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>	<20110202020531.GC423@ykcyc>
	<AANLkTim7TfnZwvFkvJ+Y8_C9vqEc0YKM6RSmOnjZwTGy@mail.gmail.com>
Message-ID: <4D498FD9.8040606@livinglogic.de>

On 02.02.11 03:16, MinRK wrote:

> [...]
> An alternative to developers writing methods in their classes would be
> for the Display system to use its own lookup table for representation
> methods, and provide an extension API for modules, IPython,
> third-parties, or users to register additional methods for classes.
>  This would be more elaborate, but would make the display system a
> better Python citizen, and would eliminate the possiblity of collision
> with other projects' private conventions.

Why not use a generic function (i.e.
http://pypi.python.org/pypi/simplegeneric) for the method registry?

Servus,
   Walter


From walter at livinglogic.de  Wed Feb  2 12:09:45 2011
From: walter at livinglogic.de (=?UTF-8?B?V2FsdGVyIETDtnJ3YWxk?=)
Date: Wed, 02 Feb 2011 18:09:45 +0100
Subject: [IPython-dev] New display methods,
 should we fine-tune our naming decision?
In-Reply-To: <AANLkTim7TfnZwvFkvJ+Y8_C9vqEc0YKM6RSmOnjZwTGy@mail.gmail.com>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>	<ii9o4q$abd$1@dough.gmane.org>	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>	<iia179$u6s$1@dough.gmane.org>	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>	<20110202020531.GC423@ykcyc>
	<AANLkTim7TfnZwvFkvJ+Y8_C9vqEc0YKM6RSmOnjZwTGy@mail.gmail.com>
Message-ID: <4D498FD9.8040606@livinglogic.de>

On 02.02.11 03:16, MinRK wrote:

> [...]
> An alternative to developers writing methods in their classes would be
> for the Display system to use its own lookup table for representation
> methods, and provide an extension API for modules, IPython,
> third-parties, or users to register additional methods for classes.
>  This would be more elaborate, but would make the display system a
> better Python citizen, and would eliminate the possiblity of collision
> with other projects' private conventions.

Why not use a generic function (i.e.
http://pypi.python.org/pypi/simplegeneric) for the method registry?

Servus,
   Walter


From robert.kern at gmail.com  Wed Feb  2 12:41:17 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 02 Feb 2011 09:41:17 -0800
Subject: [IPython-dev] New display methods,
	should we fine-tune our naming decision?
In-Reply-To: <4D498FD9.8040606@livinglogic.de>
References: <AANLkTimBTxmw2bYmBKdji84=RWjE+hF9ywguu64ophYb@mail.gmail.com>	<ii9o4q$abd$1@dough.gmane.org>	<AANLkTikebH-DQsHy0vk6W4mTXZDsKBk0cLkEXaqqXDCQ@mail.gmail.com>	<AANLkTimAPROrk0MueEob9rPbxBrJ2qrXoqK06yEaDb9x@mail.gmail.com>	<iia179$u6s$1@dough.gmane.org>	<AANLkTi=xsNxOdQiCHq1jeocsobn37MCPHKeybJD3z26_@mail.gmail.com>	<AANLkTimXMkQKxDPV88vhk8T3mxzNtU--r1w1eBZbEsD0@mail.gmail.com>	<20110202020531.GC423@ykcyc>	<AANLkTim7TfnZwvFkvJ+Y8_C9vqEc0YKM6RSmOnjZwTGy@mail.gmail.com>
	<4D498FD9.8040606@livinglogic.de>
Message-ID: <iic4vu$32p$1@dough.gmane.org>

On 2/2/11 9:09 AM, Walter D?rwald wrote:
> On 02.02.11 03:16, MinRK wrote:
>
>> [...]
>> An alternative to developers writing methods in their classes would be
>> for the Display system to use its own lookup table for representation
>> methods, and provide an extension API for modules, IPython,
>> third-parties, or users to register additional methods for classes.
>>   This would be more elaborate, but would make the display system a
>> better Python citizen, and would eliminate the possiblity of collision
>> with other projects' private conventions.
>
> Why not use a generic function (i.e.
> http://pypi.python.org/pypi/simplegeneric) for the method registry?

Generic functions (as implemented by simplegeneric) are global, and it is 
difficult to "unregister" functions. The dispatch algorithm in DisplayFormatter 
is quite similar to that of simplegeneric, though.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From sunqiang at gmail.com  Thu Feb  3 07:16:55 2011
From: sunqiang at gmail.com (sunqiang)
Date: Thu, 3 Feb 2011 20:16:55 +0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
Message-ID: <AANLkTi=urLUjVhG=DwCzGN0gsUnrtkqgKOuOX+og4sfg@mail.gmail.com>

thanks for the code, it's interesting.
On Wed, Feb 2, 2011 at 1:56 AM, Nicolas Rougier
<Nicolas.rougier at inria.fr> wrote:
>
>
> Hi all,
> I've made some time ago a kind of 'ascii imshow' function to render a numpy
> array directly within the console by exploiting the 256 colors capability of
> most modern terminal. I don't know how much interesting it can be for
> IPython but I found the code useful for myself.
> Code is located
> at:?http://www.loria.fr/~rougier/coding/software/numpy_imshow.py
> (GPL license can be replaced by the BSD one if necessary)
>
> Nicolas
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


From epatters at enthought.com  Thu Feb  3 11:41:42 2011
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 3 Feb 2011 10:41:42 -0600
Subject: [IPython-dev] PySide support for the Qt frontend
Message-ID: <F624C3CF-6AD4-4BCF-AD4F-DE89715FD492@enthought.com>

Hello all,

Now that several PySide bugs relevant to IPython have been fixed, my port of IPython's Qt code to PySide is working. (Note that even the most recent PySide beta 5 release will segfault sporadically. This is fixed in the PySide master branch and IPython will work with the upcoming RC 1 release.) My branch is here:

https://github.com/epatters/ipython/tree/pyside-support

This should be fairly uncontroversial. The only thing I would like to direct attention to is the Qt API switcher:

https://github.com/epatters/ipython/blob/pyside-support/IPython/external/qt.py

I have modeled this on the way that ETS is currently handling switching Qt APIs. Any thoughts on this, however, would be welcome.

Also, regarding GitHub etiquette: should I merge with the master branch before I submit my pull request? My branch is now several weeks behind.

Thanks,
Evan

From ellisonbg at gmail.com  Thu Feb  3 13:49:48 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 3 Feb 2011 10:49:48 -0800
Subject: [IPython-dev] PySide support for the Qt frontend
In-Reply-To: <F624C3CF-6AD4-4BCF-AD4F-DE89715FD492@enthought.com>
References: <F624C3CF-6AD4-4BCF-AD4F-DE89715FD492@enthought.com>
Message-ID: <AANLkTi=sFJNmZmGNoX6pzTrDJy6-QHAZK88PT67Z8Et6@mail.gmail.com>

Evan,

Good to hear from you.  Hope things are going well.

> Now that several PySide bugs relevant to IPython have been fixed, my port of IPython's Qt code to PySide is working. (Note that even the most recent PySide beta 5 release will segfault sporadically. This is fixed in the PySide master branch and IPython will work with the upcoming RC 1 release.) My branch is here:
>
> https://github.com/epatters/ipython/tree/pyside-support

This is great!

> This should be fairly uncontroversial. The only thing I would like to direct attention to is the Qt API switcher:
>
> https://github.com/epatters/ipython/blob/pyside-support/IPython/external/qt.py

This is god to know about.  Overall, how ready would you say that
pyside is?  Is Enthought considering shipping pyside in a version of
EPD soon?

> I have modeled this on the way that ETS is currently handling switching Qt APIs. Any thoughts on this, however, would be welcome.
>
> Also, regarding GitHub etiquette: should I merge with the master branch before I submit my pull request? My branch is now several weeks behind.

No, your stuff should be pretty independent.  We may do a rebase, but
why won't you do a pull request so everyone can have a look.

Thanks for doing this!

Cheers,

Brian

> Thanks,
> Evan
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From robert.kern at gmail.com  Thu Feb  3 14:42:19 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 03 Feb 2011 11:42:19 -0800
Subject: [IPython-dev] PySide support for the Qt frontend
In-Reply-To: <AANLkTi=sFJNmZmGNoX6pzTrDJy6-QHAZK88PT67Z8Et6@mail.gmail.com>
References: <F624C3CF-6AD4-4BCF-AD4F-DE89715FD492@enthought.com>
	<AANLkTi=sFJNmZmGNoX6pzTrDJy6-QHAZK88PT67Z8Et6@mail.gmail.com>
Message-ID: <iif0et$kje$1@dough.gmane.org>

On 2/3/11 10:49 AM, Brian Granger wrote:

> Is Enthought considering shipping pyside in a version of
> EPD soon?

We have no particular timeline in mind. I'd guess by EPD 7.1, PySide will have a 
release solid enough to include.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From epatters at enthought.com  Thu Feb  3 14:42:51 2011
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 3 Feb 2011 13:42:51 -0600
Subject: [IPython-dev] PySide support for the Qt frontend
In-Reply-To: <AANLkTi=sFJNmZmGNoX6pzTrDJy6-QHAZK88PT67Z8Et6@mail.gmail.com>
References: <F624C3CF-6AD4-4BCF-AD4F-DE89715FD492@enthought.com>
	<AANLkTi=sFJNmZmGNoX6pzTrDJy6-QHAZK88PT67Z8Et6@mail.gmail.com>
Message-ID: <BCAC19B9-0F24-45E8-B7FA-8B8BD657AF06@enthought.com>

On Feb 3, 2011, at 12:49 PM, Brian Granger wrote:

> Evan,
> 
> Good to hear from you.  Hope things are going well.

Thanks, Brian. Good to hear from you as well. I'll be working at Enthought for a while, which means that I may do some more work on IPython over the next six months or so.

> 
>> Now that several PySide bugs relevant to IPython have been fixed, my port of IPython's Qt code to PySide is working. (Note that even the most recent PySide beta 5 release will segfault sporadically. This is fixed in the PySide master branch and IPython will work with the upcoming RC 1 release.) My branch is here:
>> 
>> https://github.com/epatters/ipython/tree/pyside-support
> 
> This is great!
> 
>> This should be fairly uncontroversial. The only thing I would like to direct attention to is the Qt API switcher:
>> 
>> https://github.com/epatters/ipython/blob/pyside-support/IPython/external/qt.py
> 
> This is god to know about.  Overall, how ready would you say that
> pyside is?  Is Enthought considering shipping pyside in a version of
> EPD soon?

PySide is quickly approaching some kind of stability. It looks like an RC 1 release is coming a week or two, with the 1.0 release not too far after. Although the upcoming EPD release (7.0) will not include PySide, as it is being released on the 8th, the subsequent release (7.1) will. At some point, PySide will replace wx as the default toolkit, although Enthought will continue to ship wx with EPD.

> 
>> I have modeled this on the way that ETS is currently handling switching Qt APIs. Any thoughts on this, however, would be welcome.
>> 
>> Also, regarding GitHub etiquette: should I merge with the master branch before I submit my pull request? My branch is now several weeks behind.
> 
> No, your stuff should be pretty independent.  We may do a rebase, but
> why won't you do a pull request so everyone can have a look.

Done.

Evan

> 
> Thanks for doing this!
> 
> Cheers,
> 
> Brian
> 
>> Thanks,
>> Evan
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
> 
> 
> 
> -- 
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com



From dsdale24 at gmail.com  Fri Feb  4 09:59:52 2011
From: dsdale24 at gmail.com (Darren Dale)
Date: Fri, 4 Feb 2011 09:59:52 -0500
Subject: [IPython-dev] ipython-qtconsole and docstrings
Message-ID: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>

Hi All,

This morning I updated my zmq/pyzmq and tried running
ipython-qtconsole from the master branch. I had let my zeromq
installation get behind ipython development, and so I'm rediscovering
how awesome is the qtconsole. Having docstrings pop up at "(" is
great. I noticed though, that depending on the location of the mouse
pointer, the docstring may only briefly appear. It seems more likely
to occur if the pointer is located inside the region where the
docstring will be rendered. I'm running "ipython-qtconsole --pylab" on
a Snow Leopard machine, with native (not X-based) Qt provided by
MacPorts. Has anyone else noticed something similar?

Who is working on the qtconsole? Is there a roadmap?

Darren


From takowl at gmail.com  Fri Feb  4 12:16:51 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 4 Feb 2011 17:16:51 +0000
Subject: [IPython-dev] Fwd: Debian packaging pyzmq
In-Reply-To: <AANLkTi=nrtezoP+SE25480JbNHC00p7M7UPqCCjBzuhk@mail.gmail.com>
References: <AANLkTikZPVaYpk3DHAe2d2GeMh3B_6M6g9uVFvEgQW_Y@mail.gmail.com>
	<AANLkTi=8pQyedJDv81=RxEKPfT1ByttuMca=HxnkfA4e@mail.gmail.com>
	<AANLkTikEHSwa95tJKf=LGLiUXNqsSA2oGYtQdtXcxCSJ@mail.gmail.com>
	<AANLkTimkbVmfheyfoz1waCoHAg3U=P2Y4gvJ7zdWwBT2@mail.gmail.com>
	<AANLkTimbnpnQAm6XVGnejU0J_78LnzBNstytJrROUCx=@mail.gmail.com>
	<AANLkTi=nrtezoP+SE25480JbNHC00p7M7UPqCCjBzuhk@mail.gmail.com>
Message-ID: <AANLkTikn3cAYC-MWqPNKBZSdjhFoigeQhHm5KGp87Sn1@mail.gmail.com>

Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and hopes to
be able to get it uploaded next week. If you're running Debian or Ubuntu,
and have a few minutes to spare, please check that the packaged version is
working as expected.

The x86 version is attached, and the amd64 version is at:
http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb

Thanks,
Thomas

---------- Forwarded message ----------
From: Miguel Landaeta <miguel at miguel.cc>
Date: 4 February 2011 14:48
Subject: Re: Debian packaging pyzmq
To: Thomas Kluyver <takowl at gmail.com>


On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> Thanks Miguel. I'll need an i386 package, I'm afraid (or some pointers on
> how to build one). I think the computer is 64 bit, but I'm sticking with a
> 32 bit OS for now.

No problem, take a look at the attached package.
Cheers,

--
Miguel Landaeta, miguel at miguel.cc
secure email with PGP 0x7D8967E9 available at http://keyserver.pgp.com/
"Faith means not wanting to know what is true." -- Nietzsche
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110204/71c543f0/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: python-zmq_2.0.10.1-0miguel1_i386.deb
Type: application/x-debian-package
Size: 280340 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110204/71c543f0/attachment.deb>

From benjaminrk at gmail.com  Fri Feb  4 12:19:20 2011
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 4 Feb 2011 09:19:20 -0800
Subject: [IPython-dev] ipython-qtconsole and docstrings
In-Reply-To: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
References: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
Message-ID: <AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>

On Fri, Feb 4, 2011 at 06:59, Darren Dale <dsdale24 at gmail.com> wrote:
> Hi All,
>
> This morning I updated my zmq/pyzmq and tried running
> ipython-qtconsole from the master branch. I had let my zeromq
> installation get behind ipython development, and so I'm rediscovering
> how awesome is the qtconsole. Having docstrings pop up at "(" is
> great. I noticed though, that depending on the location of the mouse
> pointer, the docstring may only briefly appear. It seems more likely
> to occur if the pointer is located inside the region where the
> docstring will be rendered. I'm running "ipython-qtconsole --pylab" on
> a Snow Leopard machine, with native (not X-based) Qt provided by
> MacPorts. Has anyone else noticed something similar?

Yes, this is my experience, also on OSX 10.6.6, with native PyQt from
their website.  The behavior is inconsistent, but seems more likely if
the cursor is hovering over the popup, and also more likely if the
popup is large.

Another (probably) OSX-specific glitch: You cannot copy with 'cmd-c',
because pressing command triggers an event that clears the selection.

>
> Who is working on the qtconsole? Is there a roadmap?
>
> Darren
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From ellisonbg at gmail.com  Fri Feb  4 12:37:54 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 4 Feb 2011 09:37:54 -0800
Subject: [IPython-dev] Fwd: Debian packaging pyzmq
In-Reply-To: <AANLkTikn3cAYC-MWqPNKBZSdjhFoigeQhHm5KGp87Sn1@mail.gmail.com>
References: <AANLkTikZPVaYpk3DHAe2d2GeMh3B_6M6g9uVFvEgQW_Y@mail.gmail.com>
	<AANLkTi=8pQyedJDv81=RxEKPfT1ByttuMca=HxnkfA4e@mail.gmail.com>
	<AANLkTikEHSwa95tJKf=LGLiUXNqsSA2oGYtQdtXcxCSJ@mail.gmail.com>
	<AANLkTimkbVmfheyfoz1waCoHAg3U=P2Y4gvJ7zdWwBT2@mail.gmail.com>
	<AANLkTimbnpnQAm6XVGnejU0J_78LnzBNstytJrROUCx=@mail.gmail.com>
	<AANLkTi=nrtezoP+SE25480JbNHC00p7M7UPqCCjBzuhk@mail.gmail.com>
	<AANLkTikn3cAYC-MWqPNKBZSdjhFoigeQhHm5KGp87Sn1@mail.gmail.com>
Message-ID: <AANLkTinexde=5QC3SNvMT51w9PBLR5_vajYN+v1Rv2W-@mail.gmail.com>

Thomas,

This is great!  Thanks Miguel, this is huge!  Where is the link to the
2.0.10 of zeromq itself.  I will definitely test this out.

Cheers,

Brian

On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and hopes to
> be able to get it uploaded next week. If you're running Debian or Ubuntu,
> and have a few minutes to spare, please check that the packaged version is
> working as expected.
>
> The x86 version is attached, and the amd64 version is at:
> http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb
>
> Thanks,
> Thomas
>
> ---------- Forwarded message ----------
> From: Miguel Landaeta <miguel at miguel.cc>
> Date: 4 February 2011 14:48
> Subject: Re: Debian packaging pyzmq
> To: Thomas Kluyver <takowl at gmail.com>
>
>
> On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>> Thanks Miguel. I'll need an i386 package, I'm afraid (or some pointers on
>> how to build one). I think the computer is 64 bit, but I'm sticking with a
>> 32 bit OS for now.
>
> No problem, take a look at the attached package.
> Cheers,
>
> --
> Miguel Landaeta, miguel at miguel.cc
> secure email with PGP 0x7D8967E9 available at http://keyserver.pgp.com/
> "Faith means not wanting to know what is true." -- Nietzsche
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From takowl at gmail.com  Fri Feb  4 12:49:02 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 4 Feb 2011 17:49:02 +0000
Subject: [IPython-dev] Fwd: Debian packaging pyzmq
In-Reply-To: <AANLkTinexde=5QC3SNvMT51w9PBLR5_vajYN+v1Rv2W-@mail.gmail.com>
References: <AANLkTikZPVaYpk3DHAe2d2GeMh3B_6M6g9uVFvEgQW_Y@mail.gmail.com>
	<AANLkTi=8pQyedJDv81=RxEKPfT1ByttuMca=HxnkfA4e@mail.gmail.com>
	<AANLkTikEHSwa95tJKf=LGLiUXNqsSA2oGYtQdtXcxCSJ@mail.gmail.com>
	<AANLkTimkbVmfheyfoz1waCoHAg3U=P2Y4gvJ7zdWwBT2@mail.gmail.com>
	<AANLkTimbnpnQAm6XVGnejU0J_78LnzBNstytJrROUCx=@mail.gmail.com>
	<AANLkTi=nrtezoP+SE25480JbNHC00p7M7UPqCCjBzuhk@mail.gmail.com>
	<AANLkTikn3cAYC-MWqPNKBZSdjhFoigeQhHm5KGp87Sn1@mail.gmail.com>
	<AANLkTinexde=5QC3SNvMT51w9PBLR5_vajYN+v1Rv2W-@mail.gmail.com>
Message-ID: <AANLkTikGwjtepWMNn2KmP=hbviY4H43TwAgNmNwLrkpm@mail.gmail.com>

Knew there was something I'd forget ;-)

http://packages.debian.org/sid/libzmq0

Thomas

On 4 February 2011 17:37, Brian Granger <ellisonbg at gmail.com> wrote:

> Thomas,
>
> This is great!  Thanks Miguel, this is huge!  Where is the link to the
> 2.0.10 of zeromq itself.  I will definitely test this out.
>
> Cheers,
>
> Brian
>
> On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and hopes
> to
> > be able to get it uploaded next week. If you're running Debian or Ubuntu,
> > and have a few minutes to spare, please check that the packaged version
> is
> > working as expected.
> >
> > The x86 version is attached, and the amd64 version is at:
> >
> http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb<http://alioth.debian.org/%7Enomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb>
> >
> > Thanks,
> > Thomas
> >
> > ---------- Forwarded message ----------
> > From: Miguel Landaeta <miguel at miguel.cc>
> > Date: 4 February 2011 14:48
> > Subject: Re: Debian packaging pyzmq
> > To: Thomas Kluyver <takowl at gmail.com>
> >
> >
> > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some pointers
> on
> >> how to build one). I think the computer is 64 bit, but I'm sticking with
> a
> >> 32 bit OS for now.
> >
> > No problem, take a look at the attached package.
> > Cheers,
> >
> > --
> > Miguel Landaeta, miguel at miguel.cc
> > secure email with PGP 0x7D8967E9 available at http://keyserver.pgp.com/
> > "Faith means not wanting to know what is true." -- Nietzsche
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
>
>
>
> --
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110204/644045f0/attachment.html>

From epatters at enthought.com  Fri Feb  4 13:03:50 2011
From: epatters at enthought.com (Evan Patterson)
Date: Fri, 4 Feb 2011 12:03:50 -0600
Subject: [IPython-dev] ipython-qtconsole and docstrings
In-Reply-To: <AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
References: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
	<AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
Message-ID: <AANLkTi=yZCS-voVczA1b+ZXGP=vXoGYdgsQ2gse=MzB-@mail.gmail.com>

On Fri, Feb 4, 2011 at 11:19 AM, MinRK <benjaminrk at gmail.com> wrote:

> On Fri, Feb 4, 2011 at 06:59, Darren Dale <dsdale24 at gmail.com> wrote:
> > Hi All,
> >
> > This morning I updated my zmq/pyzmq and tried running
> > ipython-qtconsole from the master branch. I had let my zeromq
> > installation get behind ipython development, and so I'm rediscovering
> > how awesome is the qtconsole. Having docstrings pop up at "(" is
> > great. I noticed though, that depending on the location of the mouse
> > pointer, the docstring may only briefly appear. It seems more likely
> > to occur if the pointer is located inside the region where the
> > docstring will be rendered. I'm running "ipython-qtconsole --pylab" on
> > a Snow Leopard machine, with native (not X-based) Qt provided by
> > MacPorts. Has anyone else noticed something similar?
>
> Yes, this is my experience, also on OSX 10.6.6, with native PyQt from
> their website.  The behavior is inconsistent, but seems more likely if
> the cursor is hovering over the popup, and also more likely if the
> popup is large.
>

This is not so much bug as it is a (mis)feature in the call tip
implementation.

https://github.com/ipython/ipython/blob/master/IPython/frontend/qt/console/call_tip_widget.py

The call tip autohides after a little if the mouse cursor is outside both
the console window and the tooltip frame. I did this because I thought it
was uncouth for an application to leave a (potentially very large) tooltip
up when the mouse was elsewhere. However, this is clearly a usability issue.
Perhaps a better solution would be to autohide the tooltip only if the
console window loses focus in the window manager?

>
> Another (probably) OSX-specific glitch: You cannot copy with 'cmd-c',
> because pressing command triggers an event that clears the selection.
>

I'm working on a Mac more regularly now, so I should be able to see this and
get it fixed soon.

Evan

>
> >
> > Who is working on the qtconsole? Is there a roadmap?
> >
> > Darren
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110204/de2b7fe6/attachment.html>

From benjaminrk at gmail.com  Fri Feb  4 13:24:47 2011
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 4 Feb 2011 10:24:47 -0800
Subject: [IPython-dev] ipython-qtconsole and docstrings
In-Reply-To: <AANLkTi=yZCS-voVczA1b+ZXGP=vXoGYdgsQ2gse=MzB-@mail.gmail.com>
References: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
	<AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
	<AANLkTi=yZCS-voVczA1b+ZXGP=vXoGYdgsQ2gse=MzB-@mail.gmail.com>
Message-ID: <AANLkTikx2zP=ebO+kkcnq4OxjT15_2n2pTA3GeTObnfM@mail.gmail.com>

On Fri, Feb 4, 2011 at 10:03, Evan Patterson <epatters at enthought.com> wrote:
>
>
> On Fri, Feb 4, 2011 at 11:19 AM, MinRK <benjaminrk at gmail.com> wrote:
>>
>> On Fri, Feb 4, 2011 at 06:59, Darren Dale <dsdale24 at gmail.com> wrote:
>> > Hi All,
>> >
>> > This morning I updated my zmq/pyzmq and tried running
>> > ipython-qtconsole from the master branch. I had let my zeromq
>> > installation get behind ipython development, and so I'm rediscovering
>> > how awesome is the qtconsole. Having docstrings pop up at "(" is
>> > great. I noticed though, that depending on the location of the mouse
>> > pointer, the docstring may only briefly appear. It seems more likely
>> > to occur if the pointer is located inside the region where the
>> > docstring will be rendered. I'm running "ipython-qtconsole --pylab" on
>> > a Snow Leopard machine, with native (not X-based) Qt provided by
>> > MacPorts. Has anyone else noticed something similar?
>>
>> Yes, this is my experience, also on OSX 10.6.6, with native PyQt from
>> their website. ?The behavior is inconsistent, but seems more likely if
>> the cursor is hovering over the popup, and also more likely if the
>> popup is large.
>
> This is not so much bug as it is a (mis)feature in the call tip
> implementation.
>
> https://github.com/ipython/ipython/blob/master/IPython/frontend/qt/console/call_tip_widget.py
>
> The call tip autohides after a little if the mouse cursor is outside both
> the console window and the tooltip frame. I did this because I thought it
> was uncouth for an application to leave a (potentially very large) tooltip
> up when the mouse was elsewhere. However, this is clearly a usability issue.
> Perhaps a better solution would be to autohide the tooltip only if the
> console window loses focus in the window manager?

Thanks for the clarification, and that makes sense as a design, but
the bug is that this is not the observed behavior.  The calltip is
incorrectly dismissing itself when the cursor is within the tooltip
frame at the time of pop-up.

Based on a cursory insertion of 'print' statements, there is a surplus
of 'Leave' events triggered, which results in the calltip being
inappropriately dismissed while the cursor is over it.

-MinRK

>>
>> Another (probably) OSX-specific glitch: You cannot copy with 'cmd-c',
>> because pressing command triggers an event that clears the selection.
>
> I'm working on a Mac more regularly now, so I should be able to see this and
> get it fixed soon.
>
> Evan
>>
>> >
>> > Who is working on the qtconsole? Is there a roadmap?
>> >
>> > Darren
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


From mark.voorhies at ucsf.edu  Fri Feb  4 15:56:02 2011
From: mark.voorhies at ucsf.edu (Mark Voorhies)
Date: Fri, 4 Feb 2011 12:56:02 -0800
Subject: [IPython-dev] Fwd: Debian packaging pyzmq
In-Reply-To: <AANLkTikGwjtepWMNn2KmP=hbviY4H43TwAgNmNwLrkpm@mail.gmail.com>
References: <AANLkTikZPVaYpk3DHAe2d2GeMh3B_6M6g9uVFvEgQW_Y@mail.gmail.com>
	<AANLkTinexde=5QC3SNvMT51w9PBLR5_vajYN+v1Rv2W-@mail.gmail.com>
	<AANLkTikGwjtepWMNn2KmP=hbviY4H43TwAgNmNwLrkpm@mail.gmail.com>
Message-ID: <201102041256.03222.mark.voorhies@ucsf.edu>

Awesome!

I can test on Debian PPC and AMD64 this weekend.  Any suggested
values of "working as expected" beyond running an iPython qtconsole 
session?

--Mark

On Friday, February 04, 2011 09:49:02 am Thomas Kluyver wrote:
> Knew there was something I'd forget ;-)
> 
> http://packages.debian.org/sid/libzmq0
> 
> Thomas
> 
> On 4 February 2011 17:37, Brian Granger <ellisonbg at gmail.com> wrote:
>  
> Thomas,
>  
>  This is great!  Thanks Miguel, this is huge!  Where is the link to the
>  2.0.10 of zeromq itself.  I will definitely test this out.
>  
>  Cheers,
>  
>  Brian
>  
> 
>  On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>  > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and hopes to
>  > be able to get it uploaded next week. If you're running Debian or Ubuntu,
>  > and have a few minutes to spare, please check that the packaged version is
>  > working as expected.
>  >
>  > The x86 version is attached, and the amd64 version is at:
>  > http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb
>  >
>  > Thanks,
>  > Thomas
>  >
>  > ---------- Forwarded message ----------
>  > From: Miguel Landaeta <miguel at miguel.cc>
>  > Date: 4 February 2011 14:48
>  > Subject: Re: Debian packaging pyzmq
>  > To: Thomas Kluyver <takowl at gmail.com>
>  >
>  >
>  > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>  >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some pointers on
>  >> how to build one). I think the computer is 64 bit, but I'm sticking with a
>  >> 32 bit OS for now.
>  >
>  > No problem, take a look at the attached package.
>  > Cheers,
>  >
>  > --
>  > Miguel Landaeta, miguel at miguel.cc
>  > secure email with PGP 0x7D8967E9 available at http://keyserver.pgp.com/
>  > "Faith means not wanting to know what is true." -- Nietzsche
>  >
>  >
>  
> > _______________________________________________
>  > IPython-dev mailing list
>  > IPython-dev at scipy.org
>  > http://mail.scipy.org/mailman/listinfo/ipython-dev
>  >
>  >
>  
>  
>  
>  --
>  Brian E. Granger, Ph.D.
>  Assistant Professor of Physics
>  Cal Poly State University, San Luis Obispo
>  bgranger at calpoly.edu
>  ellisonbg at gmail.com
> 


From takowl at gmail.com  Fri Feb  4 16:08:05 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 4 Feb 2011 21:08:05 +0000
Subject: [IPython-dev] Fwd: Debian packaging pyzmq
In-Reply-To: <201102041256.03222.mark.voorhies@ucsf.edu>
References: <AANLkTikZPVaYpk3DHAe2d2GeMh3B_6M6g9uVFvEgQW_Y@mail.gmail.com>
	<AANLkTinexde=5QC3SNvMT51w9PBLR5_vajYN+v1Rv2W-@mail.gmail.com>
	<AANLkTikGwjtepWMNn2KmP=hbviY4H43TwAgNmNwLrkpm@mail.gmail.com>
	<201102041256.03222.mark.voorhies@ucsf.edu>
Message-ID: <AANLkTikU177Ok0kBKyf1BVjx+GqUbfkxdYWwqRdwf9x2@mail.gmail.com>

Well, that's a good start. pyzmq people, is there a simple way to run the
test suite against a separately installed version of pyzmq? Miguel, is there
some standard we're aiming to confirm the package meets for its inclusion?

Thomas

On 4 February 2011 20:56, Mark Voorhies <mark.voorhies at ucsf.edu> wrote:

> Awesome!
>
> I can test on Debian PPC and AMD64 this weekend.  Any suggested
> values of "working as expected" beyond running an iPython qtconsole
> session?
>
> --Mark
>
> On Friday, February 04, 2011 09:49:02 am Thomas Kluyver wrote:
> > Knew there was something I'd forget ;-)
> >
> > http://packages.debian.org/sid/libzmq0
> >
> > Thomas
> >
> > On 4 February 2011 17:37, Brian Granger <ellisonbg at gmail.com> wrote:
> >
> > Thomas,
> >
> >  This is great!  Thanks Miguel, this is huge!  Where is the link to the
> >  2.0.10 of zeromq itself.  I will definitely test this out.
> >
> >  Cheers,
> >
> >  Brian
> >
> >
> >  On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver <takowl at gmail.com>
> wrote:
> >  > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and
> hopes to
> >  > be able to get it uploaded next week. If you're running Debian or
> Ubuntu,
> >  > and have a few minutes to spare, please check that the packaged
> version is
> >  > working as expected.
> >  >
> >  > The x86 version is attached, and the amd64 version is at:
> >  >
> http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb<http://alioth.debian.org/%7Enomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb>
> >  >
> >  > Thanks,
> >  > Thomas
> >  >
> >  > ---------- Forwarded message ----------
> >  > From: Miguel Landaeta <miguel at miguel.cc>
> >  > Date: 4 February 2011 14:48
> >  > Subject: Re: Debian packaging pyzmq
> >  > To: Thomas Kluyver <takowl at gmail.com>
> >  >
> >  >
> >  > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver <takowl at gmail.com>
> wrote:
> >  >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some
> pointers on
> >  >> how to build one). I think the computer is 64 bit, but I'm sticking
> with a
> >  >> 32 bit OS for now.
> >  >
> >  > No problem, take a look at the attached package.
> >  > Cheers,
> >  >
> >  > --
> >  > Miguel Landaeta, miguel at miguel.cc
> >  > secure email with PGP 0x7D8967E9 available at
> http://keyserver.pgp.com/
> >  > "Faith means not wanting to know what is true." -- Nietzsche
> >  >
> >  >
> >
> > > _______________________________________________
> >  > IPython-dev mailing list
> >  > IPython-dev at scipy.org
> >  > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >  >
> >  >
> >
> >
> >
> >  --
> >  Brian E. Granger, Ph.D.
> >  Assistant Professor of Physics
> >  Cal Poly State University, San Luis Obispo
> >  bgranger at calpoly.edu
> >  ellisonbg at gmail.com
> >
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110204/ebf2ba9b/attachment.html>

From benjaminrk at gmail.com  Fri Feb  4 17:23:15 2011
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 4 Feb 2011 14:23:15 -0800
Subject: [IPython-dev] Fwd: Debian packaging pyzmq
In-Reply-To: <AANLkTikU177Ok0kBKyf1BVjx+GqUbfkxdYWwqRdwf9x2@mail.gmail.com>
References: <AANLkTikZPVaYpk3DHAe2d2GeMh3B_6M6g9uVFvEgQW_Y@mail.gmail.com>
	<AANLkTinexde=5QC3SNvMT51w9PBLR5_vajYN+v1Rv2W-@mail.gmail.com>
	<AANLkTikGwjtepWMNn2KmP=hbviY4H43TwAgNmNwLrkpm@mail.gmail.com>
	<201102041256.03222.mark.voorhies@ucsf.edu>
	<AANLkTikU177Ok0kBKyf1BVjx+GqUbfkxdYWwqRdwf9x2@mail.gmail.com>
Message-ID: <AANLkTim5Yaes_wKssrC7R=bihybJT_vspzEAa-dEJezp@mail.gmail.com>

On Fri, Feb 4, 2011 at 13:08, Thomas Kluyver <takowl at gmail.com> wrote:
> Well, that's a good start. pyzmq people, is there a simple way to run the
> test suite against a separately installed version of pyzmq?

With nose, you can just do:

nosetests -vvs zmq.tests

to run the zmq test suite from anywhere. It shouldn't matter how
pyzmq/zeromq are installed, as long as your PATHs are correct.

-MinRK

> Miguel, is there
> some standard we're aiming to confirm the package meets for its inclusion?
>
> Thomas
>
> On 4 February 2011 20:56, Mark Voorhies <mark.voorhies at ucsf.edu> wrote:
>>
>> Awesome!
>>
>> I can test on Debian PPC and AMD64 this weekend. ?Any suggested
>> values of "working as expected" beyond running an iPython qtconsole
>> session?
>>
>> --Mark
>>
>> On Friday, February 04, 2011 09:49:02 am Thomas Kluyver wrote:
>> > Knew there was something I'd forget ;-)
>> >
>> > http://packages.debian.org/sid/libzmq0
>> >
>> > Thomas
>> >
>> > On 4 February 2011 17:37, Brian Granger <ellisonbg at gmail.com> wrote:
>> >
>> > Thomas,
>> >
>> > ?This is great! ?Thanks Miguel, this is huge! ?Where is the link to the
>> > ?2.0.10 of zeromq itself. ?I will definitely test this out.
>> >
>> > ?Cheers,
>> >
>> > ?Brian
>> >
>> >
>> > ?On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver <takowl at gmail.com>
>> > wrote:
>> > ?> Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and
>> > hopes to
>> > ?> be able to get it uploaded next week. If you're running Debian or
>> > Ubuntu,
>> > ?> and have a few minutes to spare, please check that the packaged
>> > version is
>> > ?> working as expected.
>> > ?>
>> > ?> The x86 version is attached, and the amd64 version is at:
>> > ?>
>> > http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb
>> > ?>
>> > ?> Thanks,
>> > ?> Thomas
>> > ?>
>> > ?> ---------- Forwarded message ----------
>> > ?> From: Miguel Landaeta <miguel at miguel.cc>
>> > ?> Date: 4 February 2011 14:48
>> > ?> Subject: Re: Debian packaging pyzmq
>> > ?> To: Thomas Kluyver <takowl at gmail.com>
>> > ?>
>> > ?>
>> > ?> On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver <takowl at gmail.com>
>> > wrote:
>> > ?>> Thanks Miguel. I'll need an i386 package, I'm afraid (or some
>> > pointers on
>> > ?>> how to build one). I think the computer is 64 bit, but I'm sticking
>> > with a
>> > ?>> 32 bit OS for now.
>> > ?>
>> > ?> No problem, take a look at the attached package.
>> > ?> Cheers,
>> > ?>
>> > ?> --
>> > ?> Miguel Landaeta, miguel at miguel.cc
>> > ?> secure email with PGP 0x7D8967E9 available at
>> > http://keyserver.pgp.com/
>> > ?> "Faith means not wanting to know what is true." -- Nietzsche
>> > ?>
>> > ?>
>> >
>> > > _______________________________________________
>> > ?> IPython-dev mailing list
>> > ?> IPython-dev at scipy.org
>> > ?> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> > ?>
>> > ?>
>> >
>> >
>> >
>> > ?--
>> > ?Brian E. Granger, Ph.D.
>> > ?Assistant Professor of Physics
>> > ?Cal Poly State University, San Luis Obispo
>> > ?bgranger at calpoly.edu
>> > ?ellisonbg at gmail.com
>> >
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


From takowl at gmail.com  Fri Feb  4 18:26:56 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 4 Feb 2011 23:26:56 +0000
Subject: [IPython-dev] Fwd: Debian packaging pyzmq
In-Reply-To: <AANLkTim5Yaes_wKssrC7R=bihybJT_vspzEAa-dEJezp@mail.gmail.com>
References: <AANLkTikZPVaYpk3DHAe2d2GeMh3B_6M6g9uVFvEgQW_Y@mail.gmail.com>
	<AANLkTinexde=5QC3SNvMT51w9PBLR5_vajYN+v1Rv2W-@mail.gmail.com>
	<AANLkTikGwjtepWMNn2KmP=hbviY4H43TwAgNmNwLrkpm@mail.gmail.com>
	<201102041256.03222.mark.voorhies@ucsf.edu>
	<AANLkTikU177Ok0kBKyf1BVjx+GqUbfkxdYWwqRdwf9x2@mail.gmail.com>
	<AANLkTim5Yaes_wKssrC7R=bihybJT_vspzEAa-dEJezp@mail.gmail.com>
Message-ID: <AANLkTin1V-_MN-o=2KLvymszKt8_jiCDW-4_PcD+S2VZ@mail.gmail.com>

Well, after a bit of fiddling, I can confirm that the x86 package passes the
tests and runs the Qt Console frontend alright.

Anyone else attempting to test on Ubuntu: the dependencies specify a debian
specific version of Python. To get round this, use:
sudo dpkg -i --force-depends-version <debfile>

Once this is uploaded to Debian, I'll put in a sync request for Ubuntu.

Thomas

On 4 February 2011 22:23, MinRK <benjaminrk at gmail.com> wrote:

> On Fri, Feb 4, 2011 at 13:08, Thomas Kluyver <takowl at gmail.com> wrote:
> > Well, that's a good start. pyzmq people, is there a simple way to run the
> > test suite against a separately installed version of pyzmq?
>
> With nose, you can just do:
>
> nosetests -vvs zmq.tests
>
> to run the zmq test suite from anywhere. It shouldn't matter how
> pyzmq/zeromq are installed, as long as your PATHs are correct.
>
> -MinRK
>
> > Miguel, is there
> > some standard we're aiming to confirm the package meets for its
> inclusion?
> >
> > Thomas
> >
> > On 4 February 2011 20:56, Mark Voorhies <mark.voorhies at ucsf.edu> wrote:
> >>
> >> Awesome!
> >>
> >> I can test on Debian PPC and AMD64 this weekend.  Any suggested
> >> values of "working as expected" beyond running an iPython qtconsole
> >> session?
> >>
> >> --Mark
> >>
> >> On Friday, February 04, 2011 09:49:02 am Thomas Kluyver wrote:
> >> > Knew there was something I'd forget ;-)
> >> >
> >> > http://packages.debian.org/sid/libzmq0
> >> >
> >> > Thomas
> >> >
> >> > On 4 February 2011 17:37, Brian Granger <ellisonbg at gmail.com> wrote:
> >> >
> >> > Thomas,
> >> >
> >> >  This is great!  Thanks Miguel, this is huge!  Where is the link to
> the
> >> >  2.0.10 of zeromq itself.  I will definitely test this out.
> >> >
> >> >  Cheers,
> >> >
> >> >  Brian
> >> >
> >> >
> >> >  On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver <takowl at gmail.com>
> >> > wrote:
> >> >  > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and
> >> > hopes to
> >> >  > be able to get it uploaded next week. If you're running Debian or
> >> > Ubuntu,
> >> >  > and have a few minutes to spare, please check that the packaged
> >> > version is
> >> >  > working as expected.
> >> >  >
> >> >  > The x86 version is attached, and the amd64 version is at:
> >> >  >
> >> >
> http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb<http://alioth.debian.org/%7Enomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb>
> >> >  >
> >> >  > Thanks,
> >> >  > Thomas
> >> >  >
> >> >  > ---------- Forwarded message ----------
> >> >  > From: Miguel Landaeta <miguel at miguel.cc>
> >> >  > Date: 4 February 2011 14:48
> >> >  > Subject: Re: Debian packaging pyzmq
> >> >  > To: Thomas Kluyver <takowl at gmail.com>
> >> >  >
> >> >  >
> >> >  > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver <takowl at gmail.com>
> >> > wrote:
> >> >  >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some
> >> > pointers on
> >> >  >> how to build one). I think the computer is 64 bit, but I'm
> sticking
> >> > with a
> >> >  >> 32 bit OS for now.
> >> >  >
> >> >  > No problem, take a look at the attached package.
> >> >  > Cheers,
> >> >  >
> >> >  > --
> >> >  > Miguel Landaeta, miguel at miguel.cc
> >> >  > secure email with PGP 0x7D8967E9 available at
> >> > http://keyserver.pgp.com/
> >> >  > "Faith means not wanting to know what is true." -- Nietzsche
> >> >  >
> >> >  >
> >> >
> >> > > _______________________________________________
> >> >  > IPython-dev mailing list
> >> >  > IPython-dev at scipy.org
> >> >  > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> >  >
> >> >  >
> >> >
> >> >
> >> >
> >> >  --
> >> >  Brian E. Granger, Ph.D.
> >> >  Assistant Professor of Physics
> >> >  Cal Poly State University, San Luis Obispo
> >> >  bgranger at calpoly.edu
> >> >  ellisonbg at gmail.com
> >> >
> >> _______________________________________________
> >> IPython-dev mailing list
> >> IPython-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110204/c9547875/attachment.html>

From takowl at gmail.com  Sat Feb  5 13:23:28 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 5 Feb 2011 18:23:28 +0000
Subject: [IPython-dev] Unicode in tests
In-Reply-To: <iia1no$133$1@dough.gmane.org>
References: <AANLkTini4eTaAFfD3u6d6aQJikqULy52JV15Y6qafFCW@mail.gmail.com>
	<ii9msc$32g$1@dough.gmane.org>
	<AANLkTikQZ2R=nF_4A1kDf81wiVZzCeisBeAR8LPRJwv2@mail.gmail.com>
	<AANLkTik0zYQgGVitAOP6aomh9_2mVCHSg3soFUh+29er@mail.gmail.com>
	<AANLkTi=w6Dt7M+zXR12RS1XSxDqh=Tsd2QV2Vp7xrjDY@mail.gmail.com>
	<iia1no$133$1@dough.gmane.org>
Message-ID: <AANLkTinfyZqWuq0pM1huEx+LXDO9VeSq7zA5AE5tKxa8@mail.gmail.com>

On 1 February 2011 22:33, Robert Kern <robert.kern at gmail.com> wrote:

> The GTK tests may need to be separated. At startup, iptest currently will
> attempt to import gtk just to test if it exists. Perhaps it ought to
> exclude the
> GTK tests unless if they are explicitly requested.
>

What's the best way to deal with this? What tests use gtk?

If we're going to need two separate runs (one for gtk, one for Unicode), it
seems to me simpler to have two test scripts, rather than adding a command
line switch to determine which tests will be run.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110205/1bb48bb0/attachment.html>

From takowl at gmail.com  Sun Feb  6 11:00:28 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 6 Feb 2011 16:00:28 +0000
Subject: [IPython-dev] Selecting lines from history
Message-ID: <AANLkTinFQSYY24hOSW6h+TCQmK9MbmM3P9tD5eoYt3rJ@mail.gmail.com>

With the new history system, the history from previous sessions is stored in
the same lists as the history from the current session (input_history_raw,
input_history_parsed). So, when selecting lines with magic commands
%save/%edit/%macro, the numbers didn't match up to the prompt numbers in the
current session (except when you start with no history).

The approach I've taken to dealing with this is to add a session_offset,
which is used by these cases to translate session line numbers (starting
from 1) to indices in the history lists. This has the added benefit that,
using a default offset of -1, we don't need a blank item at the top of the
list to use 1-based indexing. In principle, you can select lines from
previous sessions using negative numbers; in practice, the - is interpreted
to indicate options, and the command fails.

https://github.com/ipython/ipython/pull/261

Other possible approaches I can think of:
- Use a dict instead of a list, with prompt numbers as keys. Previous
sessions could be stored with another form of keys, perhaps tuples (-1, 1)
(=first line of last session), and if desirable, we could come up with some
syntax to let magic commands access these. Separating sessions would also
allow a command line option to repeat the commands saved from the previous
session, e.g. after a power failure.
- Previous sessions go into a separate list from the current session. Again,
some syntax could be devised to allow magic commands to access them.
- Only load previous sessions for readline; don't allow access from magic
commands until they're re-used in the current session. I believe this is
what we did before the new history system.

What was the rationale behind saving and reloading the history as we now do?
Do we want to be able to do something like "%edit x1-x12"? That seems to
have limited value when you can't immediately see the lines you're selecting
(although of course the %hist command could have options to show them). Is
there another reason we store history that I've overlooked?

There's also the mysterious "shadow history" system, which I'm not entirely
clear about the reasons for.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110206/879d3358/attachment.html>

From satra at mit.edu  Sun Feb  6 15:47:55 2011
From: satra at mit.edu (Satrajit Ghosh)
Date: Sun, 6 Feb 2011 15:47:55 -0500
Subject: [IPython-dev] newparallel
Message-ID: <AANLkTin-H4MC-2F2sSGfBqKzX23T1ySVynEQUKfR96Z5@mail.gmail.com>

hi min, brian and fernando,

i just checked out the newparallel branch from the github repo. a few
questions:

1. is this the best place to look at documentation?

http://minrk.github.com/ipython/parallelz/index.html

(unfortunately, i could not compile the docs on my 64-bit osx machine
without wx.)

2. is dag support already available in newparallel? i.e., can i add tasks
with dependencies before executing them? if not, where is the best place for
hacking this in?

3. fernando: is your workflow branch mergeable with this? the ability to
spawn and shutdown engines as needed (especially in the context of clusters
such as SGE) has come up for discussion on the nipype side a fair bit in the
last few weeks.

4. is there a good place to read about the parallel architecture?

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110206/754530db/attachment.html>

From benjaminrk at gmail.com  Sun Feb  6 18:42:40 2011
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 6 Feb 2011 15:42:40 -0800
Subject: [IPython-dev] newparallel
In-Reply-To: <AANLkTin-H4MC-2F2sSGfBqKzX23T1ySVynEQUKfR96Z5@mail.gmail.com>
References: <AANLkTin-H4MC-2F2sSGfBqKzX23T1ySVynEQUKfR96Z5@mail.gmail.com>
Message-ID: <AANLkTinff4MqtXA4O8pJqdQ2a0LMJGrG3D49-2k5mX6o@mail.gmail.com>

On Sun, Feb 6, 2011 at 12:47, Satrajit Ghosh <satra at mit.edu> wrote:
> hi min, brian and fernando,
>
> i just checked out the newparallel branch from the github repo. a few
> questions:
>
> 1. is this the best place to look at documentation?
>
> http://minrk.github.com/ipython/parallelz/index.html

Yes, but I only just started updating those to match the newparallel
code, so they aren't fully up to date.

>
> (unfortunately, i could not compile the docs on my 64-bit osx machine
> without wx.)

wxpython 2.9 now supports cocoa, but if you want to use it with the
System default Python, you need to build from source (which takes
ages).

>
> 2. is dag support already available in newparallel? i.e., can i add tasks
> with dependencies before executing them? if not, where is the best place for
> hacking this in?

Yes, you can implement a DAG of dependencies, as seen here:
https://github.com/ipython/ipython/blob/newparallel/examples/demo/dag/dagdeps.py.

Note that you must use one of the Python schedulers (via '--scheme
lru' to ipcontrollerz), because the default pure-zmq scheduler ignores
dependencies.

There is probably some more work to go into Dependencies, specifically
regarding how to interpret job failures and unreachable DAG nodes.

>
> 3. fernando: is your workflow branch mergeable with this? the ability to
> spawn and shutdown engines as needed (especially in the context of clusters
> such as SGE) has come up for discussion on the nipype side a fair bit in the
> last few weeks.

The last commit of that branch has not been merged yet, but over the
last week I have updated the newparallel scripts to use the IPython
config system, so starting an engine is no longer the same as
represented in those scripts, but it should be easy to update them.

>
> 4. is there a good place to read about the parallel architecture?

The parallel messaging/connections documents are the best description
we have of how the underlying architecture works, but they still need
to be fleshed out:

http://minrk.github.com/ipython/development/parallel_messages.html

-MinRK

>
> cheers,
>
> satra
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


From robert.kern at gmail.com  Sun Feb  6 20:01:39 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 06 Feb 2011 19:01:39 -0600
Subject: [IPython-dev] Unicode in tests
In-Reply-To: <AANLkTinfyZqWuq0pM1huEx+LXDO9VeSq7zA5AE5tKxa8@mail.gmail.com>
References: <AANLkTini4eTaAFfD3u6d6aQJikqULy52JV15Y6qafFCW@mail.gmail.com>	<ii9msc$32g$1@dough.gmane.org>	<AANLkTikQZ2R=nF_4A1kDf81wiVZzCeisBeAR8LPRJwv2@mail.gmail.com>	<AANLkTik0zYQgGVitAOP6aomh9_2mVCHSg3soFUh+29er@mail.gmail.com>	<AANLkTi=w6Dt7M+zXR12RS1XSxDqh=Tsd2QV2Vp7xrjDY@mail.gmail.com>	<iia1no$133$1@dough.gmane.org>
	<AANLkTinfyZqWuq0pM1huEx+LXDO9VeSq7zA5AE5tKxa8@mail.gmail.com>
Message-ID: <iing9k$b2g$1@dough.gmane.org>

On 2011-02-05 12:23 , Thomas Kluyver wrote:
> On 1 February 2011 22:33, Robert Kern <robert.kern at gmail.com
> <mailto:robert.kern at gmail.com>> wrote:
>
>     The GTK tests may need to be separated. At startup, iptest currently will
>     attempt to import gtk just to test if it exists. Perhaps it ought to exclude the
>     GTK tests unless if they are explicitly requested.
>
>
> What's the best way to deal with this? What tests use gtk?

I don't think there are any tests that use gtk. iptest just checks for gtk to 
determine whether to add IPython/lib/inputhookgtk.py to the list of module to 
exclude from the test suite. This can just be made unconditional and remove the 
test for gtk and gobject. Try the attached patch.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: iptest-ungtk.diff
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110206/46e040e6/attachment.ksh>

From mark.voorhies at ucsf.edu  Sun Feb  6 23:45:24 2011
From: mark.voorhies at ucsf.edu (Mark Voorhies)
Date: Sun, 6 Feb 2011 20:45:24 -0800
Subject: [IPython-dev] Fwd: Debian packaging pyzmq
In-Reply-To: <AANLkTin1V-_MN-o=2KLvymszKt8_jiCDW-4_PcD+S2VZ@mail.gmail.com>
References: <AANLkTikZPVaYpk3DHAe2d2GeMh3B_6M6g9uVFvEgQW_Y@mail.gmail.com>
	<AANLkTim5Yaes_wKssrC7R=bihybJT_vspzEAa-dEJezp@mail.gmail.com>
	<AANLkTin1V-_MN-o=2KLvymszKt8_jiCDW-4_PcD+S2VZ@mail.gmail.com>
Message-ID: <201102062045.25082.mark.voorhies@ucsf.edu>

I can confirm that the source package works when built for PPC.

--

Built and tested on a PowerBook G4 15", freshly upgraded to Debian Squeeze.

All dependencies were installed from the squeeze respository, except:
     libzmq-dev 2.0.10-1 from sid
     cython and cython-dbg 0.13.-1 from experimental

Downloaded Miguel's source package from alioth, built with "debuild -us -uc",
    and installed with "dpkg --install python-zmq_2.0.10.1-0miguel1_powerpc.deb"

nosetests -vvs zmq.tests
   passes all tests, with one skipped due to no python 2.7:
       test messages from memoryview (only valid for python >= 2.7)

Built iPython from master (2ccc90d68) and tested with:
   ./ipython-qtconsole --pylab --paging hsplit
   %guiref
   plot(rand(100))
   display(*getfigs())

  --> works =)

Holding off on testing on my AMD64 machine until it's upgraded to Squeeze (probably later this week).

--Mark

On Friday, February 04, 2011 03:26:56 pm Thomas Kluyver wrote:
> Well, after a bit of fiddling, I can confirm that the x86 package passes the tests and runs the Qt Console frontend alright.
> 
> Anyone else attempting to test on Ubuntu: the dependencies specify a debian specific version of Python. To get round this, use:
>  sudo dpkg -i --force-depends-version <debfile>
> 
> Once this is uploaded to Debian, I'll put in a sync request for Ubuntu.
> 
> Thomas
> 
> On 4 February 2011 22:23, MinRK <benjaminrk at gmail.com> wrote:
>  
> On Fri, Feb 4, 2011 at 13:08, Thomas Kluyver <takowl at gmail.com> wrote:
>  > Well, that's a good start. pyzmq people, is there a simple way to run the
>  > test suite against a separately installed version of pyzmq?
>  
>  
> With nose, you can just do:
>  
>  nosetests -vvs zmq.tests
>  
>  to run the zmq test suite from anywhere. It shouldn't matter how
>  pyzmq/zeromq are installed, as long as your PATHs are correct.
>  
>  -MinRK
>  
> 
>  > Miguel, is there
>  > some standard we're aiming to confirm the package meets for its inclusion?
>  >
>  > Thomas
>  >
>  > On 4 February 2011 20:56, Mark Voorhies <mark.voorhies at ucsf.edu> wrote:
>  >>
>  >> Awesome!
>  >>
>  >> I can test on Debian PPC and AMD64 this weekend.  Any suggested
>  >> values of "working as expected" beyond running an iPython qtconsole
>  >> session?
>  >>
>  >> --Mark
>  >>
>  >> On Friday, February 04, 2011 09:49:02 am Thomas Kluyver wrote:
>  >> > Knew there was something I'd forget ;-)
>  >> >
>  >> > http://packages.debian.org/sid/libzmq0
>  >> >
>  >> > Thomas
>  >> >
>  >> > On 4 February 2011 17:37, Brian Granger <ellisonbg at gmail.com> wrote:
>  >> >
>  >> > Thomas,
>  >> >
>  >> >  This is great!  Thanks Miguel, this is huge!  Where is the link to the
>  >> >  2.0.10 of zeromq itself.  I will definitely test this out.
>  >> >
>  >> >  Cheers,
>  >> >
>  >> >  Brian
>  >> >
>  >> >
>  >> >  On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver <takowl at gmail.com>
>  >> > wrote:
>  >> >  > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and
>  >> > hopes to
>  >> >  > be able to get it uploaded next week. If you're running Debian or
>  >> > Ubuntu,
>  >> >  > and have a few minutes to spare, please check that the packaged
>  >> > version is
>  >> >  > working as expected.
>  >> >  >
>  >> >  > The x86 version is attached, and the amd64 version is at:
>  >> >  >
>  >> > http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb
>  >> >  >
>  >> >  > Thanks,
>  >> >  > Thomas
>  >> >  >
>  >> >  > ---------- Forwarded message ----------
>  >> >  > From: Miguel Landaeta <miguel at miguel.cc>
>  >> >  > Date: 4 February 2011 14:48
>  >> >  > Subject: Re: Debian packaging pyzmq
>  >> >  > To: Thomas Kluyver <takowl at gmail.com>
>  >> >  >
>  >> >  >
>  >> >  > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver <takowl at gmail.com>
>  >> > wrote:
>  >> >  >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some
>  >> > pointers on
>  >> >  >> how to build one). I think the computer is 64 bit, but I'm sticking
>  >> > with a
>  >> >  >> 32 bit OS for now.
>  >> >  >
>  >> >  > No problem, take a look at the attached package.
>  >> >  > Cheers,
>  >> >  >
>  >> >  > --
>  >> >  > Miguel Landaeta, miguel at miguel.cc
>  >> >  > secure email with PGP 0x7D8967E9 available at
>  >> > http://keyserver.pgp.com/
>  >> >  > "Faith means not wanting to know what is true." -- Nietzsche
>  >> >  >
>  >> >  >
>  >> >
>  >> > > _______________________________________________
>  >> >  > IPython-dev mailing list
>  >> >  > IPython-dev at scipy.org
>  >> >  > http://mail.scipy.org/mailman/listinfo/ipython-dev
>  >> >  >
>  >> >  >
>  >> >
>  >> >
>  >> >
>  >> >  --
>  >> >  Brian E. Granger, Ph.D.
>  >> >  Assistant Professor of Physics
>  >> >  Cal Poly State University, San Luis Obispo
>  >> >  bgranger at calpoly.edu
>  >> >  ellisonbg at gmail.com
>  >> >
>  >> _______________________________________________
>  >> IPython-dev mailing list
>  >> IPython-dev at scipy.org
>  >> http://mail.scipy.org/mailman/listinfo/ipython-dev
>  >
>  >
>  > _______________________________________________
>  > IPython-dev mailing list
>  > IPython-dev at scipy.org
>  > http://mail.scipy.org/mailman/listinfo/ipython-dev
>  >
>  >
> 


From takowl at gmail.com  Mon Feb  7 09:40:06 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 7 Feb 2011 14:40:06 +0000
Subject: [IPython-dev] Unicode in tests
In-Reply-To: <iing9k$b2g$1@dough.gmane.org>
References: <AANLkTini4eTaAFfD3u6d6aQJikqULy52JV15Y6qafFCW@mail.gmail.com>
	<ii9msc$32g$1@dough.gmane.org>
	<AANLkTikQZ2R=nF_4A1kDf81wiVZzCeisBeAR8LPRJwv2@mail.gmail.com>
	<AANLkTik0zYQgGVitAOP6aomh9_2mVCHSg3soFUh+29er@mail.gmail.com>
	<AANLkTi=w6Dt7M+zXR12RS1XSxDqh=Tsd2QV2Vp7xrjDY@mail.gmail.com>
	<iia1no$133$1@dough.gmane.org>
	<AANLkTinfyZqWuq0pM1huEx+LXDO9VeSq7zA5AE5tKxa8@mail.gmail.com>
	<iing9k$b2g$1@dough.gmane.org>
Message-ID: <AANLkTikxvoghBRk_GFUm+3KPFMBuuhRgrSMcpCQhbOH7@mail.gmail.com>

Thanks, Robert, that did the trick. I've moved the unicode tests into the
main test suite, and checked that each one fails if I undo the corresponding
fix.

It also caused a failure in the unicode test for inputsplitter, which I've
resolved. This has all been added to the pull request.

https://github.com/ipython/ipython/pull/252

Thomas

On 7 February 2011 01:01, Robert Kern <robert.kern at gmail.com> wrote:

> On 2011-02-05 12:23 , Thomas Kluyver wrote:
>
>> On 1 February 2011 22:33, Robert Kern <robert.kern at gmail.com
>> <mailto:robert.kern at gmail.com>> wrote:
>>
>>    The GTK tests may need to be separated. At startup, iptest currently
>> will
>>    attempt to import gtk just to test if it exists. Perhaps it ought to
>> exclude the
>>    GTK tests unless if they are explicitly requested.
>>
>>
>> What's the best way to deal with this? What tests use gtk?
>>
>
> I don't think there are any tests that use gtk. iptest just checks for gtk
> to determine whether to add IPython/lib/inputhookgtk.py to the list of
> module to exclude from the test suite. This can just be made unconditional
> and remove the test for gtk and gobject. Try the attached patch.
>
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma
>  that is made terrible by our own mad attempt to interpret it as though it
> had
>  an underlying truth."
>  -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110207/a4d5b141/attachment.html>

From source at sharpsteen.net  Mon Feb  7 21:52:23 2011
From: source at sharpsteen.net (Charlie Sharpsteen)
Date: Mon, 7 Feb 2011 18:52:23 -0800
Subject: [IPython-dev] Comments on IPython Qt widgets
Message-ID: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>

Hello IPython developers,

I was playing around with the new Qt  widgets in 0.11-dev the other day and
was very impressed.  I tried throwing together a quick plugin to add an
IPython console to the Qt-based program Quantum GIS [QGIS][1].  Creating the
plugin was easy enough as QGIS runs an embedded Python interpreter and
exposes a lot of PyQt4 functionality.  However, I noticed a couple of things
and was wondering if anyone on this list had any comments to share:


- The shell is based on a stripped down version of `ipython-qtconsole` and
works but is very, very slow for some reason.  The is a delay of a couple of
seconds between when an action is triggered and a response comes back to the
terminal.  The only guess I have that may explain this behavior is that QGIS
is a C++ application that embeds a Python interpreter whereas
ipython-qtconsole is a pure PyQt4 program.  Perhaps QGIS is not letting the
Python threads run as often as a pure PyQt4 program would.

This could be a QGIS-specific problem, but I was wondering if anyone else
had the chance to run the IPython Qt widget in an embedded interpreter and
noticed similar results?


- Currently, the only IPython kernel available for use with the Qt widget
runs the IPython interpreter in a separate process.  For QGIS, it would be
very nice to have the option of running an IPython kernel inside the same
process as the Qt widgets.  IPython would then have direct access to the
QGIS interface and would be able to inspect and operatie on the PyQt4
objects that compose the GUI providing a powerful tool for users performing
spatial analysis and developers building plugins.

Are there any plans to develop IPython kernels that do not run
in separate processes?  I spent a few days trying to build one myself by
wrapping an InteractiveShell, but I feel like I am flailing a little bit
when it comes to creating a kernel_manager that will integrate with the Qt
frontend.  If anyone has any design advice on how to implement an in-process
kernel I would love to hear it.


The plugin is available on [GitHub][2] under the tag `0.1` if anyone feels
interested in checking it out as an IPython use case.  The 0.1 version is
really quite simple---about 5-10 lines in console.py that do the actual
implementation.  To use:

  - Install QGIS
  - Clone project as a sub-directory of ~/.qgis/python/plugins/
  - Run `make` (runs PyQt4 dev tools on ui and qrc files)
  - Launch QGIS and enable Plugin through Plugins -> Manage Plugins menu


Keep up the good work on IPython development!

-Charlie

[1]: http://www.qgis.org
[2]: http://www.github.com/Sharpie/qgis-ipython/tree/0.1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110207/0cb5f259/attachment.html>

From ellisonbg at gmail.com  Mon Feb  7 23:15:38 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 7 Feb 2011 20:15:38 -0800
Subject: [IPython-dev] Comments on IPython Qt widgets
In-Reply-To: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>
References: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>
Message-ID: <AANLkTikavaN1hcqeEcrT720BVb0zdV8E73tGCbGeBZm0@mail.gmail.com>

Charlie,

> I was playing around with the new Qt ?widgets in 0.11-dev the other day and
> was very impressed. ?I tried?throwing?together a quick plugin to add an
> IPython console to the Qt-based program Quantum GIS [QGIS][1]. ?Creating the
> plugin was easy enough as QGIS runs an embedded Python interpreter and
> exposes a lot of PyQt4 functionality.??However, I noticed a couple of things
> and was wondering if anyone on this list had any comments to share:
>
> - The shell is based on a stripped down version of `ipython-qtconsole` and
> works but is very, very slow for some reason. ?The is a delay of a couple of
> seconds between when an action is triggered and a response comes back to the
> terminal. ?The only guess I have that may explain this behavior is that QGIS
> is a C++ application that embeds a Python interpreter whereas
> ipython-qtconsole is a pure PyQt4 program. ?Perhaps QGIS is not letting the
> Python threads run as often as a pure PyQt4 program would.
> This could be a QGIS-specific problem, but I was wondering if anyone else
> had the chance to run the IPython Qt widget in an embedded interpreter and
> noticed similar results?

In general the ipython-qtconsole should be quite fast.  But for that
to be the case, the qt event loop has to be running.  If the C++ is
not letting the qt event loop run often enough, you would definitely
see quite slow behavior.

> - Currently, the only IPython kernel available for use with the Qt widget
> runs the IPython interpreter in a?separate?process. ?For QGIS, it would be
> very nice to have the option of running an IPython kernel inside the same
> process as the Qt widgets. ?IPython would then have direct access to the
> QGIS interface and would be able to inspect and operatie on the PyQt4
> objects that compose the GUI providing a powerful tool for users performing
> spatial analysis and developers building plugins.
> Are there any plans to develop IPython kernels that do not run
> in?separate?processes? ?I spent a few days trying to build one myself by
> wrapping an InteractiveShell, but I feel like I am flailing a little bit
> when it comes to creating a kernel_manager that will integrate with the Qt
> frontend. ?If anyone has any design advice on how to implement an in-process
> kernel I would love to hear it.

There is interest in having an in process IPython, but I honestly
think it will not work very well at all.  This is mostly because of
the GIL.  I would simply turn the model inside out.  If you run the
QGIS code that has all the PyQt widgets in the IPython kernel (start
the kernel with the pyqt event loop running using --pylab qt) your
ipython session will have all the access that it needs.  IOW, don't
embed IPython in your app, embed your app in IPython.  If this model
doesn't work for you, we can talk more about how to get started on an
in process IPython widget, but it will be a significant amount of work
I'm afraid.

> The plugin is available on [GitHub][2] under the tag `0.1` if anyone feels
> interested in checking it out as an IPython use case. ?The 0.1 version is
> really quite simple---about 5-10 lines in console.py that do the actual
> implementation. ?To use:
> ??- Install QGIS
> ??- Clone project as a sub-directory of ~/.qgis/python/plugins/
> ??- Run `make` (runs PyQt4 dev tools on ui and qrc files)
> ??- Launch QGIS and enable Plugin through Plugins -> Manage Plugins menu
>
> Keep up the good work on IPython development!

Thanks!

Brian

> -Charlie
> [1]: http://www.qgis.org
> [2]: http://www.github.com/Sharpie/qgis-ipython/tree/0.1
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From robert.kern at gmail.com  Mon Feb  7 23:25:54 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 07 Feb 2011 22:25:54 -0600
Subject: [IPython-dev] Comments on IPython Qt widgets
In-Reply-To: <AANLkTikavaN1hcqeEcrT720BVb0zdV8E73tGCbGeBZm0@mail.gmail.com>
References: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>
	<AANLkTikavaN1hcqeEcrT720BVb0zdV8E73tGCbGeBZm0@mail.gmail.com>
Message-ID: <iiqgki$v38$1@dough.gmane.org>

On 2011-02-07 22:15 , Brian Granger wrote:
> Charlie,

>> - Currently, the only IPython kernel available for use with the Qt widget
>> runs the IPython interpreter in a separate process.  For QGIS, it would be
>> very nice to have the option of running an IPython kernel inside the same
>> process as the Qt widgets.  IPython would then have direct access to the
>> QGIS interface and would be able to inspect and operatie on the PyQt4
>> objects that compose the GUI providing a powerful tool for users performing
>> spatial analysis and developers building plugins.
>> Are there any plans to develop IPython kernels that do not run
>> in separate processes?  I spent a few days trying to build one myself by
>> wrapping an InteractiveShell, but I feel like I am flailing a little bit
>> when it comes to creating a kernel_manager that will integrate with the Qt
>> frontend.  If anyone has any design advice on how to implement an in-process
>> kernel I would love to hear it.
>
> There is interest in having an in process IPython, but I honestly
> think it will not work very well at all.  This is mostly because of
> the GIL.

I don't see why. The shell may block. This is not a big deal. This is what 
people expect of shells embedded in apps.

> I would simply turn the model inside out.  If you run the
> QGIS code that has all the PyQt widgets in the IPython kernel (start
> the kernel with the pyqt event loop running using --pylab qt) your
> ipython session will have all the access that it needs.  IOW, don't
> embed IPython in your app, embed your app in IPython.

This is not an option for most such apps.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ellisonbg at gmail.com  Mon Feb  7 23:30:04 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 7 Feb 2011 20:30:04 -0800
Subject: [IPython-dev] Comments on IPython Qt widgets
In-Reply-To: <iiqgki$v38$1@dough.gmane.org>
References: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>
	<AANLkTikavaN1hcqeEcrT720BVb0zdV8E73tGCbGeBZm0@mail.gmail.com>
	<iiqgki$v38$1@dough.gmane.org>
Message-ID: <AANLkTinYdcFokc_i54BeE1R=77FxGbrLXj9pzXHgVaJp@mail.gmail.com>

On Mon, Feb 7, 2011 at 8:25 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On 2011-02-07 22:15 , Brian Granger wrote:
>> Charlie,
>
>>> - Currently, the only IPython kernel available for use with the Qt widget
>>> runs the IPython interpreter in a separate process. ?For QGIS, it would be
>>> very nice to have the option of running an IPython kernel inside the same
>>> process as the Qt widgets. ?IPython would then have direct access to the
>>> QGIS interface and would be able to inspect and operatie on the PyQt4
>>> objects that compose the GUI providing a powerful tool for users performing
>>> spatial analysis and developers building plugins.
>>> Are there any plans to develop IPython kernels that do not run
>>> in separate processes? ?I spent a few days trying to build one myself by
>>> wrapping an InteractiveShell, but I feel like I am flailing a little bit
>>> when it comes to creating a kernel_manager that will integrate with the Qt
>>> frontend. ?If anyone has any design advice on how to implement an in-process
>>> kernel I would love to hear it.
>>
>> There is interest in having an in process IPython, but I honestly
>> think it will not work very well at all. ?This is mostly because of
>> the GIL.
>
> I don't see why. The shell may block. This is not a big deal. This is what
> people expect of shells embedded in apps.

It is not the blocking of the shell that is the problem, it is the
blocking of the *entire* app that is the problem.

>> I would simply turn the model inside out. ?If you run the
>> QGIS code that has all the PyQt widgets in the IPython kernel (start
>> the kernel with the pyqt event loop running using --pylab qt) your
>> ipython session will have all the access that it needs. ?IOW, don't
>> embed IPython in your app, embed your app in IPython.
>
> This is not an option for most such apps.

I do realize that.

Brian

> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
> ?that is made terrible by our own mad attempt to interpret it as though it had
> ?an underlying truth."
> ? -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From robert.kern at gmail.com  Mon Feb  7 23:40:46 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 07 Feb 2011 22:40:46 -0600
Subject: [IPython-dev] Comments on IPython Qt widgets
In-Reply-To: <AANLkTinYdcFokc_i54BeE1R=77FxGbrLXj9pzXHgVaJp@mail.gmail.com>
References: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>	<AANLkTikavaN1hcqeEcrT720BVb0zdV8E73tGCbGeBZm0@mail.gmail.com>	<iiqgki$v38$1@dough.gmane.org>
	<AANLkTinYdcFokc_i54BeE1R=77FxGbrLXj9pzXHgVaJp@mail.gmail.com>
Message-ID: <iiqhgf$3dd$1@dough.gmane.org>

On 2011-02-07 22:30 , Brian Granger wrote:
> On Mon, Feb 7, 2011 at 8:25 PM, Robert Kern<robert.kern at gmail.com>  wrote:
>> On 2011-02-07 22:15 , Brian Granger wrote:
>>> Charlie,
>>
>>>> - Currently, the only IPython kernel available for use with the Qt widget
>>>> runs the IPython interpreter in a separate process.  For QGIS, it would be
>>>> very nice to have the option of running an IPython kernel inside the same
>>>> process as the Qt widgets.  IPython would then have direct access to the
>>>> QGIS interface and would be able to inspect and operatie on the PyQt4
>>>> objects that compose the GUI providing a powerful tool for users performing
>>>> spatial analysis and developers building plugins.
>>>> Are there any plans to develop IPython kernels that do not run
>>>> in separate processes?  I spent a few days trying to build one myself by
>>>> wrapping an InteractiveShell, but I feel like I am flailing a little bit
>>>> when it comes to creating a kernel_manager that will integrate with the Qt
>>>> frontend.  If anyone has any design advice on how to implement an in-process
>>>> kernel I would love to hear it.
>>>
>>> There is interest in having an in process IPython, but I honestly
>>> think it will not work very well at all.  This is mostly because of
>>> the GIL.
>>
>> I don't see why. The shell may block. This is not a big deal. This is what
>> people expect of shells embedded in apps.
>
> It is not the blocking of the shell that is the problem, it is the
> blocking of the *entire* app that is the problem.

I meant "The shell may block [the app]". It would be *ideal* if the shell didn't 
block the app, but that doesn't mean that a blocking shell doesn't work well. 
Remember, every UI interaction can potentially trigger a long blocking call, 
unless if that part of the app is carefully written to avoid that. The GIL issue 
you mention applies to any GUI application written in Python, but we manage to 
make usable applications just fine.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From source at sharpsteen.net  Tue Feb  8 13:51:22 2011
From: source at sharpsteen.net (Charlie Sharpsteen)
Date: Tue, 8 Feb 2011 10:51:22 -0800
Subject: [IPython-dev] Comments on IPython Qt widgets
In-Reply-To: <AANLkTikavaN1hcqeEcrT720BVb0zdV8E73tGCbGeBZm0@mail.gmail.com>
References: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>
	<AANLkTikavaN1hcqeEcrT720BVb0zdV8E73tGCbGeBZm0@mail.gmail.com>
Message-ID: <AANLkTimfRQYYL+F4xiXUKu6Kx4O1wqd8teHAn8ytkWkL@mail.gmail.com>

Thanks for the reply Brian,

On Mon, Feb 7, 2011 at 8:15 PM, Brian Granger <ellisonbg at gmail.com> wrote:

> Charlie,
>
> > I was playing around with the new Qt  widgets in 0.11-dev the other day
> and
> > was very impressed.  I tried throwing together a quick plugin to add an
> > IPython console to the Qt-based program Quantum GIS [QGIS][1].  Creating
> the
> > plugin was easy enough as QGIS runs an embedded Python interpreter and
> > exposes a lot of PyQt4 functionality.  However, I noticed a couple of
> things
> > and was wondering if anyone on this list had any comments to share:
> >
> > - The shell is based on a stripped down version of `ipython-qtconsole`
> and
> > works but is very, very slow for some reason.  The is a delay of a couple
> of
> > seconds between when an action is triggered and a response comes back to
> the
> > terminal.  The only guess I have that may explain this behavior is that
> QGIS
> > is a C++ application that embeds a Python interpreter whereas
> > ipython-qtconsole is a pure PyQt4 program.  Perhaps QGIS is not letting
> the
> > Python threads run as often as a pure PyQt4 program would.
> > This could be a QGIS-specific problem, but I was wondering if anyone else
> > had the chance to run the IPython Qt widget in an embedded interpreter
> and
> > noticed similar results?
>
> In general the ipython-qtconsole should be quite fast.  But for that
> to be the case, the qt event loop has to be running.  If the C++ is
> not letting the qt event loop run often enough, you would definitely
> see quite slow behavior.


This is what I suspect, I would probably have to spend some quality time
with the C++ side of the code to be sure.



> > - Currently, the only IPython kernel available for use with the Qt widget
> > runs the IPython interpreter in a separate process.  For QGIS, it would
> be
> > very nice to have the option of running an IPython kernel inside the same
> > process as the Qt widgets.  IPython would then have direct access to the
> > QGIS interface and would be able to inspect and operatie on the PyQt4
> > objects that compose the GUI providing a powerful tool for users
> performing
> > spatial analysis and developers building plugins.
> > Are there any plans to develop IPython kernels that do not run
> > in separate processes?  I spent a few days trying to build one myself by
> > wrapping an InteractiveShell, but I feel like I am flailing a little bit
> > when it comes to creating a kernel_manager that will integrate with the
> Qt
> > frontend.  If anyone has any design advice on how to implement an
> in-process
> > kernel I would love to hear it.
>
> There is interest in having an in process IPython, but I honestly
> think it will not work very well at all.  This is mostly because of
> the GIL.  I would simply turn the model inside out.  If you run the
> QGIS code that has all the PyQt widgets in the IPython kernel (start
> the kernel with the pyqt event loop running using --pylab qt) your
> ipython session will have all the access that it needs.  IOW, don't
> embed IPython in your app, embed your app in IPython.  If this model
> doesn't work for you, we can talk more about how to get started on an
> in process IPython widget, but it will be a significant amount of work
> I'm afraid.


The problem with embedding QGIS in IPython is that the main app is written
in C++ and `import qgis` only works out of the box on Linux---for Windows
and OS X it is a bit of a hassle to get paths set up right.  Many things are
wrapped via SIP and it is possible to run a lot of widgets from IPython but
the main windows are not so it is currently not possible to re-launch the
app under Python.

QGIS currently includes a Python console that makes use of
code.InteractiveInterpreter from the standard library. This console it works
very well with the exception that I always find myself pounding on the tab
key in vain when I forget what I can do with a random Python object.
 Replacing the QGIS Python console with an IPython console would be awesome
and would cut out a lot of trips to the QGIS source code and various
reference manuals.

If an in-process kernel could be implemented that shared the same interface
as the QtKernelManager, then users could easily be given the option of
launching an external process for tasks where the GIL would be a bother.

I am willing to take a stab at developing an in-process widget but after
thrashing around a bit this weekend I feel like some guidance would help me
focus my efforts.



> > The plugin is available on [GitHub][2] under the tag `0.1` if anyone
> feels
> > interested in checking it out as an IPython use case.  The 0.1 version is
> > really quite simple---about 5-10 lines in console.py that do the actual
> > implementation.  To use:
> >   - Install QGIS
> >   - Clone project as a sub-directory of ~/.qgis/python/plugins/
> >   - Run `make` (runs PyQt4 dev tools on ui and qrc files)
> >   - Launch QGIS and enable Plugin through Plugins -> Manage Plugins menu
> >
> > Keep up the good work on IPython development!
>
> Thanks!
>
> Brian
>
> > -Charlie
> > [1]: http://www.qgis.org
> > [2]: http://www.github.com/Sharpie/qgis-ipython/tree/0.1
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
>
>
>
> --
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110208/2987c02e/attachment.html>

From fperez.net at gmail.com  Tue Feb  8 14:48:44 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 8 Feb 2011 11:48:44 -0800
Subject: [IPython-dev] PySide support for the Qt frontend
In-Reply-To: <BCAC19B9-0F24-45E8-B7FA-8B8BD657AF06@enthought.com>
References: <F624C3CF-6AD4-4BCF-AD4F-DE89715FD492@enthought.com>
	<AANLkTi=sFJNmZmGNoX6pzTrDJy6-QHAZK88PT67Z8Et6@mail.gmail.com>
	<BCAC19B9-0F24-45E8-B7FA-8B8BD657AF06@enthought.com>
Message-ID: <AANLkTi=DTu+VZ=vUze4AntKMSwL3aOtW_9JL1Jr8Jr1E@mail.gmail.com>

Hi Evan,

On Thu, Feb 3, 2011 at 11:42 AM, Evan Patterson <epatters at enthought.com> wrote:
> Thanks, Brian. Good to hear from you as well. I'll be working at Enthought for a while, which means that I may do some more work on IPython over the next six months or so.

Great news!  Glad to have you on board again, hope all is well on your side.

>>> Now that several PySide bugs relevant to IPython have been fixed, my port of IPython's Qt code to PySide is working. (Note that even the most recent PySide beta 5 release will segfault sporadically. This is fixed in the PySide master branch and IPython will work with the upcoming RC 1 release.) My branch is here:
>>>
>>> https://github.com/epatters/ipython/tree/pyside-support
>>
>> This is great!

Indeed, many thanks!

>>> This should be fairly uncontroversial. The only thing I would like to direct attention to is the Qt API switcher:
>>>
>>> https://github.com/epatters/ipython/blob/pyside-support/IPython/external/qt.py
>>
>> This is god to know about. ?Overall, how ready would you say that
>> pyside is? ?Is Enthought considering shipping pyside in a version of
>> EPD soon?
>
> PySide is quickly approaching some kind of stability. It looks like an RC 1 release is coming a week or two, with the 1.0 release not too far after. Although the upcoming EPD release (7.0) will not include PySide, as it is being released on the 8th, the subsequent release (7.1) will. At some point, PySide will replace wx as the default toolkit, although Enthought will continue to ship wx with EPD.
>
>>
>>> I have modeled this on the way that ETS is currently handling switching Qt APIs. Any thoughts on this, however, would be welcome.
>>>
>>> Also, regarding GitHub etiquette: should I merge with the master branch before I submit my pull request? My branch is now several weeks behind.
>>
>> No, your stuff should be pretty independent. ?We may do a rebase, but
>> why won't you do a pull request so everyone can have a look.
>
> Done.

I saw the pull request, will continue the discussion on that in there.

I'm very happy to see pyside coming along as well as it seems to, it
will really be a big improvement to the whole python/qt situation.

Cheers,

f


From fperez.net at gmail.com  Tue Feb  8 15:49:13 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 8 Feb 2011 12:49:13 -0800
Subject: [IPython-dev] No displayhook triggered for raw 'asdf'
In-Reply-To: <AANLkTin-PWUd4Vf1NmsDw6rJTnKg4i89G+LHz6L0x6z_@mail.gmail.com>
References: <AANLkTikdY0vrAoZKRwK_yjMe9C+2XaC43vHWYkNG4uG6@mail.gmail.com>
	<AANLkTikLeKS7P-Z-8Qwy=HPY4WT5zt-LAkLaw6WxN-bi@mail.gmail.com>
	<20110126230020.GI5046@phare.normalesup.org>
	<iht35t$fd8$1@dough.gmane.org>
	<AANLkTin-PWUd4Vf1NmsDw6rJTnKg4i89G+LHz6L0x6z_@mail.gmail.com>
Message-ID: <AANLkTinXxCL0h7fsMwWdK1HMXav6JMzquk0aru1NfCRC@mail.gmail.com>

Hey guys,

On Fri, Jan 28, 2011 at 1:02 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>
> Many thanks, Robert. ?For now this fixes at least the single-line
> strings, which was likely the most common problem. ?I'll keep working
> on it later, but since it was such an ugly bug I've pushed your commit
> for now.

A long international flight is often a useful thing, fix ready for review/merge:

https://github.com/ipython/ipython/pull/265

Cheers,

f


From ellisonbg at gmail.com  Tue Feb  8 17:37:09 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 8 Feb 2011 14:37:09 -0800
Subject: [IPython-dev] No displayhook triggered for raw 'asdf'
In-Reply-To: <AANLkTinXxCL0h7fsMwWdK1HMXav6JMzquk0aru1NfCRC@mail.gmail.com>
References: <AANLkTikdY0vrAoZKRwK_yjMe9C+2XaC43vHWYkNG4uG6@mail.gmail.com>
	<AANLkTikLeKS7P-Z-8Qwy=HPY4WT5zt-LAkLaw6WxN-bi@mail.gmail.com>
	<20110126230020.GI5046@phare.normalesup.org>
	<iht35t$fd8$1@dough.gmane.org>
	<AANLkTin-PWUd4Vf1NmsDw6rJTnKg4i89G+LHz6L0x6z_@mail.gmail.com>
	<AANLkTinXxCL0h7fsMwWdK1HMXav6JMzquk0aru1NfCRC@mail.gmail.com>
Message-ID: <AANLkTim-ADJj-X1p6f-cdxCC92acDsp3bxjhpa07uD6i@mail.gmail.com>

Great, I need to spend some time dealing with pull requests/review.

Cheers,

Brian

On Tue, Feb 8, 2011 at 12:49 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hey guys,
>
> On Fri, Jan 28, 2011 at 1:02 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>>
>> Many thanks, Robert. ?For now this fixes at least the single-line
>> strings, which was likely the most common problem. ?I'll keep working
>> on it later, but since it was such an ugly bug I've pushed your commit
>> for now.
>
> A long international flight is often a useful thing, fix ready for review/merge:
>
> https://github.com/ipython/ipython/pull/265
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Tue Feb  8 18:05:16 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 8 Feb 2011 15:05:16 -0800
Subject: [IPython-dev] No displayhook triggered for raw 'asdf'
In-Reply-To: <AANLkTim-ADJj-X1p6f-cdxCC92acDsp3bxjhpa07uD6i@mail.gmail.com>
References: <AANLkTikdY0vrAoZKRwK_yjMe9C+2XaC43vHWYkNG4uG6@mail.gmail.com>
	<AANLkTikLeKS7P-Z-8Qwy=HPY4WT5zt-LAkLaw6WxN-bi@mail.gmail.com>
	<20110126230020.GI5046@phare.normalesup.org>
	<iht35t$fd8$1@dough.gmane.org>
	<AANLkTin-PWUd4Vf1NmsDw6rJTnKg4i89G+LHz6L0x6z_@mail.gmail.com>
	<AANLkTinXxCL0h7fsMwWdK1HMXav6JMzquk0aru1NfCRC@mail.gmail.com>
	<AANLkTim-ADJj-X1p6f-cdxCC92acDsp3bxjhpa07uD6i@mail.gmail.com>
Message-ID: <AANLkTikLrwvTG5ndz_GVWnyA9-ahxss3R8Hfp0-qfU57@mail.gmail.com>

On Tue, Feb 8, 2011 at 2:37 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Great, I need to spend some time dealing with pull requests/review.
>
> Cheers,
>

Yes, same here :)  I feel terrible that we've gotten a lot of good
contributions recently, and we're falling behind on reviews/feedback.
I'm trying to get plugged back into everything and doing some of that
as well, because I've been *terrible* lately.  This recent excellent
post on building open source communities:

http://www.codesimplicity.com/post/open-source-community-simplified/

really made me think hard about this and how I try to organize my
priorities so that, when faced with very limited time (as I inevitably
am) I can still be effective with the project.  This section in
particular:

"""
Respond to contributions immediately.

The Bugzilla Project has a system of code reviews that requires that
all new contributions be reviewed by an experienced developer before
they can become part of Bugzilla. There have been various complaints
about the system over the years, but analyzing the survey data showed
that people leave the project because getting a review takes too long,
not because the reviews are too hard. In fact, the reviews can be as
hard as you want as long as they happen almost instantly after
somebody submits a contribution.

People don?t (usually) mind having to revise a contribution. They even
generally don?t mind revising it several times. But they do mind if
they post a patch, don?t get a review for three months, and then they
have to revise it, only to wait another three months to be told that
they have to revise it again. It?s the delay that matters, not the
level of quality control.
"""

spells out very well the importance of good feedback to new contributors.

I feel super guilty about having dropped the ball on the wave that
came from Scipy India; I hope those contributors will still keep an
interest in the project.

Anyway, the good thing about screwing up badly is that your next step
is likely to be an improvement ;)

Cheers,

f


From fperez.net at gmail.com  Tue Feb  8 20:56:41 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 8 Feb 2011 17:56:41 -0800
Subject: [IPython-dev] Making our git repo much smaller?
Message-ID: <AANLkTinGNVVfr_PZQzFNdHq2=NKvRxKf-XGGZV+7SRTb@mail.gmail.com>

Hi all,

I've always been bothered by the fact that our base git repo is much
bigger than seems necessary.  A clean clone gives:

(master)uqbar[ipython]> dsize .
4.8M	./IPython
7.2M	./docs
75M	./.git
-----------------------------------------------
Total (including smaller dirs not shown above):
87M	.

After reading on the matplotlib-devel list a tip from Pauli Virtnanen
on git compression, who suggested using:

git reflog expire --expire=0 --all
git prune
git repack -f -a -d
git gc --prune=0

I ran that on my fresh clone and now I get:

(master)uqbar[ipython]> dsize .
4.8M	./IPython
7.2M	./docs
12M	./.git
-----------------------------------------------
Total (including smaller dirs not shown above):
24M	.

This is a pretty significant reduction.  What I'm wondering is: how do
we push this benefit back up to github so that all new users who
clone/fork get the smaller repos?

I don't want to push blindly quite yet from my local copy, because I'm
not sure it's OK to do that.  I simply don't know enough: does it do
anything, does it do what we want, any damage?...

But dropping the storage area of our repo from 75M to 12M is a really
significant improvement that would especially help contributors with
low-bandwidth connections, so I'd like to make it happen on the github
side...

If anyone knows what the right step here is, I'd be grateful...

Best,

f


From robert.kern at gmail.com  Tue Feb  8 22:27:14 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 08 Feb 2011 21:27:14 -0600
Subject: [IPython-dev] Making our git repo much smaller?
In-Reply-To: <AANLkTinGNVVfr_PZQzFNdHq2=NKvRxKf-XGGZV+7SRTb@mail.gmail.com>
References: <AANLkTinGNVVfr_PZQzFNdHq2=NKvRxKf-XGGZV+7SRTb@mail.gmail.com>
Message-ID: <iit1ij$1pf$1@dough.gmane.org>

On 2011-02-08 19:56 , Fernando Perez wrote:

> I don't want to push blindly quite yet from my local copy, because I'm
> not sure it's OK to do that.  I simply don't know enough: does it do
> anything, does it do what we want, any damage?...

That sums up most of my experience with git as a whole. ;-)

My brief fiddling with my IPython checkout suggests that only the repack command 
does the major amount of compression (although following it up with gc is just 
good hygiene). None of those commands ought to do damage. Pushing to github 
doesn't seem to affect later checkouts, though. github does seem to run a 
standard no-options gc on every push, but they don't seem to repack.

http://support.github.com/discussions/feature-requests/751-garbage-collection-or-repository-repack

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From benjaminrk at gmail.com  Wed Feb  9 00:04:11 2011
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 8 Feb 2011 21:04:11 -0800
Subject: [IPython-dev] Making our git repo much smaller?
In-Reply-To: <iit1ij$1pf$1@dough.gmane.org>
References: <AANLkTinGNVVfr_PZQzFNdHq2=NKvRxKf-XGGZV+7SRTb@mail.gmail.com>
	<iit1ij$1pf$1@dough.gmane.org>
Message-ID: <AANLkTim6Dq0nsP8Ovsvkxjvtwieku9R58xYv9=2b270T@mail.gmail.com>

I've looked into this a little bit, and as far as I can tell (though not
said with any confidence), the only option you have to repack a public repo
is to pull, repack, and then push to an entirely new repo and toss out the
old one - less than ideal.

-MinRK

On Tue, Feb 8, 2011 at 19:27, Robert Kern <robert.kern at gmail.com> wrote:

> On 2011-02-08 19:56 , Fernando Perez wrote:
>
> > I don't want to push blindly quite yet from my local copy, because I'm
> > not sure it's OK to do that.  I simply don't know enough: does it do
> > anything, does it do what we want, any damage?...
>
> That sums up most of my experience with git as a whole. ;-)
>
> My brief fiddling with my IPython checkout suggests that only the repack
> command
> does the major amount of compression (although following it up with gc is
> just
> good hygiene). None of those commands ought to do damage. Pushing to github
> doesn't seem to affect later checkouts, though. github does seem to run a
> standard no-options gc on every push, but they don't seem to repack.
>
>
> http://support.github.com/discussions/feature-requests/751-garbage-collection-or-repository-repack
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma
>  that is made terrible by our own mad attempt to interpret it as though it
> had
>  an underlying truth."
>   -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110208/17d4f4f8/attachment.html>

From fperez.net at gmail.com  Wed Feb  9 00:26:59 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 8 Feb 2011 21:26:59 -0800
Subject: [IPython-dev] Making our git repo much smaller?
In-Reply-To: <iit1ij$1pf$1@dough.gmane.org>
References: <AANLkTinGNVVfr_PZQzFNdHq2=NKvRxKf-XGGZV+7SRTb@mail.gmail.com>
	<iit1ij$1pf$1@dough.gmane.org>
Message-ID: <AANLkTinMWm7VSj3Thh+bL6gUGHpu3egeZ6_hrB8d=J39@mail.gmail.com>

On Tue, Feb 8, 2011 at 7:27 PM, Robert Kern <robert.kern at gmail.com> wrote:
>
> That sums up most of my experience with git as a whole. ;-)
>
> My brief fiddling with my IPython checkout suggests that only the repack command
> does the major amount of compression (although following it up with gc is just
> good hygiene). None of those commands ought to do damage. Pushing to github
> doesn't seem to affect later checkouts, though. github does seem to run a
> standard no-options gc on every push, but they don't seem to repack.
>
> http://support.github.com/discussions/feature-requests/751-garbage-collection-or-repository-repack

Thanks for that tip... I'll post a new request to see what they say
about running a manual repack for us, perhaps.

Cheers,

f


From fperez.net at gmail.com  Wed Feb  9 01:03:15 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 8 Feb 2011 22:03:15 -0800
Subject: [IPython-dev] Making our git repo much smaller?
In-Reply-To: <AANLkTim6Dq0nsP8Ovsvkxjvtwieku9R58xYv9=2b270T@mail.gmail.com>
References: <AANLkTinGNVVfr_PZQzFNdHq2=NKvRxKf-XGGZV+7SRTb@mail.gmail.com>
	<iit1ij$1pf$1@dough.gmane.org>
	<AANLkTim6Dq0nsP8Ovsvkxjvtwieku9R58xYv9=2b270T@mail.gmail.com>
Message-ID: <AANLkTi=2QQmkvV76TeZkFADApDQ8iftiEed+3pvJweT+@mail.gmail.com>

Hey,

On Tue, Feb 8, 2011 at 9:04 PM, MinRK <benjaminrk at gmail.com> wrote:
> I've looked into this a little bit, and as far as I can tell (though not
> said with any confidence), the only option you have to repack a public repo
> is to pull, repack, and then push to an entirely new repo and toss out the
> old one - less than ideal.

I just made a request for a repack, we'll see what happens:

http://support.github.com/discussions/repos/5478-manual-repack-request-for-repo

Cheers,

f


From fperez.net at gmail.com  Wed Feb  9 14:10:46 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 9 Feb 2011 11:10:46 -0800
Subject: [IPython-dev] Making our git repo much smaller?
In-Reply-To: <AANLkTi=2QQmkvV76TeZkFADApDQ8iftiEed+3pvJweT+@mail.gmail.com>
References: <AANLkTinGNVVfr_PZQzFNdHq2=NKvRxKf-XGGZV+7SRTb@mail.gmail.com>
	<iit1ij$1pf$1@dough.gmane.org>
	<AANLkTim6Dq0nsP8Ovsvkxjvtwieku9R58xYv9=2b270T@mail.gmail.com>
	<AANLkTi=2QQmkvV76TeZkFADApDQ8iftiEed+3pvJweT+@mail.gmail.com>
Message-ID: <AANLkTimyR1q+xsnABGGk7-maHMK16X5mwB6+zmSuJv+o@mail.gmail.com>

On Tue, Feb 8, 2011 at 10:03 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>
> I just made a request for a repack, we'll see what happens:
>
> http://support.github.com/discussions/repos/5478-manual-repack-request-for-repo

Ask and ye shall receive.  They manually ran an aggressive gc pass, and now:

uqbar[junk]> dsize ipython
4.8M	ipython/IPython
7.2M	ipython/docs
11M	ipython/.git
-----------------------------------------------
Total (including smaller dirs not shown above):
23M	ipython


So all new cloners will now have a much more reasonable download.

Github is awesome :)

Cheers,

f


From fperez.net at gmail.com  Wed Feb  9 15:15:04 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 9 Feb 2011 12:15:04 -0800
Subject: [IPython-dev] newparallel
In-Reply-To: <AANLkTin-H4MC-2F2sSGfBqKzX23T1ySVynEQUKfR96Z5@mail.gmail.com>
References: <AANLkTin-H4MC-2F2sSGfBqKzX23T1ySVynEQUKfR96Z5@mail.gmail.com>
Message-ID: <AANLkTin2MeEYx9ow3-LkOKm3dnvrbvFifpxQ9OKxa0n_@mail.gmail.com>

Hey Satra,

On Sun, Feb 6, 2011 at 12:47 PM, Satrajit Ghosh <satra at mit.edu> wrote:
>
> 3. fernando: is your workflow branch mergeable with this? the ability to
> spawn and shutdown engines as needed (especially in the context of clusters
> such as SGE) has come up for discussion on the nipype side a fair bit in the
> last few weeks.

the workflow branch was really just a trivial amount of code to show
Soizic how the basic idea should work, so there isn't much in there to
really merge.

The more important question seems to be that we need to have a good
solution for the (valid) use case of engine creation and use outside
of our central scheduler, when another scheduler is in control of job
creation (say SGE or anything else).

I've been thinking about this a bit, and there seem to be two valid
scenarios to consider:

1. nipype wants to parallelize part of a pipeline where each node is
nothing but a command-line call, long enough that startup overhead is
irrelevant and with no other information to be transferred from the
'head node' (the instance running nipype itself) to the workers.

2. startup time is relevant compared to execution (higher frequency
execution) or there's information to be passed that would be
cumbersome/impossible as command-line arguments but is available to
nipy as python objects.


In situation 1, it's probably not of much value to have ipython
around, except for the case where you might want to debug a
problematic execution. In that scenario, having the engine not
terminate its execution after a problem so that you could connect to
it and play with the data/variables to understand the problem could be
very useful.  But that could be implemented in a special debug mode,
that basically runs something like

namespace = {}
try:
  exec code in namespace
except:
  start_ipython_engine(namespace)
  engine_wait()

The engine could only run if there's a problem with the execution, and
in the normal case isn't even run.


Back to #2, I think in that case there is genuine value added by
having ipython around, as it gives nipype the ability to do much more
flexible execution control and to be efficient in scenarios where
sge/similar likely wouldn't.

The trick is for us to make it very, very easy to incorporate the
'controller' parts into nipype in a way that engines can be started by
the queuing system instead of by our own cluster scripts, and that
those engines participate of the rest of the nipype execution as
needed, coming and going as dictated by the scheduler.


Cheers,

f


From benjaminrk at gmail.com  Wed Feb  9 16:56:15 2011
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 9 Feb 2011 13:56:15 -0800
Subject: [IPython-dev] newparallel
In-Reply-To: <AANLkTin2MeEYx9ow3-LkOKm3dnvrbvFifpxQ9OKxa0n_@mail.gmail.com>
References: <AANLkTin-H4MC-2F2sSGfBqKzX23T1ySVynEQUKfR96Z5@mail.gmail.com>
	<AANLkTin2MeEYx9ow3-LkOKm3dnvrbvFifpxQ9OKxa0n_@mail.gmail.com>
Message-ID: <AANLkTinfnf-CrLuRxfJkY4qTx2vgoM0Azy75hxvHvXZ6@mail.gmail.com>

On Wed, Feb 9, 2011 at 12:15, Fernando Perez <fperez.net at gmail.com> wrote:

> Hey Satra,
>
> On Sun, Feb 6, 2011 at 12:47 PM, Satrajit Ghosh <satra at mit.edu> wrote:
> >
> > 3. fernando: is your workflow branch mergeable with this? the ability to
> > spawn and shutdown engines as needed (especially in the context of
> clusters
> > such as SGE) has come up for discussion on the nipype side a fair bit in
> the
> > last few weeks.
>
> the workflow branch was really just a trivial amount of code to show
> Soizic how the basic idea should work, so there isn't much in there to
> really merge.
>

That may be, but I did merge and update it anyway ;)

The newparallel examples are now in docs/examples/newparallel


>
> The more important question seems to be that we need to have a good
> solution for the (valid) use case of engine creation and use outside
> of our central scheduler, when another scheduler is in control of job
> creation (say SGE or anything else).
>

Since starting engines amounts to running shell scripts, it's not difficult
to start engines with any *ix job system.

I am in the process of updating the Launchers from the kernel code, in order
to make this trivial on common systems.

Linking engine startup to jobs can probably (but not so nicely) be done with
the current setup in 3 steps:

1. submit IPython task that will start an engine (or just run this in the
client)

def newjob(jobID):
    jobfile="job-%s.sh"%jobID
    with open(jobfile, 'w') as f:
        f.write(<pbs boilerplate>)
        f.write("ipenginez -c jobID=%r"%jobID)
    os.system('qsub %s'%jobfile)

c[0].apply(newjob, my_jobid)

2. submit the real Task, with a dependency that will keep it on that engine

def check_jobid(jobid):
    engine_jobid = globals().get('jobID', None)
    return engine_jobid == jobid

@depend(check_jobid, my_jobid)
def mytask(args):
    dostuff()

ar = client.apply(mytask, myargs)

3. submit another task using a `follow` dependency that calls sys.exit

client.apply(lambda : sys.exit(), follow=ar)


>
> I've been thinking about this a bit, and there seem to be two valid
> scenarios to consider:
>
> 1. nipype wants to parallelize part of a pipeline where each node is
> nothing but a command-line call, long enough that startup overhead is
> irrelevant and with no other information to be transferred from the
> 'head node' (the instance running nipype itself) to the workers.
>
> 2. startup time is relevant compared to execution (higher frequency
> execution) or there's information to be passed that would be
> cumbersome/impossible as command-line arguments but is available to
> nipy as python objects.
>
>
> In situation 1, it's probably not of much value to have ipython
> around, except for the case where you might want to debug a
> problematic execution. In that scenario, having the engine not
> terminate its execution after a problem so that you could connect to
> it and play with the data/variables to understand the problem could be
> very useful.  But that could be implemented in a special debug mode,
> that basically runs something like
>
> namespace = {}
> try:
>  exec code in namespace
> except:
>  start_ipython_engine(namespace)
>  engine_wait()
>
> The engine could only run if there's a problem with the execution, and
> in the normal case isn't even run.
>
>
> Back to #2, I think in that case there is genuine value added by
> having ipython around, as it gives nipype the ability to do much more
> flexible execution control and to be efficient in scenarios where
> sge/similar likely wouldn't.
>
> The trick is for us to make it very, very easy to incorporate the
> 'controller' parts into nipype in a way that engines can be started by
> the queuing system instead of by our own cluster scripts, and that
> those engines participate of the rest of the nipype execution as
> needed, coming and going as dictated by the scheduler.
>

It sounds like what is requested is, rather than an IPython scheduler, an
SGE scheduler.  That is, the Scheduler, for every Task, starts a new Engine
(via qsub,etc.), runs the task on it, and shuts the engine down. This would
obviously be extraordinarily slow compared to the regular Scheduler.
 Perhaps the new Cluster model Brian and I have been discussing will help
with this - it will allow clients to make requests like 'start_engine' and
'stop_engine', which would be executed via the Launchers.

-MinRK


>
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110209/3238995b/attachment.html>

From gael.varoquaux at normalesup.org  Wed Feb  9 17:13:53 2011
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Wed, 9 Feb 2011 23:13:53 +0100
Subject: [IPython-dev] newparallel
In-Reply-To: <AANLkTin2MeEYx9ow3-LkOKm3dnvrbvFifpxQ9OKxa0n_@mail.gmail.com>
References: <AANLkTin-H4MC-2F2sSGfBqKzX23T1ySVynEQUKfR96Z5@mail.gmail.com>
	<AANLkTin2MeEYx9ow3-LkOKm3dnvrbvFifpxQ9OKxa0n_@mail.gmail.com>
Message-ID: <20110209221353.GB6115@phare.normalesup.org>

On Wed, Feb 09, 2011 at 12:15:04PM -0800, Fernando Perez wrote:
> In situation 1, it's probably not of much value to have ipython
> around, except for the case where you might want to debug a
> problematic execution. 

IMHO that's a hugely important point. The number one reason why I don't
do much parallel work is that a significant amount of my time on research
code is spent debuging. So I would say: please don't underestimate the
fantastic contribution that you have been making in this direction with
IPython.

Take this with a grain of salt from somebody who doesn't do grid
computing.

Ga?l


From epatters at enthought.com  Thu Feb 10 12:55:16 2011
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 10 Feb 2011 11:55:16 -0600
Subject: [IPython-dev] ipython-qtconsole and docstrings
In-Reply-To: <AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
References: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
	<AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
Message-ID: <F0EE2E2B-D017-4C18-945A-E24F8E768CDE@enthought.com>


On Feb 4, 2011, at 11:19 AM, MinRK wrote:

> On Fri, Feb 4, 2011 at 06:59, Darren Dale <dsdale24 at gmail.com> wrote:
>> Hi All,
>> 
>> This morning I updated my zmq/pyzmq and tried running
>> ipython-qtconsole from the master branch. I had let my zeromq
>> installation get behind ipython development, and so I'm rediscovering
>> how awesome is the qtconsole. Having docstrings pop up at "(" is
>> great. I noticed though, that depending on the location of the mouse
>> pointer, the docstring may only briefly appear. It seems more likely
>> to occur if the pointer is located inside the region where the
>> docstring will be rendered. I'm running "ipython-qtconsole --pylab" on
>> a Snow Leopard machine, with native (not X-based) Qt provided by
>> MacPorts. Has anyone else noticed something similar?
> 
> Yes, this is my experience, also on OSX 10.6.6, with native PyQt from
> their website.  The behavior is inconsistent, but seems more likely if
> the cursor is hovering over the popup, and also more likely if the
> popup is large.
> 
> Another (probably) OSX-specific glitch: You cannot copy with 'cmd-c',
> because pressing command triggers an event that clears the selection.

I've submitted a pull request that fixes both of these bugs.

If you're aware of other, similar issues, please send those my way as well.

Evan
> 
>> 
>> Who is working on the qtconsole? Is there a roadmap?
>> 
>> Darren
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



From fperez.net at gmail.com  Thu Feb 10 14:32:28 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 10 Feb 2011 11:32:28 -0800
Subject: [IPython-dev] ipython-qtconsole and docstrings
In-Reply-To: <F0EE2E2B-D017-4C18-945A-E24F8E768CDE@enthought.com>
References: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
	<AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
	<F0EE2E2B-D017-4C18-945A-E24F8E768CDE@enthought.com>
Message-ID: <AANLkTimvnHH8YcXsNcTp+vOK=UacrwUDMkY8__1QH3_b@mail.gmail.com>

Hi folks,

On Thu, Feb 10, 2011 at 9:55 AM, Evan Patterson <epatters at enthought.com> wrote:
>
> I've submitted a pull request that fixes both of these bugs.

For reference, this is Evan's pull request:

https://github.com/ipython/ipython/pull/266

As per my comments there, it would be great if at least one mac user
(anyone can do it, it doesn't have to be a core developer) could
independently validate this.  The code looks fine to me, but I don't
want to approve for merging code that hasn't been actually verified in
execution by at least one more person beyond its author, and I don't
have a mac set up for development that I can easily use.

Cheers,

f


From fperez.net at gmail.com  Thu Feb 10 14:39:33 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 10 Feb 2011 11:39:33 -0800
Subject: [IPython-dev] newparallel
In-Reply-To: <20110209221353.GB6115@phare.normalesup.org>
References: <AANLkTin-H4MC-2F2sSGfBqKzX23T1ySVynEQUKfR96Z5@mail.gmail.com>
	<AANLkTin2MeEYx9ow3-LkOKm3dnvrbvFifpxQ9OKxa0n_@mail.gmail.com>
	<20110209221353.GB6115@phare.normalesup.org>
Message-ID: <AANLkTikJs44zZd_uqXWnEdxAuoQpPckDE_oWxnRQf8PJ@mail.gmail.com>

On Wed, Feb 9, 2011 at 2:13 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> On Wed, Feb 09, 2011 at 12:15:04PM -0800, Fernando Perez wrote:
>> In situation 1, it's probably not of much value to have ipython
>> around, except for the case where you might want to debug a
>> problematic execution.
>
> IMHO that's a hugely important point. The number one reason why I don't
> do much parallel work is that a significant amount of my time on research
> code is spent debuging. So I would say: please don't underestimate the
> fantastic contribution that you have been making in this direction with
> IPython.

Thanks for the comments, Gael.  Indeed that was very much my take on
it, but I know you and I have similar workstyles and we really value
interactive access to 'hands-on' code and data.  I was just trying to
be somewhat neutral towards the more classic grid computing approaches
that toss interactivity overboard.

But you are right that we shouldn't shy away from highlighting the
value of this.  And it's clear to me that with a bit of api help on
our side, we can make it trivial and zero-overhead for batch-oriented
systems to offer the capability to do:

try:
  normal_execution()
except:
  open_ipython_client()

This would not even invoke ipython if things go OK, but offer the
ability to debug a problematic node interactively whenever there's a
problem (which is more often than the grid computing literature wants
you to believe ;)

So thanks for coming back to that point, it's more important than I
sometimes give it credit in a misguided attempt at moderating my
ipython/interactive bias :)

Best,

f


From benjaminrk at gmail.com  Thu Feb 10 14:47:48 2011
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 10 Feb 2011 11:47:48 -0800
Subject: [IPython-dev] ipython-qtconsole and docstrings
In-Reply-To: <AANLkTimvnHH8YcXsNcTp+vOK=UacrwUDMkY8__1QH3_b@mail.gmail.com>
References: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
	<AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
	<F0EE2E2B-D017-4C18-945A-E24F8E768CDE@enthought.com>
	<AANLkTimvnHH8YcXsNcTp+vOK=UacrwUDMkY8__1QH3_b@mail.gmail.com>
Message-ID: <AANLkTikoYHSV6L82p=zedqy53H6EphiLFKbT5AD00DCq@mail.gmail.com>

Fix verified on my machine (OSX 10.6.6, x64 System Python 2.6.1, current
release PyQt, etc.)

[also posted in PR discussion thread]

-MinRK

On Thu, Feb 10, 2011 at 11:32, Fernando Perez <fperez.net at gmail.com> wrote:

> Hi folks,
>
> On Thu, Feb 10, 2011 at 9:55 AM, Evan Patterson <epatters at enthought.com>
> wrote:
> >
> > I've submitted a pull request that fixes both of these bugs.
>
> For reference, this is Evan's pull request:
>
> https://github.com/ipython/ipython/pull/266
>
> As per my comments there, it would be great if at least one mac user
> (anyone can do it, it doesn't have to be a core developer) could
> independently validate this.  The code looks fine to me, but I don't
> want to approve for merging code that hasn't been actually verified in
> execution by at least one more person beyond its author, and I don't
> have a mac set up for development that I can easily use.
>
> Cheers,
>
> f
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110210/48c49e6c/attachment.html>

From fperez.net at gmail.com  Thu Feb 10 14:53:19 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 10 Feb 2011 11:53:19 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
Message-ID: <AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>

On Tue, Feb 1, 2011 at 9:56 AM, Nicolas Rougier
<Nicolas.rougier at inria.fr> wrote:
> I've made some time ago a kind of 'ascii imshow' function to render a numpy
> array directly within the console by exploiting the 256 colors capability of
> most modern terminal. I don't know how much interesting it can be for
> IPython but I found the code useful for myself.
> Code is located
> at:?http://www.loria.fr/~rougier/coding/software/numpy_imshow.py
> (GPL license can be replaced by the BSD one if necessary)

Want. Now. Fabulous!!

If you'd be so kind as to officialy relicense it as BSD, we'd be
thrilled to have it in IPython right away.  For the curious, this is
what Nicolas' default example produces:

http://imgur.com/iZ13v

Note, this is a true terminal-only tool, it doesn't work correctly in
the Qt console nor in a bare Linux console (i.e.  outside X11).  Both
the Qt and Linux console show spurious codes and just two lines of
weird colors.  It would be cool to see if this can be adapted to
non-xterm environments later, but for now this is great as it is.

Nicolas, if you're willing to submit this as a pull request against
our trunk, by adding the code as a new module named
IPython/extensions/ascii_imshow.py, it would be fantastic.  If you're
not famliar with the github workflow let us know and we'll be happy to
provide some pointers.

If you can't spare the time to make a git-based pull request please
let us know, it's possible someone else can collaborate with you to
finish that part of the process.

This file:

https://github.com/ipython/ipython/blob/master/docs/source/development/template.py

can serve as a template for putting your code in our standard format.
Certainly do keep your original copyright statement in addition to
(moving forward) the team one, and also add to the docstring an
authorship line

Authors
--------
* Nicolas Rougier...

But it's good to have all source files similarly structured and
documented project-wide.

In any case, great little tool, I hope we can pull it in soon!

Regards,

f


From source at sharpsteen.net  Thu Feb 10 15:02:22 2011
From: source at sharpsteen.net (Charlie Sharpsteen)
Date: Thu, 10 Feb 2011 12:02:22 -0800
Subject: [IPython-dev] ipython-qtconsole and docstrings
In-Reply-To: <AANLkTimvnHH8YcXsNcTp+vOK=UacrwUDMkY8__1QH3_b@mail.gmail.com>
References: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
	<AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
	<F0EE2E2B-D017-4C18-945A-E24F8E768CDE@enthought.com>
	<AANLkTimvnHH8YcXsNcTp+vOK=UacrwUDMkY8__1QH3_b@mail.gmail.com>
Message-ID: <AANLkTi=0MPGzLQoUbEfg14eeRAaaC4KtrmC-zshOckM=@mail.gmail.com>

On Thu, Feb 10, 2011 at 11:32 AM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hi folks,
>
> On Thu, Feb 10, 2011 at 9:55 AM, Evan Patterson <epatters at enthought.com>
> wrote:
> >
> > I've submitted a pull request that fixes both of these bugs.
>
> For reference, this is Evan's pull request:
>
> https://github.com/ipython/ipython/pull/266
>
> As per my comments there, it would be great if at least one mac user
> (anyone can do it, it doesn't have to be a core developer) could
> independently validate this.  The code looks fine to me, but I don't
> want to approve for merging code that hasn't been actually verified in
> execution by at least one more person beyond its author, and I don't
> have a mac set up for development that I can easily use.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>

Verified with `ipython-qtconsole` OS X 10.6.6 with:

   Python 2.7.1
   Qt 4.7.1
   SIP/PyQt  4.12.1/4.8.3

Before:

  Cmd-C bug:  Could not reproduce
  Call tips:  What calltips?
  Home/End behavior: Reproduced as described by issue #173

After:

  Cmd-C bug:  Still could not reproduce (good!)
  Call tips:  Whoa!  It comes with calltips?!
  Home/End behavior:  Fixed

Looks good to me.

-Charlie
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110210/bfc169f7/attachment.html>

From takowl at gmail.com  Thu Feb 10 15:04:55 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 10 Feb 2011 20:04:55 +0000
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
Message-ID: <AANLkTincmEvVb-epzX5E6FkYrge0Tj-NHSFY6FLLP53i@mail.gmail.com>

On 10 February 2011 19:53, Fernando Perez <fperez.net at gmail.com> wrote:

>  It would be cool to see if this can be adapted to
> non-xterm environments later, but for now this is great as it is.
>

Doing something similar for the the Qt console (and other graphics capable
consoles) should be easier than making the ASCII version - scipy already has
functions to turn an array into an image.

http://docs.scipy.org/doc/scipy/reference/misc.html#scipy.misc.toimage

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110210/4aa91076/attachment.html>

From satra at mit.edu  Thu Feb 10 15:14:06 2011
From: satra at mit.edu (Satrajit Ghosh)
Date: Thu, 10 Feb 2011 15:14:06 -0500
Subject: [IPython-dev] ipython-qtconsole and docstrings
In-Reply-To: <AANLkTi=0MPGzLQoUbEfg14eeRAaaC4KtrmC-zshOckM=@mail.gmail.com>
References: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
	<AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
	<F0EE2E2B-D017-4C18-945A-E24F8E768CDE@enthought.com>
	<AANLkTimvnHH8YcXsNcTp+vOK=UacrwUDMkY8__1QH3_b@mail.gmail.com>
	<AANLkTi=0MPGzLQoUbEfg14eeRAaaC4KtrmC-zshOckM=@mail.gmail.com>
Message-ID: <AANLkTik4Lcq5=BM2Y2or7A-xZb_prMdDY2RMoHvg=yGi@mail.gmail.com>

also fixed on osx x64 10.6.6 system python 2.6.1 latest pyqt

cheers,

satra


On Thu, Feb 10, 2011 at 3:02 PM, Charlie Sharpsteen
<source at sharpsteen.net>wrote:

> On Thu, Feb 10, 2011 at 11:32 AM, Fernando Perez <fperez.net at gmail.com>wrote:
>
>> Hi folks,
>>
>> On Thu, Feb 10, 2011 at 9:55 AM, Evan Patterson <epatters at enthought.com>
>> wrote:
>> >
>> > I've submitted a pull request that fixes both of these bugs.
>>
>> For reference, this is Evan's pull request:
>>
>> https://github.com/ipython/ipython/pull/266
>>
>> As per my comments there, it would be great if at least one mac user
>> (anyone can do it, it doesn't have to be a core developer) could
>> independently validate this.  The code looks fine to me, but I don't
>> want to approve for merging code that hasn't been actually verified in
>> execution by at least one more person beyond its author, and I don't
>> have a mac set up for development that I can easily use.
>>
>> Cheers,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
> Verified with `ipython-qtconsole` OS X 10.6.6 with:
>
>    Python 2.7.1
>    Qt 4.7.1
>    SIP/PyQt  4.12.1/4.8.3
>
> Before:
>
>   Cmd-C bug:  Could not reproduce
>   Call tips:  What calltips?
>   Home/End behavior: Reproduced as described by issue #173
>
> After:
>
>   Cmd-C bug:  Still could not reproduce (good!)
>   Call tips:  Whoa!  It comes with calltips?!
>   Home/End behavior:  Fixed
>
> Looks good to me.
>
> -Charlie
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110210/4c3a11c9/attachment.html>

From ellisonbg at gmail.com  Thu Feb 10 15:21:07 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 10 Feb 2011 12:21:07 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
Message-ID: <AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>

Two quick comments:

* While I think this is incredibly cool, I am not sure this belongs in
IPython.  Two reasons for this: i) it is useful for people outside of
IPython and ii) plotting/viz is outside the scope of IPython.  I think
a much better place for this would be a standalone module or
matplotlib (imagine a terminal based backend!).
* If it does go into IPython, the proper directory would be lib, not
extensions.  IPython/extensions is for IPython extensions that adhere
to the official IPython extension API.

Cheers,

Brian

On Thu, Feb 10, 2011 at 11:53 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Tue, Feb 1, 2011 at 9:56 AM, Nicolas Rougier
> <Nicolas.rougier at inria.fr> wrote:
>> I've made some time ago a kind of 'ascii imshow' function to render a numpy
>> array directly within the console by exploiting the 256 colors capability of
>> most modern terminal. I don't know how much interesting it can be for
>> IPython but I found the code useful for myself.
>> Code is located
>> at:?http://www.loria.fr/~rougier/coding/software/numpy_imshow.py
>> (GPL license can be replaced by the BSD one if necessary)
>
> Want. Now. Fabulous!!
>
> If you'd be so kind as to officialy relicense it as BSD, we'd be
> thrilled to have it in IPython right away. ?For the curious, this is
> what Nicolas' default example produces:
>
> http://imgur.com/iZ13v
>
> Note, this is a true terminal-only tool, it doesn't work correctly in
> the Qt console nor in a bare Linux console (i.e. ?outside X11). ?Both
> the Qt and Linux console show spurious codes and just two lines of
> weird colors. ?It would be cool to see if this can be adapted to
> non-xterm environments later, but for now this is great as it is.
>
> Nicolas, if you're willing to submit this as a pull request against
> our trunk, by adding the code as a new module named
> IPython/extensions/ascii_imshow.py, it would be fantastic. ?If you're
> not famliar with the github workflow let us know and we'll be happy to
> provide some pointers.
>
> If you can't spare the time to make a git-based pull request please
> let us know, it's possible someone else can collaborate with you to
> finish that part of the process.
>
> This file:
>
> https://github.com/ipython/ipython/blob/master/docs/source/development/template.py
>
> can serve as a template for putting your code in our standard format.
> Certainly do keep your original copyright statement in addition to
> (moving forward) the team one, and also add to the docstring an
> authorship line
>
> Authors
> --------
> * Nicolas Rougier...
>
> But it's good to have all source files similarly structured and
> documented project-wide.
>
> In any case, great little tool, I hope we can pull it in soon!
>
> Regards,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From epatters at enthought.com  Thu Feb 10 15:21:46 2011
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 10 Feb 2011 14:21:46 -0600
Subject: [IPython-dev] ipython-qtconsole and docstrings
In-Reply-To: <AANLkTik4Lcq5=BM2Y2or7A-xZb_prMdDY2RMoHvg=yGi@mail.gmail.com>
References: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
	<AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
	<F0EE2E2B-D017-4C18-945A-E24F8E768CDE@enthought.com>
	<AANLkTimvnHH8YcXsNcTp+vOK=UacrwUDMkY8__1QH3_b@mail.gmail.com>
	<AANLkTi=0MPGzLQoUbEfg14eeRAaaC4KtrmC-zshOckM=@mail.gmail.com>
	<AANLkTik4Lcq5=BM2Y2or7A-xZb_prMdDY2RMoHvg=yGi@mail.gmail.com>
Message-ID: <344D3E76-071E-409E-8465-A3C37E3E2007@enthought.com>

Thanks for all the testing everyone. The fixes have been merged into master.

Evan

On Feb 10, 2011, at 2:14 PM, Satrajit Ghosh wrote:

> also fixed on osx x64 10.6.6 system python 2.6.1 latest pyqt
> 
> cheers,
> 
> satra
> 
> 
> On Thu, Feb 10, 2011 at 3:02 PM, Charlie Sharpsteen <source at sharpsteen.net> wrote:
> On Thu, Feb 10, 2011 at 11:32 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi folks,
> 
> On Thu, Feb 10, 2011 at 9:55 AM, Evan Patterson <epatters at enthought.com> wrote:
> >
> > I've submitted a pull request that fixes both of these bugs.
> 
> For reference, this is Evan's pull request:
> 
> https://github.com/ipython/ipython/pull/266
> 
> As per my comments there, it would be great if at least one mac user
> (anyone can do it, it doesn't have to be a core developer) could
> independently validate this.  The code looks fine to me, but I don't
> want to approve for merging code that hasn't been actually verified in
> execution by at least one more person beyond its author, and I don't
> have a mac set up for development that I can easily use.
> 
> Cheers,
> 
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> Verified with `ipython-qtconsole` OS X 10.6.6 with:
> 
>    Python 2.7.1
>    Qt 4.7.1
>    SIP/PyQt  4.12.1/4.8.3
> 
> Before:
> 
>   Cmd-C bug:  Could not reproduce
>   Call tips:  What calltips?
>   Home/End behavior: Reproduced as described by issue #173
> 
> After:
> 
>   Cmd-C bug:  Still could not reproduce (good!)
>   Call tips:  Whoa!  It comes with calltips?!
>   Home/End behavior:  Fixed
> 
> Looks good to me.
> 
> -Charlie
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
> 
> 
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110210/861019fb/attachment.html>

From fperez.net at gmail.com  Thu Feb 10 15:24:16 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 10 Feb 2011 12:24:16 -0800
Subject: [IPython-dev] ipython-qtconsole and docstrings
In-Reply-To: <344D3E76-071E-409E-8465-A3C37E3E2007@enthought.com>
References: <AANLkTin4Ot4KDjVS8rf3OESLGERq6PU9GMM6fDVM_81s@mail.gmail.com>
	<AANLkTi=D1z=egB37VK-LbBg8r7PYcPe63dCSKD9Hc2Mt@mail.gmail.com>
	<F0EE2E2B-D017-4C18-945A-E24F8E768CDE@enthought.com>
	<AANLkTimvnHH8YcXsNcTp+vOK=UacrwUDMkY8__1QH3_b@mail.gmail.com>
	<AANLkTi=0MPGzLQoUbEfg14eeRAaaC4KtrmC-zshOckM=@mail.gmail.com>
	<AANLkTik4Lcq5=BM2Y2or7A-xZb_prMdDY2RMoHvg=yGi@mail.gmail.com>
	<344D3E76-071E-409E-8465-A3C37E3E2007@enthought.com>
Message-ID: <AANLkTimia4D-jfV2E7XE8tnKV1GemxfLPOhnuSus6hr=@mail.gmail.com>

On Thu, Feb 10, 2011 at 12:21 PM, Evan Patterson <epatters at enthought.com> wrote:
> Thanks for all the testing everyone. The fixes have been merged into master.

Much appreciated from all, testers and Evan alike.  Excellent.

regards,

f


From fperez.net at gmail.com  Thu Feb 10 15:43:51 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 10 Feb 2011 12:43:51 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
Message-ID: <AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>

On Thu, Feb 10, 2011 at 12:21 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> * While I think this is incredibly cool, I am not sure this belongs in
> IPython. ?Two reasons for this: i) it is useful for people outside of
> IPython and ii) plotting/viz is outside the scope of IPython. ?I think
> a much better place for this would be a standalone module or
> matplotlib (imagine a terminal based backend!).

While I really hope MPL gets an ascii backend at some point, that's a
lot more work.  I don't know if Nicolas is up for it :)  I think it's
OK to have it in ipython in the sense that it's small, useful code for
terminal use.  While it's true that we're not in the data viz
business, we're in the business of making a really good interactive
environment, and the terminal is one of our strengths.

Something this small, left to live on its own, is much less likely to
get noticed/used than if we ship it and everyone benefits from it
right away.  I think of this as something much like our qt console
matplotlib support: something to seamlessly make ipython that much
better for heavy numpy/scientific users.

> * If it does go into IPython, the proper directory would be lib, not
> extensions. ?IPython/extensions is for IPython extensions that adhere
> to the official IPython extension API.

Yes, my initial email was written with lib/, and then I backtracked
and thought of it as an extension.  But the api point is spot-on, so
lib it is.

Cheers,

f


From takowl at gmail.com  Fri Feb 11 08:42:35 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 11 Feb 2011 13:42:35 +0000
Subject: [IPython-dev] Fwd: RFS: DPMT: pyzmq 2.0.10.1-1
In-Reply-To: <20110211131509.GH16708@piotro.eu>
References: <AANLkTikFw+x4oLNLkTq3TsjkYXkb5qLrFoESZ-BC=r55@mail.gmail.com>
	<20110210144324.GD16708@piotro.eu>
	<AANLkTi=Fs8bQg2NFKiFWpPgUbJhrLOt9jNLHNO_0dd+S@mail.gmail.com>
	<20110211131509.GH16708@piotro.eu>
Message-ID: <AANLkTik2=-Uom3+kBx2=LZ3uHy0RMECkX5CU=MJmJL41@mail.gmail.com>

The bugfix for pyzmq made it into Debian experimental, and they found a
couple of spelling mistakes - see below.

Thomas

---------- Forwarded message ----------
From: Piotr O?arowski <piotr at debian.org>
Date: 11 February 2011 13:15
Subject: Re: RFS: DPMT: pyzmq 2.0.10.1-1
To: Miguel Landaeta <miguel at miguel.cc>
Cc: Thomas Kluyver <takowl at gmail.com>


uploaded; please send upstream a patch to fix these spelling mistakes:

zmq/core/message.so Messsages ? Messages
zmq/core/message.so Messsages ? Messages
zmq/core/socket.so Recieve ? Receive
--
http://people.debian.org/~piotr/sponsor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110211/34e10bae/attachment.html>

From erik.tollerud at gmail.com  Fri Feb 11 13:13:25 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Fri, 11 Feb 2011 10:13:25 -0800
Subject: [IPython-dev] Fwd: RFS: DPMT: pyzmq 2.0.10.1-1
In-Reply-To: <AANLkTik2=-Uom3+kBx2=LZ3uHy0RMECkX5CU=MJmJL41@mail.gmail.com>
References: <AANLkTikFw+x4oLNLkTq3TsjkYXkb5qLrFoESZ-BC=r55@mail.gmail.com>
	<20110210144324.GD16708@piotro.eu>
	<AANLkTi=Fs8bQg2NFKiFWpPgUbJhrLOt9jNLHNO_0dd+S@mail.gmail.com>
	<20110211131509.GH16708@piotro.eu>
	<AANLkTik2=-Uom3+kBx2=LZ3uHy0RMECkX5CU=MJmJL41@mail.gmail.com>
Message-ID: <AANLkTimkapweXOHA1rAYxLoc9MGbJ2rsAtE7ijSjL7o7@mail.gmail.com>

I've added another Ubuntu bug report to upgrade to this package for
Natty (https://bugs.launchpad.net/ubuntu/+source/pyzmq/+bug/717316)

On Fri, Feb 11, 2011 at 5:42 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> The bugfix for pyzmq made it into Debian experimental, and they found a
> couple of spelling mistakes - see below.
>
> Thomas
>
> ---------- Forwarded message ----------
> From: Piotr O?arowski <piotr at debian.org>
> Date: 11 February 2011 13:15
> Subject: Re: RFS: DPMT: pyzmq 2.0.10.1-1
> To: Miguel Landaeta <miguel at miguel.cc>
> Cc: Thomas Kluyver <takowl at gmail.com>
>
>
> uploaded; please send upstream a patch to fix these spelling mistakes:
>
> zmq/core/message.so Messsages ? Messages
> zmq/core/message.so Messsages ? Messages
> zmq/core/socket.so Recieve ? Receive
> --
> http://people.debian.org/~piotr/sponsor
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Erik Tollerud


From benjaminrk at gmail.com  Fri Feb 11 13:34:20 2011
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 11 Feb 2011 10:34:20 -0800
Subject: [IPython-dev] Fwd: RFS: DPMT: pyzmq 2.0.10.1-1
In-Reply-To: <AANLkTimkapweXOHA1rAYxLoc9MGbJ2rsAtE7ijSjL7o7@mail.gmail.com>
References: <AANLkTikFw+x4oLNLkTq3TsjkYXkb5qLrFoESZ-BC=r55@mail.gmail.com>
	<20110210144324.GD16708@piotro.eu>
	<AANLkTi=Fs8bQg2NFKiFWpPgUbJhrLOt9jNLHNO_0dd+S@mail.gmail.com>
	<20110211131509.GH16708@piotro.eu>
	<AANLkTik2=-Uom3+kBx2=LZ3uHy0RMECkX5CU=MJmJL41@mail.gmail.com>
	<AANLkTimkapweXOHA1rAYxLoc9MGbJ2rsAtE7ijSjL7o7@mail.gmail.com>
Message-ID: <AANLkTi=_UK-Th1uzgny4zg+rLmH7qXh=mDqPARNy3Zib@mail.gmail.com>

Great, thanks!

The typos are now fixed in master.

-MinRK

On Fri, Feb 11, 2011 at 10:13, Erik Tollerud <erik.tollerud at gmail.com>wrote:

> I've added another Ubuntu bug report to upgrade to this package for
> Natty (https://bugs.launchpad.net/ubuntu/+source/pyzmq/+bug/717316)
>
> On Fri, Feb 11, 2011 at 5:42 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> > The bugfix for pyzmq made it into Debian experimental, and they found a
> > couple of spelling mistakes - see below.
> >
> > Thomas
> >
> > ---------- Forwarded message ----------
> > From: Piotr O?arowski <piotr at debian.org>
> > Date: 11 February 2011 13:15
> > Subject: Re: RFS: DPMT: pyzmq 2.0.10.1-1
> > To: Miguel Landaeta <miguel at miguel.cc>
> > Cc: Thomas Kluyver <takowl at gmail.com>
> >
> >
> > uploaded; please send upstream a patch to fix these spelling mistakes:
> >
> > zmq/core/message.so Messsages ? Messages
> > zmq/core/message.so Messsages ? Messages
> > zmq/core/socket.so Recieve ? Receive
> > --
> > http://people.debian.org/~piotr/sponsor
> >
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
>
>
>
> --
> Erik Tollerud
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110211/7adc5b5b/attachment.html>

From takowl at gmail.com  Fri Feb 11 17:30:43 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 11 Feb 2011 22:30:43 +0000
Subject: [IPython-dev] Fwd: RFS: DPMT: pyzmq 2.0.10.1-1
In-Reply-To: <AANLkTi=_UK-Th1uzgny4zg+rLmH7qXh=mDqPARNy3Zib@mail.gmail.com>
References: <AANLkTikFw+x4oLNLkTq3TsjkYXkb5qLrFoESZ-BC=r55@mail.gmail.com>
	<20110210144324.GD16708@piotro.eu>
	<AANLkTi=Fs8bQg2NFKiFWpPgUbJhrLOt9jNLHNO_0dd+S@mail.gmail.com>
	<20110211131509.GH16708@piotro.eu>
	<AANLkTik2=-Uom3+kBx2=LZ3uHy0RMECkX5CU=MJmJL41@mail.gmail.com>
	<AANLkTimkapweXOHA1rAYxLoc9MGbJ2rsAtE7ijSjL7o7@mail.gmail.com>
	<AANLkTi=_UK-Th1uzgny4zg+rLmH7qXh=mDqPARNy3Zib@mail.gmail.com>
Message-ID: <AANLkTi=UcgqwWDfCyBqbAk3RCXbqmx7bEJ9jL13PAXV7@mail.gmail.com>

And Ubuntu has done the sync. That closes that chapter.

Thomas

On 11 February 2011 18:34, MinRK <benjaminrk at gmail.com> wrote:

> Great, thanks!
>
> The typos are now fixed in master.
>
> -MinRK
>
>
> On Fri, Feb 11, 2011 at 10:13, Erik Tollerud <erik.tollerud at gmail.com>wrote:
>
>> I've added another Ubuntu bug report to upgrade to this package for
>> Natty (https://bugs.launchpad.net/ubuntu/+source/pyzmq/+bug/717316)
>>
>> On Fri, Feb 11, 2011 at 5:42 AM, Thomas Kluyver <takowl at gmail.com> wrote:
>> > The bugfix for pyzmq made it into Debian experimental, and they found a
>> > couple of spelling mistakes - see below.
>> >
>> > Thomas
>> >
>> > ---------- Forwarded message ----------
>> > From: Piotr O?arowski <piotr at debian.org>
>> > Date: 11 February 2011 13:15
>> > Subject: Re: RFS: DPMT: pyzmq 2.0.10.1-1
>> > To: Miguel Landaeta <miguel at miguel.cc>
>> > Cc: Thomas Kluyver <takowl at gmail.com>
>> >
>> >
>> > uploaded; please send upstream a patch to fix these spelling mistakes:
>> >
>> > zmq/core/message.so Messsages ? Messages
>> > zmq/core/message.so Messsages ? Messages
>> > zmq/core/socket.so Recieve ? Receive
>> > --
>> > http://people.debian.org/~piotr/sponsor
>> >
>> >
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>>
>>
>>
>> --
>> Erik Tollerud
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110211/8f98335f/attachment.html>

From ellisonbg at gmail.com  Fri Feb 11 23:11:27 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 11 Feb 2011 20:11:27 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
Message-ID: <AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>

Hi,

On Thu, Feb 10, 2011 at 12:43 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Thu, Feb 10, 2011 at 12:21 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>> * While I think this is incredibly cool, I am not sure this belongs in
>> IPython. ?Two reasons for this: i) it is useful for people outside of
>> IPython and ii) plotting/viz is outside the scope of IPython. ?I think
>> a much better place for this would be a standalone module or
>> matplotlib (imagine a terminal based backend!).
>
> While I really hope MPL gets an ascii backend at some point, that's a
> lot more work. ?I don't know if Nicolas is up for it :) ?I think it's
> OK to have it in ipython in the sense that it's small, useful code for
> terminal use. ?While it's true that we're not in the data viz
> business, we're in the business of making a really good interactive
> environment, and the terminal is one of our strengths.

OK, if we do this, let make sure that:

* There are tests.
* It is fully documented in docstrings.

I am mostly concerned about the fact that this type of stuff tends to
get deposited with us to maintain in the future (like port to Python
3).  Ideally, these things would have maintainers that are committed
to helping out with things.

> Something this small, left to live on its own, is much less likely to
> get noticed/used than if we ship it and everyone benefits from it
> right away. ?I think of this as something much like our qt console
> matplotlib support: something to seamlessly make ipython that much
> better for heavy numpy/scientific users.
>
>> * If it does go into IPython, the proper directory would be lib, not
>> extensions. ?IPython/extensions is for IPython extensions that adhere
>> to the official IPython extension API.
>
> Yes, my initial email was written with lib/, and then I backtracked
> and thought of it as an extension. ?But the api point is spot-on, so
> lib it is.

Cheers,

Brian

> Cheers,
>
> f
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From sunqiang at gmail.com  Sat Feb 12 04:25:33 2011
From: sunqiang at gmail.com (sunqiang)
Date: Sat, 12 Feb 2011 17:25:33 +0800
Subject: [IPython-dev] Issue of install ipython-py3k
Message-ID: <AANLkTikdxipXbpzprepgXEjnLb22omVMmgr-R7dYMJQK@mail.gmail.com>

Hi,

  After git clone git://github.com/ipython/ipython-py3k.git
  "python3.2 setup.py build" raise a byte/str error, I have to change
line 370 of setupbase.py from "cfg_parser.set('commit hash',
'install_hash', repo_commit)" to "cfg_parser.set('commit hash',
'install_hash', repo_commit.decode())" to finish the setup.
I don't know if this is my own problem.
the error I am getting is:
############################################################################
============================================================================
BUILDING IPYTHON
                python: 3.2rc2+ (py3k, Feb  9 2011, 01:06:54)  [GCC 4.2.4
                        (Ubuntu 4.2.4-1ubuntu4)]
              platform: linux2

OPTIONAL DEPENDENCIES
        zope.Interface: Not found (required for parallel computing
                        capabilities)
               Twisted: Not found (required for parallel computing
                        capabilities)
              Foolscap: Not found (required for parallel computing
                        capabilities)
               OpenSSL: Not found (required if you want security in the
                        parallel computing capabilities)
                sphinx: Not found (required for building documentation)
              pygments: Not found (required for syntax highlighting
                        documentation)
                  nose: Not found (required for running the test suite)
               pexpect: no (required for running standalone doctests)
running build
running build_py
copying IPython/core/display.py -> build/lib/IPython/core
copying IPython/core/magic.py -> build/lib/IPython/core
copying IPython/core/hooks.py -> build/lib/IPython/core
copying IPython/core/formatters.py -> build/lib/IPython/core
copying IPython/core/usage.py -> build/lib/IPython/core
copying IPython/core/displayhook.py -> build/lib/IPython/core
copying IPython/core/displaypub.py -> build/lib/IPython/core
copying IPython/core/inputsplitter.py -> build/lib/IPython/core
copying IPython/core/interactiveshell.py -> build/lib/IPython/core
copying IPython/core/tests/test_inputsplitter.py -> build/lib/IPython/core/tests
copying IPython/extensions/sympy_printing.py -> build/lib/IPython/extensions
copying IPython/lib/irunner.py -> build/lib/IPython/lib
copying IPython/lib/latextools.py -> build/lib/IPython/lib
copying IPython/lib/pylabtools.py -> build/lib/IPython/lib
copying IPython/testing/tools.py -> build/lib/IPython/testing
Traceback (most recent call last):
  File "setup.py", line 254, in <module>
    setup(**setup_args)
  File "/opt/python3.2/lib/python3.2/distutils/core.py", line 149, in setup
    dist.run_commands()
  File "/opt/python3.2/lib/python3.2/distutils/dist.py", line 919, in
run_commands
    self.run_command(cmd)
  File "/opt/python3.2/lib/python3.2/distutils/dist.py", line 938, in
run_command
    cmd_obj.run()
  File "/opt/python3.2/lib/python3.2/distutils/command/build.py", line
128, in run
    self.run_command(cmd_name)
  File "/opt/python3.2/lib/python3.2/distutils/cmd.py", line 315, in run_command
    self.distribution.run_command(command)
  File "/opt/python3.2/lib/python3.2/distutils/dist.py", line 938, in
run_command
    cmd_obj.run()
  File "/home/romano/gitrepos/ipython-py3k/setupbase.py", line 370, in run
    cfg_parser.set('commit hash', 'install_hash', repo_commit)
  File "/opt/python3.2/lib/python3.2/configparser.py", line 1157, in set
    self._validate_value_types(option=option, value=value)
  File "/opt/python3.2/lib/python3.2/configparser.py", line 1146, in
_validate_value_types
    raise TypeError("option values must be strings")
TypeError: option values must be strings
############################################################################


From Nicolas.rougier at inria.fr  Sat Feb 12 04:32:54 2011
From: Nicolas.rougier at inria.fr (Nicolas Rougier)
Date: Sat, 12 Feb 2011 10:32:54 +0100
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
Message-ID: <DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>


Hi, 

Sorry for the delay, I was away from my mail.

After reading the commentaries, I realize there are still some work to do before the code can be properly integrated within IPython and I will try to make a nicer version as well as fully document it (using the template.py). For the git pull, I'm not very familiar with it and I don't know the proper command to pull or push, woudl yu have any pointer on this ?

For making the code to work with the QT console, it would require to parse the string sent to the console and interpret the ascii sequences. I did it some time ago for a gtk/ipython console but I'm not familiar at all with qt. (parsing code is of course available if one want to work on it).


As for the MPL terminal backend, it would be really cool (gnuplot got one) but I do have time right know to start such a thing.


Nicolas


On Feb 12, 2011, at 5:11 AM, Brian Granger wrote:

> Hi,
> 
> On Thu, Feb 10, 2011 at 12:43 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>> On Thu, Feb 10, 2011 at 12:21 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>> * While I think this is incredibly cool, I am not sure this belongs in
>>> IPython.  Two reasons for this: i) it is useful for people outside of
>>> IPython and ii) plotting/viz is outside the scope of IPython.  I think
>>> a much better place for this would be a standalone module or
>>> matplotlib (imagine a terminal based backend!).
>> 
>> While I really hope MPL gets an ascii backend at some point, that's a
>> lot more work.  I don't know if Nicolas is up for it :)  I think it's
>> OK to have it in ipython in the sense that it's small, useful code for
>> terminal use.  While it's true that we're not in the data viz
>> business, we're in the business of making a really good interactive
>> environment, and the terminal is one of our strengths.
> 
> OK, if we do this, let make sure that:
> 
> * There are tests.
> * It is fully documented in docstrings.
> 
> I am mostly concerned about the fact that this type of stuff tends to
> get deposited with us to maintain in the future (like port to Python
> 3).  Ideally, these things would have maintainers that are committed
> to helping out with things.
> 
>> Something this small, left to live on its own, is much less likely to
>> get noticed/used than if we ship it and everyone benefits from it
>> right away.  I think of this as something much like our qt console
>> matplotlib support: something to seamlessly make ipython that much
>> better for heavy numpy/scientific users.
>> 
>>> * If it does go into IPython, the proper directory would be lib, not
>>> extensions.  IPython/extensions is for IPython extensions that adhere
>>> to the official IPython extension API.
>> 
>> Yes, my initial email was written with lib/, and then I backtracked
>> and thought of it as an extension.  But the api point is spot-on, so
>> lib it is.
> 
> Cheers,
> 
> Brian
> 
>> Cheers,
>> 
>> f
>> 
> 
> 
> 
> -- 
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com



From ellisonbg at gmail.com  Sat Feb 12 13:42:38 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 12 Feb 2011 10:42:38 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
	<DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
Message-ID: <AANLkTinCEgMsRer6Xm9W6Rkg8zz8M6sFpWC82LipC+Xy@mail.gmail.com>

Fernando,

Do you think it would make sene to implement this as an object that
has special print methods like __pretty__, __png__, etc.  That would
make it super easy to add support for other frontends.

Cheers,

Brian

On Sat, Feb 12, 2011 at 1:32 AM, Nicolas Rougier
<Nicolas.rougier at inria.fr> wrote:
>
> Hi,
>
> Sorry for the delay, I was away from my mail.
>
> After reading the commentaries, I realize there are still some work to do before the code can be properly integrated within IPython and I will try to make a nicer version as well as fully document it (using the template.py). For the git pull, I'm not very familiar with it and I don't know the proper command to pull or push, woudl yu have any pointer on this ?
>
> For making the code to work with the QT console, it would require to parse the string sent to the console and interpret the ascii sequences. I did it some time ago for a gtk/ipython console but I'm not familiar at all with qt. (parsing code is of course available if one want to work on it).
>
>
> As for the MPL terminal backend, it would be really cool (gnuplot got one) but I do have time right know to start such a thing.
>
>
> Nicolas
>
>
> On Feb 12, 2011, at 5:11 AM, Brian Granger wrote:
>
>> Hi,
>>
>> On Thu, Feb 10, 2011 at 12:43 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>>> On Thu, Feb 10, 2011 at 12:21 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>>> * While I think this is incredibly cool, I am not sure this belongs in
>>>> IPython. ?Two reasons for this: i) it is useful for people outside of
>>>> IPython and ii) plotting/viz is outside the scope of IPython. ?I think
>>>> a much better place for this would be a standalone module or
>>>> matplotlib (imagine a terminal based backend!).
>>>
>>> While I really hope MPL gets an ascii backend at some point, that's a
>>> lot more work. ?I don't know if Nicolas is up for it :) ?I think it's
>>> OK to have it in ipython in the sense that it's small, useful code for
>>> terminal use. ?While it's true that we're not in the data viz
>>> business, we're in the business of making a really good interactive
>>> environment, and the terminal is one of our strengths.
>>
>> OK, if we do this, let make sure that:
>>
>> * There are tests.
>> * It is fully documented in docstrings.
>>
>> I am mostly concerned about the fact that this type of stuff tends to
>> get deposited with us to maintain in the future (like port to Python
>> 3). ?Ideally, these things would have maintainers that are committed
>> to helping out with things.
>>
>>> Something this small, left to live on its own, is much less likely to
>>> get noticed/used than if we ship it and everyone benefits from it
>>> right away. ?I think of this as something much like our qt console
>>> matplotlib support: something to seamlessly make ipython that much
>>> better for heavy numpy/scientific users.
>>>
>>>> * If it does go into IPython, the proper directory would be lib, not
>>>> extensions. ?IPython/extensions is for IPython extensions that adhere
>>>> to the official IPython extension API.
>>>
>>> Yes, my initial email was written with lib/, and then I backtracked
>>> and thought of it as an extension. ?But the api point is spot-on, so
>>> lib it is.
>>
>> Cheers,
>>
>> Brian
>>
>>> Cheers,
>>>
>>> f
>>>
>>
>>
>>
>> --
>> Brian E. Granger, Ph.D.
>> Assistant Professor of Physics
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu
>> ellisonbg at gmail.com
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From takowl at gmail.com  Sat Feb 12 14:11:34 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 12 Feb 2011 19:11:34 +0000
Subject: [IPython-dev] Issue of install ipython-py3k
In-Reply-To: <AANLkTikdxipXbpzprepgXEjnLb22omVMmgr-R7dYMJQK@mail.gmail.com>
References: <AANLkTikdxipXbpzprepgXEjnLb22omVMmgr-R7dYMJQK@mail.gmail.com>
Message-ID: <AANLkTimWZMftoL9k7E99avvY1TkUjqfNivhtQ54uwYEz@mail.gmail.com>

Thanks, Sunqiang, I've pushed the fix.

Keep me posted of any more problems you find using it with Python 3.2. I
probably won't upgrade until April, with the next Ubuntu release.

Technical details: The value coming from a subprocess.Popen().communicate()
was a byte string. This was being written to a config file, but the
rewritten configparser in Python 3.2 specifically checks that the option to
be set is a (unicode) str. I've actually specified that it should be decoded
as ascii: the hash digest should never include any non-ascii characters, and
we'll find out quickly if it's doing anything unexpected.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110212/68eda3e4/attachment.html>

From ellisonbg at gmail.com  Sat Feb 12 14:22:44 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 12 Feb 2011 11:22:44 -0800
Subject: [IPython-dev] No displayhook triggered for raw 'asdf'
In-Reply-To: <AANLkTikLrwvTG5ndz_GVWnyA9-ahxss3R8Hfp0-qfU57@mail.gmail.com>
References: <AANLkTikdY0vrAoZKRwK_yjMe9C+2XaC43vHWYkNG4uG6@mail.gmail.com>
	<AANLkTikLeKS7P-Z-8Qwy=HPY4WT5zt-LAkLaw6WxN-bi@mail.gmail.com>
	<20110126230020.GI5046@phare.normalesup.org>
	<iht35t$fd8$1@dough.gmane.org>
	<AANLkTin-PWUd4Vf1NmsDw6rJTnKg4i89G+LHz6L0x6z_@mail.gmail.com>
	<AANLkTinXxCL0h7fsMwWdK1HMXav6JMzquk0aru1NfCRC@mail.gmail.com>
	<AANLkTim-ADJj-X1p6f-cdxCC92acDsp3bxjhpa07uD6i@mail.gmail.com>
	<AANLkTikLrwvTG5ndz_GVWnyA9-ahxss3R8Hfp0-qfU57@mail.gmail.com>
Message-ID: <AANLkTikitPEZktazQrRp1zZPY73LUNQq99ZS9v5OptfM@mail.gmail.com>

Fernando,

> Yes, same here :) ?I feel terrible that we've gotten a lot of good
> contributions recently, and we're falling behind on reviews/feedback.
> I'm trying to get plugged back into everything and doing some of that
> as well, because I've been *terrible* lately. ?This recent excellent
> post on building open source communities:

Yes, that is the hidden side of code review and lots of (great!)
contributions.  It takes a ton of time just to review.  Between the
different projects I am involved in, I spend a significant amount of
time doing review (more than I code these days).

> http://www.codesimplicity.com/post/open-source-community-simplified/
>
> really made me think hard about this and how I try to organize my
> priorities so that, when faced with very limited time (as I inevitably
> am) I can still be effective with the project. ?This section in
> particular:
>
> """
> Respond to contributions immediately.
>
> The Bugzilla Project has a system of code reviews that requires that
> all new contributions be reviewed by an experienced developer before
> they can become part of Bugzilla. There have been various complaints
> about the system over the years, but analyzing the survey data showed
> that people leave the project because getting a review takes too long,
> not because the reviews are too hard. In fact, the reviews can be as
> hard as you want as long as they happen almost instantly after
> somebody submits a contribution.

I *completely* believe this.

> People don?t (usually) mind having to revise a contribution. They even
> generally don?t mind revising it several times. But they do mind if
> they post a patch, don?t get a review for three months, and then they
> have to revise it, only to wait another three months to be told that
> they have to revise it again. It?s the delay that matters, not the
> level of quality control.
> """

Yep.

> spells out very well the importance of good feedback to new contributors.
>
> I feel super guilty about having dropped the ball on the wave that
> came from Scipy India; I hope those contributors will still keep an
> interest in the project.
>
> Anyway, the good thing about screwing up badly is that your next step
> is likely to be an improvement ;)

:)

One thing I think we should do is figure out a way of dividing the
review workload so nothing slips through the cracks.  When a review
comes in, do we want to assign a "lead" reviewer that will make sure
it gets pushed through in a timely manner?  Sort of like journals
assign reviewers to articles.  Our current situation is like running a
peer review journal, without an editor to manage the review proces...

Cheers,

Brian

> Cheers,
>
> f
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ellisonbg at gmail.com  Sat Feb 12 14:38:00 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 12 Feb 2011 11:38:00 -0800
Subject: [IPython-dev] Comments on IPython Qt widgets
In-Reply-To: <AANLkTimfRQYYL+F4xiXUKu6Kx4O1wqd8teHAn8ytkWkL@mail.gmail.com>
References: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>
	<AANLkTikavaN1hcqeEcrT720BVb0zdV8E73tGCbGeBZm0@mail.gmail.com>
	<AANLkTimfRQYYL+F4xiXUKu6Kx4O1wqd8teHAn8ytkWkL@mail.gmail.com>
Message-ID: <AANLkTim8mSgbtzSeZR_2i6nOAO+knv2C49cgd_if0D=V@mail.gmail.com>

I think there are two possible approaches:

1.  Still use the pyzmq interface through KernelManager and the
SocketChannel objects, just use in process (threads) zeromq sockets.
While complicated, this might be the best approach.  But, it might
take some GUI event loop magic to get working (because you can't run
anything in a thread.

2.  Get rid of the zeromq stuff entirely.  This is probably easier.
To do this, I would check out the subclass of InteractiveShell in
IPython.zmq.zmqshell.  It should be pretty easy to build a subclass of
InteractiveShell that works, *but*, it leaves you without a GUI.

Min, Fernando or Evan, any thoughts on this?

Cheers,

Brian


On Tue, Feb 8, 2011 at 10:51 AM, Charlie Sharpsteen
<source at sharpsteen.net> wrote:
> Thanks for the reply Brian,
>
> On Mon, Feb 7, 2011 at 8:15 PM, Brian Granger <ellisonbg at gmail.com
>> wrote:
>
>> Charlie,
>>
>> > I was playing around with the new Qt  widgets in 0.11-dev the other day
>> and
>> > was very impressed.  I tried throwing together a quick plugin to add an
>> > IPython console to the Qt-based program Quantum GIS [QGIS][1].  Creating
>> the
>> > plugin was easy enough as QGIS runs an embedded Python interpreter and
>> > exposes a lot of PyQt4 functionality.  However, I noticed a couple of
>> things
>> > and was wondering if anyone on this list had any comments to share:
>> >
>> > - The shell is based on a stripped down version of `ipython-qtconsole`
>> and
>> > works but is very, very slow for some reason.  The is a delay of a
>> > couple
>> of
>> > seconds between when an action is triggered and a response comes back to
>> the
>> > terminal.  The only guess I have that may explain this behavior is that
>> QGIS
>> > is a C++ application that embeds a Python interpreter whereas
>> > ipython-qtconsole is a pure PyQt4 program.  Perhaps QGIS is not letting
>> the
>> > Python threads run as often as a pure PyQt4 program would.
>> > This could be a QGIS-specific problem, but I was wondering if anyone
>> > else
>> > had the chance to run the IPython Qt widget in an embedded interpreter
>> and
>> > noticed similar results?
>>
>> In general the ipython-qtconsole should be quite fast.  But for that
>> to be the case, the qt event loop has to be running.  If the C++ is
>> not letting the qt event loop run often enough, you would definitely
>> see quite slow behavior.
>
>
> This is what I suspect, I would probably have to spend some quality time
> with the C++ side of the code to be sure.
>
>
>
>> > - Currently, the only IPython kernel available for use with the Qt
>> > widget
>> > runs the IPython interpreter in a separate process.  For QGIS, it would
>> be
>> > very nice to have the option of running an IPython kernel inside the
>> > same
>> > process as the Qt widgets.  IPython would then have direct access to the
>> > QGIS interface and would be able to inspect and operatie on the PyQt4
>> > objects that compose the GUI providing a powerful tool for users
>> performing
>> > spatial analysis and developers building plugins.
>> > Are there any plans to develop IPython kernels that do not run
>> > in separate processes?  I spent a few days trying to build one myself by
>> > wrapping an InteractiveShell, but I feel like I am flailing a little bit
>> > when it comes to creating a kernel_manager that will integrate with the
>> Qt
>> > frontend.  If anyone has any design advice on how to implement an
>> in-process
>> > kernel I would love to hear it.
>>
>> There is interest in having an in process IPython, but I honestly
>> think it will not work very well at all.  This is mostly because of
>> the GIL.  I would simply turn the model inside out.  If you run the
>> QGIS code that has all the PyQt widgets in the IPython kernel (start
>> the kernel with the pyqt event loop running using --pylab qt) your
>> ipython session will have all the access that it needs.  IOW, don't
>> embed IPython in your app, embed your app in IPython.  If this model
>> doesn't work for you, we can talk more about how to get started on an
>> in process IPython widget, but it will be a significant amount of work
>> I'm afraid.
>
>
> The problem with embedding QGIS in IPython is that the main app is written
> in C++ and `import qgis` only works out of the box on Linux---for Windows
> and OS X it is a bit of a hassle to get paths set up right.  Many things are
> wrapped via SIP and it is possible to run a lot of widgets from IPython but
> the main windows are not so it is currently not possible to re-launch the
> app under Python.
>
> QGIS currently includes a Python console that makes use of
> code.InteractiveInterpreter from the standard library. This console it works
> very well with the exception that I always find myself pounding on the tab
> key in vain when I forget what I can do with a random Python object.
>  Replacing the QGIS Python console with an IPython console would be awesome
> and would cut out a lot of trips to the QGIS source code and various
> reference manuals.
>
> If an in-process kernel could be implemented that shared the same interface
> as the QtKernelManager, then users could easily be given the option of
> launching an external process for tasks where the GIL would be a bother.
>
> I am willing to take a stab at developing an in-process widget but after
> thrashing around a bit this weekend I feel like some guidance would help me
> focus my efforts.
>
>
>
>> > The plugin is available on [GitHub][2] under the tag `0.1` if anyone
>> feels
>> > interested in checking it out as an IPython use case.  The 0.1 version
>> > is
>> > really quite simple---about 5-10 lines in console.py that do the actual
>> > implementation.  To use:
>> >   - Install QGIS
>> >   - Clone project as a sub-directory of ~/.qgis/python/plugins/
>> >   - Run `make` (runs PyQt4 dev tools on ui and qrc files)
>> >   - Launch QGIS and enable Plugin through Plugins -> Manage Plugins menu
>> >
>> > Keep up the good work on IPython development!
>>
>> Thanks!
>>
>> Brian
>>
>> > -Charlie
>> > [1]: http://www.qgis.org
>> > [2]: http://www.github.com/Sharpie/qgis-ipython/tree/0.1
>> > _______________________________________________
>> > IPython-dev mailing list
>> > IPython-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-dev
>> >
>> >
>>
>>
>>
>> --
>> Brian E. Granger, Ph.D.
>> Assistant Professor of Physics
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu
>> ellisonbg at gmail.com
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From source at sharpsteen.net  Sat Feb 12 15:16:26 2011
From: source at sharpsteen.net (Charlie Sharpsteen)
Date: Sat, 12 Feb 2011 12:16:26 -0800
Subject: [IPython-dev] Comments on IPython Qt widgets
In-Reply-To: <AANLkTim8mSgbtzSeZR_2i6nOAO+knv2C49cgd_if0D=V@mail.gmail.com>
References: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>
	<AANLkTikavaN1hcqeEcrT720BVb0zdV8E73tGCbGeBZm0@mail.gmail.com>
	<AANLkTimfRQYYL+F4xiXUKu6Kx4O1wqd8teHAn8ytkWkL@mail.gmail.com>
	<AANLkTim8mSgbtzSeZR_2i6nOAO+knv2C49cgd_if0D=V@mail.gmail.com>
Message-ID: <AANLkTimaotQ0pzNsB76fVwsKGy9x6zFOBQ-a_c+W4_Mo@mail.gmail.com>

On Sat, Feb 12, 2011 at 11:38 AM, Brian Granger <ellisonbg at gmail.com> wrote:

> I think there are two possible approaches:
>
> 1.  Still use the pyzmq interface through KernelManager and the
> SocketChannel objects, just use in process (threads) zeromq sockets.
> While complicated, this might be the best approach.  But, it might
> take some GUI event loop magic to get working (because you can't run
> anything in a thread.
>

The problem with ZMQ for me is that the shell still has poor responsiveness
when embedded in QGIS.  I still don't know if this problem is specific to
QGIS or would affect other C++ Qt applications that run embedded Python
interpreters.  I don't have another C++ app handy that does this to
cross-validate with.

I did some hacking over the week to try and work around the problem that
included: switching the ZmqSocketChannel to a subclass of QThread, tweaking
the thread run priorities, fiddling with the signal connections, and nothing
gave good results.  So, the problem could be:

   1. With the way QGIS handles threads controlled by Python
   2. With the way the ZeroMQ event loop behaves when embedded in a C++ app
   3. Other

I noticed that if I interact with the console window somehow, like twiddling
the scroll wheel, while it is waiting then a refresh occurs that brings in
the results from the external kernel.  Also, I wrote a quick example in pure
PyQGIS where Python is running the whole show and it is just as responsive
as `ipython-qtconsole`. So, I suspect explanation #1 is still the most
likely cause for this behavior.

I haven't heard anything back from the QGIS-dev mailing list yet, so the ZMQ
route is currently cold for me at this point.


2.  Get rid of the zeromq stuff entirely.  This is probably easier.
> To do this, I would check out the subclass of InteractiveShell in
> IPython.zmq.zmqshell.  It should be pretty easy to build a subclass of
> InteractiveShell that works, *but*, it leaves you without a GUI.
>

I also gave this a shot but creating a very primitive "kernel manager" that
ran an InteractiveShell inside an OutputTrap that was subclassed to also
redirect io.Term.  This worked fairly well and I was able to get something
working that was responsive in QGIS and supported tab-completion.  There
were two main difficulties I ran into going this route:

- The frontend widgets seem to know a lot about the backend implementation
which makes it difficult to create a new kernel manager that does something
different under the hood.

- The ZMQ kernel manager currently forms the base class for all kernel
managers.


It seems like it might be a good idea to:

- Create an ABC for kernel managers that defines everything a frontend
widget really needs to know in order to use them.

- Replace specific references in FrontendWidgets
like self.kernel_manager.xreq_channel.execute with something a little more
abstract such as self.kernel_manager.execute and let the kernel manager
worry about the details of which socket, channel, QSignal, etc is actually
carrying the message.

These are just my thoughts, and I reserve the right to be wrong :)


To summarize a bit of a rambling email: embedding an InteractiveShell has
proven to be a very promising route.  At this point, the main question I
have concerning this implementation is:

   - Is there a better way to set the stout/stderr streams of  kernel than
having the kernel manager run every call to "run_source" inside an
OutputTrap?


Final note: you guys are doing an awesome job with the rewrite, on the whole
it has been very easy to hack on IPython without breaking everything since
things are well componentized.

-Charlie



Min, Fernando or Evan, any thoughts on this?
>
> Cheers,
>
> Brian
>
>
> On Tue, Feb 8, 2011 at 10:51 AM, Charlie Sharpsteen
> <source at sharpsteen.net> wrote:
> > Thanks for the reply Brian,
> >
> > On Mon, Feb 7, 2011 at 8:15 PM, Brian Granger <ellisonbg at gmail.com
> >> wrote:
> >
> >> Charlie,
> >>
> >> > I was playing around with the new Qt  widgets in 0.11-dev the other
> day
> >> and
> >> > was very impressed.  I tried throwing together a quick plugin to add
> an
> >> > IPython console to the Qt-based program Quantum GIS [QGIS][1].
>  Creating
> >> the
> >> > plugin was easy enough as QGIS runs an embedded Python interpreter and
> >> > exposes a lot of PyQt4 functionality.  However, I noticed a couple of
> >> things
> >> > and was wondering if anyone on this list had any comments to share:
> >> >
> >> > - The shell is based on a stripped down version of `ipython-qtconsole`
> >> and
> >> > works but is very, very slow for some reason.  The is a delay of a
> >> > couple
> >> of
> >> > seconds between when an action is triggered and a response comes back
> to
> >> the
> >> > terminal.  The only guess I have that may explain this behavior is
> that
> >> QGIS
> >> > is a C++ application that embeds a Python interpreter whereas
> >> > ipython-qtconsole is a pure PyQt4 program.  Perhaps QGIS is not
> letting
> >> the
> >> > Python threads run as often as a pure PyQt4 program would.
> >> > This could be a QGIS-specific problem, but I was wondering if anyone
> >> > else
> >> > had the chance to run the IPython Qt widget in an embedded interpreter
> >> and
> >> > noticed similar results?
> >>
> >> In general the ipython-qtconsole should be quite fast.  But for that
> >> to be the case, the qt event loop has to be running.  If the C++ is
> >> not letting the qt event loop run often enough, you would definitely
> >> see quite slow behavior.
> >
> >
> > This is what I suspect, I would probably have to spend some quality time
> > with the C++ side of the code to be sure.
> >
> >
> >
> >> > - Currently, the only IPython kernel available for use with the Qt
> >> > widget
> >> > runs the IPython interpreter in a separate process.  For QGIS, it
> would
> >> be
> >> > very nice to have the option of running an IPython kernel inside the
> >> > same
> >> > process as the Qt widgets.  IPython would then have direct access to
> the
> >> > QGIS interface and would be able to inspect and operatie on the PyQt4
> >> > objects that compose the GUI providing a powerful tool for users
> >> performing
> >> > spatial analysis and developers building plugins.
> >> > Are there any plans to develop IPython kernels that do not run
> >> > in separate processes?  I spent a few days trying to build one myself
> by
> >> > wrapping an InteractiveShell, but I feel like I am flailing a little
> bit
> >> > when it comes to creating a kernel_manager that will integrate with
> the
> >> Qt
> >> > frontend.  If anyone has any design advice on how to implement an
> >> in-process
> >> > kernel I would love to hear it.
> >>
> >> There is interest in having an in process IPython, but I honestly
> >> think it will not work very well at all.  This is mostly because of
> >> the GIL.  I would simply turn the model inside out.  If you run the
> >> QGIS code that has all the PyQt widgets in the IPython kernel (start
> >> the kernel with the pyqt event loop running using --pylab qt) your
> >> ipython session will have all the access that it needs.  IOW, don't
> >> embed IPython in your app, embed your app in IPython.  If this model
> >> doesn't work for you, we can talk more about how to get started on an
> >> in process IPython widget, but it will be a significant amount of work
> >> I'm afraid.
> >
> >
> > The problem with embedding QGIS in IPython is that the main app is
> written
> > in C++ and `import qgis` only works out of the box on Linux---for Windows
> > and OS X it is a bit of a hassle to get paths set up right.  Many things
> are
> > wrapped via SIP and it is possible to run a lot of widgets from IPython
> but
> > the main windows are not so it is currently not possible to re-launch the
> > app under Python.
> >
> > QGIS currently includes a Python console that makes use of
> > code.InteractiveInterpreter from the standard library. This console it
> works
> > very well with the exception that I always find myself pounding on the
> tab
> > key in vain when I forget what I can do with a random Python object.
> >  Replacing the QGIS Python console with an IPython console would be
> awesome
> > and would cut out a lot of trips to the QGIS source code and various
> > reference manuals.
> >
> > If an in-process kernel could be implemented that shared the same
> interface
> > as the QtKernelManager, then users could easily be given the option of
> > launching an external process for tasks where the GIL would be a bother.
> >
> > I am willing to take a stab at developing an in-process widget but after
> > thrashing around a bit this weekend I feel like some guidance would help
> me
> > focus my efforts.
> >
> >
> >
> >> > The plugin is available on [GitHub][2] under the tag `0.1` if anyone
> >> feels
> >> > interested in checking it out as an IPython use case.  The 0.1 version
> >> > is
> >> > really quite simple---about 5-10 lines in console.py that do the
> actual
> >> > implementation.  To use:
> >> >   - Install QGIS
> >> >   - Clone project as a sub-directory of ~/.qgis/python/plugins/
> >> >   - Run `make` (runs PyQt4 dev tools on ui and qrc files)
> >> >   - Launch QGIS and enable Plugin through Plugins -> Manage Plugins
> menu
> >> >
> >> > Keep up the good work on IPython development!
> >>
> >> Thanks!
> >>
> >> Brian
> >>
> >> > -Charlie
> >> > [1]: http://www.qgis.org
> >> > [2]: http://www.github.com/Sharpie/qgis-ipython/tree/0.1
> >> > _______________________________________________
> >> > IPython-dev mailing list
> >> > IPython-dev at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Brian E. Granger, Ph.D.
> >> Assistant Professor of Physics
> >> Cal Poly State University, San Luis Obispo
> >> bgranger at calpoly.edu
> >> ellisonbg at gmail.com
> >>
> >
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> >
>
>
>
> --
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110212/3552f71a/attachment.html>

From fperez.net at gmail.com  Sat Feb 12 19:56:35 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 12 Feb 2011 16:56:35 -0800
Subject: [IPython-dev] No displayhook triggered for raw 'asdf'
In-Reply-To: <AANLkTikitPEZktazQrRp1zZPY73LUNQq99ZS9v5OptfM@mail.gmail.com>
References: <AANLkTikdY0vrAoZKRwK_yjMe9C+2XaC43vHWYkNG4uG6@mail.gmail.com>
	<AANLkTikLeKS7P-Z-8Qwy=HPY4WT5zt-LAkLaw6WxN-bi@mail.gmail.com>
	<20110126230020.GI5046@phare.normalesup.org>
	<iht35t$fd8$1@dough.gmane.org>
	<AANLkTin-PWUd4Vf1NmsDw6rJTnKg4i89G+LHz6L0x6z_@mail.gmail.com>
	<AANLkTinXxCL0h7fsMwWdK1HMXav6JMzquk0aru1NfCRC@mail.gmail.com>
	<AANLkTim-ADJj-X1p6f-cdxCC92acDsp3bxjhpa07uD6i@mail.gmail.com>
	<AANLkTikLrwvTG5ndz_GVWnyA9-ahxss3R8Hfp0-qfU57@mail.gmail.com>
	<AANLkTikitPEZktazQrRp1zZPY73LUNQq99ZS9v5OptfM@mail.gmail.com>
Message-ID: <AANLkTik-tWL_b6q8aWx5d5momqA36GVvxUEy=RRZvHTe@mail.gmail.com>

On Sat, Feb 12, 2011 at 11:22 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> One thing I think we should do is figure out a way of dividing the
> review workload so nothing slips through the cracks. ?When a review
> comes in, do we want to assign a "lead" reviewer that will make sure
> it gets pushed through in a timely manner? ?Sort of like journals
> assign reviewers to articles. ?Our current situation is like running a
> peer review journal, without an editor to manage the review proces...

Yup, totally agreed.  I think Thomas is going to send soon an email on
this one, he has some good ideas to help out.

One thing to keep in mind: currently github has no way of notifying of
pull requests anyone outside of team members with write access to the
repo.  But they supposedly are working on it:

http://support.github.com/discussions/pull-requests/118-send-pull-request-notifications-to-all-watchers-or-an-external-mail-list

So hopefully soon we should be able to add the whole -dev list to get
auto-notified of pull requests, which will help in letting people know
of requests they could potentially help with.

Cheers,

f


From fperez.net at gmail.com  Sat Feb 12 19:57:12 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 12 Feb 2011 16:57:12 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
Message-ID: <AANLkTiky0r_fscRbhbTq7yLyyjJrUkKBeaMYPoE8uXYj@mail.gmail.com>

On Fri, Feb 11, 2011 at 8:11 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> OK, if we do this, let make sure that:
>
> * There are tests.
> * It is fully documented in docstrings.
>
> I am mostly concerned about the fact that this type of stuff tends to
> get deposited with us to maintain in the future (like port to Python
> 3). ?Ideally, these things would have maintainers that are committed
> to helping out with things.

Very valid points.

Cheers,

f


From fperez.net at gmail.com  Sat Feb 12 20:01:54 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 12 Feb 2011 17:01:54 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
	<DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
Message-ID: <AANLkTi=cEfem=Qy9cjLrNWMzuQC7ja8s6Z5xu+ACZ13N@mail.gmail.com>

Hi Nicolas,

On Sat, Feb 12, 2011 at 1:32 AM, Nicolas Rougier
<Nicolas.rougier at inria.fr> wrote:
> After reading the commentaries, I realize there are still some work to do before the code can be properly integrated within IPython and I will try to make a nicer version as well as fully document it (using the template.py). For the git pull, I'm not very familiar with it and I don't know the proper command to pull or push, woudl yu have any pointer on this ?
>

Yes, the first 4 sections of our developer guide document coding and
documentation guidelines, as well as the pull request process with
github:

http://ipython.github.com/ipython-doc/dev/development/index.html

If you've never used git/github and get confused with any of the
steps, please do let us know and we'll do our best to help.  While
there's a little learning curve at the beginning, I think you'll find
that learning to use git/github will pay off in many ways in your
work.  Many scientific python projects have either already moved or
are in the process of moving to git and/or github (numpy, ipython,
enthought.*, nipy.*, sympy, and over the next few days matplotlib and
scipy, likely many others too).

> For making the code to work with the QT console, it would require to parse the string sent to the console and interpret the ascii sequences. I did it some time ago for a gtk/ipython console but I'm not familiar at all with qt. (parsing code is of course available if one want to work on it).
>

Interestingly, the behavior of the Qt console was *identical* to that
of the Linux text console (the 'bare' screen console, outside of X11).
 I don't know why that would be the case, just a data point for you...

But for now, working in an xterm is already good.

Cheers,

f


From sunqiang at gmail.com  Sat Feb 12 20:22:53 2011
From: sunqiang at gmail.com (sunqiang)
Date: Sun, 13 Feb 2011 09:22:53 +0800
Subject: [IPython-dev] Issue of install ipython-py3k
In-Reply-To: <AANLkTimWZMftoL9k7E99avvY1TkUjqfNivhtQ54uwYEz@mail.gmail.com>
References: <AANLkTikdxipXbpzprepgXEjnLb22omVMmgr-R7dYMJQK@mail.gmail.com>
	<AANLkTimWZMftoL9k7E99avvY1TkUjqfNivhtQ54uwYEz@mail.gmail.com>
Message-ID: <AANLkTimPEvc7y+k4sb5K8rzzJ7WD9uea=dDkzbXsF2n=@mail.gmail.com>

Thomas, Thanks for the fix, I'll keep using ipython on python3.2 to
test my little script.

On Sun, Feb 13, 2011 at 3:11 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> Thanks, Sunqiang, I've pushed the fix.
>
> Keep me posted of any more problems you find using it with Python 3.2. I
> probably won't upgrade until April, with the next Ubuntu release.
>
> Technical details: The value coming from a subprocess.Popen().communicate()
> was a byte string. This was being written to a config file, but the
> rewritten configparser in Python 3.2 specifically checks that the option to
> be set is a (unicode) str. I've actually specified that it should be decoded
> as ascii: the hash digest should never include any non-ascii characters, and
> we'll find out quickly if it's doing anything unexpected.
>
> Thomas
>


From fperez.net at gmail.com  Sat Feb 12 20:40:11 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 12 Feb 2011 17:40:11 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTinCEgMsRer6Xm9W6Rkg8zz8M6sFpWC82LipC+Xy@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
	<DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
	<AANLkTinCEgMsRer6Xm9W6Rkg8zz8M6sFpWC82LipC+Xy@mail.gmail.com>
Message-ID: <AANLkTikzDn=0pQA2WerE+Mi2OfS370f=cePEHOursxBT@mail.gmail.com>

Hey, Sat, Feb 12, 2011 at 10:42 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> Do you think it would make sene to implement this as an object that
> has special print methods like __pretty__, __png__, etc. ?That would
> make it super easy to add support for other frontends.

Mmh, interesting...  Initially I thought of this as a purely
functional thing, a terminal version of imshow() and nothing else.
But what you're thinking of is connecting it to our extended display
system... I actually do like that, especially since it would be code
coming from IPython.

Here's a suggestion.  Considering:

1. That Nicolas is new to IPython development and not likely to know
all the details of the new display design.

2. That we're still sorting out (in the other thread) precisely how we
want to go about that design, after some of the concerns Robert
raised.

I propose that we go for now with a purely functional implementation
(basically a cleaned up, documented and tested version of Nicolas
current code, but with the same overall approach).  This will make it
easier for Nicolas to contribute his work now.  But we keep this as a
prime candidate to test out the display system, and once we finalize
the design decisions, it can be refactored in that direction.  The
refactoring would be very easy, since it's basically replacing the
current printing with assembling a string to be the return value of
the repr.

I now realize this is probably the best course of action for another
reason: in its current form, the code works even with the plain
terminal ipython that has no knowledge of the full display machinery.
If we move it to that model, I think it wouldn't quite work in our
current terminal client (correct me if I'm wrong).

Does that sound reasonable?

Cheers,

f


From fperez.net at gmail.com  Sat Feb 12 21:05:30 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 12 Feb 2011 18:05:30 -0800
Subject: [IPython-dev] All documentation now on Github via gh-pages
Message-ID: <AANLkTin+RJ7nAXKk9NhS2HMdGgDm0LENMBpD+RXpjHEq@mail.gmail.com>

Hi everyone,

Min (who did most of the work) and I just completed moving our docs
over to Github, using the gh-pages system but via the tricks we
discussed before (what I wrote for datarray) that avoid polluting the
main ipython repo with any static HTML content.

The old documentation page on the Moin wiki is now empty and simply points to:

http://ipython.github.com/ipython-doc/

This is actually all nicely generated from a real git repo:

https://github.com/ipython/ipython-doc

So that any core team member can easily push doc updates.  Note that
this repo contains actually almost zero original content (just an
index.rst page and a trivial build script), all the sources are still
in the doc/ directory of the IPython main repo.  This repo is simply a
static holder of the generated HTML/pdf so that we can take advantage
of gh-pages hosting, 'git push' for publication, and repository
administrative control for permissions (instead of me manually
managing ssh keys on the scipy.org server, which I have no time for).

Thanks to Min for adapting the previous tools and setting things up
with the CSS.

In summary, for contributing to the docs:

- for the actual manual sources, everything is in the real ipython
repo, docs/source directory like always.

- to edit the index page or the css for looks, do fork/clone the -doc
repo and make a pull request on that one (it will be handled just like
any other pull request on the main repo, but likely with quicker
turnaround since there's no major code to review there).

Cheers,

f


From fperez.net at gmail.com  Sat Feb 12 22:01:47 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 12 Feb 2011 19:01:47 -0800
Subject: [IPython-dev] Welcoming Thomas to the core/merge team
Message-ID: <AANLkTi=PeGrcHHfpGWsc90iSqtMBeSxNxoEao=-yztcg@mail.gmail.com>

Hi all,

I just wanted to welcome Thomas Kluyver to the select club of people
who mange to work on IPython instead of completing their PhDs... :)
After all his great work on the py3 port, pyzmq fixes for py3 also,
and work on the core, frequent code reviews, etc, it's a pleasure to
have one more set of hands directly participating in merging
contributions.  I hope that this will also help with reducing the
turnaround time for reviews and feedback when I or other traditional
members bottleneck out on that strange thing that passes for a normal
life.

Cheers,

f


From Nicolas.rougier at inria.fr  Sun Feb 13 07:37:44 2011
From: Nicolas.rougier at inria.fr (Nicolas Rougier)
Date: Sun, 13 Feb 2011 13:37:44 +0100
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTi=cEfem=Qy9cjLrNWMzuQC7ja8s6Z5xu+ACZ13N@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
	<DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
	<AANLkTi=cEfem=Qy9cjLrNWMzuQC7ja8s6Z5xu+ACZ13N@mail.gmail.com>
Message-ID: <AA06C61D-F114-4F37-83D7-FF1EC3267556@inria.fr>


Thanks, the guide has been very helpful (I opened a bug report for a small typo).


If I did thing properly, here is the summary:
https://github.com/rougier/ipython/compare/master...ascii_imshow

I added the possibility to display MxNx3 (RGB) or MxNx4 (RGBA) since the code is quite similar. There is no test yet since I'm not sure how to test such a visual thing.

I also tried to detect terminal number of colors using curses (if available) because I don't know if IPython already got some terminal capabilities related functions (mainly, size and number of colors).


Nicolas 




From ben.root at ou.edu  Sun Feb 13 11:48:53 2011
From: ben.root at ou.edu (Benjamin Root)
Date: Sun, 13 Feb 2011 10:48:53 -0600
Subject: [IPython-dev] No displayhook triggered for raw 'asdf'
In-Reply-To: <AANLkTik-tWL_b6q8aWx5d5momqA36GVvxUEy=RRZvHTe@mail.gmail.com>
References: <AANLkTikdY0vrAoZKRwK_yjMe9C+2XaC43vHWYkNG4uG6@mail.gmail.com>
	<AANLkTikLeKS7P-Z-8Qwy=HPY4WT5zt-LAkLaw6WxN-bi@mail.gmail.com>
	<20110126230020.GI5046@phare.normalesup.org>
	<iht35t$fd8$1@dough.gmane.org>
	<AANLkTin-PWUd4Vf1NmsDw6rJTnKg4i89G+LHz6L0x6z_@mail.gmail.com>
	<AANLkTinXxCL0h7fsMwWdK1HMXav6JMzquk0aru1NfCRC@mail.gmail.com>
	<AANLkTim-ADJj-X1p6f-cdxCC92acDsp3bxjhpa07uD6i@mail.gmail.com>
	<AANLkTikLrwvTG5ndz_GVWnyA9-ahxss3R8Hfp0-qfU57@mail.gmail.com>
	<AANLkTikitPEZktazQrRp1zZPY73LUNQq99ZS9v5OptfM@mail.gmail.com>
	<AANLkTik-tWL_b6q8aWx5d5momqA36GVvxUEy=RRZvHTe@mail.gmail.com>
Message-ID: <AANLkTikMk+dqNVCkwDyUnFJvybC8cLmijvJsko_Dvb9a@mail.gmail.com>

On Saturday, February 12, 2011, Fernando Perez <fperez.net at gmail.com> wrote:
> On Sat, Feb 12, 2011 at 11:22 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> One thing I think we should do is figure out a way of dividing the
>> review workload so nothing slips through the cracks. ?When a review
>> comes in, do we want to assign a "lead" reviewer that will make sure
>> it gets pushed through in a timely manner? ?Sort of like journals
>> assign reviewers to articles. ?Our current situation is like running a
>> peer review journal, without an editor to manage the review proces...
>
> Yup, totally agreed. ?I think Thomas is going to send soon an email on
> this one, he has some good ideas to help out.
>
> One thing to keep in mind: currently github has no way of notifying of
> pull requests anyone outside of team members with write access to the
> repo. ?But they supposedly are working on it:
>
> http://support.github.com/discussions/pull-requests/118-send-pull-request-notifications-to-all-watchers-or-an-external-mail-list
>
> So hopefully soon we should be able to add the whole -dev list to get
> auto-notified of pull requests, which will help in letting people know
> of requests they could potentially help with.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>

Actually, I get notified of pull requests for numpy through an rss
feed.  All I had to do was watch the numpy repo and add my dashboard
feed to my reader.  Not email, but it works.

Ben Root


From takowl at gmail.com  Sun Feb 13 12:45:10 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 13 Feb 2011 17:45:10 +0000
Subject: [IPython-dev] Outstanding pull requests
Message-ID: <AANLkTikvzZzsZEWyaJumic0pXbs9prd_eHoVC1QJq3mi@mail.gmail.com>

By way of my debut in the core team, I've prepared a list of all the open
pull requests, with a very brief assessment of each, in the hope that we can
start sorting them out (we're currently at 31 open requests):

Magic functions documentation (several branches from SciPy India):
[The first two listed here seem to have changed various bits of indentation
beyond the specific things they're described as changing. This makes the
diffs rather long. Do we want to copy the key changes out in new commits,
using the --author option to credit them?]
- cd: looks like some examples are at the bash prompt, not the ipython
prompt. https://github.com/ipython/ipython/pull/233/files
- profile: Has neatly documented the Python profile module. Unfortunately,
IPython's %profile command does something entirely different.
https://github.com/ipython/ipython/pull/232/files
- logstart: Seems OK, although there's an "examples" title added for logstop
with no actual examples. logstop probably doesn't need an example, but we
should tidy up the "examples" title.
https://github.com/ipython/ipython/pull/231/files
- edit: Clean diff, but giving an example for edit is tricky, because you
can't show the interactive editor. Also restates the -p option in a comment
in the example. https://github.com/ipython/ipython/pull/230/files (closed
220 as duplicate)
- pdef (it's confusing to use magic functions as the arguments for the
example, we should have a user defined function) & colors (one example is
probably enough, and the last two are wrong, using %usage instead of
%colors) https://github.com/ipython/ipython/pull/229/files
- who, whos, who_ls: Looks good. https://github.com/ipython/ipython/pull/228
- logon: Not quite sure what's happened here. Example seems to be a bunch of
Java-ish code. It probably needs to be redone from scratch.
https://github.com/ipython/ipython/pull/227
- pwd: pwd example is good, but has also added to exit (doesn't really need
an example), dirs (example doesn't really show what it's for), and color
(just an "examples" heading, no actual examples.
https://github.com/ipython/ipython/pull/226/files
- save: Clean diff, but the example and explanation aren't very clear.
Probably needs to be redone from scratch.
https://github.com/ipython/ipython/pull/225
- cpaste: Appears to have ripped out the actual function while writing the
docs, and stitched its definition onto the magic_paste function code.
Example could also be a bit clearer.
https://github.com/ipython/ipython/pull/224
- colors: Same changes (& problems) as request 229.
https://github.com/ipython/ipython/pull/223
- timeit: Too many equivalent examples.
https://github.com/ipython/ipython/pull/222/files
- alias: Already had examples, changes remove one of them. Adds two, but one
(ls) is available without aliasing, and the other is inaccurate (the example
shows the result of a man command).
https://github.com/ipython/ipython/pull/221
- pycat: Doesn't actually use a Python file, so rather defeats the point of
pycat. Not sure that an example is really helpful, as it doesn't show the
colouring you get at a terminal. https://github.com/ipython/ipython/pull/219
- hist: Formatting is off (probably related to a bug in the code). Also, the
"this is an example of..." line seems redundant.
https://github.com/ipython/ipython/pull/218/files

Whew. Of these, I recommend merging 228 (%who), and closing 225 (%save), 232
(%profile) with "try again" and 227 (%logon), for which I'm not sure that
it's even possible to do an example.

The others will need further work. Fernando, any sort of feel for how many
of the SciPy India contributors will still be in the loop to revise their
contributions?

Looking at the others:

Small pull requests:
- %hist output format: I've solved the same problem in a different way in
pull request 261, see below. https://github.com/ipython/ipython/pull/236
- Logging unicode characters (0.10 series): Good start. See my comment there
for minor concern. https://github.com/ipython/ipython/pull/249
- check_gtk: Removes a function. Mark Voorhies has tested the change and
commented that it works. I'll admit to not understanding what it's about.
https://github.com/ipython/ipython/pull/237
- Typos in zmq/blockingkernelmanager.py: Seems OK to me.
https://github.com/ipython/ipython/pull/203/files
- cProfiling ipython.py: Updated in response to Brian's concerns. Needs
reviewing again. https://github.com/ipython/ipython/pull/193
- Autocomplete in Emacs: Removes the all_completions function (which
apparently was broken). One other person says it works. Needs review.
https://github.com/ipython/ipython/pull/204
- Unicode issues [My changes]: Tests now integrated into main test suite
(thanks to help from Robert Kern). Needs review.
https://github.com/ipython/ipython/pull/252
- Wildcard module [My changes]: Fixes issue 129. The bug and the fix are
pretty much self contained, so shouldn't interfere with changes in other
code. No feedback yet. (this is pushing what I'd call small, but shouldn't
change functionality besides fixing the bug)
https://github.com/ipython/ipython/pull/251

Medium pull requests:
- Fixing up various magic commands with the new history system [My changes]:
Revised after Fernando's comments, unit tests added. Conflicts with
Madhusudan's #203, see below. https://github.com/ipython/ipython/pull/261
- %hist command bugs: This solves one of the same problems I've fixed in
#261, in a different way. https://github.com/ipython/ipython/pull/203/files
- Unbundling external libraries: Revised in response to Fernando's and
Brian's concerns. Needs reviewing again.
https://github.com/ipython/ipython/pull/191
- Pyside compatibility: Waiting for contributor to resolve bug with SVG
handling. https://github.com/ipython/ipython/pull/259
- Kernel logging: Fernando has requested a final check from Robert.
https://github.com/ipython/ipython/pull/264

Larger pull requests:
- Magic arguments decorators. Updated in response to Fernando's comments.
Needs reviewing again. https://github.com/ipython/ipython/pull/199

Not quite yet:
- HTML Notebook: Gained a lot of interest, but not quite ready yet. Possible
project for GSoC this year. (But if we're going to accumulate alternative
frontends, should we consider putting each frontend in its own repo?)
https://github.com/ipython/ipython/pull/179
- MinRK's newparallel: Work in progress. Lots of work.
https://github.com/ipython/ipython/pull/254

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110213/6649584c/attachment.html>

From ellisonbg at gmail.com  Sun Feb 13 13:10:59 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 13 Feb 2011 10:10:59 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTikzDn=0pQA2WerE+Mi2OfS370f=cePEHOursxBT@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
	<DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
	<AANLkTinCEgMsRer6Xm9W6Rkg8zz8M6sFpWC82LipC+Xy@mail.gmail.com>
	<AANLkTikzDn=0pQA2WerE+Mi2OfS370f=cePEHOursxBT@mail.gmail.com>
Message-ID: <AANLkTinPvZm4PCbWaHYyWzvT4Uhg4kM+74vGX-SKoE0W@mail.gmail.com>

Hi,

On Sat, Feb 12, 2011 at 5:40 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hey, Sat, Feb 12, 2011 at 10:42 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> Do you think it would make sene to implement this as an object that
>> has special print methods like __pretty__, __png__, etc. ?That would
>> make it super easy to add support for other frontends.
>
> Mmh, interesting... ?Initially I thought of this as a purely
> functional thing, a terminal version of imshow() and nothing else.
> But what you're thinking of is connecting it to our extended display
> system... I actually do like that, especially since it would be code
> coming from IPython.

Great.

> Here's a suggestion. ?Considering:
>
> 1. That Nicolas is new to IPython development and not likely to know
> all the details of the new display design.
>
> 2. That we're still sorting out (in the other thread) precisely how we
> want to go about that design, after some of the concerns Robert
> raised.

Definitely.

> I propose that we go for now with a purely functional implementation
> (basically a cleaned up, documented and tested version of Nicolas
> current code, but with the same overall approach). ?This will make it
> easier for Nicolas to contribute his work now. ?But we keep this as a
> prime candidate to test out the display system, and once we finalize
> the design decisions, it can be refactored in that direction. ?The
> refactoring would be very easy, since it's basically replacing the
> current printing with assembling a string to be the return value of
> the repr.

Sounds good.

> I now realize this is probably the best course of action for another
> reason: in its current form, the code works even with the plain
> terminal ipython that has no knowledge of the full display machinery.
> If we move it to that model, I think it wouldn't quite work in our
> current terminal client (correct me if I'm wrong).

Correct, but I think we could make a version that did work with a
plain terminal at the same time.

> Does that sound reasonable?

Yep.

Cheers,

Brian

> Cheers,
>
> f
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ellisonbg at gmail.com  Sun Feb 13 13:48:21 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 13 Feb 2011 10:48:21 -0800
Subject: [IPython-dev] No displayhook triggered for raw 'asdf'
In-Reply-To: <AANLkTik-tWL_b6q8aWx5d5momqA36GVvxUEy=RRZvHTe@mail.gmail.com>
References: <AANLkTikdY0vrAoZKRwK_yjMe9C+2XaC43vHWYkNG4uG6@mail.gmail.com>
	<AANLkTikLeKS7P-Z-8Qwy=HPY4WT5zt-LAkLaw6WxN-bi@mail.gmail.com>
	<20110126230020.GI5046@phare.normalesup.org>
	<iht35t$fd8$1@dough.gmane.org>
	<AANLkTin-PWUd4Vf1NmsDw6rJTnKg4i89G+LHz6L0x6z_@mail.gmail.com>
	<AANLkTinXxCL0h7fsMwWdK1HMXav6JMzquk0aru1NfCRC@mail.gmail.com>
	<AANLkTim-ADJj-X1p6f-cdxCC92acDsp3bxjhpa07uD6i@mail.gmail.com>
	<AANLkTikLrwvTG5ndz_GVWnyA9-ahxss3R8Hfp0-qfU57@mail.gmail.com>
	<AANLkTikitPEZktazQrRp1zZPY73LUNQq99ZS9v5OptfM@mail.gmail.com>
	<AANLkTik-tWL_b6q8aWx5d5momqA36GVvxUEy=RRZvHTe@mail.gmail.com>
Message-ID: <AANLkTikZ1W8Ry+6CVMJPAvk=xzPvQYBRY_a1vuVF094u@mail.gmail.com>

On Sat, Feb 12, 2011 at 4:56 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Sat, Feb 12, 2011 at 11:22 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> One thing I think we should do is figure out a way of dividing the
>> review workload so nothing slips through the cracks. ?When a review
>> comes in, do we want to assign a "lead" reviewer that will make sure
>> it gets pushed through in a timely manner? ?Sort of like journals
>> assign reviewers to articles. ?Our current situation is like running a
>> peer review journal, without an editor to manage the review proces...
>
> Yup, totally agreed. ?I think Thomas is going to send soon an email on
> this one, he has some good ideas to help out.
>
> One thing to keep in mind: currently github has no way of notifying of
> pull requests anyone outside of team members with write access to the
> repo. ?But they supposedly are working on it:
>
> http://support.github.com/discussions/pull-requests/118-send-pull-request-notifications-to-all-watchers-or-an-external-mail-list
>
> So hopefully soon we should be able to add the whole -dev list to get
> auto-notified of pull requests, which will help in letting people know
> of requests they could potentially help with.

Yes, that would be nice.

Cheers,

Brian

> Cheers,
>
> f
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ellisonbg at gmail.com  Sun Feb 13 14:05:11 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 13 Feb 2011 11:05:11 -0800
Subject: [IPython-dev] Welcoming Thomas to the core/merge team
In-Reply-To: <AANLkTi=PeGrcHHfpGWsc90iSqtMBeSxNxoEao=-yztcg@mail.gmail.com>
References: <AANLkTi=PeGrcHHfpGWsc90iSqtMBeSxNxoEao=-yztcg@mail.gmail.com>
Message-ID: <AANLkTim2f6sLkQqv2g-vSrpfXCWrrJPwjLzQntg3WVFO@mail.gmail.com>

Welcome Thomas!

Cheers,

Brian

On Sat, Feb 12, 2011 at 7:01 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi all,
>
> I just wanted to welcome Thomas Kluyver to the select club of people
> who mange to work on IPython instead of completing their PhDs... :)
> After all his great work on the py3 port, pyzmq fixes for py3 also,
> and work on the core, frequent code reviews, etc, it's a pleasure to
> have one more set of hands directly participating in merging
> contributions. ?I hope that this will also help with reducing the
> turnaround time for reviews and feedback when I or other traditional
> members bottleneck out on that strange thing that passes for a normal
> life.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From epatters at enthought.com  Sun Feb 13 14:49:17 2011
From: epatters at enthought.com (Evan Patterson)
Date: Sun, 13 Feb 2011 13:49:17 -0600
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTi=cEfem=Qy9cjLrNWMzuQC7ja8s6Z5xu+ACZ13N@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
	<DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
	<AANLkTi=cEfem=Qy9cjLrNWMzuQC7ja8s6Z5xu+ACZ13N@mail.gmail.com>
Message-ID: <AANLkTinH1E+qC3U1cUjZ65mNMHKoEEXVHFHsxD0o7FrD@mail.gmail.com>

On Sat, Feb 12, 2011 at 7:01 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hi Nicolas,
>
> On Sat, Feb 12, 2011 at 1:32 AM, Nicolas Rougier
> <Nicolas.rougier at inria.fr> wrote:
> > After reading the commentaries, I realize there are still some work to do
> before the code can be properly integrated within IPython and I will try to
> make a nicer version as well as fully document it (using the template.py).
> For the git pull, I'm not very familiar with it and I don't know the proper
> command to pull or push, woudl yu have any pointer on this ?
> >
>
> Yes, the first 4 sections of our developer guide document coding and
> documentation guidelines, as well as the pull request process with
> github:
>
> http://ipython.github.com/ipython-doc/dev/development/index.html
>
> If you've never used git/github and get confused with any of the
> steps, please do let us know and we'll do our best to help.  While
> there's a little learning curve at the beginning, I think you'll find
> that learning to use git/github will pay off in many ways in your
> work.  Many scientific python projects have either already moved or
> are in the process of moving to git and/or github (numpy, ipython,
> enthought.*, nipy.*, sympy, and over the next few days matplotlib and
> scipy, likely many others too).
>
> > For making the code to work with the QT console, it would require to
> parse the string sent to the console and interpret the ascii sequences. I
> did it some time ago for a gtk/ipython console but I'm not familiar at all
> with qt. (parsing code is of course available if one want to work on it).
> >
>
> Interestingly, the behavior of the Qt console was *identical* to that
> of the Linux text console (the 'bare' screen console, outside of X11).
>  I don't know why that would be the case, just a data point for you...
>
> But for now, working in an xterm is already good.
>

The reason is that we implement the (more useful) ANSI escape sequences in
the Qt console, but not any of the xterm-specific sequences. Since most
terminal emulators implement them for compatibility reasons, so should
we---or at least we should implement the important ones. Comparing Nicolas'
code with this:

http://invisible-island.net/xterm/ctlseqs/ctlseqs.html

it look likes we need only implement two more sequences to get his code
working. I will add this to my TODO list (but as a low priority item, I'm
afraid).

Evan

>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110213/ba1ada3e/attachment.html>

From fperez.net at gmail.com  Sun Feb 13 16:00:52 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Feb 2011 13:00:52 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTinH1E+qC3U1cUjZ65mNMHKoEEXVHFHsxD0o7FrD@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
	<DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
	<AANLkTi=cEfem=Qy9cjLrNWMzuQC7ja8s6Z5xu+ACZ13N@mail.gmail.com>
	<AANLkTinH1E+qC3U1cUjZ65mNMHKoEEXVHFHsxD0o7FrD@mail.gmail.com>
Message-ID: <AANLkTiku3V9QfkXOsf9kz10Xn8GRAEBG-CHwDoiuftCx@mail.gmail.com>

On Sun, Feb 13, 2011 at 11:49 AM, Evan Patterson <epatters at enthought.com> wrote:
> The reason is that we implement the (more useful) ANSI escape sequences in
> the Qt console, but not any of the xterm-specific sequences. Since most
> terminal emulators implement them for compatibility reasons, so should
> we---or at least we should implement the important ones.

Aha, thanks for the info, that explains it.  Sure, if you can get
around to it at some point and it's not hard to implement, all the
better.

Cheers,

f


From fperez.net at gmail.com  Sun Feb 13 16:29:29 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Feb 2011 13:29:29 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AA06C61D-F114-4F37-83D7-FF1EC3267556@inria.fr>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
	<DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
	<AANLkTi=cEfem=Qy9cjLrNWMzuQC7ja8s6Z5xu+ACZ13N@mail.gmail.com>
	<AA06C61D-F114-4F37-83D7-FF1EC3267556@inria.fr>
Message-ID: <AANLkTin2p7MYPd_qqj+yxcq2HWgdFSd4K+td2aVoRny=@mail.gmail.com>

Hi Nicolas,

On Sun, Feb 13, 2011 at 4:37 AM, Nicolas Rougier
<Nicolas.rougier at inria.fr> wrote:
> Thanks, the guide has been very helpful (I opened a bug report for a small typo).

Glad to hear that, and thanks for filing the report; Thomas already
jumped on it and the online guide should be soon updated once Thomas
sets his environment to do the gh-pages pushes (up until now only Min
and I had been pushing).

> If I did thing properly, here is the summary:
> https://github.com/rougier/ipython/compare/master...ascii_imshow

Excellent!  I would suggest that you now simply click on the 'Pull
Request' button so we can review your contribution for merge.  It's
likely to go through a few more iterations before it's merged, but
once in a pull request, we have a nice centralized location for the
discussion, you can keep fine-tuning it, and when ready it will go in
cleanly.

> I added the possibility to display MxNx3 (RGB) or MxNx4 (RGBA) since the code is quite similar. There is no test yet since I'm not sure how to test such a visual thing.

Great features.

The topic of testing is very important, so I want to address this in
some detail, hopefully this will be useful to others as well.

You are indeed correct that testing something visual is tricky.  The
point is to realize that printing is a side effect, and testing side
effects is in general quite hard.  But you can think of printing as a
two-step process: a) computing the string to be printed b) actually
sending that string to the terminal.  While b) more or less can only
be checked interactively by a human, a) can be automatically
validated.

Basically, when thinking of testing, you want to code so there's a
clean separation between the functional computation and the part that
does purely the side effects.  That is, instead of:

def show_foo(x):
  print 'abc', x, 'pqr'

you would do

def compute_foo(x):
  return 'abc %s pqr' % x

def show_foo(x):
  print compute_foo(x)

That way at least compute_foo(x) can be fully tested in a 100%
functional way.  And then a test can also be written for show_foo()
just to ensure it doesn't crash (so that any api changes in
compute_foo are correctly caught also in show_foo), but that test
doesn't need to actually validate the output.  In simple cases it may
(such as with a doctest), but in others it may be hard, like in your
situation.

Let me know if this helps.  If it's clear enough, we'll add it to the
dev guidelines in the testing section.

> I also tried to detect terminal number of colors using curses (if available) because I don't know if IPython already got some terminal capabilities related functions (mainly, size and number of colors).

No idea on that front, sorry.

Many thanks for your willingness to follow the (hopefully
straightforward) process!  We're really glad to have more
contributions to the project, and I'm excited to start seeing things
going more smoothly on this front.

Best regards,


f


From fperez.net at gmail.com  Sun Feb 13 17:05:49 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Feb 2011 14:05:49 -0800
Subject: [IPython-dev] No displayhook triggered for raw 'asdf'
In-Reply-To: <AANLkTikMk+dqNVCkwDyUnFJvybC8cLmijvJsko_Dvb9a@mail.gmail.com>
References: <AANLkTikdY0vrAoZKRwK_yjMe9C+2XaC43vHWYkNG4uG6@mail.gmail.com>
	<AANLkTikLeKS7P-Z-8Qwy=HPY4WT5zt-LAkLaw6WxN-bi@mail.gmail.com>
	<20110126230020.GI5046@phare.normalesup.org>
	<iht35t$fd8$1@dough.gmane.org>
	<AANLkTin-PWUd4Vf1NmsDw6rJTnKg4i89G+LHz6L0x6z_@mail.gmail.com>
	<AANLkTinXxCL0h7fsMwWdK1HMXav6JMzquk0aru1NfCRC@mail.gmail.com>
	<AANLkTim-ADJj-X1p6f-cdxCC92acDsp3bxjhpa07uD6i@mail.gmail.com>
	<AANLkTikLrwvTG5ndz_GVWnyA9-ahxss3R8Hfp0-qfU57@mail.gmail.com>
	<AANLkTikitPEZktazQrRp1zZPY73LUNQq99ZS9v5OptfM@mail.gmail.com>
	<AANLkTik-tWL_b6q8aWx5d5momqA36GVvxUEy=RRZvHTe@mail.gmail.com>
	<AANLkTikMk+dqNVCkwDyUnFJvybC8cLmijvJsko_Dvb9a@mail.gmail.com>
Message-ID: <AANLkTinsgG4KSARTthXogvv_wszEtN0yxroZLWBZ_PpN@mail.gmail.com>

On Sun, Feb 13, 2011 at 8:48 AM, Benjamin Root <ben.root at ou.edu> wrote:
> Actually, I get notified of pull requests for numpy through an rss
> feed. ?All I had to do was watch the numpy repo and add my dashboard
> feed to my reader. ?Not email, but it works.

Ah, thanks for that tip!  I've never really gotten hooked on RSS for
some reason, but this is may be a good solution for many.

Cheers,

f


From takowl at gmail.com  Sun Feb 13 17:29:31 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 13 Feb 2011 22:29:31 +0000
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTiku3V9QfkXOsf9kz10Xn8GRAEBG-CHwDoiuftCx@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
	<DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
	<AANLkTi=cEfem=Qy9cjLrNWMzuQC7ja8s6Z5xu+ACZ13N@mail.gmail.com>
	<AANLkTinH1E+qC3U1cUjZ65mNMHKoEEXVHFHsxD0o7FrD@mail.gmail.com>
	<AANLkTiku3V9QfkXOsf9kz10Xn8GRAEBG-CHwDoiuftCx@mail.gmail.com>
Message-ID: <AANLkTi=W5HY6pWPaSo5sR+iB-xpJ9ieKa5k=3HXT+wB5@mail.gmail.com>

On 13 February 2011 21:00, Fernando Perez <fperez.net at gmail.com> wrote:

> Aha, thanks for the info, that explains it.  Sure, if you can get
> around to it at some point and it's not hard to implement, all the
> better.
>

Though I would add that the ability to render ASCII heatmaps in the Qt
console where we can display png and svg is perhaps not terribly urgent.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110213/b54767ca/attachment.html>

From fperez.net at gmail.com  Sun Feb 13 17:30:51 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Feb 2011 14:30:51 -0800
Subject: [IPython-dev] Ascii imshow
In-Reply-To: <AANLkTi=W5HY6pWPaSo5sR+iB-xpJ9ieKa5k=3HXT+wB5@mail.gmail.com>
References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr>
	<AANLkTikKHQXqLJ9Ff1WQXZM43Sqan0VySQeJ6UJ7F3gh@mail.gmail.com>
	<AANLkTi=4XWq0zmdTwqUwJsnYeiOWr1AYLZAQ3yVa1yuK@mail.gmail.com>
	<AANLkTimcakWgsvnrCmzxWqhmn9DT-j=Fbr1cQ2GTy3XM@mail.gmail.com>
	<AANLkTinnKWgeQLojUf6qJwJUAKWLsTN_k4smV357Gtc9@mail.gmail.com>
	<DFA7C957-694C-4039-9085-C5B0E09D6F00@inria.fr>
	<AANLkTi=cEfem=Qy9cjLrNWMzuQC7ja8s6Z5xu+ACZ13N@mail.gmail.com>
	<AANLkTinH1E+qC3U1cUjZ65mNMHKoEEXVHFHsxD0o7FrD@mail.gmail.com>
	<AANLkTiku3V9QfkXOsf9kz10Xn8GRAEBG-CHwDoiuftCx@mail.gmail.com>
	<AANLkTi=W5HY6pWPaSo5sR+iB-xpJ9ieKa5k=3HXT+wB5@mail.gmail.com>
Message-ID: <AANLkTi=_NQ3+2Ev5DcENg6o2VYYx=scZGVeKQtJ_WRdD@mail.gmail.com>

On Sun, Feb 13, 2011 at 2:29 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
> Though I would add that the ability to render ASCII heatmaps in the Qt
> console where we can display png and svg is perhaps not terribly urgent.

Agreed.  Evan said it would be low on his list, so it's one of those
'for an idle moment' things.

Cheers,

f


From takowl at gmail.com  Sun Feb 13 17:51:20 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Sun, 13 Feb 2011 22:51:20 +0000
Subject: [IPython-dev] Issues with building documentation
Message-ID: <AANLkTi=eJnos+bkCS1vkRpSqWSBSv9xGebmLVmVTNw-v@mail.gmail.com>

There seem to be some issues with getting the docs to build. My guess would
be that these only show up if you've not built the docs before.

On Fernando's instructions, I went into docs and ran ./gh-pages.py. This
tries to call "git describe", which fails because it can't find annotated
tags. I peeked at the source, took a guess, and ran "./gh-pages.py dev".
This gets further, but fails when it tries to copy build/html into the new
repo, because I haven't yet built the docs at all.

Next, I try to build the html docs with "make html". This sets out, but then
falls over at:

ValueError: Could not import class or module
'IPython.Extensions.PhysicalQInteractive' specified for inheritance diagram

This looks like a leftover from old code - our extensions directory is now
all lower case, and there's no PhysicalQInteractive module there.

Is there some sort of preparation step I've overlooked? Or is this area in
need of some attention?

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110213/7bd0b350/attachment.html>

From fperez.net at gmail.com  Sun Feb 13 18:35:26 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Feb 2011 15:35:26 -0800
Subject: [IPython-dev] Issues with building documentation
In-Reply-To: <AANLkTi=eJnos+bkCS1vkRpSqWSBSv9xGebmLVmVTNw-v@mail.gmail.com>
References: <AANLkTi=eJnos+bkCS1vkRpSqWSBSv9xGebmLVmVTNw-v@mail.gmail.com>
Message-ID: <AANLkTimda8KuH7Dy3oUXaoGRjfnU_4QCnvdun0JkiN9h@mail.gmail.com>

Just for reference, no worries about this.  We've sorted it out on IRC.

On Sun, Feb 13, 2011 at 2:51 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> There seem to be some issues with getting the docs to build. My guess would
> be that these only show up if you've not built the docs before.
>
> On Fernando's instructions, I went into docs and ran ./gh-pages.py. This
> tries to call "git describe", which fails because it can't find annotated
> tags. I peeked at the source, took a guess, and ran "./gh-pages.py dev".
> This gets further, but fails when it tries to copy build/html into the new
> repo, because I haven't yet built the docs at all.
>
> Next, I try to build the html docs with "make html". This sets out, but then
> falls over at:
>
> ValueError: Could not import class or module
> 'IPython.Extensions.PhysicalQInteractive' specified for inheritance diagram
>
> This looks like a leftover from old code - our extensions directory is now
> all lower case, and there's no PhysicalQInteractive module there.
>
> Is there some sort of preparation step I've overlooked? Or is this area in
> need of some attention?
>
> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


From fperez.net at gmail.com  Sun Feb 13 19:36:21 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Feb 2011 16:36:21 -0800
Subject: [IPython-dev] Comments on IPython Qt widgets
In-Reply-To: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>
References: <AANLkTikPr1spizinkOPsYjR1iEAPZF0AU5R4qSqP+ZHx@mail.gmail.com>
Message-ID: <AANLkTimK26jZvaBBVeFanwx6tpsaHLv6LvAF-DbkuKPp@mail.gmail.com>

Dear Charlie,

Just joining the others to welcome you to the list!  It's great to see
interest and new participants with regards to the new architecture and
the Qt machinery. I hope we'll be able to help yo contribute and do
useful thins with ipython in the GIS domain.

On Mon, Feb 7, 2011 at 6:52 PM, Charlie Sharpsteen
<source at sharpsteen.net> wrote:
> Hello IPython developers,
> I was playing around with the new Qt ?widgets in 0.11-dev the other day and
> was very impressed. ?I tried?throwing?together a quick plugin to add an
> IPython console to the Qt-based program Quantum GIS [QGIS][1]. ?Creating the
> plugin was easy enough as QGIS runs an embedded Python interpreter and
> exposes a lot of PyQt4 functionality.??However, I noticed a couple of things
> and was wondering if anyone on this list had any comments to share:
>

> - Currently, the only IPython kernel available for use with the Qt widget
> runs the IPython interpreter in a?separate?process. ?For QGIS, it would be
> very nice to have the option of running an IPython kernel inside the same
> process as the Qt widgets. ?IPython would then have direct access to the
> QGIS interface and would be able to inspect and operatie on the PyQt4
> objects that compose the GUI providing a powerful tool for users performing
> spatial analysis and developers building plugins.
> Are there any plans to develop IPython kernels that do not run
> in?separate?processes? ?I spent a few days trying to build one myself by
> wrapping an InteractiveShell, but I feel like I am flailing a little bit
> when it comes to creating a kernel_manager that will integrate with the Qt
> frontend. ?If anyone has any design advice on how to implement an in-process
> kernel I would love to hear it.

I actually think that this is something we do very much need and would
like to have, despite the (absolutely valid) caveats that Brian
raised.  While it's true that an in-process kernel can block and/or
crash your entire app, it can also be very useful (see Mayavi for a
good example where even with rather gross hack remains extremely
useful).

Let me try to explain our architecture a little bit.  If this is clear
enough let me know, and I'll add it to our dev docs.  For all of this,
keep this diagram in mind:

http://ipython.github.com/ipython-doc/dev/development/messaging.html

In a two-process model, the Kernel is proxied on the client process by
a KernelManager object.  This is the object that all client code uses
to communicate with the kernel, and it's the *only* object in charge
of doing explicit zmq calls.  The rest of ipython has no idea zmq
exists, it just talks to this KernelManager object.  The basic
KernelManager lives in:

https://github.com/ipython/ipython/blob/master/IPython/zmq/kernelmanager.py

but that can't be used directly, since each front end needs to have
code to specifically handle the generated messages according to its
own UI constraints. So for example the Qt console has here its version
that wires the message handlers to Qt's event mechanisms:

https://github.com/ipython/ipython/blob/master/IPython/frontend/qt/kernelmanager.py

With this in mind, what would be needed is an object that exposes the
kernelmanager interface but runs in-process instead of out of process,
and is otherwise the same.  Then, we could compose for Qt two KM's:
one based on an out-of-process, zmq-based class (what we have today)
and another one based on such an in-process KM that doesn't use zmq.

In this manner, the code above would function identically, modulo some
modifications so it doesn't mention external kernels on shutdown and
somesuch.

Let us know if this makes sense, I think it would be great to get
something like this implemented in full.

> Keep up the good work on IPython development!

Thanks for the kind words!   We hope to have you on board for the
project.  The new architecture is actually what we think is a fairly
good and clean design, so contributions like what you're discussing
would be great to have.

Regards,

f


From fperez.net at gmail.com  Sun Feb 13 20:01:31 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Feb 2011 17:01:31 -0800
Subject: [IPython-dev] Outstanding pull requests
In-Reply-To: <AANLkTikvzZzsZEWyaJumic0pXbs9prd_eHoVC1QJq3mi@mail.gmail.com>
References: <AANLkTikvzZzsZEWyaJumic0pXbs9prd_eHoVC1QJq3mi@mail.gmail.com>
Message-ID: <AANLkTimuMcAjontWN8CcArCfF1PAgGuaKzs7OVwC0era@mail.gmail.com>

Hi Thomas,

I just wanted to thank you *enormously* for doing this.  I have to
switch gears out of ipython right now for some deadlines, but later in
the week I'll return to this, and I'll try to flush a few of the easy
ones out of the way.  Some comments below.

On Sun, Feb 13, 2011 at 9:45 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> By way of my debut in the core team, I've prepared a list of all the open
> pull requests, with a very brief assessment of each, in the hope that we can
> start sorting them out (we're currently at 31 open requests):
>
> Magic functions documentation (several branches from SciPy India):
> [The first two listed here seem to have changed various bits of indentation
> beyond the specific things they're described as changing. This makes the
> diffs rather long. Do we want to copy the key changes out in new commits,
> using the --author option to credit them?]

Good idea.  These docstring requests were done by a number of teams at
scipy india who were at the very beginning of their python/programming
experience, so they are more likely to have beginner's problems.

It's a real bummer that I dropped the ball in providing them with
early feedback, but I had a ton of travel right after and just got
overrun with life...

So definitely crediting with --author but cleaning up the commit is a
good strategy for those with unnecessary noise.

> - cd: looks like some examples are at the bash prompt, not the ipython
> prompt. https://github.com/ipython/ipython/pull/233/files
> - profile: Has neatly documented the Python profile module. Unfortunately,
> IPython's %profile command does something entirely different.
> https://github.com/ipython/ipython/pull/232/files
> - logstart: Seems OK, although there's an "examples" title added for logstop
> with no actual examples. logstop probably doesn't need an example, but we
> should tidy up the "examples" title.
> https://github.com/ipython/ipython/pull/231/files
> - edit: Clean diff, but giving an example for edit is tricky, because you
> can't show the interactive editor. Also restates the -p option in a comment
> in the example. https://github.com/ipython/ipython/pull/230/files (closed
> 220 as duplicate)
> - pdef (it's confusing to use magic functions as the arguments for the
> example, we should have a user defined function) & colors (one example is
> probably enough, and the last two are wrong, using %usage instead of
> %colors) https://github.com/ipython/ipython/pull/229/files
> - who, whos, who_ls: Looks good. https://github.com/ipython/ipython/pull/228
> - logon: Not quite sure what's happened here. Example seems to be a bunch of
> Java-ish code. It probably needs to be redone from scratch.
> https://github.com/ipython/ipython/pull/227
> - pwd: pwd example is good, but has also added to exit (doesn't really need
> an example), dirs (example doesn't really show what it's for), and color
> (just an "examples" heading, no actual examples.
> https://github.com/ipython/ipython/pull/226/files
> - save: Clean diff, but the example and explanation aren't very clear.
> Probably needs to be redone from scratch.
> https://github.com/ipython/ipython/pull/225
> - cpaste: Appears to have ripped out the actual function while writing the
> docs, and stitched its definition onto the magic_paste function code.
> Example could also be a bit clearer.
> https://github.com/ipython/ipython/pull/224
> - colors: Same changes (& problems) as request 229.
> https://github.com/ipython/ipython/pull/223
> - timeit: Too many equivalent examples.
> https://github.com/ipython/ipython/pull/222/files
> - alias: Already had examples, changes remove one of them. Adds two, but one
> (ls) is available without aliasing, and the other is inaccurate (the example
> shows the result of a man command).
> https://github.com/ipython/ipython/pull/221
> - pycat: Doesn't actually use a Python file, so rather defeats the point of
> pycat. Not sure that an example is really helpful, as it doesn't show the
> colouring you get at a terminal. https://github.com/ipython/ipython/pull/219
> - hist: Formatting is off (probably related to a bug in the code). Also, the
> "this is an example of..." line seems redundant.
> https://github.com/ipython/ipython/pull/218/files
>
> Whew. Of these, I recommend merging 228 (%who), and closing 225 (%save), 232
> (%profile) with "try again" and 227 (%logon), for which I'm not sure that
> it's even possible to do an example.
>
> The others will need further work. Fernando, any sort of feel for how many
> of the SciPy India contributors will still be in the loop to revise their
> contributions?

Thanks for going through all of this!  Unfortunately due to the delay,
and the fact that many of them were first-timers to pretty much
everything, I'm not too hopeful.  I think if there are some where we
can easily finish up what they did with minimal work, it would be a
nice gesture.  But those that require significant rework, just
dropping a comment would be sufficient.  If they do pop up again and
come back with improvements, then we can engage them better for the
future.

In my defense, I was trying to handle a sprint where 60! people wanted
to contribute, most of which had never coded a single line of Python.
So I tried to do something where at least they would learn during the
day, even if it wasn't realistic to expect actual final contributions
to the codebase to come from everyone.

> Looking at the others:
>
> Small pull requests:
> - %hist output format: I've solved the same problem in a different way in
> pull request 261, see below. https://github.com/ipython/ipython/pull/236
> - Logging unicode characters (0.10 series): Good start. See my comment there
> for minor concern. https://github.com/ipython/ipython/pull/249
> - check_gtk: Removes a function. Mark Voorhies has tested the change and
> commented that it works. I'll admit to not understanding what it's about.
> https://github.com/ipython/ipython/pull/237
> - Typos in zmq/blockingkernelmanager.py: Seems OK to me.
> https://github.com/ipython/ipython/pull/203/files
> - cProfiling ipython.py: Updated in response to Brian's concerns. Needs
> reviewing again. https://github.com/ipython/ipython/pull/193
> - Autocomplete in Emacs: Removes the all_completions function (which
> apparently was broken). One other person says it works. Needs review.
> https://github.com/ipython/ipython/pull/204
> - Unicode issues [My changes]: Tests now integrated into main test suite
> (thanks to help from Robert Kern). Needs review.
> https://github.com/ipython/ipython/pull/252
> - Wildcard module [My changes]: Fixes issue 129. The bug and the fix are
> pretty much self contained, so shouldn't interfere with changes in other
> code. No feedback yet. (this is pushing what I'd call small, but shouldn't
> change functionality besides fixing the bug)
> https://github.com/ipython/ipython/pull/251
>
> Medium pull requests:
> - Fixing up various magic commands with the new history system [My changes]:
> Revised after Fernando's comments, unit tests added. Conflicts with
> Madhusudan's #203, see below. https://github.com/ipython/ipython/pull/261
> - %hist command bugs: This solves one of the same problems I've fixed in
> #261, in a different way. https://github.com/ipython/ipython/pull/203/files
> - Unbundling external libraries: Revised in response to Fernando's and
> Brian's concerns. Needs reviewing again.
> https://github.com/ipython/ipython/pull/191
> - Pyside compatibility: Waiting for contributor to resolve bug with SVG
> handling. https://github.com/ipython/ipython/pull/259
> - Kernel logging: Fernando has requested a final check from Robert.
> https://github.com/ipython/ipython/pull/264
>
> Larger pull requests:
> - Magic arguments decorators. Updated in response to Fernando's comments.
> Needs reviewing again. https://github.com/ipython/ipython/pull/199
>
> Not quite yet:
> - HTML Notebook: Gained a lot of interest, but not quite ready yet. Possible
> project for GSoC this year. (But if we're going to accumulate alternative
> frontends, should we consider putting each frontend in its own repo?)
> https://github.com/ipython/ipython/pull/179
> - MinRK's newparallel: Work in progress. Lots of
> work.https://github.com/ipython/ipython/pull/254

Ok, thanks again for this analysis.

Over the next few days, I suggest that anyone willing to 'take
ownership' of one just pitch in and add a comment to the corresponding
one, mentioning they'll keep an eye on it.  That way, we can split
this load and claw our way back to a small number of pending ones.

Best,

f


From fperez.net at gmail.com  Mon Feb 14 03:36:19 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 14 Feb 2011 00:36:19 -0800
Subject: [IPython-dev] Configuring logging for the kernel?
Message-ID: <AANLkTi=MThqCbUzV9jSiYoamdkeVDRPS12KH+rfKuafg@mail.gmail.com>

Howdy,

I've merged:

https://github.com/ipython/ipython/pull/264

after Omar took Robert's feedback into account.  But I'm not very
experienced with the logging module, so I'm wondering what the best
pattern should be for enabling logging conveniently at the
command-line level.  Right now, applications like the Qt console have
to pass flags to the kernel (and there's some duplication there we
should remove, like the code in ipythonqt.py and entry_point that
creates the parsers).  We can pass a --log-level flag to the kernel
entry point script easily, that's not a problem.

My question has to do with how to cleanly and conveniently let the
kernel log to the screen or a file, keeping in mind that the kernel
will be running in a separate process.  The default approach to
configuring logging for libraries is to let the application using a
library set its own handlers, but in this case the app (say a Qt
console) is calling the kernel in another process, so it can't pass it
any handlers; in fact it can't pass the kernel anything other than
command-line flags.

The code path is somewhat elaborate, because options have to pass from
the user-facing client through the kernel manager and ultimately to
the actual kernel object (in ipkernel.py).  I admit I played with it
for a while and got a bit lost, so I'm looking for some ideas on this
one.  In addition, we need to keep in mind that all of the Qt console
code is still in need of a Configurable restructuring, but that can be
handled separately from this.

If anyone for now needs to re-activate message printing to debug
low-level kernel behavior, this snippet in ipkernel.py, below the
logger definition line, will do:

logger.addHandler(logging.StreamHandler())
logger.setLevel(logging.DEBUG)

But what I want to do is come up with a cleaner solution for the long run...

Cheers,

f


From robert.kern at gmail.com  Mon Feb 14 12:23:55 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 14 Feb 2011 11:23:55 -0600
Subject: [IPython-dev] Configuring logging for the kernel?
In-Reply-To: <AANLkTi=MThqCbUzV9jSiYoamdkeVDRPS12KH+rfKuafg@mail.gmail.com>
References: <AANLkTi=MThqCbUzV9jSiYoamdkeVDRPS12KH+rfKuafg@mail.gmail.com>
Message-ID: <ijbofb$6do$1@dough.gmane.org>

On 2/14/11 2:36 AM, Fernando Perez wrote:
> Howdy,
>
> I've merged:
>
> https://github.com/ipython/ipython/pull/264
>
> after Omar took Robert's feedback into account.  But I'm not very
> experienced with the logging module, so I'm wondering what the best
> pattern should be for enabling logging conveniently at the
> command-line level.  Right now, applications like the Qt console have
> to pass flags to the kernel (and there's some duplication there we
> should remove, like the code in ipythonqt.py and entry_point that
> creates the parsers).  We can pass a --log-level flag to the kernel
> entry point script easily, that's not a problem.
>
> My question has to do with how to cleanly and conveniently let the
> kernel log to the screen or a file, keeping in mind that the kernel
> will be running in a separate process.  The default approach to
> configuring logging for libraries is to let the application using a
> library set its own handlers, but in this case the app (say a Qt
> console) is calling the kernel in another process, so it can't pass it
> any handlers; in fact it can't pass the kernel anything other than
> command-line flags.
>
> The code path is somewhat elaborate, because options have to pass from
> the user-facing client through the kernel manager and ultimately to
> the actual kernel object (in ipkernel.py).  I admit I played with it
> for a while and got a bit lost, so I'm looking for some ideas on this
> one.  In addition, we need to keep in mind that all of the Qt console
> code is still in need of a Configurable restructuring, but that can be
> handled separately from this.

For now, just add command line arguments enough to use logging.basicConfig() in 
the kernel process, and don't worry about any more complicated logging 
scenarios. Just the filename/stream and level arguments are sufficient, I think. 
Untested, but probably sufficient:

   parser.add_argument('--log-file', default='stderr',
     help='The file to send logging information to [default: stderr]')
   parser.add_argument('--log-level', default='ERROR',
     choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'],
     help='The log level [default: ERROR]')

   args = parser.parse_args()
   kwds = {}
   do_logging = True
   if args.log_file.lower() == 'none':
     do_logging = False
   elif args.log_file == 'stdout':
     kwds['stream'] = sys.stdout
   elif args.log_file == 'stderr':
     kwds['stream'] = sys.stderr
   else:
     kwds['filename'] = args.log_file
   if do_logging:
     kwds['level'] = getattr(logging, args.log_level)
     logging.basicConfig(**kwds)
   else:
     root_logger = logging.getLogger()
     class NullHandler(logging.Handler):
       def emit(self, record):
         pass
     root_logger.addHandler(NullHandler())

The more general solution would be to make logging Configurable and allow the 
passing of a config file via the command line parameters.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From fperez.net at gmail.com  Mon Feb 14 13:36:40 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 14 Feb 2011 10:36:40 -0800
Subject: [IPython-dev] Configuring logging for the kernel?
In-Reply-To: <ijbofb$6do$1@dough.gmane.org>
References: <AANLkTi=MThqCbUzV9jSiYoamdkeVDRPS12KH+rfKuafg@mail.gmail.com>
	<ijbofb$6do$1@dough.gmane.org>
Message-ID: <AANLkTi=DO-2C2Tf4CMmbYKkOrC5w3MOfphHKbGEcGEpj@mail.gmail.com>

Hi Robert,

On Mon, Feb 14, 2011 at 9:23 AM, Robert Kern <robert.kern at gmail.com> wrote:
> For now, just add command line arguments enough to use logging.basicConfig() in
> the kernel process, and don't worry about any more complicated logging
> scenarios. Just the filename/stream and level arguments are sufficient, I think.
> Untested, but probably sufficient:

OK, many thanks.  That's along the lines of what I had, I was just a
little tired to finish it up and decided to get some feedback before
putting more time into it, in case I was off-base.  Indeed, we'll want
to make it properly configurable later on, but for now something like
this should suffice.

If nobody beats me to it with a coded pull request, I'll try to work
on it in the next couple of days.

Cheers,

f


From benjaminrk at gmail.com  Mon Feb 14 16:32:03 2011
From: benjaminrk at gmail.com (MinRK)
Date: Mon, 14 Feb 2011 13:32:03 -0800
Subject: [IPython-dev] Outstanding pull requests
In-Reply-To: <AANLkTimuMcAjontWN8CcArCfF1PAgGuaKzs7OVwC0era@mail.gmail.com>
References: <AANLkTikvzZzsZEWyaJumic0pXbs9prd_eHoVC1QJq3mi@mail.gmail.com>
	<AANLkTimuMcAjontWN8CcArCfF1PAgGuaKzs7OVwC0era@mail.gmail.com>
Message-ID: <AANLkTi=QDvCmRtLTGyJJTPf0wu3LJQSPtnNEtvwHjNJ-@mail.gmail.com>

Thanks for all this great work.  I'll pitch in and merge some of these.

I just have a clarifying question for direct-commit/pull-request policy.

For instance, I was looking at %who, and the docstrings look great, but I
noticed that the behavior in the code is actually wrong:

            out = [i for i in out if type(i).__name__ in typeset]

should be:

            out = [i for i in out if type(user_ns[i]).__name__ in typeset]

(since `out` is the list of *names*, not variables, so it behaved as if
everything had type 'str').  Which, first of all, is excellent, since
docstrings should help us find bugs like this one has, but here's my
question:

What is the policy on tiny bugs like this?  Should I do a one-line
pull-request, or just push a fix directly to master (with test, of course)?

-MinRK

On Sun, Feb 13, 2011 at 17:01, Fernando Perez <fperez.net at gmail.com> wrote:

> Hi Thomas,
>
> I just wanted to thank you *enormously* for doing this.  I have to
> switch gears out of ipython right now for some deadlines, but later in
> the week I'll return to this, and I'll try to flush a few of the easy
> ones out of the way.  Some comments below.
>
> On Sun, Feb 13, 2011 at 9:45 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> > By way of my debut in the core team, I've prepared a list of all the open
> > pull requests, with a very brief assessment of each, in the hope that we
> can
> > start sorting them out (we're currently at 31 open requests):
> >
> > Magic functions documentation (several branches from SciPy India):
> > [The first two listed here seem to have changed various bits of
> indentation
> > beyond the specific things they're described as changing. This makes the
> > diffs rather long. Do we want to copy the key changes out in new commits,
> > using the --author option to credit them?]
>
> Good idea.  These docstring requests were done by a number of teams at
> scipy india who were at the very beginning of their python/programming
> experience, so they are more likely to have beginner's problems.
>
> It's a real bummer that I dropped the ball in providing them with
> early feedback, but I had a ton of travel right after and just got
> overrun with life...
>
> So definitely crediting with --author but cleaning up the commit is a
> good strategy for those with unnecessary noise.
>
> > - cd: looks like some examples are at the bash prompt, not the ipython
> > prompt. https://github.com/ipython/ipython/pull/233/files
> > - profile: Has neatly documented the Python profile module.
> Unfortunately,
> > IPython's %profile command does something entirely different.
> > https://github.com/ipython/ipython/pull/232/files
> > - logstart: Seems OK, although there's an "examples" title added for
> logstop
> > with no actual examples. logstop probably doesn't need an example, but we
> > should tidy up the "examples" title.
> > https://github.com/ipython/ipython/pull/231/files
> > - edit: Clean diff, but giving an example for edit is tricky, because you
> > can't show the interactive editor. Also restates the -p option in a
> comment
> > in the example. https://github.com/ipython/ipython/pull/230/files(closed
> > 220 as duplicate)
> > - pdef (it's confusing to use magic functions as the arguments for the
> > example, we should have a user defined function) & colors (one example is
> > probably enough, and the last two are wrong, using %usage instead of
> > %colors) https://github.com/ipython/ipython/pull/229/files
> > - who, whos, who_ls: Looks good.
> https://github.com/ipython/ipython/pull/228
> > - logon: Not quite sure what's happened here. Example seems to be a bunch
> of
> > Java-ish code. It probably needs to be redone from scratch.
> > https://github.com/ipython/ipython/pull/227
> > - pwd: pwd example is good, but has also added to exit (doesn't really
> need
> > an example), dirs (example doesn't really show what it's for), and color
> > (just an "examples" heading, no actual examples.
> > https://github.com/ipython/ipython/pull/226/files
> > - save: Clean diff, but the example and explanation aren't very clear.
> > Probably needs to be redone from scratch.
> > https://github.com/ipython/ipython/pull/225
> > - cpaste: Appears to have ripped out the actual function while writing
> the
> > docs, and stitched its definition onto the magic_paste function code.
> > Example could also be a bit clearer.
> > https://github.com/ipython/ipython/pull/224
> > - colors: Same changes (& problems) as request 229.
> > https://github.com/ipython/ipython/pull/223
> > - timeit: Too many equivalent examples.
> > https://github.com/ipython/ipython/pull/222/files
> > - alias: Already had examples, changes remove one of them. Adds two, but
> one
> > (ls) is available without aliasing, and the other is inaccurate (the
> example
> > shows the result of a man command).
> > https://github.com/ipython/ipython/pull/221
> > - pycat: Doesn't actually use a Python file, so rather defeats the point
> of
> > pycat. Not sure that an example is really helpful, as it doesn't show the
> > colouring you get at a terminal.
> https://github.com/ipython/ipython/pull/219
> > - hist: Formatting is off (probably related to a bug in the code). Also,
> the
> > "this is an example of..." line seems redundant.
> > https://github.com/ipython/ipython/pull/218/files
> >
> > Whew. Of these, I recommend merging 228 (%who), and closing 225 (%save),
> 232
> > (%profile) with "try again" and 227 (%logon), for which I'm not sure that
> > it's even possible to do an example.
> >
> > The others will need further work. Fernando, any sort of feel for how
> many
> > of the SciPy India contributors will still be in the loop to revise their
> > contributions?
>
> Thanks for going through all of this!  Unfortunately due to the delay,
> and the fact that many of them were first-timers to pretty much
> everything, I'm not too hopeful.  I think if there are some where we
> can easily finish up what they did with minimal work, it would be a
> nice gesture.  But those that require significant rework, just
> dropping a comment would be sufficient.  If they do pop up again and
> come back with improvements, then we can engage them better for the
> future.
>
> In my defense, I was trying to handle a sprint where 60! people wanted
> to contribute, most of which had never coded a single line of Python.
> So I tried to do something where at least they would learn during the
> day, even if it wasn't realistic to expect actual final contributions
> to the codebase to come from everyone.
>
> > Looking at the others:
> >
> > Small pull requests:
> > - %hist output format: I've solved the same problem in a different way in
> > pull request 261, see below. https://github.com/ipython/ipython/pull/236
> > - Logging unicode characters (0.10 series): Good start. See my comment
> there
> > for minor concern. https://github.com/ipython/ipython/pull/249
> > - check_gtk: Removes a function. Mark Voorhies has tested the change and
> > commented that it works. I'll admit to not understanding what it's about.
> > https://github.com/ipython/ipython/pull/237
> > - Typos in zmq/blockingkernelmanager.py: Seems OK to me.
> > https://github.com/ipython/ipython/pull/203/files
> > - cProfiling ipython.py: Updated in response to Brian's concerns. Needs
> > reviewing again. https://github.com/ipython/ipython/pull/193
> > - Autocomplete in Emacs: Removes the all_completions function (which
> > apparently was broken). One other person says it works. Needs review.
> > https://github.com/ipython/ipython/pull/204
> > - Unicode issues [My changes]: Tests now integrated into main test suite
> > (thanks to help from Robert Kern). Needs review.
> > https://github.com/ipython/ipython/pull/252
> > - Wildcard module [My changes]: Fixes issue 129. The bug and the fix are
> > pretty much self contained, so shouldn't interfere with changes in other
> > code. No feedback yet. (this is pushing what I'd call small, but
> shouldn't
> > change functionality besides fixing the bug)
> > https://github.com/ipython/ipython/pull/251
> >
> > Medium pull requests:
> > - Fixing up various magic commands with the new history system [My
> changes]:
> > Revised after Fernando's comments, unit tests added. Conflicts with
> > Madhusudan's #203, see below.
> https://github.com/ipython/ipython/pull/261
> > - %hist command bugs: This solves one of the same problems I've fixed in
> > #261, in a different way.
> https://github.com/ipython/ipython/pull/203/files
> > - Unbundling external libraries: Revised in response to Fernando's and
> > Brian's concerns. Needs reviewing again.
> > https://github.com/ipython/ipython/pull/191
> > - Pyside compatibility: Waiting for contributor to resolve bug with SVG
> > handling. https://github.com/ipython/ipython/pull/259
> > - Kernel logging: Fernando has requested a final check from Robert.
> > https://github.com/ipython/ipython/pull/264
> >
> > Larger pull requests:
> > - Magic arguments decorators. Updated in response to Fernando's comments.
> > Needs reviewing again. https://github.com/ipython/ipython/pull/199
> >
> > Not quite yet:
> > - HTML Notebook: Gained a lot of interest, but not quite ready yet.
> Possible
> > project for GSoC this year. (But if we're going to accumulate alternative
> > frontends, should we consider putting each frontend in its own repo?)
> > https://github.com/ipython/ipython/pull/179
> > - MinRK's newparallel: Work in progress. Lots of
> > work.https://github.com/ipython/ipython/pull/254
>
> Ok, thanks again for this analysis.
>
> Over the next few days, I suggest that anyone willing to 'take
> ownership' of one just pitch in and add a comment to the corresponding
> one, mentioning they'll keep an eye on it.  That way, we can split
> this load and claw our way back to a small number of pending ones.
>
> Best,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110214/6175806b/attachment.html>

From fperez.net at gmail.com  Mon Feb 14 17:53:15 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 14 Feb 2011 14:53:15 -0800
Subject: [IPython-dev] Outstanding pull requests
In-Reply-To: <AANLkTi=QDvCmRtLTGyJJTPf0wu3LJQSPtnNEtvwHjNJ-@mail.gmail.com>
References: <AANLkTikvzZzsZEWyaJumic0pXbs9prd_eHoVC1QJq3mi@mail.gmail.com>
	<AANLkTimuMcAjontWN8CcArCfF1PAgGuaKzs7OVwC0era@mail.gmail.com>
	<AANLkTi=QDvCmRtLTGyJJTPf0wu3LJQSPtnNEtvwHjNJ-@mail.gmail.com>
Message-ID: <AANLkTi=ZYoPv2e8FdBm063DVAYLyvUiD+HmEnE1Mn8Ni@mail.gmail.com>

On Mon, Feb 14, 2011 at 1:32 PM, MinRK <benjaminrk at gmail.com> wrote:
> Thanks for all this great work. ?I'll pitch in and merge some of these.

Many thanks!

> I just have a clarifying question for direct-commit/pull-request policy.
> For instance, I was looking at %who, and the docstrings look great, but I
> noticed that the behavior in the code is actually wrong:
> ?? ??? ? ? ?out = [i for i in out if type(i).__name__ in typeset]
> should be:
> ?? ? ? ? ? ?out = [i for i in out if type(user_ns[i]).__name__ in typeset]
> (since `out` is the list of names, not variables, so it behaved as if
> everything had type 'str'). ?Which, first of all, is excellent, since
> docstrings should help us find bugs like this one has, but here's my
> question:
> What is the policy on tiny bugs like this? ?Should I do a one-line
> pull-request, or just push a fix directly to master (with test, of course)?

I've always felt that a certain level of trust, and willingness to fix
little slip-ups without getting upset if/when they happen, is the best
policy.  Core developers should feel free to make small merges like
these when they are confident that there's no danger.

I realize that it's always possible to make a mistake and to have a
small change that inadvertently introduces an error.  But I also
deeply dislike 'zero tolerance' style approaches to anything, where
human thought and judgment are eliminated in favor of blind policy.

Especially since it's true that even *with* review we can still make
mistakes (and that's OK, we'll just fix them), I don't want the idea
of review to become a straitjacket that forces every last action in
the project to require six rubber stamps.  Since an absolutist policy
would be just the *illusion* of 'no mistakes', we might as well live
with the reality and find an approach that balances things to maximize
the efficiency, fluidity and the fun of working on the project for all
involved [1].

Paraphrasing the famous quote on simplicity, we need just enough
process and policy to help things run smoothly, but no more :)

So my take is: use your judgment, and when you are confident that a
*small* fix is obvious, otherwise innocuous, and that making it
directly will just save everyone time for more complex reviews, go
ahead.  When in doubt for any reason, do ask for review.  And needless
to say, large or complex changes of any kind, or even small ones that
entail important design decisions, should always be put up for review.

Cheers,

f

[1] The fun part *is* important.  As a largely volunteer effort, we
need to ensure that people *enjoy* working on IPython (myself included
:).


From takowl at gmail.com  Mon Feb 14 18:14:23 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 14 Feb 2011 23:14:23 +0000
Subject: [IPython-dev] Outstanding pull requests
In-Reply-To: <AANLkTi=ZYoPv2e8FdBm063DVAYLyvUiD+HmEnE1Mn8Ni@mail.gmail.com>
References: <AANLkTikvzZzsZEWyaJumic0pXbs9prd_eHoVC1QJq3mi@mail.gmail.com>
	<AANLkTimuMcAjontWN8CcArCfF1PAgGuaKzs7OVwC0era@mail.gmail.com>
	<AANLkTi=QDvCmRtLTGyJJTPf0wu3LJQSPtnNEtvwHjNJ-@mail.gmail.com>
	<AANLkTi=ZYoPv2e8FdBm063DVAYLyvUiD+HmEnE1Mn8Ni@mail.gmail.com>
Message-ID: <AANLkTi=UO=nzBbbzE4GhHvrfJ10LitNcPqiC=jgBSA3_@mail.gmail.com>

On 14 February 2011 22:53, Fernando Perez <fperez.net at gmail.com> wrote:

> I've always felt that a certain level of trust, and willingness to fix
> little slip-ups without getting upset if/when they happen, is the best
> policy.  Core developers should feel free to make small merges like
> these when they are confident that there's no danger.
>

I'll second that. Even in purely utilitarian terms, the time spent finding
and undoing the occasional mistaken commit is bound to be far less than the
time saved by not having to wait for a seal of approval on every little
change.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110214/39b5e736/attachment.html>

From gokhansever at gmail.com  Tue Feb 15 13:15:13 2011
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=)
Date: Tue, 15 Feb 2011 11:15:13 -0700
Subject: [IPython-dev] Link broken on github main-page
Message-ID: <AANLkTin+_1dj02bVZw3yMmq60FsTHCnfFWO1chU_J7gJ@mail.gmail.com>

Hi,

The link is broken on the line starting with "website"...

Overview
========

Welcome to IPython. Our full documentation can be found in the ``docs/dist``
subdirectory in ``.html`` and ``.pdf`` formats, also available online at our
`website <http://ipython.scipy.org&gt;`_
<http://ipython.scipy.org%3E%60_/>.  The ``docs/source`` directory
contains
the plaintext version of these manuals.


None of the documentation seems updated to reflect github transition (from
http://ipython.github.com/ipython-doc/)

Is this still in so experimental stage?

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110215/9c7fc903/attachment.html>

From benjaminrk at gmail.com  Tue Feb 15 13:38:33 2011
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 15 Feb 2011 10:38:33 -0800
Subject: [IPython-dev] Link broken on github main-page
In-Reply-To: <AANLkTin+_1dj02bVZw3yMmq60FsTHCnfFWO1chU_J7gJ@mail.gmail.com>
References: <AANLkTin+_1dj02bVZw3yMmq60FsTHCnfFWO1chU_J7gJ@mail.gmail.com>
Message-ID: <AANLkTimQJ3ozavwhRe97p9dr+4rO4EBcvtNeau5cR0kK@mail.gmail.com>

The link is broken because GitHub doesn't support using '.txt' as a reST
extension, so it's interpreting the wrong text as a link.  We should
probably rename README.txt to README.rst, and link to the new doc page.

Is there a reason to *not* use .rst instead of .txt?

Thanks for the note.

-MinRK

On Tue, Feb 15, 2011 at 10:15, G?khan Sever <gokhansever at gmail.com> wrote:

> Hi,
>
> The link is broken on the line starting with "website"...
>
>
> Overview
> ========
>
> Welcome to IPython. Our full documentation can be found in the ``docs/dist``
> subdirectory in ``.html`` and ``.pdf`` formats, also available online at our
> `website <http://ipython.scipy.org&gt;`_ <http://ipython.scipy.org%3E%60_/>.  The ``docs/source`` directory contains
> the plaintext version of these manuals.
>
>
> None of the documentation seems updated to reflect github transition (from
> http://ipython.github.com/ipython-doc/)
>
> Is this still in so experimental stage?
>
> --
> G?khan
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110215/69b6a591/attachment.html>

From takowl at gmail.com  Tue Feb 15 14:41:53 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 15 Feb 2011 19:41:53 +0000
Subject: [IPython-dev] Examples for magic commands
Message-ID: <AANLkTimONbhgkrUmk3rWCUQ=-h4qfXFYtR_jB84CjT6v@mail.gmail.com>

OK, we've now got through most of the magic command examples from SciPy
India. Min merged one, I've amalgamated several into pull request 272, and
closed a few more. There are three left:

#219 (pycat) and #230 (edit): I don't think an example adds much to either
of these. The first case involves colour, which we can't put in the
docstrings. The second revolves around an editor which won't be in the
example.

#221 has several problems, and there were already some examples included in
it, so I think it's OK without any changes.

Should I go ahead and close those? Or do people think those commands would
benefit from examples?

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110215/49f7dede/attachment.html>

From benjaminrk at gmail.com  Tue Feb 15 15:05:35 2011
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 15 Feb 2011 12:05:35 -0800
Subject: [IPython-dev] Examples for magic commands
In-Reply-To: <AANLkTimONbhgkrUmk3rWCUQ=-h4qfXFYtR_jB84CjT6v@mail.gmail.com>
References: <AANLkTimONbhgkrUmk3rWCUQ=-h4qfXFYtR_jB84CjT6v@mail.gmail.com>
Message-ID: <AANLkTikGAziSeV2BhV2GOj2vJ3+9yoMssKE=Bq2MXPDc@mail.gmail.com>

On Tue, Feb 15, 2011 at 11:41, Thomas Kluyver <takowl at gmail.com> wrote:

> OK, we've now got through most of the magic command examples from SciPy
> India. Min merged one, I've amalgamated several into pull request 272, and
> closed a few more. There are three left:
>
> #219 (pycat) and #230 (edit): I don't think an example adds much to either
> of these. The first case involves colour, which we can't put in the
> docstrings. The second revolves around an editor which won't be in the
> example.
>
> #221 has several problems, and there were already some examples included in
> it, so I think it's OK without any changes.
>
> Should I go ahead and close those? Or do people think those commands would
> benefit from examples?
>

I'm okay on all of those, but I don't know about other people.

Thanks for doing all of this, you are a hero!

-MinRK



>
> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110215/69e57ab4/attachment.html>

From ellisonbg at gmail.com  Tue Feb 15 18:18:02 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 15 Feb 2011 15:18:02 -0800
Subject: [IPython-dev] Examples for magic commands
In-Reply-To: <AANLkTimONbhgkrUmk3rWCUQ=-h4qfXFYtR_jB84CjT6v@mail.gmail.com>
References: <AANLkTimONbhgkrUmk3rWCUQ=-h4qfXFYtR_jB84CjT6v@mail.gmail.com>
Message-ID: <AANLkTikkrR2+AT63RjaH=Dp7JzeEUeHZdbhVDwAb=7At@mail.gmail.com>

Thomas,

Thanks for your work on this.

> #219 (pycat) and #230 (edit): I don't think an example adds much to either
> of these. The first case involves colour, which we can't put in the
> docstrings. The second revolves around an editor which won't be in the
> example.
>
> #221 has several problems, and there were already some examples included in
> it, so I think it's OK without any changes.
>
> Should I go ahead and close those? Or do people think those commands would
> benefit from examples?

I am fine with just closing these.

Cheers,

Brian

> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From takowl at gmail.com  Tue Feb 15 18:57:09 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 15 Feb 2011 23:57:09 +0000
Subject: [IPython-dev] pyzmq 2.0.10.1 available in unstable
In-Reply-To: <AANLkTinKS6FT_f3PF95ibeRK=jW3e+A3S3Y6Uo2OoaHm@mail.gmail.com>
References: <AANLkTinKS6FT_f3PF95ibeRK=jW3e+A3S3Y6Uo2OoaHm@mail.gmail.com>
Message-ID: <AANLkTikPqwdacm_3tpvBPt8OVk-4NbZ4HWDOG6y05R4D@mail.gmail.com>

Thanks, Miguel.

On 15 February 2011 23:19, Miguel Landaeta <miguel at miguel.cc> wrote:

> Hello Thomas,
>
> FYI, pyzmq 2.0.10.1 has reached unstable.
>
> http://packages.debian.org/unstable/python-zmq
>
> Cheers,
>
> --
> Miguel Landaeta, miguel at miguel.cc
> secure email with PGP 0x7D8967E9 available at http://keyserver.pgp.com/
> "Faith means not wanting to know what is true." -- Nietzsche
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110215/dacccb37/attachment.html>

From ellisonbg at gmail.com  Tue Feb 15 22:48:11 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 15 Feb 2011 19:48:11 -0800
Subject: [IPython-dev] My schedule for the next week
Message-ID: <AANLkTikzOOjnkvAKM9AHv=wT_+LBVcdZkXf1JnvUWtRz@mail.gmail.com>

Hi,

I am excited to see all the activity going on.  I wanted to let you
all know that I will be mostly occupied with a conference until mid
next week.  I am going to try to catch up on things while I am
travelling, but I may be on/off a lot of the next week.

Cheers,

Brian

-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Wed Feb 16 01:52:38 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 15 Feb 2011 22:52:38 -0800
Subject: [IPython-dev] My schedule for the next week
In-Reply-To: <AANLkTikzOOjnkvAKM9AHv=wT_+LBVcdZkXf1JnvUWtRz@mail.gmail.com>
References: <AANLkTikzOOjnkvAKM9AHv=wT_+LBVcdZkXf1JnvUWtRz@mail.gmail.com>
Message-ID: <AANLkTinRPW3pBh6WatB8nfZ3DA1mvVe-3===_FO7fUjq@mail.gmail.com>

Hi everyone,

On Tue, Feb 15, 2011 at 7:48 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>
>
> I am excited to see all the activity going on. ?I wanted to let you
> all know that I will be mostly occupied with a conference until mid
> next week. ?I am going to try to catch up on things while I am
> travelling, but I may be on/off a lot of the next week.

and in a perfect storm of bad timing, I'm in the exact same boat, off
to the East Coast for the same period to a (different) conference :)

I'll do my best to catch up on the list and github, but will likely
fall behind, sorry...

Cheers,

f


From fperez.net at gmail.com  Wed Feb 16 03:02:47 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 16 Feb 2011 00:02:47 -0800
Subject: [IPython-dev] Examples for magic commands
In-Reply-To: <AANLkTikGAziSeV2BhV2GOj2vJ3+9yoMssKE=Bq2MXPDc@mail.gmail.com>
References: <AANLkTimONbhgkrUmk3rWCUQ=-h4qfXFYtR_jB84CjT6v@mail.gmail.com>
	<AANLkTikGAziSeV2BhV2GOj2vJ3+9yoMssKE=Bq2MXPDc@mail.gmail.com>
Message-ID: <AANLkTi=k03Ru7h_VYp+FOjp8VAUG8rp8Wqxho7uMJXAd@mail.gmail.com>

On Tue, Feb 15, 2011 at 12:05 PM, MinRK <benjaminrk at gmail.com> wrote:
>
> Thanks for doing all of this, you are a hero!

Indeed!


From takowl at gmail.com  Wed Feb 16 17:10:27 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 16 Feb 2011 22:10:27 +0000
Subject: [IPython-dev] History
Message-ID: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>

I've been having a think about how we store history:

At present: Commands entered are stored in two lists (raw and "translated" -
i.e. turning magic commands into function calls). These are persisted to
disk at the next input every time 60 seconds elapse, with storage in a JSON
file, which is reloaded into the same lists (and into readline history) when
starting IPython. Each command entered is also immediately persisted in the
"shadow history", a collection of files in .ipython managed by the
pickleshare DB. The output objects are also stored in a dictionary by prompt
number, but I'm less concerned with that here.

Uses:
- Readline history (getting previous commands via up arrow)
- Various magic commands (save, macro, hist) can access ranges of input,
using the prompt numbers from the current session.
- %hist -g allows searching shadow and current history with glob syntax.
- %rep can access ranges of this session's history, or single lines from
shadow history.
(These are all I've found so far - please let me know if there are others)

I'd like to propose simplifying this somewhat. Specifically, I think it
makes sense to store history on disk just once. I suggest an SQLite
database. SQLite has been in the standard library since 2.5, is lightweight,
simple and widely used (e.g. Firefox's history). More specifically, I
envisage:
- We keep the session lists of raw and translated history as they are.
- Each command is stored instantly, so we do away with the need for an
autosave timer thread. A crash at any stage should leave your entire history
intact up to the last command completed.
- We store only raw history on disk (I think raw history is what we're
looking for 90% of the time). If we want translated history, we redo the
translation on the fly (this should require minimal computation, unless I've
missed something).
- History is indexed by session number and prompt number. This provides a
sensible behaviour if we have two IPython shells open together - the second
one to be opened will be the latter session (and will be able to access
commands entered in the other session as soon as they are completed).
- For magic commands, accessing a line from a previous session could look
like "-1#9" (9th line of immediately previous session).
- On starting IPython, we load the last (~40 lines/~2 sessions) from the
database into readline history.

Thoughts? Have I overlooked some key reason we use the existing system? Is
there a better alternative to SQLite? Would you design it differently? I've
not written any code for this yet, so I'm open to ideas. But if people think
that makes sense, I'm volunteering to make it happen.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110216/71895949/attachment.html>

From fperez.net at gmail.com  Wed Feb 16 22:07:17 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 16 Feb 2011 19:07:17 -0800
Subject: [IPython-dev] Examples for magic commands
In-Reply-To: <AANLkTikkrR2+AT63RjaH=Dp7JzeEUeHZdbhVDwAb=7At@mail.gmail.com>
References: <AANLkTimONbhgkrUmk3rWCUQ=-h4qfXFYtR_jB84CjT6v@mail.gmail.com>
	<AANLkTikkrR2+AT63RjaH=Dp7JzeEUeHZdbhVDwAb=7At@mail.gmail.com>
Message-ID: <4D5C90E5.1010302@gmail.com>

On 02/15/2011 03:18 PM, Brian Granger wrote:
>> >  Should I go ahead and close those? Or do people think those commands would
>> >  benefit from examples?
> I am fine with just closing these.

No objection from me on closing them either.  It would be great to drop 
them a short word of thanks and why it's being closed, since these are 
first-time contributors and even if their work wasn't accepted we want 
to be kind to newbies...

And indeed, many many thanks Thomas for this rather heroic push to flush 
our accumulation of requests.  Way to land in the project with a splash :)

Best,

f


From fperez.net at gmail.com  Thu Feb 17 00:54:04 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 16 Feb 2011 21:54:04 -0800
Subject: [IPython-dev] Link broken on github main-page
In-Reply-To: <AANLkTimQJ3ozavwhRe97p9dr+4rO4EBcvtNeau5cR0kK@mail.gmail.com>
References: <AANLkTin+_1dj02bVZw3yMmq60FsTHCnfFWO1chU_J7gJ@mail.gmail.com>
	<AANLkTimQJ3ozavwhRe97p9dr+4rO4EBcvtNeau5cR0kK@mail.gmail.com>
Message-ID: <AANLkTi=HGE8TLJCoiMTXCMQ+67ctsj07Z-TnWp6cmvkF@mail.gmail.com>

On Tue, Feb 15, 2011 at 10:38 AM, MinRK <benjaminrk at gmail.com> wrote:
> The link is broken because GitHub doesn't support using '.txt' as a reST
> extension, so it's interpreting the wrong text as a link. ?We should
> probably rename README.txt to README.rst, and link to the new doc page.
> Is there a reason to *not* use .rst instead of .txt?

Well, docutils does suggest using txt as the reST extension, their
argument being that they ultimately are text files.  And on more
gui-oriented platforms, it was deemed as nicer for new users who could
double-click on a .txt file and get automatically a text editor
instead of a dialog about what application to use.  So historically,
we went with txt.

But I have no strong feelings on the matter, so if you guys prefer to
switch over to rst so the github setup is smoother, I won't oppose it
in any way.

Cheers,

f


From benjaminrk at gmail.com  Thu Feb 17 01:14:48 2011
From: benjaminrk at gmail.com (MinRK)
Date: Wed, 16 Feb 2011 22:14:48 -0800
Subject: [IPython-dev] Link broken on github main-page
In-Reply-To: <AANLkTi=HGE8TLJCoiMTXCMQ+67ctsj07Z-TnWp6cmvkF@mail.gmail.com>
References: <AANLkTin+_1dj02bVZw3yMmq60FsTHCnfFWO1chU_J7gJ@mail.gmail.com>
	<AANLkTimQJ3ozavwhRe97p9dr+4rO4EBcvtNeau5cR0kK@mail.gmail.com>
	<AANLkTi=HGE8TLJCoiMTXCMQ+67ctsj07Z-TnWp6cmvkF@mail.gmail.com>
Message-ID: <AANLkTinsmowDOrB+qMNfwMMOstxbhU_vHHTx1=f6m7wR@mail.gmail.com>

On Wed, Feb 16, 2011 at 21:54, Fernando Perez <fperez.net at gmail.com> wrote:

> On Tue, Feb 15, 2011 at 10:38 AM, MinRK <benjaminrk at gmail.com> wrote:
> > The link is broken because GitHub doesn't support using '.txt' as a reST
> > extension, so it's interpreting the wrong text as a link.  We should
> > probably rename README.txt to README.rst, and link to the new doc page.
> > Is there a reason to *not* use .rst instead of .txt?
>
> Well, docutils does suggest using txt as the reST extension, their
> argument being that they ultimately are text files.  And on more
> gui-oriented platforms, it was deemed as nicer for new users who could
> double-click on a .txt file and get automatically a text editor
> instead of a dialog about what application to use.  So historically,
> we went with txt.
>

That makes sense.  Basic reST with headers, sections, emphasis and blocks is
perfectly readable
as nicely formatted plaintext.  However, to me it seems like if you want a
file to read
as plaintext as well as reST, you shouldn't use the markup that is less
readable like `link text <http://hiddenlink.com>`_,
and you can use .rst as an extension when you do use the extended reST bits.

My vote:

either a) remove the hidden links for better .txt readability (and correct
GH link parsing)
or b) rename to .rst so it looks nice on GitHub

I lean towards b), but if there really is a significant number of people who
actually read READMEs but are foiled by a .rst extension, then we can stick
to a).



>
> But I have no strong feelings on the matter, so if you guys prefer to
> switch over to rst so the github setup is smoother, I won't oppose it
> in any way.
>
> Cheers,
>
> f
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110216/48734a34/attachment.html>

From fperez.net at gmail.com  Thu Feb 17 01:22:55 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 16 Feb 2011 22:22:55 -0800
Subject: [IPython-dev] Link broken on github main-page
In-Reply-To: <AANLkTinsmowDOrB+qMNfwMMOstxbhU_vHHTx1=f6m7wR@mail.gmail.com>
References: <AANLkTin+_1dj02bVZw3yMmq60FsTHCnfFWO1chU_J7gJ@mail.gmail.com>
	<AANLkTimQJ3ozavwhRe97p9dr+4rO4EBcvtNeau5cR0kK@mail.gmail.com>
	<AANLkTi=HGE8TLJCoiMTXCMQ+67ctsj07Z-TnWp6cmvkF@mail.gmail.com>
	<AANLkTinsmowDOrB+qMNfwMMOstxbhU_vHHTx1=f6m7wR@mail.gmail.com>
Message-ID: <AANLkTinrP+Ugm9opfwK-ynsPyEXynCP1gzfKo_EAYRTU@mail.gmail.com>

On Wed, Feb 16, 2011 at 10:14 PM, MinRK <benjaminrk at gmail.com> wrote:
>
> I lean towards b), but if there really is a significant number of people who
> actually read READMEs but are foiled by a .rst extension, then we can stick
> to a).

I'm OK with b) too...

f


From takowl at gmail.com  Thu Feb 17 05:45:30 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 17 Feb 2011 10:45:30 +0000
Subject: [IPython-dev] Examples for magic commands
In-Reply-To: <4D5C90E5.1010302@gmail.com>
References: <AANLkTimONbhgkrUmk3rWCUQ=-h4qfXFYtR_jB84CjT6v@mail.gmail.com>
	<AANLkTikkrR2+AT63RjaH=Dp7JzeEUeHZdbhVDwAb=7At@mail.gmail.com>
	<4D5C90E5.1010302@gmail.com>
Message-ID: <AANLkTimrWH-Xa5A8Afw=r3NRGxNQK_0B=VrxVhhnYUov@mail.gmail.com>

On 17 February 2011 03:07, Fernando Perez <fperez.net at gmail.com> wrote:

>  It would be great to drop
> them a short word of thanks and why it's being closed, since these are
> first-time contributors and even if their work wasn't accepted we want
> to be kind to newbies...
>

I tried to do something along these lines with each one I closed. I hope it
sounded sincere - I found it a bit tricky to convey that we're grateful for
their interest in contributing, even though we don't want the specific
changes.

I should add that what you did at SciPy India was an impressive feat: you
got a large group of entirely new contributors, had them download the
development codebase, make changes, and submit those changes as pull
requests that, although not perfect, mostly made sense. Hopefully it's a
good indication that there's relatively little technical friction for new
contributors.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110217/3c781473/attachment.html>

From hans_meine at gmx.net  Thu Feb 17 12:53:59 2011
From: hans_meine at gmx.net (Hans Meine)
Date: Thu, 17 Feb 2011 18:53:59 +0100
Subject: [IPython-dev] History
In-Reply-To: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
Message-ID: <201102171853.59724.hans_meine@gmx.net>

Am Mittwoch, 16. Februar 2011, um 23:10:27 schrieb Thomas Kluyver:
> I'd like to propose simplifying this somewhat. Specifically, I think it
> makes sense to store history on disk just once. I suggest an SQLite
> database.

+0.9
(it will become a little bit harder to directly access the history, compared 
to the plain text files at least)

> - We keep the session lists of raw and translated history as they are.
> - Each command is stored instantly, so we do away with the need for an
> autosave timer thread. A crash at any stage should leave your entire
> history intact up to the last command completed.

Why "completed"?  I would assume that a crashing command will still appear in 
the DB?!

> - We store only raw history on disk (I think raw history is what we're
> looking for 90% of the time). If we want translated history, we redo the
> translation on the fly (this should require minimal computation, unless
> I've missed something).

Not sure about this one - sounds fragile, I guess there was a reason to store 
the processed commands; what about processing which depends on currently 
active aliases etc.?  My fear is that this will become difficult to recompute.

OTOH, most of the time, the two commands are absolutely equal.  What about two 
DB columns, but leaving the processed command empty if it is equal to the raw 
command?  This could save a lot of space, and it would be trivial to create a 
view which provides the two full columns.

> - History is indexed by session number and prompt number.

Sounds sensible.

> This provides a
> sensible behaviour if we have two IPython shells open together - the second
> one to be opened will be the latter session (and will be able to access
> commands entered in the other session as soon as they are completed).

I would still expect only local commands to appear in the normal readline 
history, and in the default %history output.  IMO there should be optional 
modes (something along %history --all) which list the other shells' history.

> - For magic commands, accessing a line from a previous session could look
> like "-1#9" (9th line of immediately previous session).

Hmm, what about concurrent sessions?  (Versus "previous" sessions which have 
been quit.)

> - On starting IPython, we load the last (~40 lines/~2 sessions) from the
> database into readline history.

Are you kidding me?  One session equals roughly 1400 lines for me. :-)

That's a good question, how much history should be available from "previous" 
sessions.

> Thoughts?

You wrote "will be able to access commands entered in the other session as 
soon as they are completed" - does this require ipython to constantly read 
from the DB?  No, obviously there should be a history API that directly 
operates on the database.

Only for readline, we would be reading some lines from the DB initially, 
right?

HTH,
  Hans


From takowl at gmail.com  Thu Feb 17 13:46:52 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 17 Feb 2011 18:46:52 +0000
Subject: [IPython-dev] History
In-Reply-To: <201102171853.59724.hans_meine@gmx.net>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<201102171853.59724.hans_meine@gmx.net>
Message-ID: <AANLkTim9YJnz+JQ_O+mgh3X4Ot8ynT4EEMHsqo12xKPH@mail.gmail.com>

On 17 February 2011 17:53, Hans Meine <hans_meine at gmx.net> wrote:

> +0.9
> (it will become a little bit harder to directly access the history,
> compared
> to the plain text files at least)
>

True, but I don't think that's a common use case (correct me if I'm wrong).
And there are tools to access SQLite databases if you need to, e.g.
http://sqlitebrowser.sourceforge.net/

 Why "completed"?  I would assume that a crashing command will still appear
> in
> the DB?!
>

My mistake. For some reason, I thought that input was stored in history
after execution. Checking the code, it's stored before, so a crashing
command would be saved. That should have been obvious.

 Not sure about this one - sounds fragile, I guess there was a reason to
> store
> the processed commands; what about processing which depends on currently
> active aliases etc.?  My fear is that this will become difficult to
> recompute.


> OTOH, most of the time, the two commands are absolutely equal.  What about
> two
> DB columns, but leaving the processed command empty if it is equal to the
> raw
> command?  This could save a lot of space, and it would be trivial to create
> a
> view which provides the two full columns.
>

Good point, I hadn't thought of temporary aliases (I've never used them);
and good solution. While we're on the subject, what does translation
actually do? %magic commands, !system commands, aliases; anything else?


> I would still expect only local commands to appear in the normal readline
> history, and in the default %history output.  IMO there should be optional
> modes (something along %history --all) which list the other shells'
> history.
>

Yes, that's what I'd envisage. At present, %history -g is used for a glob
search over all the history; passing it no pattern is the way to list
history from all sessions.

The readline history wouldn't be updated from other sessions - so commands
you enter in shell 1 before starting shell 2 will be loaded into readline
for shell 2, but then only shell 2 commands would be added to that. Starting
a third shell would not add anything to readline in shell 2.

 Hmm, what about concurrent sessions?  (Versus "previous" sessions which
> have
> been quit.)
>

Interesting idea, but more complicated to implement. You'd have to somehow
have separate shells signal their presence. I think this is a separate
consideration - I'm looking at what we can get for free by restructuring the
history.

 Are you kidding me?  One session equals roughly 1400 lines for me. :-)
>
> That's a good question, how much history should be available from
> "previous"
> sessions.
>

But how much of that do you access through readline? It can of course be
user configurable, but we should set a sensible default. 40 is on the low
side. I assume readline is fairly fast: maybe a couple of hundred is a good
limit?


> You wrote "will be able to access commands entered in the other session as
> soon as they are completed" - does this require ipython to constantly read
> from the DB?  No, obviously there should be a history API that directly
> operates on the database.
>

Sorry, I should have been clearer about what I meant by access. Those
commands will be in the database immediately, and accessible with magic
commands (e.g. "%rep -1#9" or "%hist -g"), which will call the history API
to read the database. Other than those commands, IPython would only read the
database on startup, to push lines into readline.

Thanks for the feedback,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110217/e4a9bcfd/attachment.html>

From benjaminrk at gmail.com  Thu Feb 17 14:31:38 2011
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 17 Feb 2011 11:31:38 -0800
Subject: [IPython-dev] History
In-Reply-To: <AANLkTim9YJnz+JQ_O+mgh3X4Ot8ynT4EEMHsqo12xKPH@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<201102171853.59724.hans_meine@gmx.net>
	<AANLkTim9YJnz+JQ_O+mgh3X4Ot8ynT4EEMHsqo12xKPH@mail.gmail.com>
Message-ID: <AANLkTin4QT0v7j7GVn=eV2Be_bRy0EBHZ57pXzt1xk3r@mail.gmail.com>

Hi,

I like this idea.  It's a bit unfortunate that we are starting this
discussion while Brian and Fernando are at conferences for a week, but we
have good ideas when we have them.  It's possible there will be an objection
based on ideas of non-Python interfaces, but I think those are generally
implemented via history_request messages, which would still work and thus
not be a problem.

On Thu, Feb 17, 2011 at 10:46, Thomas Kluyver <takowl at gmail.com> wrote:

> On 17 February 2011 17:53, Hans Meine <hans_meine at gmx.net> wrote:
>
>> +0.9
>> (it will become a little bit harder to directly access the history,
>> compared
>> to the plain text files at least)
>>
>
> True, but I don't think that's a common use case (correct me if I'm wrong).
> And there are tools to access SQLite databases if you need to, e.g.
> http://sqlitebrowser.sourceforge.net/
>
>  Why "completed"?  I would assume that a crashing command will still appear
>> in
>> the DB?!
>>
>
> My mistake. For some reason, I thought that input was stored in history
> after execution. Checking the code, it's stored before, so a crashing
> command would be saved. That should have been obvious.
>
>  Not sure about this one - sounds fragile, I guess there was a reason to
>> store
>> the processed commands; what about processing which depends on currently
>> active aliases etc.?  My fear is that this will become difficult to
>> recompute.
>
>
>> OTOH, most of the time, the two commands are absolutely equal.  What about
>> two
>> DB columns, but leaving the processed command empty if it is equal to the
>> raw
>> command?  This could save a lot of space, and it would be trivial to
>> create a
>> view which provides the two full columns.
>>
>
I don't think there's any real penalty to storing both (other than doubling
the size of a very small file), so might as well.


>
>
Good point, I hadn't thought of temporary aliases (I've never used them);
> and good solution. While we're on the subject, what does translation
> actually do? %magic commands, !system commands, aliases; anything else?
>

>
>> I would still expect only local commands to appear in the normal readline
>> history, and in the default %history output.  IMO there should be optional
>> modes (something along %history --all) which list the other shells'
>> history.
>>
>
> Yes, that's what I'd envisage. At present, %history -g is used for a glob
> search over all the history; passing it no pattern is the way to list
> history from all sessions.
>
> The readline history wouldn't be updated from other sessions - so commands
> you enter in shell 1 before starting shell 2 will be loaded into readline
> for shell 2, but then only shell 2 commands would be added to that. Starting
> a third shell would not add anything to readline in shell 2.
>
>  Hmm, what about concurrent sessions?  (Versus "previous" sessions which
>> have
>> been quit.)
>>
>
> Interesting idea, but more complicated to implement. You'd have to somehow
> have separate shells signal their presence. I think this is a separate
> consideration - I'm looking at what we can get for free by restructuring the
> history.
>

I think the question here might be: what would the history look like at
startup if I've been running several sessions simultaneously, and I start a
new one?

<start IP0,IP1,IP2>
IP0: foo='bar'
IP1: a=1
IP2: b=2
IP1: c=3
<close IP0>
IP2: d=4
<start IP3>

i.e. what is the 'previous' session?  Is it 'most recently closed' or 'most
recently started' or 'most recently executed'?  You seemed to suggest above
that IP3's readline history at startup should go d,c,b,a,foo (as I continue
to press up) Is that correct?  I like this approach, but there's also value
in keeping related commands (same session) adjacent to each other, so I
don't think it's an obvious choice.  In most real cases, this probably
doesn't
make much difference.



>
>  Are you kidding me?  One session equals roughly 1400 lines for me. :-)
>>
>> That's a good question, how much history should be available from
>> "previous"
>> sessions.
>>
>
> But how much of that do you access through readline? It can of course be
> user configurable, but we should set a sensible default. 40 is on the low
> side. I assume readline is fairly fast: maybe a couple of hundred is a good
> limit?
>

I would think a default should be something like 500 at the very least.  I
use ctrl-r all the time, and basically expect that if I remember typing it,
IPython should remember as well.  If you are just using arrow-keys, then
more than a few dozen isn't particularly valuable, but reverse-i-search
should find anything I remember well enough to look for.

-MinRK


>
>
>> You wrote "will be able to access commands entered in the other session as
>> soon as they are completed" - does this require ipython to constantly read
>> from the DB?  No, obviously there should be a history API that directly
>> operates on the database.
>>
>
> Sorry, I should have been clearer about what I meant by access. Those
> commands will be in the database immediately, and accessible with magic
> commands (e.g. "%rep -1#9" or "%hist -g"), which will call the history API
> to read the database. Other than those commands, IPython would only read the
> database on startup, to push lines into readline.
>
> Thanks for the feedback,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110217/ed9ca518/attachment.html>

From satra at mit.edu  Thu Feb 17 14:36:59 2011
From: satra at mit.edu (Satrajit Ghosh)
Date: Thu, 17 Feb 2011 14:36:59 -0500
Subject: [IPython-dev] History
In-Reply-To: <AANLkTim9YJnz+JQ_O+mgh3X4Ot8ynT4EEMHsqo12xKPH@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<201102171853.59724.hans_meine@gmx.net>
	<AANLkTim9YJnz+JQ_O+mgh3X4Ot8ynT4EEMHsqo12xKPH@mail.gmail.com>
Message-ID: <AANLkTi=1qrdFQKpgjNO==DjsOLZ3webwopZ1TbDAkHdJ@mail.gmail.com>

hi thomas and hans,

just a few thoughts.

(it will become a little bit harder to directly access the history, compared
>> to the plain text files at least)
>>
>
> True, but I don't think that's a common use case (correct me if I'm wrong).
> And there are tools to access SQLite databases if you need to, e.g.
> http://sqlitebrowser.sourceforge.net/
>

history is currently in json files and yes it does get long, but reading
json is now part of the standard library. we can definitely save space by
replacing identical commands between raw and processed with only one copy.
when fernando and i worked on this in december, we increased the save
history limit in the config.


>
>
>> Hmm, what about concurrent sessions?  (Versus "previous" sessions which
>> have
>> been quit.)
>>
>
> Interesting idea, but more complicated to implement. You'd have to somehow
> have separate shells signal their presence. I think this is a separate
> consideration - I'm looking at what we can get for free by restructuring the
> history.
>

we do have concurrent history as long as the clients connect to the same
kernel. this is in fact what we implemented between the qt client and the
terminal client or combinations thereof. but this will be different from
using separate kernels.


>
>  Are you kidding me?  One session equals roughly 1400 lines for me. :-)
>>
>> That's a good question, how much history should be available from
>> "previous"
>> sessions.
>>
>
> But how much of that do you access through readline? It can of course be
> user configurable, but we should set a sensible default. 40 is on the low
> side. I assume readline is fairly fast: maybe a couple of hundred is a good
> limit?
>

are we really in memory trouble these days? or is this more a question of
load time?

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110217/ca28df88/attachment.html>

From hans_meine at gmx.net  Fri Feb 18 08:06:33 2011
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 18 Feb 2011 14:06:33 +0100
Subject: [IPython-dev] History
In-Reply-To: <AANLkTi=1qrdFQKpgjNO==DjsOLZ3webwopZ1TbDAkHdJ@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTim9YJnz+JQ_O+mgh3X4Ot8ynT4EEMHsqo12xKPH@mail.gmail.com>
	<AANLkTi=1qrdFQKpgjNO==DjsOLZ3webwopZ1TbDAkHdJ@mail.gmail.com>
Message-ID: <201102181406.33551.hans_meine@gmx.net>

Am Donnerstag, 17. Februar 2011, um 20:36:59 schrieb Satrajit Ghosh:
> are we really in memory trouble these days? or is this more a question of
> load time?

As anecdotal evidence, I once had the problem that it took ages for new (bash) 
shells to start up (sth. > 15s).  It took me quite some debugging (strace 
etc.) to understand that the problem was in fact the readline history, not 
(directly) my startup files.

I like having a history limit of 150,000 commands.  Why not?  Same as MinRK: 
As long as I dimly remember that I solved a particular problem on the 
commandline in the middle of last year, why should my computer have forgotten 
about it?  (Yes, privacy issues might arise, but there are countermeasures 
here, too.)

But I could very well live with magic commands for this use case, i.e. with 
limited readline history.

Am Donnerstag, 17. Februar 2011, um 20:31:38 schrieb MinRK:
> I don't think there's any real penalty to storing both (other than doubling
> the size of a very small file), so might as well.

Here, you're assuming that the file is "very small", while AFAICS it might be 
at least some megabytes.  So why have this redundancy?

Ciao,
  Hans


From takowl at gmail.com  Fri Feb 18 08:37:06 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 18 Feb 2011 13:37:06 +0000
Subject: [IPython-dev] History
In-Reply-To: <201102181406.33551.hans_meine@gmx.net>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTim9YJnz+JQ_O+mgh3X4Ot8ynT4EEMHsqo12xKPH@mail.gmail.com>
	<AANLkTi=1qrdFQKpgjNO==DjsOLZ3webwopZ1TbDAkHdJ@mail.gmail.com>
	<201102181406.33551.hans_meine@gmx.net>
Message-ID: <AANLkTinS=L3qVwLXx6wg5PQ8FAv68C9kSoTheqtGATNA@mail.gmail.com>

On 18 February 2011 13:06, Hans Meine <hans_meine at gmx.net> wrote:

> I like having a history limit of 150,000 commands.  Why not?  Same as
> MinRK:
> As long as I dimly remember that I solved a particular problem on the
> commandline in the middle of last year, why should my computer have
> forgotten
> about it?  (Yes, privacy issues might arise, but there are countermeasures
> here, too.)
>
> But I could very well live with magic commands for this use case, i.e. with
> limited readline history.
>

I'll ensure that this can all be tuned through a config file. I've gone with
loading the last 1000 into readline history for now, and I've not yet
implemented any limit on the database size.


> Here, you're assuming that the file is "very small", while AFAICS it might
> be
> at least some megabytes.  So why have this redundancy?
>

I tested it briefly yesterday evening. After about 140 commands, storing
both raw and translated all the time, the database was about 14kB; i.e.
about 100 bytes per line. If we assume the lines in my test were half what's
typical, 150,000 commands would be 30MB. In disk space terms, that's not
worth worrying about (unless we do IPython for embedded systems ;-)).

Here's the code:
https://github.com/takluyver/ipython/tree/sqlite-history
It's still very much a work in progress - I haven't hooked up all the magic
commands to access previous sessions. But it writes to the database, reloads
lines into readline (in the order I described), and searching the database
with "%hist -g thing" works. I'll make a pull request once it's a bit more
complete.

As an aside, there are a few odds and ends left using the pickleshare DB -
on my system, there's still syscmdlist and dhist (directory history). I've
got nothing against pickleshare - it seems to work perfectly well - but is
it worth looking at simplifying those odds and ends of storage? It looks
like it's using pickle to dump to/load from file: is it simpler to call
pickle directly?

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110218/cafe7af5/attachment.html>

From fperez.net at gmail.com  Fri Feb 18 10:48:56 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 18 Feb 2011 10:48:56 -0500
Subject: [IPython-dev] History
In-Reply-To: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
Message-ID: <AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>

Hi all,

sorry that this will be brief and not very thought-through, but I can
only sneak in short periods while at the conference...

Many thanks to Thomas for getting this work going!  But I think, since
we now have a bit more manpower and good momentum going, it's worth
thinking a little about the key points we want to hit so we end up
with something really solid.  Comments below...

On Wed, Feb 16, 2011 at 5:10 PM, Thomas Kluyver <takowl at gmail.com> wrote:

> - Each command is stored instantly, so we do away with the need for an
> autosave timer thread. A crash at any stage should leave your entire history
> intact up to the last command completed.

Instant saving has one problem: frequent disk usage prevents hard
drives from spinnning down when on battery.  The idea of an auto-save
thread on a timer with a user-controllable delay has the advantage
that the user can control their power consumption profile to fit their
needs.

On an international flight when you're trying to squeeze every last
bit of your battery, this matters a lot.  We don't want to turn
ipython into the thing that eats up your battery life to death just by
virtue of running very simple interactive commands that in principle
are purely CPU/memory resident, but because we generate lots of
sideband disk activity.

So we should keep this consideration in mind in the design.  If we
don't think about it now, it will be much harder to retrofit a decent
power profile later on.

> - We store only raw history on disk (I think raw history is what we're
> looking for 90% of the time). If we want translated history, we redo the
> translation on the fly (this should require minimal computation, unless I've
> missed something).
> I've been having a think about how we store history:
>
> At present: Commands entered are stored in two lists (raw and "translated" -
> i.e. turning magic commands into function calls). These are persisted to
> disk at the next input every time 60 seconds elapse, with storage in a JSON
> file, which is reloaded into the same lists (and into readline history) when
> starting IPython. Each command entered is also immediately persisted in the
> "shadow history", a collection of files in .ipython managed by the
> pickleshare DB. The output objects are also stored in a dictionary by prompt
> number, but I'm less concerned with that here.
>
> Uses:
> - Readline history (getting previous commands via up arrow)
> - Various magic commands (save, macro, hist) can access ranges of input,
> using the prompt numbers from the current session.
> - %hist -g allows searching shadow and current history with glob syntax.
> - %rep can access ranges of this session's history, or single lines from
> shadow history.
> (These are all I've found so far - please let me know if there are others)

No, we must store the translated history for two reasons:

1. Some translations are dynamic and context-dependent, so they can
not be recomputed later (though these are the minority).

2. More importantly, the translation process is relatively cpu
intensive, while disk space is the absolutely cheapest resource in
existence (at least at the data storage volumes we're talking about
here).  So it makes sense to store on disk these results once we have
computed them, rather than recomputing them all over later on reload.

> - History is indexed by session number and prompt number. This provides a
> sensible behaviour if we have two IPython shells open together - the second
> one to be opened will be the latter session (and will be able to access
> commands entered in the other session as soon as they are completed).
> - For magic commands, accessing a line from a previous session could look
> like "-1#9" (9th line of immediately previous session).
> - On starting IPython, we load the last (~40 lines/~2 sessions) from the
> database into readline history.
>
> Thoughts? Have I overlooked some key reason we use the existing system? Is
> there a better alternative to SQLite? Would you design it differently? I've
> not written any code for this yet, so I'm open to ideas. But if people think
> that makes sense, I'm volunteering to make it happen.

Finally, but importantly, I'm somewhat reluctant to go to sqlite until
we've fully shown that accomplishing our design goals is
hard/impossible with a simple json persisted data structure.  While
sqlite is indeed lightweight and available to us, it's also a more
complex api to program against than simply storing a list/dict on
disk.  I'm OK accepting that complexity price *if we need it*, but I'm
not convinced we do yet.  Specifically, we need to answer: what is
precisely the functionality that we want, that is hard/impossible to
implement on json and easy/possible with sqlite?

Cheers,

f


From fperez.net at gmail.com  Fri Feb 18 11:06:26 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 18 Feb 2011 11:06:26 -0500
Subject: [IPython-dev] History
In-Reply-To: <AANLkTim9YJnz+JQ_O+mgh3X4Ot8ynT4EEMHsqo12xKPH@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<201102171853.59724.hans_meine@gmx.net>
	<AANLkTim9YJnz+JQ_O+mgh3X4Ot8ynT4EEMHsqo12xKPH@mail.gmail.com>
Message-ID: <AANLkTikPjMc+qS3Kfen5z61pfsWJ9BrV6UZfP+MeZZpo@mail.gmail.com>

Hi Tomas,

On Thu, Feb 17, 2011 at 1:46 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> Good point, I hadn't thought of temporary aliases (I've never used them);
> and good solution. While we're on the subject, what does translation
> actually do? %magic commands, !system commands, aliases; anything else?

Our input processing has been fairly cleaned up, but it could still
use some more refactoring to put it all in one place so it's easier to
understand.  There are two parts to it: a) 'static' transformations
that can be applied to any input string without knowing about the
user's state/namespace.  b) dynamic ones that are only applied to
single-line input, and that are context-dependent.

The code that does the input processing of type (a) is here:

https://github.com/ipython/ipython/blob/master/IPython/core/inputsplitter.py

And the single-line prefiltering of type (b) is here:

https://github.com/ipython/ipython/blob/master/IPython/core/prefilter.py

(b) is only called in one place, by the run_single_line method:

https://github.com/ipython/ipython/blob/master/IPython/core/interactiveshell.py#L2188

I hope this helps, keep asking  and we'll try to explain things out...

Cheers,

f


From ellisonbg at gmail.com  Fri Feb 18 12:01:59 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 18 Feb 2011 09:01:59 -0800
Subject: [IPython-dev] History
In-Reply-To: <AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
Message-ID: <AANLkTi=ADq3aZFuo_E-ddpzZ5R27zxNoo_NiFfkogNw9@mail.gmail.com>

I too am a bit hesitant to go the sqlite route on this without some
careful thinking.

Cheers,

Brian

On Fri, Feb 18, 2011 at 7:48 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi all,
>
> sorry that this will be brief and not very thought-through, but I can
> only sneak in short periods while at the conference...
>
> Many thanks to Thomas for getting this work going! ?But I think, since
> we now have a bit more manpower and good momentum going, it's worth
> thinking a little about the key points we want to hit so we end up
> with something really solid. ?Comments below...
>
> On Wed, Feb 16, 2011 at 5:10 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
>> - Each command is stored instantly, so we do away with the need for an
>> autosave timer thread. A crash at any stage should leave your entire history
>> intact up to the last command completed.
>
> Instant saving has one problem: frequent disk usage prevents hard
> drives from spinnning down when on battery. ?The idea of an auto-save
> thread on a timer with a user-controllable delay has the advantage
> that the user can control their power consumption profile to fit their
> needs.
>
> On an international flight when you're trying to squeeze every last
> bit of your battery, this matters a lot. ?We don't want to turn
> ipython into the thing that eats up your battery life to death just by
> virtue of running very simple interactive commands that in principle
> are purely CPU/memory resident, but because we generate lots of
> sideband disk activity.
>
> So we should keep this consideration in mind in the design. ?If we
> don't think about it now, it will be much harder to retrofit a decent
> power profile later on.
>
>> - We store only raw history on disk (I think raw history is what we're
>> looking for 90% of the time). If we want translated history, we redo the
>> translation on the fly (this should require minimal computation, unless I've
>> missed something).
>> I've been having a think about how we store history:
>>
>> At present: Commands entered are stored in two lists (raw and "translated" -
>> i.e. turning magic commands into function calls). These are persisted to
>> disk at the next input every time 60 seconds elapse, with storage in a JSON
>> file, which is reloaded into the same lists (and into readline history) when
>> starting IPython. Each command entered is also immediately persisted in the
>> "shadow history", a collection of files in .ipython managed by the
>> pickleshare DB. The output objects are also stored in a dictionary by prompt
>> number, but I'm less concerned with that here.
>>
>> Uses:
>> - Readline history (getting previous commands via up arrow)
>> - Various magic commands (save, macro, hist) can access ranges of input,
>> using the prompt numbers from the current session.
>> - %hist -g allows searching shadow and current history with glob syntax.
>> - %rep can access ranges of this session's history, or single lines from
>> shadow history.
>> (These are all I've found so far - please let me know if there are others)
>
> No, we must store the translated history for two reasons:
>
> 1. Some translations are dynamic and context-dependent, so they can
> not be recomputed later (though these are the minority).
>
> 2. More importantly, the translation process is relatively cpu
> intensive, while disk space is the absolutely cheapest resource in
> existence (at least at the data storage volumes we're talking about
> here). ?So it makes sense to store on disk these results once we have
> computed them, rather than recomputing them all over later on reload.
>
>> - History is indexed by session number and prompt number. This provides a
>> sensible behaviour if we have two IPython shells open together - the second
>> one to be opened will be the latter session (and will be able to access
>> commands entered in the other session as soon as they are completed).
>> - For magic commands, accessing a line from a previous session could look
>> like "-1#9" (9th line of immediately previous session).
>> - On starting IPython, we load the last (~40 lines/~2 sessions) from the
>> database into readline history.
>>
>> Thoughts? Have I overlooked some key reason we use the existing system? Is
>> there a better alternative to SQLite? Would you design it differently? I've
>> not written any code for this yet, so I'm open to ideas. But if people think
>> that makes sense, I'm volunteering to make it happen.
>
> Finally, but importantly, I'm somewhat reluctant to go to sqlite until
> we've fully shown that accomplishing our design goals is
> hard/impossible with a simple json persisted data structure. ?While
> sqlite is indeed lightweight and available to us, it's also a more
> complex api to program against than simply storing a list/dict on
> disk. ?I'm OK accepting that complexity price *if we need it*, but I'm
> not convinced we do yet. ?Specifically, we need to answer: what is
> precisely the functionality that we want, that is hard/impossible to
> implement on json and easy/possible with sqlite?
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Fri Feb 18 11:54:40 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 18 Feb 2011 11:54:40 -0500
Subject: [IPython-dev] History
In-Reply-To: <AANLkTin4QT0v7j7GVn=eV2Be_bRy0EBHZ57pXzt1xk3r@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<201102171853.59724.hans_meine@gmx.net>
	<AANLkTim9YJnz+JQ_O+mgh3X4Ot8ynT4EEMHsqo12xKPH@mail.gmail.com>
	<AANLkTin4QT0v7j7GVn=eV2Be_bRy0EBHZ57pXzt1xk3r@mail.gmail.com>
Message-ID: <AANLkTinXOTsc7gDqDt0uu_d0Q3TJBUgCeu6-o+7QAMwv@mail.gmail.com>

Hi,

On Thu, Feb 17, 2011 at 2:31 PM, MinRK <benjaminrk at gmail.com> wrote:

>>> Hmm, what about concurrent sessions? ?(Versus "previous" sessions which
>>> have
>>> been quit.)
>>
>> Interesting idea, but more complicated to implement. You'd have to somehow
>> have separate shells signal their presence. I think this is a separate
>> consideration - I'm looking at what we can get for free by restructuring the
>> history.
>
> I think the question here might be: what would the history look like at
> startup if I've been running several sessions simultaneously, and I start a
> new one?
> <start IP0,IP1,IP2>
> IP0: foo='bar'
> IP1: a=1
> IP2: b=2
> IP1: c=3
> <close IP0>
> IP2: d=4
> <start IP3>
> i.e. what is the 'previous' session? ?Is it 'most recently closed' or 'most
> recently started' or 'most recently executed'? ?You seemed to suggest above
> that IP3's readline history at startup should go d,c,b,a,foo (as I continue
> to press up) Is that correct? ?I like this approach, but there's also value
> in keeping related commands (same session) adjacent to each other, so I
> don't think it's an obvious choice. ?In most real cases, this probably
> doesn't
> make much difference.

This is really the key problem to think about, and I'm not sure I have
a full solution even in my head yet.  For now, a few thoughts...

In a sense, the history problem can be seen similar to the evolution
of a git repository, where you start with the master branch (the
history of your first session ever).  If you close and open a single
session again (but no concurrent ones), then each of these operations
is akin to a new commit, just adding to the old history.  But if a
concurrent session is opened, this is like making a new branch.

However, from a user experience standpoint, we don't want to expose
this kind of complexity in everyday work: people shouldn't need to
think of doing git branch merges to manage their history :)

And in actual usage, you tend to think of your history as a linear
object running back in time.  But as we've seen clearly, it's not
linear, it's a DAG.  So the question is: how to best flatten it, so
that the user experience is automatic and shows a reasonable
linearization of the DAG, and yet it's useful and powerful (better
than today's behavior where a lot of clobbering happens from
concurrent sessions)?

If we start keeping all the sessions around, though, after a while it
will get very unwieldy to manage this, as we'll probably need to use
uuids to tag them, and tracking multiple sessions like this will be a
pain.  But we can't fully linearize on every save operation, because
we'd lose the ability to track open concurrent sessions.

I have to run to a talk soon... Just as a sketch of how it could work:
there's a notion of a master session (to borrow git's terminology
deliberately); on opening ipython, it marks it as in use (locks).  If
a second session is opened (it sees the active one as locked), a
branch is made, tagged with a uuid and timestamp.  On startup, if
there are branches available they get merged into the current view,
but grouped together with most recent available first.  And only up to
5 branches are allowed to be 'in flight', so that the system doesn't
grow too complicated.  We may want to think about a few commands for
users to manage this manually if they want...

Profiles could allow long-lived branches: there's a 'master'
per-profile, but otherwise the same logic as above applies.

Incomplete ideas as food for thought, but I have to run now...

>>> Are you kidding me? ?One session equals roughly 1400 lines for me. :-)
>>>
>>> That's a good question, how much history should be available from
>>> "previous"
>>> sessions.
>>
>> But how much of that do you access through readline? It can of course be
>> user configurable, but we should set a sensible default. 40 is on the low
>> side. I assume readline is fairly fast: maybe a couple of hundred is a good
>> limit?
>
> I would think a default should be something like 500 at the very least. ?I
> use ctrl-r all the time, and basically expect that if I remember typing it,
> IPython?should remember as well. ?If you are just using arrow-keys, then
> more than a few dozen isn't particularly valuable, but reverse-i-search
> should find anything I remember well enough to look for.

Yes, Ctrl-r should have O(1000) lines of history at least available to
it.  It's easy to use, fast, and very powerful.  Many people live and
die by it;l even if it's a somewhat more advanced readline feature,
for power users it's a very important one.

Cheers,

f


From robert.kern at gmail.com  Fri Feb 18 12:11:49 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 18 Feb 2011 11:11:49 -0600
Subject: [IPython-dev] History
In-Reply-To: <AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
Message-ID: <ijm98l$5h2$1@dough.gmane.org>

On 2/18/11 9:48 AM, Fernando Perez wrote:

> Finally, but importantly, I'm somewhat reluctant to go to sqlite until
> we've fully shown that accomplishing our design goals is
> hard/impossible with a simple json persisted data structure.  While
> sqlite is indeed lightweight and available to us, it's also a more
> complex api to program against than simply storing a list/dict on
> disk.  I'm OK accepting that complexity price *if we need it*, but I'm
> not convinced we do yet.  Specifically, we need to answer: what is
> precisely the functionality that we want, that is hard/impossible to
> implement on json and easy/possible with sqlite?

Well, if the computation resources devoted to translating raw input to their 
executable form is important, then I suspect the serialization to JSON of the 
whole history every time it gets saved would often dominate it (translation 
happens rarely; loading/saving happens often).

SQLite is safer.
SQLite allows concurrency.
SQLite allows us to store absolutely everything we have ever typed, but only 
load what we want to use in a session.
SQLite is more easily extended to also record outputs. You get a basic notebook 
format practically for free!  :-)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ellisonbg at gmail.com  Fri Feb 18 13:06:11 2011
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 18 Feb 2011 10:06:11 -0800
Subject: [IPython-dev] History
In-Reply-To: <ijm98l$5h2$1@dough.gmane.org>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
	<ijm98l$5h2$1@dough.gmane.org>
Message-ID: <AANLkTim13YPFGKeYzu599pPVP1bZJECYL+ijKx3Wb4Vr@mail.gmail.com>

On Fri, Feb 18, 2011 at 9:11 AM, Robert Kern <robert.kern at gmail.com> wrote:
> On 2/18/11 9:48 AM, Fernando Perez wrote:
>
>> Finally, but importantly, I'm somewhat reluctant to go to sqlite until
>> we've fully shown that accomplishing our design goals is
>> hard/impossible with a simple json persisted data structure. ?While
>> sqlite is indeed lightweight and available to us, it's also a more
>> complex api to program against than simply storing a list/dict on
>> disk. ?I'm OK accepting that complexity price *if we need it*, but I'm
>> not convinced we do yet. ?Specifically, we need to answer: what is
>> precisely the functionality that we want, that is hard/impossible to
>> implement on json and easy/possible with sqlite?
>
> Well, if the computation resources devoted to translating raw input to their
> executable form is important, then I suspect the serialization to JSON of the
> whole history every time it gets saved would often dominate it (translation
> happens rarely; loading/saving happens often).
>
> SQLite is safer.
> SQLite allows concurrency.
> SQLite allows us to store absolutely everything we have ever typed, but only
> load what we want to use in a session.

Absolutely, I think these points are *very* important and make sqlite
worth thinking about.

> SQLite is more easily extended to also record outputs. You get a basic notebook
> format practically for free! ?:-)

Some thoughts about this:

* The history is a chronological picture of the command that happen on
a particular engine session.
* The notebook is a "spatial" picture of a subset of commands that is
completely independent of a particular session.
* The history can contain things that are not in a notebook (from a
terminal IPython) and a notebook can contain things not in a history.
* In the past, we have envisioned the notebook server as being a
completely separate process/server than the kernel.  For examples, you
could imagine a centralized notebook server for a group that is used
by frontends/kernels running all over the world.
* Notebooks are inherently multiuser (notebooks in a notebook server
are associated with users, they can be shared, etc.).

But...it is an interesting idea to combine the kernel and notebook
server.  There could be some significant benefits for this, but it is
definitely not the model we have been thinking.  Any ideas about how
this would work?

One other thought:  possibly, something like the history storage
format should be abstracted through an API that we could implement
using sqlite/json/mongodb/etc.

Cheers,

Brian


> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
> ?that is made terrible by our own mad attempt to interpret it as though it had
> ?an underlying truth."
> ? -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From robert.kern at gmail.com  Fri Feb 18 13:44:39 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 18 Feb 2011 12:44:39 -0600
Subject: [IPython-dev] History
In-Reply-To: <AANLkTim13YPFGKeYzu599pPVP1bZJECYL+ijKx3Wb4Vr@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>	<ijm98l$5h2$1@dough.gmane.org>
	<AANLkTim13YPFGKeYzu599pPVP1bZJECYL+ijKx3Wb4Vr@mail.gmail.com>
Message-ID: <ijmemo$5gd$1@dough.gmane.org>

[OT: *please* stop Ccing me. I am on the list!]

On 2/18/11 12:06 PM, Brian Granger wrote:
> On Fri, Feb 18, 2011 at 9:11 AM, Robert Kern<robert.kern at gmail.com>  wrote:

>> SQLite is more easily extended to also record outputs. You get a basic notebook
>> format practically for free!  :-)
>
> Some thoughts about this:
>
> * The history is a chronological picture of the command that happen on
> a particular engine session.
> * The notebook is a "spatial" picture of a subset of commands that is
> completely independent of a particular session.
> * The history can contain things that are not in a notebook (from a
> terminal IPython) and a notebook can contain things not in a history.
> * In the past, we have envisioned the notebook server as being a
> completely separate process/server than the kernel.  For examples, you
> could imagine a centralized notebook server for a group that is used
> by frontends/kernels running all over the world.
> * Notebooks are inherently multiuser (notebooks in a notebook server
> are associated with users, they can be shared, etc.).
>
> But...it is an interesting idea to combine the kernel and notebook
> server.  There could be some significant benefits for this, but it is
> definitely not the model we have been thinking.  Any ideas about how
> this would work?

The only thing that is fundamentally "multiuser" about a notebook is that one 
usually makes them to share with people. That sharing can take many forms, is 
frequently just for them to view, and often the only "other person" is yourself, 
six months later. None of that necessitates (or is even facilitated) by a server 
model, so I think that is a very limiting view and substantial overhead. 
Swapping files has worked pretty well for Mathematica, the Ur-notebook.

To use a buzzword, a more "social" notebook interaction like you are talking 
about is something that's valuable and does require a server, but this is 
something built on top of the fundamental notebook idea; it's not the notebook 
itself. I think if you are chasing just the social features, you're missing out 
on a large segment of use cases that can't support the overhead of a server.

As my smiley and "basic" indicated, I don't think that it's really a notebook, 
but it is an extension feature that I've wanted for a long time: record 
everything I execute and everything that's output. Less of a Mathematica 
notebook than a lab notebook. Given that all-inclusive history, I can write 
tools that let me extract the stuff I have been doing now and make a proper 
notebook whenever you get around to implementing one. :-)

Mostly, I hate needing to make decisions before I start. Don't make me decide 
whether I want this session to be a notebook or not before I even begin typing.

> One other thought:  possibly, something like the history storage
> format should be abstracted through an API that we could implement
> using sqlite/json/mongodb/etc.

I think you just need to flesh out the HistoryManager interface a little. I 
don't think it's worth making another interface.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From takowl at gmail.com  Fri Feb 18 14:14:18 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 18 Feb 2011 19:14:18 +0000
Subject: [IPython-dev] History
In-Reply-To: <AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
Message-ID: <AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>

On 18 February 2011 15:48, Fernando Perez <fperez.net at gmail.com> wrote:

> Instant saving has one problem: frequent disk usage prevents hard
> drives from spinnning down when on battery.  The idea of an auto-save
> thread on a timer with a user-controllable delay has the advantage
> that the user can control their power consumption profile to fit their
> needs.
>

In the default configuration, I think we already have disk access on each
command as it's written to the shadow history. Admittedly that can be
disabled without affecting the main function of the program.

We could write a powersaver configuration, which disables the immediate
writes to the database, but registers an atexit handler to squirt the
session in when IPython is closed. I certainly want to cater for people
trying to eke out every last bit of battery life, but at the same time I
don't want all the users who aren't relying on a battery to miss out on what
I hope is a simpler, more powerful design for history.

Summarising your other points:
>Store translated history
Indeed - I've already changed my mind on this thanks to comments from other
people. My prototype is storing both raw and translated history

>Advantages of SQLite?
Robert's given a few. While JSON is a great tool, I don't think it's the
"one obvious way to do it" here - you have to parse it all to read any of
it, which is slow for a large file, and can lose your entire history if one
byte gets out of place. Neither is pickleshare, which is essentially a
key-value database, not designed to store a sequence. SQLite can essentially
combine what we're trying to do with both, and do it "right".

Also, I don't think the way we're reloading the history at present is tidy.
It gets lumped in with the current session, which causes the indexing
problems I already discovered. I've worked round those with session_offset,
but it's really a workaround. I could rethink the saving/reloading with
JSON, but when I tried thinking about it, I just decided JSON wasn't the
best tool for the job.

>Input processing
Thanks for the info :)

>Ordering multiple sessions:
While it's worth some thought, let's not lose too much sleep over the order.
As we've determined, if you want to go more than a few lines back in the
history, you search it somehow, in which case order doesn't matter much. I
never actually knew about Ctrl-R: I just type the first few letters of what
I want, then press up.

There are all sorts of options we could have for named sessions, for
repeating previous sessions, and so on, but I've not explored any of that
yet. As before, profiles have, for now, entirely separate history files - I
don't know if we want to keep that, or integrate them into one to allow
accessing history from other profiles.

>Tracking concurrent sessions:
Sessions can clearly leave some signal of their existence on disk, but if
they crash without clearing it up, other shells will see a zombie shell
still there. I don't think there's a payoff worth the trouble of tracking
'live' shells: we can just share the database with whatever other shells are
around.

>Readline history - 1000 lines
OK, I've made the prototype load 1000 lines by default. We can always up
that if we want.

(Robert/Brian) > Notebooks
[Snip] OK, Robert's clarified now that he meant something more like a
logbook than the UI option we've called a notebook.

Robert: storing the output is a bit trickier. We can store the repr of it
easily enough, which might be what you want. We could pickle output items
and store them, but that could quickly make the database file very large. If
you want to save a session with output, I believe you can use "%hist -of
session.txt".

(Brian) > Consistent API, alternative backends

Parts of the history_manager API can easily be abstracted - like
store_inputs and get_history. But there's a more conceptual difference in
what I'm doing, too: I'm getting rid of the idea of saving and loading
history, in favour of a model where it's immediately persisted. We can
expand that to include delayed persistence (as I've suggested above for
extending battery life), but we have to choose whether our API is written in
terms of "save" and "load", or in terms of "push cache to persistent store".

Also, while I'm all in favour of clean, well documented APIs, I think
storing history is the sort of thing that we as the creators should get
right, and users should never have to worry about. If someone wants to write
their own history manager for some reason, that's fine, but I don't think we
need to design around the assumption that people will. Let's focus on
writing a history system, not on an interface for possible history systems.

Whew! This seems to be a thread of epic e-mails. Thanks everyone for your
thoughts, and I hope I've answered any questions people had.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110218/de8c257b/attachment.html>

From robert.kern at gmail.com  Fri Feb 18 14:34:43 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 18 Feb 2011 13:34:43 -0600
Subject: [IPython-dev] History
In-Reply-To: <AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
	<AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>
Message-ID: <ijmhkk$lr8$1@dough.gmane.org>

On 2/18/11 1:14 PM, Thomas Kluyver wrote:

>  >Ordering multiple sessions:
> While it's worth some thought, let's not lose too much sleep over the order. As
> we've determined, if you want to go more than a few lines back in the history,
> you search it somehow, in which case order doesn't matter much.

Right. Just pick a simple rule that keeps sessions intact, say ordered by the 
start time of the session. The only thing to avoid is interleaving inputs from 
different sessions.

> (Robert/Brian) > Notebooks
> [Snip] OK, Robert's clarified now that he meant something more like a logbook
> than the UI option we've called a notebook.
>
> Robert: storing the output is a bit trickier. We can store the repr of it easily
> enough, which might be what you want. We could pickle output items and store
> them, but that could quickly make the database file very large.

I don't mean the actual objects, just whatever is given by the DisplayFormatter 
system and the displayhook, i.e. whatever would get sent to a remote frontend. 
I'm happy to have it be an extension functionality disabled by default, but I do 
think that it should live in the same SQLite DB to get the references right.

 > If you want to
 > save a session with output, I believe you can use "%hist -of session.txt".

Not what I'm looking for, but that functionality could be more easily (and 
efficiently) backed by the datastore I'm asking for.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From takowl at gmail.com  Fri Feb 18 16:50:50 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 18 Feb 2011 21:50:50 +0000
Subject: [IPython-dev] History
In-Reply-To: <ijmhkk$lr8$1@dough.gmane.org>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
	<AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>
	<ijmhkk$lr8$1@dough.gmane.org>
Message-ID: <AANLkTikQLwVHNHa4w+UffXeskx1aZ3MLehLD8v4x3hMX@mail.gmail.com>

On 18 February 2011 19:34, Robert Kern <robert.kern at gmail.com> wrote:

> I don't mean the actual objects, just whatever is given by the
> DisplayFormatter
> system and the displayhook, i.e. whatever would get sent to a remote
> frontend.
> I'm happy to have it be an extension functionality disabled by default, but
> I do
> think that it should live in the same SQLite DB to get the references
> right.


I don't have a problem with that, although my vote does go to having it
disabled by default. It's not altogether trivial, though, because we
currently store the input before running it, so the output isn't available
at the same time. Possible ways round that:
- Keep output in a separate table, indexed in the same way as input. This is
my preferred option, because it means the main history system doesn't need
to be aware of the optional output history. There's a slightly greater
chance of a mismatch between input and output, but the indexing is simple
enough that it shouldn't be a problem.
- Store input, then execute, and update the row with output (if any) - I
don't much like the idea of doing update statements after every command.
- Store input after execution - undesirable, because a command that crashes
IPython will not be stored.
- Store input after execution only when output history is turned on - ugly
idea.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110218/86d25ce4/attachment.html>

From robert.kern at gmail.com  Fri Feb 18 17:10:40 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 18 Feb 2011 16:10:40 -0600
Subject: [IPython-dev] History
In-Reply-To: <AANLkTikQLwVHNHa4w+UffXeskx1aZ3MLehLD8v4x3hMX@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>	<AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>	<ijmhkk$lr8$1@dough.gmane.org>
	<AANLkTikQLwVHNHa4w+UffXeskx1aZ3MLehLD8v4x3hMX@mail.gmail.com>
Message-ID: <ijmqp1$7g9$1@dough.gmane.org>

[Please do not Cc me. I am on the list!]

On 2/18/11 3:50 PM, Thomas Kluyver wrote:
> On 18 February 2011 19:34, Robert Kern <robert.kern at gmail.com
> <mailto:robert.kern at gmail.com>> wrote:
>
>     I don't mean the actual objects, just whatever is given by the DisplayFormatter
>     system and the displayhook, i.e. whatever would get sent to a remote frontend.
>     I'm happy to have it be an extension functionality disabled by default, but I do
>     think that it should live in the same SQLite DB to get the references right.
>
> I don't have a problem with that, although my vote does go to having it disabled
> by default.

I vote the same way, too. :-)

> It's not altogether trivial, though, because we currently store the
> input before running it, so the output isn't available at the same time.

That's fine. It's necessarily a separate API although I do think that it should 
be part of the HistoryManager.

> Possible ways round that:
> - Keep output in a separate table, indexed in the same way as input. This is my
> preferred option, because it means the main history system doesn't need to be
> aware of the optional output history. There's a slightly greater chance of a
> mismatch between input and output, but the indexing is simple enough that it
> shouldn't be a problem.

This sounds fine. With (session, line) as the foreign key, it seems 
straightforward to keep it synchronous. The only suggestion I would make is to 
make the store_inputs() method (and a putative store_outputs() method) take the 
execution_count explicitly rather than taking it from the shell.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From takowl at gmail.com  Fri Feb 18 17:37:44 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 18 Feb 2011 22:37:44 +0000
Subject: [IPython-dev] History
In-Reply-To: <ijmqp1$7g9$1@dough.gmane.org>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
	<AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>
	<ijmhkk$lr8$1@dough.gmane.org>
	<AANLkTikQLwVHNHa4w+UffXeskx1aZ3MLehLD8v4x3hMX@mail.gmail.com>
	<ijmqp1$7g9$1@dough.gmane.org>
Message-ID: <AANLkTin57RRDVDDv_d-15RQpE=z1NscQuFmQPgy8iM47@mail.gmail.com>

On 18 February 2011 22:10, Robert Kern <robert.kern at gmail.com> wrote:

> [Please do not Cc me. I am on the list!]
>

Sorry. If I hit reply, it e-mails you directly (GMail's not so good at
mailing lists), so I'm in the habit of doing reply all. I thought the list
was smart enough to see your e-mail address and only send it once. I'll try
to remember to take other addresses out.


> This sounds fine. With (session, line) as the foreign key, it seems
> straightforward to keep it synchronous. The only suggestion I would make is
> to
> make the store_inputs() method (and a putative store_outputs() method) take
> the
> execution_count explicitly rather than taking it from the shell.
>

OK, excellent. And yes, it's not a bad idea to take the execution_count/line
number explicitly. Done.

Coming back to Fernando's point about disk writes, I've implemented a very
simple cache (21 extra lines :) ). It can be turned on simply by setting
ip.history_manager.db_cache_size to the number of lines you want to store
before writing to disk. So, if you set it to 19, it will write on every 20th
command. For now, I've left the default at 0 - instant writes. And on exit,
any remaining cache is written, so you can set it to a very high value if
you want to minimise disk access.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110218/bae315ff/attachment.html>

From takowl at gmail.com  Sat Feb 19 16:54:29 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Sat, 19 Feb 2011 21:54:29 +0000
Subject: [IPython-dev] History
In-Reply-To: <ijmqp1$7g9$1@dough.gmane.org>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
	<AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>
	<ijmhkk$lr8$1@dough.gmane.org>
	<AANLkTikQLwVHNHa4w+UffXeskx1aZ3MLehLD8v4x3hMX@mail.gmail.com>
	<ijmqp1$7g9$1@dough.gmane.org>
Message-ID: <AANLkTimBW20FJ2Q2wmiLK=vDoM+mTq0cSjU4vhvf_Xao@mail.gmail.com>

On 18 February 2011 22:10, Robert Kern <robert.kern at gmail.com> wrote:

> This sounds fine. With (session, line) as the foreign key, it seems
> straightforward to keep it synchronous. The only suggestion I would make is
> to
> make the store_inputs() method (and a putative store_outputs() method) take
> the
> execution_count explicitly rather than taking it from the shell.
>

Basic implementation here:
https://github.com/takluyver/ipython/commit/15fbffa0d924765461c9a1d539d49daf70bc5315

The only bit missing is that I don't know off the top of my head where to
call the store_outputs method. I can have a dig in the code, or if someone
would like to point me to the best place, I'd be grateful. In the meantime,
I've tried calling the function manually, and it works as expected.

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110219/c1c2ca42/attachment.html>

From benjaminrk at gmail.com  Sun Feb 20 18:02:55 2011
From: benjaminrk at gmail.com (MinRK)
Date: Sun, 20 Feb 2011 15:02:55 -0800
Subject: [IPython-dev] GitHub Issues viewer
Message-ID: <AANLkTikKDmzHq9v5hQpCyObt1MxPp8_a9C8uCjj_s+wC@mail.gmail.com>

Hello,

Much as many of us are fond of GitHub, one of the site's major
shortcomings is the Issues page, especially if you have as many issues
as IPython does.  However, one of the great features of GitHub is its
json api, which means that anyone can write a better UI to the issue
tracker, and various people have had a run at this.  One I have found
useful is actually a demo of Cappucino, an OSX-centric webapp
framework.  Far from perfect, but it does handle the long list much
better.

You can see IPython's Issues by visiting:
http://githubissues.heroku.com/#ipython/ipython

And, if you are on OSX, you can run it as a standalone app, downloaded
from here:
http://cappuccino.org/discuss/2010/05/13/github-issues-cappuccino-app-desktop-and-web/

-MinRK


From fperez.net at gmail.com  Sun Feb 20 20:50:11 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 20 Feb 2011 20:50:11 -0500
Subject: [IPython-dev] GitHub Issues viewer
In-Reply-To: <AANLkTikKDmzHq9v5hQpCyObt1MxPp8_a9C8uCjj_s+wC@mail.gmail.com>
References: <AANLkTikKDmzHq9v5hQpCyObt1MxPp8_a9C8uCjj_s+wC@mail.gmail.com>
Message-ID: <AANLkTikVV64Z1HnucnLUt-BKnKkN_K1Q4W2XZC-psDf9@mail.gmail.com>

On Sun, Feb 20, 2011 at 6:02 PM, MinRK <benjaminrk at gmail.com> wrote:
>
> You can see IPython's Issues by visiting:
> http://githubissues.heroku.com/#ipython/ipython

Nice! Thanks for the link, very useful.

Cheers,

f


From ccordoba12 at gmail.com  Sun Feb 20 22:13:45 2011
From: ccordoba12 at gmail.com (=?ISO-8859-1?Q?Carlos_C=F3rdoba?=)
Date: Sun, 20 Feb 2011 22:13:45 -0500
Subject: [IPython-dev] GitHub Issues viewer
In-Reply-To: <AANLkTikKDmzHq9v5hQpCyObt1MxPp8_a9C8uCjj_s+wC@mail.gmail.com>
References: <AANLkTikKDmzHq9v5hQpCyObt1MxPp8_a9C8uCjj_s+wC@mail.gmail.com>
Message-ID: <4D61D869.9080905@gmail.com>

Quite cool app indeed! For those using Chrome, they can play with it 
after installing it from Chrome Web Store, at

https://chrome.google.com/webstore/detail/oaianekphgmiibhdffnoimngeolnailp

Carlos

El 20/02/11 18:02, MinRK escribi?:
> Hello,
>
> Much as many of us are fond of GitHub, one of the site's major
> shortcomings is the Issues page, especially if you have as many issues
> as IPython does.  However, one of the great features of GitHub is its
> json api, which means that anyone can write a better UI to the issue
> tracker, and various people have had a run at this.  One I have found
> useful is actually a demo of Cappucino, an OSX-centric webapp
> framework.  Far from perfect, but it does handle the long list much
> better.
>
> You can see IPython's Issues by visiting:
> http://githubissues.heroku.com/#ipython/ipython
>
> And, if you are on OSX, you can run it as a standalone app, downloaded
> from here:
> http://cappuccino.org/discuss/2010/05/13/github-issues-cappuccino-app-desktop-and-web/
>
> -MinRK
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


From kunal.t2 at gmail.com  Mon Feb 21 09:00:04 2011
From: kunal.t2 at gmail.com (kunal)
Date: Mon, 21 Feb 2011 19:30:04 +0530
Subject: [IPython-dev] GitHub Issues viewer
In-Reply-To: <AANLkTikVV64Z1HnucnLUt-BKnKkN_K1Q4W2XZC-psDf9@mail.gmail.com>
References: <AANLkTikKDmzHq9v5hQpCyObt1MxPp8_a9C8uCjj_s+wC@mail.gmail.com>
	<AANLkTikVV64Z1HnucnLUt-BKnKkN_K1Q4W2XZC-psDf9@mail.gmail.com>
Message-ID: <4D626FE4.2010107@gmail.com>

On 02/21/2011 07:20 AM, Fernando Perez wrote:
> On Sun, Feb 20, 2011 at 6:02 PM, MinRK<benjaminrk at gmail.com>  wrote:
>> You can see IPython's Issues by visiting:
>> http://githubissues.heroku.com/#ipython/ipython
> Nice! Thanks for the link, very useful.
This looks neat !
+1 much more use-able than the default github issue tracker.
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev


-- 
regards
-------
Kunal Ghosh
Dept of Computer Sc.&  Engineering.
Sir MVIT
Bangalore,India

permalink: member.acm.org/~kunal.t2
Blog:kunalghosh.wordpress.com
Website:www.kunalghosh.net46.net



From robert.kern at gmail.com  Mon Feb 21 15:31:37 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 21 Feb 2011 14:31:37 -0600
Subject: [IPython-dev] History
In-Reply-To: <AANLkTimBW20FJ2Q2wmiLK=vDoM+mTq0cSjU4vhvf_Xao@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>	<AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>	<ijmhkk$lr8$1@dough.gmane.org>	<AANLkTikQLwVHNHa4w+UffXeskx1aZ3MLehLD8v4x3hMX@mail.gmail.com>	<ijmqp1$7g9$1@dough.gmane.org>
	<AANLkTimBW20FJ2Q2wmiLK=vDoM+mTq0cSjU4vhvf_Xao@mail.gmail.com>
Message-ID: <ijui3a$n8m$1@dough.gmane.org>

On 2/19/11 3:54 PM, Thomas Kluyver wrote:
> On 18 February 2011 22:10, Robert Kern <robert.kern at gmail.com
> <mailto:robert.kern at gmail.com>> wrote:
>
>     This sounds fine. With (session, line) as the foreign key, it seems
>     straightforward to keep it synchronous. The only suggestion I would make is to
>     make the store_inputs() method (and a putative store_outputs() method) take the
>     execution_count explicitly rather than taking it from the shell.
>
>
> Basic implementation here:
> https://github.com/takluyver/ipython/commit/15fbffa0d924765461c9a1d539d49daf70bc5315
>
> The only bit missing is that I don't know off the top of my head where to call
> the store_outputs method. I can have a dig in the code, or if someone would like
> to point me to the best place, I'd be grateful. In the meantime, I've tried
> calling the function manually, and it works as expected.

The DisplayHook.log_output() method would be the right place to do it. 
Eventually, it would be nice to expand this to capture all of the different 
formats and the printed outputs, but just storing the text/plain format for 
displayhook'ed objects is enough for now.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From takowl at gmail.com  Tue Feb 22 14:15:54 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 22 Feb 2011 19:15:54 +0000
Subject: [IPython-dev] History
In-Reply-To: <AANLkTikjp3seD5K3=RLEopOBORNzhg8uR0sSam03M0Qp@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
	<AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>
	<ijmhkk$lr8$1@dough.gmane.org>
	<AANLkTikQLwVHNHa4w+UffXeskx1aZ3MLehLD8v4x3hMX@mail.gmail.com>
	<ijmqp1$7g9$1@dough.gmane.org>
	<AANLkTimBW20FJ2Q2wmiLK=vDoM+mTq0cSjU4vhvf_Xao@mail.gmail.com>
	<ijui3a$n8m$1@dough.gmane.org>
	<AANLkTikfCmVnzAtALY+qwaJT9rOD-jEAo9c4odxLqh6U@mail.gmail.com>
	<AANLkTikjp3seD5K3=RLEopOBORNzhg8uR0sSam03M0Qp@mail.gmail.com>
Message-ID: <AANLkTi=gvBXNugMtnuAUCsrND6EMO0ZesFthGsDoXtDA@mail.gmail.com>

On 21 February 2011 23:19, Robert Kern <robert.kern at gmail.com> wrote:

> Now you just have to make HistoryManager a Configurable so we can
> enable it from the configuration.
>

I think this does the trick:
https://github.com/takluyver/ipython/commit/eca335fc6efe0f479774e2a93cf95f730342bf91

I've tested uncommenting the lines in ipython_config.py, and they have the
desired effects.

Thanks for your help with the config system and traitlets,

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110222/e35b2287/attachment.html>

From fperez.net at gmail.com  Tue Feb 22 16:39:39 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 22 Feb 2011 13:39:39 -0800
Subject: [IPython-dev] History
In-Reply-To: <AANLkTi=gvBXNugMtnuAUCsrND6EMO0ZesFthGsDoXtDA@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
	<AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>
	<ijmhkk$lr8$1@dough.gmane.org>
	<AANLkTikQLwVHNHa4w+UffXeskx1aZ3MLehLD8v4x3hMX@mail.gmail.com>
	<ijmqp1$7g9$1@dough.gmane.org>
	<AANLkTimBW20FJ2Q2wmiLK=vDoM+mTq0cSjU4vhvf_Xao@mail.gmail.com>
	<ijui3a$n8m$1@dough.gmane.org>
	<AANLkTikfCmVnzAtALY+qwaJT9rOD-jEAo9c4odxLqh6U@mail.gmail.com>
	<AANLkTikjp3seD5K3=RLEopOBORNzhg8uR0sSam03M0Qp@mail.gmail.com>
	<AANLkTi=gvBXNugMtnuAUCsrND6EMO0ZesFthGsDoXtDA@mail.gmail.com>
Message-ID: <AANLkTikX7XjgzgY43tpd_qtN7CYRcWeshs5HJwPBGVOn@mail.gmail.com>

Quick question: does the work you've done in any way
impact/supersede/collide with this one:

https://github.com/ipython/ipython/pull/235

?

Cheers

f


From takowl at gmail.com  Tue Feb 22 16:49:42 2011
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 22 Feb 2011 21:49:42 +0000
Subject: [IPython-dev] History
In-Reply-To: <AANLkTikX7XjgzgY43tpd_qtN7CYRcWeshs5HJwPBGVOn@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
	<AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>
	<ijmhkk$lr8$1@dough.gmane.org>
	<AANLkTikQLwVHNHa4w+UffXeskx1aZ3MLehLD8v4x3hMX@mail.gmail.com>
	<ijmqp1$7g9$1@dough.gmane.org>
	<AANLkTimBW20FJ2Q2wmiLK=vDoM+mTq0cSjU4vhvf_Xao@mail.gmail.com>
	<ijui3a$n8m$1@dough.gmane.org>
	<AANLkTikfCmVnzAtALY+qwaJT9rOD-jEAo9c4odxLqh6U@mail.gmail.com>
	<AANLkTikjp3seD5K3=RLEopOBORNzhg8uR0sSam03M0Qp@mail.gmail.com>
	<AANLkTi=gvBXNugMtnuAUCsrND6EMO0ZesFthGsDoXtDA@mail.gmail.com>
	<AANLkTikX7XjgzgY43tpd_qtN7CYRcWeshs5HJwPBGVOn@mail.gmail.com>
Message-ID: <AANLkTimSR6wBM+p99vUMc=VSR-v_aazeByW9+gOkmKUz@mail.gmail.com>

At a glance, yes, I'm afraid so. It looks like he's solved the same thing
I've already got a pull request open for. I should have looked at the pull
requests before I started coding. He's added an "all history" option to the
%history magic command, which is more than I did.

However, I'm still hoping to persuade you (and others) of the benefits of
the SQLite history system I'm currently working on. In that case, that work
should supersede both that pull request and my open history-related pull
request. I'm just sorting out APIs, but I should have a pull request ready
before long (next couple of days).

Best wishes,
Thomas

On 22 February 2011 21:39, Fernando Perez <fperez.net at gmail.com> wrote:

> Quick question: does the work you've done in any way
> impact/supersede/collide with this one:
>
> https://github.com/ipython/ipython/pull/235
>
> ?
>
> Cheers
>
> f
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110222/3440d3af/attachment.html>

From fperez.net at gmail.com  Tue Feb 22 16:53:02 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 22 Feb 2011 13:53:02 -0800
Subject: [IPython-dev] GitHub Issues viewer
In-Reply-To: <AANLkTikKDmzHq9v5hQpCyObt1MxPp8_a9C8uCjj_s+wC@mail.gmail.com>
References: <AANLkTikKDmzHq9v5hQpCyObt1MxPp8_a9C8uCjj_s+wC@mail.gmail.com>
Message-ID: <AANLkTim-jG6AnbdKa=yCNwHcLSnTrZJLGXTkg3wC3rav@mail.gmail.com>

Hey,

On Sun, Feb 20, 2011 at 3:02 PM, MinRK <benjaminrk at gmail.com> wrote:
>
> You can see IPython's Issues by visiting:
> http://githubissues.heroku.com/#ipython/ipython
>
> And, if you are on OSX, you can run it as a standalone app, downloaded
> from here:
> http://cappuccino.org/discuss/2010/05/13/github-issues-cappuccino-app-desktop-and-web/
>

I added the link in our dev docs on the wiki:

http://ipython.scipy.org/moin/Developer_Zone

Cheers,

f


From fperez.net at gmail.com  Tue Feb 22 16:55:43 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 22 Feb 2011 13:55:43 -0800
Subject: [IPython-dev] History
In-Reply-To: <AANLkTimSR6wBM+p99vUMc=VSR-v_aazeByW9+gOkmKUz@mail.gmail.com>
References: <AANLkTikfDL-eJp=XGDJkfyzbg7jBY31R7MV8kh1qSiJT@mail.gmail.com>
	<AANLkTimQ+LdYJrErqbVZ9UR0HUyChEVjTM51Mvy_xTWS@mail.gmail.com>
	<AANLkTi=ZJKnKZemwLk8sXDYj5nbZYz+k_EpvwaWWJkYV@mail.gmail.com>
	<ijmhkk$lr8$1@dough.gmane.org>
	<AANLkTikQLwVHNHa4w+UffXeskx1aZ3MLehLD8v4x3hMX@mail.gmail.com>
	<ijmqp1$7g9$1@dough.gmane.org>
	<AANLkTimBW20FJ2Q2wmiLK=vDoM+mTq0cSjU4vhvf_Xao@mail.gmail.com>
	<ijui3a$n8m$1@dough.gmane.org>
	<AANLkTikfCmVnzAtALY+qwaJT9rOD-jEAo9c4odxLqh6U@mail.gmail.com>
	<AANLkTikjp3seD5K3=RLEopOBORNzhg8uR0sSam03M0Qp@mail.gmail.com>
	<AANLkTi=gvBXNugMtnuAUCsrND6EMO0ZesFthGsDoXtDA@mail.gmail.com>
	<AANLkTikX7XjgzgY43tpd_qtN7CYRcWeshs5HJwPBGVOn@mail.gmail.com>
	<AANLkTimSR6wBM+p99vUMc=VSR-v_aazeByW9+gOkmKUz@mail.gmail.com>
Message-ID: <AANLkTin+=sk+LXsCSZdxvFt34sSME=K6p5Qt0xgqhB7y@mail.gmail.com>

On Tue, Feb 22, 2011 at 1:49 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
> However, I'm still hoping to persuade you (and others) of the benefits of
> the SQLite history system I'm currently working on. In that case, that work
> should supersede both that pull request and my open history-related pull
> request. I'm just sorting out APIs, but I should have a pull request ready
> before long (next couple of days).

OK, no problem.  I have to say, I've reread the discussion and I'm
warming up to the idea of the sqlite approach quite a bit.  Robert's
points are very valid, and I like the idea of having a programatically
accessible store of all inputs (and possibly output reprs) that users
can get data from in any way they see fit.  And as Robert says, this
is a global store, complementary to any more document-oriented
notebook idea.

Cheers,

f


From robert.kern at gmail.com  Fri Feb 25 19:40:33 2011
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 25 Feb 2011 18:40:33 -0600
Subject: [IPython-dev] Delete pyzmq 2.0.10-1 on PyPI?
Message-ID: <ik9i62$9du$1@dough.gmane.org>

Brian, Min,

Can you delete or hide the pyzmq 2.0.10-1 release on PyPI? Due to the way PyPI 
sorts releases, it is coming up as the latest release of pyzmq.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From benjaminrk at gmail.com  Fri Feb 25 19:50:13 2011
From: benjaminrk at gmail.com (MinRK)
Date: Fri, 25 Feb 2011 16:50:13 -0800
Subject: [IPython-dev] Delete pyzmq 2.0.10-1 on PyPI?
In-Reply-To: <ik9i62$9du$1@dough.gmane.org>
References: <ik9i62$9du$1@dough.gmane.org>
Message-ID: <AANLkTikFvgzTgGnLuZbF3W1JdQg255wCZ6Ux+GzXj=4e@mail.gmail.com>

Done, thanks for catching that one.

I had missed it, since easy_install was correctly finding 2.0.10.1.

-MinRK

On Fri, Feb 25, 2011 at 16:40, Robert Kern <robert.kern at gmail.com> wrote:
> Brian, Min,
>
> Can you delete or hide the pyzmq 2.0.10-1 release on PyPI? Due to the way PyPI
> sorts releases, it is coming up as the latest release of pyzmq.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
> ?that is made terrible by our own mad attempt to interpret it as though it had
> ?an underlying truth."
> ? -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From fperez.net at gmail.com  Sat Feb 26 19:38:26 2011
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 26 Feb 2011 16:38:26 -0800
Subject: [IPython-dev] Fwd: [GitHub] SQLite history [ipython/ipython GH-282]
In-Reply-To: <ipython/ipython/pull/282@github.com>
References: <ipython/ipython/pull/282@github.com>
Message-ID: <AANLkTik_H5cyJdyxEnO+kOu9ishdrtVZ-t9akEDWaeMq@mail.gmail.com>

Hi folks,

Thomas just created this pull request for the sqlite-based history.
I'll try to have a look soon, but this is large and important enough
that more sets of eyes would be very, very welcome.

And many thanks to Thomas for implementing all this!

Best,

f


---------- Forwarded message ----------
From: takluyver <noreply at github.com>
Date: Sat, Feb 26, 2011 at 3:11 PM
Subject: [GitHub] SQLite history [ipython/ipython GH-282]
To: fperez.net at gmail.com


This was discussed previously on the mailing list. The code will
probably still need some refining, but I think it's working well
enough to put it forward.

There are two major changes involved:

History is now stored on disk as an SQLite database. This replaces
both the JSON history and the "shadow history" database. By default,
the input is stored as each line is entered, but (as Fernando
suggested), there is a configurable attribute, db_cache_size, which
lets you determine how often (in number of commands) it will write to
disk. Very large values will cause all input to be written when
IPython quits. A separate configurable attribute, db_log_output,
causes the plaintext repr of output to be written to a separate table
in the file.

Secondly, whereas history was previously treated as one long stream of
commands, it is now stored by "sessions", i.e. runs of IPython. So, if
you are particularly fond of the fifth command you gave IPython in one
session, the next time you start IPython, you can refer to the fifth
command of the last session (the syntax for magic commands is `~1/5`).
Sessions are consecutively numbered from when they start, so
interleaved commands from two shells open at the same time will be
kept separate.

This requires somewhat more complicated interfaces. There are now four
main interfaces to retrieve history from HistoryManager:

* `get_history`: Retrieves specified ranges of history, either from
the database, or the current session.
* `get_hist_from_rangestr`: Is a convenience method for magic
functions, which accepts the "~1/3-6 ?12" style range specifications.
* `get_hist_tail`: Gets the last n items from the database, ordered by
session and line. Used to populate readline (and the equivalent in the
Qt console).
* `get_hist_search`: Searches the database using glob style matching
(i.e. wildcards, not regexen).

All of these return iterators over 3-tuples, `(session, lineno,
item)`. If asked for output, `item` will be a 2-tuple of `(input,
output)`.

I've also given `%hist` the same range specification parsing as
`%save`, `%macro` and `%edit` (it would previously only accept one or
two numbers separated by a space). A single numeric argument, such as
`%hist 10` previously fetched the last 10 lines; now it gets the 10th
line of the current session. So I've added a new option, `-l` (for
last), which works with a single numeric argument. This is marginally
less convenient if that's what you want, but I think the consistency
among magic functions retrieving history is a good thing.

https://github.com/ipython/ipython/pull/282