From ellisonbg.net at gmail.com  Sun Oct  1 02:12:07 2006
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Sun, 1 Oct 2006 00:12:07 -0600
Subject: [IPython-dev] Moin issues
In-Reply-To: <E1GTaqr-0003MB-H4@mail2.sun.ac.za>
References: <E1GTaqr-0003MB-H4@mail2.sun.ac.za>
Message-ID: <6ce0ac130609302312u3bb6cdbdwb6ee5586759aea49@mail.gmail.com>

Thanks for catching this.  I think Fernando did somethings in making
the moin site the new home page the broke some of this.  I am not sure
if he wants things under the moin subdirectory or not.  Because of
this I am going to wait until monday (he is away for the weekend) to
fix this.

Thanks though

Brian

On 9/30/06, Albert Strasheim <fullung at gmail.com> wrote:
> Hello all
>
> I noticed that some links on the IPython Moin site are pointing to the wrong
> places. For example, on this page:
>
> http://ipython.scipy.org/Parallel_Computing
>
> there is a link to Parallel_Computing/FAQ but the URL Moin generates for
> this page is:
>
> http://ipython.scipy.org/moin/Parallel_Computing/FAQ
>
> which goes to a non-existant page, instead of
>
> http://ipython.scipy.org/Parallel_Computing/FAQ
>
> Regards,
>
> Albert
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://projects.scipy.org/mailman/listinfo/ipython-dev
>


From fperez.net at gmail.com  Tue Oct  3 03:01:16 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 3 Oct 2006 01:01:16 -0600
Subject: [IPython-dev] Moin issues
In-Reply-To: <E1GTaqr-0003MB-H4@mail2.sun.ac.za>
References: <E1GTaqr-0003MB-H4@mail2.sun.ac.za>
Message-ID: <db6b5ecc0610030001g5be92f1l32a08643be363c@mail.gmail.com>

On 9/30/06, Albert Strasheim <fullung at gmail.com> wrote:
> Hello all
>
> I noticed that some links on the IPython Moin site are pointing to the wrong
> places. For example, on this page:
>
> http://ipython.scipy.org/Parallel_Computing
>
> there is a link to Parallel_Computing/FAQ but the URL Moin generates for
> this page is:
>
> http://ipython.scipy.org/moin/Parallel_Computing/FAQ
>
> which goes to a non-existant page, instead of
>
> http://ipython.scipy.org/Parallel_Computing/FAQ

Thanks, Albert.  It should all be reasonably OK now, just growing
pains with Apache redirects and the virtual hosting setup at
scipy.org.  By all means report any new problems!

Cheers,

f


From fperez.net at gmail.com  Tue Oct  3 03:01:46 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 3 Oct 2006 01:01:46 -0600
Subject: [IPython-dev] emacs display problems + patch
In-Reply-To: <20060930214556.GI1607@mentat.za.net>
References: <20060930214556.GI1607@mentat.za.net>
Message-ID: <db6b5ecc0610030001n1b6ebfc2kfff9574830712401@mail.gmail.com>

On 9/30/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> Hi,
>
> Has anyone had a chance to take a look at the attached ipython.el
> patch?

I'll take care of it this week, promise.  Thanks!

f


From vivainio at gmail.com  Tue Oct  3 12:34:55 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 3 Oct 2006 19:34:55 +0300
Subject: [IPython-dev] Fwd: IPython and closing sys.stdout
In-Reply-To: <451A2727.7010305@colorado.edu>
References: <451A2727.7010305@colorado.edu>
Message-ID: <46cb515a0610030934q19277742m5bdb1239a5ddc7a5@mail.gmail.com>

On 9/27/06, Fernando Perez <Fernando.Perez at colorado.edu> wrote:

> In my opinion, the best thing to do would be to subclass InteractiveConsole
> and provide a more robust raw_input() method.  Ideally, I think you'd want
> to dup() the file descriptors for sys.{stdin,stdout} and use those instead
> of calling the builtin raw_input().  Something like (untested):
>
>      class IC(code.InteractiveConsole):
>          def __init__(self):
>              code.InteractiveConsole.__init__(self)
>              self.input = os.fdopen(os.dup(sys.stdin.fileno()))
>              self.output = os.fdopen(os.dup(sys.stdout.fileno()))
>              self.error = os.fdopen(os.dup(sys.stderr.fileno()))
>
>          def raw_input(self, prompt=""):
>              if prompt:
>                  self.output.write(prompt):
>                  self.output.flush()
>              return self.input.readline()
>
>          def write(self, data):
>              self.error.write(data)
>
> Also, the runcode() method will have to be overridden to use self.output
> instead of sys.stdout.  Those couple changes should (hopefully) insulate
> IPython from such user wackiness.

It seems code.InteractiveConsole was removed from ipython
purposefully, so this would seem to be going to the wrong direction
(Fernando?). I put the ValueError cathing (which was originally for
quit() on py2.5) back in, but there are lots of plain print statements
around ipython and intact sys.std[in | out] would seem necessary, for
a while at least.

At least now we fail with a helpful message, instead of infinite loop.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Thu Oct  5 21:59:12 2006
From: rocky at panix.com (R. Bernstein)
Date: Thu, 5 Oct 2006 21:59:12 -0400
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
Message-ID: <17701.47216.335545.353158@panix3.panix.com>

Hi - 

Sorry if I've been out of touch for awhile. I just looked to see what
it would take to change the current svn sources to use pydb
(http://bashdb.sf.net/pydb) when it is available instead of the stock
pdb. I was very pleasantly pleased to find that, at least for python
2.5, the changes are fairly small.

(Last time I attempted doing this which was earlier this year, changes
were much more onerous and extensive. It looks like the code has
greatly been cleaned up. THANKS!)

Enclosed find a patch. Here is basically what happens. First I'm only
attempting this if Python 2.5 is installed. (The idea is to start
small and simple and if this is desirable/successful we can add other
Python versions later). I try to import pydb; if that fails then pdb
is imported. To make it clearer to folks which debugger they've got,
the prompt is also changed to "ipydb>" instead of "ipdb>". 

Note that in pydb we generally add a set of parenthesis around the
prompt whenever the debugger is called recursively (command "debug"
inside of the debugger).

Since pydb has a "restart" command which doesn't make sense in the
ipython contexts, that is made the same thing as "quit". Probably
better would be to issue a message indicating what's up (and then
quit). Perhaps later.

Thanks

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython-pydb.diff
Type: application/octet-stream
Size: 3165 bytes
Desc: Patch to allow pydb to be used when it is available.
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061005/4b14be16/attachment.obj>

From rocky at panix.com  Fri Oct  6 04:14:27 2006
From: rocky at panix.com (R. Bernstein)
Date: Fri, 6 Oct 2006 04:14:27 -0400
Subject: [IPython-dev]  Patches to use pydb instead of pdb for debugging
In-Reply-To: <17701.47216.335545.353158@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
Message-ID: <17702.4195.540436.694402@panix3.panix.com>

One more file to be patched, the Emacs interface, ipython.el, since I
changed the debugger prompt when using pydb:

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython.el.diff
Type: application/octet-stream
Size: 679 bytes
Desc: Patch to also allow 'ipydb>' as a prompt
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061006/1441ab42/attachment.obj>
-------------- next part --------------

R. Bernstein writes:
 > Hi - 
 > 
 > Sorry if I've been out of touch for awhile. I just looked to see what
 > it would take to change the current svn sources to use pydb
 > (http://bashdb.sf.net/pydb) when it is available instead of the stock
 > pdb. I was very pleasantly pleased to find that, at least for python
 > 2.5, the changes are fairly small.
 > 
 > (Last time I attempted doing this which was earlier this year, changes
 > were much more onerous and extensive. It looks like the code has
 > greatly been cleaned up. THANKS!)
 > 
 > Enclosed find a patch. Here is basically what happens. First I'm only
 > attempting this if Python 2.5 is installed. (The idea is to start
 > small and simple and if this is desirable/successful we can add other
 > Python versions later). I try to import pydb; if that fails then pdb
 > is imported. To make it clearer to folks which debugger they've got,
 > the prompt is also changed to "ipydb>" instead of "ipdb>". 
 > 
 > Note that in pydb we generally add a set of parenthesis around the
 > prompt whenever the debugger is called recursively (command "debug"
 > inside of the debugger).
 > 
 > Since pydb has a "restart" command which doesn't make sense in the
 > ipython contexts, that is made the same thing as "quit". Probably
 > better would be to issue a message indicating what's up (and then
 > quit). Perhaps later.
 > 
 > Thanks
 > 
 > _______________________________________________
 > IPython-dev mailing list
 > IPython-dev at scipy.org
 > http://projects.scipy.org/mailman/listinfo/ipython-dev

From fperez.net at gmail.com  Fri Oct  6 04:20:26 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 6 Oct 2006 02:20:26 -0600
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <17702.4195.540436.694402@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
Message-ID: <db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>

On 10/6/06, R. Bernstein <rocky at panix.com> wrote:
> One more file to be patched, the Emacs interface, ipython.el, since I
> changed the debugger prompt when using pydb:

Many thanks, I'll try to integrate these soon.

Are these diffs against current SVN?  If not, could you please resend
them against SVN?  I haven't checked, so if they are, just ignore this
comment.

Regards,

f


From rocky at panix.com  Fri Oct  6 06:19:21 2006
From: rocky at panix.com (R. Bernstein)
Date: Fri, 6 Oct 2006 06:19:21 -0400
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
Message-ID: <17702.11689.407083.959718@panix3.panix.com>

The diffs are against svn, current as of Oct 6th at revision 1809. 

Some small things I just noticed and throw out for general comment.

1. Command Completion
===================
pydb adds command completion for debugger commands and
subcommands. (Subcommands are things like "info ...", "set ..." or
"show ..."). It also adds the gdb "command" command which will return
a list of completion tokens.

I've checked and (not surprisingly) the "command" command does work
under ipython. However it would be nice to hook that in so it does the
right thing when one enters "tab" (or whatever the completion key is)
and likewise when running inside Emacs. It looks like some sort of
completion is provided in both cases, although the code for this is
different (if not also providing different results).

The code inside ipython-complete() of ipython.el looks pretty
complicated; also I'm not sure where the right place to hook this in
for the non-Emacs shell would be. Suggestions and advice?

2. ? and ??
===================
Although pydb adds an "examine" command for objects (similar to Perl's
debugger), it might be nice to extend pydb so that it hooks into the
same mechanism used by ipython. (I'm referring the ?object and
??object here). This could either be done inside pydb code, but maybe
it would be neater to do inside ipython since it looks like there are
routines already for decorating debugger commands. Suggestions on
where to look to hook into '?' and '??'?

3. list command
===================
And finally, speaking of extending the debugger commands, I see that
"list" is one of the extended commands (without help docstring). But
pydb's/gdb's list command is a bit more flexible: one use as a target
module/function/methods names as well as file/line positions. Also
pydb/gdb allows '-' to indicate listing backwards, and one can change
the list size window (via "set listsize").

At least for pydb this can currently be done by decorating the
function rather than copying it since pydb is better than pdb with
respect to allowing for output redirection. As pydb currently stands,
this would be done by saving the self.msg_nocr method, changing it and
restoring it before returning. However if folks think it is better for
pydb to provide a "list" command that returns a list of strings
instead, that could easily be done in a future pydb release. For
example, if folks think that doing the above is sufficiently ugly and
think there will be other applications like ipython that might want to
embed a debugger, changing pydb may make sense. So here too, comments
and suggestions are welcome.

Fernando Perez writes:
 > On 10/6/06, R. Bernstein <rocky at panix.com> wrote:
 > > One more file to be patched, the Emacs interface, ipython.el, since I
 > > changed the debugger prompt when using pydb:
 > 
 > Many thanks, I'll try to integrate these soon.
 > 
 > Are these diffs against current SVN?  If not, could you please resend
 > them against SVN?  I haven't checked, so if they are, just ignore this
 > comment.
 > 
 > Regards,
 > 
 > f
 > 


From rocky at panix.com  Fri Oct  6 14:10:36 2006
From: rocky at panix.com (R. Bernstein)
Date: Fri, 6 Oct 2006 14:10:36 -0400
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <17702.11689.407083.959718@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
Message-ID: <17702.39964.389464.78803@panix3.panix.com>

Sorry for the deluge of posting. Some information based on recent
investigation. 

1. do_list
===========

What I wrote about being able to decorate pydb's list command is,
alas, not useful here. In looking over ipython's replacement of pdb's
do_list, I see that unspecified lines of a file (such as in a list of
strings) is not sufficient.  What's also needed is the filename, and
line numbers so that ipython can mark breakpoints and the current line
in the listing whenever they occurs.

So in pydb's CVS, I've broken up the "list" command into two parts, a
parsing part and a listing part. (In pydb the listing part is still
called "do_list" and calls the parsing part first). However ipython's
"do_list" can now also be broken up into the parsing part and a
listing part (and here it's the listing part that is still called
"do_list" but it calls a split out parsing part. I tried this out, and
will submit a patch after the previous batch goes through if there's
interest.

2. Help docstrings for decorated fns do_up and do_down
======================================================

pdb doesn't have docstrings for the "up" and "down" commands. These
are decorated in ipython. But pydb does have docstrings. In other
words in pdb if you type "help up" you get nothing, whereas in pydb
you do get the syntax and semantics of the command. Based on a recent
posting by Duncan Booth in comp.lang.python, I've written a wrapper
function which takes a decorator function and the function it
decorates and adds the docstring to the decorator function. This is of
benefit for the ipython do_up, do_down routines when pydb is used.




From fperez.net at gmail.com  Fri Oct  6 19:23:45 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 6 Oct 2006 17:23:45 -0600
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <17702.39964.389464.78803@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<17702.39964.389464.78803@panix3.panix.com>
Message-ID: <db6b5ecc0610061623o2aa4e8acn5a35cc4c6270ecb9@mail.gmail.com>

On 10/6/06, R. Bernstein <rocky at panix.com> wrote:
> Sorry for the deluge of posting. Some information based on recent
> investigation.

No problem, the discussion is welcome, and these sounds like excellent
improvements.

I'd just like to mention that at this point, I'm mostly tied up with
the new branch, so I have precious little time to touch the 0.7.x
(trunk) one.  Over the next few days I'll do some work on it which we
need for the new one, but that's about it.  So these kinds of new
developments will be mostly handled by Ville.

We should plan a 0.7.3 release soonish.  Ville: would you want this
debugging work to be part of that release?  How's your schedule
looking in the next 3-4 weeks?

Cheers,

f


From vivainio at gmail.com  Sat Oct  7 06:08:02 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 7 Oct 2006 13:08:02 +0300
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <db6b5ecc0610061623o2aa4e8acn5a35cc4c6270ecb9@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<17702.39964.389464.78803@panix3.panix.com>
	<db6b5ecc0610061623o2aa4e8acn5a35cc4c6270ecb9@mail.gmail.com>
Message-ID: <46cb515a0610070308h507e1df8wf671bcfb28a87c36@mail.gmail.com>

On 10/7/06, Fernando Perez <fperez.net at gmail.com> wrote:

> On 10/6/06, R. Bernstein <rocky at panix.com> wrote:
> > Sorry for the deluge of posting. Some information based on recent
> > investigation.
>
> No problem, the discussion is welcome, and these sounds like excellent
> improvements.

Agreed.

> We should plan a 0.7.3 release soonish.  Ville: would you want this
> debugging work to be part of that release?  How's your schedule

Well, it's a great selling point for the new release, IMO. Should look
great on some kind of "show me do" video, esp. since many don't know
about pydb. :-)

> looking in the next 3-4 weeks?

My schedule should be ok for the release soon-ish. I'll examine this
pydb stuff soon enough myself, probably starting on tuesday.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Sat Oct  7 06:14:19 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 7 Oct 2006 13:14:19 +0300
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <17702.11689.407083.959718@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
Message-ID: <46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>

On 10/6/06, R. Bernstein <rocky at panix.com> wrote:

> ??object here). This could either be done inside pydb code, but maybe
> it would be neater to do inside ipython since it looks like there are
> routines already for decorating debugger commands. Suggestions on
> where to look to hook into '?' and '??'?

Now they are handled by "handle_help" in iplib.py, but you should
create an input_prefilter hook function that catches the ? commands
and translates them to something you want, e.g. ??foo ->
_pydb.detailedhelpcommand("foo"). see ext_rescapture.py on how to
implement one.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Sat Oct  7 08:47:25 2006
From: rocky at panix.com (R. Bernstein)
Date: Sat, 7 Oct 2006 08:47:25 -0400
Subject: [IPython-dev] making pydb play nice;
	ipython release may imply pydb release
Message-ID: <17703.41437.385516.139680@panix3.panix.com>

If it is the case that there is another release in 3-4 weeks and if
it's also the case that it would include (what I hope are rather
small) patches to allow pydb to be used, then it would probably be
advantageous to have another pydb release about the same time. 

Following the good practice that ipython seems to use, I make an
announcement of a posssible release in advance of the release.  That's
just been done on the sourceforge pydb mailing list.

The reason for the concurrent release is that there have been a couple
of small changes recently in pydb's CVS that may make interaction with
ipython more user-friendly. I mentioned the change to the
implementation of the "list" command.

Here's another. In order to make it possible to debug from inside
ipython (or any Python shell), I've added pydb.runl() and pydb.runv()
calls to start a debugger session. The 'l', and 'v' suffixes are
analogous to the meaning in spawnv()/execv() and spawnl()/execl()
calls.

For example, if you are in the ipython. You can do something like this:

  In [1]: import pydb
  In [2]: pydb.runl("--threading", "--nx", "/tmp/test.py", "--option1", "arg1")
to start a debugger session. 

The pydb.runv() form might look like this:

  In [3]: args=("--threading", "--nx", "/tmp/test.py", "--option1", "arg1")
  In [4]: pydb.runv(args)

Note that although pdb doesn't support command-line options (like
"--threading"), pydb does. So "--threading" and "--nx" go to the
*debugger*, and the Python script to be debugged and its options and
arguments comes after any debugger options.

Are there any other debugging annoyances working in ipython? 

Thanks.


From vivainio at gmail.com  Sat Oct  7 11:12:10 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 7 Oct 2006 18:12:10 +0300
Subject: [IPython-dev] making pydb play nice;
	ipython release may imply pydb release
In-Reply-To: <17703.41437.385516.139680@panix3.panix.com>
References: <17703.41437.385516.139680@panix3.panix.com>
Message-ID: <46cb515a0610070812l285a6d76s6309f5e114e3e7ca@mail.gmail.com>

On 10/7/06, R. Bernstein <rocky at panix.com> wrote:

> If it is the case that there is another release in 3-4 weeks and if
> it's also the case that it would include (what I hope are rather
> small) patches to allow pydb to be used, then it would probably be
> advantageous to have another pydb release about the same time.

Have you tried making devel version pydb easy_install:able by any
chance (as in "easy_install pydb==dev")? At some point (before the
release) we could have compatible development version of both pydb and
ipython for more widespread testing than what people on these lists
can provide...

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Sat Oct  7 11:39:41 2006
From: rocky at panix.com (R. Bernstein)
Date: Sat, 7 Oct 2006 11:39:41 -0400
Subject: [IPython-dev] making pydb play nice;
	ipython release may imply pydb release
In-Reply-To: <46cb515a0610070812l285a6d76s6309f5e114e3e7ca@mail.gmail.com>
References: <17703.41437.385516.139680@panix3.panix.com>
	<46cb515a0610070812l285a6d76s6309f5e114e3e7ca@mail.gmail.com>
Message-ID: <17703.51773.137527.146040@panix3.panix.com>

I don't know what easy_install is, but I'll look into when I get a
chance.

Begin perhaps a little bit of a rant: 

pydb still uses autotools because I couldn't figure out how to get
setup.py to work as flexibly. There was a query I made a while back in
comp.lang.python asking how one can use setup.py and have it
*uninstall*. 

This concept, uninstall, have been around for a decade or so in
autotools and it's not an unreasonable thing to want to do. I just
assumed any *newer* system would do it. Using one of the Perl's
install system, to add insult to injury I think they say "This feature
is unsupported and deprecated."  I guess we should be greatful that it
doesn't also complain that we are ugly too.

Ville M. Vainio writes:
 > On 10/7/06, R. Bernstein <rocky at panix.com> wrote:
 > 
 > > If it is the case that there is another release in 3-4 weeks and if
 > > it's also the case that it would include (what I hope are rather
 > > small) patches to allow pydb to be used, then it would probably be
 > > advantageous to have another pydb release about the same time.
 > 
 > Have you tried making devel version pydb easy_install:able by any
 > chance (as in "easy_install pydb==dev")? At some point (before the
 > release) we could have compatible development version of both pydb and
 > ipython for more widespread testing than what people on these lists
 > can provide...
 > 
 > -- 
 > Ville M. Vainio - vivainio.googlepages.com
 > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
 > 


From vivainio at gmail.com  Sat Oct  7 12:06:24 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 7 Oct 2006 19:06:24 +0300
Subject: [IPython-dev] making pydb play nice;
	ipython release may imply pydb release
In-Reply-To: <17703.51773.137527.146040@panix3.panix.com>
References: <17703.41437.385516.139680@panix3.panix.com>
	<46cb515a0610070812l285a6d76s6309f5e114e3e7ca@mail.gmail.com>
	<17703.51773.137527.146040@panix3.panix.com>
Message-ID: <46cb515a0610070906t259095fdicae0b6c202c864c5@mail.gmail.com>

On 10/7/06, R. Bernstein <rocky at panix.com> wrote:

> I don't know what easy_install is, but I'll look into when I get a
> chance.

http://peak.telecommunity.com/DevCenter/setuptools

I really enjoy installing stuff via "easy_install mymodule" instead of
all the sourceforge clickety-clicking around.

> Begin perhaps a little bit of a rant:
>
> pydb still uses autotools because I couldn't figure out how to get
> setup.py to work as flexibly. There was a query I made a while back in
> comp.lang.python asking how one can use setup.py and have it
> *uninstall*.
>
> This concept, uninstall, have been around for a decade or so in
> autotools and it's not an unreasonable thing to want to do. I just
> assumed any *newer* system would do it. Using one of the Perl's

setuptools installs stuff in cleaner fashion, and remove is basically
the same thing as removing the egg directory (or the egg itself). It's
in general a much more developed system than distutils. Do check it
out at some point when you have time.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Sun Oct  8 01:11:44 2006
From: rocky at panix.com (R. Bernstein)
Date: Sun, 8 Oct 2006 01:11:44 -0400
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
Message-ID: <17704.34960.967034.430310@panix3.panix.com>

I was not have been clear on what I was hoping to do.

Let's say I'm inside the debugger and I want to get the equivalent
information for a local object "a" that I would get were I in the
shell with that variable defined and were I to type "a?".

And let's forget about how to handle the syntax of ? or ??. So let's
add a new debugger command called "pinfo". All it wants to do is call
Magic.magic_pinfo(), but somehow set up the local and global
namespaces correctly. One could add to Debugger.py the following
method which sort of works, but really isn't right. It hackily
modifies one of the dictionaries that gets used via Magic.magic_pinfo.

    def do_pinfo(self, arg):
        """Sort of The debugger equivalant of obj? for a local variable"""
        # Find a dictionary to smack before the call. We'll use the
        # alias table.
        old_ns = __IPYTHON__.shell.alias_table   # save
        __IPYTHON__.shell.alias_table = self.curframe.f_locals  # smack
        __IPYTHON__.magic_pinfo("pinfo %s" % arg)
        __IPYTHON__.shell.alias_table = old_ns   # restore

Possibly the alias table shouldn't be wiped out -- I don't know
possibly it is irrelevant in this context anyway. And the alias table
is searched last whereas local variables are searched first by Python.

Instead I think one would like a variant of Magic._ofind which has a
notion of a local and global name spaces separate from the shell's
namespace and some way for a debugger to hook in the current
local/global namespaces to that. (Could the global namespace replace
user_ns of the shell? Maybe, I don't know.)

Note that such a routine as do_pinfo really has nothing to do with
pydb at all. Short of finding incidental misfeatures in pdb (or pydb
as I've been finding) which prevent this, to first approximation it
should work equally well in both. Or any debugger; or any user
application called from the ipython shell that want's to interact with
ipython for that matter.

- - - 

As a related but separate problem, now that I've go a way to invoke
the debugger in full generaltity from ipython I'd also like to be able
run such a "pinfo" command from that.  By virtue of starting inside
ipython I have all of the ipython routines around. In particular I
have __IPYTHON__ defined. So moving that same do_pinfo routine from
Debugger.py into someplace inside pydb's code would roughly do the
same thing. But again, that code is really fairly wrong for the same
reasons.


Ville M. Vainio writes:
 > On 10/6/06, R. Bernstein <rocky at panix.com> wrote:
 > 
 > > ??object here). This could either be done inside pydb code, but maybe
 > > it would be neater to do inside ipython since it looks like there are
 > > routines already for decorating debugger commands. Suggestions on
 > > where to look to hook into '?' and '??'?
 > 
 > Now they are handled by "handle_help" in iplib.py, but you should
 > create an input_prefilter hook function that catches the ? commands
 > and translates them to something you want, e.g. ??foo ->
 > _pydb.detailedhelpcommand("foo"). see ext_rescapture.py on how to
 > implement one.
 > 
 > -- 
 > Ville M. Vainio - vivainio.googlepages.com
 > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
 > 


From fperez.net at gmail.com  Mon Oct  9 04:13:48 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 9 Oct 2006 02:13:48 -0600
Subject: [IPython-dev] timeit bug
In-Reply-To: <46cb515a0609210248i7dbb65deid4a1ce2d48a73774@mail.gmail.com>
References: <20060915085341.GA1402@mentat.za.net>
	<46cb515a0609210156o613d9852u1139b79510b5291d@mail.gmail.com>
	<46cb515a0609210248i7dbb65deid4a1ce2d48a73774@mail.gmail.com>
Message-ID: <db6b5ecc0610090113n1ddc6161heb40ce3f8cf76b0f@mail.gmail.com>

On 9/21/06, Ville M. Vainio <vivainio at gmail.com> wrote:
> On 9/21/06, Ville M. Vainio <vivainio at gmail.com> wrote:
>
> > On 9/15/06, Stefan van der Walt <stefan at sun.ac.za> wrote:

[... %timeit print 'x' bug ]

The attached patch should be a proper fix.  It's late and I'm tired so
I don't want to commit it yet; Brian is testing it because he also ran
into similar problems and we need this fixed for a demo on Tuesday, so
it's now kind of high priority.  But I don't want to commit a broken
patch.

I'll be testing it tomorrow (Monday) further and will commit if we
find no other problems (and that it really fixes everyhting).
Comments on its behavior will be appreciated.

Note that any %magic that wants to preserve quotes in its inputs can
now use parse_options(posix=False).  %timeit is a good example, but in
the chainsaw branch there are others.  Basically, magics that treat
their input as python source shouldn't modify quotes, while magics
that treat it in traditional sys.argv manner want them stripped.

Cheers,

f
-------------- next part --------------
A non-text attachment was scrubbed...
Name: shfix.diff
Type: text/x-patch
Size: 4029 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061009/7b286c83/attachment.bin>

From hans_meine at gmx.net  Mon Oct  9 05:08:00 2006
From: hans_meine at gmx.net (Hans Meine)
Date: Mon, 9 Oct 2006 11:08:00 +0200
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <46cb515a0610070308h507e1df8wf671bcfb28a87c36@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<db6b5ecc0610061623o2aa4e8acn5a35cc4c6270ecb9@mail.gmail.com>
	<46cb515a0610070308h507e1df8wf671bcfb28a87c36@mail.gmail.com>
Message-ID: <200610091108.01709.hans_meine@gmx.net>

On Saturday, 07. October 2006 12:08, Ville M. Vainio wrote:
> > We should plan a 0.7.3 release soonish.  Ville: would you want this
> > debugging work to be part of that release?  How's your schedule
>
> Well, it's a great selling point for the new release, IMO. Should look
> great on some kind of "show me do" video, esp. since many don't know
> about pydb. :-)

Another great selling point IMNSHO is the multi-line history handling (e.g. 
when fixing an error in a rather largish interactively entered for-loop).

Ciao, /  /
     /--/
    /  / ANS


From fperez.net at gmail.com  Mon Oct  9 14:07:08 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 9 Oct 2006 12:07:08 -0600
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <46cb515a0610070308h507e1df8wf671bcfb28a87c36@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<17702.39964.389464.78803@panix3.panix.com>
	<db6b5ecc0610061623o2aa4e8acn5a35cc4c6270ecb9@mail.gmail.com>
	<46cb515a0610070308h507e1df8wf671bcfb28a87c36@mail.gmail.com>
Message-ID: <db6b5ecc0610091107h77a3e9f9ud6cea56deb65430b@mail.gmail.com>

On 10/7/06, Ville M. Vainio <vivainio at gmail.com> wrote:
> On 10/7/06, Fernando Perez <fperez.net at gmail.com> wrote:

> > We should plan a 0.7.3 release soonish.  Ville: would you want this
> > debugging work to be part of that release?  How's your schedule
>
> Well, it's a great selling point for the new release, IMO. Should look
> great on some kind of "show me do" video, esp. since many don't know
> about pydb. :-)

Yes, a ShowMeDo video as part of the new release would be great.  I
know Ian has repeatedly encouraged contributions from the IPython
community, so if anyone can pick up the ball on this one, it would be
great.

I also think a short video would be a great way to introduce
/existing/ users to the neat features of a new release.  It often
happens that old users stick to what they know and never really
benefit from the good new features simply due to lack of time.  A
quick video may be an excellent way for them to realize what the
benefits of the new features are.

cheers,

f


From rocky at panix.com  Mon Oct  9 14:33:27 2006
From: rocky at panix.com (R. Bernstein)
Date: Mon, 9 Oct 2006 14:33:27 -0400
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <db6b5ecc0610091107h77a3e9f9ud6cea56deb65430b@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<17702.39964.389464.78803@panix3.panix.com>
	<db6b5ecc0610061623o2aa4e8acn5a35cc4c6270ecb9@mail.gmail.com>
	<46cb515a0610070308h507e1df8wf671bcfb28a87c36@mail.gmail.com>
	<db6b5ecc0610091107h77a3e9f9ud6cea56deb65430b@mail.gmail.com>
Message-ID: <17706.38391.388364.863106@panix3.panix.com>

I've never made a showmedo video; I'd prefer not to volunteer for this
if someone else can do it. On the other hand at a minimum I can
probably create a list of things to demo with pydb and ipython to show
what's improved.

Fernando Perez writes:
 > On 10/7/06, Ville M. Vainio <vivainio at gmail.com> wrote:
 > > On 10/7/06, Fernando Perez <fperez.net at gmail.com> wrote:
 > 
 > > > We should plan a 0.7.3 release soonish.  Ville: would you want this
 > > > debugging work to be part of that release?  How's your schedule
 > >
 > > Well, it's a great selling point for the new release, IMO. Should look
 > > great on some kind of "show me do" video, esp. since many don't know
 > > about pydb. :-)
 > 
 > Yes, a ShowMeDo video as part of the new release would be great.  I
 > know Ian has repeatedly encouraged contributions from the IPython
 > community, so if anyone can pick up the ball on this one, it would be
 > great.
 > 
 > I also think a short video would be a great way to introduce
 > /existing/ users to the neat features of a new release.  It often
 > happens that old users stick to what they know and never really
 > benefit from the good new features simply due to lack of time.  A
 > quick video may be an excellent way for them to realize what the
 > benefits of the new features are.
 > 
 > cheers,
 > 
 > f
 > 


From fperez.net at gmail.com  Mon Oct  9 14:44:39 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 9 Oct 2006 12:44:39 -0600
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <17706.38391.388364.863106@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<17702.39964.389464.78803@panix3.panix.com>
	<db6b5ecc0610061623o2aa4e8acn5a35cc4c6270ecb9@mail.gmail.com>
	<46cb515a0610070308h507e1df8wf671bcfb28a87c36@mail.gmail.com>
	<db6b5ecc0610091107h77a3e9f9ud6cea56deb65430b@mail.gmail.com>
	<17706.38391.388364.863106@panix3.panix.com>
Message-ID: <db6b5ecc0610091144u366a7258y8e694d709073ef5b@mail.gmail.com>

On 10/9/06, R. Bernstein <rocky at panix.com> wrote:
> I've never made a showmedo video; I'd prefer not to volunteer for this
> if someone else can do it. On the other hand at a minimum I can
> probably create a list of things to demo with pydb and ipython to show
> what's improved.

Don't worry, I wasn't suggesting /you/ do it: it was more a matter of
throwing the bait in the water, and waiting to see who takes it :)

But it could be a fun task for someone from the community who has a
bit of spare time in their hands.

When we get closer to release and the features are in place, we'll
suggest it also on the user list.  It would be nice if you put up your
list of ideas (a storyboard of sorts) in the wiki.  I think this would
increase the chance of someone actually doing it, if they feel there's
some guidelines already on where to start.

Cheers,

f


From stefan at sun.ac.za  Mon Oct  9 17:30:41 2006
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Mon, 9 Oct 2006 23:30:41 +0200
Subject: [IPython-dev] timeit bug
In-Reply-To: <db6b5ecc0610090113n1ddc6161heb40ce3f8cf76b0f@mail.gmail.com>
References: <20060915085341.GA1402@mentat.za.net>
	<46cb515a0609210156o613d9852u1139b79510b5291d@mail.gmail.com>
	<46cb515a0609210248i7dbb65deid4a1ce2d48a73774@mail.gmail.com>
	<db6b5ecc0610090113n1ddc6161heb40ce3f8cf76b0f@mail.gmail.com>
Message-ID: <20061009213041.GD4215@mentat.za.net>

On Mon, Oct 09, 2006 at 02:13:48AM -0600, Fernando Perez wrote:
> On 9/21/06, Ville M. Vainio <vivainio at gmail.com> wrote:
> >On 9/21/06, Ville M. Vainio <vivainio at gmail.com> wrote:
> >
> >> On 9/15/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> 
> [... %timeit print 'x' bug ]
> 
> The attached patch should be a proper fix.  It's late and I'm tired so
> I don't want to commit it yet; Brian is testing it because he also ran
> into similar problems and we need this fixed for a demo on Tuesday, so
> it's now kind of high priority.  But I don't want to commit a broken
> patch.

Thanks, Fernando!  That solves the problem for me.

Cheers
St?fan


From fperez.net at gmail.com  Tue Oct 10 00:47:04 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 9 Oct 2006 22:47:04 -0600
Subject: [IPython-dev] timeit bug
In-Reply-To: <20061009213041.GD4215@mentat.za.net>
References: <20060915085341.GA1402@mentat.za.net>
	<46cb515a0609210156o613d9852u1139b79510b5291d@mail.gmail.com>
	<46cb515a0609210248i7dbb65deid4a1ce2d48a73774@mail.gmail.com>
	<db6b5ecc0610090113n1ddc6161heb40ce3f8cf76b0f@mail.gmail.com>
	<20061009213041.GD4215@mentat.za.net>
Message-ID: <db6b5ecc0610092147x2f0e68ecj2db1ad6d7a54cd69@mail.gmail.com>

On 10/9/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> On Mon, Oct 09, 2006 at 02:13:48AM -0600, Fernando Perez wrote:
> > On 9/21/06, Ville M. Vainio <vivainio at gmail.com> wrote:
> > >On 9/21/06, Ville M. Vainio <vivainio at gmail.com> wrote:
> > >
> > >> On 9/15/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> >
> > [... %timeit print 'x' bug ]
> >
> > The attached patch should be a proper fix.  It's late and I'm tired so
> > I don't want to commit it yet; Brian is testing it because he also ran
> > into similar problems and we need this fixed for a demo on Tuesday, so
> > it's now kind of high priority.  But I don't want to commit a broken
> > patch.
>
> Thanks, Fernando!  That solves the problem for me.

Thanks for the feeedback.  Committed as [1815]

Cheers,

f


From rocky at panix.com  Wed Oct 11 23:30:45 2006
From: rocky at panix.com (R. Bernstein)
Date: Wed, 11 Oct 2006 23:30:45 -0400
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <17704.34960.967034.430310@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
Message-ID: <17709.46821.79113.757496@panix3.panix.com>

Although I don't like talking to myself, since no one else has said
anything on this topic, I've been ploughing on and here's what I've
found.

In order to make the object inspection routines more useful to an
application (like a debugger) which is run from ipython, in a test
version I've changed Magic._ofind to accept a namespace parameter.
The default is the current hard-coded list. Routines like _inspect,
pdoc, pdef, and pinfo have been modified to allow this optional
parameter. Having done this, the debugger routine becomes

    def do_pinfo(self, arg):
        """The debugger (or any application) equivalant of ?obj"""
        namespaces = [('Locals', self.curframe.f_locals),
                      ('Globals', self.curframe.f_globals)]
        __IPYTHON__.magic_pinfo("pinfo %s" % arg, namespaces=namespaces)

and similarly for do_pdef, and do_pdoc. Overall, pretty simple. 

Personally, I think a better organization though would be to pull out
the object inspection routines and make them their own class. The fact
that they are glommed with other routines like magic_lsmagic or
magic_autocall are just artifacts the current ipython interface naming
convention.


R. Bernstein writes:
 > I was not have been clear on what I was hoping to do.
 > 
 > Let's say I'm inside the debugger and I want to get the equivalent
 > information for a local object "a" that I would get were I in the
 > shell with that variable defined and were I to type "a?".
 > 
 > And let's forget about how to handle the syntax of ? or ??. So let's
 > add a new debugger command called "pinfo". All it wants to do is call
 > Magic.magic_pinfo(), but somehow set up the local and global
 > namespaces correctly. One could add to Debugger.py the following
 > method which sort of works, but really isn't right. It hackily
 > modifies one of the dictionaries that gets used via Magic.magic_pinfo.
 > 
 >     def do_pinfo(self, arg):
 >         """Sort of The debugger equivalant of obj? for a local variable"""
 >         # Find a dictionary to smack before the call. We'll use the
 >         # alias table.
 >         old_ns = __IPYTHON__.shell.alias_table   # save
 >         __IPYTHON__.shell.alias_table = self.curframe.f_locals  # smack
 >         __IPYTHON__.magic_pinfo("pinfo %s" % arg)
 >         __IPYTHON__.shell.alias_table = old_ns   # restore
 > 
 > Possibly the alias table shouldn't be wiped out -- I don't know
 > possibly it is irrelevant in this context anyway. And the alias table
 > is searched last whereas local variables are searched first by Python.
 > 
 > Instead I think one would like a variant of Magic._ofind which has a
 > notion of a local and global name spaces separate from the shell's
 > namespace and some way for a debugger to hook in the current
 > local/global namespaces to that. (Could the global namespace replace
 > user_ns of the shell? Maybe, I don't know.)
 > 
 > Note that such a routine as do_pinfo really has nothing to do with
 > pydb at all. Short of finding incidental misfeatures in pdb (or pydb
 > as I've been finding) which prevent this, to first approximation it
 > should work equally well in both. Or any debugger; or any user
 > application called from the ipython shell that want's to interact with
 > ipython for that matter.
 > 
 > - - - 
 > 
 > As a related but separate problem, now that I've go a way to invoke
 > the debugger in full generaltity from ipython I'd also like to be able
 > run such a "pinfo" command from that.  By virtue of starting inside
 > ipython I have all of the ipython routines around. In particular I
 > have __IPYTHON__ defined. So moving that same do_pinfo routine from
 > Debugger.py into someplace inside pydb's code would roughly do the
 > same thing. But again, that code is really fairly wrong for the same
 > reasons.
 > 
 > 
 > Ville M. Vainio writes:
 >  > On 10/6/06, R. Bernstein <rocky at panix.com> wrote:
 >  > 
 >  > > ??object here). This could either be done inside pydb code, but maybe
 >  > > it would be neater to do inside ipython since it looks like there are
 >  > > routines already for decorating debugger commands. Suggestions on
 >  > > where to look to hook into '?' and '??'?
 >  > 
 >  > Now they are handled by "handle_help" in iplib.py, but you should
 >  > create an input_prefilter hook function that catches the ? commands
 >  > and translates them to something you want, e.g. ??foo ->
 >  > _pydb.detailedhelpcommand("foo"). see ext_rescapture.py on how to
 >  > implement one.
 >  > 
 >  > -- 
 >  > Ville M. Vainio - vivainio.googlepages.com
 >  > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
 >  > 
 > _______________________________________________
 > IPython-dev mailing list
 > IPython-dev at scipy.org
 > http://projects.scipy.org/mailman/listinfo/ipython-dev
 > 


From vivainio at gmail.com  Thu Oct 12 02:17:01 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 12 Oct 2006 09:17:01 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <17709.46821.79113.757496@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
Message-ID: <46cb515a0610112317s4562f7a3la052466e880a45c0@mail.gmail.com>

On 10/12/06, R. Bernstein <rocky at panix.com> wrote:

>     def do_pinfo(self, arg):
>         """The debugger (or any application) equivalant of ?obj"""
>         namespaces = [('Locals', self.curframe.f_locals),
>                       ('Globals', self.curframe.f_globals)]

Any particular reasons for using tuple for namespaces instead of
separate kwargs?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Thu Oct 12 07:46:20 2006
From: rocky at panix.com (R. Bernstein)
Date: Thu, 12 Oct 2006 07:46:20 -0400
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610112317s4562f7a3la052466e880a45c0@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610112317s4562f7a3la052466e880a45c0@mail.gmail.com>
Message-ID: <17710.11020.542720.7991@panix3.panix.com>

Ville M. Vainio writes:
 > On 10/12/06, R. Bernstein <rocky at panix.com> wrote:
 > 
 > >     def do_pinfo(self, arg):
 > >         """The debugger (or any application) equivalant of ?obj"""
 > >         namespaces = [('Locals', self.curframe.f_locals),
 > >                       ('Globals', self.curframe.f_globals)]
 > 
 > Any particular reasons for using tuple for namespaces instead of
 > separate kwargs?


I'm not sure exactly what you mean. If you mean use:

        __IPYTHON__.magic_pinfo("pinfo %s" % arg, 
	                        namespaces=[('Locals', self.curframe.f_locals),
                      ('Globals', self.curframe.f_globals)])
rather than:
        namespaces = [('Locals', self.curframe.f_locals),
                      ('Globals', self.curframe.f_globals)]
        __IPYTHON__.magic_pinfo("pinfo %s" % arg, namespaces=namespaces)

I find the latter a little easier to read because it is less
cluttered. This kind of stuff reminds me of similar discussions among
C programmers of between using:
    if ( 'x' == c=getc() )
versus:
    c=getc()
   if ( 'x' == c )

Which variant is used is a matter of style. Although some style guides
recommend the latter, especially when the assignment is more complex,
many folks prefer the former. Personally, I really don't care (in both
C or Python).

If you mean something else, I don't doubt there may be cleverer ways
to do what I want: allow Magic._ofind to get passed a namespace
parameter -- currently an indirect call from magic_pinfo() via
_inspect() -- so that _ofind uses the supplied namespaces known by
do_pinfo() instead of the current hard-coded values which make sense
only in a shell.

So if you could just make the change, however you want to do it, that
would be great! Then debuggers (pdb and pydb) and other similar kinds
of applications would then have access to more of the good stuff from
ipython.

And speaking of changes how are the first set of patches coming along?
If I recall correctly you said last Tuesday you might look at.. :-)

Thanks.


From rocky at panix.com  Thu Oct 12 08:21:34 2006
From: rocky at panix.com (R. Bernstein)
Date: Thu, 12 Oct 2006 08:21:34 -0400
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610112317s4562f7a3la052466e880a45c0@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610112317s4562f7a3la052466e880a45c0@mail.gmail.com>
Message-ID: <17710.13134.871314.729500@panix3.panix.com>

Ville M. Vainio writes:
 > On 10/12/06, R. Bernstein <rocky at panix.com> wrote:
 > 
 > >     def do_pinfo(self, arg):
 > >         """The debugger (or any application) equivalant of ?obj"""
 > >         namespaces = [('Locals', self.curframe.f_locals),
 > >                       ('Globals', self.curframe.f_globals)]
 > 
 > Any particular reasons for using tuple for namespaces instead of
 > separate kwargs?

It occured to me that you might be asking why namespaces is this funny
list of tuples. If that's what you mean, it's because that's the
structure currently used in Magic._ofind.

When I first read that code I thought it odd. Personally I think
_ofind should be rewritten using a simple dictionary, e.g:

      namespaces = { 'Interactive': self.shell.user_ns, ... }
rather than 
      namespaces = [ ('Interactive', self.shell.user_ns), ... ]

I didn't do this initially because, well, I'm currently backed up in
patches, what I have works and I don't want to destroy that, and
adding my own version control on top of SVN checked-out sources is a
bit whacky.

Which reminds me, in _ofind, any particular reasons why 1 and 0 are
used for is_alias (and other seemingly boolean variables) rather than
True/False? ;-)


From vivainio at gmail.com  Thu Oct 12 11:23:22 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 12 Oct 2006 18:23:22 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <17709.46821.79113.757496@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
Message-ID: <46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>

On 10/12/06, R. Bernstein <rocky at panix.com> wrote:

> Although I don't like talking to myself, since no one else has said
> anything on this topic, I've been ploughing on and here's what I've
> found.

Yeah, a lot has been happening this last week (some of which I am only
able to speak properly next week, but might involve much more python
coding in my day job) and I haven't tested pydb yet. Sorry :-).

> In order to make the object inspection routines more useful to an
> application (like a debugger) which is run from ipython, in a test
> version I've changed Magic._ofind to accept a namespace parameter.
> The default is the current hard-coded list. Routines like _inspect,
> pdoc, pdef, and pinfo have been modified to allow this optional
> parameter. Having done this, the debugger routine becomes
>
>     def do_pinfo(self, arg):
>         """The debugger (or any application) equivalant of ?obj"""
>         namespaces = [('Locals', self.curframe.f_locals),
>                       ('Globals', self.curframe.f_globals)]
>         __IPYTHON__.magic_pinfo("pinfo %s" % arg, namespaces=namespaces)
>
> and similarly for do_pdef, and do_pdoc. Overall, pretty simple.
>
> Personally, I think a better organization though would be to pull out
> the object inspection routines and make them their own class. The fact
> that they are glommed with other routines like magic_lsmagic or
> magic_autocall are just artifacts the current ipython interface naming
> convention.

They are in the wrong place, yes, but such refactoring may not be
optimal for the "stable" branch. Instead, I'd like to see your patches
for kludging the _ofind, _inspect et. al. with optional 'namespaces'
keyword argument. I have no opposition to getting them in.

So send out patches for whatever you've got in your working copy
(preferably by friday) and I'll take a proper look at them.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Thu Oct 12 11:35:39 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 12 Oct 2006 18:35:39 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <17710.13134.871314.729500@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610112317s4562f7a3la052466e880a45c0@mail.gmail.com>
	<17710.13134.871314.729500@panix3.panix.com>
Message-ID: <46cb515a0610120835v312e0ecci6bf8467596237aca@mail.gmail.com>

On 10/12/06, R. Bernstein <rocky at panix.com> wrote:

> I didn't do this initially because, well, I'm currently backed up in
> patches, what I have works and I don't want to destroy that, and
> adding my own version control on top of SVN checked-out sources is a
> bit whacky.

Not really all that whacky, esp. if the svn is for the "official,
stable version" that is expected to work rather predictably. Too bad
svn is not a distributed vcs...

I recommend bzr (bazaar-ng) for your "own", on-top-of-svn version
control, works really well with local versions of files and no need
for repository administration.

See http://doc.bazaar-vcs.org/bzr.dev/tutorial.htm if you are interested.

> Which reminds me, in _ofind, any particular reasons why 1 and 0 are
> used for is_alias (and other seemingly boolean variables) rather than
> True/False? ;-)

I recall reading somewhere that 1 and 0 are "officially ok" as boolean
values. The fact that True and False are rather recent in python
speaks for itself...

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Thu Oct 12 12:06:31 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 12 Oct 2006 19:06:31 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
Message-ID: <46cb515a0610120906t10eee44cm47f5adb973f493cd@mail.gmail.com>

On 10/12/06, Ville M. Vainio <vivainio at gmail.com> wrote:

> coding in my day job) and I haven't tested pydb yet.

Ok, I'm trying to test it now.

First note: there appears to be no reason to not have a functional setup.py.

Attached is an eggsetup.py file that appears to install pydb quite
well, at least on this windows box. Just do "python eggsetup.py
install" (you do need to have setuptools installed, but
python-setuptools (or sth. like that) is apt-gettable and yummable on
most distros).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Thu Oct 12 12:08:12 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 12 Oct 2006 19:08:12 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610120906t10eee44cm47f5adb973f493cd@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<46cb515a0610120906t10eee44cm47f5adb973f493cd@mail.gmail.com>
Message-ID: <46cb515a0610120908o167d4479ye6a534a45ea621f@mail.gmail.com>

On 10/12/06, Ville M. Vainio <vivainio at gmail.com> wrote:

> Attached is an eggsetup.py file that appears to install pydb quite

Well, *now* it's attached.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: eggsetup.py
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061012/22f2a020/attachment.ksh>

From vivainio at gmail.com  Thu Oct 12 12:37:49 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 12 Oct 2006 19:37:49 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610120908o167d4479ye6a534a45ea621f@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<46cb515a0610120906t10eee44cm47f5adb973f493cd@mail.gmail.com>
	<46cb515a0610120908o167d4479ye6a534a45ea621f@mail.gmail.com>
Message-ID: <46cb515a0610120937u3e8efb3fr9d52444d7ab1db0c@mail.gmail.com>

On 10/12/06, Ville M. Vainio <vivainio at gmail.com> wrote:

> On 10/12/06, Ville M. Vainio <vivainio at gmail.com> wrote:
>
> > Attached is an eggsetup.py file that appears to install pydb quite
>
> Well, *now* it's attached.

And as a side note, if you have setuptools installed you should be
able to upload it to cheeseshop trivially so it can be installed
directly by "easy_install pydb":

$ python eggsetup.py bdist_egg upload
$ python eggsetup.py sdist upload

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Thu Oct 12 17:01:40 2006
From: rocky at panix.com (R. Bernstein)
Date: Thu, 12 Oct 2006 17:01:40 -0400
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
Message-ID: <17710.44340.520249.414290@panix3.panix.com>

Ville M. Vainio writes:
  ...
 > Yeah, a lot has been happening this last week (some of which I am only
 > able to speak properly next week, but might involve much more python
 > coding in my day job) and I haven't tested pydb yet. Sorry :-).
  .. 
 > They are in the wrong place, yes, but such refactoring may not be
 > optimal for the "stable" branch. Instead, I'd like to see your patches
 > for kludging the _ofind, _inspect et. al. with optional 'namespaces'
 > keyword argument. I have no opposition to getting them in.
 > 
 > So send out patches for whatever you've got in your working copy
 > (preferably by friday) and I'll take a proper look at them.
 > 

By Friday? Gee, that doesn't leave much time, so I'll send what I have
now. Let's see, a summary of the patches. First there's what I sent
before and in private email I sent a slight code improvement on that.

In that we allowed "%pdb on" and -pdb and pm to call pydb if that's
installed and Python 2.5

Above this there's 

- decorating do_up, do_down so they get the
  docstrings from pydb (there are none in pdb so it's a no op). Also 
  do_frame decorated, but again in pdb there's no "frame" command. 

- Changing the list command so that it accepts pydb/gdb syntax by decoupling
  the parsing part from the listing part

- Allowing magic_pinfo, magic_pdef, and magic_pdoc to be used inside the
  debugger. 

- There's some stuff using command pydb's completion, and this helps a
  little but private email with Alexander Schmolc suggests there may
  be some stygian stable work in (re)doing the issue of making command
  completion work correctly.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pydb.patch
Type: application/octet-stream
Size: 13940 bytes
Desc: patches to make ipython work with pydb (and other good	stuff)
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061012/727dc8d1/attachment.obj>

From Fernando.Perez at colorado.edu  Fri Oct 13 10:29:33 2006
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Fri, 13 Oct 2006 08:29:33 -0600
Subject: [IPython-dev] [Fwd: Re: making pdoc, pdef,
	and pinfo more generally useful]
Message-ID: <452FA2CD.2090505@colorado.edu>

Thanks for the suggestions.  Note that I'm forwarding this manually to the 
list since it discards non-subscriber posts (too much spam).

Cheers,

f

-------- Original Message --------
Subject:
Re: making pdoc, pdef, and pinfo more generally useful
From:
Stefan Reich?r <stefan at xsteve.at>
Date:
Fri, 13 Oct 2006 11:13:28 +0200
To:
ipython-dev at scipy.net

"Ville M. Vainio" <vivainio at gmail.com> writes:

 > > On 10/12/06, R. Bernstein <rocky at panix.com> wrote:
 > >
 >> >> I didn't do this initially because, well, I'm currently backed up in
 >> >> patches, what I have works and I don't want to destroy that, and
 >> >> adding my own version control on top of SVN checked-out sources is a
 >> >> bit whacky.
 > >
 > > Not really all that whacky, esp. if the svn is for the "official,
 > > stable version" that is expected to work rather predictably. Too bad
 > > svn is not a distributed vcs...
 > >
 > > I recommend bzr (bazaar-ng) for your "own", on-top-of-svn version
 > > control, works really well with local versions of files and no need
 > > for repository administration.
 > >
 > > See http://doc.bazaar-vcs.org/bzr.dev/tutorial.htm if you are interested.

I also think that bzr is a good solution for that kind of stuff.
However, I use Mercurial Queues (mq) for tracking svn repositories and
generating patches.

Since R. Bernstein uses Emacs, let me do some advertisement for emacs dvc:
http://www.xsteve.at/prg/emacs_dvc/index.html

I have added emacs support for mq. That way I can easily build patches
for upstream projects.

If anyone would like to try this setup, feel free to ask  ;-)


Stefan.



From vivainio at gmail.com  Fri Oct 13 11:15:29 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 13 Oct 2006 18:15:29 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <17710.44340.520249.414290@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<17710.44340.520249.414290@panix3.panix.com>
Message-ID: <46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>

On 10/13/06, R. Bernstein <rocky at panix.com> wrote:

> By Friday? Gee, that doesn't leave much time, so I'll send what I have
> now. Let's see, a summary of the patches. First there's what I sent

I committed only part of your patch (Magic.py and ipython.el) because:

1) iplib.py had a mistake I think - it doesn't account for pydb not
existing properly:

       have_pydb = False
        if sys.version[:3] >= '2.5':
            try:
                from pydb import pm
                have_pydb = True
            except ImportError:
                pass
        if not have_pydb:
            from pydb import pm   <======
        pm()

didn't you mean to say "from pdb import pm"?

2) Debugger.py patch was invalid for some reason (for eclipse's "apply
patch" tool). Was it done properly against the current svn? Please do
svn up again, resolve conflicts and send a new patch.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Fri Oct 13 12:38:22 2006
From: rocky at panix.com (R. Bernstein)
Date: Fri, 13 Oct 2006 12:38:22 -0400
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.4195.540436.694402@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<17710.44340.520249.414290@panix3.panix.com>
	<46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>
Message-ID: <17711.49406.813910.224951@panix3.panix.com>

A revised patch follows. I apologize for the problem and inconvenience
of the previous patch.

Many thanks.

Ville M. Vainio writes:
 ...
 > 1) iplib.py had a mistake I think - it doesn't account for pydb not
 > existing properly:
 > 
 >        have_pydb = False
 >         if sys.version[:3] >= '2.5':
 >             try:
 >                 from pydb import pm
 >                 have_pydb = True
 >             except ImportError:
 >                 pass
 >         if not have_pydb:
 >             from pydb import pm   <======
 >         pm()
 > 
 > didn't you mean to say "from pdb import pm"?

You are absolutely correct. 

 > 
 > 2) Debugger.py patch was invalid for some reason (for eclipse's "apply
 > patch" tool). Was it done properly against the current svn? Please do
 > svn up again, resolve conflicts and send a new patch.

Should be included below.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pydb-take2.patch
Type: application/octet-stream
Size: 9452 bytes
Desc: Remaining Debugger and iplib.py patches.
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061013/a9814409/attachment.obj>

From rocky at panix.com  Fri Oct 13 13:14:20 2006
From: rocky at panix.com (R. Bernstein)
Date: Fri, 13 Oct 2006 13:14:20 -0400
Subject: [IPython-dev] A maze of twisty little version control systems,
	mostly alike (Was [Fwd: Re: making pdoc, pdef,
	)	and pinfo more generally useful]
In-Reply-To: <452FA2CD.2090505@colorado.edu>
References: <452FA2CD.2090505@colorado.edu>
Message-ID: <17711.51564.957737.449514@panix3.panix.com>

First, thanks everyone for the suggestions on how to apply version
control on top of another version control system.

>From Stefan van der Walt's suggestion and help (sent in private
email), I have bzr running on top of the ipython svn. So the problem I
was having with backed-up patches shouldn't be an issue in the
future. Thanks Stefan v.d.W!

However I will also look into Mercurial Queues as suggested by Stefan
Reichoer. I currently use SVN, CVS, Perforce, RCS, SCCS (in the past),
and now bzr. Other than bitkeeper and GNU Arch, I wasn't aware there
*were* any more version control systems. I can't wait for a half a
dozen more to come on the scene so we can learn those as well.

But also many thanks to Stefan Reichoer thanks for mentioning and
writing dvc. I've installed that. (One little suggestion though for
the INSTALL file. A small step is missing if you get the package via
"bzr get http://bzr.xsteve.at/dvc" - I needed to run autoconf to build
configure from configure.ac)

I also noticed that Stefan R. has also written an Emacs SVN interface
too! As a PCL-CVS user This is something I've long wanted. (And it had
not escaped my careful and keep powers of observation that ipython
uses SVN.)

In sum, I'm learning a great deal from reading the ipython-dev forum.

Again - thanks.


Fernando Perez writes:
 > Thanks for the suggestions.  Note that I'm forwarding this manually to the 
 > list since it discards non-subscriber posts (too much spam).
 > 
 > Cheers,
 > 
 > f
 > 
 > -------- Original Message --------
 > Subject:
 > Re: making pdoc, pdef, and pinfo more generally useful
 > From:
 > Stefan Reich?r <stefan at xsteve.at>
 > Date:
 > Fri, 13 Oct 2006 11:13:28 +0200
 > To:
 > ipython-dev at scipy.net
 > 
 > "Ville M. Vainio" <vivainio at gmail.com> writes:
 > 
 >  > > On 10/12/06, R. Bernstein <rocky at panix.com> wrote:
 >  > >
 >  >> >> I didn't do this initially because, well, I'm currently backed up in
 >  >> >> patches, what I have works and I don't want to destroy that, and
 >  >> >> adding my own version control on top of SVN checked-out sources is a
 >  >> >> bit whacky.
 >  > >
 >  > > Not really all that whacky, esp. if the svn is for the "official,
 >  > > stable version" that is expected to work rather predictably. Too bad
 >  > > svn is not a distributed vcs...
 >  > >
 >  > > I recommend bzr (bazaar-ng) for your "own", on-top-of-svn version
 >  > > control, works really well with local versions of files and no need
 >  > > for repository administration.
 >  > >
 >  > > See http://doc.bazaar-vcs.org/bzr.dev/tutorial.htm if you are interested.
 > 
 > I also think that bzr is a good solution for that kind of stuff.
 > However, I use Mercurial Queues (mq) for tracking svn repositories and
 > generating patches.
 > 
 > Since R. Bernstein uses Emacs, let me do some advertisement for emacs dvc:
 > http://www.xsteve.at/prg/emacs_dvc/index.html
 > 
 > I have added emacs support for mq. That way I can easily build patches
 > for upstream projects.
 > 
 > If anyone would like to try this setup, feel free to ask  ;-)
 > 
 > 
 > Stefan.


From vivainio at gmail.com  Fri Oct 13 15:38:13 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 13 Oct 2006 22:38:13 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <17711.49406.813910.224951@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<db6b5ecc0610060120n174090a6p2a4ef2172774a5a6@mail.gmail.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<17710.44340.520249.414290@panix3.panix.com>
	<46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>
	<17711.49406.813910.224951@panix3.panix.com>
Message-ID: <46cb515a0610131238sf9ec617yf1763a5818cca17e@mail.gmail.com>

On 10/13/06, R. Bernstein <rocky at panix.com> wrote:

> A revised patch follows.
....

Now (after applying patches), when I quit ipydb after postmortem
debugging (%pdb 1 and %run whichpy.py):

ipydb>q
*** ERROR ***
This version of pdb has a bug and crashed.
Returning to IPython...

Any idea what's going on?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Fri Oct 13 16:11:47 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 13 Oct 2006 23:11:47 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610131238sf9ec617yf1763a5818cca17e@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<17710.44340.520249.414290@panix3.panix.com>
	<46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>
	<17711.49406.813910.224951@panix3.panix.com>
	<46cb515a0610131238sf9ec617yf1763a5818cca17e@mail.gmail.com>
Message-ID: <46cb515a0610131311m6a7dc816u8b89b2c2af9ce291@mail.gmail.com>

On 10/13/06, Ville M. Vainio <vivainio at gmail.com> wrote:
> On 10/13/06, R. Bernstein <rocky at panix.com> wrote:
>
> > A revised patch follows.
> ....
>
> Now (after applying patches), when I quit ipydb after postmortem
> debugging (%pdb 1 and %run whichpy.py):
>
> ipydb>q
> *** ERROR ***
> This version of pdb has a bug and crashed.
> Returning to IPython...

Ok, I got over that but now I get:

[lscripts]|7> %run -d whichpy.py cp
---------------------------------------------------------------------------
<type 'exceptions.AttributeError'>        Traceback (most recent call last)
q:\trunk\lscripts\<ipython console> in <module>()
Q:\ipython\IPython\iplib.py in
ipmagic(self=<IPython.iplib.InteractiveShell object at 0x00A2D4F0>,
arg_s='%run -d whichpy.py cp')
    887         else:
    888             magic_args = self.var_expand(magic_args)
--> 889             return fn(magic_args)
        fn = <bound method InteractiveShell.magic_run of
<IPython.iplib.InteractiveShell object at 0x00A2D4F0>>
        magic_args = '-d whichpy.py cp'
    890
    891     def ipalias(self,arg_s):
Q:\ipython\IPython\Magic.py in
magic_run(self=<IPython.iplib.InteractiveShell object at 0x00A2D4F0>,
parameter_s='-d whichpy.py cp', runner=None)
   1515                     maxtries = 10
   1516                     bp = int(opts.get('b',[1])[0])
-> 1517                     checkline = deb.checkline(filename,bp)
        checkline = undefined
        deb.checkline = undefined
        filename = 'whichpy.py'
        bp = 1
   1518                     if not checkline:
   1519                         for bp in range(bp+1,bp+maxtries+1):
<type 'exceptions.AttributeError'>: Pdb instance has no attribute 'checkline'

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Fri Oct 13 18:48:35 2006
From: rocky at panix.com (R. Bernstein)
Date: Fri, 13 Oct 2006 18:48:35 -0400
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610131311m6a7dc816u8b89b2c2af9ce291@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17702.11689.407083.959718@panix3.panix.com>
	<46cb515a0610070314r216949b0od51ad26a69c8df2b@mail.gmail.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<17710.44340.520249.414290@panix3.panix.com>
	<46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>
	<17711.49406.813910.224951@panix3.panix.com>
	<46cb515a0610131238sf9ec617yf1763a5818cca17e@mail.gmail.com>
	<46cb515a0610131311m6a7dc816u8b89b2c2af9ce291@mail.gmail.com>
Message-ID: <17712.6083.522575.256388@panix3.panix.com>

Ville M. Vainio writes:
 > On 10/13/06, Ville M. Vainio <vivainio at gmail.com> wrote:
 > > On 10/13/06, R. Bernstein <rocky at panix.com> wrote:
 > >
 > > > A revised patch follows.
 > > ....
 > >
 > > Now (after applying patches), when I quit ipydb after postmortem
 > > debugging (%pdb 1 and %run whichpy.py):
 > >
 > > ipydb>q
 > > *** ERROR ***
 > > This version of pdb has a bug and crashed.
 > > Returning to IPython...
 > >
 > > Any idea what's going on?
 > 
 > Ok, I got over that 

Good. What was the problem? I didn't encounter that when I initially
tried (which means the patch below might have the same problem).


> but now I get:

 > 
 > [lscripts]|7> %run -d whichpy.py cp
 > ---------------------------------------------------------------------------
...
 >    1518                     if not checkline:
 >    1519                         for bp in range(bp+1,bp+maxtries+1):
 > <type 'exceptions.AttributeError'>: Pdb instance has no attribute 'checkline'

Oy. 

When I look at that code in magic_run to set up a program to
debug it, I kringe. It looks to me like a couple of problems in pdb
are getting worked around, but I'm not really sure what those problems
are.

Something's probably wrong. Sure, maybe in the past you couldn't get
pdb changed all that quickly, but with pydb that's not the case. Talk
to me!

What I had been using to debug a program was this:

  import pydb
  pydb.runl("debug-opt1", "debug-opt2", "program", "prog-opt1", ...)

This works whether you are in ipython, the stock python shell, or
running out of your own Python program not run from a shell.

What ipython's "run -d" doesn't give you is the ablity to pass options
to the debugger. For pdb that's not an issue because, well, pdb
doesn't *have* any debugger options. 
 
That said, for now we'll limp things along. The particular error you
encountered has to do with a somewhat general function (testing to see
if a line is someplace you can put a breakpoint) which was moved out
of the debugger class into a class of auxiliary functions. If
Magic.magic_run wants to call that function in order to figure out
where to put a breakpoint, so be it. I've added that as a lambda
function in the Debugger __init__. 

But of course there were a number of other problems. For some of them
changes were more appropriate in pydb than in ipython. And as a
result, I think initially I reduce the scope even more. You need the
latest version of pydb (and Python 2.5). 

For now and testing use what's just been checked in to CVS.

Here is a revised set of patches off of the current SVN.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pydb-take3.patch
Type: application/octet-stream
Size: 10309 bytes
Desc: 3rd try at patches for pydb support.
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061013/1e3ed87d/attachment.obj>

From vivainio at gmail.com  Sat Oct 14 02:42:06 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 14 Oct 2006 09:42:06 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <17712.6083.522575.256388@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<17710.44340.520249.414290@panix3.panix.com>
	<46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>
	<17711.49406.813910.224951@panix3.panix.com>
	<46cb515a0610131238sf9ec617yf1763a5818cca17e@mail.gmail.com>
	<46cb515a0610131311m6a7dc816u8b89b2c2af9ce291@mail.gmail.com>
	<17712.6083.522575.256388@panix3.panix.com>
Message-ID: <46cb515a0610132342u285345ecoa43732c8eaa62314@mail.gmail.com>

On 10/14/06, R. Bernstein <rocky at panix.com> wrote:

> But of course there were a number of other problems. For some of them
> changes were more appropriate in pydb than in ipython. And as a
> result, I think initially I reduce the scope even more. You need the
> latest version of pydb (and Python 2.5).
>
> For now and testing use what's just been checked in to CVS.
>
> Here is a revised set of patches off of the current SVN.

I'll apply them sometime next week. In the meantime, give it some more
testing to see whether there are more gotchas.

The previous bug was fixed by this:

     def new_do_quit(self, arg):
-        __IPYTHON__.Completer.all_completions=self.old_all_completions
+
+        if hasattr(self, 'all_completions'):
+            __IPYTHON__.Completer.all_completions=self.old_all_completions
         return OldPdb.do_quit(self, arg)

(it was AttributeError - there was no old_all_completions. Also see
whether there is a cleaner fix).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Sat Oct 14 06:21:12 2006
From: rocky at panix.com (R. Bernstein)
Date: Sat, 14 Oct 2006 06:21:12 -0400
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610132342u285345ecoa43732c8eaa62314@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<17710.44340.520249.414290@panix3.panix.com>
	<46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>
	<17711.49406.813910.224951@panix3.panix.com>
	<46cb515a0610131238sf9ec617yf1763a5818cca17e@mail.gmail.com>
	<46cb515a0610131311m6a7dc816u8b89b2c2af9ce291@mail.gmail.com>
	<17712.6083.522575.256388@panix3.panix.com>
	<46cb515a0610132342u285345ecoa43732c8eaa62314@mail.gmail.com>
Message-ID: <17712.47640.683214.100950@panix3.panix.com>

Ville M. Vainio writes:
 > I'll apply them sometime next week. In the meantime, give it some more
 > testing to see whether there are more gotchas.

Sure. Will do. As explained before, part (but not all) of the problem
is that you were using something I wasn't really aware of: run -d.
What else is there to test? (See below for what I tried.)

Is there a regression test or unit test framework for ipython? That's
what I use in pydb to make sure bugs don't creep in and I've found it
extremely effective.

What I tested was %pdb (and option -pdb) and post-mortem
debugging. And the pydb.runl() command I added. I've now tried "run
-d", although it has the weakness mentioned before. It also has a
weakness in that pydb debugger commands "run" and "restart" don't
work; to fix them up incrementally means going back into that awkward
code in Magic.magic_run(), shared by pdb. Since pdb doesn't have
"restart" or "run" commands, there's no *loss* for folks who use
ipython's "run -d". So my take is to let that one go for now, and have
folks use pydb.runl() if they want some sort of restart.

And speaking of that hacky magic_run code, it occurs to me that it is
also a bit wrong for debugging. It sets a permanent break on what it
determines is the first line of the Python script. So if the first
executable line is say a loop, you keep stopping at that line which I
don't think was the intent. Better would be a *temporary* breakpoint.
(And even better to remove altogether for pydb by understanding
problem(s) this code trying to address.)

 > The previous bug was fixed by this:
 > 
 >      def new_do_quit(self, arg):
 > -        __IPYTHON__.Completer.all_completions=self.old_all_completions
 > +
 > +        if hasattr(self, 'all_completions'):
 > +            __IPYTHON__.Completer.all_completions=self.old_all_completions
 >          return OldPdb.do_quit(self, arg)
 > 
 > (it was AttributeError - there was no old_all_completions. 

Interesting for a couple of reasons. First, yes that bug is still in
the patch I sent. So I'm glad it was brought up. Second, if you found
this in testing *with* pydb installed, it means you weren't using the
current CVS sources. Again, in the last patch I've narrowed the scope
of changes so that you now *have* to use the lastest sources,
currently only in CVS if you want pydb. Again, my take is to make a
pydb release along with the ipython release.

 > Also see
 > whether there is a cleaner fix).

Well, since this is only needed for pydb, you could put that in the
pydb initialization: 

            ...
            if self.is_pydb:
               self.do_q = self.do_quit = decorate_fn_with_doc(self.new_do_quit,
                                                               OldPdb.do_quit)

But what's there isn't so bad either.


From hans_meine at gmx.net  Sat Oct 14 07:31:48 2006
From: hans_meine at gmx.net (Hans Meine)
Date: Sat, 14 Oct 2006 13:31:48 +0200
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610132342u285345ecoa43732c8eaa62314@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17712.6083.522575.256388@panix3.panix.com>
	<46cb515a0610132342u285345ecoa43732c8eaa62314@mail.gmail.com>
Message-ID: <200610141331.54330.hans_meine@gmx.net>

On Saturday 14 October 2006 08:42, Ville M. Vainio wrote:
> The previous bug was fixed by this:
>
>      def new_do_quit(self, arg):
> -        __IPYTHON__.Completer.all_completions=self.old_all_completions
> +
> +        if hasattr(self, 'all_completions'):
> +            __IPYTHON__.Completer.all_completions=self.old_all_completions
>          return OldPdb.do_quit(self, arg)
>
> (it was AttributeError - there was no old_all_completions. Also see
> whether there is a cleaner fix).

Then why don't you check for "old_all_completions" but for "all_completions"?
If that's an actual "svn diff" output I would think this fix is not necessary, 
since it cannot have fixed that problem. ;-p

Greetings,
  Hans
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061014/ec6801ca/attachment.sig>

From rocky at panix.com  Sat Oct 14 07:38:37 2006
From: rocky at panix.com (R. Bernstein)
Date: Sat, 14 Oct 2006 07:38:37 -0400
Subject: [IPython-dev] Patches to use pydb instead of pdb for debugging
In-Reply-To: <200610141331.54330.hans_meine@gmx.net>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17712.6083.522575.256388@panix3.panix.com>
	<46cb515a0610132342u285345ecoa43732c8eaa62314@mail.gmail.com>
	<200610141331.54330.hans_meine@gmx.net>
Message-ID: <17712.52285.288370.534260@panix3.panix.com>

Hans Meine writes:
 > On Saturday 14 October 2006 08:42, Ville M. Vainio wrote:
 > > The previous bug was fixed by this:
 > >
 > >      def new_do_quit(self, arg):
 > > -        __IPYTHON__.Completer.all_completions=self.old_all_completions
 > > +
 > > +        if hasattr(self, 'all_completions'):
 > > +            __IPYTHON__.Completer.all_completions=self.old_all_completions
 > >          return OldPdb.do_quit(self, arg)
 > >
 > > (it was AttributeError - there was no old_all_completions. Also see
 > > whether there is a cleaner fix).
 > 
 > Then why don't you check for "old_all_completions" but for "all_completions"?
 > If that's an actual "svn diff" output I would think this fix is not necessary, 
 > since it cannot have fixed that problem. ;-p

Yes, although what you suggest is a more obvious, direct, and robust
fix, Ville's patch I do believe was correct. You see
old_all_completions was set in only if hasattr(self,
'all_completions') was true. This was set in the Debugger __init__
routine.



From stefan at xsteve.at  Mon Oct 16 03:13:59 2006
From: stefan at xsteve.at (Stefan =?iso-8859-1?Q?Reich=F6r?=)
Date: Mon, 16 Oct 2006 09:13:59 +0200
Subject: [IPython-dev] Different behaviour: python vs. ipython when using a
	C module
Message-ID: <m0ejt8ybpk.fsf@ge-research.com>

Hi!

I use python in C/C++ Applications for scripting.

I have implemented a C/C++ library that is used in that application.
To test that library, I use python (preferred ipython).
That works very well.

Now I have added an embedded python interpreter to the C/C++ library.
That interpreter makes the library very easy customizable.

But with this setup I discovered a different behaviour when I compare
python with ipython. The python version works like I expect. When I
use ipython, it does not work.

Now I'd like to find out, if this is a bug in ipython, that can be
fixed, or if I do something wrong.

I have stripped down my project to a minimal example that fails.

I create a C extension called testmodule. This extension initializes
an additional module: the embedmodule. Importing embedmodule does work
in python, but it does not work in iypthon!

PyObject *test_func1(PyObject *self, PyObject *args)
{
  printf("testmodule.func1 called :-)\n");
  return Py_None;
}

static PyMethodDef testmoduleAccessMethods[] = {
  {"func1",           test_func1, METH_VARARGS},
  {NULL,              NULL}        /* Sentinel */
};


PyObject *embed_func2(PyObject *self, PyObject *args)
{
  printf("embedmodule.func2 called :-)\n");
  return Py_None;
}

static PyMethodDef embedmoduleAccessMethods[] = {
  {"func2",           embed_func2, METH_VARARGS},
  {NULL,              NULL}        /* Sentinel */
};

void inittestmodule() {
  Py_Initialize();
  Py_InitModule("testmodule", testmoduleAccessMethods);

  Py_InitModule("embedmodule",embedmoduleAccessMethods);
  PyImport_ImportModule("embedmodule");

  printf("Before import embedmodule\n");
  PyRun_SimpleString("import embedmodule");
  printf("After import embedmodule\n");

  PyRun_SimpleString("embedmodule.func2()");
}

When I run the example via python, I can do the following:
% python -i test-embed.py                                                                                                                                            9:05
Before import embedmodule
After import embedmodule
embedmodule.func2 called :-)
>>> dir()
['__builtins__', '__doc__', '__file__', '__name__', 'embedmodule', 'testmodule']
>>> dir(embedmodule)
['__doc__', '__name__', 'func2']
>>> dir(testmodule)
['__doc__', '__file__', '__name__', 'func1']



The following happens, when I use ipython:
% ipython test-embed.py                                                                                                                                              9:09
Before import embedmodule
---------------------------------------------------------------------------
exceptions.ImportError                               Traceback (most recent call last)

/home/srei/work/embed-python-test/<string> 

ImportError: __import__ not found
After import embedmodule
---------------------------------------------------------------------------
exceptions.NameError                                 Traceback (most recent call last)

/home/srei/work/embed-python-test/<string> 

NameError: name 'embedmodule' is not defined
Python 2.4.1 (#1, Apr  1 2005, 11:41:56) 
Type "copyright", "credits" or "license" for more information.

IPython 0.7.3.svn -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: dir()
Out[1]: 
['In',
 'Out',
 '_',
 '__',
 '__IP',
 '___',
 '__builtins__',
 '__file__',
 '__name__',
 '__nonzero__',
 '_dh',
 '_i',
 '_i1',
 '_ih',
 '_ii',
 '_iii',
 '_oh',
 'help',
 'testmodule']


Any ideas, why the call to PyRun_SimpleString("import embedmodule");
does not work?

Stefan.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: testmodule.c
Type: application/octet-stream
Size: 1021 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061016/70a37cee/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SConstruct
Type: application/octet-stream
Size: 382 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061016/70a37cee/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SConscript
Type: application/octet-stream
Size: 120 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061016/70a37cee/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-embed.py
Type: application/octet-stream
Size: 91 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20061016/70a37cee/attachment-0003.obj>

From fperez.net at gmail.com  Mon Oct 16 03:39:08 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 16 Oct 2006 01:39:08 -0600
Subject: [IPython-dev] Different behaviour: python vs. ipython when
	using a C module
In-Reply-To: <m0ejt8ybpk.fsf@ge-research.com>
References: <m0ejt8ybpk.fsf@ge-research.com>
Message-ID: <db6b5ecc0610160039w45ece530pd2ff93b2e954be25@mail.gmail.com>

On 10/16/06, Stefan Reich?r <stefan at xsteve.at> wrote:
> Hi!
>
> I use python in C/C++ Applications for scripting.
>
> I have implemented a C/C++ library that is used in that application.
> To test that library, I use python (preferred ipython).
> That works very well.
>
> Now I have added an embedded python interpreter to the C/C++ library.
> That interpreter makes the library very easy customizable.
>
> But with this setup I discovered a different behaviour when I compare
> python with ipython. The python version works like I expect. When I
> use ipython, it does not work.
>
> Now I'd like to find out, if this is a bug in ipython, that can be
> fixed, or if I do something wrong.

Well, I'm not really sure: you are doing something a bit odd, and I
honestly don't know if it's correct or if you're getting away with
murder by accident.  Your enclosing init function does this:

void inittestmodule() {
  Py_Initialize();
  Py_InitModule("testmodule", testmoduleAccessMethods);

  Py_InitModule("embedmodule",embedmoduleAccessMethods);
  PyImport_ImportModule("embedmodule");

  printf("Before import embedmodule\n");
  PyRun_SimpleString("import embedmodule");
  printf("After import embedmodule\n");

  PyRun_SimpleString("embedmodule.func2()");
}

It initializes a second internal module, 'embedmodule', inside itself.
 I don't have the foggiest idea whether this trick of creating a
module within another is kosher at the C-API  level (I know it can be
done in pure python using the 'new' module).

I realize it works in pure python, so I'd be inclined to say it's an
ipython bug: ipython should always let you do things that are valid
python.  But I honestly don't know what the solution is here, nor even
if it's not a fluke that it works for the pure python case.

I'll be glad to fix this one if we could understand better the
problem, so if anyone knows what's going on, by all means pitch in.

cheers,

f


From stefan at xsteve.at  Mon Oct 16 04:05:30 2006
From: stefan at xsteve.at (Stefan =?iso-8859-1?Q?Reich=F6r?=)
Date: Mon, 16 Oct 2006 10:05:30 +0200
Subject: [IPython-dev] Different behaviour: python vs. ipython when
	using a C module
In-Reply-To: <db6b5ecc0610160039w45ece530pd2ff93b2e954be25@mail.gmail.com>
	(Fernando Perez's message of "Mon, 16 Oct 2006 01:39:08 -0600")
References: <m0ejt8ybpk.fsf@ge-research.com>
	<db6b5ecc0610160039w45ece530pd2ff93b2e954be25@mail.gmail.com>
Message-ID: <m07iz0y9bp.fsf@ge-research.com>

"Fernando Perez" <fperez.net at gmail.com> writes:

> On 10/16/06, Stefan Reich?r <stefan at xsteve.at> wrote:
>> Hi!
>>
>> I use python in C/C++ Applications for scripting.
>>
>> I have implemented a C/C++ library that is used in that application.
>> To test that library, I use python (preferred ipython).
>> That works very well.
>>
>> Now I have added an embedded python interpreter to the C/C++ library.
>> That interpreter makes the library very easy customizable.
>>
>> But with this setup I discovered a different behaviour when I compare
>> python with ipython. The python version works like I expect. When I
>> use ipython, it does not work.
>>
>> Now I'd like to find out, if this is a bug in ipython, that can be
>> fixed, or if I do something wrong.
>
> Well, I'm not really sure: you are doing something a bit odd, and I
> honestly don't know if it's correct or if you're getting away with
> murder by accident.

Let me try to describe my problem and the solution I use at the moment
a bit mor in detail.

I extend the C library via an embedded python interpreter. This
interpreter can access different parts of the library. So I thought it
would be nice to provide this distinct parts as different modules. So
in my application there are several embedded modules.

After I have imported the embedmodule, I load a user configuration
script that has access to the embedmodule.

If there is a different way to achieve this goal, I will happily use
that.

>  Your enclosing init function does this:
>
> void inittestmodule() {
>  Py_Initialize();
>  Py_InitModule("testmodule", testmoduleAccessMethods);
>
>  Py_InitModule("embedmodule",embedmoduleAccessMethods);
>  PyImport_ImportModule("embedmodule");
>
>  printf("Before import embedmodule\n");
>  PyRun_SimpleString("import embedmodule");
>  printf("After import embedmodule\n");
>
>  PyRun_SimpleString("embedmodule.func2()");
> }
>
> It initializes a second internal module, 'embedmodule', inside itself.
> I don't have the foggiest idea whether this trick of creating a
> module within another is kosher at the C-API  level (I know it can be
> done in pure python using the 'new' module).
>
> I realize it works in pure python, so I'd be inclined to say it's an
> ipython bug: ipython should always let you do things that are valid
> python.  But I honestly don't know what the solution is here, nor even
> if it's not a fluke that it works for the pure python case.
>
> I'll be glad to fix this one if we could understand better the
> problem, so if anyone knows what's going on, by all means pitch in.

I don't know anything about ther ipython internals. But the error
message suggests, that the import functionality is not available
for C extensions, when using ipython.

The following function does not work also:

void inittestmodule() {
  Py_Initialize();
  Py_InitModule("testmodule", testmoduleAccessMethods);
  PyRun_SimpleString("import os");
}


Stefan.


From wojciechowski_m at o2.pl  Wed Oct 18 15:32:09 2006
From: wojciechowski_m at o2.pl (Marek Wojciechowski)
Date: Wed, 18 Oct 2006 19:32:09 -0000
Subject: [IPython-dev] ipython + MPI
Message-ID: <op.thmtvvwqks79d5@localhost>

Hi!
A while ago I have written a small patch to the last ipython (7.2) which  
consist mainly in writing the following function:

def mpi_raw_input(prompt):
     rank = 0; mpi = False; size = 1
     try:
         from mpi4py import MPI
         mpi = True
         COMM = MPI.COMM_WORLD
         rank = MPI.rank
	  size = MPI.size
	  line = ''
     except: pass
     sys.stdout.flush() #Just to avoid messy output
     if rank == 0:
         line = raw_input(prompt)  # Command is read by first process only
     if mpi and size > 1: line = COMM.Bcast(line, 0)  # And is broadcasted  
to all processes involved
     return line

Then at the top of iplib.py I made an assignment:  raw_input_original =  
mpi_raw_input
instead of:  raw_input_original = raw_input

I'm using mpi4py package here from http://www.cimec.org.ar/python, but any  
mpi port to python could be used here.
This slight change allows to run ipython with simple:
     mpirun -np 4 ipython (assuming you have modern enough MPI  
implementation)
All commands are read by first process only and broadcasted to all others  
for execution. With this it is possible to follow plain MPI approach to  
parallel execution:

[In]<1> 1+3
[Out][Proc0]<1> 4
[Out][Proc1]<1> 4
[Out][Proc3]<1> 4
[Out][Proc2]<1> 4

[In]<2> from mpi4py import MPI
[In]<3> if MPI.rank in [1, 3]: print "I'm process %i" %(MPI.rank)
I'm process 3
I'm process 1

[In]<5> from math import sin
[In]<6> bg sin(3)  # ipython magic works as well :)
Starting job # 0 in a separate thread.
Starting job # 0 in a separate thread.
Starting job # 0 in a separate thread.
Starting job # 0 in a separate thread.

Obviously, there are problems with standard output of ipython processes,  
as we are not always interested in output of all of them. However this can  
be handled in a similar way as the input, and was partially done (i made  
for example help messages produced by ipython to be displayed at first  
process only).

Now i observed that new ipython development introduces a parallel  
execution via "controllers" and "engines". Is the above very simple  
approach comparable in any way with this new development. How does it  
support MPI communication? What are the other advanteges of the new kernel  
above the current ipython?

Thanks for any replay

-- 
Marek Wojciechowski


From ellisonbg.net at gmail.com  Wed Oct 18 23:47:20 2006
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Wed, 18 Oct 2006 21:47:20 -0600
Subject: [IPython-dev] ipython + MPI
In-Reply-To: <op.thmtvvwqks79d5@localhost>
References: <op.thmtvvwqks79d5@localhost>
Message-ID: <6ce0ac130610182047g56de946fv430d84ae35eba95c@mail.gmail.com>

Marek,

I *really* like how you are getting ipython to work with mpi4py
interactively.  This accomplishes what pympi and the interactive mode
of mpi4py provide, but you don't have to recompile python.
For those of you who want an extremely simple way of using mpi
interactively, I think this is the best way.  I will probably start to
use this for simple stuff and I might even talk to Fernando about
adding your idea to IPython itself.  Also, I really like mpi4py.

So, how does this relate to the parallel capabilities we are working
on with IPython1:

http://ipython.scipy.org/moin/Parallel_Computing

Most importantly, there are a number of huge limitations of your
approach (and that of pympi) that make it unsuitable for serious
interactive usage:

1.  Your interactive session must be part of the mpi world.  This is
not good because it forces you to do everything in a single
interactive session.  For instance, you can't start an mpi job
interactively on a cluster, disconnect, turn off your laptop go home
and then reconnect to the running mpi job.  If your mpi job lasts a
long time this is unworkable.  With IPython1, your interactive IPython
session is typically on your local workstation and is not an mpi
process.  But, it can connect to the mpi processes (the IPython
Engines in our language) through what we are calling the IPython
Controller (which manages the Engines).  This architecture allows you
to steer an mpi job interactively and to disconnect/reconnect to that
mpi job at any time (we even have a web interface).

2.  If you run a long running command (which you probably will if you
are using mpi) your parallel ipython will block until that command is
done.  With IPython1, mpi commands (any command for that matter) can
be sent to the Engines in either blocking or non blocking mode and we
maintain a queue for each Engine.  This allows you to fire off many
long running python commands - some of which may be mpi calls - and
your local IPython session won't block at all.

3.  Plotting/Visualization.  Try typing 'import matplotlib' in your
parallel Ipython session.  If all the processes are on your local
machine you will probably get 4 or 8 or 16 identical matplotlib
instances.  If you do this on a supercomputer you will get some wierd
error or something worse.  With IPython1, there is only one
interactive instance of IPython running (most often on your local
machine).  We make is easy to ring data back from a remotely running
mpi job to this local IPython instance for plotting and visualization.

4.  Your approach only supports SIMD (single instruction/multiple
data) style parallelism.  That is, each process will always execute
the same code.  With IPython1, you can easily send different commands
to different processes to achieve arbitrary MIMD parallelism (multiple
instruction/multiple data).  This is a little more academic, but I
think it is important for developing general parallel applications.

With all this said, don't let these limitations discourage you from
pushing your ideas further.  I am all for making IPython work in
parallel contexts in as many ways as possible.  I also like your
approach because it is sooo simple and easy to use.  I will probably
use it myself for this reason.
But, I think you will find that the IPython Engine/Controller in
IPython1 make interactive parallel work much more pleasant.

For more information about all the IPython1 stuff, check out the above
web page or the following page that has talks abot this stuff:

http://ipython.scipy.org/moin/About/Presentations

Also, feel free to check the code out from our repository and try it
out.  Also please bug us with questions if you have any.

Cheers,

Brian





> Hi!
> A while ago I have written a small patch to the last ipython (7.2) which
> consist mainly in writing the following function:
>
> def mpi_raw_input(prompt):
>      rank = 0; mpi = False; size = 1
>      try:
>          from mpi4py import MPI
>          mpi = True
>          COMM = MPI.COMM_WORLD
>          rank = MPI.rank
>           size = MPI.size
>           line = ''
>      except: pass
>      sys.stdout.flush() #Just to avoid messy output
>      if rank == 0:
>          line = raw_input(prompt)  # Command is read by first process only
>      if mpi and size > 1: line = COMM.Bcast(line, 0)  # And is broadcasted
> to all processes involved
>      return line
>
> Then at the top of iplib.py I made an assignment:  raw_input_original =
> mpi_raw_input
> instead of:  raw_input_original = raw_input
>
> I'm using mpi4py package here from http://www.cimec.org.ar/python, but any
> mpi port to python could be used here.
> This slight change allows to run ipython with simple:
>      mpirun -np 4 ipython (assuming you have modern enough MPI
> implementation)
> All commands are read by first process only and broadcasted to all others
> for execution. With this it is possible to follow plain MPI approach to
> parallel execution:
>
> [In]<1> 1+3
> [Out][Proc0]<1> 4
> [Out][Proc1]<1> 4
> [Out][Proc3]<1> 4
> [Out][Proc2]<1> 4
>
> [In]<2> from mpi4py import MPI
> [In]<3> if MPI.rank in [1, 3]: print "I'm process %i" %(MPI.rank)
> I'm process 3
> I'm process 1
>
> [In]<5> from math import sin
> [In]<6> bg sin(3)  # ipython magic works as well :)
> Starting job # 0 in a separate thread.
> Starting job # 0 in a separate thread.
> Starting job # 0 in a separate thread.
> Starting job # 0 in a separate thread.
>
> Obviously, there are problems with standard output of ipython processes,
> as we are not always interested in output of all of them. However this can
> be handled in a similar way as the input, and was partially done (i made
> for example help messages produced by ipython to be displayed at first
> process only).
>
> Now i observed that new ipython development introduces a parallel
> execution via "controllers" and "engines". Is the above very simple
> approach comparable in any way with this new development. How does it
> support MPI communication? What are the other advanteges of the new kernel
> above the current ipython?
>
> Thanks for any replay
>
> --
> Marek Wojciechowski
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://projects.scipy.org/mailman/listinfo/ipython-dev
>


From Fernando.Perez at colorado.edu  Sat Oct 21 15:40:32 2006
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Sat, 21 Oct 2006 13:40:32 -0600
Subject: [IPython-dev] [Fwd: color modification of ipython]
Message-ID: <453A77B0.2080404@colorado.edu>

Hi Alec,

thanks for the report; I'm forwarding this to the dev list, where our WinXP
developers (in particular the pyreadline gang) can confirm this and apply the fix.

Note to devs: if these changes are applied, they should be windows-only, since
the reported problem does NOT happen under *nix.

Regards,

f

-------- Original Message --------
Subject: color modification of ipython
Date: Sun, 22 Oct 2006 02:33:54 +0800
From: Alec Jang <alec_jang at 126.com>
To: Fernando.Perez at colorado.edu

Dear Fernando,

I have used ipython for a long time. It works fine along with python
2.4. But when upgrading to python 2.5, a strange thing happened. The
text I typed is in blue, which I never experienced. I think it is a
uncomfortable color, because it cannot be well recognized against black
background.

I looked into ipython source, and made a little fix.

1 In coloransi.py, set Normol = '', which original value is '\033[0m'
2 In the same file, Normal comes twice, the second one also need
modification

After above modification, everything goes well.

I am using Windows Xp, and color theme is Linux.

I suppose it is not a bug, but only a little platform-related thing.

Thank you.

-- 
Best regards,
Alec Jang
   _____                  _____
  (, /  |   /)           (, /
    /---|  //  _  _        /  _  __   _
 ) /    |_(/__(/_(__   ___/__(_(_/ (_(_/_
(_/                  /   /          .-/
                    (__ /          (_/



From jorgen.stenarson at bostream.nu  Sun Oct 22 06:45:38 2006
From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Sun, 22 Oct 2006 12:45:38 +0200
Subject: [IPython-dev] [Fwd: color modification of ipython]
In-Reply-To: <453A77B0.2080404@colorado.edu>
References: <453A77B0.2080404@colorado.edu>
Message-ID: <453B4BD2.8060107@bostream.nu>

Hi Alec,
I do not see this problem when running ipython-pyreadline under python 
2.5 on windows XP. So my question is what versions of ipython and 
pyreadline are you using? Perhaps it is only a question of upgrading one 
or the other.

Regards,
J?rgen


Fernando Perez skrev:
> Hi Alec,
> 
> thanks for the report; I'm forwarding this to the dev list, where our WinXP
> developers (in particular the pyreadline gang) can confirm this and apply the fix.
> 
> Note to devs: if these changes are applied, they should be windows-only, since
> the reported problem does NOT happen under *nix.
> 
> Regards,
> 
> f
> 
> -------- Original Message --------
> Subject: color modification of ipython
> Date: Sun, 22 Oct 2006 02:33:54 +0800
> From: Alec Jang <alec_jang at 126.com>
> To: Fernando.Perez at colorado.edu
> 
> Dear Fernando,
> 
> I have used ipython for a long time. It works fine along with python
> 2.4. But when upgrading to python 2.5, a strange thing happened. The
> text I typed is in blue, which I never experienced. I think it is a
> uncomfortable color, because it cannot be well recognized against black
> background.
> 
> I looked into ipython source, and made a little fix.
> 
> 1 In coloransi.py, set Normol = '', which original value is '\033[0m'
> 2 In the same file, Normal comes twice, the second one also need
> modification
> 
> After above modification, everything goes well.
> 
> I am using Windows Xp, and color theme is Linux.
> 
> I suppose it is not a bug, but only a little platform-related thing.
> 
> Thank you.
> 



From jorgen.stenarson at bostream.nu  Sun Oct 22 15:37:34 2006
From: jorgen.stenarson at bostream.nu (=?UTF-8?B?SsO2cmdlbiBTdGVuYXJzb24=?=)
Date: Sun, 22 Oct 2006 21:37:34 +0200
Subject: [IPython-dev] [Fwd: color modification of ipython]
In-Reply-To: <453B8F49.6010700@126.com>
References: <453A77B0.2080404@colorado.edu> <453B4BD2.8060107@bostream.nu>
	<453B8F49.6010700@126.com>
Message-ID: <453BC87E.8080809@bostream.nu>

Alec,

as I said before I do not see your behaviour.

I think it may be related to the handling of default color. I have made 
some
changes today to improve he situation but I'm not sure if they will help 
you.
If you can test the trunk version from svn I would be happy.

If this doesn't  help. Please activate the debug setting in the
pyreadline.ini file (by changing debug_output("off") to debug_output("on"))
this will result in a log file (pyreadline_debug_output.txt) being created
in the work directory where you start python, in the beginning of the file
there will be a line that logs the initial color attribute. Please 
report this value.
(This value is only reported by the trunk version not the 1.3 release 
version)

You can also try to use the set_input_color function in the ini file to 
force
the color you want. (I don't think this function was available in the 
1.3 release
version but I'm not sure so you would have to upgrade to trunk).


/J?rgen


Alec Jang skrev:
> Hi,
> 
> I am using pyreadline 1.3 and ipython 0.7.2, all packages were newly
> installed, so I suppose it's not a upgrading problem.
> 
> Attachment is a screenshot before I hack the source.
> 
> Thank you for your reply.
> 
> Regards,
> Alec Jang
> 2006-10-22
> 
> J?gen Stenarson wrote:
>> Hi Alec,
>> I do not see this problem when running ipython-pyreadline under python
>> 2.5 on windows XP. So my question is what versions of ipython and
>> pyreadline are you using? Perhaps it is only a question of upgrading
>> one or the other.
>>
>> Regards,
>> J?gen
>>
>>
>> Fernando Perez skrev:
>>> Hi Alec,
>>>
>>> thanks for the report; I'm forwarding this to the dev list, where our
>>> WinXP
>>> developers (in particular the pyreadline gang) can confirm this and
>>> apply the fix.
>>>
>>> Note to devs: if these changes are applied, they should be
>>> windows-only, since
>>> the reported problem does NOT happen under *nix.
>>>
>>> Regards,
>>>
>>> f
>>>
>>> -------- Original Message --------
>>> Subject: color modification of ipython
>>> Date: Sun, 22 Oct 2006 02:33:54 +0800
>>> From: Alec Jang <alec_jang at 126.com>
>>> To: Fernando.Perez at colorado.edu
>>>
>>> Dear Fernando,
>>>
>>> I have used ipython for a long time. It works fine along with python
>>> 2.4. But when upgrading to python 2.5, a strange thing happened. The
>>> text I typed is in blue, which I never experienced. I think it is a
>>> uncomfortable color, because it cannot be well recognized against black
>>> background.
>>>
>>> I looked into ipython source, and made a little fix.
>>>
>>> 1 In coloransi.py, set Normol = '', which original value is '\033[0m'
>>> 2 In the same file, Normal comes twice, the second one also need
>>> modification
>>>
>>> After above modification, everything goes well.
>>>
>>> I am using Windows Xp, and color theme is Linux.
>>>
>>> I suppose it is not a bug, but only a little platform-related thing.
>>>
>>> Thank you.
>>>
>>
>>
> 
> 
> 
> ------------------------------------------------------------------------
> 



From walter at livinglogic.de  Mon Oct 23 07:49:42 2006
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 23 Oct 2006 13:49:42 +0200
Subject: [IPython-dev] path 2.1
Message-ID: <453CAC56.8080007@livinglogic.de>

There's a new version of the path module available (2.1). It simplifies
subclassing, which would simplify the ifile class in ipipe. (Here's the
list of changes:
http://www.jorendorff.com/articles/python/path/changes.html)

So, do we want to update the version of the path module in IPython?

Servus,
   Walter


From rocky at panix.com  Mon Oct 23 12:26:37 2006
From: rocky at panix.com (R. Bernstein)
Date: Mon, 23 Oct 2006 12:26:37 -0400
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610132342u285345ecoa43732c8eaa62314@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17704.34960.967034.430310@panix3.panix.com>
	<17709.46821.79113.757496@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<17710.44340.520249.414290@panix3.panix.com>
	<46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>
	<17711.49406.813910.224951@panix3.panix.com>
	<46cb515a0610131238sf9ec617yf1763a5818cca17e@mail.gmail.com>
	<46cb515a0610131311m6a7dc816u8b89b2c2af9ce291@mail.gmail.com>
	<17712.6083.522575.256388@panix3.panix.com>
	<46cb515a0610132342u285345ecoa43732c8eaa62314@mail.gmail.com>
Message-ID: <17724.60733.290174.21034@panix3.panix.com>

Ville M. Vainio writes:
 > On 10/14/06, R. Bernstein <rocky at panix.com> wrote:
 > 
 > > But of course there were a number of other problems. For some of them
 > > changes were more appropriate in pydb than in ipython. And as a
 > > result, I think initially I reduce the scope even more. You need the
 > > latest version of pydb (and Python 2.5).
 > >
 > > For now and testing use what's just been checked in to CVS.
 > >
 > > Here is a revised set of patches off of the current SVN.
 > 
 > I'll apply them sometime next week. 

[This referred to last week.]  

Any news? 



From vivainio at gmail.com  Mon Oct 23 14:05:27 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 23 Oct 2006 21:05:27 +0300
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <17724.60733.290174.21034@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<17710.44340.520249.414290@panix3.panix.com>
	<46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>
	<17711.49406.813910.224951@panix3.panix.com>
	<46cb515a0610131238sf9ec617yf1763a5818cca17e@mail.gmail.com>
	<46cb515a0610131311m6a7dc816u8b89b2c2af9ce291@mail.gmail.com>
	<17712.6083.522575.256388@panix3.panix.com>
	<46cb515a0610132342u285345ecoa43732c8eaa62314@mail.gmail.com>
	<17724.60733.290174.21034@panix3.panix.com>
Message-ID: <46cb515a0610231105x5d4adbd2ob1d82a811ddeae97@mail.gmail.com>

On 10/23/06, R. Bernstein <rocky at panix.com> wrote:

>  > > Here is a revised set of patches off of the current SVN.
>  >
>  > I'll apply them sometime next week.
>
> [This referred to last week.]
>
> Any news?

Not yet; I haven't tried upgrading my pydb yet to the cvs version
(because, essentially, it would only be you and me testing it in that
case, plus perhaps a few random people who run both svn ipython and
pydb from cvs). I think you'd better but out a pydb release / release
candidate out first, then we apply the patches to ipython to get more
testing coverage. If you think that the pydb from cvs works ok with
your current version of ipython, that is.

It's just that I'm a bit hesitant to change the ipython svn to depend
on a module that is only available via cvs (as opposed to, say, an
easy_install:able module). Or what do you think?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Mon Oct 23 14:23:07 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 23 Oct 2006 21:23:07 +0300
Subject: [IPython-dev] path 2.1
In-Reply-To: <453CAC56.8080007@livinglogic.de>
References: <453CAC56.8080007@livinglogic.de>
Message-ID: <46cb515a0610231123s4f6fe281pe6ad95aaa7acbcdb@mail.gmail.com>

On 10/23/06, Walter D?rwald <walter at livinglogic.de> wrote:

> There's a new version of the path module available (2.1). It simplifies
> subclassing, which would simplify the ifile class in ipipe. (Here's the
> list of changes:
> http://www.jorendorff.com/articles/python/path/changes.html)
>
> So, do we want to update the version of the path module in IPython?

Definitely (and I already committed it - thanks for the heads up).

It's evident from the past that path.py versions are here for the long haul.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From walter at livinglogic.de  Mon Oct 23 18:07:50 2006
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue, 24 Oct 2006 00:07:50 +0200
Subject: [IPython-dev] path 2.1
In-Reply-To: <46cb515a0610231123s4f6fe281pe6ad95aaa7acbcdb@mail.gmail.com>
References: <453CAC56.8080007@livinglogic.de>
	<46cb515a0610231123s4f6fe281pe6ad95aaa7acbcdb@mail.gmail.com>
Message-ID: <453D3D36.8000205@livinglogic.de>

Ville M. Vainio wrote:

> On 10/23/06, Walter D?rwald <walter at livinglogic.de> wrote:
> 
>> There's a new version of the path module available (2.1). It simplifies
>> subclassing, which would simplify the ifile class in ipipe. (Here's the
>> list of changes:
>> http://www.jorendorff.com/articles/python/path/changes.html)
>>
>> So, do we want to update the version of the path module in IPython?
> 
> Definitely (and I already committed it - thanks for the heads up).

Thanks! What about a ChangeLog entry?

> It's evident from the past that path.py versions are here for the long
> haul.

OK, I've updated ipipe accordingly.

Servus,
   Walter


From rocky at panix.com  Mon Oct 23 23:20:33 2006
From: rocky at panix.com (R. Bernstein)
Date: Mon, 23 Oct 2006 23:20:33 -0400
Subject: [IPython-dev] making pdoc, pdef,
	and pinfo more generally useful (Was Re: Patches to use pydb
	instead of pdb for debugging)
In-Reply-To: <46cb515a0610231105x5d4adbd2ob1d82a811ddeae97@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<46cb515a0610120823w2a5889f3j1bcde2b705ebf611@mail.gmail.com>
	<17710.44340.520249.414290@panix3.panix.com>
	<46cb515a0610130815l63de37a2le31c579f36003957@mail.gmail.com>
	<17711.49406.813910.224951@panix3.panix.com>
	<46cb515a0610131238sf9ec617yf1763a5818cca17e@mail.gmail.com>
	<46cb515a0610131311m6a7dc816u8b89b2c2af9ce291@mail.gmail.com>
	<17712.6083.522575.256388@panix3.panix.com>
	<46cb515a0610132342u285345ecoa43732c8eaa62314@mail.gmail.com>
	<17724.60733.290174.21034@panix3.panix.com>
	<46cb515a0610231105x5d4adbd2ob1d82a811ddeae97@mail.gmail.com>
Message-ID: <17725.34433.756977.122038@panix3.panix.com>

When you write "ipython svn to depend on a module.." it may be that
either I haven't explained how the current patches work well enough,
or you are overdramatizing the situation.

There is no dependency on pydb. If you don't have pydb you get pdb. If
you do have pydb and it isn't the right version, you still get pdb.

That said, another release will be put out on Oct 27th or thereabouts,
anyway. Recently I put together a showmedo demo:
 http://showmedo.com/videos/video?name=pythonBernsteinPydbIntro&fromSeriesID=28

In that I use a feature someone requested that is currently only in
CVS. But also possibly the pydb release will also facilitate things
for ipython.

As for easy_install, I think we need a working setup.py. The setup.py
provided in another thread fails for a number of reasons. I don't
doubt that it will happen someday.


Ville M. Vainio writes:
 > On 10/23/06, R. Bernstein <rocky at panix.com> wrote:
 > 
 > >  > > Here is a revised set of patches off of the current SVN.
 > >  >
 > >  > I'll apply them sometime next week.
 > >
 > > [This referred to last week.]
 > >
 > > Any news?
 > 
 > Not yet; I haven't tried upgrading my pydb yet to the cvs version
 > (because, essentially, it would only be you and me testing it in that
 > case, plus perhaps a few random people who run both svn ipython and
 > pydb from cvs). I think you'd better but out a pydb release / release
 > candidate out first, then we apply the patches to ipython to get more
 > testing coverage. If you think that the pydb from cvs works ok with
 > your current version of ipython, that is.
 > 
 > It's just that I'm a bit hesitant to change the ipython svn to depend
 > on a module that is only available via cvs (as opposed to, say, an
 > easy_install:able module). Or what do you think?
 > 
 > -- 
 > Ville M. Vainio - vivainio.googlepages.com
 > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
 > 


From fperez.net at gmail.com  Thu Oct 26 18:53:18 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 26 Oct 2006 16:53:18 -0600
Subject: [IPython-dev] Fwd: Patches to use pydb instead of pdb for debugging
In-Reply-To: <29829.213.53.148.66.1161711516.squirrel@mail.zettai.net>
References: <17701.47216.335545.353158@panix3.panix.com>
	<46cb515a0610070308h507e1df8wf671bcfb28a87c36@mail.gmail.com>
	<db6b5ecc0610091107h77a3e9f9ud6cea56deb65430b@mail.gmail.com>
	<17706.38391.388364.863106@panix3.panix.com>
	<db6b5ecc0610091144u366a7258y8e694d709073ef5b@mail.gmail.com>
	<33686.81.76.40.241.1160421251.squirrel@mail.zettai.net>
	<17706.41849.177166.458362@panix3.panix.com>
	<17962.213.53.148.66.1160472857.squirrel@mail.zettai.net>
	<db6b5ecc0610100927lc467124hdf81239af71d70d2@mail.gmail.com>
	<29829.213.53.148.66.1161711516.squirrel@mail.zettai.net>
Message-ID: <db6b5ecc0610261553o3c79d656jf84667374a28e2a4@mail.gmail.com>

Hi all,

given the recent collaboration with the pydb project, which will
appear in ipython's next release, the email below may be of interest
to IPython users.

Pydb is an improved python debugger, useable in its own right, but
which is seeing better integration with IPython thanks to work by its
author and Ville.  Big thanks to both for this work!

Regards,

f

---------- Forwarded message ----------
From: Ian Ozsvald <ian at showmedo.com>
Date: Oct 24, 2006 11:38 AM
Subject: Re: [IPython-dev] Patches to use pydb instead of pdb for debugging
To: Fernando Perez <fperez.net at gmail.com>
Cc: Kyran <kyran.dale at gmail.com>, rocky at panix.com


Hi Fernando.  Just a quick mail for you - Rocky has recorded a pydb
introduction for us (centre, in gold):
http://showmedo.com/

Perhaps you'll want to mention it in the IPython threads?  I know that
pydb isn't integral to IPython yet, but this might be quite informative to
interested users on the dev lists?  He walks through using pydb to solve a
debugging problem, it is a rather nice video.

Also, in my latest two-part series introducing on-line resources for
Python newbies:
http://showmedo.com/videos/video?name=pythonOzsvaldIntroToPyResources2&fromSeriesID=27
I mention IPython, along with matplotlib, TurboGears, Django and wxPython
as 'established projects to check out'.  These are all projects that I'm
involved with, I figured newbies could do with a pointer in the right
direction :-)

Cheers,
Ian.


From rocky at panix.com  Thu Oct 26 23:38:29 2006
From: rocky at panix.com (R. Bernstein)
Date: Thu, 26 Oct 2006 23:38:29 -0400
Subject: [IPython-dev] pydb 1.19 released, thoughts about integration
In-Reply-To: <db6b5ecc0610261553o3c79d656jf84667374a28e2a4@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<46cb515a0610070308h507e1df8wf671bcfb28a87c36@mail.gmail.com>
	<db6b5ecc0610091107h77a3e9f9ud6cea56deb65430b@mail.gmail.com>
	<17706.38391.388364.863106@panix3.panix.com>
	<db6b5ecc0610091144u366a7258y8e694d709073ef5b@mail.gmail.com>
	<33686.81.76.40.241.1160421251.squirrel@mail.zettai.net>
	<17706.41849.177166.458362@panix3.panix.com>
	<17962.213.53.148.66.1160472857.squirrel@mail.zettai.net>
	<db6b5ecc0610100927lc467124hdf81239af71d70d2@mail.gmail.com>
	<29829.213.53.148.66.1161711516.squirrel@mail.zettai.net>
	<db6b5ecc0610261553o3c79d656jf84667374a28e2a4@mail.gmail.com>
Message-ID: <17729.32565.872208.221156@panix3.panix.com>

As requested and promised, a new version of pydb (1.19) has been
released, and this release has the patches to make it more ipython
friendly. (It also has other accumulated improvements requested too.)

I'd like to possibly reiterate some thoughts about ipython/pydb
integration and possibly go further.

To date, what I've noticed is that things are a bit more complicated
in ipython because it took the necessary stance that the debugger
interfaces/API couldn't be changed. I think things could be simplified
by working across these two programs.

But, at least on the pydb side, the resources to work on this are
limited. So limiting initial exposure is perhaps a good
thing. Therefore to get the *full* benefit of what's there, you need
Python 2.5 and pydb 1.19 (and of course a future ipython
release). Older versions of python could be accomodated depending on
how much crud one wanted to add to ipython.

But what are plans in the chainsaw branch? I see what looks to me
really ugly code in "run -d". Is that code going to be in the chainsaw
branch as well?. Is the chainsaw branch going to support older
versions of Python which require duplicating code from say pdb?




From vivainio at gmail.com  Fri Oct 27 10:11:40 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 27 Oct 2006 17:11:40 +0300
Subject: [IPython-dev] pydb 1.19 released, thoughts about integration
In-Reply-To: <17729.32565.872208.221156@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17706.38391.388364.863106@panix3.panix.com>
	<db6b5ecc0610091144u366a7258y8e694d709073ef5b@mail.gmail.com>
	<33686.81.76.40.241.1160421251.squirrel@mail.zettai.net>
	<17706.41849.177166.458362@panix3.panix.com>
	<17962.213.53.148.66.1160472857.squirrel@mail.zettai.net>
	<db6b5ecc0610100927lc467124hdf81239af71d70d2@mail.gmail.com>
	<29829.213.53.148.66.1161711516.squirrel@mail.zettai.net>
	<db6b5ecc0610261553o3c79d656jf84667374a28e2a4@mail.gmail.com>
	<17729.32565.872208.221156@panix3.panix.com>
Message-ID: <46cb515a0610270711k58a4bb0et683d4e7d1397b159@mail.gmail.com>

On 10/27/06, R. Bernstein <rocky at panix.com> wrote:

> But what are plans in the chainsaw branch? I see what looks to me
> really ugly code in "run -d". Is that code going to be in the chainsaw
> branch as well?. Is the chainsaw branch going to support older
> versions of Python which require duplicating code from say pdb?

You can already override the %run magic command through ipapi
(IPython/ipapi.py), by using expose_magic().

We could look into exposing various hooks to allow plugging in
different levels of debugging support. The current debugger support is
pre-ipapi stuff and too "wired in" for it's own good, but if we can
clean ip up through hooks and other extensions mechanisms (as opposed
to wiring in more stuff), so much the better.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Fri Oct 27 15:36:58 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 27 Oct 2006 22:36:58 +0300
Subject: [IPython-dev] pydb 1.19 released, thoughts about integration
In-Reply-To: <17730.22974.688887.337485@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17962.213.53.148.66.1160472857.squirrel@mail.zettai.net>
	<db6b5ecc0610100927lc467124hdf81239af71d70d2@mail.gmail.com>
	<29829.213.53.148.66.1161711516.squirrel@mail.zettai.net>
	<db6b5ecc0610261553o3c79d656jf84667374a28e2a4@mail.gmail.com>
	<17729.32565.872208.221156@panix3.panix.com>
	<46cb515a0610270735u473d95f5v82919b354d72b503@mail.gmail.com>
	<17730.18243.354371.813555@panix3.panix.com>
	<46cb515a0610271121kf610602w3d331e3fea497c71@mail.gmail.com>
	<17730.22974.688887.337485@panix3.panix.com>
Message-ID: <46cb515a0610271236v4d328353n31db33fa95ac701@mail.gmail.com>

On 10/27/06, R. Bernstein <rocky at panix.com> wrote:

> I tried that. And I assume you were in the directory that ipython was
> located in which is what I tried. The interesting question is what's
> the value of frame.f_code.co_filename and if that's a valid file name,
> where is it? Probably %pdb on would let you extract that information.

Ok, now I ran %pdb and %run -d ipython.py and got this traceback:

Q:\ipython\IPython\Debugger.py in __init__(self=<IPython.Debugger.Pdb
instance at 0x00F17C88>, color_scheme='Linux', completekey=None,
stdin=None, stdout=None)
    103             # IPython changes...
    104             self.prompt = prompt # The default prompt is '(Pdb)'
    105             self.is_pydb = prompt == 'ipydb>'
    106
    107             if self.is_pydb:
    108
    109                 # iplib.py's ipalias seems to want pdb's checkline
    110                 # which located in pydb.fn
    111                 import pydb.fns
    112                 self.checkline = lambda filename, lineno: \
    113                                  pydb.fns.checkline(self,
filename, lineno)
    114
    115                 self.curframe = None
    116                 self.do_restart = self.new_do_restart
    117
--> 118                 self.old_all_completions =
__IPYTHON__.Completer.all_completions
        self.old_all_completions = undefined
        global __IPYTHON__.Completer.all_completions = undefined
    119
__IPYTHON__.Completer.all_completions=self.all_completions
    120
    121                 # Do we have access to pydb's list command parser?
    122                 self.do_list =
decorate_fn_with_doc(self.list_command_pydb,
    123                                                     OldPdb.do_list)
    124                 self.do_l     = self.do_list
    125                 self.do_frame = decorate_fn_with_doc(self.new_do_frame,
    126                                                      OldPdb.do_frame)
    127
    128             self.aliases = {}
    129
    130             # Create color table: we copy the default one from
the traceback
    131             # module and add a few attributes needed for debugging
    132             self.color_scheme_table = ExceptionColors.copy()
    133
<type 'exceptions.AttributeError'>: 'InteractiveShell' object has no
attribute 'Completer'
**********************************************************************

>  > BTW, does pydb nuke the readline history in %run -d for you? I mean
>  > that, after the debug session is over, you can't go back to commands
>  > you executed before %run -d.
>
> If you run "set history save on", the history is be saved in
> $HOME/.pydbhist. The default is not to save the history, same as
> gdb. You could put this command in your $HOME/.pydbrc if that's what
> you want. But when I just tried it I don't see that .pydbrc is getting
> read. I'll check into this.

But the whole history is lost, including "normal" ipython stuff that
precedes %run -d. This is quite undesirable default behaviour.

> Yes, I think a separate %pydb magic would be better for a couple of
> reasons. First %run -d has no provision for sending options to the
> debugger. Second, because %run -d copies some of the top-level pdb
> debugger loop, a soft restart which is basically raising an exception
> caught in that loop can't done without more modification. To add it to
> that code would be increasing the wrongness, if not also the
> complexity.
>
> The way I've been using pydb inside ipython is real simple:
>   import pydb  # Needed once
>   pydb.runl(args1, arg2, arg3...)
>
> If the runl could be run via magic that'd be great.

Great, we're going to go with separate %pydb then. Screw %run -d for
pydb purposes, we should reserve it for plain pdb use only.

> There is perhaps one slight subtlety or ugliness. The above is the
> programming way to do things, not a command-line way. When you run any
> python program from the script's standpoint it gets the arguments as a
> list. When you run "%run -d" what's getting sent is a string. You
> can't just do a split on spaces to turn this string into a parameter
> list because there may be embedded blanks in string
> options/parameters.

shlex.split helps here:

[ipython]|4> shlex.split('hello world "and good luck"')
         <4> ['hello', 'world', 'and good luck']

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Fri Oct 27 16:53:22 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 27 Oct 2006 14:53:22 -0600
Subject: [IPython-dev] pydb 1.19 released, thoughts about integration
In-Reply-To: <db6b5ecc0610271339y50c89785x985aededc42e1260@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<29829.213.53.148.66.1161711516.squirrel@mail.zettai.net>
	<db6b5ecc0610261553o3c79d656jf84667374a28e2a4@mail.gmail.com>
	<17729.32565.872208.221156@panix3.panix.com>
	<46cb515a0610270735u473d95f5v82919b354d72b503@mail.gmail.com>
	<17730.18243.354371.813555@panix3.panix.com>
	<46cb515a0610271121kf610602w3d331e3fea497c71@mail.gmail.com>
	<17730.22974.688887.337485@panix3.panix.com>
	<46cb515a0610271236v4d328353n31db33fa95ac701@mail.gmail.com>
	<db6b5ecc0610271339y50c89785x985aededc42e1260@mail.gmail.com>
Message-ID: <db6b5ecc0610271353u3bc96f4fy354634c87d843dde@mail.gmail.com>

On 10/27/06, Ville M. Vainio <vivainio at gmail.com> wrote:

> shlex.split helps here:
>
> [ipython]|4> shlex.split('hello world "and good luck"')
>          <4> ['hello', 'world', 'and good luck']

You can use genutils.arg_split() from SVN r.1845 (I just committed
it).  It's a thin shlex.split wrapper, but it lets you keep your
quotes nicely in case you need them:

In [3]: genutils.arg_split('hello world "and good luck"')
Out[3]: ['hello', 'world', '"and good luck"']

In [4]: genutils.arg_split('hello world "and good luck"',posix=True)
Out[4]: ['hello', 'world', 'and good luck']

The first case is needed if you want a magic that sees the original
strings as single units (which we need in magics to extract the
magic's command line flags while preserving the original strings for
the downstream program).

Cheers,

f


From fperez.net at gmail.com  Fri Oct 27 17:03:39 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 27 Oct 2006 15:03:39 -0600
Subject: [IPython-dev] pydb 1.19 released, thoughts about integration
In-Reply-To: <17729.32565.872208.221156@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17706.38391.388364.863106@panix3.panix.com>
	<db6b5ecc0610091144u366a7258y8e694d709073ef5b@mail.gmail.com>
	<33686.81.76.40.241.1160421251.squirrel@mail.zettai.net>
	<17706.41849.177166.458362@panix3.panix.com>
	<17962.213.53.148.66.1160472857.squirrel@mail.zettai.net>
	<db6b5ecc0610100927lc467124hdf81239af71d70d2@mail.gmail.com>
	<29829.213.53.148.66.1161711516.squirrel@mail.zettai.net>
	<db6b5ecc0610261553o3c79d656jf84667374a28e2a4@mail.gmail.com>
	<17729.32565.872208.221156@panix3.panix.com>
Message-ID: <db6b5ecc0610271403q3715fd29ub0f352cb43d19c29@mail.gmail.com>

Just to fill in a little...

On 10/26/06, R. Bernstein <rocky at panix.com> wrote:
> As requested and promised, a new version of pydb (1.19) has been
> released, and this release has the patches to make it more ipython
> friendly. (It also has other accumulated improvements requested too.)
>
> I'd like to possibly reiterate some thoughts about ipython/pydb
> integration and possibly go further.
>
> To date, what I've noticed is that things are a bit more complicated
> in ipython because it took the necessary stance that the debugger
> interfaces/API couldn't be changed. I think things could be simplified
> by working across these two programs.
>
> But, at least on the pydb side, the resources to work on this are
> limited. So limiting initial exposure is perhaps a good
> thing. Therefore to get the *full* benefit of what's there, you need
> Python 2.5 and pydb 1.19 (and of course a future ipython
> release). Older versions of python could be accomodated depending on
> how much crud one wanted to add to ipython.

We'll certainly be supporting 2.4 for quite a while.  There is
currently not one single *nix distro (counting Apple OSX as one) that
defaults to Python 2.5, not even the fresh-off-the-oven Fedora Core 6
and Ubuntu Edgy Eft, both released in the last 72 hours.  These still
use 2.4.x (x=3 for FC6 and x=4c1 for Edgy Eft, which I installed
yesterday).

And Ubuntu Dapper, bound to have a long life in research labs (due to
its long term support) uses 2.4, so for now we'll keep 2.4 as a
requirement.  Right now, making code be 2.5-only would alienate a
pretty large fraction of ipython's user base,  I suspect (including a
user I consider particularly important: me ;)

> But what are plans in the chainsaw branch? I see what looks to me
> really ugly code in "run -d". Is that code going to be in the chainsaw
> branch as well?. Is the chainsaw branch going to support older
> versions of Python which require duplicating code from say pdb?

The chainsaw branch will eventually offer all of the normal ipython
functionality, including magic support.  But hopefully we'll have a
much cleaner infrastructure in place to do so.  Much of the code in
some of the bigger magics (%run and %edit are the worse) grew simply
by historical accretion, and is one big spaghetti mess in bad need of
a cleanup.

And thanks for this release!

Cheers,

f


From rocky at panix.com  Fri Oct 27 20:39:18 2006
From: rocky at panix.com (R. Bernstein)
Date: Fri, 27 Oct 2006 20:39:18 -0400
Subject: [IPython-dev] ipython/pydb integration - readline
Message-ID: <17730.42678.268147.727141@panix3.panix.com>

I thought I'd bring to a wider development audience some of the recent
observations and issues with ipython and pydb integration. 

As with pdoc/pdef, I don't think the issues are strictly tied to pydb,
but any program one %run's that uses readline.

(Ville: I'm adding observations I've come across subsequently so this
doesn't exactly follow the email exchanges or that history.)

The short one sentence summary is that right now there is a single
(global) readline history gets used inside %run (if not other places)
and really the history from an application's %run should separate from
ipython's history.

Both ipython and pydb and use readline. When pydb exits it can either
write its command history, or not save it in which case it is
truncated. The default is to truncate the history, and I believe the
effect seen in ipython then, is that after pydb leaves the *ipython*
history has artifacts of whatever the pydb setting was. That is if
pydb set to truncate the history, then ipython's history is truncated;
if not, then the history contains *pydb* entries in addition to the
ipython entries.

Sort of. I say this because the ipython "history" command seems to
work right. So does pydb's equivalent: "show commands".

There sort of is an implication that adding pydb "broke" things, but
in thinking and looking at this deeper, it strikes me that the
readline handling in ipython is currently a little bit broken. Just
broken in a different way. And I'd be curious to learn if others have
noticed this as think I have.

First, *any* program that decides to use readline that is %run from
ipython (whether -d or not) will have the above issue. So as was
initially observed, we have interference in the program-to-ipython
side. But you also have it in the ipython-to-program side. That is,
inside pdb whenever pdb prompts, *ipython* command completions are
offered.  This isn't all that helpful. And as mentioned above
"previous-command" (in Emacs: Ctrl-P) spans across the two, which also
doesn't seem right.

I think the right solution then would be for ipython to save its
history (if history saving is on), truncate the history if it can, run
the command, and the reinstate the history by rereading it from the
saved history file. Again, this works for all programs, not just pydb.

- - - -

One last strongly related topic which includes Fenando's comment about
the necessity of Python 2.4 support. It was the following readline
workaround in Debugger.py around line 68 that caused me to initially
decided to punt on Python < 2.5:


    if sys.version[:3] >= '2.5':
        def __init__(self,color_scheme='NoColor',completekey=None,
                     stdin=None, stdout=None):
            ... # I'll come back to this below
    else:
        # Ugly hack: for Python 2.3-2.4, we can't call the parent constructor,
        # because it binds readline and breaks tab-completion.  This means we
        # have to COPY the constructor here.
        def __init__(self,color_scheme='NoColor'):
        ... #1

I think the intent at #1 above is basically to copy the code in pdb's
__init__ constructor omitting "import readline" and the code associated
with implications of that.

So it looks to me like the test on version 2.5 has more to do with
testing the version of *pdb* used than anything else version 2.4 or
version 2.5. Is this correct?

Assuming this is the case, when I look the 2.4 pdb versus the 2.5 pdb,
I see THAT THEY BOTH IMPORT READLINE. In other words the 2.5 branch
("I'll come back to this below") should have pdb code copied minus the
import readline. This has to be a separate branch still because it is
different code. But none of it appears, so I'd be curious to learn why
it's not needed in the 2.5 branch.

Perhaps. And perhaps not. Because if the real solution is to:
 - save the command history file, 
 - truncate it, 
 - call "run" (or post-mortem)
 - restore the history file by rereading it
then perhaps the above is moot.

Comments?


From vivainio at gmail.com  Sat Oct 28 03:52:45 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 28 Oct 2006 10:52:45 +0300
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <17730.42678.268147.727141@panix3.panix.com>
References: <17730.42678.268147.727141@panix3.panix.com>
Message-ID: <46cb515a0610280052r27edd6bbg8b73d8d279adf816@mail.gmail.com>

On 10/28/06, R. Bernstein <rocky at panix.com> wrote:

> Perhaps. And perhaps not. Because if the real solution is to:
>  - save the command history file,
>  - truncate it,
>  - call "run" (or post-mortem)
>  - restore the history file by rereading it

To cut the long story short, this is the best solution and I just
committed it to SVN (rev 1846).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Sat Oct 28 04:47:37 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 28 Oct 2006 11:47:37 +0300
Subject: [IPython-dev] Extensions/pydb_ipy.py added
Message-ID: <46cb515a0610280147u5a310ffev181ec352bc9f83cd@mail.gmail.com>

I've now added pydb_ipy.py in Extensions, importing of which creates a
new %pydb magic:

[ipython]|2> import pydb_ipy
[ipython]|3> %pydb a.py
(q:\ipython\a.py:1):  <module>
(Pydb) l
  1  -> print "hello"
  2     print "world"
[EOF]
(Pydb) c
hello
world
The program finished and will be restarted
(q:\ipython\a.py:1):  <module>
(Pydb) q
[ipython]|4> %pydb a.py

It appeard we need to do some SystemExit catching there also because
pydb exits with sys.exit() on many error situations.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Sat Oct 28 04:57:05 2006
From: rocky at panix.com (R. Bernstein)
Date: Sat, 28 Oct 2006 04:57:05 -0400
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <46cb515a0610280052r27edd6bbg8b73d8d279adf816@mail.gmail.com>
References: <17730.42678.268147.727141@panix3.panix.com>
	<46cb515a0610280052r27edd6bbg8b73d8d279adf816@mail.gmail.com>
Message-ID: <17731.7009.270093.52153@panix3.panix.com>

Ville M. Vainio reports (out of order):
 > To cut the long story short, this is the best solution and I just
 > committed it to SVN (rev 1846).

Great! Thanks! 

I looked at what was committed though and have a couple of comments.
They are in line below.

And Ville continues..
 > On 10/28/06, R. Bernstein <rocky at panix.com> wrote:
 > 
 > > Perhaps. And perhaps not. Because if the real solution is to:
 > >  - save the command history file,
 > >  - truncate it,

I don't see this step. Shouldn't there be a readline.clear_history()
after the save?

 > >  - call "run" (or post-mortem)

I see the save/restore for "%run", but not for post-mortem(). And this
may be a slightly more involved change, because what may be needed is
not to invoke the the debugger's post-mortem() directly in the setup
hook as is done now, but to invoke with a new wrapping routine which
surrounds the post_mortem with the same steps used in %run.

And once this is done, then all of that other code in Debugger.py to
define class Pdb and copy code from pdb can be removed/simplified!
Basically the non Python 2.5 branch can look exactly like the Python
2.5 branch, and then we can drop the requirement that pydb get loaded
only for version 2.5. Right?

(I'm not completely sure, because we haven't gotten a more
authoritative answer than my guess as to what that code does, and why
that code is *not* needed in Python 2.5 but is in Python 2.4)


From vivainio at gmail.com  Sat Oct 28 05:10:45 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 28 Oct 2006 12:10:45 +0300
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <17731.7009.270093.52153@panix3.panix.com>
References: <17730.42678.268147.727141@panix3.panix.com>
	<46cb515a0610280052r27edd6bbg8b73d8d279adf816@mail.gmail.com>
	<17731.7009.270093.52153@panix3.panix.com>
Message-ID: <46cb515a0610280210v1264adaeqd89ca5dd9a3e8a50@mail.gmail.com>

On 10/28/06, R. Bernstein <rocky at panix.com> wrote:

>  > > Perhaps. And perhaps not. Because if the real solution is to:
>  > >  - save the command history file,
>  > >  - truncate it,
>
> I don't see this step. Shouldn't there be a readline.clear_history()
> after the save?

I don't think so, necessarily. More stuff in history == better, IMO.
Even if it's not strictly in the debugger domain.

>  > >  - call "run" (or post-mortem)
>
> I see the save/restore for "%run", but not for post-mortem(). And this
> may be a slightly more involved change, because what may be needed is
> not to invoke the the debugger's post-mortem() directly in the setup
> hook as is done now, but to invoke with a new wrapping routine which
> surrounds the post_mortem with the same steps used in %run.

Will do.

> And once this is done, then all of that other code in Debugger.py to
> define class Pdb and copy code from pdb can be removed/simplified!
> Basically the non Python 2.5 branch can look exactly like the Python
> 2.5 branch, and then we can drop the requirement that pydb get loaded
> only for version 2.5. Right?

Probably. Though here your guess is as good as mine, and Fernando will
probably want to chip in.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Sat Oct 28 05:26:42 2006
From: rocky at panix.com (R. Bernstein)
Date: Sat, 28 Oct 2006 05:26:42 -0400
Subject: [IPython-dev]  Extensions/pydb_ipy.py added
In-Reply-To: <46cb515a0610280147u5a310ffev181ec352bc9f83cd@mail.gmail.com>
References: <46cb515a0610280147u5a310ffev181ec352bc9f83cd@mail.gmail.com>
Message-ID: <17731.8786.756329.573337@panix3.panix.com>

Ville M. Vainio writes:
 > I've now added pydb_ipy.py in Extensions, importing of which creates a
 > new %pydb magic:

Again - thanks! But, er, isn't the name ipy_pydb.py? (That's what I
see at revision 1849).

And given that the same thing is to be done for post_mortem, It might
be neater to write a generic history-wrapping routine sort of as was
done to wrap pydb routines and but add their docstrings.

By the way, pydb.runl(*argl) is in fact the same thing as
pydb.runv(argl) and I think the latter just more straightforward.


From rocky at panix.com  Sat Oct 28 05:36:38 2006
From: rocky at panix.com (R. Bernstein)
Date: Sat, 28 Oct 2006 05:36:38 -0400
Subject: [IPython-dev] pydb 1.19 released, thoughts about integration
In-Reply-To: <db6b5ecc0610271403q3715fd29ub0f352cb43d19c29@mail.gmail.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<17706.38391.388364.863106@panix3.panix.com>
	<db6b5ecc0610091144u366a7258y8e694d709073ef5b@mail.gmail.com>
	<33686.81.76.40.241.1160421251.squirrel@mail.zettai.net>
	<17706.41849.177166.458362@panix3.panix.com>
	<17962.213.53.148.66.1160472857.squirrel@mail.zettai.net>
	<db6b5ecc0610100927lc467124hdf81239af71d70d2@mail.gmail.com>
	<29829.213.53.148.66.1161711516.squirrel@mail.zettai.net>
	<db6b5ecc0610261553o3c79d656jf84667374a28e2a4@mail.gmail.com>
	<17729.32565.872208.221156@panix3.panix.com>
	<db6b5ecc0610271403q3715fd29ub0f352cb43d19c29@mail.gmail.com>
Message-ID: <17731.9382.515217.236549@panix3.panix.com>

Fernando Perez writes:
 ...
 > We'll certainly be supporting 2.4 for quite a while.  

Okay. Thanks for the information!

 > The chainsaw branch will eventually offer all of the normal ipython
 > functionality, including magic support.  But hopefully we'll have a
 > much cleaner infrastructure in place to do so.  

As I think we are discovering via ipython/pydb integration, it is not
clear that a number of these issues current discovered have been
adequately documented or documented as areas where it is known how to
improve. Two cases in point are

 - how to allow applications run from ipython to use ipython's object
   help routines
 - how to handle readline issues that running program needs

Please correct me if I'm wrong, but if the chain-saw branch has a
cleaner way to handle that what's in the normal branch, that would be
helpful to know; even if for technical reasons it currently can't be
applied there.


From vivainio at gmail.com  Sat Oct 28 06:36:58 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 28 Oct 2006 13:36:58 +0300
Subject: [IPython-dev] pydb 1.19 released, thoughts about integration
In-Reply-To: <17731.9382.515217.236549@panix3.panix.com>
References: <17701.47216.335545.353158@panix3.panix.com>
	<33686.81.76.40.241.1160421251.squirrel@mail.zettai.net>
	<17706.41849.177166.458362@panix3.panix.com>
	<17962.213.53.148.66.1160472857.squirrel@mail.zettai.net>
	<db6b5ecc0610100927lc467124hdf81239af71d70d2@mail.gmail.com>
	<29829.213.53.148.66.1161711516.squirrel@mail.zettai.net>
	<db6b5ecc0610261553o3c79d656jf84667374a28e2a4@mail.gmail.com>
	<17729.32565.872208.221156@panix3.panix.com>
	<db6b5ecc0610271403q3715fd29ub0f352cb43d19c29@mail.gmail.com>
	<17731.9382.515217.236549@panix3.panix.com>
Message-ID: <46cb515a0610280336s7f771081rb76729deb4585554@mail.gmail.com>

On 10/28/06, R. Bernstein <rocky at panix.com> wrote:

> Please correct me if I'm wrong, but if the chain-saw branch has a
> cleaner way to handle that what's in the normal branch, that would be
> helpful to know; even if for technical reasons it currently can't be
> applied there.

Both chainsaw and stable branch should share the same kind of ipapi
(IPython/ipapi.py) - and possibly also many of the hooks. Extensions
points should be implemented in ipapi and as hooks, and they stand a
reasonable chance to work on both ipython versions.

Object discovery and introspection could be there, but I'm not quite
sure, because of the "printing" nature of introspection (coloring
etc.).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Sat Oct 28 06:47:44 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 28 Oct 2006 13:47:44 +0300
Subject: [IPython-dev] Extensions/pydb_ipy.py added
In-Reply-To: <17731.8786.756329.573337@panix3.panix.com>
References: <46cb515a0610280147u5a310ffev181ec352bc9f83cd@mail.gmail.com>
	<17731.8786.756329.573337@panix3.panix.com>
Message-ID: <46cb515a0610280347n73539bdt6810659b7584e3a@mail.gmail.com>

On 10/28/06, R. Bernstein <rocky at panix.com> wrote:
> Ville M. Vainio writes:
>  > I've now added pydb_ipy.py in Extensions, importing of which creates a
>  > new %pydb magic:
>
> Again - thanks! But, er, isn't the name ipy_pydb.py? (That's what I
> see at revision 1849).

It is, I renamed it (to follow the established convention for Extensions)

> And given that the same thing is to be done for post_mortem, It might
> be neater to write a generic history-wrapping routine sort of as was
> done to wrap pydb routines and but add their docstrings.

Yeah, makes sense & will do.

> By the way, pydb.runl(*argl) is in fact the same thing as
> pydb.runv(argl) and I think the latter just more straightforward.

Ok; though does the existence of *list syntax not make pydb.runv()
slightly redundant?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Sat Oct 28 07:18:54 2006
From: rocky at panix.com (R. Bernstein)
Date: Sat, 28 Oct 2006 07:18:54 -0400
Subject: [IPython-dev] Extensions/pydb_ipy.py added
In-Reply-To: <46cb515a0610280347n73539bdt6810659b7584e3a@mail.gmail.com>
References: <46cb515a0610280147u5a310ffev181ec352bc9f83cd@mail.gmail.com>
	<17731.8786.756329.573337@panix3.panix.com>
	<46cb515a0610280347n73539bdt6810659b7584e3a@mail.gmail.com>
Message-ID: <17731.15518.833930.352892@panix3.panix.com>

Ville M. Vainio writes:
 > On 10/28/06, R. Bernstein <rocky at panix.com> wrote:
 > > Ville M. Vainio writes:
 > >  > I've now added pydb_ipy.py in Extensions, importing of which creates a
 > >  > new %pydb magic:
 > >
 > > Again - thanks! But, er, isn't the name ipy_pydb.py? (That's what I
 > > see at revision 1849).
 > 
 > It is, I renamed it (to follow the established convention for Extensions)

Ok. Thanks for the clarification.

 > 
 > > And given that the same thing is to be done for post_mortem, It might
 > > be neater to write a generic history-wrapping routine sort of as was
 > > done to wrap pydb routines and but add their docstrings.
 > 
 > Yeah, makes sense & will do.

Many thanks! 

But one other thing regarding call_pydb(). And I'm not really sure how
to fix. So if anyone has suggestions...

pydb.runv() should be using the ipython Pdb class which has pdef, pdoc
commands added and not the pydb one.

 > 
 > > By the way, pydb.runl(*argl) is in fact the same thing as
 > > pydb.runv(argl) and I think the latter just more straightforward.
 > 
 > Ok; though does the existence of *list syntax not make pydb.runv()
 > slightly redundant?

Well, in the os module there is: 
  execl(	    path, arg0, arg1, ...)
  execle(	    path, arg0, arg1, ..., env)
  execlp(	    file, arg0, arg1, ...)
  execlpe(    file, arg0, arg1, ..., env)
  execv(	    path, args)
  execve(	    path, args, env)
  execvp(	    file, args)
  execvpe(    file, args, env)

  spawnl(	      mode, path, ...)
  spawnle(      mode, path, ..., env)
  spawnlp(      mode, file, ...)
  spawnlpe(     mode, file, ..., env)
  spawnv(	      mode, path, args)
  spawnve(      mode, path, args, env)
  spawnvp(      mode, file, args)
  spawnvpe(     mode, file, args, env)

The "run" suffixes were drawn from this (which in turn are drawn from
the C library system calls.)


From vivainio at gmail.com  Sat Oct 28 08:41:33 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 28 Oct 2006 15:41:33 +0300
Subject: [IPython-dev] Extensions/pydb_ipy.py added
In-Reply-To: <17731.15518.833930.352892@panix3.panix.com>
References: <46cb515a0610280147u5a310ffev181ec352bc9f83cd@mail.gmail.com>
	<17731.8786.756329.573337@panix3.panix.com>
	<46cb515a0610280347n73539bdt6810659b7584e3a@mail.gmail.com>
	<17731.15518.833930.352892@panix3.panix.com>
Message-ID: <46cb515a0610280541n1ccc4691o4a47f1ca404b826c@mail.gmail.com>

On 10/28/06, R. Bernstein <rocky at panix.com> wrote:

> But one other thing regarding call_pydb(). And I'm not really sure how
> to fix. So if anyone has suggestions...
>
> pydb.runv() should be using the ipython Pdb class which has pdef, pdoc
> commands added and not the pydb one.

Would it make sense to import pydb.pydb module in pydb_ipy.py, and
inject the ipython version of Pdb class to the pydb module namespace?
That way main() would instantiate the ipython version...


>  > Ok; though does the existence of *list syntax not make pydb.runv()
>  > slightly redundant?
>
> Well, in the os module there is:
>   execl(            path, arg0, arg1, ...)
>   execle(           path, arg0, arg1, ..., env)
>   execlp(           file, arg0, arg1, ...)
>   execlpe(    file, arg0, arg1, ..., env)
>   execv(            path, args)
>   execve(           path, args, env)
>   execvp(           file, args)
>   execvpe(    file, args, env)
>
>   spawnl(             mode, path, ...)
>   spawnle(      mode, path, ..., env)
>   spawnlp(      mode, file, ...)
>   spawnlpe(     mode, file, ..., env)
>   spawnv(             mode, path, args)
>   spawnve(      mode, path, args, env)
>   spawnvp(      mode, file, args)
>   spawnvpe(     mode, file, args, env)
>
> The "run" suffixes were drawn from this (which in turn are drawn from
> the C library system calls.)

Those functions were introduced long before *arglist syntax (on caller
side)... I'm not sure whether anyone actualle "likes" those functions
anymore. In C they make perfect sense, of course.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Sat Oct 28 11:47:14 2006
From: rocky at panix.com (R. Bernstein)
Date: Sat, 28 Oct 2006 11:47:14 -0400
Subject: [IPython-dev] Extensions/pydb_ipy.py added
In-Reply-To: <46cb515a0610280541n1ccc4691o4a47f1ca404b826c@mail.gmail.com>
References: <46cb515a0610280147u5a310ffev181ec352bc9f83cd@mail.gmail.com>
	<17731.8786.756329.573337@panix3.panix.com>
	<46cb515a0610280347n73539bdt6810659b7584e3a@mail.gmail.com>
	<17731.15518.833930.352892@panix3.panix.com>
	<46cb515a0610280541n1ccc4691o4a47f1ca404b826c@mail.gmail.com>
Message-ID: <17731.31618.644784.714712@panix3.panix.com>

Ville M. Vainio writes:
 > On 10/28/06, R. Bernstein <rocky at panix.com> wrote:
 > 
 > > But one other thing regarding call_pydb(). And I'm not really sure how
 > > to fix. So if anyone has suggestions...
 > >
 > > pydb.runv() should be using the ipython Pdb class which has pdef, pdoc
 > > commands added and not the pydb one.
 > 
 > Would it make sense to import pydb.pydb module in pydb_ipy.py, and
 > inject the ipython version of Pdb class to the pydb module namespace?
 > That way main() would instantiate the ipython version...

Good idea! That's be great!

By the way this brings up something else that's been puzzling me:
changing the prompt (Pdb) versus ipdb>. Here's the concern.  If you
are running say from emacs (or something else that is watching output)
the regexp looks for those prompts. The more variation we have, the
more variation we need in those kinds of front-ends. Already I
experienced confusion in those when the wrong regexps are used.  Also,
already there is too much complexity in the regexps.

I'm a culprit of prompt proliferation myself. But here's my rational.
I've changed the prompt from (Pdb) to (Pydb) because the command set
and way things get output (e.g. breakpoint listings, call stack
listings) is already sufficiently different that I think this cue is
needed to help folks keep in mind which command set to use. And such
front-end would need to know too for the same reason.

But I'm not sure the same could be said for the distinction say of
being in a post-mortem called from ipython, Python, or a shell.  One
would know the distinction between which is which because of the way
the output is shown. So is it really necessary to *also* change the
prompt?

 > 
 > 
 > >  > Ok; though does the existence of *list syntax not make pydb.runv()
 > >  > slightly redundant?
 > >
 > > Well, in the os module there is:
 > >   execl(            path, arg0, arg1, ...)
 > >   execle(           path, arg0, arg1, ..., env)
 > >   execlp(           file, arg0, arg1, ...)
 > >   execlpe(    file, arg0, arg1, ..., env)
 > >   execv(            path, args)
 > >   execve(           path, args, env)
 > >   execvp(           file, args)
 > >   execvpe(    file, args, env)
 > >
 > >   spawnl(             mode, path, ...)
 > >   spawnle(      mode, path, ..., env)
 > >   spawnlp(      mode, file, ...)
 > >   spawnlpe(     mode, file, ..., env)
 > >   spawnv(             mode, path, args)
 > >   spawnve(      mode, path, args, env)
 > >   spawnvp(      mode, file, args)
 > >   spawnvpe(     mode, file, args, env)
 > >
 > > The "run" suffixes were drawn from this (which in turn are drawn from
 > > the C library system calls.)
 > 
 > Those functions were introduced long before *arglist syntax (on caller
 > side)... I'm not sure whether anyone actualle "likes" those functions
 > anymore. 

Maybe - I don't know. The *arglist syntax is a been around for a
number of years, and I'll bet you'll find lots of code that was
written/modified after it was around that uses "v" forms. Can you
point me to some discussion on this, like a PEP which substantiates
this claim?

I think of this as sort of in that category of things in ipython where
some places 1/0 is used for True/False. If you want to make it your
style to not use the "v" forms, sure, okay. Personally, I like to
reduce complexity and increase clarity. The v forms which also appear
as a prefix in the C libraries "v*print*", have probably a longer
history than the l suffix forms.


From Fernando.Perez at colorado.edu  Sat Oct 28 14:15:40 2006
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Sat, 28 Oct 2006 12:15:40 -0600
Subject: [IPython-dev] [FWD] IPythonShellEmbed() disturbs regular (embedded)
	Python traceback
Message-ID: <45439E4C.9040100@colorado.edu>

Forwarded to the list, please note that you need to be subscribed to post (we 
get way too much spam otherwise).

I'll reply on-list, sorry for the delay, this slipped in a pile of other email.

Cheers,

f


# ORIGINAL MESSAGE:

Subject:
IPythonShellEmbed() disturbs regular (embedded) Python traceback
From:
"Remy X.O. Martin" <rmvsxop at gmail.com>
Date:
Sat, 30 Sep 2006 15:13:24 +0200
To:
ipython-dev at scipy.org

Hello,

I am embedding Python in one of my C apps, and use IPythonShellEmbed
to support an interactive Python shell, using

      if( !called ){
           PyRun_SimpleString( "from IPython.Shell import IPythonShellEmbed" );
           PyRun_SimpleString( "xgraph.ipshell=IPythonShellEmbed()" );
           PyRun_SimpleString( "xgraph.ipshell.set_banner('Entering
xgraph.ipshell IPython shell')" );
           PyRun_SimpleString( "xgraph.ipshell.set_exit_msg('Leaving
xgraph.ipshell IPython shell')" );
            // Not clear if this is needed or actually does anything:
           PyRun_SimpleString( "xgraph.ipshell.restore_system_completer()" );
           called= 1;
      }


If I call this upon initialising the embedded interpreter, (some)
syntax errors in user code no longer generate useful error messages:

def RedrawNow:
      return None

---------------------------------------------------------------------------
None                                      Traceback (most recent call last)


None: None


If I do *not* initialise the interactive shell, I get the usual message

def RedrawNow:
      return None

   File "<string>", line 1
     def RedrawNow:
                  ^
SyntaxError: invalid syntax


This is of course quite annoying: execution of the Python code stops
at the offending statement, but there is no way to find out what that
statement was.

To be honest, I don't see why IPython would become active outside an
eventual call to xgraph.ipshell() as defined above...?!

Regards,
R.M.


From dd55 at cornell.edu  Sat Oct 28 14:34:30 2006
From: dd55 at cornell.edu (Darren Dale)
Date: Sat, 28 Oct 2006 14:34:30 -0400
Subject: [IPython-dev] object inspection behavior
Message-ID: <200610281434.31302.dd55@cornell.edu>

I was wondering if it is desireable (and possible) for object inspection to 
hide attributes and methods that begin with a leading underscore. 

For example, I have some code that reads a datafile with named columns and 
returns an object with each data column as one of the objects attributes. 
When I want to inspect the object to see what was recorded in the file, I 
don't really need to see the __whatever__ methods that get inherited 
from "object". I would also prefer not to list any methods or attributes that 
the developer has masked with an underscore to make clear they are not 
public.

Is it possible to modify ipython such that tab would not list these 
methods/attributes, but perhaps something like ctrl+tab would?

Thanks,
Darren


From fperez.net at gmail.com  Sat Oct 28 15:52:33 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 28 Oct 2006 13:52:33 -0600
Subject: [IPython-dev] emacs display problems + patch
In-Reply-To: <20060930214556.GI1607@mentat.za.net>
References: <20060930214556.GI1607@mentat.za.net>
Message-ID: <db6b5ecc0610281252h7c9ad0fdp591229511538af1d@mail.gmail.com>

On 9/30/06, Stefan van der Walt <stefan at sun.ac.za> wrote:
> Hi,
>
> Has anyone had a chance to take a look at the attached ipython.el
> patch?

After an unacceptably long delay, entirely my fault, it's been applied
as of r1851.  Thanks!!!

Take care,

f


From fperez.net at gmail.com  Sat Oct 28 16:14:22 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 28 Oct 2006 14:14:22 -0600
Subject: [IPython-dev] object inspection behavior
In-Reply-To: <200610281434.31302.dd55@cornell.edu>
References: <200610281434.31302.dd55@cornell.edu>
Message-ID: <db6b5ecc0610281314o1625157m843e3831b85e5c9a@mail.gmail.com>

On 10/28/06, Darren Dale <dd55 at cornell.edu> wrote:
> I was wondering if it is desireable (and possible) for object inspection to
> hide attributes and methods that begin with a leading underscore.
>
> For example, I have some code that reads a datafile with named columns and
> returns an object with each data column as one of the objects attributes.
> When I want to inspect the object to see what was recorded in the file, I
> don't really need to see the __whatever__ methods that get inherited
> from "object". I would also prefer not to list any methods or attributes that
> the developer has masked with an underscore to make clear they are not
> public.
>
> Is it possible to modify ipython such that tab would not list these
> methods/attributes, but perhaps something like ctrl+tab would?

>From the ipythonrc file:



# (iii) readline_omit__names: normally hitting <tab> after a '.' in a name
# will complete all attributes of an object, including all the special methods
# whose names start with single or double underscores (like __getitem__ or
# __class__).

# This variable allows you to control this completion behavior:

# readline_omit__names 1 -> completion will omit showing any names starting
# with two __, but it will still show names starting with one _.

# readline_omit__names 2 -> completion will omit all names beginning with one
# _ (which obviously means filtering out the double __ ones).

# Even when this option is set, you can still see those names by explicitly
# typing a _ after the period and hitting <tab>: 'name._<tab>' will always
# complete attribute names starting with '_'.

# This option is off by default so that new users see all attributes of any
# objects they are dealing with.

readline_omit__names 0


This should give you all the flexibility you need.

Cheers,

f


From dd55 at cornell.edu  Sat Oct 28 19:57:20 2006
From: dd55 at cornell.edu (Darren Dale)
Date: Sat, 28 Oct 2006 19:57:20 -0400
Subject: [IPython-dev] object inspection behavior
In-Reply-To: <db6b5ecc0610281314o1625157m843e3831b85e5c9a@mail.gmail.com>
References: <200610281434.31302.dd55@cornell.edu>
	<db6b5ecc0610281314o1625157m843e3831b85e5c9a@mail.gmail.com>
Message-ID: <200610281957.21445.dd55@cornell.edu>

On Saturday 28 October 2006 4:14 pm, Fernando Perez wrote:
> On 10/28/06, Darren Dale <dd55 at cornell.edu> wrote:
> > I was wondering if it is desireable (and possible) for object inspection
> > to hide attributes and methods that begin with a leading underscore.
> >
> > For example, I have some code that reads a datafile with named columns
> > and returns an object with each data column as one of the objects
> > attributes. When I want to inspect the object to see what was recorded in
> > the file, I don't really need to see the __whatever__ methods that get
> > inherited from "object". I would also prefer not to list any methods or
> > attributes that the developer has masked with an underscore to make clear
> > they are not public.
> >
> > Is it possible to modify ipython such that tab would not list these
> > methods/attributes, but perhaps something like ctrl+tab would?
>
> From the ipythonrc file:
>
>
>
> # (iii) readline_omit__names: normally hitting <tab> after a '.' in a name
> # will complete all attributes of an object, including all the special
> methods # whose names start with single or double underscores (like
> __getitem__ or # __class__).
>
> # This variable allows you to control this completion behavior:
>
> # readline_omit__names 1 -> completion will omit showing any names starting
> # with two __, but it will still show names starting with one _.
>
> # readline_omit__names 2 -> completion will omit all names beginning with
> one # _ (which obviously means filtering out the double __ ones).
>
> # Even when this option is set, you can still see those names by explicitly
> # typing a _ after the period and hitting <tab>: 'name._<tab>' will always
> # complete attribute names starting with '_'.
>
> # This option is off by default so that new users see all attributes of any
> # objects they are dealing with.
>
> readline_omit__names 0
>
>
> This should give you all the flexibility you need.

Ah, that was a bit of an oversite on my part. I should have known you would 
have included support for this already, I didn't think to look in the rc 
file. Thanks a lot!

Darren


From jorgen.stenarson at bostream.nu  Sun Oct 29 11:47:51 2006
From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Sun, 29 Oct 2006 17:47:51 +0100
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <17730.42678.268147.727141@panix3.panix.com>
References: <17730.42678.268147.727141@panix3.panix.com>
Message-ID: <4544DB37.8070302@bostream.nu>

I would also like to see the ability to have separate history buffers 
for different modes of ipython. I find it a bit annoying to have to wade 
through pdb history when I'm back in standard ipython.

Another thing I'd love to see is the ability to have a context sensitive 
tab completion. For instance when completing a magic like %run it would 
only complete on filenames and the command switches of %run.

If this would require new functionality in the readline module we have 
the possibilty to experiment in pyreadline. Unfortunately at the moment 
pyreadline does not work under unix but for the next version I have done 
work to make it easier to create interfaces to different consoles. If 
anyone would like to pitch in and create something for unix I'm happy to 
include it and will help answer questions about how things work in the 
codebase.

In fact the pyreadline refactor branch is coming along nicely (I use it 
my self daily now). It also has some nice new features: more of the 
editing commands of readline have been implemented, copy paste have been 
improved so you can do selection of text using shift and arrow keys, a 
vi mode. Anyone that would like to take it for a spin can check it out 
from:  http://ipython.scipy.org/svn/ipython/pyreadline/branches/refactor 
using svn.

/J?rgen

R. Bernstein skrev:
> The short one sentence summary is that right now there is a single
> (global) readline history gets used inside %run (if not other places)
> and really the history from an application's %run should separate from
> ipython's history.
> 
> Both ipython and pydb and use readline. When pydb exits it can either
> write its command history, or not save it in which case it is
> truncated. The default is to truncate the history, and I believe the
> effect seen in ipython then, is that after pydb leaves the *ipython*
> history has artifacts of whatever the pydb setting was. That is if
> pydb set to truncate the history, then ipython's history is truncated;
> if not, then the history contains *pydb* entries in addition to the
> ipython entries.
> 
> Sort of. I say this because the ipython "history" command seems to
> work right. So does pydb's equivalent: "show commands".
> 
> There sort of is an implication that adding pydb "broke" things, but
> in thinking and looking at this deeper, it strikes me that the
> readline handling in ipython is currently a little bit broken. Just
> broken in a different way. And I'd be curious to learn if others have
> noticed this as think I have.
> 
> First, *any* program that decides to use readline that is %run from
> ipython (whether -d or not) will have the above issue. So as was
> initially observed, we have interference in the program-to-ipython
> side. But you also have it in the ipython-to-program side. That is,
> inside pdb whenever pdb prompts, *ipython* command completions are
> offered.  This isn't all that helpful. And as mentioned above
> "previous-command" (in Emacs: Ctrl-P) spans across the two, which also
> doesn't seem right.
> 
> I think the right solution then would be for ipython to save its
> history (if history saving is on), truncate the history if it can, run
> the command, and the reinstate the history by rereading it from the
> saved history file. Again, this works for all programs, not just pydb.
> 
> - - - -
> 
> One last strongly related topic which includes Fenando's comment about
> the necessity of Python 2.4 support. It was the following readline
> workaround in Debugger.py around line 68 that caused me to initially
> decided to punt on Python < 2.5:
> 
> 
>     if sys.version[:3] >= '2.5':
>         def __init__(self,color_scheme='NoColor',completekey=None,
>                      stdin=None, stdout=None):
>             ... # I'll come back to this below
>     else:
>         # Ugly hack: for Python 2.3-2.4, we can't call the parent constructor,
>         # because it binds readline and breaks tab-completion.  This means we
>         # have to COPY the constructor here.
>         def __init__(self,color_scheme='NoColor'):
>         ... #1
> 
> I think the intent at #1 above is basically to copy the code in pdb's
> __init__ constructor omitting "import readline" and the code associated
> with implications of that.
> 
> So it looks to me like the test on version 2.5 has more to do with
> testing the version of *pdb* used than anything else version 2.4 or
> version 2.5. Is this correct?
> 
> Assuming this is the case, when I look the 2.4 pdb versus the 2.5 pdb,
> I see THAT THEY BOTH IMPORT READLINE. In other words the 2.5 branch
> ("I'll come back to this below") should have pdb code copied minus the
> import readline. This has to be a separate branch still because it is
> different code. But none of it appears, so I'd be curious to learn why
> it's not needed in the 2.5 branch.
> 
> Perhaps. And perhaps not. Because if the real solution is to:
>  - save the command history file, 
>  - truncate it, 
>  - call "run" (or post-mortem)
>  - restore the history file by rereading it
> then perhaps the above is moot.
> 
> Comments?
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://projects.scipy.org/mailman/listinfo/ipython-dev
> 



From vivainio at gmail.com  Sun Oct 29 13:02:35 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sun, 29 Oct 2006 20:02:35 +0200
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <4544DB37.8070302@bostream.nu>
References: <17730.42678.268147.727141@panix3.panix.com>
	<4544DB37.8070302@bostream.nu>
Message-ID: <46cb515a0610291002w1c598002tce0302b9ae55e4d7@mail.gmail.com>

On 10/29/06, J?rgen Stenarson <jorgen.stenarson at bostream.nu> wrote:

> Another thing I'd love to see is the ability to have a context sensitive
> tab completion. For instance when completing a magic like %run it would
> only complete on filenames and the command switches of %run.
>
> If this would require new functionality in the readline module we have
> the possibilty to experiment in pyreadline. Unfortunately at the moment

This doesn't require functionality in pyreadline, it's perfectly
doable in ipython alone. This feature will probably be in the next
ipython release, even.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Sun Oct 29 15:50:29 2006
From: rocky at panix.com (R. Bernstein)
Date: Sun, 29 Oct 2006 15:50:29 -0500
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <4544DB37.8070302@bostream.nu>
References: <17730.42678.268147.727141@panix3.panix.com>
	<4544DB37.8070302@bostream.nu>
Message-ID: <17733.5141.531684.734361@panix3.panix.com>

J?rgen Stenarson writes:
 > I would also like to see the ability to have separate history buffers 
 > for different modes of ipython. I find it a bit annoying to have to wade 
 > through pdb history when I'm back in standard ipython.

I believe this has been fixed by Ville in SVN and he posted something
to that effect. Again, the fix is not just for pdb, but all %run
programs. This covers the pdb-to-ipython direction I originally
mentioned. However the other direction -- ipython history bleeding
into debuggers -- has not been changed, and apparently Ville seems to
prefer this. I don't, but it's not something to I'd care to press on,
unless others feel that way. And I gather that's not the case.

 > Another thing I'd love to see is the ability to have a context sensitive 
 > tab completion. For instance when completing a magic like %run it would 
 > only complete on filenames and the command switches of %run.

Inside pydb, you will get command completion for debugger command
names. So in that sense there is some context-sensitive
completion. But inside an emacs shell (not via ipython), pydb will
also give you completion on a subcommand name (e.g. set/show/info
commands).  There's something odd in ipython's completion interface
that seems to prevent this (via ipython.el) whereas in emacs, it
doesn't.

I had a private discussion about this with Alexander Schmolck in the
context of improving the behavior in emacs and ipython.el. For each
instance of the problems uncovered, my understanding was that the
completion mechanism in ipython needs improvement first; to date
nobody has been willing to undertake fixing it.

And at present I fall in this category too - I'll be happy if some
sort of minimal pydb interaction gets into the next release; at that
point I'll reassess the effort-to-benefit ratio.

 > 
 > If this would require new functionality in the readline module 

As Ville has confirmed, the problem is not in the readline module(s)
but in ipython's completion interface. If this has been addressed at
the interface or API level in the chain-saw branch, I'd be interested
to learn of that fact.

 > we have 
 > the possibilty to experiment in pyreadline. Unfortunately at the moment 
 > pyreadline does not work under unix but for the next version I have done 
 > work to make it easier to create interfaces to different consoles. If 
 > anyone would like to pitch in and create something for unix I'm happy to 
 > include it and will help answer questions about how things work in the 
 > codebase.
 > 
 > In fact the pyreadline refactor branch is coming along nicely (I use it 
 > my self daily now). It also has some nice new features: more of the 
 > editing commands of readline have been implemented, copy paste have been 
 > improved so you can do selection of text using shift and arrow keys, a 
 > vi mode. Anyone that would like to take it for a spin can check it out 
 > from:  http://ipython.scipy.org/svn/ipython/pyreadline/branches/refactor 
 > using svn.
 > 
 > /J?rgen
 > 
 > R. Bernstein skrev:
 > > The short one sentence summary is that right now there is a single
 > > (global) readline history gets used inside %run (if not other places)
 > > and really the history from an application's %run should separate from
 > > ipython's history.
 > > 
 > > Both ipython and pydb and use readline. When pydb exits it can either
 > > write its command history, or not save it in which case it is
 > > truncated. The default is to truncate the history, and I believe the
 > > effect seen in ipython then, is that after pydb leaves the *ipython*
 > > history has artifacts of whatever the pydb setting was. That is if
 > > pydb set to truncate the history, then ipython's history is truncated;
 > > if not, then the history contains *pydb* entries in addition to the
 > > ipython entries.
 > > 
 > > Sort of. I say this because the ipython "history" command seems to
 > > work right. So does pydb's equivalent: "show commands".
 > > 
 > > There sort of is an implication that adding pydb "broke" things, but
 > > in thinking and looking at this deeper, it strikes me that the
 > > readline handling in ipython is currently a little bit broken. Just
 > > broken in a different way. And I'd be curious to learn if others have
 > > noticed this as think I have.
 > > 
 > > First, *any* program that decides to use readline that is %run from
 > > ipython (whether -d or not) will have the above issue. So as was
 > > initially observed, we have interference in the program-to-ipython
 > > side. But you also have it in the ipython-to-program side. That is,
 > > inside pdb whenever pdb prompts, *ipython* command completions are
 > > offered.  This isn't all that helpful. And as mentioned above
 > > "previous-command" (in Emacs: Ctrl-P) spans across the two, which also
 > > doesn't seem right.
 > > 
 > > I think the right solution then would be for ipython to save its
 > > history (if history saving is on), truncate the history if it can, run
 > > the command, and the reinstate the history by rereading it from the
 > > saved history file. Again, this works for all programs, not just pydb.
 > > 
 > > - - - -
 > > 
 > > One last strongly related topic which includes Fenando's comment about
 > > the necessity of Python 2.4 support. It was the following readline
 > > workaround in Debugger.py around line 68 that caused me to initially
 > > decided to punt on Python < 2.5:
 > > 
 > > 
 > >     if sys.version[:3] >= '2.5':
 > >         def __init__(self,color_scheme='NoColor',completekey=None,
 > >                      stdin=None, stdout=None):
 > >             ... # I'll come back to this below
 > >     else:
 > >         # Ugly hack: for Python 2.3-2.4, we can't call the parent constructor,
 > >         # because it binds readline and breaks tab-completion.  This means we
 > >         # have to COPY the constructor here.
 > >         def __init__(self,color_scheme='NoColor'):
 > >         ... #1
 > > 
 > > I think the intent at #1 above is basically to copy the code in pdb's
 > > __init__ constructor omitting "import readline" and the code associated
 > > with implications of that.
 > > 
 > > So it looks to me like the test on version 2.5 has more to do with
 > > testing the version of *pdb* used than anything else version 2.4 or
 > > version 2.5. Is this correct?
 > > 
 > > Assuming this is the case, when I look the 2.4 pdb versus the 2.5 pdb,
 > > I see THAT THEY BOTH IMPORT READLINE. In other words the 2.5 branch
 > > ("I'll come back to this below") should have pdb code copied minus the
 > > import readline. This has to be a separate branch still because it is
 > > different code. But none of it appears, so I'd be curious to learn why
 > > it's not needed in the 2.5 branch.
 > > 
 > > Perhaps. And perhaps not. Because if the real solution is to:
 > >  - save the command history file, 
 > >  - truncate it, 
 > >  - call "run" (or post-mortem)
 > >  - restore the history file by rereading it
 > > then perhaps the above is moot.
 > > 
 > > Comments?
 > > _______________________________________________
 > > IPython-dev mailing list
 > > IPython-dev at scipy.org
 > > http://projects.scipy.org/mailman/listinfo/ipython-dev
 > > 
 > 


From vivainio at gmail.com  Sun Oct 29 16:06:31 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sun, 29 Oct 2006 23:06:31 +0200
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <17733.5141.531684.734361@panix3.panix.com>
References: <17730.42678.268147.727141@panix3.panix.com>
	<4544DB37.8070302@bostream.nu>
	<17733.5141.531684.734361@panix3.panix.com>
Message-ID: <46cb515a0610291306s142878f4k729f5f14d947aa56@mail.gmail.com>

On 10/29/06, R. Bernstein <rocky at panix.com> wrote:

>  > If this would require new functionality in the readline module
>
> As Ville has confirmed, the problem is not in the readline module(s)
> but in ipython's completion interface. If this has been addressed at
> the interface or API level in the chain-saw branch, I'd be interested
> to learn of that fact.

No need for chainsaw, I'll add a:

complete_symbol(line, pos_in_line) -> [completions]

(or something to that effect) hook into the stable branch soon.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Sun Oct 29 16:29:29 2006
From: rocky at panix.com (R. Bernstein)
Date: Sun, 29 Oct 2006 16:29:29 -0500
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <46cb515a0610291306s142878f4k729f5f14d947aa56@mail.gmail.com>
References: <17730.42678.268147.727141@panix3.panix.com>
	<4544DB37.8070302@bostream.nu>
	<17733.5141.531684.734361@panix3.panix.com>
	<46cb515a0610291306s142878f4k729f5f14d947aa56@mail.gmail.com>
Message-ID: <17733.7481.460297.914793@panix3.panix.com>

Ville M. Vainio writes:
 > On 10/29/06, R. Bernstein <rocky at panix.com> wrote:
 > 
 > >  > If this would require new functionality in the readline module
 > >
 > > As Ville has confirmed, the problem is not in the readline module(s)
 > > but in ipython's completion interface. If this has been addressed at
 > > the interface or API level in the chain-saw branch, I'd be interested
 > > to learn of that fact.
 > 
 > No need for chainsaw, I'll add a:
 > 
 > complete_symbol(line, pos_in_line) -> [completions]

I think this would suffice or at least go in the right direction.

For what it's worth, this was Alexander Schmolck's view on the matter:

  What should really happen IMO, is that ipython offer some sensible
  completion interface to editor-shells etc. that is agnostic of the
  mode (plain, p(y)db or whatever else the future may hold) and just
  take the whole command line so far and return the possible
  completions for the last word of that line. That would also make
  things like e.g. filename completion for contexts such as 
     open("<TAB> 
  possible (I'm pretty sure ipython outside of emacs already
  offers better context sensitive completion, although I never run
  ipython outside emacs).

So to reconcile these two views, basically the calling program would
have to find where the last "word" begins. Probably okay.

Looking to see how this matches the corresponding GNU Emacs interface
gud-gdb-complete-command() I see that that the routine picks out the
last word rather than being told where the last word starts. Since
finding word boundaries may be a little application specific, perhaps
passing in a position is not so bad. But maybe better though would to
make "pos_in_line" optional and use a default find last word search to
pick the word out.

Don't know. Just throwing all of this out for thought. 


From jorgen.stenarson at bostream.nu  Sun Oct 29 16:33:41 2006
From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Sun, 29 Oct 2006 22:33:41 +0100
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <17733.5141.531684.734361@panix3.panix.com>
References: <17730.42678.268147.727141@panix3.panix.com>	<4544DB37.8070302@bostream.nu>
	<17733.5141.531684.734361@panix3.panix.com>
Message-ID: <45451E35.8050208@bostream.nu>

R. Bernstein skrev:

>  > 
>  > If this would require new functionality in the readline module 
> 
> As Ville has confirmed, the problem is not in the readline module(s)
> but in ipython's completion interface. If this has been addressed at
> the interface or API level in the chain-saw branch, I'd be interested
> to learn of that fact.
> 
The reason for my comment on perhaps needing some new functionality in 
readline was the comment below present in completer.py. I have no idea 
if that comment is still valid though. I would be happy to work on this 
(if it is desirable) if someone can explain what is needed.


     def file_matches(self, text):
         """Match filneames, expanding ~USER type strings.

         Most of the seemingly convoluted logic in this completer is an
         attempt to handle filenames with spaces in them.  And yet it's not
         quite perfect, because Python's readline doesn't expose all of the
         GNU readline details needed for this to be done correctly.


From vivainio at gmail.com  Sun Oct 29 17:30:11 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 30 Oct 2006 00:30:11 +0200
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <17733.7481.460297.914793@panix3.panix.com>
References: <17730.42678.268147.727141@panix3.panix.com>
	<4544DB37.8070302@bostream.nu>
	<17733.5141.531684.734361@panix3.panix.com>
	<46cb515a0610291306s142878f4k729f5f14d947aa56@mail.gmail.com>
	<17733.7481.460297.914793@panix3.panix.com>
Message-ID: <46cb515a0610291430k3aab9bcx9c56e3e33238950@mail.gmail.com>

On 10/29/06, R. Bernstein <rocky at panix.com> wrote:

> Looking to see how this matches the corresponding GNU Emacs interface
> gud-gdb-complete-command() I see that that the routine picks out the
> last word rather than being told where the last word starts. Since
> finding word boundaries may be a little application specific, perhaps
> passing in a position is not so bad. But maybe better though would to
> make "pos_in_line" optional and use a default find last word search to
> pick the word out.

pos_in_line is necessary because the cursor could be positioned like
this on the moment of pressing tab:

cp /var/lo[cursor] /home/foo

I thought passing the last "symbol" as convenience (in this example,
'/var/lo') could be handy, but it would bloat the hook too much.

I guess I'll pass a "Struct" object, where the hook would be:

def my_complete_symbol(event):
 print event.line, event.pos_in_line, event.symbol, event.command

(where 'command' would be the first word).

symbol and command are there purely for the convenience.

I think the completion hooks should be dispatchable by both of:

- Dictionary lookup with 'command' (for performance)
- Regexp (where the hooks with empty regexp are always tried).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From rocky at panix.com  Mon Oct 30 07:36:50 2006
From: rocky at panix.com (R. Bernstein)
Date: Mon, 30 Oct 2006 07:36:50 -0500
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <46cb515a0610291430k3aab9bcx9c56e3e33238950@mail.gmail.com>
References: <17730.42678.268147.727141@panix3.panix.com>
	<4544DB37.8070302@bostream.nu>
	<17733.5141.531684.734361@panix3.panix.com>
	<46cb515a0610291306s142878f4k729f5f14d947aa56@mail.gmail.com>
	<17733.7481.460297.914793@panix3.panix.com>
	<46cb515a0610291430k3aab9bcx9c56e3e33238950@mail.gmail.com>
Message-ID: <17733.61922.526760.241483@panix3.panix.com>

Here's a suggestion for a what I think is a simpler interface.

First I hadn't realized that previously:

   complete_symbol(line, pos_in_line) -> [completions]

pos_in_line referred to the *end* of the word to complete rather than
the beginning of it, and that there was an implicit search *back* from
that to find the word start boundary. I just assumed the word
completion went from pos_in_line to the end of the string. So that
also means that I hadn't realized that there was some implied regular
expression that was getting used to search back from. And with the
previous interface that still seems to happen in some situations.

Let's start out with the simplest definition:

   complete_symbol(line) -> [completions]

or should it be (completions)?

What this would do is give completions of the last "word" in line with
an implicit search back to find the word in line. But as you point
out, the word one wants to complete might not be the last word but be
a word somewhere in the middle of the line, and one wants to keep the
context to the right in the string the same. Okay then let's try:

   complete_symbol(line, endword_pos=-1) -> [completions]

But now there is an observation that finding the word may be context
sensitive. In the "Struct" or object interface, this is solved by
having the caller pick out that part and pass that. I think it's
called "symbol". But two other solutions I think that are simpler at
least from the caller's standpoint is to give the position of the
start of the word or a regexp to use to search back to instead of the
default look-for-blank (or maybe its look-for-non-alphanum) regular
expression that is implied previously.

Allowing a start position I think would be the simplest:

   complete_symbol(line, endword_pos=-1, startword_pos=None) -> [completions]

But if one really wants that regexp, because it saves the caller from
having to do a search first (and presumably within the context of the
caller it knows what the right regexp should be) then:

   complete_symbol(line, endword_pos=-1, re_str="[ \t]") 
   -> [completions]
or 
   complete_symbol(line, endword_pos=-1, startword_pos=None, re_str='[ \t]') 
   -> [completions]
   # Note: re_str is used *only* if startword_pos is None

Here's why I find this nicer. My guess is more than half of the time
the call needs only to be:

   complete_symbol(line)

and 90% of the time (actually 100% in the cases in either debugger pdb
or pydb)
   complete_symbol(line, endword_pos)

will suffice. The thing about the object parameter interface is that
the caller has to create it. And the line portion is *not* optional,
so why not make that a required parameter? It reduces run-time errors
at least in that aspect. And I don't see a need for giving the
"command" part because at least in my uses, I arrange for what the
valid completions can be *in advance* of the completion calls.


Ville M. Vainio writes:
 > On 10/29/06, R. Bernstein <rocky at panix.com> wrote:
 > 
 > > Looking to see how this matches the corresponding GNU Emacs interface
 > > gud-gdb-complete-command() I see that that the routine picks out the
 > > last word rather than being told where the last word starts. Since
 > > finding word boundaries may be a little application specific, perhaps
 > > passing in a position is not so bad. But maybe better though would to
 > > make "pos_in_line" optional and use a default find last word search to
 > > pick the word out.
 > 
 > pos_in_line is necessary because the cursor could be positioned like
 > this on the moment of pressing tab:
 > 
 > cp /var/lo[cursor] /home/foo
 > 
 > I thought passing the last "symbol" as convenience (in this example,
 > '/var/lo') could be handy, but it would bloat the hook too much.
 > 
 > I guess I'll pass a "Struct" object, where the hook would be:
 > 
 > def my_complete_symbol(event):
 >  print event.line, event.pos_in_line, event.symbol, event.command
 > 
 > (where 'command' would be the first word).
 > 
 > symbol and command are there purely for the convenience.
 > 
 > I think the completion hooks should be dispatchable by both of:
 > 
 > - Dictionary lookup with 'command' (for performance)
 > - Regexp (where the hooks with empty regexp are always tried).
 > 
 > -- 
 > Ville M. Vainio - vivainio.googlepages.com
 > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
 > 


From vivainio at gmail.com  Mon Oct 30 08:33:54 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 30 Oct 2006 15:33:54 +0200
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <45451E35.8050208@bostream.nu>
References: <17730.42678.268147.727141@panix3.panix.com>
	<4544DB37.8070302@bostream.nu>
	<17733.5141.531684.734361@panix3.panix.com>
	<45451E35.8050208@bostream.nu>
Message-ID: <46cb515a0610300533i6774ae3ciebaa96e11adf7cca@mail.gmail.com>

On 10/29/06, J?rgen Stenarson <jorgen.stenarson at bostream.nu> wrote:

> The reason for my comment on perhaps needing some new functionality in
> readline was the comment below present in completer.py. I have no idea
> if that comment is still valid though. I would be happy to work on this
> (if it is desirable) if someone can explain what is needed.

The "dirty" code requirement is still valid, but I don't see a need to
deal with it through editing pyreadline at this point.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Mon Oct 30 08:47:10 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 30 Oct 2006 15:47:10 +0200
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <17733.61922.526760.241483@panix3.panix.com>
References: <17730.42678.268147.727141@panix3.panix.com>
	<4544DB37.8070302@bostream.nu>
	<17733.5141.531684.734361@panix3.panix.com>
	<46cb515a0610291306s142878f4k729f5f14d947aa56@mail.gmail.com>
	<17733.7481.460297.914793@panix3.panix.com>
	<46cb515a0610291430k3aab9bcx9c56e3e33238950@mail.gmail.com>
	<17733.61922.526760.241483@panix3.panix.com>
Message-ID: <46cb515a0610300547i1175ecd5uf4633398580af54c@mail.gmail.com>

On 10/30/06, R. Bernstein <rocky at panix.com> wrote:

> called "symbol". But two other solutions I think that are simpler at
> least from the caller's standpoint is to give the position of the
> start of the word or a regexp to use to search back to instead of the
> default look-for-blank (or maybe its look-for-non-alphanum) regular
> expression that is implied previously.

I think the caller's simplicity is slightly irrelevant here - the data
structure would be constructed only once and passed to possibly
several hooks, which need to be fast.

> Here's why I find this nicer. My guess is more than half of the time
> the call needs only to be:
>
>    complete_symbol(line)

As I see it, the easiest (and probably most typical) use is:

if event.command == 'apt-get':
  return [c for c in ['dist-upgrade','upgrade', 'update']  if
c.startswith(event.symbol)]

> will suffice. The thing about the object parameter interface is that
> the caller has to create it. And the line portion is *not* optional,

It's easy & cheap to create (and importantly, extend, without having
to alter the existing hooks).

> so why not make that a required parameter? It reduces run-time errors

In my example above, the use of whole 'line' is optional.

> at least in that aspect. And I don't see a need for giving the
> "command" part because at least in my uses, I arrange for what the
> valid completions can be *in advance* of the completion calls.

Again, see the example above. The 'command' makes it really easy to
implement many of the hooks.

A drawback with event object in hooks is that the signature of the
hook function does not fully document the behaviour of the hook, but I
believe we can be quite informative about it in our exceptions and
docstrings.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Mon Oct 30 08:52:38 2006
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 30 Oct 2006 15:52:38 +0200
Subject: [IPython-dev] ipython/pydb integration - readline
In-Reply-To: <46cb515a0610300547i1175ecd5uf4633398580af54c@mail.gmail.com>
References: <17730.42678.268147.727141@panix3.panix.com>
	<4544DB37.8070302@bostream.nu>
	<17733.5141.531684.734361@panix3.panix.com>
	<46cb515a0610291306s142878f4k729f5f14d947aa56@mail.gmail.com>
	<17733.7481.460297.914793@panix3.panix.com>
	<46cb515a0610291430k3aab9bcx9c56e3e33238950@mail.gmail.com>
	<17733.61922.526760.241483@panix3.panix.com>
	<46cb515a0610300547i1175ecd5uf4633398580af54c@mail.gmail.com>
Message-ID: <46cb515a0610300552l606ae00ckd2c2de62b94ccd6b@mail.gmail.com>

On 10/30/06, Ville M. Vainio <vivainio at gmail.com> wrote:

> I think the caller's simplicity is slightly irrelevant here - the data
> structure would be constructed only once and passed to possibly
> several hooks, which need to be fast.

And to avoid a misunderstanding, the 'caller' is always IPython, and
not user/extender customizable. The hooks are always the callees, and
are implemented by those who wish to extend IPython.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Mon Oct 30 15:19:10 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 30 Oct 2006 13:19:10 -0700
Subject: [IPython-dev] Functional Trac user registration
Message-ID: <db6b5ecc0610301219i43be5370tdbdd206dbd8f5b6e@mail.gmail.com>

Hi all,

with a huge thank you to Jeff Strunk from Enthought, we now have
functional registration for new users in Trac!  This means that anyone
can now submit tickets to IPython, by simply making a user account on
the Trac system.  There's a 'Register' link at the top right, which
will take you here:

http://projects.scipy.org/ipython/ipython/register

After that you can make new tickets as needed.  The registration step
is needed because otherwise the flood of spam to the system is
unstoppable.

We'll update the wiki and make prominent links to this soon.

This completes our transition from the old Roundup/manual-ugly
homepage to the Trac/Moin environment in terms of needed
functionality.

Regards,

f