[IPython-dev] ipython/pydb integration - readline

[R. Bernstein]

I thought I'd bring to a wider development audience some of the recent
observations and issues with ipython and pydb integration. 

As with pdoc/pdef, I don't think the issues are strictly tied to pydb,
but any program one %run's that uses readline.

(Ville: I'm adding observations I've come across subsequently so this
doesn't exactly follow the email exchanges or that history.)

The short one sentence summary is that right now there is a single
(global) readline history gets used inside %run (if not other places)
and really the history from an application's %run should separate from
ipython's history.

Both ipython and pydb and use readline. When pydb exits it can either
write its command history, or not save it in which case it is
truncated. The default is to truncate the history, and I believe the
effect seen in ipython then, is that after pydb leaves the *ipython*
history has artifacts of whatever the pydb setting was. That is if
pydb set to truncate the history, then ipython's history is truncated;
if not, then the history contains *pydb* entries in addition to the
ipython entries.

Sort of. I say this because the ipython "history" command seems to
work right. So does pydb's equivalent: "show commands".

There sort of is an implication that adding pydb "broke" things, but
in thinking and looking at this deeper, it strikes me that the
readline handling in ipython is currently a little bit broken. Just
broken in a different way. And I'd be curious to learn if others have
noticed this as think I have.

First, *any* program that decides to use readline that is %run from
ipython (whether -d or not) will have the above issue. So as was
initially observed, we have interference in the program-to-ipython
side. But you also have it in the ipython-to-program side. That is,
inside pdb whenever pdb prompts, *ipython* command completions are
offered.  This isn't all that helpful. And as mentioned above
"previous-command" (in Emacs: Ctrl-P) spans across the two, which also
doesn't seem right.

I think the right solution then would be for ipython to save its
history (if history saving is on), truncate the history if it can, run
the command, and the reinstate the history by rereading it from the
saved history file. Again, this works for all programs, not just pydb.

- - - -

One last strongly related topic which includes Fenando's comment about
the necessity of Python 2.4 support. It was the following readline
workaround in Debugger.py around line 68 that caused me to initially
decided to punt on Python < 2.5:


    if sys.version[:3] >= '2.5':
        def __init__(self,color_scheme='NoColor',completekey=None,
                     stdin=None, stdout=None):
            ... # I'll come back to this below
    else:
        # Ugly hack: for Python 2.3-2.4, we can't call the parent constructor,
        # because it binds readline and breaks tab-completion.  This means we
        # have to COPY the constructor here.
        def __init__(self,color_scheme='NoColor'):
        ... #1

I think the intent at #1 above is basically to copy the code in pdb's
__init__ constructor omitting "import readline" and the code associated
with implications of that.

So it looks to me like the test on version 2.5 has more to do with
testing the version of *pdb* used than anything else version 2.4 or
version 2.5. Is this correct?

Assuming this is the case, when I look the 2.4 pdb versus the 2.5 pdb,
I see THAT THEY BOTH IMPORT READLINE. In other words the 2.5 branch
("I'll come back to this below") should have pdb code copied minus the
import readline. This has to be a separate branch still because it is
different code. But none of it appears, so I'd be curious to learn why
it's not needed in the 2.5 branch.

Perhaps. And perhaps not. Because if the real solution is to:
 - save the command history file, 
 - truncate it, 
 - call "run" (or post-mortem)
 - restore the history file by rereading it
then perhaps the above is moot.

Comments?