[IPython-dev] Update on IPython recent developments

[Fernando Perez]

On 1/25/07, Fernando Perez <fperez.net at gmail.com> wrote:

> Scratch that.  When I said 'last piece' above, it's because I'd given
> up on the possibility of doctesting code that spawns subprocesses, due
> to the fact that their stdout file descriptors are NOT the sys.stdout
> of the original process.  This was stated in the 'limitations' section
> here:
>
> http://projects.scipy.org/ipython/ipython/browser/ipython/branches/saw/ipython1/test/ipdoctest.py
>
> I now realized that there /is/ a solution to this problem using
> pexpect, so before I commit the lot I'll solve this problem as well.
> This will really leave us with a system where it will be /very easy/
> to add doctests for pretty much any of ipython's functionality.  I'm
> sure there will still  be a few dark corners, but I'm now determined
> to reduce that count to zero.

Done.  We now support 3 types of doctests:

1. Plain python ones (that's just what the doctest module does)
2. IPython-syntax ones, but which do NOT need direct access to the OS.
 Since these can be run in-process, just doing a syntax conversion
(ipython->python) is enough.

And as of today,

3. IPython examples that call aliases, system calls, use !cmd, etc.
These MUST be run in a separate process via pexpect, so we can access
the underlying file descriptors for output capture.  In this mode we
lose a few things, like the doctest sophisticated exception analysis,
but by and large this works great.  Doctests with exceptions or
fancier needs can always be coded in mode #2 above, so I don't see
this as any major limitation.

  These doctests simply need to be marked with

  # ipython-doctest: EXTERNAL


The code needs much cleanup, which I'll do as soon as I catch a
breather and will then commit it.

It's worth noting that unfortunately none of this works under Win32,
because there is no equivalent of pexpect there.  The author seems to
be working on a win32 port, but nothing exists yet.  Since pexpect is
a critical component of this, the test suite will simply not work
under win32.  Patches to at least induce more graceful failure will be
welcome (I have no clue how badly the current code will explode).

Linux, BSD, OSX and related should all be fine (if not, I'll consider
that a bug).

Cheers,

f