[IPython-dev] iptest and the nasty twisted bug

[Fernando Perez]

Hi all,

I've spent hours trying to fix the interaction between twisted and
ipdoctest, to no avail.  I have no clue what's going on, the problem
is intermittent and makes no sense to me.  But what I realized is that
at least us developers have a simple, if clunky, workaround so that we
can run  the test suite by default in a twisted-robust way, while also
running with full ipdoctest support.

If you now run:

iptest [any options] [any submodules]

it will  NOT load ipdoctest but won't cause any Twisted problems.

Once you're happy that you didn't break Twisted, run

iptest --with-ipdoctest [any options] [any submodules]

This MAY give a Twisted AlreadyCalledError exception at the end, but
it will also correctly load up all of the ipython-specific tests and
doctests.

The above can be made easier with a trivial  shell  alias

alias iptest2='iptest --with-ipdoctest'


So that you can run

iptest ...
# Twisted happy
iptest2 ...
# ignore possible Twisted error, this checks all the rest.

I know this is not ideal, but at least it does give us a way to run
all the proper IPython tests and doctests.  I have not the foggiest
idea how that error in Twisted is occurring: I'm not a Twisted expert,
and the behavior is both intermittent and, to me, nonsensical.  The
ipdoctest plugin in principle doesn't do anything with Twisted or any
global state (its changes to global state are happening anyway even
now, so that's not the problem).  I'm completely puzzled  by this.

**** But it is CRITICAL that all developers run the full suite with
--with-ipdoctest ALWAYS before committing code  ****

It may be annoying to have to run  the test suite twice, but that's
the only way we can get test coverage of ipython magics and
extensions.  In my branch there are already 90 more tests picked up
this way, and this is only bound to grow as we add ipython-specific
tests.  So these MUST be always run.

I'd love to find a cleaner solution,  and I'm sure we will eventually.
 But for now, this is how it will need to be done...

Cheers,

f