[IPython-dev] correct test-suite

Mon Jul 26 03:18:50 EDT 2010

Am Sun, 25 Jul 2010 19:49:18 -0700
schrieb Fernando Perez <fperez.net at gmail.com>:

> Hi Tom,
> 
> On Sun, Jul 18, 2010 at 8:14 AM, Thomas Spura
> <tomspur at fedoraproject.org> wrote:
> > There is now a Makefile, so it's nicer to run repetive tasks in the
> > repository, but currently there is only 'make test-suite', which
> > should run the test suite.
> > (now = in branch my_fix_test_suite at github:
> > http://github.com/tomspur/ipython/commits/my_fix_test_suite)
> 
> Unfortunately this approach does not work if IPython isn't installed
> system-wide, because it breaks the twisted tests:
> 
> ===============================================================================
> [ERROR]: IPython.kernel
> 
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/dist-packages/twisted/trial/runner.py",
> line 651, in loadByNames
>     things.append(self.findByName(name))
>   File "/usr/lib/python2.6/dist-packages/twisted/trial/runner.py",
> line 461, in findByName
>     return reflect.namedAny(name)
>   File "/usr/lib/python2.6/dist-packages/twisted/python/reflect.py",
> line 471, in namedAny
>     raise ObjectNotFound('%r does not name an object' % (name,))
> twisted.python.reflect.ObjectNotFound: 'IPython.kernel' does not name
> an object
> -------------------------------------------------------------------------------
> 
> as well as some others.  And the manual sys.path manipulations done
> here:
> 
> http://github.com/tomspur/ipython/commit/7abd52e0933aa57a082db4623c791630bc0671ea#L0R51
> 
> should be avoided in production code, which iptest.py is.
> 
> I'm not opposed to having a top-level makefile, but the one in your
> commit can't be merged for this reason.  Additionally, it makes
> targets for things like tar/zip generation that should be done as
> distutils commands instead (and they are done by some of the scripts
> in tools/).
> 
> In practice, I find that simply having the current working IPython/
> directory symlinked in my PYTHONPATH lets me run the iptest script at
> any time without further PYTHONPATH manipulations.

I did now something similar and linked
$(pythondir)/site-packages/IPython -> git repository. Seems to work
pretty well.

> 
> I'm happy to find a solution to running the test suite without
> installing, but this one doesn't seem to work robustly (and I'd
> already backed off a while ago some more hackish things I'd tried,
> precisely for being too brittle).
> 
> Until we have a really clean solution, we'll have a test suite that
> can only be run from an installed IPython (or equivalently, a setup
> that finds the source tree transparently, which is what I use).
> 
> > One failing test pointed out, that there is a programming error in
> > IPython/Shell.py and is now corrected in this commit:
> > http://github.com/tomspur/ipython/commit/7e7988ee9e7c35b2e5302725ebdf6c22135f334e
> 
> This one I did just cherry-pick and push, as it was really a clean
> bug, thanks for the fix:
> 
> http://github.com/ipython/ipython/commit/a469f3d77cf794b33ac20cf9d3f2246387423808
> 
> > But now, there is a problem with test: "Test that object's __del__
> > methods are called on exit." in IPython/core/tests/test_run.py:146.
> 
> I think that's all caused by the problems you are seeing from your
> method of running the tests.  On my system, all tests do pass cleanly
> right now:
> 
> **********************************************************************
> Test suite completed for system with the following information:
> IPython version: 0.11.alpha1.git
> BZR revision   : 0
> Platform info  : os.name -> posix, sys.platform -> linux2
>                : Linux-2.6.32-24-generic-i686-with-Ubuntu-10.04-lucid
> Python info    : 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
> [GCC 4.4.3]
> 
> Tools and libraries available at test time:
>    curses foolscap gobject gtk pexpect twisted wx wx.aui
> zope.interface
> 
> Tools and libraries NOT available at test time:
>    objc
> 
> Ran 10 test groups in 46.446s
> 
> Status:
> OK
> ####

Here, they don't... That's why, I didn't look too closely to the
failing tests in my branches. I'll try to fix the failures in current
master on my side first, because it seems some other dependencies are
doing something wrong I guess...

**********************************************************************
Test suite completed for system with the following information:
IPython version: 0.11.alpha1.git
BZR revision   : 0
Platform info  : os.name -> posix, sys.platform -> linux2
               :
Linux-2.6.33.6-147.fc13.x86_64-x86_64-with-fedora-13-Goddard Python
info    : 2.6.4 (r264:75706, Jun  4 2010, 18:20:31) [GCC 4.4.4 20100503
(Red Hat 4.4.4-2)]

Tools and libraries available at test time:
   curses foolscap gobject gtk pexpect twisted wx wx.aui zope.interface

Tools and libraries NOT available at test time:
   objc

Ran 10 test groups in 68.690s

Status:
ERROR - 2 out of 10 test groups failed.
----------------------------------------
Runner failed: IPython.core
You may wish to rerun this one individually, with:
/usr/bin/python /usr/lib/python2.6/site-packages/IPython/testing/iptest.py
IPython.core

----------------------------------------
Runner failed: IPython.extensions
You may wish to rerun this one individually, with:
/usr/bin/python /usr/lib/python2.6/site-packages/IPython/testing/iptest.py
IPython.extensions

> 
> I also commented on your bundled_libs branch: it can't be merged
> because it also breaks most of the Twisted tests.  Until the test
> suite passes 100% we can't merge those changes, though I do very much
> like the idea of better organizing externals, so I hope you can sort
> out the issue.  Do let us know as soon as you can fix those and we can
> try again.

Will do. :)

> So I think right now we have merged everything that is mergeable from
> you, right?  Please go ahead and file pull requests again if you do
> update these (since those trigger an email and that makes it easier to
> keep track of what's been done).

I think, it's quite handy to see on which commit you run ipython. e.g.
when receiving bug reports.
So this commit could be merged:
http://github.com/tomspur/ipython/commit/936858ba3e6648a8bc0031cb76a94643dcdb080a

(This also works, when the IPython directory is symlinked somewhere
else.)

I'll rework all other commits and will do a pull request again.

Thanks.
   Tom