[IPython-dev] Proposal to get unit test coverage as soon as possible

[Brian Granger]

> I suppose we should try to cover as much of the classic ipython code
> as possible. Even if we can't easily ensure that something behaves
> correctly without lot of stubbing work (as opposed to something
> obvious such as crashing), we should at least *excercise* as much of
> the code as possible.

Yes, definitely.  For example, I just added a get_ipython_dir function
to genutils.py and the best way of testing it is too simply all it.
Of course, with nose and all the new testing capabilities that
Fernando has added, in many cases we can do more details testing.
Fernando, do you have plans on writing up some Sphinx docs in the
developer section about testing?  I know that your testing stuff does
*way* more than we are taking advantage of.

> If thinking about perfect test case prevents you
> from writing anything, stop thinking and start writing ;-).

Amen!

> Additionally, I don't think unit tests need to go through code review
> process. They won't break anything, and when they themselves break it
> is quite obvious why ;-)

I disagree with this and here it why:

* I just spent at least 3-5 full days fixing test code that was broken
in subtle ways for the 0.9 release.  The underlying code was just
fine, the tests were broken in some subtle manner.

* Getting test code right is often more difficult than getting the
underlying code right.  There are subtleties with Twisted, platform
dependencies, tests that take a very long time to run, etc., etc.

* Tests need to be documented just like regular code.

* More and more, our tests need to follow certain conventions.  For
example, we now have decorators that can be applied to test that
should be skipped on particular platforms, like @skip_win32, etc.

* IPython is getting big enough that no single person can keep track
of all of everything.  This is particularly difficult when different
parts of our code are coupled in seemingly unrelated ways.  For
example the usage of thread in pylab and in the parallel computing
stuff in IPython.kernel.

I do think that all of these things apply to testing the "older" parts
of IPython as well.

Also, I like applying code review uniformly to our code as it
encourages a very open and honest development process where we all
learn to give and take productive criticism.  One of the biggest
reasons I want universal code review, is that I want *my* code
reviwed.  Not that I write bad code, but I do at times get lazy and/or
miss things.

So, I vote for no exceptions to code review for tests

PS - because I do want everyone to start writing lots of tests, I
think it is important that all of us really make an effort to do code
reviews for this stuff promptly.

Cheers,

Brian

> --
> Ville M. Vainio
> http://tinyurl.com/vainio
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>