As some of you have seen (at least Stephen), I filed a ticket this morning about increasing testing coverage. The other night Anthony and I met up in NYC and he had something of an "intervention" about the sufficiency of answer testing for yt; it didn't take too much work on his part to convince me that we should be testing not just against a gold standard, but also performing unit tests. In the past I had eschewed unit testing simply because the task of mocking data was quite tricky, and by adding tests that use smaller bits we could cover unit testable areas with answer testing.
But, this isn't really a good strategy. Let's move to having both. The testing infrastructure he recommends is the nearly-omnipresent nose:
The ticket to track this is here:
There are a couple sub-items here:
1) NumPy's nose test plugins provide a lot of necessary functionality that we have reimplemented in the answer testing utilities. I'd like to start using the numpy plugins, which include things like conditional test execution, array comparisons, "slow" tests, etc etc. 2) We can evaluate, using conditional test execution, moving to nose for answer testing. But that's not on the agenda now. 3) Writing tests for nose is super easy, and running them is too. Just do:
nosetest -w yt/
when in your source directory.
4) I've written a simple sample here:
5) I'll handle writing up some mock data that doesn't require shipping lots of binary files, which can then be used for checking things that absolutely require hierarchies.
The way to organize tests is easy. Inside each directory with testable items create a new directory called "tests", and in here toss some scripts. You can stick a bunch of functions in those scripts.
Anyway, I'm going to start writing more of these (in the main yt repo, and this change will be grafted there as well) and I'll write back once the data mocking is ready. I'd like it if we started encouraging or even mandating simple tests (and/or answer tests) for functionality that gets added, but that's a discussion that should be held separately.
The items on the ticket:
Is anyone willing to claim any additional items that they will help write unit tests for?