skipping one unittest assertion?

[Roy Smith]

In article <87wriah4qg.fsf at benfinney.id.au>,
 Ben Finney <ben+python at benfinney.id.au> wrote:

> Roy Smith <roy at panix.com> writes:
> 
> > There's one particular assertion in a test method which depends on 
> > production code that hasn't been written yet.  I could factor that out 
> > into its own test methods and @skip that, but it would be cleaner to be 
> > able to mark the particular assertion.
> 
> I think that's backward. If there are multiple conditions to assert,
> then it's much cleaner to put them in distinct test case methods.
> 
> Each test case should be testing one thing: if it fails, it should be
> for exactly one reason.

Well, yeah, that's certainly the XP/unit-test doctrine.  In practice 
however, tests often get written as "do a bunch of stuff to get some 
result, then make a bunch of assertions about that result".

Sure, you could make each of those assertions a distinct method, and 
factor the "do a bunch of stuff" into setUp().  Which probably means 
factoring those methods out into a new TestCase subclass.  At some 
point, however practicality trumps doctrine.

I notice the Python internal tests adopt exactly the same attitude.  
Here's an example from 3.2's test_urllib.py:

    def test_fileno(self):
        file_num = self.returned_obj.fileno()
        self.assertIsInstance(file_num,
                              int,
                             "fileno() did not return an int")
        self.assertEqual(os.read(file_num, len(self.text)),
               self.text,
               "Reading on the file descriptor returned by fileno() "
               "did not return the expected text")

I've got a remote API which at some point returns a URL (wrapped up in a 
larger JSON-ized object).  I want to make several assertions about the 
object.  It should be valid JSON.  It should unpack to yield an object 
(well, dict in python) with certain attributes.  Those values of those 
attributes should be of the correct types, etc.  So:

        self.assertIsInstance(station['id'], int)
        self.assertIsInstance(station['name'], unicode)
        self.assertIsInstance(station['cover_url'], unicode)

The next assertion (the one that I know will fail because the server 
code hasn't been written yet) is that cover_url refers to a get-able 
jpeg image.  It would be convenient and useful to write the assertion 
now and mark that I expect it to fail until the server code catches up.

Breaking all this up into distinct test methods would be silly.  In 
fact, it would be wrong.  If they were different methods, they would not 
be making multiple assertions against one object retrieved from the 
external service, they would be making a series of assertions against a 
series of different objects.  And, unlike pure unit-test doctrine would 
like to believe, external services have state.  And per-interaction cost 
(in our case, real cost; some queries result in us paying a few 
mili-cents to a third-party data provider :-))

I guess I'll just mark the whole method @expectedFailure and let it go 
at that.