[Tutor] Do not understand code snippet from "26.8. test — Regression tests package for Python"

[Martin A. Brown]

Greetings boB,

>2)  The big question:  What is the program flow for this program?  I
>am not seeing the order of execution here.  How is the unittest module
>handling the execution of this? 

This is a very good question and one that was (at one time) 
inobvious to me, as well.

When you write a program, you usually have a clear notion of where 
the program will begin and then can follow its execution path (from 
"if __name__ == '__main__'") through your functions, class 
instantiations and method calls.  This is the execution structure of 
your program.  (I'd imagine you have used print and/or logging to 
debug program flow, as well....)

Testing flow is different.

The sequence of test execution has nothing to do with your program 
structure.  This is utterly intentional.

[ background / digression ] By breaking down a complex program into 
smaller testable pieces, you can have more assurance that your 
program is doing exactly what you intend.  Since you are breaking 
the program into smaller pieces, those pieces can (and should) be 
runnable and tested without requiring any of the other pieces.

Usually (almost always) tests are run in isolation.  This allows you 
to control exactly the arguments, environment and conditions of each 
test.

You may know most of the above already, but I repeat it because 
these facts help explain why testing tools work as they do...
[ end digression ]

Now, to the unittest module.

(Please note, I'm not an expert on unittest internals, so I may get 
a detail or two wrong.  Nonetheless, I hope that my answer will help 
you orient yourself around what's happening.)

When you run a tool to collect your tests and execute them, the path 
through your pieces of code under test has no connection whatsoever 
to process flow through your program.

The basic flow looks like this:

  * find the test cases / test suites you have written
  * run each of the tests independently, i.e. isolated conditions
  * report on the success/failure of each test case, test suite and 
    the whole batch

See below for more detail of the mechanics inside the unittest 
module.  What happens when you execute your testing suite?  Let's 
say you run:

  $ python -m unittest

(Unless there is customization of TestLoader TestSuite and/or 
TestRunner) the following sequence occurs:

  1. the Python interpreter starts up

  2. Python loads the unittest module and passes control to unittest

  3. unittest.main creates an instance of unittest.TestLoader [0]

  4. unittest.TestLoader scans the filesystem, collecting a list of 
     tests to run from:

       - any test suites subclassed from unittest.TestSuite [1]
       - any test cases subclassed unittest.TestCase [2]

  5. unittest.TestLoader imports anything it found and returns the 
     list of tests to the main testing program loop

  6. the main loop passes the tests to the unittest.TextTestRunner [3],
     which executes each test and (probably) produces some output 
     telling you either that your hard work has paid off or that 
     something is still wrong

Your next question is why do the mixins work?  And how do they work?

I'll make a few observations:

  - [on unittest] the unit testing tools use classes because it's a 
    natural way to accommodate the goal of reproducibly setting up 
    arguments and/or an environment for each test (note that each 
    TestCase can have its own setUp() and tearDown() methods; this 
    allows isolation)

  - [on unittest] each test collected by the TestLoader can be any 
    Python class (as long as it is also derived from 
    unittest.TestCase)

  - [on your classes] your classes use a multiple inheritance 
    model, deriving from TestFuncAcceptsSequencesMixin; when 
    instantiated, they'll have all of the expected TestCase methods
    and the method called 'test_func'

In more detail, you have created three different classes, each of 
which is derived from unittest.TestCase (I'm showing just the 
signatures):

  class AcceptLists(TestFuncAcceptsSequencesMixin, unittest.TestCase):
  class AcceptStrings(TestFuncAcceptsSequencesMixin, unittest.TestCase): 
  class AcceptTuples(TestFuncAcceptsSequencesMixin, unittest.TestCase):

Here's what's happening:

  - TestLoader finds the files that contains the above classes (probably
    named 'test_something.py')

  - Testloader imports the file 'test_something.py'; this defines your
    classes: AcceptLists, AcceptStrings and AcceptTuples (or will 
    produce a traceback if the code does not import; try breaking 
    your code and you should see that the import of your test code 
    fails during the TestLoader phase)

  - TestLoader appends the now-defined classes: AcceptLists,
    AcceptStrings and AcceptTuples to the list of tests

  - control passes back to main and then to TestRunner

  - for each unittest.TestCase in the list of tests, TestRunner will:

    - create an instance T from the defined class

    - for each method name starting with 'test_' (you have only 
      'test_func') TestRunner will:

      - execute the T.setUp() method if it exists

      - TestRunner will execute the method 'test_func'

      - collect the success / failure and any outputs

      - report on the success / failure

  - produce some final summary output and set the exit code 
    accordingly (os.EX_OK means success, anything else is failure)

>The ending comment in the docs' example cited reads:
>
>"When using this pattern, remember that all classes that inherit from
>unittest.TestCase are run as tests. The Mixin class in the example
>above does not have any data and so can’t be run by itself, thus it
>does not inherit from unittest.TestCase."
>
>This suggests to me that unittest "uses" the bottom three classes, but
>even though each of the three inherits from the class
>TestFuncAcceptsSequenceMixin, those classes don't have any methods
>that they call on that class, so how does its code get run?  I suspect
>that the mixin's concepts is where I am stumbling.  I have yet to find
>a reference that is making things clear to me, though I will continue
>searching and reading.

I hope my long-winded explanation amkes that a bit clearer.

Good luck,

-Martin

 [0] https://docs.python.org/3/library/unittest.html#unittest.TestLoader
 [1] https://docs.python.org/3/library/unittest.html#unittest.TestSuite
 [2] https://docs.python.org/3/library/unittest.html#unittest.TestCase
 [3] https://docs.python.org/3/library/unittest.html#unittest.TextTestRunner

http://www.drdobbs.com/testing/unit-testing-with-python/240165163

-- 
Martin A. Brown
http://linux-ip.net/