[Python-ideas] ABC: what about the method arguments ?

[Andrew Bennetts]

Terry Reedy wrote:
[...]
> Since unittests have been brought up, I have a idea and question.
> Can this work? Split the current test suite for a concrete class
> that implements one of the ABCs into concrete-specific and
> ABC-general portions, with the abstract part parameterized by
> concrete class.

FWIW, bzr's test suite has this facility, and bzr plugins that implement
various bzr interfaces will have tests for those interfaces
automatically applied.  (Being a Python 2.4+ project, bzr doesn't
actually use the ABCs feature, but we certainly use the concept of
"interface with many implemenations".)

E.g. if you define a new Transport (in bzr terms, a thing like FTP,
HTTP, etc) you probably want to make sure it complies with bzrlib's
expectations for Transports.  So you can include a get_test_permutations
function in your module that returns a list of (transport_class,
server_class) pairs.  [Unsurprisingly you need a test server to run
against, although for transports like LocalTransport (local filesystem
access) they can be very simple.]

It works very well, and is very useful both for bzrlib itself and
plugins.  We have “per-implementation” tests for: branch, bzrdir,
repository, interrepository, merger, transport, tree, workingtree,
uifactory, and more.  Look for bzrlib/tests/per_*.

It's not necessarily easy to write all those tests.  The more complex an
interface, the more likely it is you'll have many tests for that
interface that don't really apply to all implementations — for instance
some Transports are read-only, or don't support list_dir, etc.  So tests
that involve those need to specifically check for that capability and
raise NotApplicable, and finding the exact right way to do that can be
tricky.  It's often easier to say “if isinstance(thing,
ParticularImplementation): ...”, but that quickly erodes the
applicability of those tests for new implementations.

Also tricky is when the setup or even assertions for some tests needs to
vary considerably by implementation: how complicated is your
parameterisation interface going to have to be?

bzr has found it worthwhile, so I do encourage trying it.  I'd use
Robert Collins' http://launchpad.net/testscenarios library if I were
providing this infrastructure in a suite that doesn't already have this
approach; it's basically a distillation of the infrastructure developed
in bzrlib.tests.

-Andrew.