[Python-Dev] PEP 399: Pure Python/C Accelerator Module Compatibiilty Requirements

[exarkun at twistedmatrix.com]

On 16 Apr, 11:03 pm, steve at pearwood.info wrote:
>Brett Cannon wrote:
>>In the grand python-dev tradition of "silence means acceptance", I 
>>consider
>>this PEP finalized and implicitly accepted.
>
>How long does that silence have to last?
>
>I didn't notice a definition of what counts as "100% branch coverage". 
>Apologies if I merely failed to notice it, but I think it should be 
>explicitly defined.
>
>Presumably it means that any time you have an explicit branch 
>(if...elif...else, try...except...else, for...else, etc.) you need a 
>test that goes down each branch. But it isn't clear to me whether it's 
>sufficient to test each branch in isolation, or whether you need to 
>test all combinations.
>
>That is, if you have five branches, A or B, C or D, E or F, G or H, I 
>or J, within a single code unit (function? something else?), is it 
>sufficient to have at least one test that goes down each of A...J, or 
>do you need to explicitly test each of:
>
>A-C-E-G-I
>A-C-E-G-J
>A-C-E-H-I
>A-C-E-H-J
>A-C-F-G-I
>...
>B-D-F-H-J
>
>(10 tests versus 32 tests).
>
>If the latter, this could become impractical *very* fast. But if not, I 
>don't see how we can claim 100% coverage when there are code paths 
>which are never tested.

The mostly commonly used definition of branch coverage is that each 
outcome of each individual branch is executed, not that all possible 
combinations of all branches in a unit are executed.  I haven't heard 
anyone in this thread propose the latter, only the former.

"100% coverage" by itself is certainly ambiguous.
>
>At the very least, I think you need to explicitly define what you mean 
>by "100% branch coverage". Possibly this will assist in the 
>disagreement between you and Antoine re "100% versus "comprehensive" 
>coverage.

I suspect that everyone who has said "branch coverage" in this thread 
has intended the definition given above (and encourage anyone who meant 
something else to clarify their position).

Jean-Paul