[Python-Dev] peps: Update PEP 399 to include comments from python-dev.

[Stefan Behnel]

Antoine Pitrou, 13.04.2011 02:07:
> On Tue, 12 Apr 2011 19:50:34 -0400
> Tres Seaver wrote:
>> Trying to accelerate existing code which doesn't have the coverage is
>> insane:  how can you know that the accelerator doesn't subtly change the
>> semantics without tests?
>
> Well, why do you think tests guarantee that the semantics are the same?
> Tests are not a magic bullet. "Covering" a code path doesn't ensure
> that every possible behaviour is accounted for.

This is particularly true when it comes to input types. There are different 
protocols out there that people use in their code, iteration vs. item 
access being only the most famous ones, inheritance vs. wrapping being 
another issue. Duck-typed Python code may work with a lot more input types 
than C code, even with 100% test coverage. This has been partly mentioned 
in the PEP, but not as clearly in the context of test coverage. Tests can 
only catch issues with the input they use themselves, not with all input 
the code will encounter in the wild.

However, I think we are really discussing a theoretical issue here. All the 
PEP is trying to achieve is to raise the bar for C code in the stdlib, for 
exactly the reason that it can easily introduce subtle semantic differences 
in comparison to generic Python code.

I think it would help to point out in the PEP that code that fails to touch 
the theoretical 100% test coverage bar is not automatically excluded from 
integration, but needs solid reasoning, review and testing in the wild in 
order to be considered an equivalent alternative implementation. But then 
again, this should actually be required anyway, even for code with an 
exceedingly high test coverage.

Stefan