[Python-checkins] peps: Update PEP 399 to include comments from python-dev.
brett.cannon
python-checkins at python.org
Tue Apr 12 23:59:53 CEST 2011
http://hg.python.org/peps/rev/24d68b6329e9
changeset: 3865:24d68b6329e9
user: Brett Cannon <brett at python.org>
date: Tue Apr 12 14:59:45 2011 -0700
summary:
Update PEP 399 to include comments from python-dev.
files:
pep-0399.txt | 94 +++++++++++++++++++++++++--------------
1 files changed, 59 insertions(+), 35 deletions(-)
diff --git a/pep-0399.txt b/pep-0399.txt
--- a/pep-0399.txt
+++ b/pep-0399.txt
@@ -8,19 +8,19 @@
Content-Type: text/x-rst
Created: 04-Apr-2011
Python-Version: 3.3
-Post-History:
+Post-History: 04-Apr-2011, 12-Apr-2011
Abstract
========
The Python standard library under CPython contains various instances
of modules implemented in both pure Python and C (either entirely or
-partially). This PEP requires that in these instances that both the
-Python and C code *must* be semantically identical (except in cases
-where implementation details of a VM prevents it entirely). It is also
-required that new C-based modules lacking a pure Python equivalent
-implementation get special permissions to be added to the standard
-library.
+partially). This PEP requires that in these instances that the
+C code *must* pass the test suite used for the pure Python code
+so as to act as much as a drop-in replacement as possible
+(C- and VM-specific tests are exempt). It is also required that new
+C-based modules lacking a pure Python equivalent implementation get
+special permissions to be added to the standard library.
Rationale
@@ -48,18 +48,22 @@
mandating that all new modules added to Python's standard library
*must* have a pure Python implementation _unless_ special dispensation
is given. This makes sure that a module in the stdlib is available to
-all VMs and not just to CPython.
+all VMs and not just to CPython (pre-existing modules that do not meet
+this requirement are exempt, although there is nothing preventing
+someone from adding in a pure Python implementation retroactively).
Re-implementing parts (or all) of a module in C (in the case
of CPython) is still allowed for performance reasons, but any such
-accelerated code must semantically match the pure Python equivalent to
-prevent divergence. To accomplish this, the pure Python and C code must
-be thoroughly tested with the *same* test suite to verify compliance.
+accelerated code must pass the same test suite (sans VM- or C-specific
+tests) to verify semantics and prevent divergence. To accomplish this,
+the test suite for the module must have 100% branch coverage of the
+pure Python implementation before the acceleration code may be added.
This is to prevent users from accidentally relying
on semantics that are specific to the C code and are not reflected in
-the pure Python implementation that other VMs rely upon, e.g., in
-CPython 3.2.0, ``heapq.heappop()`` raises different exceptions
-depending on whether the accelerated C code is used or not::
+the pure Python implementation that other VMs rely upon. For example,
+in CPython 3.2.0, ``heapq.heappop()`` does an explicit type
+check in its accelerated C code while the Python code uses duck
+typing::
from test.support import import_fresh_module
@@ -77,13 +81,13 @@
try:
c_heapq.heappop(Spam())
except TypeError:
- # "heap argument must be a list"
+ # Explicit type check failure: "heap argument must be a list"
pass
try:
py_heapq.heappop(Spam())
except AttributeError:
- # "'Foo' object has no attribute 'pop'"
+ # Duck typing failure: "'Foo' object has no attribute 'pop'"
pass
This kind of divergence is a problem for users as they unwittingly
@@ -99,11 +103,11 @@
Starting in Python 3.3, any modules added to the standard library must
have a pure Python implementation. This rule can only be ignored if
the Python development team grants a special exemption for the module.
-Typically the exemption would be granted only when a module wraps a
+Typically the exemption will be granted only when a module wraps a
specific C-based library (e.g., sqlite3_). In granting an exemption it
-will be recognized that the module will most likely be considered
-exclusive to CPython and not part of Python's standard library that
-other VMs are expected to support. Usage of ``ctypes`` to provide an
+will be recognized that the module will be considered exclusive to
+CPython and not part of Python's standard library that other VMs are
+expected to support. Usage of ``ctypes`` to provide an
API for a C library will continue to be frowned upon as ``ctypes``
lacks compiler guarantees that C code typically relies upon to prevent
certain errors from occurring (e.g., API changes).
@@ -126,18 +130,34 @@
implementation of the ``csv`` module and maintaining it) then such
code will be accepted.
-Any accelerated code must be semantically identical to the pure Python
-implementation. The only time any semantics are allowed to be
-different are when technical details of the VM providing the
-accelerated code prevent matching semantics from being possible, e.g.,
-a class being a ``type`` when implemented in C. The semantics
-equivalence requirement also dictates that no public API be provided
-in accelerated code that does not exist in the pure Python code.
-Without this requirement people could accidentally come to rely on a
-detail in the accelerated code which is not made available to other VMs
-that use the pure Python implementation. To help verify that the
-contract of semantic equivalence is being met, a module must be tested
-both with and without its accelerated code as thoroughly as possible.
+This requirement does not apply to modules already existing as only C
+code in the standard library. It is acceptable to retroactively add a
+pure Python implementation of a module implemented entirely in C, but
+in those instances the C version is considered the reference
+implementation in terms of expected semantics.
+
+Any new accelerated code must act as a drop-in replacement as close
+to the pure Python implementation as reasonable. Technical details of
+the VM providing the accelerated code are allowed to differ as
+necessary, e.g., a class being a ``type`` when implemented in C. To
+verify that the Python and equivalent C code operate as similarly as
+possible, both code bases must be tested using the same tests which
+apply to the pure Python code (tests specific to the C code or any VM
+do not follow under this requirement). To make sure that the test
+suite is thorough enough to cover all relevant semantics, the tests
+must have 100% branch coverage for the Python code being replaced by
+C code. This will make sure that the new acceleration code will
+operate as much like a drop-in replacement for the Python code is as
+possible.
+
+Acting as a drop-in replacement also dictates that no public API be
+provided in accelerated code that does not exist in the pure Python
+code. Without this requirement people could accidentally come to rely
+on a detail in the accelerated code which is not made available to
+other VMs that use the pure Python implementation. To help verify
+that the contract of semantic equivalence is being met, a module must
+be tested both with and without its accelerated code as thoroughly as
+possible.
As an example, to write tests which exercise both the pure Python and
C accelerated versions of a module, a basic idiom can be followed::
@@ -189,9 +209,13 @@
if __name__ == '__main__':
test_main()
-Thoroughness of the test can be verified using coverage measurements
-with branching coverage on the pure Python code to verify that all
-possible scenarios are tested using (or not using) accelerator code.
+
+If this test were to provide 100% branch coverage for
+``heapq.heappop()`` in the pure Python implementation then the
+accelerated C code would be allowed to be added to CPython's standard
+library. If it did not, then the test suite would need to be updated
+until 100% branch coverage was provided before the accelerated C code
+could be added.
Copyright
--
Repository URL: http://hg.python.org/peps
More information about the Python-checkins
mailing list