"embedsignature" and "autotestdict" features in Py3.4
Hi, recent changes in Py3.4 affect the functionality of the two directives "embedsignature" and "autotestdict". The so-called "argument clinic" extracts any embedded signatures from docstrings and moves them into a new property "__text_signature__". This is done at runtime, i.e. when either "__doc__" or "__text_signature__" are requested. http://hg.python.org/cpython/file/6a3e09cd96f3/Objects/methodobject.c#l182 I personally consider this a feature (and it works nicely with the signatures that Cython embeds), but you may or may not agree. It broke some of our own doctests, at least, because the "__doc__" value that we tested for was no longer the same. Regarding the "autotestdict", CPython also got smarter and now finds doctests embedded in C implemented functions all by itself. This is clearly an improvement compared to the previous behaviour. However, this also means that it now executes both the doctest of the function itself and its copy in the generated "__test__" dict (stored in the module under that name), meaning that it executes all function doctests twice. This also broke some of Cython's own doctests for us, which modified global state and thus failed the second time. I'm yet unsure if we should try to do something about this. Currently, the explicit doctest dict is generated by default, so all code that uses doctests in function docstrings suffers from this. We might be able to make Cython a bit smarter so that these docstrings are left out of the test dict when (C-)compiling in CPython 3.4. Deciding which ones are still needed might be a bit tricky, though... Stefan
On Fri, Nov 29, 2013 at 8:03 AM, Stefan Behnel <stefan_ml@behnel.de> wrote:
Hi,
recent changes in Py3.4 affect the functionality of the two directives "embedsignature" and "autotestdict".
The so-called "argument clinic" extracts any embedded signatures from docstrings and moves them into a new property "__text_signature__". This is done at runtime, i.e. when either "__doc__" or "__text_signature__" are requested.
http://hg.python.org/cpython/file/6a3e09cd96f3/Objects/methodobject.c#l182
I personally consider this a feature (and it works nicely with the signatures that Cython embeds), but you may or may not agree. It broke some of our own doctests, at least, because the "__doc__" value that we tested for was no longer the same.
I'm fine with this new behavior.
Regarding the "autotestdict", CPython also got smarter and now finds doctests embedded in C implemented functions all by itself. This is clearly an improvement compared to the previous behaviour. However, this also means that it now executes both the doctest of the function itself and its copy in the generated "__test__" dict (stored in the module under that name), meaning that it executes all function doctests twice. This also broke some of Cython's own doctests for us, which modified global state and thus failed the second time.
I'm yet unsure if we should try to do something about this. Currently, the explicit doctest dict is generated by default, so all code that uses doctests in function docstrings suffers from this. We might be able to make Cython a bit smarter so that these docstrings are left out of the test dict when (C-)compiling in CPython 3.4. Deciding which ones are still needed might be a bit tricky, though...
I think it makes sense to exclude these from the __test__ dict at C compile time--is it the cpdef ones that it can pick up now? - Robert
Stefan Behnel, 29.11.2013 17:03:
recent changes in Py3.4 affect the functionality of the two directives "embedsignature" and "autotestdict".
The so-called "argument clinic" extracts any embedded signatures from docstrings and moves them into a new property "__text_signature__". This is done at runtime, i.e. when either "__doc__" or "__text_signature__" are requested.
http://hg.python.org/cpython/file/6a3e09cd96f3/Objects/methodobject.c#l182
I personally consider this a feature (and it works nicely with the signatures that Cython embeds), but you may or may not agree. It broke some of our own doctests, at least, because the "__doc__" value that we tested for was no longer the same.
This feature has essentially been reverted and can thus be expected to not go into the final CPython 3.4 release. The reason was that it interfered with existing docstrings (as noted above), so they invented a new scheme "sig=..." for generated signatures to only extract those created by the argument clinic itself. For Cython, this means that CPython no longer touches the docstrings, so the embedded signature string continues to appear where users can see it and tools like Sphinx or epydoc parse it. Eventually, Cython should generate a real Python level representation of the signature, but that's not easy to get "right" due to the C-to-Python mapping, and because the "Signature" class is not available in older Python versions and generally not meant to be used manually. See PEP 362 for some details. http://www.python.org/dev/peps/pep-0362/ Stefan
participants (2)
-
Robert Bradshaw -
Stefan Behnel