Dear Python developers,
I would like to request a review of PEP 575, which is about changing the classes used for built-in functions and Python functions and methods. The text of the PEP can be found at
https://www.python.org/dev/peps/pep-0575/
No substantial changes to the contents of the PEP were made compared to the first posting. However, many details have been changed, clarified or added, based on comments from the initial discussion thread and the work on an implementation.
My implementation is at
https://github.com/jdemeyer/cpython/tree/pep575
This is certainly not meant to be ready to merge upstream; in particular, the Python test suite does not fully pass. Nevertheless, it should be good enough to review the PEP. If the PEP would be accepted, I plan to continue working on the implementation, including adding tests and documentation.
Jeroen.
On 13 April 2018 at 02:12, Jeroen Demeyer J.Demeyer@ugent.be wrote:
Dear Python developers,
I would like to request a review of PEP 575, which is about changing the classes used for built-in functions and Python functions and methods. The text of the PEP can be found at
https://www.python.org/dev/peps/pep-0575/
No substantial changes to the contents of the PEP were made compared to the first posting. However, many details have been changed, clarified or added, based on comments from the initial discussion thread and the work on an implementation.
I'm personally +1 on this version of the PEP (thank you for putting it
together!), but have one request for clarification related to
__doc__: at first glance, it isn't obvious why there are two
different ways of storing the docstring (either m_ml->ml_doc or a
dedicated func_code field).
Those technically could be consolidated by always using m_ml->ml_doc, and changing the definition of base_function such that m_ml always pointed to a PyMethodDef instance.
However, that's less than ideal when the doc string is already a Python object, since you need to convert between "const char " and "PyObject " when accessing the field from Python (and vice versa when setting it).
There's also a section in the rationale which refers to METH_USRx flags, which I'm guessing from context are an idea you were considering proposing, but eventually dropped from the rest of the PEP. These remaining references to the concept just need to be cleaned up.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Apr 12, 2018, at 9:12 AM, Jeroen Demeyer J.Demeyer@UGent.be wrote:
I would like to request a review of PEP 575, which is about changing the classes used for built-in functions and Python functions and methods. The text of the PEP can be found at
Thanks for doing this work. The PEP is well written and I'm +1 on the general idea of what it's trying to do (I'm still taking in all the details).
It would be nice to have a section that specifically discusses the implications with respect to other existing function-like tooling: classmethod, staticmethod, partial, itemgetter, attrgetter, methodgetter, etc.
Also, please mention the backward compatibility issue that will arise for code that currently relies on types.MethodType, types.BuiltinFunctionType, types.BuiltinMethodType, etc. For example, I would need to update the code in random._randbelow(). That code uses the existing builtin-vs-pure-python type distinctions to determine whether either the random() or getrandbits() methods have been overridden. This is likely an easy change for me to make, but there may be code like it the wild, code that would be broken if the distinction is lost.
Raymond
On 12/04/18 17:12, Jeroen Demeyer wrote:
Dear Python developers,
I would like to request a review of PEP 575, which is about changing the classes used for built-in functions and Python functions and methods. The text of the PEP can be found at
The motivation of PEP 575 is to allow introspection of built-in functions and to allow functions implemented in Python to be re-implemented in C.
These are excellent goals.
The PEP then elaborates a complex class hierarchy, and various extensions to the C API. This adds a considerable maintainance burden and restricts future changes and optimisations to CPython.
While a unified interface makes sense, a unified class hierarchy and implementation, IMO, do not.
The hierarchy also seems to force classes that are dissimilar to share a common base-class. Bound-methods may be callables, but they are not functions, they are a pair of a function and a "self" object.
As the PEP points out, Cython functions are able to mimic Python functions, why not do the same for CPython builtin-functions?
As an aside, rather than unifying the classes of all non-class callables, CPython's builtin-function class could be split in two. Currently it is both a bound-method and a function. The name 'builtin_function_or_method' is a give away :)
Consider the most common "function" and "method" classes:
class C: ... def f(self): pass
# "functions"
type(C.f)
<class 'function'> type(len)
<class 'builtin_function_or_method'> type(list.append)
<class 'method_descriptor'> type(int.__add__)
<class 'wrapper_descriptor'>
# "bound-methods"
type(C().f)
<class 'method'> type([].append)
<class 'builtin_function_or_method'> type(1 .__add__)
<class 'method-wrapper'>
IMO, there are so many versions of "function" and "bound-method", that a unified class hierarchy and the resulting restriction to the implementation will make implementing a unified interface harder, not easier.
For "functions", all that is needed is to specify an interface, say a single property "__signature__". Then all that a class that wants to be a "function" need do is have a "__signature__" property and be callable.
For "bound-methods", we should reuse the interface of 'method'; two properties, "__func__" and "__self__".
Cheers, Mark.
On 30 April 2018 at 23:38, Mark Shannon mark@hotpy.org wrote:
>
On 12/04/18 17:12, Jeroen Demeyer wrote:
Dear Python developers,
I would like to request a review of PEP 575, which is about changing the classes used for built-in functions and Python functions and methods. The text of the PEP can be found at
The motivation of PEP 575 is to allow introspection of built-in functions and to allow functions implemented in Python to be re-implemented in C.
These are excellent goals.
That summary misses the 3rd goal, which is the one that answers your other questions: to allow 3rd party extension modules access to the hot paths in CPython that are currently restricted to true built-in and Python native functions (without making those hot paths measurably slower).
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-
Jeroen Demeyer -
Mark Shannon -
Nick Coghlan -
Raymond Hettinger