Weird io.OpenWrapper hack to use a function as method
Hi, The io module provides an open() function. It also provides an OpenWrapper which only exists to be able to store open as a method (class or instance method). In the _pyio module, pure Python implementation of the io module, OpenWrapper is implemented as: class OpenWrapper: """Wrapper for builtins.open Trick so that open won't become a bound method when stored as a class variable (as dbm.dumb does). See initstdio() in Python/pylifecycle.c. """ def __new__(cls, *args, **kwargs): return open(*args, **kwargs) I would like to remove this class which is causing troubles in the PEP 597 implementation, but I don't know how. Simplified problem: --- def func(): print("my func") class MyClass: method = func func() # A MyClass.method() # B obj = MyClass() obj.method() # C --- With this syntax, A and B work, but C fails with TypeError: func() takes 0 positional arguments but 1 was given. If I decorate func() with @staticmethod, B and C work, but A fails with TypeError: 'staticmethod' object is not callable. Is OpenWrapper the only way to have a callable object which works in the 3 variants A, B and C? A, B and C work if MyClass is modified to use staticmethod: class MyClass: method = staticmethod(func) Victor -- Night gathers, and now my watch begins. It shall not end until my death.
tl; dr *Maybe* staticmethod could be modified to become callable? io.open is a built-in function (type "builtin_function_or_method"). It behaves differently than _pyio.open which is a Python function (type "function"). The difference is in the LOAD_METHOD bytecode which uses __get__() descriptor if available. Built-in function has no __get__() method and so the function is used directly as a method. Python function has a __get__() descriptor which returns the function unchanged when a class method is requested, and create a bound method if an instance method is requested: --- def func(): ... FunctionType = type(func) class MyClass: method = func class_method = MyClass.method assert class_method is func assert class_method is FunctionType.__get__(func, None, MyClass) obj = MyClass() instance_method = obj.method assert instance_method.__self__ is obj assert instance_method.__func__ is func # each __get__() call creates a new bound method assert instance_method == FunctionType.__get__(func, obj, type(obj)) --- @staticmethod decorator avoids the creation of the bound method: --- def func(): ... class MyClass: method = staticmethod(func) # method = MyClass.method attr = MyClass.__dict__['method'] method = type(attr).__get__(attr, None, MyClass) assert method is func --- The drawback is that the object created by staticmethod cannot be called :-( The following code raises a TypeError: --- wrapped = staticmethod(func) wrapped() --- *Maybe* staticmethod could be modified to become callable? Victor On Wed, Mar 31, 2021 at 2:34 PM Victor Stinner <vstinner@python.org> wrote:
Hi,
The io module provides an open() function. It also provides an OpenWrapper which only exists to be able to store open as a method (class or instance method). In the _pyio module, pure Python implementation of the io module, OpenWrapper is implemented as:
class OpenWrapper: """Wrapper for builtins.open
Trick so that open won't become a bound method when stored as a class variable (as dbm.dumb does).
See initstdio() in Python/pylifecycle.c. """ def __new__(cls, *args, **kwargs): return open(*args, **kwargs)
I would like to remove this class which is causing troubles in the PEP 597 implementation, but I don't know how. Simplified problem: --- def func(): print("my func")
class MyClass: method = func
func() # A MyClass.method() # B obj = MyClass() obj.method() # C ---
With this syntax, A and B work, but C fails with TypeError: func() takes 0 positional arguments but 1 was given.
If I decorate func() with @staticmethod, B and C work, but A fails with TypeError: 'staticmethod' object is not callable.
Is OpenWrapper the only way to have a callable object which works in the 3 variants A, B and C?
A, B and C work if MyClass is modified to use staticmethod:
class MyClass: method = staticmethod(func)
Victor -- Night gathers, and now my watch begins. It shall not end until my death.
-- Night gathers, and now my watch begins. It shall not end until my death.
A friend, Antoine Rozo, wrote a variant of staticmethod which is callable. With this decorator, it works in A, B and C cases: --- class simplefunction: def __init__(self, func): self.func = func def __get__(self, owner, instance): return self.func def __call__(self, *args, **kwargs): return self.func(*args, **kwargs) @simplefunction def func(): print("my func") class MyClass: method = func func() # A MyClass.method() # B MyClass().method() # C --- It works without the __get__() method, but in this case, we go through the __call__() indirection for A, B and C cases, rather than only going through __call__() indirection in A case. Victor
On 3/31/21 6:49 AM, Victor Stinner wrote:
tl; dr *Maybe* staticmethod could be modified to become callable?
There have been other requests to make staticmethod callable, one of them being https://bugs.python.org/issue20309 +1 for having it done. -- ~Ethan~
I created https://bugs.python.org/issue43682 "Make static methods created by @staticmethod callable". Oh, I didn't know this old bpo-20309 issue closed as "not a bug". But it proposed to modify many other wrappers like @classmethod. I only propose to make static methods created @staticmethod callable. Victor On Wed, Mar 31, 2021 at 4:19 PM Ethan Furman <ethan@stoneleaf.us> wrote:
On 3/31/21 6:49 AM, Victor Stinner wrote:
tl; dr *Maybe* staticmethod could be modified to become callable?
There have been other requests to make staticmethod callable, one of them being
https://bugs.python.org/issue20309
+1 for having it done.
-- ~Ethan~ _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KITFURNW... Code of Conduct: http://python.org/psf/codeofconduct/
-- Night gathers, and now my watch begins. It shall not end until my death.
Honestly it has always bugged me that staticmethod only becomes callable through __get__. So I think this would be fine. But I don't know if there might be unintended consequences.
On 31 Mar 2021, at 13:34, Victor Stinner <vstinner@python.org> wrote:
def func(): print("my func")
This would work for the example given of a func with no args. But cannot check it called with the right number. def func(*args): print("my func") A signature like this would be a hard nut to crack. def func(fixed, *args): print("my func", fixed, args) Barry
Do we need _pyio at all? Does PyPy or any other Python implementation use it? On Wed, Mar 31, 2021 at 9:36 PM Victor Stinner <vstinner@python.org> wrote:
Hi,
The io module provides an open() function. It also provides an OpenWrapper which only exists to be able to store open as a method (class or instance method). In the _pyio module, pure Python implementation of the io module, OpenWrapper is implemented as:
class OpenWrapper: """Wrapper for builtins.open
Trick so that open won't become a bound method when stored as a class variable (as dbm.dumb does).
See initstdio() in Python/pylifecycle.c. """ def __new__(cls, *args, **kwargs): return open(*args, **kwargs)
I would like to remove this class which is causing troubles in the PEP 597 implementation, but I don't know how. Simplified problem: --- def func(): print("my func")
class MyClass: method = func
func() # A MyClass.method() # B obj = MyClass() obj.method() # C ---
With this syntax, A and B work, but C fails with TypeError: func() takes 0 positional arguments but 1 was given.
If I decorate func() with @staticmethod, B and C work, but A fails with TypeError: 'staticmethod' object is not callable.
Is OpenWrapper the only way to have a callable object which works in the 3 variants A, B and C?
A, B and C work if MyClass is modified to use staticmethod:
class MyClass: method = staticmethod(func)
Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QZ7SFW3I... Code of Conduct: http://python.org/psf/codeofconduct/
-- Inada Naoki <songofacandy@gmail.com>
On Wed., Mar. 31, 2021, 18:56 Inada Naoki, <songofacandy@gmail.com> wrote:
Do we need _pyio at all? Does PyPy or any other Python implementation use it?
https://www.python.org/dev/peps/pep-0399/ would suggest rolling back Python support is something to avoid.
On Wed, Mar 31, 2021 at 9:36 PM Victor Stinner <vstinner@python.org> wrote:
Hi,
The io module provides an open() function. It also provides an OpenWrapper which only exists to be able to store open as a method (class or instance method). In the _pyio module, pure Python implementation of the io module, OpenWrapper is implemented as:
class OpenWrapper: """Wrapper for builtins.open
Trick so that open won't become a bound method when stored as a class variable (as dbm.dumb does).
See initstdio() in Python/pylifecycle.c. """ def __new__(cls, *args, **kwargs): return open(*args, **kwargs)
I would like to remove this class which is causing troubles in the PEP 597 implementation, but I don't know how. Simplified problem: --- def func(): print("my func")
class MyClass: method = func
func() # A MyClass.method() # B obj = MyClass() obj.method() # C ---
With this syntax, A and B work, but C fails with TypeError: func() takes 0 positional arguments but 1 was given.
If I decorate func() with @staticmethod, B and C work, but A fails with TypeError: 'staticmethod' object is not callable.
Is OpenWrapper the only way to have a callable object which works in the 3 variants A, B and C?
A, B and C work if MyClass is modified to use staticmethod:
class MyClass: method = staticmethod(func)
Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/QZ7SFW3I...
Code of Conduct: http://python.org/psf/codeofconduct/
-- Inada Naoki <songofacandy@gmail.com> _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BOSZENKZ... Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Apr 1, 2021 at 11:52 AM Brett Cannon <brett@python.org> wrote:
On Wed., Mar. 31, 2021, 18:56 Inada Naoki, <songofacandy@gmail.com> wrote:
Do we need _pyio at all? Does PyPy or any other Python implementation use it?
https://www.python.org/dev/peps/pep-0399/ would suggest rolling back Python support is something to avoid.
Thank you. If we obey PEP 399, we need an easy way to keep consistency between Python function and C function. -- Inada Naoki <songofacandy@gmail.com>
participants (6)
-
Barry Scott
-
Brett Cannon
-
Ethan Furman
-
Guido van Rossum
-
Inada Naoki
-
Victor Stinner