Add __name__ to functools.partial object
Hey folks, I propose adding __name__ to functools.partial.
get_name = functools.partial(input, "name: ") get_name() name: hi 'hi' get_name.__name__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'functools.partial' object has no attribute '__name__' get_name.func <built-in function input> get_name.func.__name__ 'input'
We could set __name__ based off of partial.func.__name__ or we could try to set it to something like 'partial calling func.__name__' If the callable doesn't have a name, we could fall back to a None __name__ or set it to something generic. Even lambdas have __name__ set:
l = lambda: input('name: ') l.__name__ '<lambda>'
This proposal makes __name__ on partial objects more useful than the current behavior of __name__ on lambda objects as well. We could port over similar functionality to lambda if we'd like. - Charlie Scott Machalow
+0 Questions: 1. What's the use case for partial having __name__? 2. Does this imply it should have __qualname__ as well? 3. What name would be given to (an inherently anonymous) lambda? Notes: 1. I would prefer __name__ to be more qualifying like its repr (e.g. partial(foo, "x") → "<partial foo>") On Mon, 2022-08-29 at 21:31 -0700, Charles Machalow wrote:
Hey folks,
I propose adding __name__ to functools.partial.
get_name = functools.partial(input, "name: ") get_name() name: hi 'hi' get_name.__name__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'functools.partial' object has no attribute '__name__' get_name.func <built-in function input> get_name.func.__name__ 'input'
We could set __name__ based off of partial.func.__name__ or we could try to set it to something like 'partial calling func.__name__'
If the callable doesn't have a name, we could fall back to a None __name__ or set it to something generic.
Even lambdas have __name__ set:
l = lambda: input('name: ') l.__name__ '<lambda>'
This proposal makes __name__ on partial objects more useful than the current behavior of __name__ on lambda objects as well. We could port over similar functionality to lambda if we'd like.
- Charlie Scott Machalow _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WK3FO3... Code of Conduct: http://python.org/psf/codeofconduct/
1: There are cases where one may need the __name__/__qualname__ of a given callable. If someone uses partial to create a new callable, there is no __name__/__qualname__ given. In my particular case, I'm logging what callback function is passed to a different function... if someone uses partial, there is no __name__/__qualname__ which leads to a current traceback... of course i can work around it but still was an odd case to me. Per the docs on functools.partial: "Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords" ... which made me initially think that in order to behave like the passed in function: it should have __name__ and __qualname__... like the func did. 2: I would say have both __qualname__ and __name__. Both could be based off of __name__/__qualname__ of the passed in func. 3: This would be more difficult since you would have to disassemble the lambda to figure out the called method (or methods)... We can table the lambda discussion for the purpose of this idea. I recall that typically it is preferred to use partial over lambdas, so this could be an additional functionality/benefit of using partial over lambda. Notes: ... __name__ being something like partial(foo, "x") would be fine with me... I just feel as though something should be there. - Charlie Scott Machalow On Mon, Aug 29, 2022 at 9:56 PM Paul Bryan <pbryan@anode.ca> wrote:
+0
Questions:
1. What's the use case for partial having __name__? 2. Does this imply it should have __qualname__ as well? 3. What name would be given to (an inherently anonymous) lambda?
Notes:
1. I would prefer __name__ to be more qualifying like its repr (e.g. partial(foo, "x") → "<partial foo>")
On Mon, 2022-08-29 at 21:31 -0700, Charles Machalow wrote:
Hey folks,
I propose adding __name__ to functools.partial.
get_name = functools.partial(input, "name: ") get_name() name: hi 'hi' get_name.__name__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'functools.partial' object has no attribute '__name__' get_name.func <built-in function input> get_name.func.__name__ 'input'
We could set __name__ based off of partial.func.__name__ or we could try to set it to something like 'partial calling func.__name__'
If the callable doesn't have a name, we could fall back to a None __name__ or set it to something generic.
Even lambdas have __name__ set:
l = lambda: input('name: ') l.__name__ '<lambda>'
This proposal makes __name__ on partial objects more useful than the current behavior of __name__ on lambda objects as well. We could port over similar functionality to lambda if we'd like.
- Charlie Scott Machalow _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WK3FO3... Code of Conduct: http://python.org/psf/codeofconduct/
partial Objects¶ partial objects are callable objects created by partial(). They have
Is there a non-performance regressive way to proxy attr access to func.__name__ of the partial function (or method; Callable)? Would this affect documentation generation tools like e.g. sphinx-spidoc, which IIRC use __name__ and probably now __qualname__ for generating argspecs in RST for HTML and LaTeX? - https://docs.python.org/3/library/inspect.html - functions and methods have __name__ and __qualname__ - see: sphinx.utils.inspect - https://docs.python.org/3/library/functools.html#functools.partial - https://docs.python.org/3/library/functools.html#functools.partialmethod - https://docs.python.org/3/library/functools.html#partial-objects three read-only attributes:
partial.func A callable object or function. Calls to the partial object will be
forwarded to func with new arguments and keywords.
partial.args The leftmost positional arguments that will be prepended to the
positional arguments provided to a partial object call.
partial.keywords The keyword arguments that will be supplied when the partial object is
called.
partial objects are like function objects in that they are callable, weak referencable, and can have attributes. There are some important differences. For instance, the __name__ and __doc__ attributes are not created automatically. Also, partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.
- https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html - https://www.sphinx-doc.org/en/master/_modules/sphinx/ext/autodoc.html : 18 references to __qualname__, https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L49-L66 : ```python def unwrap_all(obj: Any, *, stop: Optional[Callable] = None) -> Any: """ Get an original object from wrapped object (unwrapping partials, wrapped functions, and other decorators). """ while True: if stop and stop(obj): return obj elif ispartial(obj): obj = obj.func elif inspect.isroutine(obj) and hasattr(obj, '__wrapped__'): obj = obj.__wrapped__ # type: ignore elif isclassmethod(obj): obj = obj.__func__ elif isstaticmethod(obj): obj = obj.__func__ else: return obj ``` From https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L173-L1... : ```python def unpartial(obj: Any) -> Any: """Get an original object from partial object. This returns given object itself if not partial. """ while ispartial(obj): obj = obj.func return obj def ispartial(obj: Any) -> bool: """Check if the object is partial.""" return isinstance(obj, (partial, partialmethod)) ``` - https://github.com/sphinx-doc/sphinx/issues/4826#issuecomment-808699254 https://docs.python.org/3/library/stdtypes.html#definition.__name__ https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarc...
Callable types > User defined functions does list both __name__ and __qualname__
### Limitations With nested functions (and classes defined inside functions), the dotted
Is there a non-performance regressive way to proxy attr access to __name__ of the partially curried (?) function? From "PEP 3155 – Qualified name for classes and functions" https://peps.python.org/pep-3155/#limitations : path will not be walkable programmatically as a function’s namespace is not available from the outside. It will still be more helpful to the human reader than the bare __name__.
As the __name__ attribute, the __qualname__ attribute is computed
statically and it will not automatically follow rebinding. From https://wrapt.readthedocs.io/en/latest/wrappers.html#proxy-object-attributes :
Proxy Object Attributes When an attempt is made to access an attribute from the proxy, the same named attribute would in normal circumstances be accessed from the wrapped object. When updating an attributes value, or deleting the attribute, that change will also be reflected in the wrapped object.
From https://docs.python.org/3/library/weakref.html#weakref.proxy :
weakref.proxy(object[, callback])¶ Return a proxy to object which uses a weak reference. This supports use of the proxy in most contexts instead of requiring the explicit dereferencing used with weak reference objects. The returned object will have a type of either ProxyType or CallableProxyType, depending on whether object is callable. Proxy objects are not hashable regardless of the referent; this avoids a number of problems related to their fundamentally mutable nature, and prevent their use as dictionary keys. callback is the same as the parameter of the same name to the ref() function.
On Tue, Aug 30, 2022, 1:14 AM Charles Machalow <csm10495@gmail.com> wrote:
1: There are cases where one may need the __name__/__qualname__ of a given callable. If someone uses partial to create a new callable, there is no __name__/__qualname__ given. In my particular case, I'm logging what callback function is passed to a different function... if someone uses partial, there is no __name__/__qualname__ which leads to a current traceback... of course i can work around it but still was an odd case to me.
Per the docs on functools.partial: "Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords" ... which made me initially think that in order to behave like the passed in function: it should have __name__ and __qualname__... like the func did.
2: I would say have both __qualname__ and __name__. Both could be based off of __name__/__qualname__ of the passed in func.
3: This would be more difficult since you would have to disassemble the lambda to figure out the called method (or methods)... We can table the lambda discussion for the purpose of this idea. I recall that typically it is preferred to use partial over lambdas, so this could be an additional functionality/benefit of using partial over lambda.
Notes: ... __name__ being something like partial(foo, "x") would be fine with me... I just feel as though something should be there.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 9:56 PM Paul Bryan <pbryan@anode.ca> wrote:
+0
Questions:
1. What's the use case for partial having __name__? 2. Does this imply it should have __qualname__ as well? 3. What name would be given to (an inherently anonymous) lambda?
Notes:
1. I would prefer __name__ to be more qualifying like its repr (e.g. partial(foo, "x") → "<partial foo>")
On Mon, 2022-08-29 at 21:31 -0700, Charles Machalow wrote:
Hey folks,
I propose adding __name__ to functools.partial.
get_name = functools.partial(input, "name: ") get_name() name: hi 'hi' get_name.__name__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'functools.partial' object has no attribute '__name__' get_name.func <built-in function input> get_name.func.__name__ 'input'
We could set __name__ based off of partial.func.__name__ or we could try to set it to something like 'partial calling func.__name__'
If the callable doesn't have a name, we could fall back to a None __name__ or set it to something generic.
Even lambdas have __name__ set:
l = lambda: input('name: ') l.__name__ '<lambda>'
This proposal makes __name__ on partial objects more useful than the current behavior of __name__ on lambda objects as well. We could port over similar functionality to lambda if we'd like.
- Charlie Scott Machalow _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WK3FO3... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ONAEBC... Code of Conduct: http://python.org/psf/codeofconduct/
We may be able to do __name__/__qualname__ as a property to make it evaluate when called as opposed to computed once on creation. That way we just work with .func upon call so no need for extra references, etc. As for documentation generation tools, it may be different at first, though I believe the existing ispartial checks would catch partials still. If they want to (in a new version) swap to using __name__/__qualname__ that should be fine, but this likely wouldn't inherently break existing tools. - Charlie Scott Machalow On Mon, Aug 29, 2022 at 11:08 PM Wes Turner <wes.turner@gmail.com> wrote:
Is there a non-performance regressive way to proxy attr access to func.__name__ of the partial function (or method; Callable)?
Would this affect documentation generation tools like e.g. sphinx-spidoc, which IIRC use __name__ and probably now __qualname__ for generating argspecs in RST for HTML and LaTeX?
- https://docs.python.org/3/library/inspect.html - functions and methods have __name__ and __qualname__ - see: sphinx.utils.inspect
- https://docs.python.org/3/library/functools.html#functools.partial - https://docs.python.org/3/library/functools.html#functools.partialmethod - https://docs.python.org/3/library/functools.html#partial-objects
partial Objects¶ partial objects are callable objects created by partial(). They have three read-only attributes:
partial.func A callable object or function. Calls to the partial object will be forwarded to func with new arguments and keywords.
partial.args The leftmost positional arguments that will be prepended to the positional arguments provided to a partial object call.
partial.keywords The keyword arguments that will be supplied when the partial object is called.
partial objects are like function objects in that they are callable, weak referencable, and can have attributes. There are some important differences. For instance, the __name__ and __doc__ attributes are not created automatically. Also, partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.
- https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html - https://www.sphinx-doc.org/en/master/_modules/sphinx/ext/autodoc.html : 18 references to __qualname__,
https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L49-L66 :
```python def unwrap_all(obj: Any, *, stop: Optional[Callable] = None) -> Any: """ Get an original object from wrapped object (unwrapping partials, wrapped functions, and other decorators). """ while True: if stop and stop(obj): return obj elif ispartial(obj): obj = obj.func elif inspect.isroutine(obj) and hasattr(obj, '__wrapped__'): obj = obj.__wrapped__ # type: ignore elif isclassmethod(obj): obj = obj.__func__ elif isstaticmethod(obj): obj = obj.__func__ else: return obj ```
From https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L173-L1... :
```python def unpartial(obj: Any) -> Any: """Get an original object from partial object. This returns given object itself if not partial. """ while ispartial(obj): obj = obj.func
return obj
def ispartial(obj: Any) -> bool: """Check if the object is partial.""" return isinstance(obj, (partial, partialmethod)) ```
- https://github.com/sphinx-doc/sphinx/issues/4826#issuecomment-808699254
https://docs.python.org/3/library/stdtypes.html#definition.__name__
https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarc...
Callable types > User defined functions does list both __name__ and __qualname__
Is there a non-performance regressive way to proxy attr access to __name__ of the partially curried (?) function?
From "PEP 3155 – Qualified name for classes and functions" https://peps.python.org/pep-3155/#limitations :
### Limitations With nested functions (and classes defined inside functions), the dotted path will not be walkable programmatically as a function’s namespace is not available from the outside. It will still be more helpful to the human reader than the bare __name__.
As the __name__ attribute, the __qualname__ attribute is computed statically and it will not automatically follow rebinding.
From https://wrapt.readthedocs.io/en/latest/wrappers.html#proxy-object-attributes :
Proxy Object Attributes When an attempt is made to access an attribute from the proxy, the same named attribute would in normal circumstances be accessed from the wrapped object. When updating an attributes value, or deleting the attribute, that change will also be reflected in the wrapped object.
From https://docs.python.org/3/library/weakref.html#weakref.proxy :
weakref.proxy(object[, callback])¶ Return a proxy to object which uses a weak reference. This supports use of the proxy in most contexts instead of requiring the explicit dereferencing used with weak reference objects. The returned object will have a type of either ProxyType or CallableProxyType, depending on whether object is callable. Proxy objects are not hashable regardless of the referent; this avoids a number of problems related to their fundamentally mutable nature, and prevent their use as dictionary keys. callback is the same as the parameter of the same name to the ref() function.
On Tue, Aug 30, 2022, 1:14 AM Charles Machalow <csm10495@gmail.com> wrote:
1: There are cases where one may need the __name__/__qualname__ of a given callable. If someone uses partial to create a new callable, there is no __name__/__qualname__ given. In my particular case, I'm logging what callback function is passed to a different function... if someone uses partial, there is no __name__/__qualname__ which leads to a current traceback... of course i can work around it but still was an odd case to me.
Per the docs on functools.partial: "Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords" ... which made me initially think that in order to behave like the passed in function: it should have __name__ and __qualname__... like the func did.
2: I would say have both __qualname__ and __name__. Both could be based off of __name__/__qualname__ of the passed in func.
3: This would be more difficult since you would have to disassemble the lambda to figure out the called method (or methods)... We can table the lambda discussion for the purpose of this idea. I recall that typically it is preferred to use partial over lambdas, so this could be an additional functionality/benefit of using partial over lambda.
Notes: ... __name__ being something like partial(foo, "x") would be fine with me... I just feel as though something should be there.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 9:56 PM Paul Bryan <pbryan@anode.ca> wrote:
+0
Questions:
1. What's the use case for partial having __name__? 2. Does this imply it should have __qualname__ as well? 3. What name would be given to (an inherently anonymous) lambda?
Notes:
1. I would prefer __name__ to be more qualifying like its repr (e.g. partial(foo, "x") → "<partial foo>")
On Mon, 2022-08-29 at 21:31 -0700, Charles Machalow wrote:
Hey folks,
I propose adding __name__ to functools.partial.
get_name = functools.partial(input, "name: ") get_name() name: hi 'hi' get_name.__name__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'functools.partial' object has no attribute '__name__' get_name.func <built-in function input> get_name.func.__name__ 'input'
We could set __name__ based off of partial.func.__name__ or we could try to set it to something like 'partial calling func.__name__'
If the callable doesn't have a name, we could fall back to a None __name__ or set it to something generic.
Even lambdas have __name__ set:
l = lambda: input('name: ') l.__name__ '<lambda>'
This proposal makes __name__ on partial objects more useful than the current behavior of __name__ on lambda objects as well. We could port over similar functionality to lambda if we'd like.
- Charlie Scott Machalow _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WK3FO3... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ONAEBC... Code of Conduct: http://python.org/psf/codeofconduct/
Would a property or a copy be faster for existing and possible use cases? In practice, how frequently will __qual/name__ be called on partials? - Copying __qual/name__ would definitely be a performance regression - There are probably as many use cases for partials as other methods of functional composition, - __qual/name__ support is not yet extant - it's faster to run e.g. a grid search *without* partials, due to function call overhead, due to scope allocation on the stack in stackful pythons [1] [1] Hyper Parameter Search > Scaling hyperparameter searches https://ml.dask.org/hyper-parameter-search.html#scaling-hyperparameter-searc... [2] Pipeline caching in TPOT http://epistasislab.github.io/tpot/using/#pipeline-caching-in-tpot #parallel-training-with-dask ; TPOT generates actual python source code instead of an ensemble of partials On Tue, Aug 30, 2022, 12:07 PM Charles Machalow <csm10495@gmail.com> wrote:
We may be able to do __name__/__qualname__ as a property to make it evaluate when called as opposed to computed once on creation. That way we just work with .func upon call so no need for extra references, etc.
As for documentation generation tools, it may be different at first, though I believe the existing ispartial checks would catch partials still. If they want to (in a new version) swap to using __name__/__qualname__ that should be fine, but this likely wouldn't inherently break existing tools.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 11:08 PM Wes Turner <wes.turner@gmail.com> wrote:
Is there a non-performance regressive way to proxy attr access to func.__name__ of the partial function (or method; Callable)?
Would this affect documentation generation tools like e.g. sphinx-spidoc, which IIRC use __name__ and probably now __qualname__ for generating argspecs in RST for HTML and LaTeX?
- https://docs.python.org/3/library/inspect.html - functions and methods have __name__ and __qualname__ - see: sphinx.utils.inspect
- https://docs.python.org/3/library/functools.html#functools.partial - https://docs.python.org/3/library/functools.html#functools.partialmethod - https://docs.python.org/3/library/functools.html#partial-objects
partial Objects¶ partial objects are callable objects created by partial(). They have three read-only attributes:
partial.func A callable object or function. Calls to the partial object will be forwarded to func with new arguments and keywords.
partial.args The leftmost positional arguments that will be prepended to the positional arguments provided to a partial object call.
partial.keywords The keyword arguments that will be supplied when the partial object is called.
partial objects are like function objects in that they are callable, weak referencable, and can have attributes. There are some important differences. For instance, the __name__ and __doc__ attributes are not created automatically. Also, partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.
- https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html - https://www.sphinx-doc.org/en/master/_modules/sphinx/ext/autodoc.html : 18 references to __qualname__,
https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L49-L66 :
```python def unwrap_all(obj: Any, *, stop: Optional[Callable] = None) -> Any: """ Get an original object from wrapped object (unwrapping partials, wrapped functions, and other decorators). """ while True: if stop and stop(obj): return obj elif ispartial(obj): obj = obj.func elif inspect.isroutine(obj) and hasattr(obj, '__wrapped__'): obj = obj.__wrapped__ # type: ignore elif isclassmethod(obj): obj = obj.__func__ elif isstaticmethod(obj): obj = obj.__func__ else: return obj ```
From https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L173-L1... :
```python def unpartial(obj: Any) -> Any: """Get an original object from partial object. This returns given object itself if not partial. """ while ispartial(obj): obj = obj.func
return obj
def ispartial(obj: Any) -> bool: """Check if the object is partial.""" return isinstance(obj, (partial, partialmethod)) ```
- https://github.com/sphinx-doc/sphinx/issues/4826#issuecomment-808699254
https://docs.python.org/3/library/stdtypes.html#definition.__name__
https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarc...
Callable types > User defined functions does list both __name__ and __qualname__
Is there a non-performance regressive way to proxy attr access to __name__ of the partially curried (?) function?
From "PEP 3155 – Qualified name for classes and functions" https://peps.python.org/pep-3155/#limitations :
### Limitations With nested functions (and classes defined inside functions), the dotted path will not be walkable programmatically as a function’s namespace is not available from the outside. It will still be more helpful to the human reader than the bare __name__.
As the __name__ attribute, the __qualname__ attribute is computed statically and it will not automatically follow rebinding.
From https://wrapt.readthedocs.io/en/latest/wrappers.html#proxy-object-attributes :
Proxy Object Attributes When an attempt is made to access an attribute from the proxy, the same named attribute would in normal circumstances be accessed from the wrapped object. When updating an attributes value, or deleting the attribute, that change will also be reflected in the wrapped object.
From https://docs.python.org/3/library/weakref.html#weakref.proxy :
weakref.proxy(object[, callback])¶ Return a proxy to object which uses a weak reference. This supports use of the proxy in most contexts instead of requiring the explicit dereferencing used with weak reference objects. The returned object will have a type of either ProxyType or CallableProxyType, depending on whether object is callable. Proxy objects are not hashable regardless of the referent; this avoids a number of problems related to their fundamentally mutable nature, and prevent their use as dictionary keys. callback is the same as the parameter of the same name to the ref() function.
On Tue, Aug 30, 2022, 1:14 AM Charles Machalow <csm10495@gmail.com> wrote:
1: There are cases where one may need the __name__/__qualname__ of a given callable. If someone uses partial to create a new callable, there is no __name__/__qualname__ given. In my particular case, I'm logging what callback function is passed to a different function... if someone uses partial, there is no __name__/__qualname__ which leads to a current traceback... of course i can work around it but still was an odd case to me.
Per the docs on functools.partial: "Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords" ... which made me initially think that in order to behave like the passed in function: it should have __name__ and __qualname__... like the func did.
2: I would say have both __qualname__ and __name__. Both could be based off of __name__/__qualname__ of the passed in func.
3: This would be more difficult since you would have to disassemble the lambda to figure out the called method (or methods)... We can table the lambda discussion for the purpose of this idea. I recall that typically it is preferred to use partial over lambdas, so this could be an additional functionality/benefit of using partial over lambda.
Notes: ... __name__ being something like partial(foo, "x") would be fine with me... I just feel as though something should be there.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 9:56 PM Paul Bryan <pbryan@anode.ca> wrote:
+0
Questions:
1. What's the use case for partial having __name__? 2. Does this imply it should have __qualname__ as well? 3. What name would be given to (an inherently anonymous) lambda?
Notes:
1. I would prefer __name__ to be more qualifying like its repr (e.g. partial(foo, "x") → "<partial foo>")
On Mon, 2022-08-29 at 21:31 -0700, Charles Machalow wrote:
Hey folks,
I propose adding __name__ to functools.partial.
> get_name = functools.partial(input, "name: ") > get_name() name: hi 'hi' > get_name.__name__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'functools.partial' object has no attribute '__name__' > get_name.func <built-in function input> > get_name.func.__name__ 'input'
We could set __name__ based off of partial.func.__name__ or we could try to set it to something like 'partial calling func.__name__'
If the callable doesn't have a name, we could fall back to a None __name__ or set it to something generic.
Even lambdas have __name__ set:
> l = lambda: input('name: ') > l.__name__ '<lambda>'
This proposal makes __name__ on partial objects more useful than the current behavior of __name__ on lambda objects as well. We could port over similar functionality to lambda if we'd like.
- Charlie Scott Machalow _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WK3FO3... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ONAEBC... Code of Conduct: http://python.org/psf/codeofconduct/
Actually, there is a good motive IMO for a partial function to have __name__ and __qualname__: the code one is passing a partial function might expect these attributes to be presented in the callable it get. It is just a matter of unifying the interface for callables that are often used as arguments in calls, and as such, even if __name__ and __qualname__ are fixed and immutable strings, this would be an improvement. (say, a partial callable __name__ could be fixed to "<partial>" just as a lambda's __name__ is "<lambda>") On Tue, Aug 30, 2022 at 4:29 PM Wes Turner <wes.turner@gmail.com> wrote:
Would a property or a copy be faster for existing and possible use cases? In practice, how frequently will __qual/name__ be called on partials?
- Copying __qual/name__ would definitely be a performance regression
- There are probably as many use cases for partials as other methods of functional composition, - __qual/name__ support is not yet extant
- it's faster to run e.g. a grid search *without* partials, due to function call overhead, due to scope allocation on the stack in stackful pythons [1]
[1] Hyper Parameter Search > Scaling hyperparameter searches
https://ml.dask.org/hyper-parameter-search.html#scaling-hyperparameter-searc...
[2] Pipeline caching in TPOT http://epistasislab.github.io/tpot/using/#pipeline-caching-in-tpot #parallel-training-with-dask ; TPOT generates actual python source code instead of an ensemble of partials
On Tue, Aug 30, 2022, 12:07 PM Charles Machalow <csm10495@gmail.com> wrote:
We may be able to do __name__/__qualname__ as a property to make it evaluate when called as opposed to computed once on creation. That way we just work with .func upon call so no need for extra references, etc.
As for documentation generation tools, it may be different at first, though I believe the existing ispartial checks would catch partials still. If they want to (in a new version) swap to using __name__/__qualname__ that should be fine, but this likely wouldn't inherently break existing tools.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 11:08 PM Wes Turner <wes.turner@gmail.com> wrote:
Is there a non-performance regressive way to proxy attr access to func.__name__ of the partial function (or method; Callable)?
Would this affect documentation generation tools like e.g. sphinx-spidoc, which IIRC use __name__ and probably now __qualname__ for generating argspecs in RST for HTML and LaTeX?
- https://docs.python.org/3/library/inspect.html - functions and methods have __name__ and __qualname__ - see: sphinx.utils.inspect
- https://docs.python.org/3/library/functools.html#functools.partial - https://docs.python.org/3/library/functools.html#functools.partialmethod - https://docs.python.org/3/library/functools.html#partial-objects
partial Objects¶ partial objects are callable objects created by partial(). They have three read-only attributes:
partial.func A callable object or function. Calls to the partial object will be forwarded to func with new arguments and keywords.
partial.args The leftmost positional arguments that will be prepended to the positional arguments provided to a partial object call.
partial.keywords The keyword arguments that will be supplied when the partial object is called.
partial objects are like function objects in that they are callable, weak referencable, and can have attributes. There are some important differences. For instance, the __name__ and __doc__ attributes are not created automatically. Also, partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.
- https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html - https://www.sphinx-doc.org/en/master/_modules/sphinx/ext/autodoc.html : 18 references to __qualname__,
https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L49-L66 :
```python def unwrap_all(obj: Any, *, stop: Optional[Callable] = None) -> Any: """ Get an original object from wrapped object (unwrapping partials, wrapped functions, and other decorators). """ while True: if stop and stop(obj): return obj elif ispartial(obj): obj = obj.func elif inspect.isroutine(obj) and hasattr(obj, '__wrapped__'): obj = obj.__wrapped__ # type: ignore elif isclassmethod(obj): obj = obj.__func__ elif isstaticmethod(obj): obj = obj.__func__ else: return obj ```
From https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L173-L1... :
```python def unpartial(obj: Any) -> Any: """Get an original object from partial object. This returns given object itself if not partial. """ while ispartial(obj): obj = obj.func
return obj
def ispartial(obj: Any) -> bool: """Check if the object is partial.""" return isinstance(obj, (partial, partialmethod)) ```
- https://github.com/sphinx-doc/sphinx/issues/4826#issuecomment-808699254
https://docs.python.org/3/library/stdtypes.html#definition.__name__
https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarc...
Callable types > User defined functions does list both __name__ and __qualname__
Is there a non-performance regressive way to proxy attr access to __name__ of the partially curried (?) function?
From "PEP 3155 – Qualified name for classes and functions" https://peps.python.org/pep-3155/#limitations :
### Limitations With nested functions (and classes defined inside functions), the dotted path will not be walkable programmatically as a function’s namespace is not available from the outside. It will still be more helpful to the human reader than the bare __name__.
As the __name__ attribute, the __qualname__ attribute is computed statically and it will not automatically follow rebinding.
From https://wrapt.readthedocs.io/en/latest/wrappers.html#proxy-object-attributes :
Proxy Object Attributes When an attempt is made to access an attribute from the proxy, the same named attribute would in normal circumstances be accessed from the wrapped object. When updating an attributes value, or deleting the attribute, that change will also be reflected in the wrapped object.
From https://docs.python.org/3/library/weakref.html#weakref.proxy :
weakref.proxy(object[, callback])¶ Return a proxy to object which uses a weak reference. This supports use of the proxy in most contexts instead of requiring the explicit dereferencing used with weak reference objects. The returned object will have a type of either ProxyType or CallableProxyType, depending on whether object is callable. Proxy objects are not hashable regardless of the referent; this avoids a number of problems related to their fundamentally mutable nature, and prevent their use as dictionary keys. callback is the same as the parameter of the same name to the ref() function.
On Tue, Aug 30, 2022, 1:14 AM Charles Machalow <csm10495@gmail.com> wrote:
1: There are cases where one may need the __name__/__qualname__ of a given callable. If someone uses partial to create a new callable, there is no __name__/__qualname__ given. In my particular case, I'm logging what callback function is passed to a different function... if someone uses partial, there is no __name__/__qualname__ which leads to a current traceback... of course i can work around it but still was an odd case to me.
Per the docs on functools.partial: "Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords" ... which made me initially think that in order to behave like the passed in function: it should have __name__ and __qualname__... like the func did.
2: I would say have both __qualname__ and __name__. Both could be based off of __name__/__qualname__ of the passed in func.
3: This would be more difficult since you would have to disassemble the lambda to figure out the called method (or methods)... We can table the lambda discussion for the purpose of this idea. I recall that typically it is preferred to use partial over lambdas, so this could be an additional functionality/benefit of using partial over lambda.
Notes: ... __name__ being something like partial(foo, "x") would be fine with me... I just feel as though something should be there.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 9:56 PM Paul Bryan <pbryan@anode.ca> wrote:
+0
Questions:
1. What's the use case for partial having __name__? 2. Does this imply it should have __qualname__ as well? 3. What name would be given to (an inherently anonymous) lambda?
Notes:
1. I would prefer __name__ to be more qualifying like its repr (e.g. partial(foo, "x") → "<partial foo>")
On Mon, 2022-08-29 at 21:31 -0700, Charles Machalow wrote:
Hey folks,
I propose adding __name__ to functools.partial.
>> get_name = functools.partial(input, "name: ") >> get_name() name: hi 'hi' >> get_name.__name__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'functools.partial' object has no attribute '__name__' >> get_name.func <built-in function input> >> get_name.func.__name__ 'input'
We could set __name__ based off of partial.func.__name__ or we could try to set it to something like 'partial calling func.__name__'
If the callable doesn't have a name, we could fall back to a None __name__ or set it to something generic.
Even lambdas have __name__ set:
>> l = lambda: input('name: ') >> l.__name__ '<lambda>'
This proposal makes __name__ on partial objects more useful than the current behavior of __name__ on lambda objects as well. We could port over similar functionality to lambda if we'd like.
- Charlie Scott Machalow _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WK3FO3... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ONAEBC... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YGARIN... Code of Conduct: http://python.org/psf/codeofconduct/
I'd likely propose doing both as properties so that we don't need to hold onto either of them... I was trying this out when I hit: ``` C:\Users\csm10495\Desktop\cpython\Scripts>ipython Could not import runpy module Traceback (most recent call last): File "<frozen importlib._bootstrap>", line 1172, in _find_and_load File "<frozen importlib._bootstrap>", line 1143, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 684, in _load_unlocked File "<frozen importlib._bootstrap>", line 976, in exec_module File "<frozen runpy>", line 15, in <module> File "<frozen importlib._bootstrap>", line 1172, in _find_and_load File "<frozen importlib._bootstrap>", line 1143, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 684, in _load_unlocked File "<frozen importlib._bootstrap>", line 976, in exec_module File "<frozen importlib.util>", line 14, in <module> File "C:\Users\csm10495\Desktop\cpython\Lib\contextlib.py", line 7, in <module> from functools import wraps File "C:\Users\csm10495\Desktop\cpython\Lib\functools.py", line 276, in <module> class partial: TypeError: type __qualname__ must be a str, not property ``` ... which is https://bugs.python.org/issue19073.. But for __name__ it seems to work: ``` In [1]: from functools import partial In [2]: from sys import getsizeof In [3]: hello = partial(print, "hello") In [4]: getsizeof(hello) Out[4]: 72 In [5]: hello.__name__ Out[5]: 'partial(print)' ``` and for the simple test, added this to the partial definition: ``` @property def __name__(self): return f"partial({self.func.__name__})" ``` Of course it probably wouldn't be too bad to add in the args/kwargs to the __name__ (and eventually __qualname__) as well. This was with commenting out using the C-version of partial and just playing with the Python version. If folks think this (cleaned up) could make upstream, I'd be happy to try to get a PR together for first allowing __qualname__ to be a property then what we're talking about here. - Charlie Scott Machalow On Tue, Aug 30, 2022 at 7:08 PM Joao S. O. Bueno <gwidion@gmail.com> wrote:
Actually, there is a good motive IMO for a partial function to have __name__ and __qualname__: the code one is passing a partial function might expect these attributes to be presented in the callable it get.
It is just a matter of unifying the interface for callables that are often used as arguments in calls, and as such, even if __name__ and __qualname__ are fixed and immutable strings, this would be an improvement. (say, a partial callable __name__ could be fixed to "<partial>" just as a lambda's __name__ is "<lambda>")
On Tue, Aug 30, 2022 at 4:29 PM Wes Turner <wes.turner@gmail.com> wrote:
Would a property or a copy be faster for existing and possible use cases? In practice, how frequently will __qual/name__ be called on partials?
- Copying __qual/name__ would definitely be a performance regression
- There are probably as many use cases for partials as other methods of functional composition, - __qual/name__ support is not yet extant
- it's faster to run e.g. a grid search *without* partials, due to function call overhead, due to scope allocation on the stack in stackful pythons [1]
[1] Hyper Parameter Search > Scaling hyperparameter searches
https://ml.dask.org/hyper-parameter-search.html#scaling-hyperparameter-searc...
[2] Pipeline caching in TPOT http://epistasislab.github.io/tpot/using/#pipeline-caching-in-tpot #parallel-training-with-dask ; TPOT generates actual python source code instead of an ensemble of partials
On Tue, Aug 30, 2022, 12:07 PM Charles Machalow <csm10495@gmail.com> wrote:
We may be able to do __name__/__qualname__ as a property to make it evaluate when called as opposed to computed once on creation. That way we just work with .func upon call so no need for extra references, etc.
As for documentation generation tools, it may be different at first, though I believe the existing ispartial checks would catch partials still. If they want to (in a new version) swap to using __name__/__qualname__ that should be fine, but this likely wouldn't inherently break existing tools.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 11:08 PM Wes Turner <wes.turner@gmail.com> wrote:
Is there a non-performance regressive way to proxy attr access to func.__name__ of the partial function (or method; Callable)?
Would this affect documentation generation tools like e.g. sphinx-spidoc, which IIRC use __name__ and probably now __qualname__ for generating argspecs in RST for HTML and LaTeX?
- https://docs.python.org/3/library/inspect.html - functions and methods have __name__ and __qualname__ - see: sphinx.utils.inspect
- https://docs.python.org/3/library/functools.html#functools.partial - https://docs.python.org/3/library/functools.html#functools.partialmethod - https://docs.python.org/3/library/functools.html#partial-objects
partial Objects¶ partial objects are callable objects created by partial(). They have three read-only attributes:
partial.func A callable object or function. Calls to the partial object will be forwarded to func with new arguments and keywords.
partial.args The leftmost positional arguments that will be prepended to the positional arguments provided to a partial object call.
partial.keywords The keyword arguments that will be supplied when the partial object is called.
partial objects are like function objects in that they are callable, weak referencable, and can have attributes. There are some important differences. For instance, the __name__ and __doc__ attributes are not created automatically. Also, partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.
- https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html - https://www.sphinx-doc.org/en/master/_modules/sphinx/ext/autodoc.html : 18 references to __qualname__,
https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L49-L66 :
```python def unwrap_all(obj: Any, *, stop: Optional[Callable] = None) -> Any: """ Get an original object from wrapped object (unwrapping partials, wrapped functions, and other decorators). """ while True: if stop and stop(obj): return obj elif ispartial(obj): obj = obj.func elif inspect.isroutine(obj) and hasattr(obj, '__wrapped__'): obj = obj.__wrapped__ # type: ignore elif isclassmethod(obj): obj = obj.__func__ elif isstaticmethod(obj): obj = obj.__func__ else: return obj ```
From https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L173-L1... :
```python def unpartial(obj: Any) -> Any: """Get an original object from partial object. This returns given object itself if not partial. """ while ispartial(obj): obj = obj.func
return obj
def ispartial(obj: Any) -> bool: """Check if the object is partial.""" return isinstance(obj, (partial, partialmethod)) ```
- https://github.com/sphinx-doc/sphinx/issues/4826#issuecomment-808699254
https://docs.python.org/3/library/stdtypes.html#definition.__name__
https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarc...
Callable types > User defined functions does list both __name__ and __qualname__
Is there a non-performance regressive way to proxy attr access to __name__ of the partially curried (?) function?
From "PEP 3155 – Qualified name for classes and functions" https://peps.python.org/pep-3155/#limitations :
### Limitations With nested functions (and classes defined inside functions), the dotted path will not be walkable programmatically as a function’s namespace is not available from the outside. It will still be more helpful to the human reader than the bare __name__.
As the __name__ attribute, the __qualname__ attribute is computed statically and it will not automatically follow rebinding.
From https://wrapt.readthedocs.io/en/latest/wrappers.html#proxy-object-attributes :
Proxy Object Attributes When an attempt is made to access an attribute from the proxy, the same named attribute would in normal circumstances be accessed from the wrapped object. When updating an attributes value, or deleting the attribute, that change will also be reflected in the wrapped object.
From https://docs.python.org/3/library/weakref.html#weakref.proxy :
weakref.proxy(object[, callback])¶ Return a proxy to object which uses a weak reference. This supports use of the proxy in most contexts instead of requiring the explicit dereferencing used with weak reference objects. The returned object will have a type of either ProxyType or CallableProxyType, depending on whether object is callable. Proxy objects are not hashable regardless of the referent; this avoids a number of problems related to their fundamentally mutable nature, and prevent their use as dictionary keys. callback is the same as the parameter of the same name to the ref() function.
On Tue, Aug 30, 2022, 1:14 AM Charles Machalow <csm10495@gmail.com> wrote:
1: There are cases where one may need the __name__/__qualname__ of a given callable. If someone uses partial to create a new callable, there is no __name__/__qualname__ given. In my particular case, I'm logging what callback function is passed to a different function... if someone uses partial, there is no __name__/__qualname__ which leads to a current traceback... of course i can work around it but still was an odd case to me.
Per the docs on functools.partial: "Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords" ... which made me initially think that in order to behave like the passed in function: it should have __name__ and __qualname__... like the func did.
2: I would say have both __qualname__ and __name__. Both could be based off of __name__/__qualname__ of the passed in func.
3: This would be more difficult since you would have to disassemble the lambda to figure out the called method (or methods)... We can table the lambda discussion for the purpose of this idea. I recall that typically it is preferred to use partial over lambdas, so this could be an additional functionality/benefit of using partial over lambda.
Notes: ... __name__ being something like partial(foo, "x") would be fine with me... I just feel as though something should be there.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 9:56 PM Paul Bryan <pbryan@anode.ca> wrote:
+0
Questions:
1. What's the use case for partial having __name__? 2. Does this imply it should have __qualname__ as well? 3. What name would be given to (an inherently anonymous) lambda?
Notes:
1. I would prefer __name__ to be more qualifying like its repr (e.g. partial(foo, "x") → "<partial foo>")
On Mon, 2022-08-29 at 21:31 -0700, Charles Machalow wrote:
Hey folks,
I propose adding __name__ to functools.partial.
>>> get_name = functools.partial(input, "name: ") >>> get_name() name: hi 'hi' >>> get_name.__name__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'functools.partial' object has no attribute '__name__' >>> get_name.func <built-in function input> >>> get_name.func.__name__ 'input'
We could set __name__ based off of partial.func.__name__ or we could try to set it to something like 'partial calling func.__name__'
If the callable doesn't have a name, we could fall back to a None __name__ or set it to something generic.
Even lambdas have __name__ set:
>>> l = lambda: input('name: ') >>> l.__name__ '<lambda>'
This proposal makes __name__ on partial objects more useful than the current behavior of __name__ on lambda objects as well. We could port over similar functionality to lambda if we'd like.
- Charlie Scott Machalow _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WK3FO3... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ONAEBC... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YGARIN... Code of Conduct: http://python.org/psf/codeofconduct/
+0.1 On Tue, Aug 30, 2022, 11:15 PM Charles Machalow <csm10495@gmail.com> wrote:
``` and for the simple test, added this to the partial definition:
``` @property def __name__(self): return f"partial({self.func.__name__})" ``` Of course it probably wouldn't be too bad to add in the args/kwargs to the __name__ (and eventually __qualname__) as well.
That'd be real nice. IIRC default repr()'s present __name__ but not (yet) __qualname__? From ```quote Note Parameter values are passed as-is to tests (no copy whatsoever). For example, if you pass a list or a dict as a parameter value, and the test case code mutates it, the mutations will be reflected in subsequent test case calls. Note pytest by default escapes any non-ascii characters used in unicode strings for the parametrization because it has several downsides. If however you would like to use unicode strings in parametrization and see them in the terminal as is (non-escaped), use this option in your pytest.ini: [pytest] disable_test_id_escaping_and_forfeit_all_rights_to_community_support = True Keep in mind however that this might cause unwanted side effects and even bugs depending on the OS used and plugins currently installed, so use it at your own risk ``` Re: punycode / idna: https://gist.github.com/pessom/b6c8c4d55296e5439403ae3cc942fbbc :
x="🖥️"; print((x, x.encode('idna')); assert x = x.encode('idna').decode('idna')
This was with commenting out using the C-version of partial and just playing with the Python version. If folks think this (cleaned up) could make upstream, I'd be happy to try to get a PR together for first allowing __qualname__ to be a property then what we're talking about here.
+0.1
- Charlie Scott Machalow
On Tue, Aug 30, 2022 at 7:08 PM Joao S. O. Bueno <gwidion@gmail.com> wrote:
Actually, there is a good motive IMO for a partial function to have __name__ and __qualname__: the code one is passing a partial function might expect these attributes to be presented in the callable it get.
It is just a matter of unifying the interface for callables that are often used as arguments in calls, and as such, even if __name__ and __qualname__ are fixed and immutable strings, this would be an improvement. (say, a partial callable __name__ could be fixed to "<partial>" just as a lambda's __name__ is "<lambda>")
On Tue, Aug 30, 2022 at 4:29 PM Wes Turner <wes.turner@gmail.com> wrote:
Would a property or a copy be faster for existing and possible use cases? In practice, how frequently will __qual/name__ be called on partials?
- Copying __qual/name__ would definitely be a performance regression
- There are probably as many use cases for partials as other methods of functional composition, - __qual/name__ support is not yet extant
- it's faster to run e.g. a grid search *without* partials, due to function call overhead, due to scope allocation on the stack in stackful pythons [1]
[1] Hyper Parameter Search > Scaling hyperparameter searches
https://ml.dask.org/hyper-parameter-search.html#scaling-hyperparameter-searc...
[2] Pipeline caching in TPOT http://epistasislab.github.io/tpot/using/#pipeline-caching-in-tpot #parallel-training-with-dask ; TPOT generates actual python source code instead of an ensemble of partials
On Tue, Aug 30, 2022, 12:07 PM Charles Machalow <csm10495@gmail.com> wrote:
We may be able to do __name__/__qualname__ as a property to make it evaluate when called as opposed to computed once on creation. That way we just work with .func upon call so no need for extra references, etc.
As for documentation generation tools, it may be different at first, though I believe the existing ispartial checks would catch partials still. If they want to (in a new version) swap to using __name__/__qualname__ that should be fine, but this likely wouldn't inherently break existing tools.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 11:08 PM Wes Turner <wes.turner@gmail.com> wrote:
Is there a non-performance regressive way to proxy attr access to func.__name__ of the partial function (or method; Callable)?
Would this affect documentation generation tools like e.g. sphinx-spidoc, which IIRC use __name__ and probably now __qualname__ for generating argspecs in RST for HTML and LaTeX?
- https://docs.python.org/3/library/inspect.html - functions and methods have __name__ and __qualname__ - see: sphinx.utils.inspect
- https://docs.python.org/3/library/functools.html#functools.partial - https://docs.python.org/3/library/functools.html#functools.partialmethod - https://docs.python.org/3/library/functools.html#partial-objects
partial Objects¶ partial objects are callable objects created by partial(). They have three read-only attributes:
partial.func A callable object or function. Calls to the partial object will be forwarded to func with new arguments and keywords.
partial.args The leftmost positional arguments that will be prepended to the positional arguments provided to a partial object call.
partial.keywords The keyword arguments that will be supplied when the partial object is called.
partial objects are like function objects in that they are callable, weak referencable, and can have attributes. There are some important differences. For instance, the __name__ and __doc__ attributes are not created automatically. Also, partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.
- https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html - https://www.sphinx-doc.org/en/master/_modules/sphinx/ext/autodoc.html : 18 references to __qualname__,
https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L49-L66 :
```python def unwrap_all(obj: Any, *, stop: Optional[Callable] = None) -> Any: """ Get an original object from wrapped object (unwrapping partials, wrapped functions, and other decorators). """ while True: if stop and stop(obj): return obj elif ispartial(obj): obj = obj.func elif inspect.isroutine(obj) and hasattr(obj, '__wrapped__'): obj = obj.__wrapped__ # type: ignore elif isclassmethod(obj): obj = obj.__func__ elif isstaticmethod(obj): obj = obj.__func__ else: return obj ```
From https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L173-L1... :
```python def unpartial(obj: Any) -> Any: """Get an original object from partial object. This returns given object itself if not partial. """ while ispartial(obj): obj = obj.func
return obj
def ispartial(obj: Any) -> bool: """Check if the object is partial.""" return isinstance(obj, (partial, partialmethod)) ```
- https://github.com/sphinx-doc/sphinx/issues/4826#issuecomment-808699254
https://docs.python.org/3/library/stdtypes.html#definition.__name__
https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarc...
Callable types > User defined functions does list both __name__ and __qualname__
Is there a non-performance regressive way to proxy attr access to __name__ of the partially curried (?) function?
From "PEP 3155 – Qualified name for classes and functions" https://peps.python.org/pep-3155/#limitations :
### Limitations With nested functions (and classes defined inside functions), the dotted path will not be walkable programmatically as a function’s namespace is not available from the outside. It will still be more helpful to the human reader than the bare __name__.
As the __name__ attribute, the __qualname__ attribute is computed statically and it will not automatically follow rebinding.
From https://wrapt.readthedocs.io/en/latest/wrappers.html#proxy-object-attributes :
Proxy Object Attributes When an attempt is made to access an attribute from the proxy, the same named attribute would in normal circumstances be accessed from the wrapped object. When updating an attributes value, or deleting the attribute, that change will also be reflected in the wrapped object.
From https://docs.python.org/3/library/weakref.html#weakref.proxy :
weakref.proxy(object[, callback])¶ Return a proxy to object which uses a weak reference. This supports use of the proxy in most contexts instead of requiring the explicit dereferencing used with weak reference objects. The returned object will have a type of either ProxyType or CallableProxyType, depending on whether object is callable. Proxy objects are not hashable regardless of the referent; this avoids a number of problems related to their fundamentally mutable nature, and prevent their use as dictionary keys. callback is the same as the parameter of the same name to the ref() function.
On Tue, Aug 30, 2022, 1:14 AM Charles Machalow <csm10495@gmail.com> wrote:
1: There are cases where one may need the __name__/__qualname__ of a given callable. If someone uses partial to create a new callable, there is no __name__/__qualname__ given. In my particular case, I'm logging what callback function is passed to a different function... if someone uses partial, there is no __name__/__qualname__ which leads to a current traceback... of course i can work around it but still was an odd case to me.
Per the docs on functools.partial: "Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords" ... which made me initially think that in order to behave like the passed in function: it should have __name__ and __qualname__... like the func did.
2: I would say have both __qualname__ and __name__. Both could be based off of __name__/__qualname__ of the passed in func.
3: This would be more difficult since you would have to disassemble the lambda to figure out the called method (or methods)... We can table the lambda discussion for the purpose of this idea. I recall that typically it is preferred to use partial over lambdas, so this could be an additional functionality/benefit of using partial over lambda.
Notes: ... __name__ being something like partial(foo, "x") would be fine with me... I just feel as though something should be there.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 9:56 PM Paul Bryan <pbryan@anode.ca> wrote:
> +0 > > Questions: > > 1. What's the use case for partial having __name__? > 2. Does this imply it should have __qualname__ as well? > 3. What name would be given to (an inherently anonymous) lambda? > > Notes: > > 1. I would prefer __name__ to be more qualifying like its repr (e.g. > partial(foo, "x") → "<partial foo>") > > > On Mon, 2022-08-29 at 21:31 -0700, Charles Machalow wrote: > > Hey folks, > > I propose adding __name__ to functools.partial. > > >>> get_name = functools.partial(input, "name: ") > >>> get_name() > name: hi > 'hi' > >>> get_name.__name__ > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > AttributeError: 'functools.partial' object has no attribute > '__name__' > >>> get_name.func > <built-in function input> > >>> get_name.func.__name__ > 'input' > > We could set __name__ based off of partial.func.__name__ or we could > try to set it to something like 'partial calling func.__name__' > > If the callable doesn't have a name, we could fall back to a None > __name__ or set it to something generic. > > Even lambdas have __name__ set: > > >>> l = lambda: input('name: ') > >>> l.__name__ > '<lambda>' > > This proposal makes __name__ on partial objects more useful than the > current behavior of __name__ on lambda objects as well. We could port over > similar functionality to lambda if we'd like. > > - Charlie Scott Machalow > _______________________________________________ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-leave@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/WK3FO3... > Code of Conduct: http://python.org/psf/codeofconduct/ > > > _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ONAEBC... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YGARIN... Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Aug 31, 2022, 12:24 AM Wes Turner <wes.turner@gmail.com> wrote:
+0.1
On Tue, Aug 30, 2022, 11:15 PM Charles Machalow <csm10495@gmail.com> wrote:
``` and for the simple test, added this to the partial definition:
``` @property def __name__(self): return f"partial({self.func.__name__})" ``` Of course it probably wouldn't be too bad to add in the args/kwargs to the __name__ (and eventually __qualname__) as well.
That'd be real nice. IIRC default repr()'s present __name__ but not (yet) __qualname__?
From
https://docs.pytest.org/en/latest/how-to/parametrize.html#pytest-mark-parame...
```quote Note Parameter values are passed as-is to tests (no copy whatsoever).
For example, if you pass a list or a dict as a parameter value, and the test case code mutates it, the mutations will be reflected in subsequent test case calls.
Note pytest by default escapes any non-ascii characters used in unicode strings for the parametrization because it has several downsides. If however you would like to use unicode strings in parametrization and see them in the terminal as is (non-escaped), use this option in your pytest.ini:
[pytest] disable_test_id_escaping_and_forfeit_all_rights_to_community_support = True
Keep in mind however that this might cause unwanted side effects and even bugs depending on the OS used and plugins currently installed, so use it at your own risk ```
Re: punycode / idna: https://gist.github.com/pessom/b6c8c4d55296e5439403ae3cc942fbbc :
x="🖥️"; print((x, x.encode('idna')); assert x = x.encode('idna').decode('idna')
This was with commenting out using the C-version of partial and just playing with the Python version. If folks think this (cleaned up) could make upstream, I'd be happy to try to get a PR together for first allowing __qualname__ to be a property then what we're talking about here.
+0.1
- Charlie Scott Machalow
On Tue, Aug 30, 2022 at 7:08 PM Joao S. O. Bueno <gwidion@gmail.com> wrote:
Actually, there is a good motive IMO for a partial function to have __name__ and __qualname__: the code one is passing a partial function might expect these attributes to be presented in the callable it get.
It is just a matter of unifying the interface for callables that are often used as arguments in calls, and as such, even if __name__ and __qualname__ are fixed and immutable strings, this would be an improvement. (say, a partial callable __name__ could be fixed to "<partial>" just as a lambda's __name__ is "<lambda>")
On Tue, Aug 30, 2022 at 4:29 PM Wes Turner <wes.turner@gmail.com> wrote:
Would a property or a copy be faster for existing and possible use cases? In practice, how frequently will __qual/name__ be called on partials?
- Copying __qual/name__ would definitely be a performance regression
- There are probably as many use cases for partials as other methods of functional composition, - __qual/name__ support is not yet extant
- it's faster to run e.g. a grid search *without* partials, due to function call overhead, due to scope allocation on the stack in stackful pythons [1]
[1] Hyper Parameter Search > Scaling hyperparameter searches
https://ml.dask.org/hyper-parameter-search.html#scaling-hyperparameter-searc...
[2] Pipeline caching in TPOT http://epistasislab.github.io/tpot/using/#pipeline-caching-in-tpot #parallel-training-with-dask ; TPOT generates actual python source code instead of an ensemble of partials
On Tue, Aug 30, 2022, 12:07 PM Charles Machalow <csm10495@gmail.com> wrote:
We may be able to do __name__/__qualname__ as a property to make it evaluate when called as opposed to computed once on creation. That way we just work with .func upon call so no need for extra references, etc.
As for documentation generation tools, it may be different at first, though I believe the existing ispartial checks would catch partials still. If they want to (in a new version) swap to using __name__/__qualname__ that should be fine, but this likely wouldn't inherently break existing tools.
- Charlie Scott Machalow
On Mon, Aug 29, 2022 at 11:08 PM Wes Turner <wes.turner@gmail.com> wrote:
Is there a non-performance regressive way to proxy attr access to func.__name__ of the partial function (or method; Callable)?
Would this affect documentation generation tools like e.g. sphinx-spidoc, which IIRC use __name__ and probably now __qualname__ for generating argspecs in RST for HTML and LaTeX?
- https://docs.python.org/3/library/inspect.html - functions and methods have __name__ and __qualname__ - see: sphinx.utils.inspect
- https://docs.python.org/3/library/functools.html#functools.partial - https://docs.python.org/3/library/functools.html#functools.partialmethod - https://docs.python.org/3/library/functools.html#partial-objects
> partial Objects¶ > partial objects are callable objects created by partial(). They have three read-only attributes: > > partial.func > A callable object or function. Calls to the partial object will be forwarded to func with new arguments and keywords. > > partial.args > The leftmost positional arguments that will be prepended to the positional arguments provided to a partial object call. > > partial.keywords > The keyword arguments that will be supplied when the partial object is called.
> partial objects are like function objects in that they are callable, weak referencable, and can have attributes. There are some important differences. For instance, the __name__ and __doc__ attributes are not created automatically. Also, partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.
- https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html - https://www.sphinx-doc.org/en/master/_modules/sphinx/ext/autodoc.html : 18 references to __qualname__,
https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L49-L66 :
```python def unwrap_all(obj: Any, *, stop: Optional[Callable] = None) -> Any: """ Get an original object from wrapped object (unwrapping partials, wrapped functions, and other decorators). """ while True: if stop and stop(obj): return obj elif ispartial(obj): obj = obj.func elif inspect.isroutine(obj) and hasattr(obj, '__wrapped__'): obj = obj.__wrapped__ # type: ignore elif isclassmethod(obj): obj = obj.__func__ elif isstaticmethod(obj): obj = obj.__func__ else: return obj ```
From https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L173-L1... :
```python def unpartial(obj: Any) -> Any: """Get an original object from partial object. This returns given object itself if not partial. """ while ispartial(obj): obj = obj.func
return obj
def ispartial(obj: Any) -> bool: """Check if the object is partial.""" return isinstance(obj, (partial, partialmethod)) ```
- https://github.com/sphinx-doc/sphinx/issues/4826#issuecomment-808699254
https://docs.python.org/3/library/stdtypes.html#definition.__name__
https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarc... > Callable types > User defined functions does list both __name__ and __qualname__
Is there a non-performance regressive way to proxy attr access to __name__ of the partially curried (?) function?
From "PEP 3155 – Qualified name for classes and functions" https://peps.python.org/pep-3155/#limitations :
> ### Limitations > With nested functions (and classes defined inside functions), the dotted path will not be walkable programmatically as a function’s namespace is not available from the outside. It will still be more helpful to the human reader than the bare __name__. > > As the __name__ attribute, the __qualname__ attribute is computed statically and it will not automatically follow rebinding.
From https://wrapt.readthedocs.io/en/latest/wrappers.html#proxy-object-attributes :
> Proxy Object Attributes > When an attempt is made to access an attribute from the proxy, the same named attribute would in normal circumstances be accessed from the wrapped object. When updating an attributes value, or deleting the attribute, that change will also be reflected in the wrapped object.
From https://docs.python.org/3/library/weakref.html#weakref.proxy :
> weakref.proxy(object[, callback])¶ > Return a proxy to object which uses a weak reference. This supports use of the proxy in most contexts instead of requiring the explicit dereferencing used with weak reference objects. The returned object will have a type of either ProxyType or CallableProxyType, depending on whether object is callable. Proxy objects are not hashable regardless of the referent; this avoids a number of problems related to their fundamentally mutable nature, and prevent their use as dictionary keys. callback is the same as the parameter of the same name to the ref() function.
On Tue, Aug 30, 2022, 1:14 AM Charles Machalow <csm10495@gmail.com> wrote:
> 1: There are cases where one may need the __name__/__qualname__ of a > given callable. If someone uses partial to create a new callable, there is > no __name__/__qualname__ given. In my particular case, I'm logging what > callback function is passed to a different function... if someone uses > partial, there is no __name__/__qualname__ which leads to a current > traceback... of course i can work around it but still was an odd case to me. > > Per the docs on functools.partial: > "Return a new partial object which when called will behave like func > called with the positional arguments args and keyword arguments keywords" > ... which made me initially think that in order to behave like the > passed in function: it should have __name__ and __qualname__... like the > func did. > > 2: I would say have both __qualname__ and __name__. Both could be > based off of __name__/__qualname__ of the passed in func. > > 3: This would be more difficult since you would have to > disassemble the lambda to figure out the called method (or methods)... We > can table the lambda discussion for the purpose of this idea. I recall that > typically it is preferred to use partial over lambdas, so this could be an > additional functionality/benefit of using partial over lambda. > > Notes: > ... __name__ being something like partial(foo, "x") would be fine > with me... I just feel as though something should be there. > > - Charlie Scott Machalow > > > On Mon, Aug 29, 2022 at 9:56 PM Paul Bryan <pbryan@anode.ca> wrote: > >> +0 >> >> Questions: >> >> 1. What's the use case for partial having __name__? >> 2. Does this imply it should have __qualname__ as well? >> 3. What name would be given to (an inherently anonymous) lambda? >> >> Notes: >> >> 1. I would prefer __name__ to be more qualifying like its repr >> (e.g. partial(foo, "x") → "<partial foo>") >> >> >> On Mon, 2022-08-29 at 21:31 -0700, Charles Machalow wrote: >> >> Hey folks, >> >> I propose adding __name__ to functools.partial. >> >> >>> get_name = functools.partial(input, "name: ") >> >>> get_name() >> name: hi >> 'hi' >> >>> get_name.__name__ >> Traceback (most recent call last): >> File "<stdin>", line 1, in <module> >> AttributeError: 'functools.partial' object has no attribute >> '__name__' >> >>> get_name.func >> <built-in function input> >> >>> get_name.func.__name__ >> 'input' >> >> We could set __name__ based off of partial.func.__name__ or we >> could try to set it to something like 'partial calling >> func.__name__' >> >> If the callable doesn't have a name, we could fall back to a None >> __name__ or set it to something generic. >> >> Even lambdas have __name__ set: >> >> >>> l = lambda: input('name: ') >> >>> l.__name__ >> '<lambda>' >> >> This proposal makes __name__ on partial objects more useful than >> the current behavior of __name__ on lambda objects as well. We could port >> over similar functionality to lambda if we'd like. >> >> - Charlie Scott Machalow >> _______________________________________________ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-leave@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/WK3FO3... >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >> _______________________________________________ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-leave@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/ONAEBC... > Code of Conduct: http://python.org/psf/codeofconduct/ > _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YGARIN... Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Aug 30, 2022 at 03:28:06PM -0400, Wes Turner wrote:
- Copying __qual/name__ would definitely be a performance regression
Doubtful that it would be meaningful. It's just a lookup and assignment. From the perspective of the partial object, it's just self.__name__ = func.__name__ (give or take), and that's not likely to be expensive, especially if implemented in C as I think partial objects are. The string itself doesn't have to be copied, just the reference to it, which is fast. And its just a once-off cost for creating the partial object, it isn't going to slow down calling them. Even if the cost of this is measurable, it is unlikely to be significant to anyone unless their entire app is just creating millions of partial objects without actually calling them or using them in any way. -- Steve
On Mon, Aug 29, 2022 at 09:31:25PM -0700, Charles Machalow wrote:
Hey folks,
I propose adding __name__ to functools.partial.
participants (5)
-
Charles Machalow -
Joao S. O. Bueno -
Paul Bryan -
Steven D'Aprano -
Wes Turner