I'd like to suggest what I think would be a simple addition to `def` and `class` blocks. I don't know if calling those "Assignment Blocks" is accurate, but I just mean to refer to block syntaxes that assign to a name. Anyway, I propose a combined return-def structure, and optionally also allowing a return-class version. Omitting the name would be allowable, as well. This would only apply to a `def` or `class` statement made as the last part of the function body, of course. def ignore_exc(exc_type): return def (func): @wraps(func) return def (*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass Thanks for considering and for any comments, thoughts, or feedback on the idea!
Would you mind providing a bit more details about your proposal? What exactly are those "Assignment Blocks" supposed to do? If I understand your proposal correctly you want this: def my_func(): return def(): print("abc") to be the same as this: def my_func(): def inner_func(): print("abc") return inner_func But this is only my assumption as your proposal doesn't give very much details. Maybe you should provide a few simple examples and explain what they are supposed to do or what they should be equivalent to. Your example is quite complex and to me it's not clear at all what it is supposed to mean. Also and probably most importantly what is the reason you want this? Are there any good usecases where this would help? If my assumption above is correct this just looks like a bit of syntactic sugar that IMO isn't really neccessary. It doesn't really improve readability or save many characters. The existing way to do this is totally fine. Benedikt Am 24.10.2018 um 15:18 schrieb Calvin Spealman:
I'd like to suggest what I think would be a simple addition to `def` and `class` blocks. I don't know if calling those "Assignment Blocks" is accurate, but I just mean to refer to block syntaxes that assign to a name. Anyway, I propose a combined return-def structure, and optionally also allowing a return-class version. Omitting the name would be allowable, as well.
This would only apply to a `def` or `class` statement made as the last part of the function body, of course.
def ignore_exc(exc_type): return def (func): @wraps(func) return def (*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass
Thanks for considering and for any comments, thoughts, or feedback on the idea!
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
My idea is not "assignment blocks" those already exist. `def` and `class` blocks are both syntaxes that assign to some name. I'm just using the term to refer to them as a group. The proposal is just being able to return them. These two examples become equivalent: def ignore_exc(exc_type): return def (func): @wraps(func) return def (*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass def ignore_exc(exc_type): def decorator(func): @wraps(func) def wrapped_func(*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass return wrapped_func return decorator On Wed, Oct 24, 2018 at 9:51 AM Benedikt Werner <1benediktwerner@gmail.com> wrote:
Would you mind providing a bit more details about your proposal?
What exactly are those "Assignment Blocks" supposed to do?
If I understand your proposal correctly you want this:
def my_func(): return def(): print("abc")
to be the same as this:
def my_func(): def inner_func(): print("abc") return inner_func
But this is only my assumption as your proposal doesn't give very much details. Maybe you should provide a few simple examples and explain what they are supposed to do or what they should be equivalent to.
Your example is quite complex and to me it's not clear at all what it is supposed to mean.
Also and probably most importantly what is the reason you want this? Are there any good usecases where this would help?
If my assumption above is correct this just looks like a bit of syntactic sugar that IMO isn't really neccessary. It doesn't really improve readability or save many characters. The existing way to do this is totally fine.
Benedikt
Am 24.10.2018 um 15:18 schrieb Calvin Spealman:
I'd like to suggest what I think would be a simple addition to `def` and `class` blocks. I don't know if calling those "Assignment Blocks" is accurate, but I just mean to refer to block syntaxes that assign to a name. Anyway, I propose a combined return-def structure, and optionally also allowing a return-class version. Omitting the name would be allowable, as well.
This would only apply to a `def` or `class` statement made as the last part of the function body, of course.
def ignore_exc(exc_type): return def (func): @wraps(func) return def (*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass
Thanks for considering and for any comments, thoughts, or feedback on the idea!
_______________________________________________ Python-ideas mailing listPython-ideas@python.orghttps://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 24/10/2018 15:04, Calvin Spealman wrote:
My idea is not "assignment blocks" those already exist. `def` and `class` blocks are both syntaxes that assign to some name. I'm just using the term to refer to them as a group.
The proposal is just being able to return them. These two examples become equivalent:
def ignore_exc(exc_type): return def (func): @wraps(func) return def (*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass
def ignore_exc(exc_type): def decorator(func): @wraps(func) def wrapped_func(*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass return wrapped_func return decorator
Essentially this is a limited multi-line lambda. Either people are going to be surprised that you can only use it in a return statement or you have to open the whole can of worms about multi-line lambdas. Good luck on the latter. -- Rhodri James *-* Kynesim Ltd
On Wed, Oct 24, 2018 at 7:55 AM Rhodri James <rhodri@kynesim.co.uk> wrote:
On 24/10/2018 15:04, Calvin Spealman wrote:
My idea is not "assignment blocks" those already exist. `def` and `class` blocks are both syntaxes that assign to some name. I'm just using the term to refer to them as a group.
The proposal is just being able to return them. These two examples become equivalent:
def ignore_exc(exc_type): return def (func): @wraps(func) return def (*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass
def ignore_exc(exc_type): def decorator(func): @wraps(func) def wrapped_func(*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass return wrapped_func return decorator
Essentially this is a limited multi-line lambda. Either people are going to be surprised that you can only use it in a return statement or you have to open the whole can of worms about multi-line lambdas. Good luck on the latter.
Let's close that can quickly. Syntactically this is much simpler because because there's no trouble with switching between expression-mode and statement-mode. Also note that syntactically it is clearly a special form of `def` statement -- it can even be decorated! So let's review the proposal as a shorthand for defining a function and immediately returning it. It saves one line plus picking a name. I personally don't think that's enough of a benefit to warrant the extra syntactic complexity (even if modest). -- --Guido van Rossum (python.org/~guido)
In addition to what GVR wrote, a generator function would have to have its own syntax - backward compatibility would be another issue. Virendra Tripathi Santa Clara, CA 415-910-4955 tripsvt@gmail.com On Wed, Oct 24, 2018 at 2:31 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Calvin Spealman wrote:
def ignore_exc(exc_type): return def (func): ...
Something very similar has been suggested before, you might like to review the previous discussion.
-- Greg
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 25/10/2018 02:15, Virendra Tripathi wrote:
In addition to what GVR wrote, a generator function would have to have its own syntax - backward compatibility would be another issue.
No it wouldn't. This is about returning the function/generator/class itself, not its results. I still don't think it's worth it, mind you.
On Wed, Oct 24, 2018 at 2:31 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Calvin Spealman wrote:
def ignore_exc(exc_type): return def (func): ...
Something very similar has been suggested before, you might like to review the previous discussion. -- Rhodri James *-* Kynesim Ltd
On Wed, Oct 24, 2018 at 09:18:14AM -0400, Calvin Spealman wrote:
I'd like to suggest what I think would be a simple addition to `def` and `class` blocks. I don't know if calling those "Assignment Blocks" is accurate, but I just mean to refer to block syntaxes that assign to a name. Anyway, I propose a combined return-def structure, and optionally also allowing a return-class version. Omitting the name would be allowable, as well.
This would only apply to a `def` or `class` statement made as the last part of the function body, of course.
def ignore_exc(exc_type): return def (func): @wraps(func) return def (*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass
Your example is too complex for me this early in the morning -- I can't tell what it actually *does*, as it is obscured by what looks like a bunch of irrelevent code. I *think* you are proposing the following syntax. Am I right? return def (func): # body of func which is equivalent to: def func: # body of func return func And similar for classes: return class (K): # body of K being equivalent to: class K: # body of K return K Is it intentional that your example function takes no arguments? If the function did, where would the parameter list go in your syntax? Aside from saving one line, what is the purpose of this? -- Steve
The idea is okay, but I'm not keen about the syntax, because it doesn't read well as English. Currently "def" can be read as the verb "define", but "return define" doesn't make sense. Maybe it would be better as a pseudo-decorator: @return def f(args): ... But I'm inclined to agree with Guido that this isn't a frequent enough thing to warrant special syntax. -- Greg
On Wed, Oct 24, 2018 at 4:41 PM Steven D'Aprano <steve@pearwood.info> wrote:
I'd like to suggest what I think would be a simple addition to `def` and `class` blocks. I don't know if calling those "Assignment Blocks" is accurate, but I just mean to refer to block syntaxes that assign to a name. Anyway, I propose a combined return-def structure, and optionally also allowing a return-class version. Omitting the name would be allowable, as well.
This would only apply to a `def` or `class` statement made as the last
On Wed, Oct 24, 2018 at 09:18:14AM -0400, Calvin Spealman wrote: part
of the function body, of course.
def ignore_exc(exc_type): return def (func): @wraps(func) return def (*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass
Your example is too complex for me this early in the morning -- I can't tell what it actually *does*, as it is obscured by what looks like a bunch of irrelevent code.
I *think* you are proposing the following syntax. Am I right?
return def (func): # body of func
which is equivalent to:
def func: # body of func return func
And similar for classes:
return class (K): # body of K
being equivalent to:
class K: # body of K return K
Is it intentional that your example function takes no arguments? If the function did, where would the parameter list go in your syntax?
Aside from saving one line, what is the purpose of this?
The point is not saving a line or typing, but saving a thought. Expressing the intent of the factory function more clearly. Decorators don't do more than "saving one line", either. But the biggest reason I'd like something like this is that it solves a *specific* version of the multi-line anonymous function that comes up over and over and over again, and maybe by chipping away at those use-cases we can stop seeing *that* debate over and over and over again. :-) So, no, it doesn't save a lot of typing, but I'm never interested in that. I don't spend a lot of time typing code, I spend it reading code, and something like this would certainly help there, imho.
-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
The point is not saving a line or typing, but saving a thought. Expressing the intent of the factory function more clearly.
Could you give some other usage examples? To me it seems like a nicer and less confusing way to create decorators is warranted but could then be more specialized and this much nicer. We wouldn't necessarily get to that future point by adding super tiny improvements to what we have, but instead take a step back and rethink what the syntax could be. Personally I often just copy paste an existing decorator and then tweak it instead of writing from scratch because it's too confusing and error prone to do that. / Anders
On Thu, Oct 25, 2018 at 9:28 AM Anders Hovmöller <boxed@killingar.net> wrote:
The point is not saving a line or typing, but saving a thought. Expressing the intent of the factory function more clearly.
Could you give some other usage examples?
To me it seems like a nicer and less confusing way to create decorators is warranted but could then be more specialized and this much nicer. We wouldn't necessarily get to that future point by adding super tiny improvements to what we have, but instead take a step back and rethink what the syntax could be. Personally I often just copy paste an existing decorator and then tweak it instead of writing from scratch because it's too confusing and error prone to do that.
You know what, I *can't* think of other good examples than slightly improved decorators. So, maybe its not that great an idea if it can't be applied to at least a few more use cases. / Anders
On Thu, Oct 25, 2018 at 7:41 AM Calvin Spealman <cspealma@redhat.com> wrote:
On Thu, Oct 25, 2018 at 9:28 AM Anders Hovmöller <boxed@killingar.net> wrote:
[Calvin]
The point is not saving a line or typing, but saving a thought. Expressing the intent of the factory function more clearly.
[...]
You know what, I *can't* think of other good examples than slightly improved decorators. So, maybe its not that great an idea if it can't be applied to at least a few more use cases.
I appreciate that you're saving a thought, and I think it isn't a bad idea. It is indeed a very common pattern in decorators. But it would be a bit of a one-trick pony, since there aren't many *other* situations where it's useful. Compared to decorators themselves (which also, as was pointed out, do so much more than "saving a line") the use case is just much narrower. So unless we find more use cases, or until we can convince ourselves that we can use `def (args): block` in all expression contexts, I guess it'll have to remain an idea. Thank you though! It was a fascinating one. -- --Guido van Rossum (python.org/~guido)
Calvin Spealman wrote:
You know what, I /can't/ think of other good examples than slightly improved decorators. So, maybe its not that great an idea if it can't be applied to at least a few more use cases.
The class version might be useful for things like namedtuple that dynamically create classes. But that's even rarer than decorators. -- Greg
Was: Return for assignment blocks Anders Hovmöller wrote:
Personally I often just copy paste an existing decorator and then tweak it instead of writing from scratch because it's too confusing and error prone to do that.
Thank you, Anders, for your honesty. I also find writing decorators a bit hard. It seems to be something I have to learn anew each time I do it. Particularly for the pattern @deco(arg1, arg2) def fn(arg3, arg4): # function body Perhaps doing something with partial might help here. Anyone here interested in exploring this? -- Jonathan
Jonathan Fine wrote:
I also find writing decorators a bit hard. It seems to be something I have to learn anew each time I do it. Particularly for the pattern
@deco(arg1, arg2) def fn(arg3, arg4): # function body
Perhaps doing something with partial might help here. Anyone here interested in exploring this?
I can't think of a way that partial would help. But would you find it easier if you could do something like this? class deco(Decorator): def __init__(self, arg1, arg2): self.arg1 = arg1 self.arg2 = arg2 def invoke(self, func, arg3, arg4): # function body Implementation: class Decorator: def __call__(self, func): self._wrapped_function = func return self._wrapper def _wrapper(self, *args, **kwds): return self.invoke(self._wrapped_function, *args, **kwds) -- Greg
Dear python enthusiasts, Writing python decorators is indeed quite a tideous process, in particular when you wish to add arguments, and in particular in two cases : all optional arguments, and one mandatory argument. Indeed in these two cases there is a need to disambiguate between no-parenthesis and with-parenthesis usage. After having struggled with this pattern for two years in various open source and industrial projects, I ended up writing a library to hopefully solve this once and for all: https://smarie.github.io/python-decopatch/ . It is extensively tested (203 tests) against many combinations of signature/calls. I would gladly appreciate any feedback ! Please note that there is a "PEP proposal draft" in the project page because I belive that the best a library can do will always be a poor workaround, where the interpreter or stdlib could really fix it properly. Sorry for not providing too much implementation details in that page, my knowledge of the python interpreter is unfortunately quite limited. -- Finally there is an additional topic around decorators : people tend to believe that decorators and function wrappers are the same, which is absolutely not the case. I used the famous `decorator` lib in many projects but I was not satisfied because it was solving both issues at the same time, maintaining the confusion. I therefore proposed https://smarie.github.io/python-makefun/ . In particular it provides an equivalent of `@functools.wraps` that is truly signature-preserving (based on the same recipe than `decorator`). Once again, any feedback would be gladly appreciated ! Kind regards Sylvain -----Message d'origine----- De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org> De la part de Greg Ewing Envoyé : vendredi 26 octobre 2018 00:04 À : python-ideas <python-ideas@python.org> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators [External email: Use caution with links and attachments] ________________________________ Jonathan Fine wrote:
I also find writing decorators a bit hard. It seems to be something I have to learn anew each time I do it. Particularly for the pattern
@deco(arg1, arg2) def fn(arg3, arg4): # function body
Perhaps doing something with partial might help here. Anyone here interested in exploring this?
I can't think of a way that partial would help. But would you find it easier if you could do something like this? class deco(Decorator): def __init__(self, arg1, arg2): self.arg1 = arg1 self.arg2 = arg2 def invoke(self, func, arg3, arg4): # function body Implementation: class Decorator: def __call__(self, func): self._wrapped_function = func return self._wrapper def _wrapper(self, *args, **kwds): return self.invoke(self._wrapped_function, *args, **kwds) -- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.pyth... Code of Conduct: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org... ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
What advantage do you perceive decopatch to have over wrapt? ( https://github.com/GrahamDumpleton/wrapt) On Tue, Mar 12, 2019, 5:37 AM Sylvain MARIE via Python-ideas < python-ideas@python.org> wrote:
Dear python enthusiasts,
Writing python decorators is indeed quite a tideous process, in particular when you wish to add arguments, and in particular in two cases : all optional arguments, and one mandatory argument. Indeed in these two cases there is a need to disambiguate between no-parenthesis and with-parenthesis usage.
After having struggled with this pattern for two years in various open source and industrial projects, I ended up writing a library to hopefully solve this once and for all: https://smarie.github.io/python-decopatch/ . It is extensively tested (203 tests) against many combinations of signature/calls. I would gladly appreciate any feedback !
Please note that there is a "PEP proposal draft" in the project page because I belive that the best a library can do will always be a poor workaround, where the interpreter or stdlib could really fix it properly. Sorry for not providing too much implementation details in that page, my knowledge of the python interpreter is unfortunately quite limited.
--
Finally there is an additional topic around decorators : people tend to believe that decorators and function wrappers are the same, which is absolutely not the case. I used the famous `decorator` lib in many projects but I was not satisfied because it was solving both issues at the same time, maintaining the confusion. I therefore proposed https://smarie.github.io/python-makefun/ . In particular it provides an equivalent of `@functools.wraps` that is truly signature-preserving (based on the same recipe than `decorator`). Once again, any feedback would be gladly appreciated !
Kind regards
Sylvain
-----Message d'origine----- De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org> De la part de Greg Ewing Envoyé : vendredi 26 octobre 2018 00:04 À : python-ideas <python-ideas@python.org> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators
[External email: Use caution with links and attachments]
________________________________
Jonathan Fine wrote:
I also find writing decorators a bit hard. It seems to be something I have to learn anew each time I do it. Particularly for the pattern
@deco(arg1, arg2) def fn(arg3, arg4): # function body
Perhaps doing something with partial might help here. Anyone here interested in exploring this?
I can't think of a way that partial would help. But would you find it easier if you could do something like this?
class deco(Decorator):
def __init__(self, arg1, arg2): self.arg1 = arg1 self.arg2 = arg2
def invoke(self, func, arg3, arg4): # function body
Implementation:
class Decorator:
def __call__(self, func): self._wrapped_function = func return self._wrapper
def _wrapper(self, *args, **kwds): return self.invoke(self._wrapped_function, *args, **kwds)
-- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.pyth... Code of Conduct: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org...
______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________ _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Mar 12, 2019 at 09:36:41AM +0000, Sylvain MARIE via Python-ideas wrote:
I therefore proposed https://smarie.github.io/python-makefun/ . In particular it provides an equivalent of `@functools.wraps` that is truly signature-preserving
Tell us more about that please. I'm very interested in getting decorators preserve the original signature. -- Steven
David, Steven, Thanks for your interest ! As you probably know, decorators and function wrappers are *completely different concepts*. A decorator can directly return the decorated function (or class), it does not have to return a wrapper. Even more, it can entirely replace the decorated item with something else (not even a function or class!). Try it: it is possible to write a decorator to replace a function with an integer, even though it is probably not quite useful :) `decopatch` helps you write decorators, whatever they are. It "just" solves the annoying issue of having to handle the no-parenthesis and with-parenthesis calls. In addition as a 'goodie', it proposes two development styles: *nested* (you have to return a function) and *flat* (you directly write what will happen when the decorator is applied to something). -- Now about creating signature-preserving function wrappers (in a decorator, or outside a decorator - again, that's not related). That use case is supposed to be covered by functools.wrapt. Unfortunately as explained here https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/5510... this is not the case because with functools.wrapt: - the wrapper code will execute even when the provided arguments are invalid. - the wrapper code cannot easily access an argument using its name, from the received *args, **kwargs. Indeed one would have to handle all cases (positional, keyword, default) and therefore to use something like Signature.bind(). For this reason I proposed a replacement in `makefun`: https://smarie.github.io/python-makefun/#signature-preserving-function-wrapp... -- Now bridging the gap. Of course a very interesting use cases for decorators is to create decorators that create a signature-preserving wrapper. It is possible to combine decopatch and makefun for this: https://smarie.github.io/python-decopatch/#3-creating-function-wrappers . Decopatch even proposes a "double-flat" development style where you directly write the wrapper body, as explained in the doc. Did I answer your questions ? Thanks again for the quick feedback ! Best, Sylvain -----Message d'origine----- De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org> De la part de Steven D'Aprano Envoyé : mardi 12 mars 2019 12:30 À : python-ideas@python.org Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators [External email: Use caution with links and attachments] ________________________________ On Tue, Mar 12, 2019 at 09:36:41AM +0000, Sylvain MARIE via Python-ideas wrote:
I therefore proposed https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma rie.github.io%2Fpython-makefun%2F&data=02%7C01%7Csylvain.marie%40s e.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae 68fef%7C0%7C0%7C636879872385158085&sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2 Fekk35bnYGvmEFJyCXaLDyLm9I%3D&reserved=0 . In particular it provides an equivalent of `@functools.wraps` that is truly signature-preserving
Tell us more about that please. I'm very interested in getting decorators preserve the original signature. -- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.pyth... Code of Conduct: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org... ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
David I realize that you were pointing at the 'wrapt' library not the functools.wrapt library.
From what I have tried with that library, it does not solve the issue solved with decopatch.
Sylvain -----Message d'origine----- De : Sylvain MARIE Envoyé : mardi 12 mars 2019 14:53 À : Steven D'Aprano <steve@pearwood.info>; python-ideas@python.org; David Mertz <mertz@gnosis.cx> Objet : RE: [Python-ideas] Problems (and solutions?) in writing decorators David, Steven, Thanks for your interest ! As you probably know, decorators and function wrappers are *completely different concepts*. A decorator can directly return the decorated function (or class), it does not have to return a wrapper. Even more, it can entirely replace the decorated item with something else (not even a function or class!). Try it: it is possible to write a decorator to replace a function with an integer, even though it is probably not quite useful :) `decopatch` helps you write decorators, whatever they are. It "just" solves the annoying issue of having to handle the no-parenthesis and with-parenthesis calls. In addition as a 'goodie', it proposes two development styles: *nested* (you have to return a function) and *flat* (you directly write what will happen when the decorator is applied to something). -- Now about creating signature-preserving function wrappers (in a decorator, or outside a decorator - again, that's not related). That use case is supposed to be covered by functools.wrapt. Unfortunately as explained here https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/5510... this is not the case because with functools.wrapt: - the wrapper code will execute even when the provided arguments are invalid. - the wrapper code cannot easily access an argument using its name, from the received *args, **kwargs. Indeed one would have to handle all cases (positional, keyword, default) and therefore to use something like Signature.bind(). For this reason I proposed a replacement in `makefun`: https://smarie.github.io/python-makefun/#signature-preserving-function-wrapp... -- Now bridging the gap. Of course a very interesting use cases for decorators is to create decorators that create a signature-preserving wrapper. It is possible to combine decopatch and makefun for this: https://smarie.github.io/python-decopatch/#3-creating-function-wrappers . Decopatch even proposes a "double-flat" development style where you directly write the wrapper body, as explained in the doc. Did I answer your questions ? Thanks again for the quick feedback ! Best, Sylvain -----Message d'origine----- De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org> De la part de Steven D'Aprano Envoyé : mardi 12 mars 2019 12:30 À : python-ideas@python.org Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators [External email: Use caution with links and attachments] ________________________________ On Tue, Mar 12, 2019 at 09:36:41AM +0000, Sylvain MARIE via Python-ideas wrote:
I therefore proposed https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma rie.github.io%2Fpython-makefun%2F&data=02%7C01%7Csylvain.marie%40s e.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae 68fef%7C0%7C0%7C636879872385158085&sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2 Fekk35bnYGvmEFJyCXaLDyLm9I%3D&reserved=0 . In particular it provides an equivalent of `@functools.wraps` that is truly signature-preserving
Tell us more about that please. I'm very interested in getting decorators preserve the original signature. -- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.pyth... Code of Conduct: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org... ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
The wrapt module I linked to (not funtools.wraps) provides all the capabilities you mention since 2013. It allows mixed use of decorators as decorator factories. It has a flat style. There are some minor API difference between your libraries and wrapt, but the concept is very similar. Since yours is something new, I imagine you perceive some win over what wrapt does. On Tue, Mar 12, 2019, 9:52 AM Sylvain MARIE <sylvain.marie@se.com> wrote:
David, Steven,
Thanks for your interest !
As you probably know, decorators and function wrappers are *completely different concepts*. A decorator can directly return the decorated function (or class), it does not have to return a wrapper. Even more, it can entirely replace the decorated item with something else (not even a function or class!). Try it: it is possible to write a decorator to replace a function with an integer, even though it is probably not quite useful :)
`decopatch` helps you write decorators, whatever they are. It "just" solves the annoying issue of having to handle the no-parenthesis and with-parenthesis calls. In addition as a 'goodie', it proposes two development styles: *nested* (you have to return a function) and *flat* (you directly write what will happen when the decorator is applied to something). -- Now about creating signature-preserving function wrappers (in a decorator, or outside a decorator - again, that's not related). That use case is supposed to be covered by functools.wrapt. Unfortunately as explained here https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/5510... this is not the case because with functools.wrapt: - the wrapper code will execute even when the provided arguments are invalid. - the wrapper code cannot easily access an argument using its name, from the received *args, **kwargs. Indeed one would have to handle all cases (positional, keyword, default) and therefore to use something like Signature.bind().
For this reason I proposed a replacement in `makefun`: https://smarie.github.io/python-makefun/#signature-preserving-function-wrapp... -- Now bridging the gap. Of course a very interesting use cases for decorators is to create decorators that create a signature-preserving wrapper. It is possible to combine decopatch and makefun for this: https://smarie.github.io/python-decopatch/#3-creating-function-wrappers . Decopatch even proposes a "double-flat" development style where you directly write the wrapper body, as explained in the doc.
Did I answer your questions ? Thanks again for the quick feedback ! Best,
Sylvain
-----Message d'origine----- De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org> De la part de Steven D'Aprano Envoyé : mardi 12 mars 2019 12:30 À : python-ideas@python.org Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators
[External email: Use caution with links and attachments]
________________________________
On Tue, Mar 12, 2019 at 09:36:41AM +0000, Sylvain MARIE via Python-ideas wrote:
I therefore proposed https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma rie.github.io%2Fpython-makefun%2F&data=02%7C01%7Csylvain.marie%40s e.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae 68fef%7C0%7C0%7C636879872385158085&sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2 Fekk35bnYGvmEFJyCXaLDyLm9I%3D&reserved=0 . In particular it provides an equivalent of `@functools.wraps` that is truly signature-preserving
Tell us more about that please. I'm very interested in getting decorators preserve the original signature.
-- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.pyth... Code of Conduct: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org...
______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
The documentation for wrapt mentions: Decorators With Optional Arguments <https://wrapt.readthedocs.io/en/latest/decorators.html#decorators-with-optio...> Although opinion can be mixed about whether the pattern is a good one, if the decorator arguments all have default values, it is also possible to implement decorators which have optional arguments. As Graham hints in his docs, I think repurposing decorator factories as decorators is an antipattern. Explicit is better than implicit. While I *do* understands that what decotools and makefun do are technically independent, I'm not sure I ever want them independently in practice. I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers. On Tue, Mar 12, 2019, 10:18 AM David Mertz <mertz@gnosis.cx> wrote:
The wrapt module I linked to (not funtools.wraps) provides all the capabilities you mention since 2013. It allows mixed use of decorators as decorator factories. It has a flat style.
There are some minor API difference between your libraries and wrapt, but the concept is very similar. Since yours is something new, I imagine you perceive some win over what wrapt does.
On Tue, Mar 12, 2019, 9:52 AM Sylvain MARIE <sylvain.marie@se.com> wrote:
David, Steven,
Thanks for your interest !
As you probably know, decorators and function wrappers are *completely different concepts*. A decorator can directly return the decorated function (or class), it does not have to return a wrapper. Even more, it can entirely replace the decorated item with something else (not even a function or class!). Try it: it is possible to write a decorator to replace a function with an integer, even though it is probably not quite useful :)
`decopatch` helps you write decorators, whatever they are. It "just" solves the annoying issue of having to handle the no-parenthesis and with-parenthesis calls. In addition as a 'goodie', it proposes two development styles: *nested* (you have to return a function) and *flat* (you directly write what will happen when the decorator is applied to something). -- Now about creating signature-preserving function wrappers (in a decorator, or outside a decorator - again, that's not related). That use case is supposed to be covered by functools.wrapt. Unfortunately as explained here https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/5510... this is not the case because with functools.wrapt: - the wrapper code will execute even when the provided arguments are invalid. - the wrapper code cannot easily access an argument using its name, from the received *args, **kwargs. Indeed one would have to handle all cases (positional, keyword, default) and therefore to use something like Signature.bind().
For this reason I proposed a replacement in `makefun`: https://smarie.github.io/python-makefun/#signature-preserving-function-wrapp... -- Now bridging the gap. Of course a very interesting use cases for decorators is to create decorators that create a signature-preserving wrapper. It is possible to combine decopatch and makefun for this: https://smarie.github.io/python-decopatch/#3-creating-function-wrappers . Decopatch even proposes a "double-flat" development style where you directly write the wrapper body, as explained in the doc.
Did I answer your questions ? Thanks again for the quick feedback ! Best,
Sylvain
-----Message d'origine----- De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org> De la part de Steven D'Aprano Envoyé : mardi 12 mars 2019 12:30 À : python-ideas@python.org Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators
[External email: Use caution with links and attachments]
________________________________
On Tue, Mar 12, 2019 at 09:36:41AM +0000, Sylvain MARIE via Python-ideas wrote:
I therefore proposed https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma rie.github.io%2Fpython-makefun%2F&data=02%7C01%7Csylvain.marie%40s e.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae 68fef%7C0%7C0%7C636879872385158085&sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2 Fekk35bnYGvmEFJyCXaLDyLm9I%3D&reserved=0 . In particular it provides an equivalent of `@functools.wraps` that is truly signature-preserving
Tell us more about that please. I'm very interested in getting decorators preserve the original signature.
-- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.pyth... Code of Conduct: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org...
______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
Thanks David,
I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers.
Nice ! I did not realize ; good job here, congrats! ;) -- I carefully read the documentation you pointed at, https://wrapt.readthedocs.io/en/latest/decorators.html#decorators-with-optio... This is the example shown by the author: def with_optional_arguments(wrapped=None, myarg1=1, myarg2=2): if wrapped is None: return functools.partial(with_optional_arguments, myarg1=myarg1, myarg2=myarg2) @wrapt.decorator def wrapper(wrapped, instance, args, kwargs): return wrapped(*args, **kwargs) return wrapper(wrapped) As you can see: * the developer has to explicitly handle the no-parenthesis case (the first two lines of code). * And in the next lines of the doc you see his recommendations “For this to be used in this way, it is a requirement that the decorator arguments be supplied as keyword arguments. If using Python 3, the requirement to use keyword only arguments can again be enforced using the keyword only argument syntax.” * Finally, but this is just a comment: this is not “flat” mode but nested mode (the code returns a decorator that returns a function wrapper) So if I’m not misleading, the problem is not really solved. Or at least, not the way I would like the problem to be solved : it is solved here (a) only if the developer takes extra care and (b) reduces the way the decorator can be used (no positional args). This is precisely because I was frustrated by all these limitations that depend on the desired signature that I wrote decopatch. As a developer I do not want to care about which trick to use in which situation (mandatory args, optional args, var-positional args..). My decorators may change signature during the development cycle, and if I frequently had to change trick during development as I changed the signature - that is a bit tiring. -- Concerning creation of signature-preserving wrappers: @wrapt.decorator is not signature preserving, I just checked it. You can check it with the following experiment: def dummy(wrapped): @wrapt.decorator def wrapper(wrapped, instance, args, kwargs): print("wrapper called") return wrapped(*args, **kwargs) return wrapper(wrapped) @dummy def function(a, b): pass If you call function(1) you will see that “wrapper called” is displayed before the TypeError is raised… The signature-preserving equivalent of @wrapt.decorator, @decorator.decorator, is the source of inspiration for makefun. You can see `makefun` as a generalization of the core of `decorator`. --
I'm not sure I ever want them (decopatch and makefun) independently in practice
I totally understand. But some projects actually need makefun and not decopatch because their need is different: they just want to create a function dynamically. This is low-level tooling, really. So at least now there is a clear separation of concerns (and dedicated issues management/roadmap, which is also quite convenient. Not to mention readability !). To cover your concern: decopatch depends on makefun, so both come at the same time when you install decopatch, and decopatch by default relies on makefun when you use it in “double-flat” mode to create wrappers as explained here https://smarie.github.io/python-decopatch/#even-simpler -- Thanks again for this discussion! It is challenging but it is necessary, to make sure I did not answer a non-existent need ;) Kind regards -- Sylvain De : David Mertz <mertz@gnosis.cx> Envoyé : mardi 12 mars 2019 15:30 À : Sylvain MARIE <sylvain.marie@se.com> Cc : Steven D'Aprano <steve@pearwood.info>; python-ideas <python-ideas@python.org> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators [External email: Use caution with links and attachments] ________________________________ The documentation for wrapt mentions: Decorators With Optional Arguments Although opinion can be mixed about whether the pattern is a good one, if the decorator arguments all have default values, it is also possible to implement decorators which have optional arguments. As Graham hints in his docs, I think repurposing decorator factories as decorators is an antipattern. Explicit is better than implicit. While I *do* understands that what decotools and makefun do are technically independent, I'm not sure I ever want them independently in practice. I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers. On Tue, Mar 12, 2019, 10:18 AM David Mertz <mertz@gnosis.cx<mailto:mertz@gnosis.cx>> wrote: The wrapt module I linked to (not funtools.wraps) provides all the capabilities you mention since 2013. It allows mixed use of decorators as decorator factories. It has a flat style. There are some minor API difference between your libraries and wrapt, but the concept is very similar. Since yours is something new, I imagine you perceive some win over what wrapt does. On Tue, Mar 12, 2019, 9:52 AM Sylvain MARIE <sylvain.marie@se.com<mailto:sylvain.marie@se.com>> wrote: David, Steven, Thanks for your interest ! As you probably know, decorators and function wrappers are *completely different concepts*. A decorator can directly return the decorated function (or class), it does not have to return a wrapper. Even more, it can entirely replace the decorated item with something else (not even a function or class!). Try it: it is possible to write a decorator to replace a function with an integer, even though it is probably not quite useful :) `decopatch` helps you write decorators, whatever they are. It "just" solves the annoying issue of having to handle the no-parenthesis and with-parenthesis calls. In addition as a 'goodie', it proposes two development styles: *nested* (you have to return a function) and *flat* (you directly write what will happen when the decorator is applied to something). -- Now about creating signature-preserving function wrappers (in a decorator, or outside a decorator - again, that's not related). That use case is supposed to be covered by functools.wrapt. Unfortunately as explained here https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/55102697#55102697<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F308999%2Fwhat-does-functools-wraps-do%2F55102697%2355102697&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186393926&sdata=Dmkd0Ld1HsuCJtJDCS6GDUgwH3GO66zwYMpZ9Ow%2B18k%3D&reserved=0> this is not the case because with functools.wrapt: - the wrapper code will execute even when the provided arguments are invalid. - the wrapper code cannot easily access an argument using its name, from the received *args, **kwargs. Indeed one would have to handle all cases (positional, keyword, default) and therefore to use something like Signature.bind(). For this reason I proposed a replacement in `makefun`: https://smarie.github.io/python-makefun/#signature-preserving-function-wrappers<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmarie.github.io%2Fpython-makefun%2F%23signature-preserving-function-wrappers&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186403936&sdata=SO319cmnKXiUK6Zek%2FSioHIhFfnQCZRpAF3kim84mv8%3D&reserved=0> -- Now bridging the gap. Of course a very interesting use cases for decorators is to create decorators that create a signature-preserving wrapper. It is possible to combine decopatch and makefun for this: https://smarie.github.io/python-decopatch/#3-creating-function-wrappers<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmarie.github.io%2Fpython-decopatch%2F%233-creating-function-wrappers&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186403936&sdata=FpCGLYwJHFJMWmStJhI%2F7NzXxmJPwC3hf9afg2zfXik%3D&reserved=0> . Decopatch even proposes a "double-flat" development style where you directly write the wrapper body, as explained in the doc. Did I answer your questions ? Thanks again for the quick feedback ! Best, Sylvain -----Message d'origine----- De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org<mailto:se.com@python.org>> De la part de Steven D'Aprano Envoyé : mardi 12 mars 2019 12:30 À : python-ideas@python.org<mailto:python-ideas@python.org> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators [External email: Use caution with links and attachments] ________________________________ On Tue, Mar 12, 2019 at 09:36:41AM +0000, Sylvain MARIE via Python-ideas wrote:
I therefore proposed https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma rie.github.io<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Frie.github.io&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186413941&sdata=8QNKZapq0Z9VQtREcqt4f8EcKbTteHWqiAhD1kVnDyc%3D&reserved=0>%2Fpython-makefun%2F&data=02%7C01%7Csylvain.marie%40s e.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fe.com&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186413941&sdata=73ekIlXJ7nwxcsHYz91JotQ%2F0eGH7h6YC2e3U5pqXow%3D&reserved=0>%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae 68fef%7C0%7C0%7C636879872385158085&sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2 Fekk35bnYGvmEFJyCXaLDyLm9I%3D&reserved=0 . In particular it provides an equivalent of `@functools.wraps` that is truly signature-preserving
Tell us more about that please. I'm very interested in getting decorators preserve the original signature. -- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org<mailto:Python-ideas@python.org> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas&data=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085&sdata=XcYfEginmDF7kIpGGA0XxDZKpUn9e4p2zPFk7UAruYg%3D&reserved=0<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186423950&sdata=X1o6EwD3M3jw6n%2Fqrsx0IFoBK8N2nsoSq4nKDgcmAZQ%3D&reserved=0> Code of Conduct: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F&data=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085&sdata=20ZrtVQZbpQ54c96veSXIOfEK7rKy0ggj0omTZg3ri8%3D&reserved=0<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186423950&sdata=gatcCdLn8s9Zhh26txhKRsqHIYkKpEghOTNmVJ%2BDZJw%3D&reserved=0> ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
One of the nice things in wrapt is that Dumpleton lets you use the same decorator for functions, regular methods, static methods, and class methods. Does yours handle that sort of "polymorphism"? FWIW, thanks for the cool work with your libraries! I don't think I will want the specific with-or-without parens feature, since it feels too implicit. Typing `@deco_factory()` really isn't too much work for me to use the two characters extra. But given that I feel the option is an antipattern, I don't want to add core language features to make the pattern easier. Both you and Graham Dumpleton have found workarounds to get that behavior when it is wanted, but I don't want it to be "too easy." FWIW... I think I'd be tempted to use a metaclass approach so that both the class and instance are callable. The class would be called with a single function argument (i.e. a decorator), but if called with any other signature it would manufacture a callable instance that was parameterized by the initialization arguments (i.e. a decorator factory). Actually, I haven't looked at your actual code, maybe that's what you do. Best, David... On Tue, Mar 12, 2019 at 12:44 PM Sylvain MARIE <sylvain.marie@se.com> wrote:
Thanks David,
I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers.
Nice ! I did not realize ; good job here, congrats! ;)
--
I carefully read the documentation you pointed at, https://wrapt.readthedocs.io/en/latest/decorators.html#decorators-with-optio...
This is the example shown by the author:
def with_optional_arguments(wrapped=None, myarg1=1, myarg2=2):
if wrapped is None:
return functools.partial(with_optional_arguments,
myarg1=myarg1, myarg2=myarg2)
@wrapt.decorator
def wrapper(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)
return wrapper(wrapped)
As you can see:
- the developer has to explicitly handle the no-parenthesis case (the first two lines of code). - And in the next lines of the doc you see his recommendations “For this to be used in this way, it is a requirement that the decorator arguments be supplied as keyword arguments. If using Python 3, the requirement to use keyword only arguments can again be enforced using the keyword only argument syntax.” - Finally, but this is just a comment: this is not “flat” mode but nested mode (the code returns a decorator that returns a function wrapper)
So if I’m not misleading, the problem is not really solved. Or at least, not the way I would like the problem to be solved : it is solved here (a) only if the developer takes extra care and (b) reduces the way the decorator can be used (no positional args). This is precisely because I was frustrated by all these limitations that depend on the desired signature that I wrote decopatch. As a developer I do not want to care about which trick to use in which situation (mandatory args, optional args, var-positional args..). My decorators may change signature during the development cycle, and if I frequently had to change trick during development as I changed the signature - that is a bit tiring.
--
Concerning creation of signature-preserving wrappers: @wrapt.decorator is not signature preserving, I just checked it. You can check it with the following experiment:
*def *dummy(wrapped): @wrapt.decorator *def *wrapper(wrapped, instance, args, kwargs): print(*"wrapper called"*) *return *wrapped(*args, **kwargs) *return *wrapper(wrapped)
@dummy *def *function(a, b): *pass*
If you call
function(1)
you will see that “wrapper called” is displayed before the TypeError is raised…
The signature-preserving equivalent of @wrapt.decorator, @decorator.decorator, is the source of inspiration for makefun. You can see `makefun` as a generalization of the core of `decorator`.
--
I'm not sure I ever want them (decopatch and makefun) independently in practice
I totally understand.
But some projects actually need makefun and not decopatch because their need is different: they just want to create a function dynamically. This is low-level tooling, really.
So at least now there is a clear separation of concerns (and dedicated issues management/roadmap, which is also quite convenient. Not to mention readability !).
To cover your concern: decopatch depends on makefun, so both come at the same time when you install decopatch, and decopatch by default relies on makefun when you use it in “double-flat” mode to create wrappers as explained here https://smarie.github.io/python-decopatch/#even-simpler
--
Thanks again for this discussion! It is challenging but it is necessary, to make sure I did not answer a non-existent need ;)
Kind regards
--
Sylvain
*De :* David Mertz <mertz@gnosis.cx> *Envoyé :* mardi 12 mars 2019 15:30 *À :* Sylvain MARIE <sylvain.marie@se.com> *Cc :* Steven D'Aprano <steve@pearwood.info>; python-ideas < python-ideas@python.org> *Objet :* Re: [Python-ideas] Problems (and solutions?) in writing decorators
[External email: Use caution with links and attachments] ------------------------------
The documentation for wrapt mentions:
Decorators With Optional Arguments
Although opinion can be mixed about whether the pattern is a good one, if the decorator arguments all have default values, it is also possible to implement decorators which have optional arguments.
As Graham hints in his docs, I think repurposing decorator factories as decorators is an antipattern. Explicit is better than implicit.
While I *do* understands that what decotools and makefun do are technically independent, I'm not sure I ever want them independently in practice. I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers.
On Tue, Mar 12, 2019, 10:18 AM David Mertz <mertz@gnosis.cx> wrote:
The wrapt module I linked to (not funtools.wraps) provides all the capabilities you mention since 2013. It allows mixed use of decorators as decorator factories. It has a flat style.
There are some minor API difference between your libraries and wrapt, but the concept is very similar. Since yours is something new, I imagine you perceive some win over what wrapt does.
On Tue, Mar 12, 2019, 9:52 AM Sylvain MARIE <sylvain.marie@se.com> wrote:
David, Steven,
Thanks for your interest !
As you probably know, decorators and function wrappers are *completely different concepts*. A decorator can directly return the decorated function (or class), it does not have to return a wrapper. Even more, it can entirely replace the decorated item with something else (not even a function or class!). Try it: it is possible to write a decorator to replace a function with an integer, even though it is probably not quite useful :)
`decopatch` helps you write decorators, whatever they are. It "just" solves the annoying issue of having to handle the no-parenthesis and with-parenthesis calls. In addition as a 'goodie', it proposes two development styles: *nested* (you have to return a function) and *flat* (you directly write what will happen when the decorator is applied to something). -- Now about creating signature-preserving function wrappers (in a decorator, or outside a decorator - again, that's not related). That use case is supposed to be covered by functools.wrapt. Unfortunately as explained here https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/5510... <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F308999%2Fwhat-does-functools-wraps-do%2F55102697%2355102697&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186393926&sdata=Dmkd0Ld1HsuCJtJDCS6GDUgwH3GO66zwYMpZ9Ow%2B18k%3D&reserved=0> this is not the case because with functools.wrapt: - the wrapper code will execute even when the provided arguments are invalid. - the wrapper code cannot easily access an argument using its name, from the received *args, **kwargs. Indeed one would have to handle all cases (positional, keyword, default) and therefore to use something like Signature.bind().
For this reason I proposed a replacement in `makefun`: https://smarie.github.io/python-makefun/#signature-preserving-function-wrapp... <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmarie.github.io%2Fpython-makefun%2F%23signature-preserving-function-wrappers&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186403936&sdata=SO319cmnKXiUK6Zek%2FSioHIhFfnQCZRpAF3kim84mv8%3D&reserved=0> -- Now bridging the gap. Of course a very interesting use cases for decorators is to create decorators that create a signature-preserving wrapper. It is possible to combine decopatch and makefun for this: https://smarie.github.io/python-decopatch/#3-creating-function-wrappers <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmarie.github.io%2Fpython-decopatch%2F%233-creating-function-wrappers&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186403936&sdata=FpCGLYwJHFJMWmStJhI%2F7NzXxmJPwC3hf9afg2zfXik%3D&reserved=0> . Decopatch even proposes a "double-flat" development style where you directly write the wrapper body, as explained in the doc.
Did I answer your questions ? Thanks again for the quick feedback ! Best,
Sylvain
-----Message d'origine----- De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org> De la part de Steven D'Aprano Envoyé : mardi 12 mars 2019 12:30 À : python-ideas@python.org Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators
[External email: Use caution with links and attachments]
________________________________
On Tue, Mar 12, 2019 at 09:36:41AM +0000, Sylvain MARIE via Python-ideas wrote:
I therefore proposed https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma rie.github.io <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Frie.github.io&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186413941&sdata=8QNKZapq0Z9VQtREcqt4f8EcKbTteHWqiAhD1kVnDyc%3D&reserved=0> %2Fpython-makefun%2F&data=02%7C01%7Csylvain.marie%40s e.com <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fe.com&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186413941&sdata=73ekIlXJ7nwxcsHYz91JotQ%2F0eGH7h6YC2e3U5pqXow%3D&reserved=0> %7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae 68fef%7C0%7C0%7C636879872385158085&sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2 Fekk35bnYGvmEFJyCXaLDyLm9I%3D&reserved=0 . In particular it provides an equivalent of `@functools.wraps` that is truly signature-preserving
Tell us more about that please. I'm very interested in getting decorators preserve the original signature.
-- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.pyth... <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186423950&sdata=X1o6EwD3M3jw6n%2Fqrsx0IFoBK8N2nsoSq4nKDgcmAZQ%3D&reserved=0> Code of Conduct: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org... <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F&data=02%7C01%7Csylvain.marie%40se.com%7Ca7dee43a3bdf494e37c508d6a6f73f7d%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879978186423950&sdata=gatcCdLn8s9Zhh26txhKRsqHIYkKpEghOTNmVJ%2BDZJw%3D&reserved=0>
______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
Thanks David Sorry for not getting back to you earlier, I made a bunch of releases of makefun in the meantime. In particular I fixed a few bugs and added an equivalent of `functools.partial` .
One of the nice things in wrapt is that Dumpleton lets you use the same decorator for functions, regular methods, static methods, and class methods. Does yours handle that sort of "polymorphism"?
It should, but I will check it thoroughly – I’ll let you know
I don't think I will want the specific with-or-without parens feature, since it feels too implicit. Typing `@deco_factory()` really isn't too much work for me to use the two characters extra.
I totally understand your point of view. However on the other hand, many very popular open source projects out there have the opposite point of view and provide decorators that can seamlessly be used with and without arguments (pytest, attrs, click, etc.). So after a while users get used to this behavior and expect it from all libraries. Making it easy to implement is therefore something quite important for developers not to spend time on this useless “feature”. Kind regards Sylvain De : David Mertz <mertz@gnosis.cx> Envoyé : mardi 12 mars 2019 19:15 À : Sylvain MARIE <sylvain.marie@se.com> Cc : Steven D'Aprano <steve@pearwood.info>; python-ideas <python-ideas@python.org> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators [External email: Use caution with links and attachments] ________________________________ One of the nice things in wrapt is that Dumpleton lets you use the same decorator for functions, regular methods, static methods, and class methods. Does yours handle that sort of "polymorphism"? FWIW, thanks for the cool work with your libraries! I don't think I will want the specific with-or-without parens feature, since it feels too implicit. Typing `@deco_factory()` really isn't too much work for me to use the two characters extra. But given that I feel the option is an antipattern, I don't want to add core language features to make the pattern easier. Both you and Graham Dumpleton have found workarounds to get that behavior when it is wanted, but I don't want it to be "too easy." FWIW... I think I'd be tempted to use a metaclass approach so that both the class and instance are callable. The class would be called with a single function argument (i.e. a decorator), but if called with any other signature it would manufacture a callable instance that was parameterized by the initialization arguments (i.e. a decorator factory). Actually, I haven't looked at your actual code, maybe that's what you do. Best, David... On Tue, Mar 12, 2019 at 12:44 PM Sylvain MARIE <sylvain.marie@se.com<mailto:sylvain.marie@se.com>> wrote: Thanks David,
I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers.
Nice ! I did not realize ; good job here, congrats! ;) -- I carefully read the documentation you pointed at, https://wrapt.readthedocs.io/en/latest/decorators.html#decorators-with-optional-arguments<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwrapt.readthedocs.io%2Fen%2Flatest%2Fdecorators.html%23decorators-with-optional-arguments&data=02%7C01%7Csylvain.marie%40se.com%7Ceba5365e816845b8258808d6a716b054%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636880113229744945&sdata=hpuzWdGlzmfJypjI7tIMubrUYPaLN8MoS9BBlMIHGAY%3D&reserved=0> This is the example shown by the author: def with_optional_arguments(wrapped=None, myarg1=1, myarg2=2): if wrapped is None: return functools.partial(with_optional_arguments, myarg1=myarg1, myarg2=myarg2) @wrapt.decorator def wrapper(wrapped, instance, args, kwargs): return wrapped(*args, **kwargs) return wrapper(wrapped) As you can see: * the developer has to explicitly handle the no-parenthesis case (the first two lines of code). * And in the next lines of the doc you see his recommendations “For this to be used in this way, it is a requirement that the decorator arguments be supplied as keyword arguments. If using Python 3, the requirement to use keyword only arguments can again be enforced using the keyword only argument syntax.” * Finally, but this is just a comment: this is not “flat” mode but nested mode (the code returns a decorator that returns a function wrapper) So if I’m not misleading, the problem is not really solved. Or at least, not the way I would like the problem to be solved : it is solved here (a) only if the developer takes extra care and (b) reduces the way the decorator can be used (no positional args). This is precisely because I was frustrated by all these limitations that depend on the desired signature that I wrote decopatch. As a developer I do not want to care about which trick to use in which situation (mandatory args, optional args, var-positional args..). My decorators may change signature during the development cycle, and if I frequently had to change trick during development as I changed the signature - that is a bit tiring. -- Concerning creation of signature-preserving wrappers: @wrapt.decorator is not signature preserving, I just checked it. You can check it with the following experiment: def dummy(wrapped): @wrapt.decorator def wrapper(wrapped, instance, args, kwargs): print("wrapper called") return wrapped(*args, **kwargs) return wrapper(wrapped) @dummy def function(a, b): pass If you call function(1) you will see that “wrapper called” is displayed before the TypeError is raised… The signature-preserving equivalent of @wrapt.decorator, @decorator.decorator, is the source of inspiration for makefun. You can see `makefun` as a generalization of the core of `decorator`. --
I'm not sure I ever want them (decopatch and makefun) independently in practice
I totally understand. But some projects actually need makefun and not decopatch because their need is different: they just want to create a function dynamically. This is low-level tooling, really. So at least now there is a clear separation of concerns (and dedicated issues management/roadmap, which is also quite convenient. Not to mention readability !). To cover your concern: decopatch depends on makefun, so both come at the same time when you install decopatch, and decopatch by default relies on makefun when you use it in “double-flat” mode to create wrappers as explained here https://smarie.github.io/python-decopatch/#even-simpler<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmarie.github.io%2Fpython-decopatch%2F%23even-simpler&data=02%7C01%7Csylvain.marie%40se.com%7Ceba5365e816845b8258808d6a716b054%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636880113229744945&sdata=XkrT0r6C1yxDtX74YxMLBcnwvk4QdGlYCNLBECURnpM%3D&reserved=0> -- Thanks again for this discussion! It is challenging but it is necessary, to make sure I did not answer a non-existent need ;) Kind regards -- Sylvain De : David Mertz <mertz@gnosis.cx<mailto:mertz@gnosis.cx>> Envoyé : mardi 12 mars 2019 15:30 À : Sylvain MARIE <sylvain.marie@se.com<mailto:sylvain.marie@se.com>> Cc : Steven D'Aprano <steve@pearwood.info<mailto:steve@pearwood.info>>; python-ideas <python-ideas@python.org<mailto:python-ideas@python.org>> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators [External email: Use caution with links and attachments] ________________________________ The documentation for wrapt mentions: Decorators With Optional Arguments Although opinion can be mixed about whether the pattern is a good one, if the decorator arguments all have default values, it is also possible to implement decorators which have optional arguments. As Graham hints in his docs, I think repurposing decorator factories as decorators is an antipattern. Explicit is better than implicit. While I *do* understands that what decotools and makefun do are technically independent, I'm not sure I ever want them independently in practice. I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers. On Tue, Mar 12, 2019, 10:18 AM David Mertz <mertz@gnosis.cx<mailto:mertz@gnosis.cx>> wrote: The wrapt module I linked to (not funtools.wraps) provides all the capabilities you mention since 2013. It allows mixed use of decorators as decorator factories. It has a flat style. There are some minor API difference between your libraries and wrapt, but the concept is very similar. Since yours is something new, I imagine you perceive some win over what wrapt does. On Tue, Mar 12, 2019, 9:52 AM Sylvain MARIE <sylvain.marie@se.com<mailto:sylvain.marie@se.com>> wrote: David, Steven, Thanks for your interest ! As you probably know, decorators and function wrappers are *completely different concepts*. A decorator can directly return the decorated function (or class), it does not have to return a wrapper. Even more, it can entirely replace the decorated item with something else (not even a function or class!). Try it: it is possible to write a decorator to replace a function with an integer, even though it is probably not quite useful :) `decopatch` helps you write decorators, whatever they are. It "just" solves the annoying issue of having to handle the no-parenthesis and with-parenthesis calls. In addition as a 'goodie', it proposes two development styles: *nested* (you have to return a function) and *flat* (you directly write what will happen when the decorator is applied to something). -- Now about creating signature-preserving function wrappers (in a decorator, or outside a decorator - again, that's not related). That use case is supposed to be covered by functools.wrapt. Unfortunately as explained here https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/55102697#55102697<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F308999%2Fwhat-does-functools-wraps-do%2F55102697%2355102697&data=02%7C01%7Csylvain.marie%40se.com%7Ceba5365e816845b8258808d6a716b054%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636880113229754958&sdata=KqOxLc6%2FpStoc4sfIV1BqAAutbrMxQXfqmCb3yznlY8%3D&reserved=0> this is not the case because with functools.wrapt: - the wrapper code will execute even when the provided arguments are invalid. - the wrapper code cannot easily access an argument using its name, from the received *args, **kwargs. Indeed one would have to handle all cases (positional, keyword, default) and therefore to use something like Signature.bind(). For this reason I proposed a replacement in `makefun`: https://smarie.github.io/python-makefun/#signature-preserving-function-wrappers<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmarie.github.io%2Fpython-makefun%2F%23signature-preserving-function-wrappers&data=02%7C01%7Csylvain.marie%40se.com%7Ceba5365e816845b8258808d6a716b054%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636880113229754958&sdata=5yNDNnhCjpTctqX951So2cnR2YqzxTiqp%2F%2BBj7XbibE%3D&reserved=0> -- Now bridging the gap. Of course a very interesting use cases for decorators is to create decorators that create a signature-preserving wrapper. It is possible to combine decopatch and makefun for this: https://smarie.github.io/python-decopatch/#3-creating-function-wrappers<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsmarie.github.io%2Fpython-decopatch%2F%233-creating-function-wrappers&data=02%7C01%7Csylvain.marie%40se.com%7Ceba5365e816845b8258808d6a716b054%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636880113229764963&sdata=4n3fuJLLekEOwdmddmMjP00XocHei4dWqV5%2FJosoTR0%3D&reserved=0> . Decopatch even proposes a "double-flat" development style where you directly write the wrapper body, as explained in the doc. Did I answer your questions ? Thanks again for the quick feedback ! Best, Sylvain -----Message d'origine----- De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org<mailto:se.com@python.org>> De la part de Steven D'Aprano Envoyé : mardi 12 mars 2019 12:30 À : python-ideas@python.org<mailto:python-ideas@python.org> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators [External email: Use caution with links and attachments] ________________________________ On Tue, Mar 12, 2019 at 09:36:41AM +0000, Sylvain MARIE via Python-ideas wrote:
I therefore proposed https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma rie.github.io<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Frie.github.io&data=02%7C01%7Csylvain.marie%40se.com%7Ceba5365e816845b8258808d6a716b054%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636880113229774972&sdata=AaBx8DaTf0SJZT6msZo8RlHnkxGHV0JaSBJ2DNXa3IA%3D&reserved=0>%2Fpython-makefun%2F&data=02%7C01%7Csylvain.marie%40s e.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fe.com&data=02%7C01%7Csylvain.marie%40se.com%7Ceba5365e816845b8258808d6a716b054%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636880113229774972&sdata=o4qqJGFIUrLp3433N%2FIiUethE9Rtd91GiRZXOvYfxHQ%3D&reserved=0>%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae 68fef%7C0%7C0%7C636879872385158085&sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2 Fekk35bnYGvmEFJyCXaLDyLm9I%3D&reserved=0 . In particular it provides an equivalent of `@functools.wraps` that is truly signature-preserving
Tell us more about that please. I'm very interested in getting decorators preserve the original signature. -- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org<mailto:Python-ideas@python.org> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas&data=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085&sdata=XcYfEginmDF7kIpGGA0XxDZKpUn9e4p2zPFk7UAruYg%3D&reserved=0<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas&data=02%7C01%7Csylvain.marie%40se.com%7Ceba5365e816845b8258808d6a716b054%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636880113229784972&sdata=o9KkGrtGOIkL%2BhMYU169rlUoTeQRrFvetVm6dF2ejys%3D&reserved=0> Code of Conduct: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F&data=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085&sdata=20ZrtVQZbpQ54c96veSXIOfEK7rKy0ggj0omTZg3ri8%3D&reserved=0<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F&data=02%7C01%7Csylvain.marie%40se.com%7Ceba5365e816845b8258808d6a716b054%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636880113229784972&sdata=PSas3%2FPwjJetvYbwe8vfo4PdpXHV74Nx6VTQHBZF91I%3D&reserved=0> ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________ -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
Sylvain MARIE via Python-ideas writes:
I totally understand your point of view. However on the other hand, many very popular open source projects out there have the opposite point of view and provide decorators that can seamlessly be used with and without arguments (pytest, attrs, click, etc.). So after a while users get used to this behavior and expect it from all libraries. Making it easy to implement is therefore something quite important for developers not to spend time on this useless “feature”.
That doesn't follow. You can also take it that "educating users to know the difference between a decorator and a decorator factory is therefore something quite important for developers not to spend time on this useless 'feature'." I'm not a fan of either position. I don't see why developers of libraries who want to provide this to their users shouldn't have "an easy way to do it", but I also don't see a good reason to encourage syntactic ambiguity by providing it in the standard library. I think this is a feature that belongs in the area of "you *could* do it, but *should* you?" If the answer is "maybe", IMO PyPI is the right solution for distribution. Steve -- Associate Professor Division of Policy and Planning Science http://turnbull.sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull@sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN
Stephen
If the answer is "maybe", IMO PyPI is the right solution for distribution.
Very wise words, I understand this point. However as of today it is *not* possible to write such a library in a complete way, without an additional tool from the language itself. Trust me, I tried very hard :). Indeed `my_decorator(foo)` when foo is a callable will always look like `@my_decorator` applied to function foo, because that's how the language is designed. So there is always one remaining ambiguous case to cover, and I have to rely on some ugly tricks such as asking users for a custom disambiguator. So if the decision is to let community-provided libraries like `decopatch` solve this problem, would you please consider providing us with the missing information? For example a new method in the `inspect` package could be `inspect.is_decorator_call(frame)`, that would return True if and only if the given frame is a decorator usage call as in `@my_decorator`. That piece would be enough - I would gladly take care of the rest in `decopatch`. Thanks for the feedback ! Sylvain -----Message d'origine----- De : Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> Envoyé : vendredi 15 mars 2019 04:56 À : Sylvain MARIE <sylvain.marie@se.com> Cc : David Mertz <mertz@gnosis.cx>; python-ideas <python-ideas@python.org> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators [External email: Use caution with links and attachments] ________________________________ Sylvain MARIE via Python-ideas writes:
I totally understand your point of view. However on the other hand, > many very popular open source projects out there have the opposite > point of view and provide decorators that can seamlessly be used > with and without arguments (pytest, attrs, click, etc.). So after a > while users get used to this behavior and expect it from all > libraries. Making it easy to implement is therefore something quite > important for developers not to spend time on this useless > “feature”.
That doesn't follow. You can also take it that "educating users to know the difference between a decorator and a decorator factory is therefore something quite important for developers not to spend time on this useless 'feature'." I'm not a fan of either position. I don't see why developers of libraries who want to provide this to their users shouldn't have "an easy way to do it", but I also don't see a good reason to encourage syntactic ambiguity by providing it in the standard library. I think this is a feature that belongs in the area of "you *could* do it, but *should* you?" If the answer is "maybe", IMO PyPI is the right solution for distribution. Steve -- Associate Professor Division of Policy and Planning Science https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fturnbull.sk... Faculty of Systems and Information Email: turnbull@sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
On Wed, Mar 20, 2019 at 12:37 AM Sylvain MARIE via Python-ideas <python-ideas@python.org> wrote:
Stephen
If the answer is "maybe", IMO PyPI is the right solution for distribution.
Very wise words, I understand this point. However as of today it is *not* possible to write such a library in a complete way, without an additional tool from the language itself. Trust me, I tried very hard :). Indeed `my_decorator(foo)` when foo is a callable will always look like `@my_decorator` applied to function foo, because that's how the language is designed. So there is always one remaining ambiguous case to cover, and I have to rely on some ugly tricks such as asking users for a custom disambiguator.
Fair point. Though the custom disambiguator could be as simple as using a keyword argument - "my_decorator(x=foo)" is not going to look like "@my_decorator \n def foo".
Sylvain MARIE via Python-ideas wrote:
`my_decorator(foo)` when foo is a callable will always look like `@my_decorator` applied to function foo, because that's how the language is designed.
I don't think it's worth doing anything to change that. Everywhere else in the language, there's a very clear distinction between 'foo' and 'foo()', and you confuse them at your peril. I don't see why decorators should be any different. -- Greg
Also: @something def fun(): ... Is exactly the same as: def fun() ... fun = something(fun) So you can’t make a distinction based whether a given usage is as a decoration. -CHB On Tue, Mar 19, 2019 at 12:26 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Sylvain MARIE via Python-ideas wrote:
`my_decorator(foo)` when foo is a callable will always look like `@my_decorator` applied to function foo, because that's how the language is designed.
I don't think it's worth doing anything to change that. Everywhere else in the language, there's a very clear distinction between 'foo' and 'foo()', and you confuse them at your peril. I don't see why decorators should be any different.
-- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
All of your answers are true,
(Chris) "my_decorator(x=foo)" is not going to look like "@my_decorator \n def foo".
That’s one of the many ways that `decopatch` uses to perform the disambiguation, indeed. But that’s already the user helping the lib, not the other way round
(Christopher) @something applied on def fun is exactly the same as something(fun)
True. However applying decorator manually is already for advanced users, so users of a decopatche-d decorator will not mind calling something()(fun). In fact it is consistent with when they use it with arguments : something(arg)(fun). Another criterion is : how easy would it be to implement an inspect.is_decorator_call(frame) method returning True if and only if frame is the @ statement ? If that’s fairly easy, well, I’m pretty sure that this is good stuff. From a naïve user, not accustomed with the long history of this language, is very strange that an operator such as @ (the decorator one, not the other one) is completely not detectable by code, while there are so many hooks available in python for all other operators (+, -, etc.). Eventually that’s obviously your call, I’m just there to give feedback from what I see of the python libs development community. -- Sylvain De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org> De la part de Christopher Barker Envoyé : mercredi 20 mars 2019 06:50 À : Greg Ewing <greg.ewing@canterbury.ac.nz> Cc : python-ideas <python-ideas@python.org> Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators [External email: Use caution with links and attachments] ________________________________ Also: @something def fun(): ... Is exactly the same as: def fun() ... fun = something(fun) So you can’t make a distinction based whether a given usage is as a decoration. -CHB On Tue, Mar 19, 2019 at 12:26 PM Greg Ewing <greg.ewing@canterbury.ac.nz<mailto:greg.ewing@canterbury.ac.nz>> wrote: Sylvain MARIE via Python-ideas wrote:
`my_decorator(foo)` when foo is a callable will always look like `@my_decorator` applied to function foo, because that's how the language is designed.
I don't think it's worth doing anything to change that. Everywhere else in the language, there's a very clear distinction between 'foo' and 'foo()', and you confuse them at your peril. I don't see why decorators should be any different. -- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org<mailto:Python-ideas@python.org> https://mail.python.org/mailman/listinfo/python-ideas<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas&data=02%7C01%7Csylvain.marie%40se.com%7C2e9308c9904c40f3702408d6acf7fc3a%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636886578423739625&sdata=%2BSHOm4PYwt6p%2F6Of69z0o19sH%2FimE4YsUFFtaEMlYCc%3D&reserved=0> Code of Conduct: http://python.org/psf/codeofconduct/<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F&data=02%7C01%7Csylvain.marie%40se.com%7C2e9308c9904c40f3702408d6acf7fc3a%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636886578423739625&sdata=uO%2Bq8AahHYhCVObE%2Fq3Tn%2BODC0vsdjWfX2niuuNgemE%3D&reserved=0> -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________
On Thu, Oct 25, 2018 at 08:44:44AM -0400, Calvin Spealman wrote:
On Wed, Oct 24, 2018 at 4:41 PM Steven D'Aprano <steve@pearwood.info> wrote: [...]
I *think* you are proposing the following syntax. Am I right?
return def (func): # body of func
which is equivalent to:
def func: # body of func return func [...] Aside from saving one line, what is the purpose of this?
The point is not saving a line or typing, but saving a thought. Expressing the intent of the factory function more clearly.
I don't find that it expresses the intent clearly. Perhaps because I have read, and written, many factory functions that post-process the inner function in some fashion, to me creating the inner function is just one step out of possibly many steps, no more special than the others and not deserving of any special syntax. To give a toy example, adding an extra attribute to the function: def factory(argument): def inner(): ... inner.spam = "spam" return inner Another common pattern for me is changing the name of the inner function. Most of the time wraps() does this, but if the factory doesn't wrap another function, I frequently change the inner function name manually: inner.__name__ = some_name() or "default" Admittedly even for me, the pattern of returning the function immediately after it is created (usually after being decorated by functools.wraps) is very *common*, but I don't find it *special* enough to justify dedicated syntax.
Decorators don't do more than "saving one line", either.
I didn't say they did.
But the biggest reason I'd like something like this is that it solves a *specific* version of the multi-line anonymous function that comes up over and over and over again, and maybe by chipping away at those use-cases we can stop seeing *that* debate over and over and over again. :-)
I don't see the connection. "def" will still be a statement, not an expression, so you can't put it in expressions with or without a leading "return" statement: x = [spam, eggs, return def func(): block, aardvark] nor can you put it in a lambda. So I don't see why this would help with the anonymous code block requests.
So, no, it doesn't save a lot of typing, but I'm never interested in that. I don't spend a lot of time typing code, I spend it reading code, and something like this would certainly help there, imho.
I don't think it will "certainly" help. I think you're projecting your own opinion onto others. I don't find it very readable, and neither have some others who commented. When I read a function def, I read it as short for "define", an imperative command. "return def(ine)" doesn't read like English to me. I also see return, expect an expression, and instead find a statement, so I find it jarring. If *everything was an expression* in Python, perhaps I would agree with you, but then we wouldn't need special syntax because we could already write "return def ...". But given that we do have the statement vs expression dichotomy in Python, making def a special case but only in return statements causes an exception in my brain when I read it :-) -- Steve
participants (14)
-
Anders Hovmöller -
Benedikt Werner -
Calvin Spealman -
Chris Angelico -
Christopher Barker -
David Mertz -
Greg Ewing -
Guido van Rossum -
Jonathan Fine -
Rhodri James -
Stephen J. Turnbull -
Steven D'Aprano -
Sylvain MARIE -
Virendra Tripathi