One of the nice things in wrapt is that Dumpleton lets you use the same decorator for functions, regular methods, static methods, and class methods.  Does yours handle that sort of "polymorphism"?

FWIW, thanks for the cool work with your libraries! 

I don't think I will want the specific with-or-without parens feature, since it feels too implicit.  Typing `@deco_factory()` really isn't too much work for me to use the two characters extra.  But given that I feel the option is an antipattern, I don't want to add core language features to make the pattern easier.  Both you and Graham Dumpleton have found workarounds to get that behavior when it is wanted, but I don't want it to be "too easy."

FWIW... I think I'd be tempted to use a metaclass approach so that both the class and instance are callable.  The class would be called with a single function argument (i.e. a decorator), but if called with any other signature it would manufacture a callable instance that was parameterized by the initialization arguments (i.e. a decorator factory).  Actually, I haven't looked at your actual code, maybe that's what you do.

Best, David...

On Tue, Mar 12, 2019 at 12:44 PM Sylvain MARIE <sylvain.marie@se.com> wrote:

Thanks David,

 

> I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers.

 

Nice ! I did not realize ; good job here, congrats! ;)

 

--

I carefully read the documentation you pointed at, https://wrapt.readthedocs.io/en/latest/decorators.html#decorators-with-optional-arguments

This is the example shown by the author:

 

def with_optional_arguments(wrapped=None, myarg1=1, myarg2=2):

    if wrapped is None:

        return functools.partial(with_optional_arguments,

                myarg1=myarg1, myarg2=myarg2)

 

    @wrapt.decorator

    def wrapper(wrapped, instance, args, kwargs):

        return wrapped(*args, **kwargs)

 

    return wrapper(wrapped)

 

As you can see:

  • the developer has to explicitly handle the no-parenthesis case (the first two lines of code).
  • And in the next lines of the doc you see his recommendations “For this to be used in this way, it is a requirement that the decorator arguments be supplied as keyword arguments. If using Python 3, the requirement to use keyword only arguments can again be enforced using the keyword only argument syntax.”
  • Finally, but this is just a comment: this is not “flat” mode but nested mode (the code returns a decorator that returns a function wrapper)

 

So if I’m not misleading, the problem is not really solved. Or at least, not the way I would like the problem to be solved : it is solved here (a) only if the developer takes extra care and (b) reduces the way the decorator can be used (no positional args). This is precisely because I was frustrated by all these limitations that depend on the desired signature that I wrote decopatch. As a developer I do not want to care about which trick to use in which situation (mandatory args, optional args, var-positional args..). My decorators may change signature during the development cycle, and if I frequently had to change trick during development as I changed the signature - that is a bit tiring.

 

--

Concerning creation of signature-preserving wrappers: @wrapt.decorator is not signature preserving, I just checked it. You can check it with the following experiment:

 

def dummy(wrapped):
    @wrapt.decorator
    def wrapper(wrapped, instance, args, kwargs):
        print("wrapper called")
        return wrapped(*args, **kwargs)
    return wrapper(wrapped)

 

@dummy
def function(a, b):
    pass

 

If you call

 

function(1)

 

you will see that “wrapper called” is displayed before the TypeError is raised…

 

The signature-preserving equivalent of @wrapt.decorator, @decorator.decorator, is the source of inspiration for makefun. You can see `makefun` as a generalization of the core of `decorator`.

 

--

> I'm not sure I ever want them (decopatch and makefun) independently in practice

 

I totally understand.

But some projects actually need makefun and not decopatch because their need is different: they just want to create a function dynamically. This is low-level tooling, really.

So at least now there is a clear separation of concerns (and dedicated issues management/roadmap, which is also quite convenient. Not to mention readability !).

To cover your concern: decopatch depends on makefun, so both come at the same time when you install decopatch, and decopatch by default relies on makefun when you use it in “double-flat” mode to create wrappers as explained here https://smarie.github.io/python-decopatch/#even-simpler

--

 

Thanks again for this discussion! It is challenging but it is necessary, to make sure I did not answer a non-existent need ;)

Kind regards

 

--

Sylvain

 

De : David Mertz <mertz@gnosis.cx>
Envoyé : mardi 12 mars 2019 15:30
À : Sylvain MARIE <sylvain.marie@se.com>
Cc : Steven D'Aprano <steve@pearwood.info>; python-ideas <python-ideas@python.org>
Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators

 

[External email: Use caution with links and attachments]

 

The documentation for wrapt mentions:

 

Decorators With Optional Arguments

Although opinion can be mixed about whether the pattern is a good one, if the decorator arguments all have default values, it is also possible to implement decorators which have optional arguments. 

As Graham hints in his docs, I think repurposing decorator factories as decorators is an antipattern. Explicit is better than implicit.

 

While I *do* understands that what decotools and makefun do are technically independent, I'm not sure I ever want them independently in practice. I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers.

On Tue, Mar 12, 2019, 10:18 AM David Mertz <mertz@gnosis.cx> wrote:

The wrapt module I linked to (not funtools.wraps) provides all the capabilities you mention since 2013. It allows mixed use of decorators as decorator factories. It has a flat style. 

 

There are some minor API difference between your libraries and wrapt, but the concept is very similar. Since yours is something new, I imagine you perceive some win over what wrapt does.

On Tue, Mar 12, 2019, 9:52 AM Sylvain MARIE <sylvain.marie@se.com> wrote:

David, Steven,

Thanks for your interest !

As you probably know, decorators and function wrappers are *completely different concepts*. A decorator can directly return the decorated function (or class), it does not have to return a wrapper. Even more, it can entirely replace the decorated item with something else (not even a function or class!). Try it: it is possible to write a decorator to replace a function with an integer, even though it is probably not quite useful :)

`decopatch` helps you write decorators, whatever they are. It "just" solves the annoying issue of having to handle the no-parenthesis and with-parenthesis calls. In addition as a 'goodie', it proposes two development styles: *nested* (you have to return a function) and *flat* (you directly write what will happen when the decorator is applied to something).
--
Now about creating signature-preserving function wrappers (in a decorator, or outside a decorator - again, that's not related). That use case is supposed to be covered by functools.wrapt. Unfortunately as explained here https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/55102697#55102697 this is not the case because with functools.wrapt:
 - the wrapper code will execute even when the provided arguments are invalid.
 - the wrapper code cannot easily access an argument using its name, from the received *args, **kwargs. Indeed one would have to handle all cases (positional, keyword, default) and therefore to use something like Signature.bind().

For this reason I proposed a replacement in `makefun`: https://smarie.github.io/python-makefun/#signature-preserving-function-wrappers
--
Now bridging the gap. Of course a very interesting use cases for decorators is to create decorators that create a signature-preserving wrapper. It is possible to combine decopatch and makefun for this: https://smarie.github.io/python-decopatch/#3-creating-function-wrappers .
Decopatch even proposes a "double-flat" development style where you directly write the wrapper body, as explained in the doc.

Did I answer your questions ?
Thanks again for the quick feedback !
Best,

Sylvain

-----Message d'origine-----
De : Python-ideas <python-ideas-bounces+sylvain.marie=se.com@python.org> De la part de Steven D'Aprano
Envoyé : mardi 12 mars 2019 12:30
À : python-ideas@python.org
Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators

[External email: Use caution with links and attachments]

________________________________



On Tue, Mar 12, 2019 at 09:36:41AM +0000, Sylvain MARIE via Python-ideas wrote:

> I therefore proposed
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma
> rie.github.io%2Fpython-makefun%2F&amp;data=02%7C01%7Csylvain.marie%40s
> e.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae
> 68fef%7C0%7C0%7C636879872385158085&amp;sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2
> Fekk35bnYGvmEFJyCXaLDyLm9I%3D&amp;reserved=0 . In particular it
> provides an equivalent of `@functools.wraps` that is truly
> signature-preserving

Tell us more about that please. I'm very interested in getting decorators preserve the original signature.


--
Steven
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas&amp;data=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085&amp;sdata=XcYfEginmDF7kIpGGA0XxDZKpUn9e4p2zPFk7UAruYg%3D&amp;reserved=0
Code of Conduct: https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F&amp;data=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085&amp;sdata=20ZrtVQZbpQ54c96veSXIOfEK7rKy0ggj0omTZg3ri8%3D&amp;reserved=0

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
______________________________________________________________________


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
______________________________________________________________________



--
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.