Thanks David,

 

> I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers.

 

Nice ! I did not realize ; good job here, congrats! ;)

 

--

I carefully read the documentation you pointed at, https://wrapt.readthedocs.io/en/latest/decorators.html#decorators-with-optional-arguments

This is the example shown by the author:

 

def with_optional_arguments(wrapped=None, myarg1=1, myarg2=2):

    if wrapped is None:

        return functools.partial(with_optional_arguments,

                myarg1=myarg1, myarg2=myarg2)

 

    @wrapt.decorator

    def wrapper(wrapped, instance, args, kwargs):

        return wrapped(*args, **kwargs)

 

    return wrapper(wrapped)

 

As you can see:

 

So if I’m not misleading, the problem is not really solved. Or at least, not the way I would like the problem to be solved : it is solved here (a) only if the developer takes extra care and (b) reduces the way the decorator can be used (no positional args). This is precisely because I was frustrated by all these limitations that depend on the desired signature that I wrote decopatch. As a developer I do not want to care about which trick to use in which situation (mandatory args, optional args, var-positional args..). My decorators may change signature during the development cycle, and if I frequently had to change trick during development as I changed the signature - that is a bit tiring.

 

--

Concerning creation of signature-preserving wrappers: @wrapt.decorator is not signature preserving, I just checked it. You can check it with the following experiment:

 

def dummy(wrapped):
   
@wrapt.decorator
   
def wrapper(wrapped, instance, args, kwargs):
       
print("wrapper called")
       
return wrapped(*args, **kwargs)
   
return wrapper(wrapped)

 

@dummy
def function(a, b):
   
pass

 

If you call

 

function(1)

 

you will see that “wrapper called” is displayed before the TypeError is raised…

 

The signature-preserving equivalent of @wrapt.decorator, @decorator.decorator, is the source of inspiration for makefun. You can see `makefun` as a generalization of the core of `decorator`.

 

--

> I'm not sure I ever want them (decopatch and makefun) independently in practice

 

I totally understand.

But some projects actually need makefun and not decopatch because their need is different: they just want to create a function dynamically. This is low-level tooling, really.

So at least now there is a clear separation of concerns (and dedicated issues management/roadmap, which is also quite convenient. Not to mention readability !).

To cover your concern: decopatch depends on makefun, so both come at the same time when you install decopatch, and decopatch by default relies on makefun when you use it in “double-flat” mode to create wrappers as explained here https://smarie.github.io/python-decopatch/#even-simpler

--

 

Thanks again for this discussion! It is challenging but it is necessary, to make sure I did not answer a non-existent need ;)

Kind regards

 

--

Sylvain

 

De : David Mertz <mertz@gnosis.cx>
Envoyé : mardi 12 mars 2019 15:30
À : Sylvain MARIE <sylvain.marie@se.com>
Cc : Steven D'Aprano <steve@pearwood.info>; python-ideas <python-ideas@python.org>
Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators

 

[External email: Use caution with links and attachments]


 

The documentation for wrapt mentions:

 

Decorators With Optional Arguments

Although opinion can be mixed about whether the pattern is a good one, if the decorator arguments all have default values, it is also possible to implement decorators which have optional arguments. 

As Graham hints in his docs, I think repurposing decorator factories as decorators is an antipattern. Explicit is better than implicit.

 

While I *do* understands that what decotools and makefun do are technically independent, I'm not sure I ever want them independently in practice. I did write the book _Functional Programming in Python_, so I'm not entirely unfamiliar with function wrappers.

On Tue, Mar 12, 2019, 10:18 AM David Mertz <mertz@gnosis.cx> wrote:

The wrapt module I linked to (not funtools.wraps) provides all the capabilities you mention since 2013. It allows mixed use of decorators as decorator factories. It has a flat style. 

 

There are some minor API difference between your libraries and wrapt, but the concept is very similar. Since yours is something new, I imagine you perceive some win over what wrapt does.

On Tue, Mar 12, 2019, 9:52 AM Sylvain MARIE <sylvain.marie@se.com> wrote:

David, Steven,

Thanks for your interest !

As you probably know, decorators and function wrappers are *completely different concepts*. A decorator can directly return the decorated function (or class), it does not have to return a wrapper. Even more, it can entirely replace the decorated item with something else (not even a function or class!). Try it: it is possible to write a decorator to replace a function with an integer, even though it is probably not quite useful :)

`decopatch` helps you write decorators, whatever they are. It "just" solves the annoying issue of having to handle the no-parenthesis and with-parenthesis calls. In addition as a 'goodie', it proposes two development styles: *nested* (you have to return a function) and *flat* (you directly write what will happen when the decorator is applied to something).
--
Now about creating signature-preserving function wrappers (in a decorator, or outside a decorator - again, that's not related). That use case is supposed to be covered by functools.wrapt. Unfortunately as explained here
https://stackoverflow.com/questions/308999/what-does-functools-wraps-do/55102697#55102697 this is not the case because with functools.wrapt:
 - the wrapper code will execute even when the provided arguments are invalid.
 - the wrapper code cannot easily access an argument using its name, from the received *args, **kwargs. Indeed one would have to handle all cases (positional, keyword, default) and therefore to use something like Signature.bind().

For this reason I proposed a replacement in `makefun`:
https://smarie.github.io/python-makefun/#signature-preserving-function-wrappers
--
Now bridging the gap. Of course a very interesting use cases for decorators is to create decorators that create a signature-preserving wrapper. It is possible to combine decopatch and makefun for this:
https://smarie.github.io/python-decopatch/#3-creating-function-wrappers .
Decopatch even proposes a "double-flat" development style where you directly write the wrapper body, as explained in the doc.

Did I answer your questions ?
Thanks again for the quick feedback !
Best,

Sylvain

-----Message d'origine-----
De : Python-ideas <python-ideas-bounces+sylvain.marie=
se.com@python.org> De la part de Steven D'Aprano
Envoyé : mardi 12 mars 2019 12:30
À :
python-ideas@python.org
Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators

[External email: Use caution with links and attachments]

________________________________



On Tue, Mar 12, 2019 at 09:36:41AM +0000, Sylvain MARIE via Python-ideas wrote:

> I therefore proposed
>
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsma
>
rie.github.io%2Fpython-makefun%2F&amp;data=02%7C01%7Csylvain.marie%40s
>
e.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae
> 68fef%7C0%7C0%7C636879872385158085&amp;sdata=nB9p9V%2BJ7gk%2Fsc%2BA5%2
> Fekk35bnYGvmEFJyCXaLDyLm9I%3D&amp;reserved=0 . In particular it
> provides an equivalent of `@functools.wraps` that is truly
> signature-preserving

Tell us more about that please. I'm very interested in getting decorators preserve the original signature.


--
Steven
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-ideas&amp;data=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085&amp;sdata=XcYfEginmDF7kIpGGA0XxDZKpUn9e4p2zPFk7UAruYg%3D&amp;reserved=0
Code of Conduct:
https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpython.org%2Fpsf%2Fcodeofconduct%2F&amp;data=02%7C01%7Csylvain.marie%40se.com%7C579232e7e10e475314c708d6a6de9d23%7C6e51e1adc54b4b39b5980ffe9ae68fef%7C0%7C0%7C636879872385158085&amp;sdata=20ZrtVQZbpQ54c96veSXIOfEK7rKy0ggj0omTZg3ri8%3D&amp;reserved=0

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
______________________________________________________________________


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
______________________________________________________________________