[Python-ideas] Add 'composable' decorator to functools (with @ matrix multiplication)

[Erik Bray]

On Wed, May 6, 2015 at 11:48 AM, Guido van Rossum <guido at python.org> wrote:
> I realize this is still python-ideas, but does this really leave functions
> with multiple arguments completely out of the picture (except as the first
> stage in the pipeline)?

I'm not sure exactly what this is in response to, but in the case of
astropy.modeling, any function
at any point in the chain can take multiple arguments, as long as the
function its is composed with
returns the right number of outputs (as a tuple).

There is also a "Mapping" object that allows remapping arguments so if
for example you want to
swap the return values of one function before passing them as inputs
to the next function.  You can
also duplicate outputs, drop outputs, inline new ones, etc.  It can
get quite arbitrarily complex, and there
aren't enough facilities in place yet to visualize complicated
function compositions as I would like.  But
this is already being put to good use.

Nothing about this is particular to astropy.modeling--the same
approach could be used in a generic function
composition operator.

Erik

> On Wed, May 6, 2015 at 8:42 AM, Erik Bray <erik.m.bray at gmail.com> wrote:
>>
>> On Wed, May 6, 2015 at 11:21 AM, Ivan Levkivskyi <levkivskyi at gmail.com>
>> wrote:
>> > Dear Erik,
>> >
>> > Thank you for the link! I agree that this idea is too raw for stdlib
>> > (there
>> > are problems with many argument functions, keyword arguments, etc.)
>> > Concerning the shell | vs. matrix @ I think it is a good idea to have
>> > both... but with different order.
>> > I mean in shell logic f | g means g (f (x)), while for matrix
>> > multiplication
>> > f @ g means f(g(x)).
>> > The former is probably more natural for people with more "programming"
>> > background, while the latter is more natural for people with a
>> > "scientific"
>> > background.
>> > We could now do good for both, since we now have a new operator.
>>
>> Absolutely!  I've found that it takes a little work sometimes for
>> scientific users to wrap
>> their heads around the
>>
>> g | f
>>
>> syntax.  Once Python 3.5 is out I might add support for "f @ g" as
>> well, though I'm wary
>> of having more than one way to do it.  Worth trying out though, so
>> thanks for the idea.
>>
>> Erik
>>
>> > On 6 May 2015 at 16:10, Erik Bray <erik.m.bray at gmail.com> wrote:
>> >>
>> >> On Wed, May 6, 2015 at 9:20 AM, Ivan Levkivskyi <levkivskyi at gmail.com>
>> >> wrote:
>> >> > Dear all,
>> >> >
>> >> > The matrix multiplication operator @ is going to be introduced in
>> >> > Python
>> >> > 3.5
>> >> > and I am thinking about the following idea:
>> >> >
>> >> > The semantics of matrix multiplication is the composition of the
>> >> > corresponding linear transformations.
>> >> > A linear transformation is a particular example of a more general
>> >> > concept -
>> >> > functions.
>> >> > The latter are frequently composed with ("wrap") each other. For
>> >> > example:
>> >> >
>> >> > plot(real(sqrt(data)))
>> >> >
>> >> > However, it is not very readable in case of many wrapping layers.
>> >> > Therefore,
>> >> > it could be useful to employ
>> >> > the matrix multiplication operator @ for indication of function
>> >> > composition.
>> >> > This could be done by such (simplified) decorator:
>> >> >
>> >> > class composable:
>> >> >
>> >> >     def __init__(self, func):
>> >> >         self.func = func
>> >> >
>> >> >     def __call__(self, arg):
>> >> >         return self.func(arg)
>> >> >
>> >> >     def __matmul__(self, other):
>> >> >         def composition(*args, **kwargs):
>> >> >             return self.func(other(*args, **kwargs))
>> >> >         return composable(composition)
>> >> >
>> >> > I think using such decorator with functions that are going to be
>> >> > deeply
>> >> > wrapped
>> >> > could improve readability.
>> >> > You could compare (note that only the outermost function should be
>> >> > decorated):
>> >> >
>> >> > plot(sorted(sqrt(real(data_array)))) vs. (plot @ sorted @ sqrt @
>> >> > real)
>> >> > (data_array)
>> >> >
>> >> > I think the latter is more readable, also compare
>> >> >
>> >> > def sunique(lst):
>> >> >     return sorted(list(set(lst)))
>> >> >
>> >> > vs.
>> >> >
>> >> > sunique = sorted @ list @ set
>> >> >
>> >> > Apart from readability, there are following pros of the proposed
>> >> > decorator:
>> >> >
>> >> > 1. Similar semantics as for matrix multiplication.
>> >> > 2. Same symbol for composition as for decorators.
>> >> > 3. The symbol @ resembles mathematical notation for function
>> >> > composition: ∘
>> >> >
>> >> > I think it could be a good idea to add such a decorator to the stdlib
>> >> > functools module.
>> >>
>> >> In the astropy.modeling package, which consists largely of collection
>> >> of fancy wrappers around analytic functions,
>> >> we used the pipe operator | (that is, __or__) to implement function
>> >> composition, as demonstrated here:
>> >>
>> >>
>> >>
>> >> http://docs.astropy.org/en/stable/modeling/compound-models.html#model-composition
>> >>
>> >> I do like the idea of using the new @ operator for this purpose--it
>> >> makes sense as a generalization of linear operators,
>> >> and it just looks a little more like the circle operator often used
>> >> for functional composition.  On the other hand
>> >> I'm also fond of the choice to use |, for the similarity to UNIX shell
>> >> pipe operations, as long as it can't be confused with
>> >> __or__.  Point being something like this could be implemented now with
>> >> __or__.
>> >>
>> >> I think this is simple enough that it doesn't need to be in the
>> >> stdlib, especially if there are different ways people
>> >> would like to do this.  But I do like the idea.
>> >>
>> >> Erik
>> >
>> >
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)