[Python-ideas] Function composition (was no subject)

[Andrew Barnert]

On May 9, 2015, at 01:36, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> 
> Andrew Barnert writes:
>>> On May 8, 2015, at 19:58, Stephen J. Turnbull <stephen at xemacs.org> wrote:
>>> 
>>> Koos Zevenhoven writes:
>>> 
>>>> As a random example, (root @ mean @ square)(x) would produce the right 
>>>> order for rms when using [2].
>>> 
>>> Hardly interesting. :-)  The result is an exception, as root and square
>>> are conceptually scalar-to-scalar, while mean is sequence-to-scalar.
>> 
>> Unless you're using an elementwise square and an array-to-scalar
>> mean, like the ones in NumPy,
> 
> Erm, why would square be elementwise and root not?  I would suppose
> that everything is element-wise in Numpy (not a user yet).

Most functions in NumPy are elementwise when applied to arrays, but can also be applied to scalars. So, square is elementwise because it's called on an array, root is scalar because it's called on a scalar. (In fact, root could also be elementwise--aggregating functions like mean can be applied across just one axis of a 2D or higher array, reducing it by one dimension, if you want.)

Before you try it, this sounds like a complicated nightmare that can't possibly work in practice. But play with it for just a few minutes and it's completely natural. (Except for a few cases where you want some array-wide but not element-wise operation, most famously matrix multiplication, which is why we now have the @ operator to play with.)

>> in which case it works perfectly well...
> 
> But that's an aspect of my point (evidently, obscure).  Conceptually,
> as taught in junior high school or so, root and square are scalar-to-
> scalar.  If you are working in a context such as Numpy where it makes
> sense to assume they are element-wise and thus composable, the context
> should provide the compose operator(s).  

I was actually thinking on these lines: what if @ didn't work on types.FunctionType, but did work on numpy.ufunc (the name for the "universal function" type that knows how to broadcast across arrays but also work on scalars)? That's something NumPy could implement without any help from the core language. (Methods are a minor problem here, but it's obvious how to solve them, so I won't get into it.) And if it turned out to be useful all over the place in NumPy, that might turn up some great uses for the idiomatic non-NumPy Python, or it might show that, like elementwise addition, it's really more a part of NumPy than of Python.

But of course that's more of a proposal for NumPy than for Python.

> Without that context, Koos's
> example looks like a TypeError.

>> But Koos's example, even if it was possibly inadvertent, shows that
>> I may be wrong about that. Maybe compose together with element-wise
>> operators actually _is_ sufficient for something beyond toy
>> examples.
> 
> Of course it is!<wink />  I didn't really think there was any doubt
> about that.  

I think there was, and still is. People keep coming up with abstract toy examples, but as soon as someone tries to give a good real example, it only makes sense with NumPy (Koos's) or with some syntax that Python doesn't have (yours), because to write them with actual Python functions would actually be ugly and verbose (my version of yours). 

I don't think that's a coincidence. You didn't write "map square" because you don't know how to think in Python, but because using compose profitably inherently implies not thinking in Python. (Except, maybe, in the case of NumPy... which is a different idiom.) Maybe someone has a bunch of obvious good use cases for compose that don't also require other functions, operators, or syntax we don't have, but so far, nobody's mentioned one.