[Python-ideas] Partial operator (and 'third-party methods' and 'piping') [was Re: Function composition (was no subject)]

[Steven D'Aprano]

On Sun, May 10, 2015 at 11:06:21PM +0300, Koos Zevenhoven wrote:

> So, -> would be an operator with a precedence similar to .attribute 
> access (but lower than .attribute):

Dot . is not an operator. If I remember correctly, the docs describe it 
as a delimiter.

>  # The simple definition of what it does:
>  arg->func   # equivalent to functools.partial(func, arg)

I believe you require that -> is applied before function application, so 

arg->func  # returns partial(func, arg)
arg->func(x)  # returns partial(func, arg)(x)
arg->(func(x))  # returns partial(func(x), arg)

> This would allow for instance:
>  arg -> spam() -> cheese(kind = 'gouda') -> eggs()

I am having a lot of difficulty seeing that as anything other than "call 
spam with no arguments, then apply arg to the result". But, teasing it 
apart with the precedence I established above:

arg->spam()  # returns partial(spam, arg)() == spam(arg)
""" -> cheese  # returns partial(cheese, spam(arg))

""" (kind='gouda')  # returns partial(cheese, spam(arg))(kind='gouda')
                    # == cheese(spam(arg), kind='gouda')

""" -> eggs  # returns partial(eggs, cheese(spam(arg), kind='gouda'))

""" ()  # calls the previous partial, with no arguments, giving:
        # partial(eggs, cheese(spam(arg), kind='gouda'))()
        # == eggs(cheese(spam(arg), kind='gouda'))


> which would be equivalent to eggs(cheese(spam(arg), kind = 'gouda'))

Amazingly, you are correct! :-)

I think this demonstrates an abuse of partial and the sort of thing that 
gives functional idioms a bad name. To tease this apart and understand 
what it does was very difficult to me. And I don't understand the point 
of creating partial applications that you are then immediately going to 
call, that just adds an extra layer of indirection to slow the code 
down. If you write partial(len, 'foo')() instead of just len('foo'), 
something has gone drastically wrong.

So instead of

arg->spam()->cheese(kind='gouda')->eggs()

which includes *three* partial objects which are immediately called, 
wouldn't it be easier to just call the functions in the first place?

eggs(cheese(spam(arg), kind='gouda'))

It will certainly be more efficient!


Let's run through a simple chain with no parens:

a -> b  # partial(b, a)
a -> b -> c  # partial(c, partial(b, a))
a -> b -> c -> d  # partial(d, partial(c, partial(b, a)))

I'm not seeing why I would want to write something like that.


Let's apply multiple arguments:

a -> func  # partial(func, a)
b -> (a -> func)  # partial(partial(func, a), b)
c -> (b -> (a -> func))  # partial(partial(partial(func, a), b), c)

Perhaps a sufficiently clever implementation of partial could optimize 
partial(partial(func, a), b) to just a single layer of indirection 
partial(func, a, b), so it's not *necessarily* as awful as it looks. (I 
would expect a function composition operator to do the same.)

Note that we have to write the second argument first, and bracket the 
second arrow clause. Writing it the "obvious" way is wrong:

a -> b -> func  # partial(func, partial(b, a))


I think this is imaginative but hard to read, hard to understand, 
hard to use correctly, inefficient, and even if used correctly, there 
are not very many times that you would need it.


> Or even together together with the proposed @ composition:
>  rms = root @ mean @ square->map     # for an iterable non-numpy argument

I think that a single arrow may be reasonable as syntactic sugar for 
partial, but once you start chaining them, it all falls apart into a 
mess. That, in my mind, is a sign that the idea doesn't scale. We can 
chain dots with no problem:

fe.fi.fo.fum

and function calls in numerous ways:

foo(bar(baz()))
foo(bar)(baz)

and although they can get hard to read just because of the sheer number 
of components, they are not conceptually difficult. But chaining arrows 
is conceptually difficult even with as few as two arrows.

I think the problem here is that partial application is an N-ary 
operation. This is not Haskell where single-argument currying is 
enforced everywhere! You're trying to perform something which 
conceptually takes N arguments partial(func, 1, 2, 3, ..., N) using only 
a operator which can only take two arguments a->b. Things are going to 
get messy.


> And here's something I find quite interesting. Together with 
> @singledispatch from 3.4 (or possibly an enhanced version using type 
> annotations in the future?), one could add 'third-party methods' to 
> classes in other libraries without monkey patching. A dummy example:
> 
> from numpy import array
> my_list = [1,2,3]
> my_array = array(my_list)
> my_mean = my_array.mean()  # This currently works in numpy
> 
> from rmslib import rms
> my_rms = my_array->rms()  # efficient rms for numpy arrays
> my_other_rms = my_list->rms()  # rms that works on any iterable

That looks cute, but isn't very interesting. Effectively, you've 
invented a new (and less efficient) syntax for calling a function:

spam->eggs(cheese)  # eggs(spam, cheese)

It's less efficient because it builds a partial object first, so instead 
of one call you end up with two, and a temporary object that gets thrown 
away immediately after it is used. Yes, you could keep the partial 
object around, but as your example shows, you don't. And because it is 
cute, people will write:

a->func(), b->func(), c->func()

and not realise that it creates three partial functions before calling 
them. Writing:

func(a), func(b), func(c)

will avoid that needless overhead.



-- 
Steve