[Python-ideas] Partial operator (and 'third-party methods' and 'piping') [was Re: Function composition (was no subject)]

[Douglas La Rocca]

I agree here--I don't think a special operator for functools.partial is desirable. The proposal seems to suggest something between an ordinary lambda expression and Haskell's (>>=) bind.

I expect the arrow to work as it does in Haskell and julia for anonymous functions

    x, *xs -> <some expr, x and xs in locals()>

Or in (very-)pseudo notation

    (->) argspec expr

Then bind (>>=) also comes to mind because it takes a value on the left and a function on the right. But doesn't have the nice things you get with monads.

These become non-issues if functions either explicitly accept and bind one argument at a time (currying/incremental binding), or if a @curried decorator is used. Or Gregory's @arrow decorator (which I've just now discovered!).

So

    arg -> spam() -> cheese(kind = 'gouda') -> eggs()

would be (with composition) written as

    compose(spam, cheese(kind='gouda'), eggs)(arg)

If you want to wrap `cheese` to avoid the awkwardness, you can do

    >>> cheese_kind = lambda kind: lambda *args, kind=kind, **kwargs: cheese(kind=kind)(*args, **kwargs)
    >>> compose(spam, cheese_kind('gouda'), eggs)(arg)

Then if you don't like two explicit sequential function calls, i.e. f(x)(y), there are ways to sugar it up, like

    def single_value_pipeline(fn):
        def wrapper(x, **kwargs):
            return compose(lambda *_: x, *fn(**kwargs))()
        return wrapper

Which would hide `compose` altogether (very anti-PEP8!) and allow binding keyword names across the pipeline:

    @single_value_pipeline
    def breakfast(cheese_kind='gouda'):
        return (spam, 
                cheese(kind=cheese_kind), 
                eggs)

    breakfast(arg, kind='something other than gouda') # what is gouda anyway?!

(`single_value_pipeline` is perhaps a bad name though...)

________________________________________
From: Python-ideas <python-ideas-bounces+larocca=abiresearch.com at python.org> on behalf of Steven D'Aprano <steve at pearwood.info>
Sent: Sunday, May 10, 2015 9:44 PM
To: python-ideas at python.org
Subject: Re: [Python-ideas] Partial operator (and 'third-party methods' and     'piping') [was Re: Function composition (was no subject)]

On Sun, May 10, 2015 at 11:06:21PM +0300, Koos Zevenhoven wrote:

> So, -> would be an operator with a precedence similar to .attribute
> access (but lower than .attribute):

Dot . is not an operator. If I remember correctly, the docs describe it
as a delimiter.

>  # The simple definition of what it does:
>  arg->func   # equivalent to functools.partial(func, arg)

I believe you require that -> is applied before function application, so

arg->func  # returns partial(func, arg)
arg->func(x)  # returns partial(func, arg)(x)
arg->(func(x))  # returns partial(func(x), arg)

> This would allow for instance:
>  arg -> spam() -> cheese(kind = 'gouda') -> eggs()

I am having a lot of difficulty seeing that as anything other than "call
spam with no arguments, then apply arg to the result". But, teasing it
apart with the precedence I established above:

arg->spam()  # returns partial(spam, arg)() == spam(arg)
""" -> cheese  # returns partial(cheese, spam(arg))

""" (kind='gouda')  # returns partial(cheese, spam(arg))(kind='gouda')
                    # == cheese(spam(arg), kind='gouda')

""" -> eggs  # returns partial(eggs, cheese(spam(arg), kind='gouda'))

""" ()  # calls the previous partial, with no arguments, giving:
        # partial(eggs, cheese(spam(arg), kind='gouda'))()
        # == eggs(cheese(spam(arg), kind='gouda'))


> which would be equivalent to eggs(cheese(spam(arg), kind = 'gouda'))

Amazingly, you are correct! :-)

I think this demonstrates an abuse of partial and the sort of thing that
gives functional idioms a bad name. To tease this apart and understand
what it does was very difficult to me. And I don't understand the point
of creating partial applications that you are then immediately going to
call, that just adds an extra layer of indirection to slow the code
down. If you write partial(len, 'foo')() instead of just len('foo'),
something has gone drastically wrong.

So instead of

arg->spam()->cheese(kind='gouda')->eggs()

which includes *three* partial objects which are immediately called,
wouldn't it be easier to just call the functions in the first place?

eggs(cheese(spam(arg), kind='gouda'))

It will certainly be more efficient!


Let's run through a simple chain with no parens:

a -> b  # partial(b, a)
a -> b -> c  # partial(c, partial(b, a))
a -> b -> c -> d  # partial(d, partial(c, partial(b, a)))

I'm not seeing why I would want to write something like that.


Let's apply multiple arguments:

a -> func  # partial(func, a)
b -> (a -> func)  # partial(partial(func, a), b)
c -> (b -> (a -> func))  # partial(partial(partial(func, a), b), c)

Perhaps a sufficiently clever implementation of partial could optimize
partial(partial(func, a), b) to just a single layer of indirection
partial(func, a, b), so it's not *necessarily* as awful as it looks. (I
would expect a function composition operator to do the same.)

Note that we have to write the second argument first, and bracket the
second arrow clause. Writing it the "obvious" way is wrong:

a -> b -> func  # partial(func, partial(b, a))


I think this is imaginative but hard to read, hard to understand,
hard to use correctly, inefficient, and even if used correctly, there
are not very many times that you would need it.


> Or even together together with the proposed @ composition:
>  rms = root @ mean @ square->map     # for an iterable non-numpy argument

I think that a single arrow may be reasonable as syntactic sugar for
partial, but once you start chaining them, it all falls apart into a
mess. That, in my mind, is a sign that the idea doesn't scale. We can
chain dots with no problem:

fe.fi.fo.fum

and function calls in numerous ways:

foo(bar(baz()))
foo(bar)(baz)

and although they can get hard to read just because of the sheer number
of components, they are not conceptually difficult. But chaining arrows
is conceptually difficult even with as few as two arrows.

I think the problem here is that partial application is an N-ary
operation. This is not Haskell where single-argument currying is
enforced everywhere! You're trying to perform something which
conceptually takes N arguments partial(func, 1, 2, 3, ..., N) using only
a operator which can only take two arguments a->b. Things are going to
get messy.


> And here's something I find quite interesting. Together with
> @singledispatch from 3.4 (or possibly an enhanced version using type
> annotations in the future?), one could add 'third-party methods' to
> classes in other libraries without monkey patching. A dummy example:
>
> from numpy import array
> my_list = [1,2,3]
> my_array = array(my_list)
> my_mean = my_array.mean()  # This currently works in numpy
>
> from rmslib import rms
> my_rms = my_array->rms()  # efficient rms for numpy arrays
> my_other_rms = my_list->rms()  # rms that works on any iterable

That looks cute, but isn't very interesting. Effectively, you've
invented a new (and less efficient) syntax for calling a function:

spam->eggs(cheese)  # eggs(spam, cheese)

It's less efficient because it builds a partial object first, so instead
of one call you end up with two, and a temporary object that gets thrown
away immediately after it is used. Yes, you could keep the partial
object around, but as your example shows, you don't. And because it is
cute, people will write:

a->func(), b->func(), c->func()

and not realise that it creates three partial functions before calling
them. Writing:

func(a), func(b), func(c)

will avoid that needless overhead.



--
Steve
_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/