[Python-ideas] Function composition (was no subject)

Sun May 10 08:06:37 CEST 2015

Thanks! Not sure what took me so long to get on the python lists, but I finally did and to my excitement you were talking about my favorite topic!

---

For replacing the need to write `lambda x: x...` inside compositions *in a limited set of cases*, you could use a sort of "doppelganger" type/metaclass:

class ThisType(type):
    def __getattr__(cls, attr):
        def attribute(*args, **kwargs):
            def method(this):
                this_attr = getattr(this, attr)
                if callable(this_attr):
                    return this_attr(*args, **kwargs)
                else:
                    return this_attr
            return method
        return attribute
    def __call__(cls, *args, **kwargs):
        def decorator(fn):
            return fn(*args, **kwargs)
        return decorator
    def __getitem__(cls, item):
        return lambda x: x[item]

class this(metaclass=ThisType): pass

Basically, it records whatever is done to it, then returns a function that takes a value and does those things to the value. So any call, __getattr__ with arguments, and __getitem__ you'd want to do with a value mid-pipe would be staged or set up by doing them to `this`.

So rather than writing 
>>> compose(lambda s: s.strip('<>'), lambda s: s.lower())('<HTML>')

you can write

>>> compose(this.strip('<>'), this.lower())('<HTML>')
'html'

or

>>> compose(float, this.__str__)('1')
'1.0'

But there are two caveats:

Property attributes would need to be *called*, which feels weird when you already know an API well, so e.g.

>>> from lxml import html
>>> html.fromstring('<b>bold text</b>').text
'bold text'
>>> compose(html.fromstring, this.text())('<b>bold text</b>')
'bold text'

It's also a bit weird because attributes that return functions/methods/callables *aren't* called (like above with `this.__str__`: `__str__` is a method of `float`).

Second caveat is that nothing past the __getitem__ and __getattr__ will work, so e.g.

>>> from pandas import DataFrame
>>> df = DataFrame([1]*2, columns=['A','B'])
   A  B
0  1  1
1  1  1
>>> compose(this.applymap(str), this['A'])(df)
0    1
1    1
Name: A, dtype: object
>>> compose(this.applymap(str), this['A'], this.shape())(df)
(2,)

...but...

>>> compose(this.applymap(str), this['A'].shape)(df)
AttributeError: 'function' object has no attribute 'shape'

________________________________________
From: Python-ideas <python-ideas-bounces+larocca=abiresearch.com at python.org> on behalf of Steven D'Aprano <steve at pearwood.info>
Sent: Sunday, May 10, 2015 2:01 AM
To: python-ideas at python.org
Subject: Re: [Python-ideas] Function composition (was no subject)

On Sun, May 10, 2015 at 04:58:29AM +0000, Douglas La Rocca wrote:
> (Newcomer here.)
>
> I use function composition pretty extensively. I've found it to be
> incredibly powerful, but can lead to bad practices. Certain other
> drawbacks are there as well, like unreadable tracebacks. But in many
> cases there are real benefits. And for data pipelines where you want
> to avoid state and mutation it works well.

Thanks for the well-thought out and very detailed post!

The concrete experience you bring to this discussion is a welcome change
from all the theoretical "wouldn't it be nice (or awful) if ..." from
many of us, and I include myself. The fact that you have extensive
experience with using function composition in practice, and can point
out the benefits and disadvantages, is great.

--
Steve
_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/