Re: [Python-ideas] Function composition (was no subject)
On 2015-05-09 21:16, Steven D'Aprano wrote:
On Sat, May 09, 2015 at 11:38:38AM -0400, Ron Adam wrote:
How about an operator for partial?
root @ mean @ map $ square(xs) Apart from the little matter that Guido has said that $ will never be used as an operator in Python, what is the association between $ and partial?
Most other operators have either been used for centuries e.g. + and - or at least decades e.g. * for multiplication because ASCII doesn't have the × symbol. The barrier to using a completely arbitrary symbol with no association to the function it plays should be considered very high.
I would only support an operator for function composition if it was at least close to the standard operators used for function composition in other areas. @ at least suggests the ∘ used in mathematics, e.g. sin∘cos, but | is used in pipelining languages and shells and could be considered, e.g. ls | wc.
My own preference would be to look at @ as the closest available ASCII symbol to ∘ and use it for left-to-right composition, and | for left-to-right function application. E.g.
(spam @ eggs @ cheese)(arg) is equivalent to spam(eggs(cheese(arg)))
(spam | eggs | cheese)(arg) is equivalent to cheese(eggs(spam(arg)))
also known as compose() and rcompose(). We can read "@" as "of", "spam of eggs of cheese of arg", and | as a pipe, "spam(arg) piped to eggs piped to cheese".
For me these are by far the most logical ones too, for exactly the same reasons (and because of the connection of @ with matrix multiplication and operators that operate from the left).
It's a pity we can't match the shell syntax and write:
spam(args)|eggs|cheese
but that would have a completely different meaning.
But it does not need to have a different meaning. You could in addition have: spam @ eggs @ cheese @ arg # equivalent to spam(eggs(cheese(arg))) arg | spam | eggs | cheese # equivalent to cheese(eggs(spam(arg))) Here, arg would thus be recognized as not a function. In this version, your example of spam(args)|eggs|cheese would do exactly the same operation as (spam | eggs | cheese)(args) :-).
David Beazley has a tutorial on using coroutines in pipelines:
http://www.dabeaz.com/coroutines/
where he ends up writing this:
f = open("access-log") follow(f, grep('python', printer()))
Coroutines grep() and printer() make up the pipeline. I cannot help but feel that the | syntax would be especially powerful for this sort of data processing purpose:
# could this work using some form of function composition? follow(f, grep('python')|printer)
This seems promising! -- Koos
pipeline operator may be confusing with bitwise operator. In this case : eggs = arg | spam | cheese Is eggs a composed function or string of bits ? 2015-05-09 21:15 GMT+02:00 Koos Zevenhoven <koos.zevenhoven@aalto.fi>:
On 2015-05-09 21:16, Steven D'Aprano wrote:
On Sat, May 09, 2015 at 11:38:38AM -0400, Ron Adam wrote:
How about an operator for partial?
root @ mean @ map $ square(xs)
Apart from the little matter that Guido has said that $ will never be used as an operator in Python, what is the association between $ and partial?
Most other operators have either been used for centuries e.g. + and - or at least decades e.g. * for multiplication because ASCII doesn't have the × symbol. The barrier to using a completely arbitrary symbol with no association to the function it plays should be considered very high.
I would only support an operator for function composition if it was at least close to the standard operators used for function composition in other areas. @ at least suggests the ∘ used in mathematics, e.g. sin∘cos, but | is used in pipelining languages and shells and could be considered, e.g. ls | wc.
My own preference would be to look at @ as the closest available ASCII symbol to ∘ and use it for left-to-right composition, and | for left-to-right function application. E.g.
(spam @ eggs @ cheese)(arg) is equivalent to spam(eggs(cheese(arg)))
(spam | eggs | cheese)(arg) is equivalent to cheese(eggs(spam(arg)))
also known as compose() and rcompose(). We can read "@" as "of", "spam of eggs of cheese of arg", and | as a pipe, "spam(arg) piped to eggs piped to cheese".
For me these are by far the most logical ones too, for exactly the same reasons (and because of the connection of @ with matrix multiplication and operators that operate from the left).
It's a pity we can't match the shell syntax and write:
spam(args)|eggs|cheese
but that would have a completely different meaning.
But it does not need to have a different meaning. You could in addition have:
spam @ eggs @ cheese @ arg # equivalent to spam(eggs(cheese(arg)))
arg | spam | eggs | cheese # equivalent to cheese(eggs(spam(arg)))
Here, arg would thus be recognized as not a function.
In this version, your example of spam(args)|eggs|cheese would do exactly the same operation as (spam | eggs | cheese)(args) :-).
David Beazley has a tutorial on using coroutines in pipelines:
http://www.dabeaz.com/coroutines/
where he ends up writing this:
f = open("access-log") follow(f, grep('python', printer()))
Coroutines grep() and printer() make up the pipeline. I cannot help but feel that the | syntax would be especially powerful for this sort of data processing purpose:
# could this work using some form of function composition? follow(f, grep('python')|printer)
This seems promising!
-- Koos
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Nobody convinced by arrow operator ? like: arg -> spam -> eggs -> cheese or cheese <- eggs <- spam <- arg This also make sense with annotations: def func(x:type1, y:type2) -> type3: pass we expect func to return type3(func(x, y)) 2015-05-09 22:41 GMT+02:00 Gregory Salvan <apieum@gmail.com>:
pipeline operator may be confusing with bitwise operator. In this case : eggs = arg | spam | cheese
Is eggs a composed function or string of bits ?
2015-05-09 21:15 GMT+02:00 Koos Zevenhoven <koos.zevenhoven@aalto.fi>:
On 2015-05-09 21:16, Steven D'Aprano wrote:
On Sat, May 09, 2015 at 11:38:38AM -0400, Ron Adam wrote:
How about an operator for partial?
root @ mean @ map $ square(xs)
Apart from the little matter that Guido has said that $ will never be used as an operator in Python, what is the association between $ and partial?
Most other operators have either been used for centuries e.g. + and - or at least decades e.g. * for multiplication because ASCII doesn't have the × symbol. The barrier to using a completely arbitrary symbol with no association to the function it plays should be considered very high.
I would only support an operator for function composition if it was at least close to the standard operators used for function composition in other areas. @ at least suggests the ∘ used in mathematics, e.g. sin∘cos, but | is used in pipelining languages and shells and could be considered, e.g. ls | wc.
My own preference would be to look at @ as the closest available ASCII symbol to ∘ and use it for left-to-right composition, and | for left-to-right function application. E.g.
(spam @ eggs @ cheese)(arg) is equivalent to spam(eggs(cheese(arg)))
(spam | eggs | cheese)(arg) is equivalent to cheese(eggs(spam(arg)))
also known as compose() and rcompose(). We can read "@" as "of", "spam of eggs of cheese of arg", and | as a pipe, "spam(arg) piped to eggs piped to cheese".
For me these are by far the most logical ones too, for exactly the same reasons (and because of the connection of @ with matrix multiplication and operators that operate from the left).
It's a pity we can't match the shell syntax and write:
spam(args)|eggs|cheese
but that would have a completely different meaning.
But it does not need to have a different meaning. You could in addition have:
spam @ eggs @ cheese @ arg # equivalent to spam(eggs(cheese(arg)))
arg | spam | eggs | cheese # equivalent to cheese(eggs(spam(arg)))
Here, arg would thus be recognized as not a function.
In this version, your example of spam(args)|eggs|cheese would do exactly the same operation as (spam | eggs | cheese)(args) :-).
David Beazley has a tutorial on using coroutines in pipelines:
http://www.dabeaz.com/coroutines/
where he ends up writing this:
f = open("access-log") follow(f, grep('python', printer()))
Coroutines grep() and printer() make up the pipeline. I cannot help but feel that the | syntax would be especially powerful for this sort of data processing purpose:
# could this work using some form of function composition? follow(f, grep('python')|printer)
This seems promising!
-- Koos
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Sun, May 10, 2015 at 12:03:24AM +0200, Gregory Salvan wrote:
Nobody convinced by arrow operator ?
like: arg -> spam -> eggs -> cheese or cheese <- eggs <- spam <- arg
Absolutely not! If we were designing a new language from scratch, I might consider arrow operators. I think that they are cute. But this proposal is going to be hard enough to get approval using *existing* operators, | __or__ and @ __mat_mul__ (if I remember the dunder methods correctly). To convince people that we should support function composition as a built-in feature, using NEW operators that will need the parser changed to recognise, and new dunder methods, well, that will be virtually impossible. numpy is one of the biggest and most important user bases for Python, and it took them something like ten years and multiple failed attempts to get enough support for adding the @ operator. You *might* just have a chance for a -> right arrow operator, just barely, but the left arrow <- operator is, I'm pretty sure, doomed to failure. The problem is that the parser would need to distinguish these two cases: f<-x # f left-arrow x f<-x # f less than minus x and I don't think that is possible with Python's parser. -- Steve
Gregory Salvan writes:
Nobody convinced by arrow operator ?
like: arg -> spam -> eggs -> cheese or cheese <- eggs <- spam <- arg
Yuck. There are living languages (R) that use an arrow as an assignment operator, and others (or perhaps you consider C a zombie language <wink/>) that uses one as a member operator. I would prefer the C++ pipe operators, ie, << and >>. But that's just bikeshedding a moot point; I doubt most people would be favorable to introducing more operator symbols for this purpose, and I personally would be opposed. If functools was more popular and its users were screaming for operators the way the numerical folk screamed for a matrix multiplication operator, I'd be more sympathetic. But they're not screaming that I can hear. To give an idea of how difficult it is to get an operator added, it took at least a decade to get the matrix multiplication operator added after it was first proposed, and two of the key steps were first the introduction of unary "@" for decorator application (another case that screamed for a new operator), and then the proponents dropping the "@@" operator from their proposal.
I agree an operator is really unnecessary, especially because parens would be needed anyway for subexpressions. A LISP-like syntax would work better than something trying to imitate Haskell. make_breakfast = (make_spam make_eggs(2, 'overeasy') make_cheese) make_breakfast() It also has the sense of collecting and reifying a series of functions rather than declaring a tuple or list or some other data structure. Why bother with the left/right associative issues of infix operators? Suppose you wanted a simple quick composition of a few functions where the expression would be called right away. With infix it might look like (list @ sorted @ ','.join)('a string of chars to be sorted, then joined on commas') But you already have the parens so why not just (list sorted ','.join)('a string ...') On May 10, 2015, at 1:53 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Gregory Salvan writes:
Nobody convinced by arrow operator ?
like: arg -> spam -> eggs -> cheese or cheese <- eggs <- spam <- arg
Yuck. There are living languages (R) that use an arrow as an assignment operator, and others (or perhaps you consider C a zombie language <wink/>) that uses one as a member operator. I would prefer the C++ pipe operators, ie, << and >>.
But that's just bikeshedding a moot point; I doubt most people would be favorable to introducing more operator symbols for this purpose, and I personally would be opposed. If functools was more popular and its users were screaming for operators the way the numerical folk screamed for a matrix multiplication operator, I'd be more sympathetic. But they're not screaming that I can hear.
To give an idea of how difficult it is to get an operator added, it took at least a decade to get the matrix multiplication operator added after it was first proposed, and two of the key steps were first the introduction of unary "@" for decorator application (another case that screamed for a new operator), and then the proponents dropping the "@@" operator from their proposal. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Stephen J. Turnbull, ok, I was wong about community expectation, I thougth functools would be more popular with new symbols. Personnally, I made a wide use of functionnal paradigm but except when I need an heavy use of partial and reduce, the simple fact of importing functools and use the "partial" function has a higher cost than making it differently. That's also because python syntax is really convenient and lambda, decorators, iterators... allow a lot of things.
On Sat, May 09, 2015 at 10:41:24PM +0200, Gregory Salvan wrote:
pipeline operator may be confusing with bitwise operator. In this case : eggs = arg | spam | cheese
Is eggs a composed function or string of bits ?
Or a set? I think it is okay to overload operators and give them different meanings: z = x + y Is z a number, a string, a list, a tuple? Something else? In practice, we rely on sensible names or context to understand overloaded operators, if you see foo = search | grep | log | process it = (foo(x) for x in data) run(it) it should be fairly obvious from context that foo is not a set or string of bits :-) -- Steve
On Sat, May 09, 2015 at 10:15:21PM +0300, Koos Zevenhoven wrote:
On 2015-05-09 21:16, Steven D'Aprano wrote:
[...]
It's a pity we can't match the shell syntax and write:
spam(args)|eggs|cheese
but that would have a completely different meaning.
But it does not need to have a different meaning.
It *should* have a different meaning. I want it to have a different meaning. Python is not the shell and spam(args) could be a factory function which itself returns a callable, e.g. partial, or a decorator. We cannot match the shell syntax because Python can do so much more than the shell.
You could in addition have:
spam @ eggs @ cheese @ arg # equivalent to spam(eggs(cheese(arg)))
arg | spam | eggs | cheese # equivalent to cheese(eggs(spam(arg)))
Here, arg would thus be recognized as not a function.
No. I think it is absolutely vital to distinguish by syntax the difference between composition and function application, and not try to "do what I mean". DWIM software has a bad history of doing the wrong thing. Every other kind of callable uses obj(arg) to call it: types, functions, methods, partial objects, etc. We shouldn't make function composition try to be different. If I write sqrt@100 I should get a runtime error, not 10. I don't mind if the error is delayed until I actually try to call the composed object, but at some point I should get a TypeError that 100 is not callable. -- Steve
participants (5)
-
Douglas La Rocca
-
Gregory Salvan
-
Koos Zevenhoven
-
Stephen J. Turnbull
-
Steven D'Aprano