Infix matrix-multiply, but not general infix operators?
A couple months back, Andrew Barnet brought up the idea of adding a way to apply functions in an infix style. It really didnt gain much traction. (Subject line "Infix Functions") Nathaniel Smith has just proposed a PEP to add an infix matrix multiply operator, and everyone seems to love it. I honestly am surprised at the difference in reaction. Why are we so quick to add a single-purpose punctuation mark, but reject a named infix operator that works for many general cases? Is the matrix-multiply use case so special? I dont oppose the PEP, and I dont mean to derail it. And I know there was quite a bit of discussion on Andrew's email. But I cant help feel that Andrew's idea didnt get the appreciation it should. (But hey, I'm biased because I like the idea :)) Jared QUICK SUMMARY OF ANDREW'S EMAIL: (Not a proposal for syntax or naming) m `cross` n m `dot` n a `Pair` b a `Tree` (b `Tree` c `Tree` d) `Tree` e And of course the matrix multiply (and matrix div that was briefly discussed) are implementable too: m `mmul` n m `mdiv` n
On 20 March 2014 17:41, Jared Grubb <jared.grubb@gmail.com> wrote:
A couple months back, Andrew Barnet brought up the idea of adding a way to apply functions in an infix style. It really didnt gain much traction. (Subject line "Infix Functions")
Nathaniel Smith has just proposed a PEP to add an infix matrix multiply operator, and everyone seems to love it.
I honestly am surprised at the difference in reaction. Why are we so quick to add a single-purpose punctuation mark, but reject a named infix operator that works for many general cases? Is the matrix-multiply use case so special?
Yes, basically - a large portion of the PEP is about pointing out that matrix multiplication really is that special. All past attempts at getting syntax for it *have* been generalised along that lines of this suggestion from Andrew, and have ended up failing on the grounds of "just use a method or function instead". Nathaniel's PEP is about looking at those past proposals and the experience of the numeric computing community and asking "Which operator have we *really* missed?". And they realised that it was only one: a way to spell matrix multiplication, since "*" has already been claimed for element-wise multiplication. The generalised proposals failed because they were still clumsy for the matrix multiplication use case, and there wasn't a compelling justification for the extra complexity that came with the generalisation. By contrast, Nathaniel's latest PEP looked at solving the *exact* problem the numeric community has: a clean and concise way to spell matrix multiplication. It hit the sweet spot of design proposals: a clean targeted solution that is as simple as it can possibly be, while still solving the problem that needs to be solved. It may have *looked* deceptively fast in terms of core devs saying "yes, that's a good idea", but the speed makes more sense if you start the timer from some of the earlier PEPs referenced from Nathaniel's one - the "matrix multiplication is clumsy compared to MATLAB" problem has been around for as long as people have been doing numeric computing in Python. Cheers, Nick.
I dont oppose the PEP, and I dont mean to derail it. And I know there was quite a bit of discussion on Andrew's email. But I cant help feel that Andrew's idea didnt get the appreciation it should. (But hey, I'm biased because I like the idea :))
Jared
QUICK SUMMARY OF ANDREW'S EMAIL: (Not a proposal for syntax or naming)
m `cross` n m `dot` n
a `Pair` b a `Tree` (b `Tree` c `Tree` d) `Tree` e
And of course the matrix multiply (and matrix div that was briefly discussed) are implementable too:
m `mmul` n m `mdiv` n
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, Mar 20, 2014 at 6:41 PM, Jared Grubb <jared.grubb@gmail.com> wrote:
But I cant help feel that Andrew's idea didnt get the appreciation it should. (But hey, I'm biased because I like the idea :))
Jared
QUICK SUMMARY OF ANDREW'S EMAIL: (Not a proposal for syntax or naming)
m `cross` n m `dot` n
Sometimes a proposal fails because no good syntax can be found for it. The strongest objection to my PEP 463 on exception expressions is "I don't like the syntax", and to date nobody's come up with a syntax that everyone likes. (Four proposals in the PEP itself, all somewhat flawed.) In the above case, the proposal somewhat falls down for the same reason. Backticks have been, by BDFL fiat, banned from syntax; either that decision has to be reversed, or some other syntax has to be figured out. People will love or hate the proposal based on the tentative syntax, and the "grit on Tim's monitor" check, and such. ChrisA
Chris Angelico writes:
figured out. People will love or hate the proposal based on the tentative syntax, and the "grit on Tim's monitor" check, and such.
Why the lets-use-some-punctuation-for-syntax crowd doesn't just buy Tim a lifetime subscription to monitor-protecting clearsheets so he can remove the grit every day, I just don't know! Win-win AFAICS....
On Thu, Mar 20, 2014 at 12:41:40AM -0700, Jared Grubb wrote:
A couple months back, Andrew Barnet brought up the idea of adding a way to apply functions in an infix style. It really didnt gain much traction. (Subject line "Infix Functions")
Nathaniel Smith has just proposed a PEP to add an infix matrix multiply operator, and everyone seems to love it.
I honestly am surprised at the difference in reaction. Why are we so quick to add a single-purpose punctuation mark, but reject a named infix operator that works for many general cases? Is the matrix-multiply use case so special?
Yes. Every day, there are probably thousands of numpy users (to say nothing of other matrix-multiplication library users) who do matrix multiplication. I can't think of the last time I've wanted to invent my own infix operator, and if I ever did, I'd probably be the only person using it. I don't think I'm too unusual. Custom infix operators for arbitrary functions are extremely niche, because apart from matrix multiplication, nearly all the common, useful ones are already available. We have + - * / & ^ | etc., that covers the common cases. The ones remaining are (apart from matrix multiplication) uncommon. The beauty of infix operators is that they give an extremely compact representation, which is valuable for mathematics but not so much for general purpose computing. Consequently, it would be useful to write something like: matrix @ vector versus matrix.mult(vector) because the symbol for the operator is so compact. Having to write it as this instead: matrix`mult`vector saves you one character over the method syntax, or even costs you one character if you write it like this: matrix `mult` vector which is not enough of a saving over the method syntax to make it worthwhile. Having operators like + and * is a good thing, but if we didn't have them, and had to write this instead: x`plus`y a`times`b there'd be no advantage to using pseudo-operators as shown. You might as well use regular method calls: x.plus(y) a.times(b) (which you'll note is already pseudo-infix). So, in my opinion, until such time as Python supports Unicode symbols (around Python 5000 perhaps?) as operators, there is no point in trying to fake them with spelled-out names. If you're going to use a spelled-out name, use a method, or function. -- Steven
On Mar 20, 2014, at 3:24, Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Mar 20, 2014 at 12:41:40AM -0700, Jared Grubb wrote:
The beauty of infix operators is that they give an extremely compact representation, which is valuable for mathematics but not so much for general purpose computing. Consequently, it would be useful to write something like:
matrix @ vector
versus
matrix.mult(vector)
because the symbol for the operator is so compact. Having to write it as this instead:
matrix`mult`vector
saves you one character over the method syntax,
It's not about saving one character, it's about saving the (possibly overly-nested) parentheses. The PEP explains that, as does the earlier thread, and all of the previous threads and PEPs on related ideas, so I won't go into it again. However, the other answers are right; matrix multiplication _is_ special. As an alternative to the 2000 PEP that proposed a suite of new operators by allowing ~ as a prefix to 11 existing operators )or to the other proposal to allow any string of operator characters to be defined as an operator), I think general infix wins. However, the NumPy community is no longer suggesting either of those; after 14 years of living with the limitations of the current system, they've decided that only one operator is really a problem. And against that proposal, general infix loses. One is a good number of special cases for the same reason 11 is a bad number of special cases. Of course I still like my general infix proposal, but even if we had it, I think @ for matmul might be worth adding anyway.
On 03/20/2014 12:06 PM, Andrew Barnert wrote:
Of course I still like my general infix proposal, but even if we had it, I think @ for matmul might be worth adding anyway.
I like that idea also, but ... s = 'a' 'b' 'c' Is 'b' a string op or a string? To make it work, it needs an introducer. Which kind of defeats some of the advantage. Or it needs to be defined with a keyword. This actually makes sense as it's changes the effect of a name. (like local, non-local, and global.) def foo(...): symbol b s = 'a' b 'c' One of the objects would need a method to get the symbol, and the other argument. Cheers, Ron
On Thu, Mar 20, 2014 at 4:14 PM, Ron Adam <ron3200@gmail.com> wrote:
def foo(...): symbol b
s = 'a' b 'c'
Actually makes me think of decorators, which means that you wouldn't even need new tokens or keywords: from functools import make_operator bop = make_operator(b) s = 'a' bop 'b' class Ops: @make_operator def cat(left, right): return left + right t = 'a' Ops.cat 'b' It reduces the grit, but also reduces certain kinds of obviousness. bwm
On Thu, Mar 20, 2014 at 05:04:25PM -0400, Brandon W Maister wrote:
Actually makes me think of decorators, which means that you wouldn't even need new tokens or keywords:
But you would require new syntax. Your suggested syntax s = 'a' bop 'b' is currently a syntax error. Keep in mind that any changes to syntax would allow bop to be any arbitrary object. Python cannot tell at compile-time whether bop is an "operator" or a list or a string or a float, only at run-time.
from functools import make_operator bop = make_operator(b)
Do you have any suggestion as to how make_operator would work? What should it do? What sort of object is bop? -- Steven
On Mar 20, 2014 8:16 PM, "Steven D'Aprano" <steve@pearwood.info> wrote:
On Thu, Mar 20, 2014 at 05:04:25PM -0400, Brandon W Maister wrote:
Actually makes me think of decorators, which means that you wouldn't
even
need new tokens or keywords:
But you would require new syntax. Your suggested syntax
s = 'a' bop 'b'
is currently a syntax error.
100% agree, I was careful with my language, if not my thinking :-)
Keep in mind that any changes to syntax would allow bop to be any arbitrary object. Python cannot tell at compile-time whether bop is an "operator" or a list or a string or a float, only at run-time.
I admit that when writing I didn't think about the fact that I was suggesting adding dynamic names to what is currently a static table.
from functools import make_operator bop = make_operator(b)
Do you have any suggestion as to how make_operator would work? What should it do?
Originally I was thinking something along the lines of "just add it to the 'operators' dict" but obviously that won't work since there's no such thing. Before I defend the idea any further I need to point out that I don't think it's a *good* idea. I kind of like the idea of arbitrary infix functions, so I threw out an idea that I haven't seen before. We could, hypothetically, create an "operators" dict (or dicts) alongside locals and globals. Then, if an unexpected token shows up in operator position (which is probably well-defined?) before a SyntaxError is raised the token is looked up in the operators dict. I think that the biggest problem I have with that solution is that philosophically it seems like it would take Python a bit far into the lisp-2 camp, which I would not like.
What sort of object is bop? A regular function that takes one or two args. Honestly the nice thing about the "look it up somewhere if it causes the right kind of SyntaxError" solution is that it would probably also allow arbitrary unary operators. Although why anyone would want *that* is beyond me, so maybe it's not nice :-)
bwm
On Fri, Mar 21, 2014 at 12:28 PM, Brandon W Maister <bwmaister@gmail.com> wrote:
We could, hypothetically, create an "operators" dict (or dicts) alongside locals and globals. Then, if an unexpected token shows up in operator position (which is probably well-defined?) before a SyntaxError is raised the token is looked up in the operators dict.
The SyntaxError is raised at compilation time; locals and globals are at run time. The only way that would work is if you do something like this: operators["b"] = lambda left, right: whatever import module_using_b ChrisA
On Mar 20, 2014, at 19:49, Chris Angelico <rosuav@gmail.com> wrote:
On Fri, Mar 21, 2014 at 12:28 PM, Brandon W Maister <bwmaister@gmail.com> wrote:
We could, hypothetically, create an "operators" dict (or dicts) alongside locals and globals. Then, if an unexpected token shows up in operator position (which is probably well-defined?) before a SyntaxError is raised the token is looked up in the operators dict.
The SyntaxError is raised at compilation time; locals and globals are at run time. The only way that would work is if you do something like this:
operators["b"] = lambda left, right: whatever import module_using_b
Or, alternatively, instead of a SyntaxError this would compile into a call to b, which would then give a NameError at runtime. And I suppose you could argue that's just taking dynamic typing one step further. But I seriously doubt anyone actually wants typos like "esle" to compile successfully... Also, imagine what it would look like to chain these: spam spam eggs baked beans spam spam Maybe the parser can figure out that the spam, baked, and spam are operators and the spam, eggs, beans, and spam are operands, but what hope does any human have? Obviously that example is over the top, but try it with a tree constructor or the other examples from this thread and it's still painful. The Haskell-style backticks solve all of these problems: the coder can't accidentally write an infix operator when he meant something else, the parser can distinguish infix operators from typos at compile time, and the reader can tell (unless he's stolen Tim's monitor grit) which things are operators and which are operands. (Of course you could deliberately confuse people with weird spacing, but Python doesn't have to make it impossible to write intentionally obfuscated code, it just has to avoid making it easy to write accidentally obfuscated code.)
On 03/20/2014 08:40 PM, Andrew Barnert wrote:
Also, imagine what it would look like to chain these:
spam spam eggs baked beans spam spam
Maybe the parser can figure out that the spam, baked, and spam are operators and the spam, eggs, beans, and spam are operands, but what hope does any human have?
Why would spam be able to be both a variable and an operator? Hmm, perhaps because of the first-class citizen thing -- so the leading grit says "make it work" and without it is just passing it around. -- ~Ethan~
On Fri, Mar 21, 2014 at 4:01 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/20/2014 08:40 PM, Andrew Barnert wrote:
Also, imagine what it would look like to chain these:
spam spam eggs baked beans spam spam
Maybe the parser can figure out that the spam, baked, and spam are operators and the spam, eggs, beans, and spam are operands, but what hope does any human have?
Why would spam be able to be both a variable and an operator? Hmm, perhaps because of the first-class citizen thing -- so the leading grit says "make it work" and without it is just passing it around.
If the creation of an operator is done by putting something into the current namespace, then it has to be available as some sort of object. On the other hand, if it's something done as a declaration, or a pre-import keyword mapping, then it should be a keyword (like other word operators, "if" and "else" and such), and unavailable for assignment. In theory, this could be done as: from __aether__ import spam as operator where "as operator", instead of specifying the name to bind to, specifies that it be something other than a name. (Preferably, it should use a keyword, rather than "operator", which is a valid name.) This would go immediately under future statements, and would affect the compilation of that module: any instance of the word "spam" is now a keyword operator, so the above would be invalid. I'm still not convinced it would be a good thing, but this at least gives a per-module way to create operators. ChrisA
Chris Angelico <rosuav@gmail.com> wrote:
where "as operator", instead of specifying the name to bind to, specifies that it be something other than a name. (Preferably, it should use a keyword, rather than "operator", which is a valid name.) This would go immediately under future statements, and would affect the compilation of that module: any instance of the word "spam" is now a keyword operator, so the above would be invalid. I'm still not convinced it would be a good thing, but this at least gives a per-module way to create operators.
Cluttering the syntax with arbitrary operators is a very bad idea, IMHO. Sturla
From: Python-ideas [mailto:python-ideas-bounces+andy.henshaw=gtri.gatech.edu@python.org] On Behalf Of Ethan Furman
On 03/20/2014 08:40 PM, Andrew Barnert wrote:
Also, imagine what it would look like to chain these:
spam spam eggs baked beans spam spam
Maybe the parser can figure out that the spam, baked, and spam are operators and the spam, eggs, beans, and spam are operands, but what hope does any human have?
Why would spam be able to be both a variable and an operator? Hmm, perhaps because of the first-class citizen thing -- so the leading grit says "make it work" and without it is just passing it around.
Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo
On Mar 21, 2014, at 10:19 AM, "Henshaw, Andy" <Andy.Henshaw@gtri.gatech.edu> wrote:
From: Python-ideas [mailto:python-ideas-bounces+andy.henshaw=gtri.gatech.edu@python.org] On Behalf Of Ethan Furman
On 03/20/2014 08:40 PM, Andrew Barnert wrote:
Also, imagine what it would look like to chain these:
spam spam eggs baked beans spam spam
Maybe the parser can figure out that the spam, baked, and spam are operators and the spam, eggs, beans, and spam are operands, but what hope does any human have?
Why would spam be able to be both a variable and an operator? Hmm, perhaps because of the first-class citizen thing -- so the leading grit says "make it work" and without it is just passing it around.
Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo
For those whose first language is not USA English: http://en.wikipedia.org/wiki/Buffalo_Buffalo_Buffalo_Buffalo_Buffalo_Buffalo Thanks, Cem Karan
On 03/21/2014 07:19 AM, Henshaw, Andy wrote:
From: Python-ideas [mailto:python-ideas-bounces+andy.henshaw=gtri.gatech.edu@python.org] On Behalf Of Ethan Furman
On 03/20/2014 08:40 PM, Andrew Barnert wrote:
Also, imagine what it would look like to chain these:
spam spam eggs baked beans spam spam
Maybe the parser can figure out that the spam, baked, and spam are operators and the spam, eggs, beans, and spam are operands, but what hope does any human have?
Why would spam be able to be both a variable and an operator? Hmm, perhaps because of the first-class citizen thing -- so the leading grit says "make it work" and without it is just passing it around.
Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo
Python, however, is not English, and would only recognize two different objects from the above, so that would be a SyntaxError. -- ~Ethan~
spam `spam` eggs `baked` beans `spam` spam
I'm not sure if the backticks help. When you read that, I think you're reading the spacing not the ticks. You can already do something very similar by using built-in operators to "quote" the custom "operators", if you set the appropriate operator overloads.
spam *spam* eggs <baked> beans /spam/ spam
"<spam>" is potentially the most readable, for it's parenthesis like shape , but you can't overload the "and" it expands into. Mark Daoust On Thu, Mar 20, 2014 at 11:40 PM, Andrew Barnert <abarnert@yahoo.com> wrote:
On Mar 20, 2014, at 19:49, Chris Angelico <rosuav@gmail.com> wrote:
On Fri, Mar 21, 2014 at 12:28 PM, Brandon W Maister <bwmaister@gmail.com> wrote:
We could, hypothetically, create an "operators" dict (or dicts) alongside locals and globals. Then, if an unexpected token shows up in operator position (which is probably well-defined?) before a SyntaxError is raised the token is looked up in the operators dict.
The SyntaxError is raised at compilation time; locals and globals are at run time. The only way that would work is if you do something like this:
operators["b"] = lambda left, right: whatever import module_using_b
Or, alternatively, instead of a SyntaxError this would compile into a call to b, which would then give a NameError at runtime. And I suppose you could argue that's just taking dynamic typing one step further. But I seriously doubt anyone actually wants typos like "esle" to compile successfully...
Also, imagine what it would look like to chain these:
spam spam eggs baked beans spam spam
Maybe the parser can figure out that the spam, baked, and spam are operators and the spam, eggs, beans, and spam are operands, but what hope does any human have? Obviously that example is over the top, but try it with a tree constructor or the other examples from this thread and it's still painful.
The Haskell-style backticks solve all of these problems: the coder can't accidentally write an infix operator when he meant something else, the parser can distinguish infix operators from typos at compile time, and the reader can tell (unless he's stolen Tim's monitor grit) which things are operators and which are operands. (Of course you could deliberately confuse people with weird spacing, but Python doesn't have to make it impossible to write intentionally obfuscated code, it just has to avoid making it easy to write accidentally obfuscated code.) _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Fri, Mar 21, 2014 at 3:06 PM, Mark Daoust <daoust.mj@gmail.com> wrote:
spam `spam` eggs `baked` beans `spam` spam
I'm not sure if the backticks help.
Backticks never help. They hide amongst the beefier characters. That's one reason why $(...) is preferred by most Bash users and repr() by most Python users. Let them die the quiet death they deserve. Skip
On Mar 21, 2014, at 13:06, Mark Daoust <daoust.mj@gmail.com> wrote:
spam `spam` eggs `baked` beans `spam` spam
I'm not sure if the backticks help.
When you read that, I think you're reading the spacing not the ticks.
No, you're reading a combination of the two. Compare it to the same without backticks: >>> spam `spam` eggs `baked` brand `spam` spam >>> spam spam eggs baked beans spam spam >>> spam spam eggs baked beans spam spam There's no way to differentiate the operators from the operands with _just_ spacing, but with spacing plus anything else, there is. And lest you think this is just an artifact of passing spam to spam, here it is with more realistic names; >>> mata mmul matb mmul matb mmul vec >>> mata `mmul` matb `mmul` matb `mmul` vec No matter how much you like or dislike the latter, the former is clearly a lot worse.
You can already do something very similar by using built-in operators to "quote" the custom "operators", if you set the appropriate operator overloads.
Yes, I mentioned the |op| hack the last time I brought this up, and two other people suggested it after that. But if you want to go over it again: no symbol has a precedence right below the bitwise ops, comparison operators don't work because of chaining, there are types that will handle __op__ so your fake operator doesn't get a chance to do its __rop__, there's no way to prevent "2 |op" from being a valid but confusing expression (and there's a temptation to use it as a half-assed form of operator sectioning), it's hard for syntax highlighters and linters and the like to understand |op| is an operator, there's no way to restrict arbitrary expressions as operators, you have to declare all infixable operators with a decorator (as opposed to having the language choose whether all functions are infixable, or we use a decorator, or something else), any symbol you use already has another meaning that will confuse people, it's a bit less efficient, and, finally, it's not a consistent language feature (having |op| be an operator in one module, *op* in another, etc. makes the language harder to learn).
participants (14)
-
Andrew Barnert -
Brandon W Maister -
Cem Karan -
Chris Angelico -
Ethan Furman -
Henshaw, Andy -
Jared Grubb -
Mark Daoust -
Nick Coghlan -
Ron Adam -
Skip Montanaro -
Stephen J. Turnbull -
Steven D'Aprano -
Sturla Molden