The @update operator for dictionaries
I've been thinking that it might be easier, in the long term, to make a big step and allow >>> a @update= b as valid Python. What do you think? (I hope it will look nicer once syntax highlighted.) For clarity, this would proceed via a.__iat_update__(b), and (a @update b) would be similarly defined. As major disadvantage, of course, will be that >>> guido@python.org would no longer be valid Python! And also we might have fewer animated discussions. -- Jonathan
Anders Hovmöller wrote:
I don't understand what you mean. Can you provide examples that show the state of the dicts before and after and what the syntax would be the equivalent of in current python?
If a.__radd__ exists, then a += b is equivalent to a = a.__radd__(b) Similarly, if a.__iat_update__ exists then a @update= b would be equivalent to a = a.__iat_update__(b) Here's an implementation def __iat_update__(self, other): self.update(other) return self Thus, 'b' would be unchanged, and 'a' would be the same dictionary as before, but updated with 'b'. I hope this helps. -- Jonathan
On Sun, Mar 10, 2019 at 3:16 AM Jonathan Fine <jfine2358@gmail.com> wrote:
With something this long, how is it better from just writing: a = a.update_with(b) ? What's the point of an operator, especially if - by your own statement - it will backward-incompatibly change the language grammar (in ways that I've yet to understand, since you haven't really been clear on that)? ChrisA
Chris Angelico suggested that a = a.update_with(b) would be better than a @update= b One of the key points of += is that parent.child['toy'].wheel[3].speed += 1 increases the speed that that wheel by 1, without having to write parent.child['toy'].wheel[3].speed = parent.child['toy'].wheel[3].speed + 1 To answer Chris's other points. It not me, but Chris and Steve who want to bind dict.update to an operator, namely '+'. I'm suggested that if you do that, why not call the operator 'update'. Finally, we don't yet have any real idea how much difficulty the grammar change would cause. -- Jonathan
On Sun, Mar 10, 2019 at 3:46 AM Jonathan Fine <jfine2358@gmail.com> wrote:
No, we don't, because you have yet to say what the grammar change would BE. Changing language grammar is a big deal. You don't just say "oh, we should do this, and hey, it's gonna break code". Steven's proposal (not mine, btw, unless you meant some other Chris?) involves giving meaning to "x + y" for different values of x and y, but doesn't change the grammar at all. If you're proposing a completely new meaning for completely new syntax, *be clear* about what you are proposing. ChrisA
Jonathan Fine wrote:
One reason would be that '+' is short, whereas 'update' is long. A large part of the reason that common operations are written using infix operators is that the operator symbols used are very compact. That benefit disappears if your operator is an entire word. -- Greg
On Sat, Mar 9, 2019 at 4:14 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I suppose people bring up Haskell too much, but it does work in Haskell. People write things like (item `notElem` list) all the time and it's readable enough. In Haskell, though, it's sugar for (notElem item list), or notElem(item, list) in Pythonish syntax. In Python, it'd in most cases be sugar for a method call, in which the method name already appears in infix position, so the benefit is less clear. Given that Python's so-called augmented assignments are really mutating operations in disguise anyway (x op= y is not equivalent to x = x op y when x is mutable), I don't see any advantage of a new assignment syntax over the existing mutating methods. I.e., instead of x @update= y, you can just write x.update(y).
It might also be worth considering YAML's own dict merge operator, the "<<" operator, as in https://yaml.org/type/merge.html as this is the existing Python's shift operator added to dict and will require no change to the synatx:: a = a << b Meitham On 03/10, Chris Angelico wrote:
-- Meitham Jamaa http://meitham.com GPG Fingerprint: 3934D0B2
I really like this suggestion. It captures the asymmetry, since we could have a = a >> b to merge with the other dictionary's keys taking precedence. My instinct is that a = a << b would take b's values when keys collide and a = a >> b would take a's values when keys collide. I'd be very interested to know if this matches most peoples' intuitions. On Sat, 9 Mar 2019 at 18:44, Meitham Jamaa <m@meitham.com> wrote:
Maybe it's just the C++ IO piping that makes me like it, but these actually seem intuitive to me, whereas `+` or even `|` leaves me queasy. On Sat, Mar 9, 2019 at 7:40 PM Ian Foote <ian@feete.org> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
Are you all REALY=LU proposing more operators? Adding @ made sense because there was an important use case for which there was no existing operator to use. But in this case, we have + and | available, both of which are pretty good options. Finally, which dicts are a very important ue ase, do we want to add an operator just for that? What would it mean for sets, for instance? I have to say, the whole discussion seems to me to be a massive bike-shedding exercise -- the original proposal was to simply define + for dicts. It's totally reasonable to not like that idea at all, or to propose that | is a better option, but this has really gone off the rails! I guess I say that because this wasn't started with a critical use-case that really needed a solution, but rather: "the + operator isn't being used for dicts, why not make a semi-common operation easily available" So my opinion, which I'm re-stating: using + to merge dicts is simple, non-disruptive, and unlikely to really confuse anyone - so why not? ( | would be OK, too, though I think a tad less accessible to newbies) But I don't think that having an operator to merge dicts is a critical use-case worth of adding a new operator or new syntax to the to the language. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Sat, Mar 09, 2019 at 03:33:25PM +0000, Jonathan Fine wrote:
Are we supposed to know what that does?
For clarity, this would proceed via a.__iat_update__(b), and (a @update b) would be similarly defined.
For an explanation to be clear, you actually have to give an explanation. You can start with explaining the difference between the two different examples you give: a@update=b a@update b and what the default __iat_update__ does. Is this supposed to be unique to update, or will there be an infinite number of special dunder methods for arbitrary method names? a@spam=b # calls __iat_spam__ Does this apply to only dicts, or is it applicable to every object?
New syntax which breaks existing code is not likely to be accepted without a *really* good reason. -- Steven
Steven D'Aprano asked me to explain the difference between
a@update=b a@update b
It is the same as the difference between a += b a + b The one uses a._iadd__(b), a.__add__(b) and so on. For the other, replace 'add' by 'at_update', and '+' by '@update'. By the way, the expressions a@update b a @update b are to be equivalent, but a @ update b is something else (and not valid Python).
Is this supposed to be unique to update, or will there be an infinite number of special dunder methods for arbitrary method names?
There are (many) numbers between 1 and infinity. If a programmer defines __at_python__ on type(guido) then guido@python will have semantics. Steve wrote:
New syntax which breaks existing code is not likely to be accepted without a *really* good reason.
I'd like to see some real-world examples of code that would be broken. As I recall, most or all of the code examples in the python-ideas thread on the '@' operator actually write ' @ '. So they would be good. https://mail.python.org/pipermail/python-ideas/2014-March/027053.html And if otherwise a good idea, we can use the from __future__ trick to maintain compatibility. -- Jonathan
On Sun, Mar 10, 2019 at 3:37 AM Jonathan Fine <jfine2358@gmail.com> wrote:
Can you start by actually defining the change to the grammar? You've casually thrown out the comment that there'll be breakage, without saying exactly what you're proposing to change. Currently, "x @ y" is defined as an operator, with the same precedence as other multiplication/division operators: https://github.com/python/cpython/blob/master/Grammar/Grammar#L106 If the actual Grammar file is too hard to work with, define in high level terms what you're trying to change, perhaps by referencing this table: https://docs.python.org/3/reference/expressions.html#operator-precedence ChrisA
On Sat, Mar 09, 2019 at 04:34:01PM +0000, Jonathan Fine wrote:
It already has meaning: it calls the @ operator with operands "guido" and "python".
The interpreter doesn't distinguish between "a @ b" and "a@b". Spaces around operators are always optional. The last thing we're going to do is repeat Ruby's design mistake of making code dependent on spaces around operators. Define a function in Ruby with a default value: def a(x=4) x+2 end and then evaluate the expressions: a + 1 a+ 1 a+1 a +1 The results you get will be 7, 7, 7 and 3. -- Steven
Steven D'Aprano wrote
The last thing we're going to do is repeat Ruby's design mistake of making code dependent on spaces around operators.
I'd say that Ruby's mistake was encouraging programmers to write code that was hard for human beings to read accurately and quickly. As we say in Python, readability counts. And that's part of the PEP process. Let me clarify the additions to the grammar. 'foo' is a valid Python variable name '@foo=' is to be a valid incremental assignment operator '@foo' is to be a valid Python binary operator For clarity, we keep '@' and '@=' as binary and incremental assignment operators. The worst possible example of ambiguity and incompatibility is perhaps a@b+1 which is valid Python both before and after, but with different syntax a @ (b + 1) # Before a @b (+1) # After To return to Steve's point. A natural example of code using the extended '@' syntax, which is hard to read accurately and quickly, would probably be fatal to this suggestion. Finally, in the discussion of the '@' operator (which Steve was part of), the point was made many times that using the '@' operator made the matrix code easier to read and understand. This was a major force in the proposal. A major reason for this was its alignment with standard mathematical notation. https://mail.python.org/pipermail/python-ideas/2014-March/027053.html I'm suggesting that the grammar allow us, if we wish, to write c = a @cup b for the union of two sets. And len(A @cup B) == len(A) + len(B) - len(A @cap B) is the very useful https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle Here, 'cup' and 'cap' are the visual names for the union and intersection operators for sets. -- Jonathan
On Sun, Mar 10, 2019 at 4:22 AM Jonathan Fine <jfine2358@gmail.com> wrote:
Actually, due to operator precedence, the current interpretation is: (a @ b) + 1 That's a massive compatibility break. You're making it so the presence of whitespace around an operator not just changes its precedence, but actually changes "b" from a value to a token. There's a huge difference between: x.upper and x+upper One of them looks up the name "upper" as an attribute of whatever object 'x' is, and the other evaluates "upper" in the current context (looking for a local or global variable, or a built-in). Atoms and values are fundamentally different; you can replace a simple name with an expression (since they're both values), but you can't do that with an atom: x+(dispatch["upper"]) # can do exactly the same thing as x+upper x.(dispatch["upper"]) # SyntaxError You're proposing to change the @ symbol from being like the first example to being like the second... but ONLY if there's the right pattern of whitespace. I hope that this has 0% chance of happening. You'll do better to pick some other symbol, such that you're giving meaning to something that is currently an error. At least that way, there won't be code that behaves drastically differently on 3.8 and 3.9. ChrisA
SUMMARY Acknowledgement of an error, and clarification of behaviour of '@b' vs '@ b'. I claim that once '@b' is accepted as an operator, the behaviour is perfectly natural and obvious. AN ERROR In a previous post, I mispoke. I should have written a@b+1 is valid Python before and after, but with different syntax. (a @ b) + 1 # Before - operator is '@'. a @b (+1) # After - operator is '@b'. Chris Angelico kindly pointed out that my Before value was wrong. Thank you, Chris. WHITE SPACE AND OPERATORS Chris also correctly points that '@ b' is parses as the '@' operator followed by the identifier 'b' '@b' parses as above (BEFORE) '@b' parses as the '@b' operator (AFTER) He then correctly says that in my proposal the lack of whitespace after an operator can cause the operator to absorb a following identifier. However, something similar ALREADY happens in Python. >>> a = nota = True >>> not a False >>> nota True Today, whenever a Python operator ends in a letter, and is followed by an identifier, white space is or some other delimiter is required between the two. Python, rightly, refuse to guess that 'notary' might be 'not ary'. Here is another example >>> e = note = None >>> e is not e False >>> e is note True This is not quite what's happening with '@b'. With 'is not e' the following identifier 'e' absorbs the 'not' from the operator to create 'note'. And finally >>> False is not None True >>> False is (not None) False The 'natural language' operators appear in https://docs.python.org/3/reference/expressions.html#operator-precedence In my suggestion, '@' consumes for as long it can, first a letter, and then name characters. This is exactly the same as with 'a' or 'b'. I think this is a problem, but not nearly so bad as Chris suggests. Some people have argued that the proposed semantics for dict + dict are natural and obvious, once the behaviour of Python elsewhere is understood. I claim the same for '@b' and '@ b', once we allow '@b' as an operator (which was the whole purpose of the proposal. By the way, it's likely that most users won't know that '@' by itself is an operator, until they come to use matrices. -- Jonathan
On Sun, Mar 10, 2019 at 6:07 AM Jonathan Fine <jfine2358@gmail.com> wrote:
Python's grammar is defined in terms of tokens. There is a specific token 'not' which can be used in three ways: either 'not' followed by a valid expression, or as part of the operators 'not in' and 'is not', both of which are then followed (and preceded) by expressions. If the parser sees 'note', it doesn't see the token 'not'; it sees a NAME token. That's not the case with '@'. There is no way that '@foo' could be a single NAME token, because an at sign cannot be part of a NAME. There is a VAST difference, both to humans and to the parser, between "note" and "@e", and the fact that "not e" is different from "note" does not mean that "@e" can be different from "@ e". As Anders says, pick something that's not currently valid syntax and then you won't be up against this problem. ChrisA
I'm adopting an idea suggested by Anders and Chris. To allow us better to focus on the main idea and purpose, I've replaced '@' by '$' in the initial suggestion. And if the main idea is accepted, we can if needed have an secondary discussion regarding the details of the syntax. Here's the restatement. I've also changed the subject line. I've been thinking that it might be easier, in the long term, to make a big step and allow >>> a $update= b as valid Python. What do you think? (I hope it will look nicer once syntax highlighted.) For clarity, this would proceed via something like a.__idl_update__(b), and (a $update b) would be similarly defined. (Here, 'idl' stand for 'incremented dollar'.) -- Jonathan
A good starting point for discussing the main idea is: PEP 465 -- A dedicated infix operator for matrix multiplication https://www.python.org/dev/peps/pep-0465 Matrix multiplication is one of many special binary mathematical operators. PEP 465 successfully argues the merits of introducing a special operator for matrix multiplication. This thread starts from a discussion of the merits of binding dict.update to an operator. (For update, '+', '|' and '<<' the leading candidate symbols.) Matrices and linear algebra are not the only part of mathematics that is usually expressed with infix operators. Thus, I suggest that the main questions are: 1. In practice, how important are additional infix operators to the Python community? 2. Can we harmoniously extend Python to accommodate these new operators? Here, from PEP 465, are some highlights of the benefits. <quote> Infix @ dramatically improves matrix code usability at all stages of programmer interaction. A large proportion of scientific code is written by people who are experts in their domain, but are not experts in programming. For these kinds of users, whose programming knowledge is fragile, the existence of a transparent mapping between formulas and code often means the difference between succeeding and failing to write that code at all. </endquote> Most mathematical and scientific formulas can be written in LaTeX notation, which gives standard names for the infix operators mathematicians use. There is no transparent and obvious mapping from the present operators to those used in mathematics. https://docs.python.org/3/reference/lexical_analysis.html?#operators Using Unicode symbols for the math operators is probably unwise. Better, I suggest, is to use the LaTeX names. There is some evidence (the wish to bind dict.update to an infix operator) that outside of mathematics there is a demand for custom infix operators. -- Jonathan
This is a horrible idea. I proposed to Mr. Fine earlier that we adopt a << operator. d1 << d2 merges d2 into a copy of d1 and returns it, with keys from d2 overriding keys from d2. On Sat, Mar 9, 2019 at 4:50 PM Jonathan Fine <jfine2358@gmail.com> wrote:
On 09/03/2019 21:13, Jonathan Fine wrote:
I think it's horribly Perl-like, but I'll reserve judgement until you tell us exactly what you expect this to achieve. So far you appear to be trying to create random operators to no great purpose. What is the problem you are seeking to solve here, and why is it worse than the problems you are creating? -- Rhodri James *-* Kynesim Ltd
On Sat, 9 Mar 2019 at 18:22, Jonathan Fine <jfine2358@gmail.com> wrote:
You can use the infix module for that, which allows you to write - a @cup@ b - a &cup& b - a <<cup>> b - a ^cup^ b - a **cup** b - a /cup/ b - a %cup% b - a |cup| b Use that for a while, and if you like it, you can help the authors spread the word. - Pål
Anders Hovmöller wrote:
I don't understand what you mean. Can you provide examples that show the state of the dicts before and after and what the syntax would be the equivalent of in current python?
If a.__radd__ exists, then a += b is equivalent to a = a.__radd__(b) Similarly, if a.__iat_update__ exists then a @update= b would be equivalent to a = a.__iat_update__(b) Here's an implementation def __iat_update__(self, other): self.update(other) return self Thus, 'b' would be unchanged, and 'a' would be the same dictionary as before, but updated with 'b'. I hope this helps. -- Jonathan
On Sun, Mar 10, 2019 at 3:16 AM Jonathan Fine <jfine2358@gmail.com> wrote:
With something this long, how is it better from just writing: a = a.update_with(b) ? What's the point of an operator, especially if - by your own statement - it will backward-incompatibly change the language grammar (in ways that I've yet to understand, since you haven't really been clear on that)? ChrisA
Chris Angelico suggested that a = a.update_with(b) would be better than a @update= b One of the key points of += is that parent.child['toy'].wheel[3].speed += 1 increases the speed that that wheel by 1, without having to write parent.child['toy'].wheel[3].speed = parent.child['toy'].wheel[3].speed + 1 To answer Chris's other points. It not me, but Chris and Steve who want to bind dict.update to an operator, namely '+'. I'm suggested that if you do that, why not call the operator 'update'. Finally, we don't yet have any real idea how much difficulty the grammar change would cause. -- Jonathan
On Sun, Mar 10, 2019 at 3:46 AM Jonathan Fine <jfine2358@gmail.com> wrote:
No, we don't, because you have yet to say what the grammar change would BE. Changing language grammar is a big deal. You don't just say "oh, we should do this, and hey, it's gonna break code". Steven's proposal (not mine, btw, unless you meant some other Chris?) involves giving meaning to "x + y" for different values of x and y, but doesn't change the grammar at all. If you're proposing a completely new meaning for completely new syntax, *be clear* about what you are proposing. ChrisA
Jonathan Fine wrote:
One reason would be that '+' is short, whereas 'update' is long. A large part of the reason that common operations are written using infix operators is that the operator symbols used are very compact. That benefit disappears if your operator is an entire word. -- Greg
On Sat, Mar 9, 2019 at 4:14 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I suppose people bring up Haskell too much, but it does work in Haskell. People write things like (item `notElem` list) all the time and it's readable enough. In Haskell, though, it's sugar for (notElem item list), or notElem(item, list) in Pythonish syntax. In Python, it'd in most cases be sugar for a method call, in which the method name already appears in infix position, so the benefit is less clear. Given that Python's so-called augmented assignments are really mutating operations in disguise anyway (x op= y is not equivalent to x = x op y when x is mutable), I don't see any advantage of a new assignment syntax over the existing mutating methods. I.e., instead of x @update= y, you can just write x.update(y).
It might also be worth considering YAML's own dict merge operator, the "<<" operator, as in https://yaml.org/type/merge.html as this is the existing Python's shift operator added to dict and will require no change to the synatx:: a = a << b Meitham On 03/10, Chris Angelico wrote:
-- Meitham Jamaa http://meitham.com GPG Fingerprint: 3934D0B2
I really like this suggestion. It captures the asymmetry, since we could have a = a >> b to merge with the other dictionary's keys taking precedence. My instinct is that a = a << b would take b's values when keys collide and a = a >> b would take a's values when keys collide. I'd be very interested to know if this matches most peoples' intuitions. On Sat, 9 Mar 2019 at 18:44, Meitham Jamaa <m@meitham.com> wrote:
Maybe it's just the C++ IO piping that makes me like it, but these actually seem intuitive to me, whereas `+` or even `|` leaves me queasy. On Sat, Mar 9, 2019 at 7:40 PM Ian Foote <ian@feete.org> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
Are you all REALY=LU proposing more operators? Adding @ made sense because there was an important use case for which there was no existing operator to use. But in this case, we have + and | available, both of which are pretty good options. Finally, which dicts are a very important ue ase, do we want to add an operator just for that? What would it mean for sets, for instance? I have to say, the whole discussion seems to me to be a massive bike-shedding exercise -- the original proposal was to simply define + for dicts. It's totally reasonable to not like that idea at all, or to propose that | is a better option, but this has really gone off the rails! I guess I say that because this wasn't started with a critical use-case that really needed a solution, but rather: "the + operator isn't being used for dicts, why not make a semi-common operation easily available" So my opinion, which I'm re-stating: using + to merge dicts is simple, non-disruptive, and unlikely to really confuse anyone - so why not? ( | would be OK, too, though I think a tad less accessible to newbies) But I don't think that having an operator to merge dicts is a critical use-case worth of adding a new operator or new syntax to the to the language. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Sat, Mar 09, 2019 at 03:33:25PM +0000, Jonathan Fine wrote:
Are we supposed to know what that does?
For clarity, this would proceed via a.__iat_update__(b), and (a @update b) would be similarly defined.
For an explanation to be clear, you actually have to give an explanation. You can start with explaining the difference between the two different examples you give: a@update=b a@update b and what the default __iat_update__ does. Is this supposed to be unique to update, or will there be an infinite number of special dunder methods for arbitrary method names? a@spam=b # calls __iat_spam__ Does this apply to only dicts, or is it applicable to every object?
New syntax which breaks existing code is not likely to be accepted without a *really* good reason. -- Steven
Steven D'Aprano asked me to explain the difference between
a@update=b a@update b
It is the same as the difference between a += b a + b The one uses a._iadd__(b), a.__add__(b) and so on. For the other, replace 'add' by 'at_update', and '+' by '@update'. By the way, the expressions a@update b a @update b are to be equivalent, but a @ update b is something else (and not valid Python).
Is this supposed to be unique to update, or will there be an infinite number of special dunder methods for arbitrary method names?
There are (many) numbers between 1 and infinity. If a programmer defines __at_python__ on type(guido) then guido@python will have semantics. Steve wrote:
New syntax which breaks existing code is not likely to be accepted without a *really* good reason.
I'd like to see some real-world examples of code that would be broken. As I recall, most or all of the code examples in the python-ideas thread on the '@' operator actually write ' @ '. So they would be good. https://mail.python.org/pipermail/python-ideas/2014-March/027053.html And if otherwise a good idea, we can use the from __future__ trick to maintain compatibility. -- Jonathan
On Sun, Mar 10, 2019 at 3:37 AM Jonathan Fine <jfine2358@gmail.com> wrote:
Can you start by actually defining the change to the grammar? You've casually thrown out the comment that there'll be breakage, without saying exactly what you're proposing to change. Currently, "x @ y" is defined as an operator, with the same precedence as other multiplication/division operators: https://github.com/python/cpython/blob/master/Grammar/Grammar#L106 If the actual Grammar file is too hard to work with, define in high level terms what you're trying to change, perhaps by referencing this table: https://docs.python.org/3/reference/expressions.html#operator-precedence ChrisA
On Sat, Mar 09, 2019 at 04:34:01PM +0000, Jonathan Fine wrote:
It already has meaning: it calls the @ operator with operands "guido" and "python".
The interpreter doesn't distinguish between "a @ b" and "a@b". Spaces around operators are always optional. The last thing we're going to do is repeat Ruby's design mistake of making code dependent on spaces around operators. Define a function in Ruby with a default value: def a(x=4) x+2 end and then evaluate the expressions: a + 1 a+ 1 a+1 a +1 The results you get will be 7, 7, 7 and 3. -- Steven
Steven D'Aprano wrote
The last thing we're going to do is repeat Ruby's design mistake of making code dependent on spaces around operators.
I'd say that Ruby's mistake was encouraging programmers to write code that was hard for human beings to read accurately and quickly. As we say in Python, readability counts. And that's part of the PEP process. Let me clarify the additions to the grammar. 'foo' is a valid Python variable name '@foo=' is to be a valid incremental assignment operator '@foo' is to be a valid Python binary operator For clarity, we keep '@' and '@=' as binary and incremental assignment operators. The worst possible example of ambiguity and incompatibility is perhaps a@b+1 which is valid Python both before and after, but with different syntax a @ (b + 1) # Before a @b (+1) # After To return to Steve's point. A natural example of code using the extended '@' syntax, which is hard to read accurately and quickly, would probably be fatal to this suggestion. Finally, in the discussion of the '@' operator (which Steve was part of), the point was made many times that using the '@' operator made the matrix code easier to read and understand. This was a major force in the proposal. A major reason for this was its alignment with standard mathematical notation. https://mail.python.org/pipermail/python-ideas/2014-March/027053.html I'm suggesting that the grammar allow us, if we wish, to write c = a @cup b for the union of two sets. And len(A @cup B) == len(A) + len(B) - len(A @cap B) is the very useful https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle Here, 'cup' and 'cap' are the visual names for the union and intersection operators for sets. -- Jonathan
On Sun, Mar 10, 2019 at 4:22 AM Jonathan Fine <jfine2358@gmail.com> wrote:
Actually, due to operator precedence, the current interpretation is: (a @ b) + 1 That's a massive compatibility break. You're making it so the presence of whitespace around an operator not just changes its precedence, but actually changes "b" from a value to a token. There's a huge difference between: x.upper and x+upper One of them looks up the name "upper" as an attribute of whatever object 'x' is, and the other evaluates "upper" in the current context (looking for a local or global variable, or a built-in). Atoms and values are fundamentally different; you can replace a simple name with an expression (since they're both values), but you can't do that with an atom: x+(dispatch["upper"]) # can do exactly the same thing as x+upper x.(dispatch["upper"]) # SyntaxError You're proposing to change the @ symbol from being like the first example to being like the second... but ONLY if there's the right pattern of whitespace. I hope that this has 0% chance of happening. You'll do better to pick some other symbol, such that you're giving meaning to something that is currently an error. At least that way, there won't be code that behaves drastically differently on 3.8 and 3.9. ChrisA
SUMMARY Acknowledgement of an error, and clarification of behaviour of '@b' vs '@ b'. I claim that once '@b' is accepted as an operator, the behaviour is perfectly natural and obvious. AN ERROR In a previous post, I mispoke. I should have written a@b+1 is valid Python before and after, but with different syntax. (a @ b) + 1 # Before - operator is '@'. a @b (+1) # After - operator is '@b'. Chris Angelico kindly pointed out that my Before value was wrong. Thank you, Chris. WHITE SPACE AND OPERATORS Chris also correctly points that '@ b' is parses as the '@' operator followed by the identifier 'b' '@b' parses as above (BEFORE) '@b' parses as the '@b' operator (AFTER) He then correctly says that in my proposal the lack of whitespace after an operator can cause the operator to absorb a following identifier. However, something similar ALREADY happens in Python. >>> a = nota = True >>> not a False >>> nota True Today, whenever a Python operator ends in a letter, and is followed by an identifier, white space is or some other delimiter is required between the two. Python, rightly, refuse to guess that 'notary' might be 'not ary'. Here is another example >>> e = note = None >>> e is not e False >>> e is note True This is not quite what's happening with '@b'. With 'is not e' the following identifier 'e' absorbs the 'not' from the operator to create 'note'. And finally >>> False is not None True >>> False is (not None) False The 'natural language' operators appear in https://docs.python.org/3/reference/expressions.html#operator-precedence In my suggestion, '@' consumes for as long it can, first a letter, and then name characters. This is exactly the same as with 'a' or 'b'. I think this is a problem, but not nearly so bad as Chris suggests. Some people have argued that the proposed semantics for dict + dict are natural and obvious, once the behaviour of Python elsewhere is understood. I claim the same for '@b' and '@ b', once we allow '@b' as an operator (which was the whole purpose of the proposal. By the way, it's likely that most users won't know that '@' by itself is an operator, until they come to use matrices. -- Jonathan
On Sun, Mar 10, 2019 at 6:07 AM Jonathan Fine <jfine2358@gmail.com> wrote:
Python's grammar is defined in terms of tokens. There is a specific token 'not' which can be used in three ways: either 'not' followed by a valid expression, or as part of the operators 'not in' and 'is not', both of which are then followed (and preceded) by expressions. If the parser sees 'note', it doesn't see the token 'not'; it sees a NAME token. That's not the case with '@'. There is no way that '@foo' could be a single NAME token, because an at sign cannot be part of a NAME. There is a VAST difference, both to humans and to the parser, between "note" and "@e", and the fact that "not e" is different from "note" does not mean that "@e" can be different from "@ e". As Anders says, pick something that's not currently valid syntax and then you won't be up against this problem. ChrisA
I'm adopting an idea suggested by Anders and Chris. To allow us better to focus on the main idea and purpose, I've replaced '@' by '$' in the initial suggestion. And if the main idea is accepted, we can if needed have an secondary discussion regarding the details of the syntax. Here's the restatement. I've also changed the subject line. I've been thinking that it might be easier, in the long term, to make a big step and allow >>> a $update= b as valid Python. What do you think? (I hope it will look nicer once syntax highlighted.) For clarity, this would proceed via something like a.__idl_update__(b), and (a $update b) would be similarly defined. (Here, 'idl' stand for 'incremented dollar'.) -- Jonathan
A good starting point for discussing the main idea is: PEP 465 -- A dedicated infix operator for matrix multiplication https://www.python.org/dev/peps/pep-0465 Matrix multiplication is one of many special binary mathematical operators. PEP 465 successfully argues the merits of introducing a special operator for matrix multiplication. This thread starts from a discussion of the merits of binding dict.update to an operator. (For update, '+', '|' and '<<' the leading candidate symbols.) Matrices and linear algebra are not the only part of mathematics that is usually expressed with infix operators. Thus, I suggest that the main questions are: 1. In practice, how important are additional infix operators to the Python community? 2. Can we harmoniously extend Python to accommodate these new operators? Here, from PEP 465, are some highlights of the benefits. <quote> Infix @ dramatically improves matrix code usability at all stages of programmer interaction. A large proportion of scientific code is written by people who are experts in their domain, but are not experts in programming. For these kinds of users, whose programming knowledge is fragile, the existence of a transparent mapping between formulas and code often means the difference between succeeding and failing to write that code at all. </endquote> Most mathematical and scientific formulas can be written in LaTeX notation, which gives standard names for the infix operators mathematicians use. There is no transparent and obvious mapping from the present operators to those used in mathematics. https://docs.python.org/3/reference/lexical_analysis.html?#operators Using Unicode symbols for the math operators is probably unwise. Better, I suggest, is to use the LaTeX names. There is some evidence (the wish to bind dict.update to an infix operator) that outside of mathematics there is a demand for custom infix operators. -- Jonathan
This is a horrible idea. I proposed to Mr. Fine earlier that we adopt a << operator. d1 << d2 merges d2 into a copy of d1 and returns it, with keys from d2 overriding keys from d2. On Sat, Mar 9, 2019 at 4:50 PM Jonathan Fine <jfine2358@gmail.com> wrote:
On 09/03/2019 21:13, Jonathan Fine wrote:
I think it's horribly Perl-like, but I'll reserve judgement until you tell us exactly what you expect this to achieve. So far you appear to be trying to create random operators to no great purpose. What is the problem you are seeking to solve here, and why is it worse than the problems you are creating? -- Rhodri James *-* Kynesim Ltd
On Sat, 9 Mar 2019 at 18:22, Jonathan Fine <jfine2358@gmail.com> wrote:
You can use the infix module for that, which allows you to write - a @cup@ b - a &cup& b - a <<cup>> b - a ^cup^ b - a **cup** b - a /cup/ b - a %cup% b - a |cup| b Use that for a while, and if you like it, you can help the authors spread the word. - Pål
participants (13)
-
Anders Hovmöller -
Ben Rudiak-Gould -
Chris Angelico -
Christopher Barker -
David Mertz -
Greg Ewing -
Ian Foote -
James Lu -
Jonathan Fine -
Meitham Jamaa -
Pål Grønås Drange -
Rhodri James -
Steven D'Aprano