Reverse assignment operators (=+, =-, =*, =/, =//, =**, =%)
Hello, I would like to propose some reversed assignment operators. Python already has some assignment operators: a += c is the same as a = a + c a -= c is the same as a = a - c a *= c is the same as a = a * c a /= c is the same as a = a / c a //= c is the same as a = a // c a %= c is the same as a = a % c a **= c is the same as a = a ** c What I would like to propose is the creation of the reverse: a =+ c is the same as a = c + a a =- c is the same as a = c - a a =* c is the same as a = c * a a =/ c is the same as a = c / a a =// c is the same as a = c // a a =% c is the same as a = c % a a =** c is the same as a = c ** a For addition (+= and =+), multiplication (*= and =*) and subtraction (-= and =-) of numbers it would give the same result, but addition (+= and =+) and multiplication (*= and =*) when used with strings return a different result. All the other operations (=/, =%, =** and =//) return a different result. I think it is simple and easy to understand, therefore pythonic. :) Best regards, JM
On Sat, Nov 12, 2016 at 10:08 PM João Matos <jcrmatos@gmail.com> wrote:
What I would like to propose is the creation of the reverse: a =+ c is the same as a = c + a
a =+5 already means a becomes 5
a =- c is the same as a = c - a
a =-5 already means a becomes -5 Elazar
Hello, What I propose is not a = +5 (see the space between the = and + sign) but a =+ 5 (no space between the = and + sign, and there is a space between + and 5) Exactly the same notation (only reversed) as the current assignment operators. Best regards, JM On 12-11-2016 20:15, אלעזר wrote:
On 2016-11-12 12:18, João Matos wrote:> Hello,
The thing is that a =+ 5 is already valid syntax, and means the same as a = +5. You don't need spaces around operators in Python. So your proposal would have the change the behavior of existing syntax, which pretty much makes it a nonstarter. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On 2016-11-12 18:08, João Matos wrote:
a =- c is the same as a = c - a
This would break compatibility. a =- c is a = -c -- Bernardo Sulzbach http://www.mafagafogigante.org/ mafagafogigante@gmail.com
I think a stronger argument to reject this (even for Python 4, and even if we could somehow get over the syntactic ambiguity) is that it's confusing. `+=` and `=+` are just visually too close, and the distinction is not all that intuitive. At least for `a += b` the expansion to `a = a + b` is a simple contraction. But for the proposed `a =+ b` the expansion would be a transposition: `a = b + a`. Given the uncommon use case this just isn't worth the potential confusion. (Also, there are other possible meanings one could guess for `a =+ b`, given the meaning of `a += b`. E.g., because it's the exact mirror image, you could guess that it means `b = a + b`.) On Sat, Nov 12, 2016 at 12:23 PM, João Matos <jcrmatos@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On 12 November 2016 at 21:08, João Matos <jcrmatos@gmail.com> wrote:
A good syntax example: a = sum (a, c) a = mul (a, c) a = div (a, c) Another good syntax, I'm not a fan of, but at least intuitive and learnt in school: a = a + c a = a * c a = a / c Bad syntax, not readable: a += c a -= c a *= c And your proposal: slightly different looking, but as bad as above, at least I don't see any improvement of the look, actually it makes even harder to see the beginning of the right part of assignment, so IMO it is merely worsening of already bad syntax. As for me, I would even prohibit all these += for the sake of readability. Mikhail
On Sun, Nov 13, 2016 at 4:00 PM, Mikhail V <mikhailwas@gmail.com> wrote:
Except that there is no "div" in python 3.x, and "sum(a, c)" does not add "a" and "c".
How is using a function better than using an operator? Especially considering the ambiguity issues you just demonstrated. The fact the operator version is part of the core language while the function versions aren't even builtins (they are in the "operator" module) suggests to me that the function version is not the preferred version in Python.
What, specifically, is not readable about this syntax?
As for me, I would even prohibit all these += for the sake of readability.
Great, so I will have to make a copy of my 500 MB array every time I want to do a simple mathematical operation on it rather than being able to do the operation in-place.
On 14 November 2016 at 00:08, Todd <toddrjen@gmail.com> wrote:
It is kind of clear from the context, that I am speaking of syntax and not how things are working under the hood, or? If a compiler cannot optimize "a = a + 1" into an in-place operation, that is misfortune. In numpy one can define in-place calulations, if I am not mistaken, with np.vectorize. A better option would be to support unary operators so the user can write directly without assignment: inc (a, 1) Would mean in-place increment "a" with 1
Well, if you find this syntax readable, then I am afraid we are on the opposite sides of barricades ;) Mikhail
On Mon, Nov 14, 2016 at 01:19:30AM +0100, Mikhail V wrote: [...]
A good syntax example:
a = sum (a, c)
There is a reason why mathematicians and accountants use symbols as a compact notation for common functions, instead of purely functional notation. Here is a real-world example, the formula for compound interest: A = P*(1+r/100)**n Compare that to: A = mul(P, pow(sum(1, div(r, 100)), n)) Even Reverse Polish Notation is easier to read than that series of functions: P 1 r 100 / + n ** * Don't misunderstand me, functions are important, and over-use of cryptic symbols that are only meaningful to an expert will hurt readability and maintainability of code. But people have needed to do arithmetic for over six thousand years, and there is no good substitute for compact operators for basic operations. [...]
That's not how Python works. Or at least, not without an extremely powerful and smart compiler, like PyPy. In Python, integers (and floats) are immutable objects. They have to be immutable, otherwise you would have things like this: x = 1 y = x x = x + 1 # changes the object 1 in place print(y*10) # expect 10, but get 20 That's how lists work, because they are mutable: py> x = [1] py> y = x py> x[0] = x[0] + 1 py> print(y[0]*10) # expect 1*10 = 10 20 A "sufficiently smart" compiler can work around this, as PyPy does under some circumstances, but you shouldn't expect this optimization to be simple. [...]
That is impossible with Python's executation model, in particular the "pass by object sharing" calling convention. I expect that this would require some extremely big changes to the way the compiler works, possibly a complete re-design, in order to allow pass by reference semantics. The biggest problem is that even if the compiler could choose between calling conventions, it would have to make that decision at runtime. It won't know until runtime that inc() requires pass by reference. That would make function calls even slower than they are now. -- Steve
On 14 November 2016 at 12:16, Steven D'Aprano <steve@pearwood.info> wrote:
I agree. Actually I meant that both are good examples. In some cases I still find function-style better, so this: A = P*(1+r/100)**n I would tend to write it like: A = P * pow((1 + r/100) , n) For me it is slightly more readable, since ** already makes the equation inconsistent. And of course you cannot use glyphs for all possible operations, it will be total mess. Most of problems with function-style equations come from limitations of representation so for Courier font for examples the brackets are too small and makes it hard to read. With good font and rendering there no such problems. Some equation editors allow even different sized brackets - the outer extend more and more when I add nested equations, so it looks way better.
Thanks a lot for a great explanation! So for current Python behavior, writing x = x + 1 and x += 1 Would mean the same in runtime? Namely creates a copy and assigns the value x+1 to x. Or there is still some overhead at parsing stage? Then however I see even less sense in using such a shortcut, it would be good to dismiss this notation, since it makes hard to read the code. As for Numpy, it uses anyway its own approaches, so in-place should use own syntax, e.g. like numpy.sum(x_, a) to do it in-place. Mikhail
Currently, Numpy takes advantage of __iadd__ and friends by performing the operation in-place; there is no copying or other object created. Numpy is very thinly C, for better and worse (which is also likely where the += syntax came from). If you're doing vast amounts of numeric computation, it quickly pays off to learn a little about how C likes to store arrays in memory and that doing things like creating a whole new array of the same size and having to free up the old one for every operation isn't a great idea. I strongly dislike the notion that I'd have to use arbitrary function calls to add one to an entire array, or multiply it by 2, or add it to another array, etc. As a minor nit, np.vectorize doesn't make in-place functions, it simply makes a naive, element-wise function act as though it was vectorized. The name is unfortunate, because it does nothing to speed it up, and it usually slows it down (because of that layer). Reworking code to avoid copying large arrays by doing in-place operations is the preferred method, though not always possible. Nick On Mon, Nov 14, 2016 at 12:21 PM, Mikhail V <mikhailwas@gmail.com> wrote:
On 14 November 2016 at 19:57, Nick Timkovich <prometheus235@gmail.com> wrote:
I can understand you good. But imagine, if Numpy would allow you to simply write: A = A + 1 Which would bring you directly to same internal procedure as A += 1. So it does not currently, why? I've tested now A += 99 against A = A + 99 and there is indeed a 30% speed difference. So it functions different. Would you then still want to write += ? I never would. Also I think to implement this syntax would be almost trivial, it should just take the A = A part and do the rest as usual. And this for equally sized arrays: A = A + B Should just add B values to A values in-place. Now it gives me also ~30% speed difference compared to A += B. Mikhail
On Mon, Nov 14, 2016 at 3:42 PM, Mikhail V <mikhailwas@gmail.com> wrote:
First, because the language doesn't allow it. But more fundamentally, sometimes we don't want A = A + 1 to be the same as A += 1. Making a copy is sometimes what you want. Having both versions lets you control when you make a copy rather than being forced to always make a copy or always not.
The Python language doesn't allow it. numpy can only work with the information provided to it be the language, and the information needed to do that sort of thing is not provided to classes. Nor should it in my opinion, this is one of those fundamental operations that I think absolutely must be consistent. What you are talking about is an enormous backwards-compatibility break. It is simply not going to happen, it would break every mutable class. And you still have not provided any reason we should want to do it this way.
On 14 November 2016 at 21:52, Todd <toddrjen@gmail.com> wrote:
we don't want A = A + 1 to be the same as A += 1 "
This sounds quite frustrating for me, what else could this be but A += 1? Would I want a copy with same name? How that will make sense? Making a copy is how it works now, ok. But even now there are differences, eg: A = B [1:10] is not a copy but a reference creation operation. So there are different meanings of =.
I don't want to break anything, no. I don't have such deep knowledge of all cases, but that is very interesting how actually it must break *everything*. Say I scan through lines and find "= " and then see that on the left is an numpy array, which already indicates a unusual variable. And if look on the right side and first argument is again A, cannot I decide to make an exception and redirect it to A.__add__() or what must be there. It does already do an overloading of all these "+=" for numpy. so what will fundamentally break everything I am not sure.
And you still have not provided any reason we should want to do it this way.
If let alone numpy for now and take only numeric Python variables, I find the += syntax very bad readable. I must literally stop and lean towards the monitor to decipher the line. And I am not even telling that you *cannot* do it with all possible operations: there are standard math, ORing ,XORing, conditionals, bitshifting etc. Is not this obvious? And what other reason apart from readability there can be at all? Again, how about TOOWTDI principle? some write a = a + 1, some a += 1 ? It does not add any pleasure for reading code.
You shouldn't be using a variable-width font for coding anyway. That is going to cause all sorts of problems (indentation not matching up, for example). You should use a fixed-width font.
No no, I should not use monowidth fonts and nobody ever should. Most of those "indentation" problems are just though-out, All you need is IDE with good tabulation support. The readability difference between monowidth and real fonts is *huge*. Currently I am forced to use monowidth, since it is VIM's limitation and it bothers me a lot. Exception is tables with numbers, but those should not be inputed in same manner as strings, it is other medium type. Mikhail
This is just a matter of understanding the actual semantics of Python, which is not the same as other languages such as C. In Python the "assignment operator" is a way of *binding a value*. So writing: a = some_expression Is ALWAYS and ONLY binding the value of `some_expression` to the name `a`. The fact that `a` might occur within `some_expression` is irrelevant to these semantics (although it *does* mean that subsequent code won't use `a` to refer to whatever value it used to have. In clear and distinct contrast, "augmented assignment" is a way of calling a method on an object. So writing: a += some_expression Means EXACTLY: a.__iadd__(some_expression) I recognize this can have the same effect if `__iadd__()` is not explicitly defined, and when that is true, it will fall back to using `__add__()` as the implementation. This is described in: https://www.python.org/dev/peps/pep-0203/ To see the difference, e.g.:
On Mon, Nov 14, 2016 at 2:13 PM, Mikhail V <mikhailwas@gmail.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On 14 November 2016 at 22:13, Mikhail V <mikhailwas@gmail.com> wrote:
For a mutable class, A = A + something is fundamentally different from A += something. Before proposing fundamental changes to Python's semantics, please make sure that you at least understand those semantics first, and explain why your proposal is justified. There are a lot of people on this list, and the cumulative time spent reading your posts is therefore quite significant. You owe it to all those readers to ensure that your proposals are at least feasible *in the context of the Python language*. Paul
On 15 November 2016 at 00:34, Paul Moore <p.f.moore@gmail.com> wrote:
Ok I am calmed down already. But how do you jump to lists already? I started an example with integers. I just want to increment an integer and I don't want to see any += in my code, it strains me. And that is exact reason I hate C syntax. Then Todd started about Numpy arrays. Ok, I just commented what I find for clearer syntax with array increment. And now lists, mutable classes... I don't use classes in my programs. I could propose something but how, if we mix everything in one big pile? Then my proposal is to make typed variables first, so I could at least consider use cases. Mikhail
Mikhail
Mikhail, what Paul probably means here is that python 'operators' are actually 'syntactic sugar' for functions (it is not recommended to call these functions directly, but it is possible): e.g. if you use 'a = a + 1', python under the hood interprets it as 'a = a.__add__(1)', but if you use 'a += 1' it uses 'a.__iadd__(1)'. All python operators are implementable under functions, which is part of the spec. For integers or strings, that may not be as visible to the end user, but if you want to change the behaviour of operators, please first look at the docs [1] and try to understand them, and understand why they exists (inconclusive examples: [2). I hope that this clears up the misconceptions you may have about how python operators work. -Matthias [1]: https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types [2]: e.g. to make your library more efficient (like the operators in numpy for more efficient array operations), or adding operator functionality for numerical classes (like the decimal.Decimal class).
On 15 November 2016 at 02:59, Matthias welp <boekewurm@gmail.com> wrote:
More specifically, the meaning of the syntax a = a + b does not depend on the types of the operators - it always means a = a.__add__(b) (with some added complexity for __radd__). So you can't make it do something different "for integers" because at this stage of interpretation, Python hasn't looked at the types of the objects in the expression. Hence my example, which uses lists to show an "obvious" case where a = a + b and a += b differ. It's harder to show a convincing example when a has an immutable type like int, because the fundamental difference between the 2 constructs is how they treat mutable values. If you're proposing a = a + b to introspect at runtime the type of a, and produce different bytecode depending on the answer, you're proposing a fundamental change to the runtime semantics of Python (such that the resulting language is arguably not even Python any more). That's not to say it can't be done, just that it's not in scope for "ideas about new features for Python" in any practical sense. Paul
Mikhail V writes:
But how do you jump to lists already?
Because "+" and "+=" are operators which may be defined for any objects. Paul explained *why* he chose to to do that elsewhere. My point is that the semantics "a += b" *is* "type(a).__iadd__(a, b)" is true for all objects. It is a fundamental consistency in Python. It may make what one might consider odd behavior fall into its proper place. Thus, in explaining this kind of thing it is often useful (YMMV) to "jump" to a different type that supports the same behavior to see how a proposed change can cause inconsistency.
If you *really* want to limit this to *integers* in *your* code, that's easy enough. Just don't use it. If by "my code" you mean "code I might have to read"[1], you're just out of luck, even for the very special case of integers. (1) Guido himself preferred *not* to have augmented assignment way back when, but bowed to popular demand. You are in a tiny minority, opposing the preference of a large and loud majority. (2) Guido could remove those operators, but it would break a lot of existing code that uses them, and that would not be acceptable.[2] (3) It would be a gratuitous special case, a fundamental inconsistency in Python semantics. So it's not just calming down. You need to understand that if you want to use a popular language with batteries included and a huge range of battle-tested very useful add-ons, you're going to have to accept that you will often be reading code that you would refuse to write unless forced to by court order. You see, you're wasting your own time here by trying to explain. It's just your preference, which in this matter is as valid as any other *so no explanation is necessary*. But the opinion that "+=" for integers is useful and readable was the majority when it was installed, and backward compability rules out changing it now. And I really don't think you have to worry about "=+". Footnotes: [1] And IMHO it is Pythonic to interpret "in my code" that way. Ie, it is Pythonic to be readable to as many Python developers as possible. But it would smooth discussion to distinguish the "my code" that you *write* from the "my code" that you *read* in your posts. (I guess English is a second language for you, it may not be worth it and *we* will adapt. But it does help, if you would like to try.) [2] Theoretically, he has the power. But he also has promised no such backward incompatible changes "ever again", so practically impossible. Yes, IMO this would be a breaking change big enough to compare to Unicode in 3.0.
Well, yes. But it is defined in particular ways in the built in types and the stdlib. And those ways are consistent-- I.e. They do the same thing as __add__. And we REALLY wouldn't want it any other way. But the __i*__ operators are a bit of a wart -- they mean different things for mutable and immutable types. This is because they were added to satisfy two use-cases: Syntactic sugar for common and simple operations like incrementing an integer. Compact syntax for in-place operations. As numbers are immutable, you can't have in-place operations. Period. So it was handy to use the same notation for both. And I don't think anyone wanted to add TWO new sets of operators. This dual use does cause confusion occasionally, but not that often in practice. -CHB
On 15 November 2016 at 14:04, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Mikhail V writes:
But how do you jump to lists already?
First, thank you for the input, and time spent to clear things here. I must admit I cannot "jump" for this time. Not only because I don't know inner nature of *all* possible implementions and cannot analyse them all in coverable amount of time, but also because I cannot draw logical connection between "appending to a list" and "incrementing an integer value" expressings, even if for some reason we want to use same syntax for both. Otherwise it would make extremely hard to contribute ideas for syntax improvement from not "python-dev" people. Of course only if one wants to recieve such ideas.
I tend to use word *my* since I can carry responsibility only for things I know from my experience. I hear often *we* in western political TV shows and this sounds not so good for my ears. But it is more of cultural thing I think. And I must read not only *my own* code of course. I personally never use += operators if I can.
Good to know that at least this being discussed, but not so good that majority wins. As for Python's numbers I personally would without hesitation remove the += and Co. I was jus curious how bad would it "break" things? Currently I cannot run my 2.7 code with 3.x Python, so I need to convert it. Probably not so obvious, how to convert the code with numeric operators automatically, since there are no type annotations in code... All in all, the issues of compatibility and majority with habits are there, I just think of "python-ideas" merely as discussion about possible future syntax improvements, simply because it relates to my specialization. But I also try to learn the technical part. To summarize things: I may be well losing my time explaining why += is bad, that is true. It is next to impossible to explain it, unless one can just *see* it. But apart from minority/majority, there is only one truth, so I would say, if "we" could consider such a slight rearrangement of how we *call* the things: Now I see many people call += a "syntactic sugar", and for me it is rather a "syntactic defect", so it is something on the opposite. One can say it is only opinion, ok, but if at least a pair of people will stop writing +=, then I could say, my mission was not a fail. Mikhail
The "just see it" isn't likely to go very far. My impression is that maybe 5% of people find '+=' and friends confusing or bothersome. Probably 15% are neutral (including me). And 80% think 'a += 1' is simply MORE readable and more easily understood than 'a = a+1'. Folks who come from C-family languages especially like the augmented assignment. It's very common for the inplace and binary operators to do the same thing. For those cases, it really is just syntax sugar. But enough importent libraries like NumPy make a distinction tray a highly value having __iadd__() and friends. On Nov 15, 2016 1:03 PM, "Mikhail V" <mikhailwas@gmail.com> wrote:
On 11/14/2016 03:42 PM, Mikhail V wrote:
If you take this file: test.py ---------------------------------------------- a = a + 1 a += 1 ---------------------------------------------- And you look at the bytecode it produces you get the following: ---------------------------------------------- $ python -m dis test.py 1 0 LOAD_NAME 0 (a) 3 LOAD_CONST 0 (1) 6 BINARY_ADD 7 STORE_NAME 0 (a) 2 10 LOAD_NAME 0 (a) 13 LOAD_CONST 0 (1) 16 INPLACE_ADD 17 STORE_NAME 0 (a) 20 LOAD_CONST 1 (None) 23 RETURN_VALUE ---------------------------------------------- That shows that the first and second lines are compiled _differently_. The point is that in the first line, numpy does not have the information necessary to know that "a + 1" will be assigned back to a. In the second case it does. This is presumably why they couldn't do the optimization you desire without large changes in cpython. On the second point, personally I prefer writing "a += 1" to "a = a + 1". I think it's clearer and would keep using it even if the two were equally efficient. But we are all allowed our opinions... Cheers, Thomas
On Mon, Nov 14, 2016 at 1:21 PM, Mikhail V <mikhailwas@gmail.com> wrote:
Okay, then make a proposal to get the operators included as built-ins. Once you have gotten that approved, then we can start talking about which syntax is better. But the idea that people should import a module to do basic mathematical operations is a non-starter.
You shouldn't be using a variable-width font for coding anyway. That is going to cause all sorts of problems (indentation not matching up, for example). You should use a fixed-width font. But if brackets are a problem, I don't see how using more brackets is a solution (which is the case with function calls). On the contrary, I would think you would want to minimize the number of brackets.
No, that is only the case for immutable types like floats. For mutable types like lists then x = x + y and x += y Are not the same thing. The first makes a new object, while the second does an in-place operation. Sometimes you want a new object, sometimes you don't. Having both versions allows you to control that.
First, there is no "_" suffix for variables that numpy could use. The syntax you are suggesting isn't possible without a major change to the basic python grammar and interpreter. Second, again, "sum(x, a)" does not add "x" and "a". Third, again, why? You still haven't given any reason we should prefer this syntax. If this is just your personal preference, please say that. But if you expect people to add a bunch of new built-ins, implement your new suffix syntax, and implement your massive changes to how python operations and function calls work, then you are going to need something more than personal preference.
On Sat, Nov 12, 2016 at 10:08 PM João Matos <jcrmatos@gmail.com> wrote:
What I would like to propose is the creation of the reverse: a =+ c is the same as a = c + a
a =+5 already means a becomes 5
a =- c is the same as a = c - a
a =-5 already means a becomes -5 Elazar
Hello, What I propose is not a = +5 (see the space between the = and + sign) but a =+ 5 (no space between the = and + sign, and there is a space between + and 5) Exactly the same notation (only reversed) as the current assignment operators. Best regards, JM On 12-11-2016 20:15, אלעזר wrote:
On 2016-11-12 12:18, João Matos wrote:> Hello,
The thing is that a =+ 5 is already valid syntax, and means the same as a = +5. You don't need spaces around operators in Python. So your proposal would have the change the behavior of existing syntax, which pretty much makes it a nonstarter. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On 2016-11-12 18:08, João Matos wrote:
a =- c is the same as a = c - a
This would break compatibility. a =- c is a = -c -- Bernardo Sulzbach http://www.mafagafogigante.org/ mafagafogigante@gmail.com
I think a stronger argument to reject this (even for Python 4, and even if we could somehow get over the syntactic ambiguity) is that it's confusing. `+=` and `=+` are just visually too close, and the distinction is not all that intuitive. At least for `a += b` the expansion to `a = a + b` is a simple contraction. But for the proposed `a =+ b` the expansion would be a transposition: `a = b + a`. Given the uncommon use case this just isn't worth the potential confusion. (Also, there are other possible meanings one could guess for `a =+ b`, given the meaning of `a += b`. E.g., because it's the exact mirror image, you could guess that it means `b = a + b`.) On Sat, Nov 12, 2016 at 12:23 PM, João Matos <jcrmatos@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On 12 November 2016 at 21:08, João Matos <jcrmatos@gmail.com> wrote:
A good syntax example: a = sum (a, c) a = mul (a, c) a = div (a, c) Another good syntax, I'm not a fan of, but at least intuitive and learnt in school: a = a + c a = a * c a = a / c Bad syntax, not readable: a += c a -= c a *= c And your proposal: slightly different looking, but as bad as above, at least I don't see any improvement of the look, actually it makes even harder to see the beginning of the right part of assignment, so IMO it is merely worsening of already bad syntax. As for me, I would even prohibit all these += for the sake of readability. Mikhail
On Sun, Nov 13, 2016 at 4:00 PM, Mikhail V <mikhailwas@gmail.com> wrote:
Except that there is no "div" in python 3.x, and "sum(a, c)" does not add "a" and "c".
How is using a function better than using an operator? Especially considering the ambiguity issues you just demonstrated. The fact the operator version is part of the core language while the function versions aren't even builtins (they are in the "operator" module) suggests to me that the function version is not the preferred version in Python.
What, specifically, is not readable about this syntax?
As for me, I would even prohibit all these += for the sake of readability.
Great, so I will have to make a copy of my 500 MB array every time I want to do a simple mathematical operation on it rather than being able to do the operation in-place.
On 14 November 2016 at 00:08, Todd <toddrjen@gmail.com> wrote:
It is kind of clear from the context, that I am speaking of syntax and not how things are working under the hood, or? If a compiler cannot optimize "a = a + 1" into an in-place operation, that is misfortune. In numpy one can define in-place calulations, if I am not mistaken, with np.vectorize. A better option would be to support unary operators so the user can write directly without assignment: inc (a, 1) Would mean in-place increment "a" with 1
Well, if you find this syntax readable, then I am afraid we are on the opposite sides of barricades ;) Mikhail
On Mon, Nov 14, 2016 at 01:19:30AM +0100, Mikhail V wrote: [...]
A good syntax example:
a = sum (a, c)
There is a reason why mathematicians and accountants use symbols as a compact notation for common functions, instead of purely functional notation. Here is a real-world example, the formula for compound interest: A = P*(1+r/100)**n Compare that to: A = mul(P, pow(sum(1, div(r, 100)), n)) Even Reverse Polish Notation is easier to read than that series of functions: P 1 r 100 / + n ** * Don't misunderstand me, functions are important, and over-use of cryptic symbols that are only meaningful to an expert will hurt readability and maintainability of code. But people have needed to do arithmetic for over six thousand years, and there is no good substitute for compact operators for basic operations. [...]
That's not how Python works. Or at least, not without an extremely powerful and smart compiler, like PyPy. In Python, integers (and floats) are immutable objects. They have to be immutable, otherwise you would have things like this: x = 1 y = x x = x + 1 # changes the object 1 in place print(y*10) # expect 10, but get 20 That's how lists work, because they are mutable: py> x = [1] py> y = x py> x[0] = x[0] + 1 py> print(y[0]*10) # expect 1*10 = 10 20 A "sufficiently smart" compiler can work around this, as PyPy does under some circumstances, but you shouldn't expect this optimization to be simple. [...]
That is impossible with Python's executation model, in particular the "pass by object sharing" calling convention. I expect that this would require some extremely big changes to the way the compiler works, possibly a complete re-design, in order to allow pass by reference semantics. The biggest problem is that even if the compiler could choose between calling conventions, it would have to make that decision at runtime. It won't know until runtime that inc() requires pass by reference. That would make function calls even slower than they are now. -- Steve
On 14 November 2016 at 12:16, Steven D'Aprano <steve@pearwood.info> wrote:
I agree. Actually I meant that both are good examples. In some cases I still find function-style better, so this: A = P*(1+r/100)**n I would tend to write it like: A = P * pow((1 + r/100) , n) For me it is slightly more readable, since ** already makes the equation inconsistent. And of course you cannot use glyphs for all possible operations, it will be total mess. Most of problems with function-style equations come from limitations of representation so for Courier font for examples the brackets are too small and makes it hard to read. With good font and rendering there no such problems. Some equation editors allow even different sized brackets - the outer extend more and more when I add nested equations, so it looks way better.
Thanks a lot for a great explanation! So for current Python behavior, writing x = x + 1 and x += 1 Would mean the same in runtime? Namely creates a copy and assigns the value x+1 to x. Or there is still some overhead at parsing stage? Then however I see even less sense in using such a shortcut, it would be good to dismiss this notation, since it makes hard to read the code. As for Numpy, it uses anyway its own approaches, so in-place should use own syntax, e.g. like numpy.sum(x_, a) to do it in-place. Mikhail
Currently, Numpy takes advantage of __iadd__ and friends by performing the operation in-place; there is no copying or other object created. Numpy is very thinly C, for better and worse (which is also likely where the += syntax came from). If you're doing vast amounts of numeric computation, it quickly pays off to learn a little about how C likes to store arrays in memory and that doing things like creating a whole new array of the same size and having to free up the old one for every operation isn't a great idea. I strongly dislike the notion that I'd have to use arbitrary function calls to add one to an entire array, or multiply it by 2, or add it to another array, etc. As a minor nit, np.vectorize doesn't make in-place functions, it simply makes a naive, element-wise function act as though it was vectorized. The name is unfortunate, because it does nothing to speed it up, and it usually slows it down (because of that layer). Reworking code to avoid copying large arrays by doing in-place operations is the preferred method, though not always possible. Nick On Mon, Nov 14, 2016 at 12:21 PM, Mikhail V <mikhailwas@gmail.com> wrote:
On 14 November 2016 at 19:57, Nick Timkovich <prometheus235@gmail.com> wrote:
I can understand you good. But imagine, if Numpy would allow you to simply write: A = A + 1 Which would bring you directly to same internal procedure as A += 1. So it does not currently, why? I've tested now A += 99 against A = A + 99 and there is indeed a 30% speed difference. So it functions different. Would you then still want to write += ? I never would. Also I think to implement this syntax would be almost trivial, it should just take the A = A part and do the rest as usual. And this for equally sized arrays: A = A + B Should just add B values to A values in-place. Now it gives me also ~30% speed difference compared to A += B. Mikhail
On Mon, Nov 14, 2016 at 3:42 PM, Mikhail V <mikhailwas@gmail.com> wrote:
First, because the language doesn't allow it. But more fundamentally, sometimes we don't want A = A + 1 to be the same as A += 1. Making a copy is sometimes what you want. Having both versions lets you control when you make a copy rather than being forced to always make a copy or always not.
The Python language doesn't allow it. numpy can only work with the information provided to it be the language, and the information needed to do that sort of thing is not provided to classes. Nor should it in my opinion, this is one of those fundamental operations that I think absolutely must be consistent. What you are talking about is an enormous backwards-compatibility break. It is simply not going to happen, it would break every mutable class. And you still have not provided any reason we should want to do it this way.
On 14 November 2016 at 21:52, Todd <toddrjen@gmail.com> wrote:
we don't want A = A + 1 to be the same as A += 1 "
This sounds quite frustrating for me, what else could this be but A += 1? Would I want a copy with same name? How that will make sense? Making a copy is how it works now, ok. But even now there are differences, eg: A = B [1:10] is not a copy but a reference creation operation. So there are different meanings of =.
I don't want to break anything, no. I don't have such deep knowledge of all cases, but that is very interesting how actually it must break *everything*. Say I scan through lines and find "= " and then see that on the left is an numpy array, which already indicates a unusual variable. And if look on the right side and first argument is again A, cannot I decide to make an exception and redirect it to A.__add__() or what must be there. It does already do an overloading of all these "+=" for numpy. so what will fundamentally break everything I am not sure.
And you still have not provided any reason we should want to do it this way.
If let alone numpy for now and take only numeric Python variables, I find the += syntax very bad readable. I must literally stop and lean towards the monitor to decipher the line. And I am not even telling that you *cannot* do it with all possible operations: there are standard math, ORing ,XORing, conditionals, bitshifting etc. Is not this obvious? And what other reason apart from readability there can be at all? Again, how about TOOWTDI principle? some write a = a + 1, some a += 1 ? It does not add any pleasure for reading code.
You shouldn't be using a variable-width font for coding anyway. That is going to cause all sorts of problems (indentation not matching up, for example). You should use a fixed-width font.
No no, I should not use monowidth fonts and nobody ever should. Most of those "indentation" problems are just though-out, All you need is IDE with good tabulation support. The readability difference between monowidth and real fonts is *huge*. Currently I am forced to use monowidth, since it is VIM's limitation and it bothers me a lot. Exception is tables with numbers, but those should not be inputed in same manner as strings, it is other medium type. Mikhail
This is just a matter of understanding the actual semantics of Python, which is not the same as other languages such as C. In Python the "assignment operator" is a way of *binding a value*. So writing: a = some_expression Is ALWAYS and ONLY binding the value of `some_expression` to the name `a`. The fact that `a` might occur within `some_expression` is irrelevant to these semantics (although it *does* mean that subsequent code won't use `a` to refer to whatever value it used to have. In clear and distinct contrast, "augmented assignment" is a way of calling a method on an object. So writing: a += some_expression Means EXACTLY: a.__iadd__(some_expression) I recognize this can have the same effect if `__iadd__()` is not explicitly defined, and when that is true, it will fall back to using `__add__()` as the implementation. This is described in: https://www.python.org/dev/peps/pep-0203/ To see the difference, e.g.:
On Mon, Nov 14, 2016 at 2:13 PM, Mikhail V <mikhailwas@gmail.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On 14 November 2016 at 22:13, Mikhail V <mikhailwas@gmail.com> wrote:
For a mutable class, A = A + something is fundamentally different from A += something. Before proposing fundamental changes to Python's semantics, please make sure that you at least understand those semantics first, and explain why your proposal is justified. There are a lot of people on this list, and the cumulative time spent reading your posts is therefore quite significant. You owe it to all those readers to ensure that your proposals are at least feasible *in the context of the Python language*. Paul
On 15 November 2016 at 00:34, Paul Moore <p.f.moore@gmail.com> wrote:
Ok I am calmed down already. But how do you jump to lists already? I started an example with integers. I just want to increment an integer and I don't want to see any += in my code, it strains me. And that is exact reason I hate C syntax. Then Todd started about Numpy arrays. Ok, I just commented what I find for clearer syntax with array increment. And now lists, mutable classes... I don't use classes in my programs. I could propose something but how, if we mix everything in one big pile? Then my proposal is to make typed variables first, so I could at least consider use cases. Mikhail
Mikhail
Mikhail, what Paul probably means here is that python 'operators' are actually 'syntactic sugar' for functions (it is not recommended to call these functions directly, but it is possible): e.g. if you use 'a = a + 1', python under the hood interprets it as 'a = a.__add__(1)', but if you use 'a += 1' it uses 'a.__iadd__(1)'. All python operators are implementable under functions, which is part of the spec. For integers or strings, that may not be as visible to the end user, but if you want to change the behaviour of operators, please first look at the docs [1] and try to understand them, and understand why they exists (inconclusive examples: [2). I hope that this clears up the misconceptions you may have about how python operators work. -Matthias [1]: https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types [2]: e.g. to make your library more efficient (like the operators in numpy for more efficient array operations), or adding operator functionality for numerical classes (like the decimal.Decimal class).
On 15 November 2016 at 02:59, Matthias welp <boekewurm@gmail.com> wrote:
More specifically, the meaning of the syntax a = a + b does not depend on the types of the operators - it always means a = a.__add__(b) (with some added complexity for __radd__). So you can't make it do something different "for integers" because at this stage of interpretation, Python hasn't looked at the types of the objects in the expression. Hence my example, which uses lists to show an "obvious" case where a = a + b and a += b differ. It's harder to show a convincing example when a has an immutable type like int, because the fundamental difference between the 2 constructs is how they treat mutable values. If you're proposing a = a + b to introspect at runtime the type of a, and produce different bytecode depending on the answer, you're proposing a fundamental change to the runtime semantics of Python (such that the resulting language is arguably not even Python any more). That's not to say it can't be done, just that it's not in scope for "ideas about new features for Python" in any practical sense. Paul
Mikhail V writes:
But how do you jump to lists already?
Because "+" and "+=" are operators which may be defined for any objects. Paul explained *why* he chose to to do that elsewhere. My point is that the semantics "a += b" *is* "type(a).__iadd__(a, b)" is true for all objects. It is a fundamental consistency in Python. It may make what one might consider odd behavior fall into its proper place. Thus, in explaining this kind of thing it is often useful (YMMV) to "jump" to a different type that supports the same behavior to see how a proposed change can cause inconsistency.
If you *really* want to limit this to *integers* in *your* code, that's easy enough. Just don't use it. If by "my code" you mean "code I might have to read"[1], you're just out of luck, even for the very special case of integers. (1) Guido himself preferred *not* to have augmented assignment way back when, but bowed to popular demand. You are in a tiny minority, opposing the preference of a large and loud majority. (2) Guido could remove those operators, but it would break a lot of existing code that uses them, and that would not be acceptable.[2] (3) It would be a gratuitous special case, a fundamental inconsistency in Python semantics. So it's not just calming down. You need to understand that if you want to use a popular language with batteries included and a huge range of battle-tested very useful add-ons, you're going to have to accept that you will often be reading code that you would refuse to write unless forced to by court order. You see, you're wasting your own time here by trying to explain. It's just your preference, which in this matter is as valid as any other *so no explanation is necessary*. But the opinion that "+=" for integers is useful and readable was the majority when it was installed, and backward compability rules out changing it now. And I really don't think you have to worry about "=+". Footnotes: [1] And IMHO it is Pythonic to interpret "in my code" that way. Ie, it is Pythonic to be readable to as many Python developers as possible. But it would smooth discussion to distinguish the "my code" that you *write* from the "my code" that you *read* in your posts. (I guess English is a second language for you, it may not be worth it and *we* will adapt. But it does help, if you would like to try.) [2] Theoretically, he has the power. But he also has promised no such backward incompatible changes "ever again", so practically impossible. Yes, IMO this would be a breaking change big enough to compare to Unicode in 3.0.
Well, yes. But it is defined in particular ways in the built in types and the stdlib. And those ways are consistent-- I.e. They do the same thing as __add__. And we REALLY wouldn't want it any other way. But the __i*__ operators are a bit of a wart -- they mean different things for mutable and immutable types. This is because they were added to satisfy two use-cases: Syntactic sugar for common and simple operations like incrementing an integer. Compact syntax for in-place operations. As numbers are immutable, you can't have in-place operations. Period. So it was handy to use the same notation for both. And I don't think anyone wanted to add TWO new sets of operators. This dual use does cause confusion occasionally, but not that often in practice. -CHB
On 15 November 2016 at 14:04, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Mikhail V writes:
But how do you jump to lists already?
First, thank you for the input, and time spent to clear things here. I must admit I cannot "jump" for this time. Not only because I don't know inner nature of *all* possible implementions and cannot analyse them all in coverable amount of time, but also because I cannot draw logical connection between "appending to a list" and "incrementing an integer value" expressings, even if for some reason we want to use same syntax for both. Otherwise it would make extremely hard to contribute ideas for syntax improvement from not "python-dev" people. Of course only if one wants to recieve such ideas.
I tend to use word *my* since I can carry responsibility only for things I know from my experience. I hear often *we* in western political TV shows and this sounds not so good for my ears. But it is more of cultural thing I think. And I must read not only *my own* code of course. I personally never use += operators if I can.
Good to know that at least this being discussed, but not so good that majority wins. As for Python's numbers I personally would without hesitation remove the += and Co. I was jus curious how bad would it "break" things? Currently I cannot run my 2.7 code with 3.x Python, so I need to convert it. Probably not so obvious, how to convert the code with numeric operators automatically, since there are no type annotations in code... All in all, the issues of compatibility and majority with habits are there, I just think of "python-ideas" merely as discussion about possible future syntax improvements, simply because it relates to my specialization. But I also try to learn the technical part. To summarize things: I may be well losing my time explaining why += is bad, that is true. It is next to impossible to explain it, unless one can just *see* it. But apart from minority/majority, there is only one truth, so I would say, if "we" could consider such a slight rearrangement of how we *call* the things: Now I see many people call += a "syntactic sugar", and for me it is rather a "syntactic defect", so it is something on the opposite. One can say it is only opinion, ok, but if at least a pair of people will stop writing +=, then I could say, my mission was not a fail. Mikhail
The "just see it" isn't likely to go very far. My impression is that maybe 5% of people find '+=' and friends confusing or bothersome. Probably 15% are neutral (including me). And 80% think 'a += 1' is simply MORE readable and more easily understood than 'a = a+1'. Folks who come from C-family languages especially like the augmented assignment. It's very common for the inplace and binary operators to do the same thing. For those cases, it really is just syntax sugar. But enough importent libraries like NumPy make a distinction tray a highly value having __iadd__() and friends. On Nov 15, 2016 1:03 PM, "Mikhail V" <mikhailwas@gmail.com> wrote:
On 11/14/2016 03:42 PM, Mikhail V wrote:
If you take this file: test.py ---------------------------------------------- a = a + 1 a += 1 ---------------------------------------------- And you look at the bytecode it produces you get the following: ---------------------------------------------- $ python -m dis test.py 1 0 LOAD_NAME 0 (a) 3 LOAD_CONST 0 (1) 6 BINARY_ADD 7 STORE_NAME 0 (a) 2 10 LOAD_NAME 0 (a) 13 LOAD_CONST 0 (1) 16 INPLACE_ADD 17 STORE_NAME 0 (a) 20 LOAD_CONST 1 (None) 23 RETURN_VALUE ---------------------------------------------- That shows that the first and second lines are compiled _differently_. The point is that in the first line, numpy does not have the information necessary to know that "a + 1" will be assigned back to a. In the second case it does. This is presumably why they couldn't do the optimization you desire without large changes in cpython. On the second point, personally I prefer writing "a += 1" to "a = a + 1". I think it's clearer and would keep using it even if the two were equally efficient. But we are all allowed our opinions... Cheers, Thomas
On Mon, Nov 14, 2016 at 1:21 PM, Mikhail V <mikhailwas@gmail.com> wrote:
Okay, then make a proposal to get the operators included as built-ins. Once you have gotten that approved, then we can start talking about which syntax is better. But the idea that people should import a module to do basic mathematical operations is a non-starter.
You shouldn't be using a variable-width font for coding anyway. That is going to cause all sorts of problems (indentation not matching up, for example). You should use a fixed-width font. But if brackets are a problem, I don't see how using more brackets is a solution (which is the case with function calls). On the contrary, I would think you would want to minimize the number of brackets.
No, that is only the case for immutable types like floats. For mutable types like lists then x = x + y and x += y Are not the same thing. The first makes a new object, while the second does an in-place operation. Sometimes you want a new object, sometimes you don't. Having both versions allows you to control that.
First, there is no "_" suffix for variables that numpy could use. The syntax you are suggesting isn't possible without a major change to the basic python grammar and interpreter. Second, again, "sum(x, a)" does not add "x" and "a". Third, again, why? You still haven't given any reason we should prefer this syntax. If this is just your personal preference, please say that. But if you expect people to add a bunch of new built-ins, implement your new suffix syntax, and implement your massive changes to how python operations and function calls work, then you are going to need something more than personal preference.
participants (17)
-
Bernardo Sulzbach -
Brendan Barnwell -
Chris Barker - NOAA Federal -
David Mertz -
Guido van Rossum -
Jelle Zijlstra -
João Matos -
Matthias welp -
Mikhail V -
MRAB -
Nick Timkovich -
Paul Moore -
Stephen J. Turnbull -
Steven D'Aprano -
Thomas Nyberg -
Todd -
אלעזר