Re: [Python-ideas] Python-ideas Digest, Vol 141, Issue 145
I could, but I don't think that justifies not having this functionality in python standard. From the language experience perspective, numpy is often a pain to install on most systems. If I'm designing card games and I just want to run a quick monte carlo simulation, the experience should be as smooth as possible. This is something I think most students will expect while learning python, especially if they're implementing algorithms. On Mon, Aug 27, 2018 at 4:24 AM <python-ideas-request@python.org> wrote:
Op ma 27 aug. 2018 om 23:18 schreef James Lu <jamtlu@gmail.com>:
To be really honest, if you want to run it as "smooth as possible" you'll NEED numpy anyway. If you have an algoritmn of any significant mathematical complexity, numpy's Fortran implementation will beat out pure python easily. I don't have any experience with monte carlo simulations myself, but if you're doing this any significant amount of times you'll be better off creating a numpy array. And personally I just grab Anaconda. It knows how to install wherever, and includes numpy and a lot of other things you might need. If you want it smooth, for beginners, that'd be my first recommendation. (Actually, I started using it because I had trouble installing numpy stack on a new system back when I was a noob.) Most students learning python will probably be on windows, so just down loading an installer and clicking through it should be more than familiar for them.
On Tue, 28 Aug 2018 at 08:12, Jacco van Dorp <j.van.dorp@deonet.nl> wrote:
Numpy is easy to install: $ pip install numpy Should work on OSX, Windows and Linux. In all cases this should download a precompiled binary wheel. It used to be more difficult but improvements in packaging (wheels, manylinux etc) and the good work of the numpy folks have made this painless now. Scipy (on Windows) is a different story. -- Oscar
On Wed, Aug 29, 2018 at 01:15:46PM +0100, Oscar Benjamin wrote:
[steve@ando ~]$ pip install numpy Collecting numpy /usr/local/lib/python3.5/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:315: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning. SNIMissingWarning Could not fetch URL https://pypi.python.org/simple/numpy/: There was a problem confirming the ssl certificate: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645) - skipping Could not find a version that satisfies the requirement numpy (from versions: ) No matching distribution found for numpy I'm sure pip is great, but honestly I've never been able to get it to work reliably, ever, on four different machines using four different Linux distros. In any case, the answer "just use Numpy" isn't really relevant to the question about adding new syntax. -- Steve
On Wed, 29 Aug 2018 at 13:26, Steven D'Aprano <steve@pearwood.info> wrote:
I've not seen this sort of error come up commonly, so I suspect there's something particular to how you have your environment(s) set up. However...
In any case, the answer "just use Numpy" isn't really relevant to the question about adding new syntax.
Indeed. And nor is discussion about pip issues, so I'll leave it there. If you want help getting pip to work, I'd suggest raising an issue on the pip tracker. Paul.
Executive summary: Much of this is my opinion, and as a newer poster, your opinion is *more* valuable than mine (fresh eyes and all that). What I hope you'll take from this post is a more precise understanding of the customary criteria that are used in evaluating a proposed feature on python-ideas and python-dev. Apologies for breaking the thread, but per Subject it's not really part of the thread. James Lu writes:
I could, but I don't think that justifies not having this functionality in python standard.
What's "this"? Note:
When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-ideas digest..."
I had to edit the Subject. I'm going to assume you mean
1. Re: Unpacking iterables for augmented assignment (Matthew Einhorn)
So. The Python development community generally doesn't require justification for refusing to implement new functionality. Rather, because Python has become a big and very complete programming environment, and a fairly large language, implementing new syntax requires that a feature increase expressiveness substantially. In the case in point, the destructuring assignments a, b = b, a w, x, y, z = z, w, y, x can be interpreted as "swapping" or "permuting", and AIUI that's why they were included. They express the intent better than tmp = a a = b b = tmp del tmp and I don't want to even think about how to do the 4-variable version without 4 temporary variables. By comparison, x, y += a, b is neither more expressive, nor easier to read, nor significantly harder to type, than x += a y += b as far as I can see. Note that this is the "not harder to type" criterion normally used in discussing new Python features, not something I've invented.
This is something I think most students will expect while learning python, especially if they're implementing algorithms.
I suppose this claim is unlikely to be accepted without testimony of several teachers of Python. In my own experience, I explicitly teach my students that the destructuring assignment is *for* permuting, and I have *not even once* been asked if it works for augmented assignments. By contrast, students with knowledge of other languages (especially C-like languages) invariably "just use" augmented assignments and ask if there's some spelling for increment and decrement expressions. Obviously, my teaching approach biases the result, but if nobody ever overcomes that bias, I do not think it is an expected or needed feature. Steve
On Tue, Aug 28, 2018 at 6:05 PM, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
When you have completely different variables, sure. But what if - like in the swap example - they're the same variables? def frobnicate(): a, b, c = 1, 10, 100 while True: a, b, c += b, c, a yield a On the first iteration, this would be: a += 10 b += 100 c += 1 I don't have an actual use-case, but to be fair, I also have very few use-cases for swapping/permuting. -0 on adding it, but it's not an illogical feature. ChrisA
Chris Angelico writes:
Good point. a, b, c = a + b, b + c, c + a is "good enough" for this particular case, I'd say, and has the advantage that it immediately generalizes to mixed operations: a, b, c = a + b, b - c, c * a I will say that the fact that neither of us has an immediate use case in mind is really the point of my previous post. Swaps frequently come up in sorting algorithms and heap structure modifications. Swapping alone might be enough to justify the destructuring assignment. Steve
So we currently have iterable unpacking: a, b, c = x # x better be an iterable of exactly 3 values as well as tuple packing: x = a, b, c # sets x to a tuple of 3 values (a, b, c) and we can combine these, e.g.: a, b, c = x, y, z and this still creates an intermediate, anonymous tuple from the values on the right. (At least in the formal semantics -- it may be optimized away if that can be done safely.) This leads to the following idioms: a, b = b, a # swap variables a, b, c = b, c, a # rotate variables That's all well established (and I have to admit that the elegance of the swap idiom was one reason to add this to the earliest version of the language). The proposal on the table is to see if there's a logical extension for augmented assignment. If we look at this: a, b, c = x, y, z and notice that it's equivalent to this: a = x b = y c = z we might propose (as the OP did) that this: a, b, c += x, y, z could be made equivalent to this: a += x b += y c += z but the question is, what would this do? a, b, c += x Presumably it requires that x is an iterable of 3 values, so it would be translated to this first: x0, x1, x2 = x a, b, c += x0, x1, x2 However, a user who doesn't typically think about the actual semantics of iterable unpacking and tuple packing might think this would instead mean the following: a += x b += x c += x IOW they might think that this is a clever way to increment three variables at once: a, b, c += 1 This ambiguity (in naive users' minds) leads me to frown upon the proposal. If we were to ignore the naive view, we could definitely give semantics to `a, b, c += ...` -- basically it would unpack the RHS into the right number of elements (if needed) and call __iadd__ pairwise. But I am not so keen, because the ambiguity of `a, b, c += x`. Perhaps someone can do some research and unearth real code that contains series of += assignments that would become more readable by collapsing them into a single line using the proposed construct. -- --Guido van Rossum (python.org/~guido)
On Tue, Aug 28, 2018 at 12:57 PM Guido van Rossum <guido@python.org> wrote:
This sort of broadcasting is often quite convenient in Numpy, where many arithmetic operators will accept scalars or vectors and generally "do the right thing" in the context. It gets complicated in higher dimensions, but for 0/1-dimensions you don't need an axis-selector. abc = np.array([1, 2, 3]) abc += 1 print(abc) # [2 3 4] abc += [10, 100, 1000] print(abc) # [ 12 103 1004] The quirks I see are with + being a concatenation operator for lists/tuples. a, b = (10, 20) # a, b += (1, 2) # a, b == 11, 22? (a, b) + (1, 2) # (10, 20, 1, 2) Nick
Guido wrote
Here's some notes towards finding (or constructing) such examples. First note that it's awkward to unpack a, b = b, a # swap values because this simple solution does not work (as we need a temporary) a = b b = a so perhaps our example should require unpacking, to avoid the use of temporarys. Now consider, for example, wibble(4).wobble.cheese += 8 This has semantics tmp1 = wibble(4).wobble tmp2 = tmp1.cheese tmp1.cheese = tmp2 + 8 which is different to and harder to write than wibble(4).wobble.cheese = wibble(4).wobble.cheese + 8 so perhaps our example should involve an assignable expression, such as wibble(4).wobble.cheese. So here's a concocted example. ns.aaa.a, ns.bbb.b, += 2 * ns.bbb.b, 3 * ns.aaa.a I describe this as a namespace-of-values, whose values change at least logically all at the same time. In other words, it's something like vector = apply(matrix, vector) This example can be expanded by first computing the values, and then doing the assignment. tmp1, tmp2 = ns.bbb.b, ns.aaa.a ns.aaa.a += 2 * tmp2 ns.bbb.b += 3 * tmp1 The following avoids the double lookup of ns.aaa.a (and of ns.bbb.b). tmp1, tmp2 = ns.bbb.b, ns.aaa.a ns.aaa.a = tmp1 + 2 * tmp2 ns.bbb.b = tmp2 + 3 * tmp1 So far, this is the best artificial example I've come up with. -- Jonathan
Guido van Rossum wrote:
But not without violating the principle that lhs += rhs is equivalent to lhs = lhs.__iadd__(lhs) Granted, this rule inevitably leads to an "unpacking sequence of wrong length" error in the case we're considering, so we wouldn't be changing any useful existing semantics, but it's still making the rules more complicated.
Someone who thinks that might also think a, b, c = 1 is a way to assign 1 to a, b and c, but we don't seem to be worried about that possible misunderstanding. This does raise the question of what the corresponding generalisation of multiple assignment targets might be. Would you write a += b += c += 1 or a = b = c += 1 ? And if the former, do we allow different operations on each target, e.g. a += b -= c *= 2 ? -- Greg
On Tue, Aug 28, 2018 at 6:37 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
(Corrected: lhs = lhs.__iadd__(rhs)) Since lhs here is neither a list nor a tuple, how is it violated? Or rather, how is it any more of a special case than in this syntax: # Neither name-binding or setitem/setattr. [a,b,c] = items If lhs is a Numpy array, then: a_b_c += x, y, z is equivalent to: a_b_c = a_b_c.__iadd__((x,y,z)) We can translate the original example: a, b, c += x, y, z to: a, b, c = target_list(a,b,c).__iadd__((x,y,z)) where `target_list` is a virtual (not as in "virtual function") type for target list constructs.
On Fri, Sep 7, 2018 at 4:11 AM, Franklin? Lee <leewangzhong+python@gmail.com> wrote:
What is the virtual type here, and what does its __iadd__ method do? I don't understand you here. Can you go into detail? Suppose I'm the author of the class that all six of these objects are instances of; can I customize the effect of __iadd__ here in some way, and if so, how? ChrisA
On Thu, Sep 6, 2018 at 2:23 PM Chris Angelico <rosuav@gmail.com> wrote:
I shouldn't have used jargon I had to look up myself. The following are equivalent and compile down to the same code: a, b, c = lst [a, b, c] = lst The left hand side is not an actual list (even though it looks like one). The brackets are optional. The docs call the left hand side a target list: https://docs.python.org/3/reference/simple_stmts.html#assignment-statements "Target list" is not a real type. You can't construct such an object, or hold one in memory. You can't make a class that emulates it (without interpreter-specific hacks), because it is a collection of its names, not a collection of values. target_list.__iadd__ also does not exist, because target_list does not exist. However, target_list can be thought of as a virtual type, a type that the compiler compiles away. We can then consider target_list.__iadd__ as a virtual operator, which the compiler will understand but hide from the runtime. I was making the point that, because the __iadd__ in the example does not refer to list.__iadd__, but rather a virtual target_list.__iadd__, there is not yet a violation of the rule.
On Fri, Sep 7, 2018 at 4:38 AM, Franklin? Lee <leewangzhong+python@gmail.com> wrote:
A target list is a syntactic element, like a name, or an operator, or a "yield" statement. You can't construct one, because it isn't an object type. It's not a "virtual type". It's a completely different sort of thing.
What you're suggesting is on par with trying to say that: for += 5 should be implemented as: current_loop.__iadd__(5) where "current_loop" doesn't really exist, but it's a virtual type that represents a 'for' loop. That doesn't make sense, because there is no object in Python to represent the loop. There is no class/type that represents all loops, on which a method like this could be added. The word 'for' is part of the grammar, not the object model. And "target list" is the same. There's no way to attach an __iadd__ method to something that doesn't exist. So for your proposal to work, you would need to break that rule, and give a *different* meaning to this. ChrisA
n Thu, Sep 6, 2018 at 2:47 PM Chris Angelico <rosuav@gmail.com> wrote:
I didn't think I gave the impression that I was complaining about not being able to construct it. I gave an explanation for how it isn't a real type, because you asked how you could modify the behavior, and because I wanted to give an explanation for more than just you. There are constructs that correspond to types (such as slices and functions). There are those that don't. We call `3:2` (in the right context) a slice, even though it's technically a construct which is compiled down to a `slice` object. I see no problem there. I called it a "virtual type" and explained why I called it that. You reject the use of that term, but you don't even acknowledge that I gave reasons for it.
I explained how target_list could be thought of as a special imaginary type which only exists in the compiler's "mind", and then extended that to an imaginary method on that type. Of course your example shows absurdity: you didn't try to say how a for-loop is like an object in the first place.
But I'm not using the word `for`. I am using constructs like `[a,b,c]` (where it is not a list). At least use `(for x in y: z) += 5` as your example. You're effectively accusing me of trying to make `[` (a single token, not a full construct) an object. Your argument here is that there is no Python object to represent a loop, but that really means there's no _runtime_ object to represent a loop. I already said that target lists don't exist in memory (i.e. runtime). "Target list" does exist, just not as a runtime type. It exists as an abstraction not available to the runtime, and we can extend that abstraction in ways not available to the runtime. That means that you can't attach it during the runtime. It does not mean you can't reason with it during compile-time.
So for your proposal to work, you would need to break that rule, and give a *different* meaning to this.
It is not my proposal. I was questioning how there was a rule violation about x+=y translating to `x = x.__iadd__(y)`. You're talking about a different, made-up rule about how syntactical constructs can't correspond to compile-time imaginary objects or runtime objects. But there are syntactical constructs that DO correspond to runtime types (slice, list, class), there are those which don't but can (let's not get into that), there are those which can stay compile-time (f-strings, target lists), and there are those which probably can't be thought of as types at all (import).
Hi Franklin Lee Thank you for your message. You wrote:
Yes, we can.I think all are agreed that that such semantics for a, b, c += x, y, z could be provided in a future version of Python. At present we get >>> a, b, c += [4, 5, 6] SyntaxError: illegal expression for augmented assignment Where we're not agreed, I think, is whether doing so would be a good idea. The proposers think it is a good idea. However, unless the proposers convince sufficient users that it is a good idea to do so, it probably won't be added to Python. By the way, I think it's easier to get users for a pure Python module (and hence perhaps get it into the standard library) than it is to make a language syntax and semantics change. And I also like that things are this way. -- Jonathan
Rather,
because Python has become a big and very complete programming
environment, and a fairly large language, implementing new syntax
requires that a feature increase expressiveness substantially. That makes sense.
By comparison, x, y += a, b
is neither more expressive, nor easier to read, nor significantly
harder to type, than
x += a
y += b I agree. I contend that x, y += func(...) is more readable than the three- statement alternative with a namespace pollution. On Tue, Aug 28, 2018 at 4:05 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
In 20 years of programming Python, I have never wanted to augment two distinct variables by two distinct return values from a function. I'm sure it's possible to construct some situation where that would be convenient, and presumably the OP actually encountered that. But the need is exceedingly rare, and absolutely not worth enabling syntax that is confusing in other ways and introduces inconsistencies in overall semantics. On Thu, Aug 30, 2018, 8:16 PM James Lu <jamtlu@gmail.com> wrote:
Executive summary: Although I'm obviously not particularly in favor of this feature, my opinion is no more valuable than anyone else's. The point of this post is to show how features have been advocated successfully in the past. James Lu writes:
I'm sure it is. But what's the context? That is, just saying you think it's more readable isn't enough. What has succeeded in the past (in approximate order of effectiveness) has been 1. A reasonably comprehensive survey of the Python stdlib to find use cases for the "ugly" construct, and then providing a few "side by side" comparisons "before and after" the new syntax, as well as statistics for the frequency of the "ugly" construct, and how often translation to the new feature improved the code. 2. A similar exercise for some other large, reasonably well-known code base by a respected project. 3. A few examples from some code base. Here's an example of the kind of analysis I mean. I went looking for examples of swaps in the Python sources by grepping for "sort". I found several in Tools/demo/sortvisu.py, but strangely enough, although it uses sequence assignment for the swap, it does this: def swap(self, i, j): if i == j: return self.countswap() item = self.items[i] other = self.items[j] self.items[i], self.items[j] = other, item item.swapwith(other) The reason is that this is not a sorting application as such, but rather a demo of tkinter. So the sort is instrumented in several ways, and in particular the items being swapped know "where they are" in the visualization (the item.swapwith(other) call). Thus the temporary variables item and other are not spurious. I'm not sure if I think def swap(self, i, j): if i == j: return self.countswap() item, other = self.items[i], self.items[j] self.items[i], self.items[j] = other, item item.swapwith(other) is less readable, but I don't think it's more readable. And with the instrumentation removed, I'm not sure whether I prefer def swap(self, i, j): if i == j: return self.items[i], self.items[j] = self.items[j], self.items[i] or def swap(self, i, j): if i == j: return item, other = self.items[i], self.items[j] self.items[i], self.items[j] = other, item It's pretty marginal. Finally, I think if i != j: self.items[i], self.items[j] = self.items[j], self.items[i] inlined instead of self.swap(i, j) perceptibly harms readability, although it would be somewhat faster due to eliminating the function call, and if speed were at a premium I would prefer the inline swap to the introduction of any temporary variables.[1] My guess is that a lot of realistic cases where you apparently *could* use augmented assignment of sequences to avoid introducing one or more temporary variables are going to have some of the same features that the variables are actually useful beyond the augmented assignment, or the expressions on one side of the assignment or both are sufficiently complex that introducing the temporary variable with a descriptive name improves readability. But the proof of that will be in doing the work. Steve Footnotes: [1] And if you're reading this, Victor, of course I'd benchmark before using the less readable version in production. :-)
Op ma 27 aug. 2018 om 23:18 schreef James Lu <jamtlu@gmail.com>:
To be really honest, if you want to run it as "smooth as possible" you'll NEED numpy anyway. If you have an algoritmn of any significant mathematical complexity, numpy's Fortran implementation will beat out pure python easily. I don't have any experience with monte carlo simulations myself, but if you're doing this any significant amount of times you'll be better off creating a numpy array. And personally I just grab Anaconda. It knows how to install wherever, and includes numpy and a lot of other things you might need. If you want it smooth, for beginners, that'd be my first recommendation. (Actually, I started using it because I had trouble installing numpy stack on a new system back when I was a noob.) Most students learning python will probably be on windows, so just down loading an installer and clicking through it should be more than familiar for them.
On Tue, 28 Aug 2018 at 08:12, Jacco van Dorp <j.van.dorp@deonet.nl> wrote:
Numpy is easy to install: $ pip install numpy Should work on OSX, Windows and Linux. In all cases this should download a precompiled binary wheel. It used to be more difficult but improvements in packaging (wheels, manylinux etc) and the good work of the numpy folks have made this painless now. Scipy (on Windows) is a different story. -- Oscar
On Wed, Aug 29, 2018 at 01:15:46PM +0100, Oscar Benjamin wrote:
[steve@ando ~]$ pip install numpy Collecting numpy /usr/local/lib/python3.5/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:315: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning. SNIMissingWarning Could not fetch URL https://pypi.python.org/simple/numpy/: There was a problem confirming the ssl certificate: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645) - skipping Could not find a version that satisfies the requirement numpy (from versions: ) No matching distribution found for numpy I'm sure pip is great, but honestly I've never been able to get it to work reliably, ever, on four different machines using four different Linux distros. In any case, the answer "just use Numpy" isn't really relevant to the question about adding new syntax. -- Steve
On Wed, 29 Aug 2018 at 13:26, Steven D'Aprano <steve@pearwood.info> wrote:
I've not seen this sort of error come up commonly, so I suspect there's something particular to how you have your environment(s) set up. However...
In any case, the answer "just use Numpy" isn't really relevant to the question about adding new syntax.
Indeed. And nor is discussion about pip issues, so I'll leave it there. If you want help getting pip to work, I'd suggest raising an issue on the pip tracker. Paul.
Executive summary: Much of this is my opinion, and as a newer poster, your opinion is *more* valuable than mine (fresh eyes and all that). What I hope you'll take from this post is a more precise understanding of the customary criteria that are used in evaluating a proposed feature on python-ideas and python-dev. Apologies for breaking the thread, but per Subject it's not really part of the thread. James Lu writes:
I could, but I don't think that justifies not having this functionality in python standard.
What's "this"? Note:
When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-ideas digest..."
I had to edit the Subject. I'm going to assume you mean
1. Re: Unpacking iterables for augmented assignment (Matthew Einhorn)
So. The Python development community generally doesn't require justification for refusing to implement new functionality. Rather, because Python has become a big and very complete programming environment, and a fairly large language, implementing new syntax requires that a feature increase expressiveness substantially. In the case in point, the destructuring assignments a, b = b, a w, x, y, z = z, w, y, x can be interpreted as "swapping" or "permuting", and AIUI that's why they were included. They express the intent better than tmp = a a = b b = tmp del tmp and I don't want to even think about how to do the 4-variable version without 4 temporary variables. By comparison, x, y += a, b is neither more expressive, nor easier to read, nor significantly harder to type, than x += a y += b as far as I can see. Note that this is the "not harder to type" criterion normally used in discussing new Python features, not something I've invented.
This is something I think most students will expect while learning python, especially if they're implementing algorithms.
I suppose this claim is unlikely to be accepted without testimony of several teachers of Python. In my own experience, I explicitly teach my students that the destructuring assignment is *for* permuting, and I have *not even once* been asked if it works for augmented assignments. By contrast, students with knowledge of other languages (especially C-like languages) invariably "just use" augmented assignments and ask if there's some spelling for increment and decrement expressions. Obviously, my teaching approach biases the result, but if nobody ever overcomes that bias, I do not think it is an expected or needed feature. Steve
On Tue, Aug 28, 2018 at 6:05 PM, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
When you have completely different variables, sure. But what if - like in the swap example - they're the same variables? def frobnicate(): a, b, c = 1, 10, 100 while True: a, b, c += b, c, a yield a On the first iteration, this would be: a += 10 b += 100 c += 1 I don't have an actual use-case, but to be fair, I also have very few use-cases for swapping/permuting. -0 on adding it, but it's not an illogical feature. ChrisA
Chris Angelico writes:
Good point. a, b, c = a + b, b + c, c + a is "good enough" for this particular case, I'd say, and has the advantage that it immediately generalizes to mixed operations: a, b, c = a + b, b - c, c * a I will say that the fact that neither of us has an immediate use case in mind is really the point of my previous post. Swaps frequently come up in sorting algorithms and heap structure modifications. Swapping alone might be enough to justify the destructuring assignment. Steve
So we currently have iterable unpacking: a, b, c = x # x better be an iterable of exactly 3 values as well as tuple packing: x = a, b, c # sets x to a tuple of 3 values (a, b, c) and we can combine these, e.g.: a, b, c = x, y, z and this still creates an intermediate, anonymous tuple from the values on the right. (At least in the formal semantics -- it may be optimized away if that can be done safely.) This leads to the following idioms: a, b = b, a # swap variables a, b, c = b, c, a # rotate variables That's all well established (and I have to admit that the elegance of the swap idiom was one reason to add this to the earliest version of the language). The proposal on the table is to see if there's a logical extension for augmented assignment. If we look at this: a, b, c = x, y, z and notice that it's equivalent to this: a = x b = y c = z we might propose (as the OP did) that this: a, b, c += x, y, z could be made equivalent to this: a += x b += y c += z but the question is, what would this do? a, b, c += x Presumably it requires that x is an iterable of 3 values, so it would be translated to this first: x0, x1, x2 = x a, b, c += x0, x1, x2 However, a user who doesn't typically think about the actual semantics of iterable unpacking and tuple packing might think this would instead mean the following: a += x b += x c += x IOW they might think that this is a clever way to increment three variables at once: a, b, c += 1 This ambiguity (in naive users' minds) leads me to frown upon the proposal. If we were to ignore the naive view, we could definitely give semantics to `a, b, c += ...` -- basically it would unpack the RHS into the right number of elements (if needed) and call __iadd__ pairwise. But I am not so keen, because the ambiguity of `a, b, c += x`. Perhaps someone can do some research and unearth real code that contains series of += assignments that would become more readable by collapsing them into a single line using the proposed construct. -- --Guido van Rossum (python.org/~guido)
On Tue, Aug 28, 2018 at 12:57 PM Guido van Rossum <guido@python.org> wrote:
This sort of broadcasting is often quite convenient in Numpy, where many arithmetic operators will accept scalars or vectors and generally "do the right thing" in the context. It gets complicated in higher dimensions, but for 0/1-dimensions you don't need an axis-selector. abc = np.array([1, 2, 3]) abc += 1 print(abc) # [2 3 4] abc += [10, 100, 1000] print(abc) # [ 12 103 1004] The quirks I see are with + being a concatenation operator for lists/tuples. a, b = (10, 20) # a, b += (1, 2) # a, b == 11, 22? (a, b) + (1, 2) # (10, 20, 1, 2) Nick
Guido wrote
Here's some notes towards finding (or constructing) such examples. First note that it's awkward to unpack a, b = b, a # swap values because this simple solution does not work (as we need a temporary) a = b b = a so perhaps our example should require unpacking, to avoid the use of temporarys. Now consider, for example, wibble(4).wobble.cheese += 8 This has semantics tmp1 = wibble(4).wobble tmp2 = tmp1.cheese tmp1.cheese = tmp2 + 8 which is different to and harder to write than wibble(4).wobble.cheese = wibble(4).wobble.cheese + 8 so perhaps our example should involve an assignable expression, such as wibble(4).wobble.cheese. So here's a concocted example. ns.aaa.a, ns.bbb.b, += 2 * ns.bbb.b, 3 * ns.aaa.a I describe this as a namespace-of-values, whose values change at least logically all at the same time. In other words, it's something like vector = apply(matrix, vector) This example can be expanded by first computing the values, and then doing the assignment. tmp1, tmp2 = ns.bbb.b, ns.aaa.a ns.aaa.a += 2 * tmp2 ns.bbb.b += 3 * tmp1 The following avoids the double lookup of ns.aaa.a (and of ns.bbb.b). tmp1, tmp2 = ns.bbb.b, ns.aaa.a ns.aaa.a = tmp1 + 2 * tmp2 ns.bbb.b = tmp2 + 3 * tmp1 So far, this is the best artificial example I've come up with. -- Jonathan
Guido van Rossum wrote:
But not without violating the principle that lhs += rhs is equivalent to lhs = lhs.__iadd__(lhs) Granted, this rule inevitably leads to an "unpacking sequence of wrong length" error in the case we're considering, so we wouldn't be changing any useful existing semantics, but it's still making the rules more complicated.
Someone who thinks that might also think a, b, c = 1 is a way to assign 1 to a, b and c, but we don't seem to be worried about that possible misunderstanding. This does raise the question of what the corresponding generalisation of multiple assignment targets might be. Would you write a += b += c += 1 or a = b = c += 1 ? And if the former, do we allow different operations on each target, e.g. a += b -= c *= 2 ? -- Greg
On Tue, Aug 28, 2018 at 6:37 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
(Corrected: lhs = lhs.__iadd__(rhs)) Since lhs here is neither a list nor a tuple, how is it violated? Or rather, how is it any more of a special case than in this syntax: # Neither name-binding or setitem/setattr. [a,b,c] = items If lhs is a Numpy array, then: a_b_c += x, y, z is equivalent to: a_b_c = a_b_c.__iadd__((x,y,z)) We can translate the original example: a, b, c += x, y, z to: a, b, c = target_list(a,b,c).__iadd__((x,y,z)) where `target_list` is a virtual (not as in "virtual function") type for target list constructs.
On Fri, Sep 7, 2018 at 4:11 AM, Franklin? Lee <leewangzhong+python@gmail.com> wrote:
What is the virtual type here, and what does its __iadd__ method do? I don't understand you here. Can you go into detail? Suppose I'm the author of the class that all six of these objects are instances of; can I customize the effect of __iadd__ here in some way, and if so, how? ChrisA
On Thu, Sep 6, 2018 at 2:23 PM Chris Angelico <rosuav@gmail.com> wrote:
I shouldn't have used jargon I had to look up myself. The following are equivalent and compile down to the same code: a, b, c = lst [a, b, c] = lst The left hand side is not an actual list (even though it looks like one). The brackets are optional. The docs call the left hand side a target list: https://docs.python.org/3/reference/simple_stmts.html#assignment-statements "Target list" is not a real type. You can't construct such an object, or hold one in memory. You can't make a class that emulates it (without interpreter-specific hacks), because it is a collection of its names, not a collection of values. target_list.__iadd__ also does not exist, because target_list does not exist. However, target_list can be thought of as a virtual type, a type that the compiler compiles away. We can then consider target_list.__iadd__ as a virtual operator, which the compiler will understand but hide from the runtime. I was making the point that, because the __iadd__ in the example does not refer to list.__iadd__, but rather a virtual target_list.__iadd__, there is not yet a violation of the rule.
On Fri, Sep 7, 2018 at 4:38 AM, Franklin? Lee <leewangzhong+python@gmail.com> wrote:
A target list is a syntactic element, like a name, or an operator, or a "yield" statement. You can't construct one, because it isn't an object type. It's not a "virtual type". It's a completely different sort of thing.
What you're suggesting is on par with trying to say that: for += 5 should be implemented as: current_loop.__iadd__(5) where "current_loop" doesn't really exist, but it's a virtual type that represents a 'for' loop. That doesn't make sense, because there is no object in Python to represent the loop. There is no class/type that represents all loops, on which a method like this could be added. The word 'for' is part of the grammar, not the object model. And "target list" is the same. There's no way to attach an __iadd__ method to something that doesn't exist. So for your proposal to work, you would need to break that rule, and give a *different* meaning to this. ChrisA
n Thu, Sep 6, 2018 at 2:47 PM Chris Angelico <rosuav@gmail.com> wrote:
I didn't think I gave the impression that I was complaining about not being able to construct it. I gave an explanation for how it isn't a real type, because you asked how you could modify the behavior, and because I wanted to give an explanation for more than just you. There are constructs that correspond to types (such as slices and functions). There are those that don't. We call `3:2` (in the right context) a slice, even though it's technically a construct which is compiled down to a `slice` object. I see no problem there. I called it a "virtual type" and explained why I called it that. You reject the use of that term, but you don't even acknowledge that I gave reasons for it.
I explained how target_list could be thought of as a special imaginary type which only exists in the compiler's "mind", and then extended that to an imaginary method on that type. Of course your example shows absurdity: you didn't try to say how a for-loop is like an object in the first place.
But I'm not using the word `for`. I am using constructs like `[a,b,c]` (where it is not a list). At least use `(for x in y: z) += 5` as your example. You're effectively accusing me of trying to make `[` (a single token, not a full construct) an object. Your argument here is that there is no Python object to represent a loop, but that really means there's no _runtime_ object to represent a loop. I already said that target lists don't exist in memory (i.e. runtime). "Target list" does exist, just not as a runtime type. It exists as an abstraction not available to the runtime, and we can extend that abstraction in ways not available to the runtime. That means that you can't attach it during the runtime. It does not mean you can't reason with it during compile-time.
So for your proposal to work, you would need to break that rule, and give a *different* meaning to this.
It is not my proposal. I was questioning how there was a rule violation about x+=y translating to `x = x.__iadd__(y)`. You're talking about a different, made-up rule about how syntactical constructs can't correspond to compile-time imaginary objects or runtime objects. But there are syntactical constructs that DO correspond to runtime types (slice, list, class), there are those which don't but can (let's not get into that), there are those which can stay compile-time (f-strings, target lists), and there are those which probably can't be thought of as types at all (import).
Hi Franklin Lee Thank you for your message. You wrote:
Yes, we can.I think all are agreed that that such semantics for a, b, c += x, y, z could be provided in a future version of Python. At present we get >>> a, b, c += [4, 5, 6] SyntaxError: illegal expression for augmented assignment Where we're not agreed, I think, is whether doing so would be a good idea. The proposers think it is a good idea. However, unless the proposers convince sufficient users that it is a good idea to do so, it probably won't be added to Python. By the way, I think it's easier to get users for a pure Python module (and hence perhaps get it into the standard library) than it is to make a language syntax and semantics change. And I also like that things are this way. -- Jonathan
Rather,
because Python has become a big and very complete programming
environment, and a fairly large language, implementing new syntax
requires that a feature increase expressiveness substantially. That makes sense.
By comparison, x, y += a, b
is neither more expressive, nor easier to read, nor significantly
harder to type, than
x += a
y += b I agree. I contend that x, y += func(...) is more readable than the three- statement alternative with a namespace pollution. On Tue, Aug 28, 2018 at 4:05 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
In 20 years of programming Python, I have never wanted to augment two distinct variables by two distinct return values from a function. I'm sure it's possible to construct some situation where that would be convenient, and presumably the OP actually encountered that. But the need is exceedingly rare, and absolutely not worth enabling syntax that is confusing in other ways and introduces inconsistencies in overall semantics. On Thu, Aug 30, 2018, 8:16 PM James Lu <jamtlu@gmail.com> wrote:
Executive summary: Although I'm obviously not particularly in favor of this feature, my opinion is no more valuable than anyone else's. The point of this post is to show how features have been advocated successfully in the past. James Lu writes:
I'm sure it is. But what's the context? That is, just saying you think it's more readable isn't enough. What has succeeded in the past (in approximate order of effectiveness) has been 1. A reasonably comprehensive survey of the Python stdlib to find use cases for the "ugly" construct, and then providing a few "side by side" comparisons "before and after" the new syntax, as well as statistics for the frequency of the "ugly" construct, and how often translation to the new feature improved the code. 2. A similar exercise for some other large, reasonably well-known code base by a respected project. 3. A few examples from some code base. Here's an example of the kind of analysis I mean. I went looking for examples of swaps in the Python sources by grepping for "sort". I found several in Tools/demo/sortvisu.py, but strangely enough, although it uses sequence assignment for the swap, it does this: def swap(self, i, j): if i == j: return self.countswap() item = self.items[i] other = self.items[j] self.items[i], self.items[j] = other, item item.swapwith(other) The reason is that this is not a sorting application as such, but rather a demo of tkinter. So the sort is instrumented in several ways, and in particular the items being swapped know "where they are" in the visualization (the item.swapwith(other) call). Thus the temporary variables item and other are not spurious. I'm not sure if I think def swap(self, i, j): if i == j: return self.countswap() item, other = self.items[i], self.items[j] self.items[i], self.items[j] = other, item item.swapwith(other) is less readable, but I don't think it's more readable. And with the instrumentation removed, I'm not sure whether I prefer def swap(self, i, j): if i == j: return self.items[i], self.items[j] = self.items[j], self.items[i] or def swap(self, i, j): if i == j: return item, other = self.items[i], self.items[j] self.items[i], self.items[j] = other, item It's pretty marginal. Finally, I think if i != j: self.items[i], self.items[j] = self.items[j], self.items[i] inlined instead of self.swap(i, j) perceptibly harms readability, although it would be somewhat faster due to eliminating the function call, and if speed were at a premium I would prefer the inline swap to the introduction of any temporary variables.[1] My guess is that a lot of realistic cases where you apparently *could* use augmented assignment of sequences to avoid introducing one or more temporary variables are going to have some of the same features that the variables are actually useful beyond the augmented assignment, or the expressions on one side of the assignment or both are sufficiently complex that introducing the temporary variable with a descriptive name improves readability. But the proof of that will be in doing the work. Steve Footnotes: [1] And if you're reading this, Victor, of course I'd benchmark before using the less readable version in production. :-)
participants (15)
-
Chris Angelico -
David Mertz -
Franklin? Lee -
Greg Ewing -
Guido van Rossum -
Jacco van Dorp -
James Lu -
Jonathan Fine -
MRAB -
Nick Timkovich -
Oscar Benjamin -
Paul Moore -
Robert Vanden Eynde -
Stephen J. Turnbull -
Steven D'Aprano