Inline assignments using "given" clauses
(Note: Guido's already told me off-list that he doesn't like the way this spelling reads, but I wanted to share it anyway since it addresses one of the recurring requests in the PEP 572 discussions for a more targeted proposal that focused specifically on the use cases that folks had agreed were reasonable potential use cases for inline assignment expressions. I'll also note that another potential concern with this specific proposal is that even though "given" wasn't used as a term in any easily discovered Python APIs back when I first wrote PEP 3150, it's now part of the Hypothesis testing API, so adopting it as a keyword now would be markedly more disruptive than it might have been historically) Recapping the use cases where the inline assignment capability received the most agreement regarding being potentially more readable than the status quo: 1. Making an "exactly one branch is executed" construct clearer than is the case for nested if statements: if m := pattern.search(data): ... elif m := other_pattern.search(data): ... else: ... 2. Replacing a loop-and-a-half construct: while m := pattern.search(remaining_data): ... 3. Sharing values between filtering clauses and result expressions in comprehensions: result = [(x, y, x/y) for x in data if (y := f(x))] The essence of the given clause concept would be to modify *these specific cases* (at least initially) to allow the condition expression to be followed by an inline assignment, of the form "given TARGET = EXPR". (Note: being able to implement such a syntactic constraint is a general consequence of using a ternary notation rather than a binary one, since it allows the construct to start with an arbitrary expression, without requiring that expression to be both the result of the operation *and* the value bound to a name - it isn't unique to the "given" keyword specifically) While the leading keyword would allow TARGET to be an arbitrary assignment target without much chance for confusion, it could also be restricted to simple names instead (as has been done for PEP 572. With that spelling, the three examples above would become: # Exactly one branch is executed here if m given m = pattern.search(data): ... elif m given m = other_pattern.search(data)): ... else: ... # This name is rebound on each trip around the loop while m given m = pattern.search(remaining_data): ... # "f(x)" is only evaluated once on each iteration result = [(x, y, x/y) for x in data if y given y = f(x)] Constraining the syntax that way (at least initially) would avoid poking into any dark corners of Python's current scoping and expression execution ordering semantics, while still leaving the door open to later making "result given NAME = expr" a general purpose ternary operator that returns the LHS, while binding the RHS to the given name as a side effect. Using a new keyword (rather than a symbol) would make the new construct easier to identify and search for, but also comes with all the downsides of introducing a new keyword. (Hence the not-entirely-uncommon suggestion of using "with" for a purpose along these lines, which runs into a different set of problems related to trying to use "with" for two distinct and entirely unrelated purposes). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
04.05.18 15:06, Nick Coghlan пише:
Sorry, but these examples don't look as good examples for inline assignments to me. I think that all these cases can be written better without using the inline assignment.
This case can be better handled by combining patterns in a single regular expression. pattern = re.compile('(?P<foo>pattern1)|(?P<bar>pattern2)|...') m = pattern.search(data) if not m: # this can be omitted if the pattern is always found ... elif m.group('foo'): ... elif m.group('bar'): ... See for example gettext.py where this pattern is used.
This case can be better handled by re.finditer(). for m in pattern.finditer(remaining_data): ... In more complex cases it is handy to write a simple generator function and iterate its result. The large number of similar cases are covered by a two-argument iter().
There are a lot of ways of writing this. PEP 572 mentions them. Different ways are used in real code depending on preferences of the author. Actually the number of these cases is pretty low in comparison with the total number of comprehensions. It is possible to express an assignment in comprehensions with the "for var in [value]" idiom, and this idiom is more powerful than PEP 572 in this case because it allows to perform an assignment before the first 'for'. But really complex comprehensions could be better written as 'for' statements with explicit adding to the collection or yielding.
[Nick Coghlan <ncoghlan@gmail.com>]
That deserves more thought. I started my paying career working on a Fortran compiler, a language which, by design, had no reserved words (although plenty of keywords). The language itself (and vendor-specific extensions) never had to suffer "but adding a new keyword could break old code!" consequences. In practice that worked out very well, Yes, you _could_ write hard-to-read code using language keywords as, e.g., identifier names too, but, no, absolutely nobody did that outside of "stupid Fortran tricks" posts on Usenet ;-) It had the _intended_ effect in practice: no breakage of old code just because the language grew new constructs. It's no longer the case that Python avoided that entirely, since "async def", "async for", and "async with" statements were added _without_ making "async" a new reserved word. It may require pain in the parser, but it's often doable anyway. At this stage in Python's life, adding new _reserved_ words "should be" an extremely high bar - but adding new non-reserved keywords (like "async") should be a much lower bar. That said, I expect it's easier in general to add a non-reserved keyword introducing a statement (like "async") than one buried inside expressions ("given").
On Fri, May 4, 2018 at 11:11 AM, Tim Peters <tim.peters@gmail.com> wrote:
Do note that this was a temporary solution. In 3.5 we introduced this hack. In 3.6, other uses of `async` and `await` became deprecated (though you'd have to use `python -Wall` to get a warning). In 3.7, it's a syntax error.
I'd also say that the difficulty of Googling for the meaning of ":=" shouldn't be exaggerated. Currently you can search for "python operators" and get tons of sites that list all operators. I also note that Google seems to be getting smarter about non-alphabetic searches -- I just searched for "python << operator" and the first hit was https://wiki.python.org/moin/BitwiseOperators -- --Guido van Rossum (python.org/~guido)
On Fri, May 4, 2018, 11:35 Guido van Rossum <guido@python.org> wrote:
Without adding hits to the search algorithm, this will remain the case. Google must have clicks to rank up. Right now there is no page, nothing on a high "Google juice" page like python.org, no one searching for it, and no mass of people clicking on it. no SO questions, etc. there is a transient response for all change. uniqueness and length of search term is just a faster one. All python syntax is findable eventually due to popularity. plus a better search is "why would I use...in python" or similar. = python also doesn't bring up anything interesting that wouldn't be had because of just "python". The details are too mundane and/or technical and everyone knows already.
that being said, if := had been (theoretically) included from the beginning, would people continue to have issues with it? unlikely, but I can't know. familiarity will cure many of these issues of readability or symbolic disagreement no matter what is chosen (well, to a point). it's unfortunate that changes have to be made up front with so little information like that, so I'm not advocating anything based on this, just pointing it out. I do think post hoc assignment will cause a cognitive load, like trying to figure out which variable is the iterator, and having to keep two contexts till the end of a comp with one given statement. [f(x) + a for all a in blah given x=1] not worse than a double nested comp though.
On Sat, May 5, 2018 at 5:27 AM, Matt Arcidy <marcidy@gmail.com> wrote:
Did you try? I searched for 'python :=' and for 'python colon equals' and got this hit each time: https://stackoverflow.com/questions/26000198/what-does-colon-equal-in-python... Which, incidentally, now has a response to it citing PEP 572. Good ol' Stack Overflow. ChrisA
[Tim]
[Guido]
See my "that deserves more thought" at the start, but wrt future cases then ;-) In 3.5 and 3.6, "everything just works" for everyone. In 3.7 the implementation gets churned again, to go out of its way to break the handful of code using "async" as an identifier. It's obvious who that hurts, but who does that really benefit? My experience with Fortran convinces me nobody would _actually_ be confused even if they wrote code like: async def frobnicate(async=True): if async: async with ... But nobody would actually do that. Then again, "but people _could_ do that!" barely registers with me because the nobody-actually-does-it theoretical possibilities were so much worse in Fortran, so I tend to tune that kind of argument out reflexively. For example, whitespace was also irrelevant in Fortran, and these two statements mean radically different things: D O1 0I=1 00,30 0 D O1 0I=1 00.30 0 The first is like: for I in range(100, 301): # the block ends at the next statement with label 10 The seconds is like: DO10I = 100.300 All actual Fortran code spells them like this instead: DO 10 I = 100, 300 DO10I = 100.300 The differing intents are obvious at a glance then - although, yup, to the compiler the difference is solely due to that one uses a comma where the other a period. I'm not suggesting Python go anywhere near _that_ far ;-) Just as far as considering that there's no actual harm in Fortran allowing "DO" to be a variable name too. Nobody is even tempted to think that "DO" might mean "DO loop" in, e.g., DO = 4 X = FUNC(DO) X = DO(Y) IF (DO.OR.DONTDO) GOTO 10 etc. People generally _don't_ use Fortran keywords as identifiers despite that they can, but it's a real boon for the relatively rare older code that failed to anticipate keywords added after it was written.
I've noted before that people don't seem to have trouble finding the meaning of Python's "is", "and", and "or" either. But Googling for "is" (etc) on its own isn't the way to do it ;-)
Ya - most arguments are crap ;-)
On Fri, May 4, 2018 at 1:53 PM, Tim Peters <tim.peters@gmail.com> wrote:
IIUC, Javascript has also gone all-in on contextual keywords. The realities of browser deployment mean they simply cannot have flag days or break old code, ever, meaning that contextual keywords are really the only kind they can add at all. So their async/await uses the same kind of trick that Python 3.5 did, and I believe they plan to keep it that way forever. FWIW. -n -- Nathaniel J. Smith -- https://vorpus.org
I agree it would be useful to have new keywords without being reserved, and we could even go with mechanism like infix operators created by user. It would allow things like [given for x in range(5) given given = x+1] or even [given for given in range(given) given given = given + 1] haha, but as other pointed out, it would be called Bad practice ^^ By the way, I still prefer "where" to "given". Le sam. 5 mai 2018 à 10:52, Greg Ewing <greg.ewing@canterbury.ac.nz> a écrit :
I think that "expr as y" was discarded too quickly. It would be a syntax completely familiar to Python programmers, and the new semantics would be obvious. That choice would also be in line with the Zen of Python. The special cases that may arise over "except" and "with" can be worked out and documented. Cheers, On Sat, May 5, 2018 at 5:22 AM, Robert Vanden Eynde <robertve92@gmail.com> wrote:
-- Juancarlo *Añez*
On Sat, May 05, 2018 at 06:24:07AM -0400, Juancarlo Añez wrote:
I think that "expr as y" was discarded too quickly.
This discussion started back in *February*. I don't think "too quickly" applies to ANYTHING about it. https://mail.python.org/pipermail/python-ideas/2018-February/048971.html And Chris' first draft of the PEP: https://mail.python.org/pipermail/python-ideas/2018-February/049041.html I have been one of the main proponents of "as". See, for example: https://mail.python.org/pipermail/python-ideas/2018-April/049880.html At least, I *was*. I'm now satisfied that "as" is the wrong solution, and I don't think it was discarded too quickly. Even though it makes me sad that "as" is not suitable, I'm satisfied that the problems with it would require too high a price to solve. The major problem is that it will clash with "except as" and "with as" statements. Of course we *could* introduce some sort of special treatment, possibly as simple as simply banning the use of binding- assignments inside except/with statements, but such special rules add complexity, make the feature less useful, harder to learn, and more surprising. Allowing or disallowing particular expressions after a certain keywork ought to be a last resort. Even though this was my preferred solution, I've now come to change my mind and think this would have been a mistake. (Thanks Chris for sticking to your guns and rejecting "as".) You say:
The special cases that may arise over "except" and "with" can be worked out and documented.
but there's no "may" about this. Using "as" does clash, it's not a matter of whether or not it will clash, we know it will. And it's easy to say that it "can" be worked out, but unless you have a concrete proposal to work it out, that's not really an argument in favour for "as", it is just a hope. Guido has also correctly pointed out that will "as" is used to bind names in other contexts, it doesn't *quite* work the same as regular = assignment. Again, the "with" statement is especially relevant: with expression as name does not bind the value of the expression to name, except by coincidence. It actually binds the value of expession.__enter__() to name. I still, and probably always will, like the look of result = (expression as spam) + spam**2 but I'm realistic to realise that it isn't practical. -- Steve
On Fri, May 4, 2018 at 8:06 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I think this is a step in the right direction. I stayed away from the PEP 572 discussions because while intuitively it felt wrong, I could not formulate what exactly was wrong with the assignment expressions proposals. This proposal has finally made me realize why I did not like PEP 572. The strong expression vs. statement dichotomy is one of the key features that set Python apart from many other languages and it makes Python programs much easier to understand. Right from the title, "Assignment Expressions", PEP 572 was set to destroy the very feature that in my view is responsible for much of Python's success. Unlike PEP 572, Nick's proposal does not feel like changing the syntax of Python expressions, instead it feels like an extension to the if-, while- and for-statements syntax. (While comprehensions are expressions, for the purposes of this proposal I am willing to view them as for-statements in disguise.)
On Fri, May 4, 2018 at 6:56 PM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
This is what makes me uncomfortable too. As Dijkstra once wrote: "our intellectual powers are rather geared to master static relations and ... our powers to visualize processes evolving in time are relatively poorly developed. For that reason we should do (as wise programmers aware of our limitations) our utmost to shorten the conceptual gap between the static program and the dynamic process, to make the correspondence between the program (spread out in text space) and the process (spread out in time) as trivial as possible." [1] Normally, Python code strongly maps *time* onto *vertical position*: one side-effect per line. Of course there is some specific order-of-operations for everything inside an individual line that the interpreter has to keep track of, but I basically never have to care about that myself. But by definition, := involves embedding side-effects within expressions, so suddenly I do have to care after all. Except... for the three cases Nick wrote above, where the side-effect occurs at the very end of the evaluation. And these also seem to be the three cases that have the most compelling use cases anyway. So restricting to just those three cases makes it much more palatable to me. (I won't comment on Nick's actual proposal, which is a bit more complicated than those examples, since it allows things like 'if m.group(1) given m = ...'.) (And on another note, I also wonder if all this pent-up desire to enrich the syntax of comprehensions means that we should add some kind of multi-line version of comprehensions, that doesn't require the awkwardness of explicitly accumulating a list or creating a nested function to yield out of. Not sure what that would look like, but people sure seem to want it.) -n [1] This is from "Go to statement considered harmful". Then a few lines later he uses a sequence of assignment statements as an example, and says that the wonderful thing about this example is that there's a 1-1 correspondence between lines and distinguishable program states, which is also uncannily apropos. -- Nathaniel J. Smith -- https://vorpus.org
On Fri, May 04, 2018 at 09:56:10PM -0400, Alexander Belopolsky wrote:
I'm not so sure that the "expression versus statement" dichotomy is as strong or as useful as Alexander says. If it were, we'd be writing much more imperative code, as if it were 1970 and we were using BASIC. Some of the greatest Python successes have been to add expression forms of what used to be purely imperative statements: - comprehensions (for-loops); - ternary if (if...else statement). In addition, we have more overlap between statements and expressions: - an expression form of def (lambda); - a functional form of the import statement (importlib.import_module, before that people used to use __import__(); - expression forms of augmented assignment (ordinary + etc operators), which is a rare case where the expression form came first and the imperative command came afterwards. I may have missed some. Also notable is that Python does not have "procedures" (pure statement callables). We use functions instead, and simply ignore the returned result. If the difference between statements and expressions was really a dichotomy, we would only need One Way to do these things, not two. I also have seen many people disappointed that Python doesn't treat "everything as an expression". The lack of assignment-expressions is a turn-off for some people: https://mail.python.org/pipermail/python-list/2018-May/732890.html See also: https://stackoverflow.com/questions/50090868/why-are-assignments-not-allowed... -- Steve
[Nick Coghlan <ncoghlan@gmail.com>]
I'm not clear on what "these specific cases" are, specifically ;-) Conditions in "if", "elif", and "while" statement expressions? Restricted to one "given" clause, or can they chain? In a listcomp, is it one "given" clause per "if", or after at most one "if"? Or is an "if" even needed at all in a listcomp? For example, [(f(x)**2, f(x)**3) for x in xs] has no conditions, and [(fx := f(x))**2, fx**3) for x in xs] is one reasonable use for binding expressions. [(fx**2, fx**3) for x in xs given fx = f(x)] reads better, although it's initially surprising (to my eyes) to find fx defined "at the end". But no more surprising than the current: [(fx**2, fx**3) for x in xs for fx in [f(x)]] trick.
The problem with complex targets in general assignment expressions is that, despite trying, I found no plausible use case for (at least) unpacking syntax. As in, e.g., x, y = func_returning_twople() if x**2 + y**2 > 9: # i.e., distance > 3, but save expensive sqrt The names can be _unpacked_ in a general assignment expression, but there appears to be no sane way then to _use_ the names in the test. This may be as good as it gets: if [(x, y := func_returning_twople()). x**2 + y**2 > 9][-1]: That reminds me of the hideous (condition and [T] or [F])[0] idiom I "invented" long ago to get the effect (in all cases) of the current T if condition else F That was intended to be goofy fun at the time, but I was appalled to see people later use it ;-) It''s certain sanest as if x**2 + y**2 > 9 given x, y = func_returning_twople(): "given" really shines there!
Which is OK. The one-letter variable name obscures that it doesn't actually reduce _redundancy_, though. That is, in the current match = pattern.search(data) if match: it's obviously less redundant typing as: if match := pattern.search(data): In if match given match = pattern.search(data): the annoying visual redundancy (& typing) persists.
# This name is rebound on each trip around the loop while m given m = pattern.search(remaining_data):
Also fine, but also doesn't reduce redundancy.
# "f(x)" is only evaluated once on each iteration result = [(x, y, x/y) for x in data if y given y = f(x)]
As above, the potential usefulness of "given" in a listcomp doesn't really depend on having a conditional. Or on having a listcomp either, for that matter ;-) r2, r3 = fx**2, fx**3 given fx = f(x) One more, a lovely (to my eyes) binding expression simplification requiring two bindings in an `if` test, taken from real-life code I happened to write during the PEP discussion: diff = x - x_base if diff: g = gcd(diff, n) if g > 1: return g collapsed to the crisp & clear: if (diff := x - x_base) and (g := gcd(diff, n)) > 1: return g If only one trailing "given" clause can be given per `if` test expression, presumably I couldn't do that without trickery. If it's more general, if (diff given diff = x _ xbase) and g > 1 given g = gcd(diff, n): reads worse to my eyes (perhaps because of the "visual redundancy" thing again), while if diff and g > 1 given diff = x - x_base given g = gcd(diff, n): has my eyes darting all over the place, and wondering which of the trailing `given` clauses executes first.
...
On 5 May 2018 at 13:36, Tim Peters <tim.peters@gmail.com> wrote:
Exactly the 3 cases presented (if/elif/while conditions). The usage in comprehensions would mirror the usage in if statements, and avoid allowing name bindings in arbitrary locations due to the implicity nested scopes used by comprehensions. Conditional expressions would be initially omitted since including them would allow arbitrary name bindings in arbitrary locations via the quirky "x if x given x = expr else x" spelling, and because "else" isn't as distinctive an ending token as "given ... :", "given ... )", "given ... ]", or "given ... }".
There were a couple key reasons I left the "for x in y" case out of the initial proposal: 1. The "for x in y" header is already quite busy, especially when tuple unpacking is used in the assignment target 2. Putting the "given" clause at the end would make it ambiguous as to whether it's executed once when setting up the iterator, or on every iteration 3. You can stick in an explicit "if True" if you don't need the given variable in the filter condition [(fx**2, fx**3) for x in xs if True given fx = f(x)] And then once you've had an entire release where the filter condition was mandatory for the comprehension form, allowing the "if True" in "[(fx**2, fx**3) for x in xs given fx = f(x)]" to be implicit would be less ambiguous. [snip]
Yep, that's why I don't have the same immediate reaction of "It would need to be limited to simple names as targets" reaction as I do for assignment expressions. It might still be a good restriction to start out with, though (especially if we wanted to allow multiple name bindings in a single given clause). [snip] The one-letter variable name obscures that it doesn't
Right, but that's specific to the case where the desired condition really is just "bool(target)". That's certainly likely to be a *common* use case, but if we decide that it's *that* particular flavour of redundancy that really bothers us, then there's always the "if expr as name:" spelling (similar to the way that Python had "a and b" and "a or b" logical control flow operators long before it got "a if c else b"). One more, a lovely (to my eyes) binding expression simplification
I was actually thinking that if we did want to allow multiple assignments, and we limited targets to single names, we could just use a comma as a separator: if diff and g > 1 given diff = x - x_base, g = gcd(diff, n): return g Similar to import statements, optional parentheses could be included in the grammar, allowing the name bindings to be split across multiple lines: if diff and g > 1 given ( diff = x - x_base, g = gcd(diff, n), ): return g (Other potential separators would be ";", but that reads weirdly to me since my brain expects the semi-colon to end the entire statement, and "and", but that feels overly verbose, while also being overly different from its regular meaning)
I find that last effect is lessened when using the comma as a separator within the given clause rather than repeating the keyword itself. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
[Nick]
And some people claim ":=" would make Python harder to teach ;-) [Tim]
I contrived that specific "use case", of course - I actually didn't stumble into any real code where multiple targets would benefit to my eyes. Perhaps because, as you noted above of `"for x in y" headers`, multiple-target assignment statements are often quite busy already too (I have no interest in cramming as much logic as possible into each line - but "sparse is better than dense" doesn't also mean "almost empty is better than sparse" ;-) ).
(especially if we wanted to allow multiple name bindings in a single given clause).
Right, but that's specific to the case where the desired condition really is just "bool(target)".
Not only. If the result _needs_ to be used N times in total in the test, binding expressions allow for that, but `given` requires N+1 instances of the name (the "extra one" to establish the name to begin with). For example, where `probable_prime()` returns `True` or `False`, and `bool(candidate)` is irrelevant: # highbit is a power of 2 >= 2; create a random prime # whose highest bit is highbit while (not probable_prime(candidate) given candidate = highbit | randrange(1, highbit, 2)): pass versus while not probable_prime(candidate := highbit | randrange(1, highbit, 2)): pass There I picked a "long" name to make the redundancy visually annoying ;-)
That's certainly likely to be a *common* use case,
In all the code I looked at where I believe a gimmick like this would actually help, it was indeed by far _most_ common that the result only needed to be used once in the test. In all such cases, the binding expression spelling of the test requires one instance of the name, and the `given` spelling two.
Reducing each redundancy is a small win to me, but reaches "importance" because it's so frequent. Binding expressions have more uses than _just_ that, though. But I'm sure I don't know what they all are. When a _general_ feature is added, people find surprising uses for it. For example, at times I'd love to write code like this, but can't: while any(n % p == 0 for p in small_primes): # divide p out - but what is p? Generator expressions prevent me from seeing which value of `p` succeeded. While that's often "a feature", sometimes it's a PITA. I don't know whether this binding-expression stab would work instead (I'm not sure the PEP realized there's "an issue" here, about the intended scope for `thisp`): while any(n % (thisp := p) == 0 for p in small_primes): n //= thisp If that is made to work, I think that counts as "a surprising use" (capturing a witness for `any(genexp)` and a counterexample for `all(genexp)`, both of which are wanted at times, but neither of which `any()`/`all()` will ever support on their own).. I suppose I could do it with `given` like so: while p is not None given p = next( (p for p in small_primes if n % p == 0), None): n //= p but at that point I'd pay to go back to the original loop-and-a-half ;-)
I expect that 's bound to be confusing, because the assignment _statement_ diff = x - x_base, g = gcd(diff, n) groups very differently than intended: diff = (x - x_base, g) = gcd(diff, n) And that's a syntax error. With enclosing parens, expectations change, and then people would expect it to work like specifying keyword arguments instead:
Keyword arguments work as a syntactic model (group as intended), but not semantically: if they really were keyword arguments, `x - x_base` and `gcd(diff, n)` would both be evaluated _before_ any bindings occurred. So it's more quirky `given`-specific rules no matter how you cut it. The closest bit of Python syntax that captures the grouping (but only partially), and the "left-to-right, with each binding in turn visible to later expressions" semantics, is the semicolon. Which would create even weirder expectations :-(
Yup.
I find that last effect is lessened when using the comma as a separator within the given clause rather than repeating the keyword itself.
Definitely. I tend to believe Python has "slightly more than enough" meanings for commas already, though. But using commas and _requiring_ parens for more than one `given` binding seems least surprising to me overall. Then again, everyone already knows what ":=" means. They just dislike it because so many major languages already have it -)
On Sun, May 06, 2018 at 02:00:35AM +1000, Nick Coghlan wrote:
o_O I'm replying to an email which is a week old. I haven't seen anyone other than Tim comment on that "if True" boilerplate. Are you still proposing that? If not, you can ignore the following.
I'm sorry, perhaps I'm having another slow day, but I don't see the ambiguity in the first place. And I *certainly* do not see how adding in a logically superfluorous "if True" would make it unambiguous. For comparison sake, here is the PEP 572 proposal compared to yours: [((fx := f(x))**2, fx**3) for x in xs] I'd read that as for each x in xs, let fx be f of x, return fx squared and fx cube [(fx**2, fx**3) for x in xs if True given fx = f(x)] which I'd read as for each x in xs, if True, return fx squared and fx cube, given fx is f of x and then wonder why on earth the test is there. Explain to me again why this boilerplate is necessary, please. Especially since you're already suggesting that in a future release it could be dropped without changing the meaning. [Tim Peters]
That really isn't the case. if result.method() given result = compare(data, arg): versus the PEP 572 syntax: if (result := compare(data, arg)).method(): So it certainly isn't just the bool(target) case that is redundant. -- Steve
On 06May2018 02:00, Nick Coghlan <ncoghlan@gmail.com> wrote:
I'm well behind, but... this! This turns "given" into a +0.8 for me. That's really nice. It reads clearly too. I was hitherto in the "expression as name" camp, which I gather is already rejected. Cheers, Cameron Simpson <cs@cskk.id.au>
On 13May2018 07:07, Cameron Simpson <cs@cskk.id.au> wrote:
And if we're still worried about new keywords or reserved words, this: if diff and g > 1 with ( diff = x - x_base, g = gcd(diff, n), ): return g or: if diff and g > 1 with ( x - x_base as diff, gcd(diff, n) as g ): return g or even: if diff and g > 1 with ( x - x_base, gcd(diff, n), ) as diff, g: return g read nearly as well to my eye. My main point here is that "with" works as well as "given" in this form from an English prose point of view. Cheers, Cameron Simpson <cs@cskk.id.au>
My main point here is that "with" works as well as "given" in this form from an English prose point of view.
+1 for "with...as", -1 for ":=" About affecting existing contexts, it seems that "with..as" would create a new context just for the expression, and the control statement it is embedded in, similar to what the current "with" statement does. These are semantics that are really easy to explain. Cheers!
On Sun, May 13, 2018 at 8:47 AM, Juancarlo Añez <apalala@gmail.com> wrote:
The trouble with every variant involving 'with' is that the semantics LOOK similar, but are subtly different. The current 'with' statement doesn't create a subscope; the only "context" it creates is regarding the resource represented by the context manager. For instance, opening a file in a 'with' block will close the file at the end of the block - but you still have a (closed) file object. Using "with... as" for name bindings wouldn't call __enter__ or __exit__, so it won't create that kind of context; and whether it creates a subscope for the variable or not, it's not going to match the 'with' statement. ChrisA
That is all valid, but it still would be familiar, and easier to explain. Python already uses "in", which is used in other languages to introduce context. The statement structure of "with...as" seems desirable, just asking for a word that is not "with" or "given". I don't remember if "when" was already rejected. http://www.thesaurus.com/browse/with?s=t http://www.thesaurus.com/browse/given?s=t http://www.thesaurus.com/browse/considering?s=t http://www.thesaurus.com/browse/assume?s=t http://www.thesaurus.com/browse/when?s=t Cheers! -- Juancarlo *Añez*
On Sat, May 12, 2018 at 07:36:33PM -0400, Juancarlo Añez wrote:
Python already uses "in", which is used in other languages to introduce context.
Fortunately, we don't have to come up with syntax that works with other languages, only Python.
The statement structure of "with...as" seems desirable
But it's not a statement, its an expression.
just asking for a word that is not "with" or "given".
How about "antidisestablishmentarianism"? That's unlikely to be used in many programs, so we could make it a keyword. *wink* I jest, of course. But I don't think "with" reads well, and given doesn't really work for me either *as prose*. In my experience mathematicians put the given *before* the statement: Given a, b, c three sides of a triangle, then Area = sqrt(s*(s-a)*(s-b)*(s-c)) where s = (a + b + c)/2 is the semi-perimeter of the triangle. For the record, that is almost exactly what I wrote for a student earlier today, and its not just me, it is very similar to the wording used on both Wolfram Mathworld and Wikipedia's pages on Heron's Formula. http://mathworld.wolfram.com/HeronsFormula.html https://en.wikipedia.org/wiki/Heron%27s_formula Putting "given" after the expression is backwards. -- Steve
On 2018-05-13 04:23, Steven D'Aprano wrote:
Yes, but that's because we're ruling out the use of "where". At this point I would be fine with "snicklefritz" as the keyword. The point is that I want to put SOMETHING after the expression, and this is not at all unusual. See for instance Wikipedia pages on the Reimann zeta function (https://en.wikipedia.org/wiki/Riemann_zeta_function#Definition), gravitation equation (https://en.wikipedia.org/wiki/Gravity#Newton%27s_theory_of_gravitation), and compound interest (https://en.wikipedia.org/wiki/Compound_interest#Mathematics_of_interest_rate...). If we have to use the word "given" even though the word mathematicians would use in that position is "where", that's not such a big deal. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Sun, May 13, 2018, 11:28 Brendan Barnwell <brenbarn@brenbarn.net> wrote:
it is a big deal. postfix requires more cognitive load, we will have no idea up front what's going on except for trivial exames. more givens, more cognitive load. if you think spending that is fine for you, I can't argue, but to say it doesn't matter isn't correct. 2.exames which get far worse for complex cases. left for the for can be as complex.as.you wish. 1: [ x + y for t in range(10) ... ] 2: x = 10 y = 20 [ x + y for t in range(10) ...] up till you read ... you have no idea there even will be a substitution. The lower is even worse, you think you know, but then have to redo the whole problem with new information. also : mathematicians don't just put the _word_ "given", they put givens, things that are known or assumed to be true. Axioms and definitions, where definitions assign names to values. This is for formal arguements. reassigning values is handled in post fix occasionally once it is clear what x and y are. but that's not what we are talking about if the name doesn't exist already. again, you want to use given, that's fine, but the math argument is wrong, as is the "it doesn't matter" argument, assuming the current neurological model for working memory continues to hold. Maybe the difference is small, especially after familiarity sets in, but that doesn't mean the difference in load isn't there. it will only increase for more complex statements with more givens.
On 2018-05-13 11:53, Matt Arcidy wrote:
Sorry, but that's nonsense. In the message you're replying to, what I'm arguing "doesn't matter" is the WORD used to do the postfixing. Are you seriously saying that the "neurological model for working memory" means that "x + y where x = foo" is easier to understand, on a fundamental neurological basis, than "x + y given x = foo"? Since you're the one who was advocating for objective measures, I'm sure you'll understand if I want some solid objective evidence for that! If you're saying that the entire concept of postfixing the givens (rather than putting them before the expression) creates greater cognitive load, then how do you explain the use of this format in the Wikipedia articles I linked, not to mention various math papers, texts, etc. throughout the world? Finally, even if we allow that postfixed-givens does increase cognitive load on a single, initial reading, that's not sufficient. Part of what I'm saying is that on LATER readings it's faster to see the overall expression first, because you don't have to plow through the definitions of the givens. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Sun, May 13, 2018 at 2:54 PM Matt Arcidy <marcidy@gmail.com> wrote:
I usually read the vertical bar "given" in set-builder notation: {x | 10 < x < 100, x is prime} Conditional probabilities, expectations, entropy, etc.: P(X | Y), E(X| Y), H(X | Y), And, the so-called "evaluation bar": https://math.stackexchange.com/questions/52651/what-is-the-name-of-the-verti... In words, I agree that where is probably better in text because writing "given the gravitational constant G" puts the subject last, unlike "where G is the gravitational constant". However, the given proposal for Python is putting the subject first: "given x = y".
Most (if not all) uses of the vertical are read as "given" and they all put the givens to the right of it.
I think Peter tried to outline this earlier, but what he was laying out wasn't clear to me at first. There seem to be 4 variations when it comes to assignment expressions. I'm going to try to ignore exact keywords here since we can sort those out once we have settled on which variation we prefer. 1. infix: TARGET := EXPR 2. infix: EXPR as TARGET 3. prefix: let TARGET = EXPR in ANOTHER_EXPR 4. postfix: ANOTHER_EXPR given TARGET = EXPR Both 1 and 2 may appear in the context of a larger expression where TARGET may or may not be used: 1. 99 + (TARGET := EXPR) ** 2 + TARGET 2. 99 + (EXPR as TARGET) ** 2 + TARGET 3 and 4 require that TARGET appear in ANOTHER_EXPR, even if TARGET is the only thing contained in that expression, whereas with 1 and 2, TARGET need not be used again. Example I: 1. x := 10 2. 10 as x 3. let x = 10 in x 4. x given x = 10 In the simple case where the goal of the assignment expression is to bind the EXPR to the TARGET so that TARGET can be used in a future statement, 1 and 2 are clearly the most straightforward because they do not require ANOTHER_EXPR. # Please ignore that m.group(2) doesn't do anything useful here Example II: 1. if m := re.match(...): m.group(2) 2. if re.match(...) as m: res = m.group(2) 3. if let m = re.match(...) in m: m.group(2) 4. if m given m = re.match(...): m.group(2) I also think expressions that use "or" or "and" to make a compound expression benefit from the infix style, mostly because each sub-expression stands on its own and is only made longer with the repetition of TARGET: Example III: 1. if (diff := x - x_base) and (g := gcd(diff, n)) > 1: ... 2. if (x - x_base as diff) and (gcd(diff, n) as g) > 1: ... 3. if (let diff = x - x_base in diff) and (let g = gcd(diff, n) in g > 1): ... 4. if (diff given diff = x - x_base) and (g > 1 given g = gcd(diff, n)): ... In the more complex case where TARGET is reused in the expression, I find 3 and 4 to benefit as there is a separation of the binding from its usage. I can consider each expression separately and I don't have to deal with the assignment side effects at the same time. I believe this is what Neil is mostly arguing for. # Borrowing from Andre, please forgive any mathematical problems like division by 0 Example IV: 1: [(-b/(2*a) + (D := sqrt( (b/(2*a))**2 - c/a), -b/(2*a) - D) for a in range(10) for b in range(10) for c in range(10) if D >= 0] 2: [(-b/(2*a) + (sqrt( (b/(2*a))**2 - c/a as D), -b/(2*a) - D) for a in range(10) for b in range(10) for c in range(10) if D >= 0] 3. [let D = sqrt( (b/(2*a))**2 - c/a) in (-b/(2*a) + D, -b/(2*a) - D) for a in range(10) for b in range(10) for c in range(10) if D >= 0] 4. [(-b/(2*a) + D, -b/(2*a) - D) for a in range(10) for b in range(10) for c in range(10) if D >= 0 given D = sqrt( (b/(2*a))**2 - c/a)] Also in the case with multiple bindings I find that 3 and 4 benefit over 1 and 2: Example V: 1. [(x := f(y := (z := f(i) ** 2) + 1)) for i in range(10)] 2. [(f((f(i) ** 2 as z) + 1 as y) as x) for i in range(10)] 3. [let x = f(y), y = z + 1, z = f(i) ** 2 in x for i in range(10)] # maybe the order of the let expressions should be reversed? 4. [x given x = f(y) given y = z + 1 given z = f(i) ** 2 for i in range(10)] No matter which variation we prefer, there are plenty of arguments to be made that multiple assignment expressions in a single expression or usage of the TARGET later in the expression is harder to work with in most cases,. And since 1 and 2 (at least to me) are more difficult to parse in those situations, I'm more likely to push back on whoever writes that code to do it another way or split it into multiple statements. I feel that Steven prefers 1, mostly for the reason that it makes Examples I, II, and III easier to write and easier to read. Neil prefers 4 because Examples I, II, and II still aren't that bad with 4, and are easier to work with in Examples IV and V. If you feel that Examples IV and V should be written differently in the first place, you probably prefer infix (1 or 2). If you feel that Examples IV and V are going to be written anyway and you want them to be as readable as possible, you probably prefer prefix (3) or postfix (4). If you want to know what all the TARGETs are assigned to up front, you probably prefer 1 or 3 (for reading from left to right). If you want to see how the TARGET is used in the larger expression up front and are willing to read to the end to find out if or where the TARGET has been defined, you probably prefer 4. In my mind, all 4 variations have merit. I think I prefer prefix or postfix (postfix feels very natural to me) because I believe more complex expressions should be separateable (Neil argues better than I can for this). But Steven has gone a long way to convince me that the sky won't fall if we choose an infix variation because in practice our better angels will push us away from using expressions that are too complex. Prefix vs postfix is a discussion worth having if we decide that infix isn't the right choice. I would love to see us reach consensus (too optimistic?) or at least an acknowledgment of the explicit tradeoffs for whichever variation we ultimately choose. -- Nick ----- Original message ----- From: Matt Arcidy <marcidy@gmail.com> To: Brendan Barnwell <brenbarn@brenbarn.net> Cc: "python-ideas" <python-ideas@python.org> Subject: Re: [Python-ideas] Inline assignments using "given" clauses Date: Sun, 13 May 2018 11:53:20 -0700 On Sun, May 13, 2018, 11:28 Brendan Barnwell <brenbarn@brenbarn.net> wrote:
it is a big deal. postfix requires more cognitive load, we will have no idea up front what's going on except for trivial exames. more givens, more cognitive load. if you think spending that is fine for you, I can't argue, but to say it doesn't matter isn't correct. 2.exames which get far worse for complex cases. left for the for can be as complex.as.you wish. 1: [ x + y for t in range(10) ... ] 2: x = 10 y = 20 [ x + y for t in range(10) ...] up till you read ... you have no idea there even will be a substitution. The lower is even worse, you think you know, but then have to redo the whole problem with new information. also : mathematicians don't just put the _word_ "given", they put givens, things that are known or assumed to be true. Axioms and definitions, where definitions assign names to values. This is for formal arguements. reassigning values is handled in post fix occasionally once it is clear what x and y are. but that's not what we are talking about if the name doesn't exist already. again, you want to use given, that's fine, but the math argument is wrong, as is the "it doesn't matter" argument, assuming the current neurological model for working memory continues to hold. Maybe the difference is small, especially after familiarity sets in, but that doesn't mean the difference in load isn't there. it will only increase for more complex statements with more givens.
_________________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Sun, May 13, 2018 at 4:30 PM Nick Malaguti <python@fwdaddr.fastmail.fm> wrote:
Yes.
In my mind, your given clauses are upside-down. The way I see it, code like this: for a in range(10): if a != 5: for b in range(10): for c in range(10): D = sqrt((b/(2*a))**2 - c/a) if D >= 0: yield (-b/(2*a) + D, -b/(2*a) - D) should be cast into a generator like this: ((-b/(2*a) + D, -b/(2*a) - D) for a in range(10) if a != 5 for b in range(10) for c in range(10) given D = sqrt( (b/(2*a))**2 - c/a) if D >= 0) Just leave everything in the order you wrote it except the yield statement turns into a bare expression in front. Regarding prefix versus postfix, we already do postfix binding using "for" clauses and I want to be able to interleave given clauses with for clauses as above to prevent a bracketing mess that prefix would require.
On 13May2018 08:55, Chris Angelico <rosuav@gmail.com> wrote:
For myself, I'm not a fan of a narrow scope. I'd be happy with an inline assignment landing in the function scope (or whatever the current innermost scope is). Like normal assignments. In how many other places does Python adopt a narrower scope than the function/method/class/module? Consider: r = re.compile("bah(baz)") if m given m = r.match("foo barbaz"): thing = m.group(1) I _want_ to access "m" after the expression. My other dislike of narrow scopes is shadowing. I had an unpleasant experience last year working in Go, which has its own ":=" assignment. While Go's := semantics are a little different to the assignment expressions being considered here, it has an IMO dangerous shadowing effect. I'm going to digress into an explaination of this issue in Go here because I want to highlight my dislike of easy accidental shadowing, which is a problem in many areas, and I think that conflating inline assignment with an implied narrow scope in Python would make this worse. In Go you can declare variables 2 ways: var ok = True and: ok := True Also, because Go doesn't use exceptions a common idiom is to return a success/fail value and the task's result: ok, result := do_something(...) That handily declares "ok" and "result" and gives them values. Because the ":=" is so convenient, you might often declare variables as they get used: ok, result1 := do_something() ok, result2 := do_something_else() and so on. You're allowed to mix existing names with new (undeclared) names provided there's at least one new name left of the ":=". So far this is all just ordinary assignments, with convenient declaration included. But consider the function that calls other functions: func thing_outer(x) { ok, result1 := do_something() if ok { ok, result2 := do_something_else() if ok { fmt.Println("ok!") } } return ok, result } That's how it might go in Pythonic form. But Go lets you preceed the test condition with a statement, somewhat like the C "for (setup values; test; advance)" form: for (i=0; i<10; i++) So you may wish to write the function like this: func thing_outer(x) { if ok, result1 := do_something(); ok { if ok, result2 := do_something_else(); ok { fmt.Println("ok!") } } return ok, result } and here is where the scoping causes a disaster. In Go, the declarations _within_ the "if" statement have the "if" statement as their scope. In particular, the inner "ok" shadows the outer "ok", such that an inner failure (setting "ok" to false) doesn't affect the outer "ok" which was true. And so the overall function returns apparent success. And that is entirely enabled by the implied scoping in the "if" statement. The above example is subtly wrong because I'm doing this from memory, but I had a very simple real world example bite me this way and it was a PITA to debug because the effect was so surprising. So much so that from then on I pretty much eschewed the ":=" declaration as a dangerous construct and went with "var" all the time, effectively giving me _one_ scope within any function. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sat, May 12, 2018 at 5:54 PM Cameron Simpson <cs@cskk.id.au> wrote:
I love given, but that's the one thing I don't like. I prefer this: if (diff and g > 1 given diff = x - x_base given g = gcd(diff, n)): return g —just like for and if subexpressions. Doing this can also open up weirdness if someone tries to roll something like: a = f(), # Make a tuple of length 1 into a given statement. Now, where do you up the parentheses? given ( a = (f(),), b = whatever? ) Seems weird.
On 12/05/2018 23:52, Neil Girdhar wrote:
I don't like the consecutive "given"s. Reading it aloud in English suggests to me that the second "given" should be evaluated before the first, which I'm sure is not the intention. (Just as I think that multiple for-loops inside a comprehension sound the wrong way round. :-( But that ship has sailed.) Rob Cliffe
[attributions lost - sorry, but I can't get 'em back] ...
I'm well behind, but... this! This turns "given" into a +0.8 for me.
That's really nice. It reads clearly too.
Since that was my example to begin with, I think it's fair to point out that they all miss a key part of the original example: this code is working with multi-thousand bit integers, and calling gcd() is expensive. It was a key point that if (diff := x - x_base) and (g := gcd(diff, n)) > 1: return g didn't call gcd() _at all_ unless `diff` was non-zero. The original real-life code was: diff = x - x_base if diff: g = gcd(diff, n) if g > 1: return g
if (diff := x - x_base) and (g := gcd(diff, n)) > 1: return g
I don't see the advantage in that succinctness: g = special_gcd(x - x_base, n) if g: return g The code bases I work on constantly move towards having the next guy grok what's going on just by reading the code. It could also be: if special_gcd(x - x_base, n) as g: return g Cheers! Juancarlo *Añez*
On Sun, May 13, 2018 at 10:27 AM, Juancarlo Añez <apalala@gmail.com> wrote:
Now I have to go read elsewhere to figure out what "special_gcd" does. With Tim's original code, I could see the effect right there. It might not seem significant with a single function, but if you had this situation come up a dozen times, now you have a dozen functions, each one used only once. You have to go a LONG way out of line to find the meaning of this name. Remember: Giving a function a useful name means figuring out a name that means you do not need to read the function's body to understand what it does. It's easy to say "just make it a function", but that's utterly useless if the next reader has to grok the function's implementation to understand its usage. ChrisA
On Sat, May 12, 2018 at 08:27:53PM -0400, Juancarlo Añez wrote:
That's an excellent point. What's "special_gcd" and how does it differ from normal gcd? How am I supposed to grok that just from reading the code above? Do I have to dig into the source of special_gcd to understand it? What happens if the normal gcd would return zero? Is that going to lead to a bug?
No it can't, because "as" is not an option. -- Steven
On Mon, May 14, 2018 at 12:20 AM, Juancarlo Añez <apalala@gmail.com> wrote:
Okay, then. What happens to the succinctness? On Sun, May 13, 2018 at 10:27 AM, Juancarlo Añez <apalala@gmail.com> wrote:
It's actually this: def special_gcd(diff, n): return diff and gcd(diff, n) g = special_gcd(x - x_base, n) if g: return g Yes, very succinct. You can't have it both ways; either you need a name that completely defines the function, such that it can be placed out-of-line without needing to be read; or you're just creating an inline helper function, which doesn't shorten your code at all, and just adds another layer of wrapping around everything. Remember, you're contrasting with one of two options: either a nested if, or an inline 'and' with the ability to capture values. What you've created here is a multi-line way of capturing a value. That's all. ChrisA
On 13May2018 00:51, MRAB <python@mrabarnett.plus.com> wrote:
Yes, but in a slightly larger scope such as a cascaded if/elif/.../else sequence embedding the "given" into the expression has 2 advantages: it stops the if-else cascading across the terminal purely to embed assignments before the next test, and it lets one clearly associate these particular assignments as relevant to the test itself. Obviously there are circumstances for each, and I expect overuse of "given" degrades readability. However, I still find Nick's "if this given that" formulation quite compelling as a logical form. Cheers, Cameron Simpson <cs@cskk.id.au>
On Fri, May 4, 2018 at 3:06 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Well, I think it looks good. Nice job Nick! I like that it leaves the "=" alone finally (but it's maybe the reason why I am biased towards this syntax :) It has clear look so as not to confuse with conditionals and the variable is emphasized so it's well understood what it means. I wish I could like the whole idea of inline assignments more now - but well, it is just what it is. Something that can help on rare occasion. But as said, your variant feels right to me. Unlike with most other proposed constructs here my first impression - this is Python. If the new keyword is too much, there are some options too choose from. (though I've no idea whether this is plausible due to parsing ambiguities) if m given m = pattern.search(data): vs: if m with m = pattern.search(data): if m def m = pattern.search(data): if m as m = pattern.search(data): Yes, all these reads strange, but IMO understandable. BTW, using your keyword, in comprehensions vs the original idea of Chris then merely I suppose: result = [(x, y, x/y) given y = f(x) for x in data if y ] vs result = [(x, y, x/y) with y = f(x) for x in data if y ] result = [(x, y, x/y) def y = f(x) for x in data if y ] result = [(x, y, x/y) as y = f(x) for x in data if y ] vs result = [(x, y, x/y) for x in data if y given y = f(x)] vs result = [(x, y, x/y) for x in data if y with y = f(x)] result = [(x, y, x/y) for x in data if y def y = f(x)] result = [(x, y, x/y) for x in data if y as y = f(x)] I can't say for sure what is better but the former order seems to be slightly more appealing for some reson. [Tim]
I disagree, I think it is not redundant but it's just how it is - namely it does not introduce 'implicitness' in the first place, and that is what I like about Nick's variant. But of course it is less compact. PS: recently I have discovered interesting fact: it seems one can already use 'inline assignment' with current syntax. E.g.: if exec("m = input()") or m: print (m) It seem to work as inline assignment correctly. Yes it is just coincidence, BUT: this has the wanted effect! - hides one line in "if" and "while" - leaves the assignment statement (or is it expression now? ;) - has explicit appearance So we have it already? Mikhail
On 6 May 2018 at 00:33, Mikhail V <mikhailwas@gmail.com> wrote:
Not really, since: 1. exec with a plain string has a very high runtime cost (the compiler is pretty slow, since we assume the results will be cached) 2. exec'ed strings are generally opaque to static analysis tools 3. writing back to the local namespace via exec doesn't work consistently at function scope (and assuming PEP 558 is eventually accepted, will some day reliably *never* work) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 4 May 2018 at 22:06, Nick Coghlan <ncoghlan@gmail.com> wrote:
Since I genuinely don't think this idea is important enough to disrupt hypothesis's public API, I've been pondering potential synonyms that are less likely to be common in real world code, while still being comprehensible and useful mnemonics for the proposed functionality. The variant I've most liked is the word "letting" (in the sense of "while letting these names have these values, do ..."): # Exactly one branch is executed here if m letting m = pattern.search(data): ... elif m letting m = other_pattern.search(data)): ... else: ... # This name is rebound on each trip around the loop while m letting m = pattern.search(remaining_data): ... # "f(x)" is only evaluated once on each iteration result = [(x, y, x/y) for x in data if y letting y = f(x)] # Tim's "bind two expressions" example if diff and g > 1 letting diff = x - x_base, g = gcd(diff, n): return g # The "bind two expressions" example across multiple lines while diff and g > 1 letting ( diff = x - x_base, g = gcd(diff, n), ): ... # Do something with diff and g Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 6 May 2018 at 02:06, Nick Coghlan <ncoghlan@gmail.com> wrote:
I've also been pondering Tim's suggestion of instead enhancing the code generation pipeline's native support for pseudo-keywords in a way that can be explicitly represented in the Grammar and AST, rather than having to be hacked in specifically every time we want to introduce a new keyword without a mandatory __future__ statement (and without creating major compatibility headaches for code that uses those new keywords as attribute or variable names). While I haven't actually tried this out yet, the way I'm thinking that might look is to add the following nodes to the grammar: name_plus: NAME | pseudo_keyword pseudo_keyword: 'given' and the replace all direct uses of 'NAME' in the grammar with 'name_plus'. That way, to allow a new keyword to be used as a name in addition to its syntactic use case, we'd add it to the pseudo_keyword list in addition to adding it to . To avoid ambiguities in the grammar, this could only be done for keywords that *can't* be used to start a new expression or statement (so it wouldn't have been sufficient for the async/await case, since 'async' can start statements, and 'await' can start both statements and expressions). So if Guido's view on the out-of-order execution approach to inline name binding softens, I think this would be a better approach to pursue than making a more awkward keyword choice. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, 4 May 2018 at 05:07 Nick Coghlan <ncoghlan@gmail.com> wrote:
My brain wants to drop the variable name in front of 'given': if given m = pattern.search(data): while given m = pattern.search(remaining_data): Maybe it's because the examples use such a short variable name? if match given match = pattern.search(data): vs. if given match = pattern.search(data); Nope, I still like mine more. ;) -Brett
On 8 May 2018 at 04:19, Brett Cannon <brett@python.org> wrote:
Does that change if the condition isn't just "bool(name)"? For example: if y > 0 given y = f(x): ... That's the situation where I strongly prefer the postfix operator spelling, since it's pretty clear how I should pronounce it (i.e. "if y is greater than zero, given y is set to f-of-x, then ..."). By contrast, while a variety of plausible suggestions have been made, I still don't really know how to pronounce "if (y := f(x)) > 0:)" in a way that's going to be clear to an English-speaking listener (aside from pronouncing it the same way as I'd pronounce the version using "given", but that then raises the question of "Why isn't it written the way it is pronounced?"). I do agree with Tim that the name repetition would strongly encourage the use of short names rather than long ones (since you're always typing them at least twice), such that we'd probably see code like: while not probable_prime(n) given (n = highbit | randrange(1, highbit, 2)): pass Rather than the more explicit: while not probable_prime(candidate) given (candidate = highbit | randrange(1, highbit, 2)): pass However, I'd still consider both of those easier to follow than: while not probable_prime(candidate := highbit | randrange(1, highbit, 2)): pass since it's really unclear to me that "candidate" in the latter form is a positional argument being bound to a name in the local environment, and *not* a keyword argument being passed to "probable_prime". I've also been pondering what the given variant might look like as a generally available postfix operator, rather than being restricted to if/elif/while clauses, and I think that would have interesting implications for the flexibility of its usage in comprehensions, since there would now be *three* places where "given" could appear (as is already the case for the inline binding operator spelling): - in the result expression - in the iterable expression - in the filter expression That is: [(x, y, x - y) given y = f(x) for x in data] [(x, data) for x in data given data = get_data()] [(x, y, x/y) for x in data if y given y = f(x)] Rather than: [(x, y := f(x), x - y) for x in data] [(x, data) for x in data := get_data()] [(x, y, x/y) for x in data if y := f(x)] Opening it up that way would allow for some odd usages that might need to be discouraged in PEP 8 (like explicitly preferring "probable_prime(n) given n = highbit | randrange(1, highbit, 2)" to "probable_prime(n given n = highbit | randrange(1, highbit, 2))"), but it would probably still be simpler overall than attempting to restrict the construct purely to if/elif/while. Even as a generally available postfix keyword, "given" should still be amenable to the treatment where it could be allowed as a variable name in a non-operator context (since we don't allow two adjacent expressions to imply a function call, it's only prefix keywords that have to be disallowed as names to avoid ambiguity in the parser). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I'm sorry, but unless there's a sudden landslide of support for 'given' in favor of ':=', I'm really not going to consider it. I'd pronounce "if (x := y) > 0" as either "if y (assigned to x) is greater than zero" or "if x (assigned from y) is greater than zero". On Thu, May 10, 2018 at 6:39 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On 10/05/18 14:44, Guido van Rossum wrote:
I'm sorry, but unless there's a sudden landslide of support for 'given' in favor of ':=', I'm really not going to consider it.
OK, this is my ha'p'th in favour of 'given', for what little it's worth. The more I see of general assignment expressions, the less I like them. All I really want is a less clumsy way to write while true: thing_to_do = get_something_to_do() if thing_to_do == GET_OUT_OF_HERE: break # else do stuff -- Rhodri James *-* Kynesim Ltd
If it just needs a stream of +1s, I personally like the "given" approach much more than the ":=" approach, for all of the many reasons repeated many times in the various email chains. (I preferred it as "as", but that's been struck down already) (and if it's between ":=" and not having them at all, I would rather just not have them)
Please no, it's not that easy. I can easily generate a stream of +1s or -1s for any proposal. I'd need well-reasoned explanations and it would have to come from people who are willing to spend significant time writing it up eloquently. Nick has tried his best and failed to convince me. So the bar is high. (Also note that most of the examples that have been brought up lately were meant to illustrate the behavior in esoteric corner cases while I was working out the fine details of the semantics. Users should use this feature sparingly and stay very far away of those corner cases -- but they have to be specified in order to be able to implement this thing.) On Thu, May 10, 2018 at 10:26 AM, marky1991 . <marky1991@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On 2018-05-10 16:10, Guido van Rossum wrote:
Poor prospects, then, but I'll do my best. I think the most obvious argument (to me) favouring `given` over `:=` is that it separates the two things it's doing: if m.group(2) given m = pattern.search(data): as opposed to the more-nested := version: if (m := pattern.search(data)).group(2): which, at least to me, is more complicated to think about because it feels like it's making the .group() something to do with the assignment. Put another way, I think your use of parentheses when discussing the *pronunciation* of this thing is telling. It feels as though one needs to start in the middle and then go in both directions at once, first explaining the origin (or destination) of the operand in question in a parenthesized offshoot, and then switching context and describing what is done to it. It's midly mentally taxing. I'm sure we can all live with that, but I don't want to: Python's exceptionally-readable syntax is one of the bigger reasons I choose it. There's a striking parallel in C, where the well-known idiom: while ((c = getchar()) != EOF) ... has an obviously-nicer alternative: while (c = getchar(), c != EOF) ... Most people I show this to agree that it's nicer, despite the fact that it manages to repeat a variable name *and* use the comma operator. I don't have proof, but I'd suggest that unwrapping that layer of context for the reader imparts a significant benefit. The C example also provides a convenient test: if you think the former example is nicer, I can just give up now ;)
On Thu, 10 May 2018 at 16:49, Ed Kellett <e+python-ideas@kellett.im> wrote:
IMHO, all these toy examples don't translate well to the real world because they tend to use very short variable names while in real world [good written code] tends to select longer more descriptive variable names. Try replacing "c" with a longer name, like input_command, then it becomes: while ((input_command = getchar()) != EOF) ... while (input_command = getchar(), input_command != EOF) ... In the second example, having to type the variable name twice is an annoyance that adds almost nothing to readability, so I would definitely prefer the first one. The "given" proposals have the same issue. (a shame we can't use "as", for reasons already stated, it would have been perfect otherwise) -- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert
Gustavo Carneiro wrote:
I don't believe that's always true. It depends on the context. Sometimes, using long variable names can make code *harder* to read. I don't think there's anything unrealistic about this example: if m given m = pattern.match(the_string): nugget = m.group(2) Most people's short-term memory is good enough to remember that "m" refers to the match object while they read the next couple of lines. IMO, using a longer name would serve no purpose and would just clutter things up. -- Greg
On 11.05.2018 09:33, Greg Ewing wrote:
I gather we don't talk about interactive usage. So, I grepped through part of our code-base. Like it or not, it's almost always called "match" there. Like Gustavo, I also have the feeling that long-living, real-world code tends to have more descriptive names than all those toy examples/arguments. Cheers, Sven
On 11 May 2018 at 03:33, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I've been thinking about this problem, and I think for the If/elif/while cases it's actually possible to allow the "binding is the same as the condition" case to be simplified to: if command = pattern.match(the_string): ... elif command = other_pattern.match(the_string): ... while data = read_data(): ... Allowing this would be part of the definition of the if/elif/while statement headers, rather than a general purpose assignment expression. The restriction of the LHS to a simple name target would need to be in the AST generator rather than in the grammar, but it's hardly the only case where we do that kind of thing. Switching to the given expression form would then only be necessary in cases where the condition *wasn't* the same as the binding target. A similar enhancement could be made to conditional expressions (adjusting their grammar to permit "EXPR if NAME = EXPR else EXPR") and filter clauses in comprehensions (allowing "EXPR for TARGET in EXPR if NAME = EXPR"). In essence, "if", "elif", and "while" would all allow for an "implied given" clause in order to simplify the 90% case where the desired condition and the bound expression are the same. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 11.05.2018 13:43, Nick Coghlan wrote:
I can imagine that this will cover 80 to 90% of the usecases and it's readable as well.
Not sure if that is too much for now. List comprehensions tend to be longer than expected, the same goes for the ternary expression. Maybe, we could start with the 90% case and whether the need for more arises.
Exactly. Maybe, even here: let's do just the 90% case. And if a lot of people need finer control, we can reconsider :=/given/as. Regards, Sven
[Gustavo Carneiro]
[Greg Ewing]
[Nick Coghlan]
Unless there's some weird font problem on my machine, that looks like a single "equals sign". In which case we'd be reproducing C's miserable confusion about whether: if (i = 1) was a too-hastily-typed spelling of the intended: if (i == 1) or whether they were thinking "equals" and typed "=" by mistake. If so, that would get an instant -1 from any number of core devs, who have vivid painful memories of being burned by that in C. That's not just speculation - it came up a number of times in the PEP 572 threads.
Spell it ":=" (colon equals) instead, and a few core devs would stop objecting ;-)
On Fri, May 11, 2018 at 3:33 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Indeed. A thought just occurred to me. Maybe we need to instigate a cultural shift where people think about style guides as less dictated by hard-coded rules that were "passed down from the mountain" and more as derived from research that we can all understand about usability. A lot more is known about how human perception and various types of memory and learning work than it was when the "7 things plus/minus 2" rule was invented ( https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two). It would be fascinating to imagine a future where language designers could talk about such topic with as much confidence as they talk about the efficiency of hash tables. -- --Guido van Rossum (python.org/~guido)
On Fri, May 11, 2018 at 12:33:10PM -0400, Guido van Rossum wrote:
That would be fantastic, but it's a real uphill battle. How long have we known about the effects of low-contrast on readability? https://www.wired.com/2016/10/how-the-web-became-unreadable/ http://contrastrebellion.com/ And don't get me started on the awful, awful choices made for chart design and data visualisation. We know how to make good charts: https://en.wikipedia.org/wiki/Edward_Tufte and yet: https://i.redd.it/0w0y1r1ba8x01.jpg -- Steve
On 2018-05-10 17:10, Gustavo Carneiro wrote: literally hundreds of others do the same, and I've never seen anyone spell it `input_command`. I do the same in Python in a few contexts, usually where the variable's meaning is very clear from its usage, or where the code in question doesn't know or care what the variable is used for: for i in range(10): def get_const(self, x): for k in self.defaults.keys(): etc., and if we had `given` clauses, I'd probably do this there too. I think one of the advantages of that syntax is that it makes it extremely clear what's going on: if m given m = re.match(...): I don't need to try to stuff an English description of m into its name, because the `given` clause already describes it perfectly.
On 10 May 2018 at 11:10, Guido van Rossum <guido@python.org> wrote:
I raised this with some of the folks that were still here at the Education Summit (similar to what I did for data classes at the PyCon Australia education seminar last year), but whereas the reactions to data classes were "as easy or easier to teach as traditional classes", the reaction for this for the folks that I asked was almost entirely negative - the most positive reaction was "Yes, if it's as a wholesale replacement for the '=' spelling, since that sometimes gets confused with mathematical equality". As an *addition* to the existing spelling, and especially with the now proposed leaking semantics in comprehension scopes, it was "No, that would just confuse out students". It's one thing adding syntactic and semantic complexity for the sake of something that significantly increases the language's expressive power (which is what the original sublocal scopes proposal was aiming for: the ability to more readily express constrained scoping and name shadowing without explicit name aliasing and del statements), it's something else entirely to do it for the sake of purely cosmetic tweaks like flattening the occasional nested if-else chain or replacing a loop-and-a-half with an embedded assignment. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hi all. I've been lurking for a little while on this discussion and I thought I might contribute some thoughts. One of my hurdles for ":=" is understanding when I should use it rather than "=". Should I use it everywhere? Should I use it only where I can't use regular "="? Is it a personal choice? Will it become so common that I need to think harder because some people will use it really frequently or intermix them? I don't want to see "=" vs ":=" become like semicolons in JavaScript. When I work on a different codebase, am I going to have to follow an "always" or "never" for binding expressions? Maybe this is all overblown and PEP8 direction will keep everyone on the same page, but I guess I worry about there being 2 very similar, but not the same, ways to do it. What I really like about "given" is it makes it a lot clearer when I should use it. No one is going to want to write x given x = f() if they can just write x = f() If you need a binding expression in a comprehension or an if or while statement, you'll know the pattern of using "given" to save that loop and a half or to call a function and bind its result while iterating. Just like you know when to use a ternary if to save that extra temporary variable - there's little confusion about when to use a ternary, especially since a few if statements quickly prove clearer to read. 10 if x == 5 else 9 if x == 2 else 8 if x == 3 else 100 looks much better as: if x == 5: result = 10 elif x == 2: result = 9 elif x == 3: result = 8 else: result = 100 I feel the same way about given. If you feel tempted to go overboard with: x given x = y * 2 given y = z + 3 given z = f() Which should be equivalent to: x := (y := ((z := f()) + 3)) * 2 hopefully you'll think, "maybe I should just make 3 statements instead?" And also I have no trouble following what that statement actually does when using given. I didn't need any parenthesis to make sure I didn't bind the wrong expressions and I don't have to read it from the inside out. Each sub-expression is complete rather than being mixed together (even though I have to read it from right to left). I feel like the strongest argument for ":=" is for all the situations where someone will actually want a binding expression in real code, ":=" is more succinct. I'm just concerned that when given a new binding expression hammer, everything is going to look like a nail and all the places where someone could really benefit from a binding expression will be drowned out by the unnecessary usage of ":=" (and its side effects). -- Nick ----- Original message ----- From: Guido van Rossum <guido@python.org> To: "marky1991 ." <marky1991@gmail.com> Cc: "python-ideas" <python-ideas@python.org> Subject: Re: [Python-ideas] Inline assignments using "given" clauses Date: Thu, 10 May 2018 11:10:50 -0400 Please no, it's not that easy. I can easily generate a stream of +1s or -1s for any proposal. I'd need well-reasoned explanations and it would have to come from people who are willing to spend significant time writing it up eloquently. Nick has tried his best and failed to convince me. So the bar is high. (Also note that most of the examples that have been brought up lately were meant to illustrate the behavior in esoteric corner cases while I was working out the fine details of the semantics. Users should use this feature sparingly and stay very far away of those corner cases -- but they have to be specified in order to be able to implement this thing.) On Thu, May 10, 2018 at 10:26 AM, marky1991 . <marky1991@gmail.com> wrote:> If it just needs a stream of +1s, I personally like the "given"
-- --Guido van Rossum (python.org/~guido) _________________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, May 10, 2018 at 10:31:00PM -0400, Nick Malaguti wrote:
When should I use "x = x + 1" and when should I use "x += 1"? When should I write alist.sort() and when sorted(alist)? When should I use a list comp and when should I use a for-loop? I could give a dozen more examples. We always have a choice in writing code. There is almost never *only* one way to do it. Nevertheless, we manage, and I believe that most of the things which perplex us today will be so trivially easy to understand tomorrow that we'll wonder what the fuss was all about. It is sometimes humbling to remember back at the things that we swore were terrible, terrible mistakes. I hated augmented assignment, until I gave in and started using them. I hated != instead of <> and I actually seriously considered using "from __future__ import barry_as_FLUFL" for a while. Depending on whether you are a "glass half full" or "glass half empty" kind of guy, you could say either that: - we're irrationally risk adverse, and would rather miss out on something fantastic than risk something slightly not so good; - or like Stockholm Syndrome victims, we can get used to, and even learn to enjoy, the most awful crap if it goes on long enough. (Or possibly both at the same time.) Personally, I think that in hindsight my dislike of != was irrational and rather silly, rather than a prophetic realisation that the dropping of <> began the ruination of Python. YMMV.
I don't want to see "=" vs ":=" become like semicolons in JavaScript.
The difference between = and := is nothing like automatic semicolon insertion in Javascript. The Python interpreter is never going to (only sometimes) insert a colon to make your "name = expression" work and it is never going to turn two statements into a single broken expression because you used = instead of := or vice versa. Of course people can still screw up the precedence and get the wrong results, but the same applies to all operators and the same solution applies: when in doubt, add parentheses to make it clear. Some arguments against := are better than others: will it encourage people to write unreadable one-liners instead of using multiple statements? Maybe, but I think most people will show more sense and restraint. You haven't seen many quadruply-nested list comprehensions, or ternary if operators nested ten deep, have you? I haven't.
When I work on a different codebase, am I going to have to follow an "always" or "never" for binding expressions?
Yes, of course you are. Some places will be conservative and say Never, and some will be gung ho and say Always, but the majority will say "Use them when they make the code better". Just like when you work on some code bases you will have to always follow PEP 8, and when you work on other code bases, you will have to follow different rules.
You seem to have two contradictory opinions here. Paraphrasing: "I worry that the same people who never abuse ternary if by writing obfuscated one-liners will suddenly turn around and abuse := binding expressions by writing obfuscated one-liners." and "clearly nobody will abuse 'given' binding expressions by writing obfuscated one-liners, because they don't abuse ternary if to write obfuscated one-liners." I understand being pessimistic about the common-sense of my fellow programmers. I understand being optimistic about the common-sense of my fellow programmers. I don't understand doing both at the same time. If people will abuse := they will abuse "given". If they won't abuse "given", they surely won't abuse := either. Since we have over a quarter of a century of experience showing that the Python community, as a whole, tends not to abuse syntax to write unreadable one-liners, I think that fears about people abusing := are overblown. -- Steve
Guido van Rossum wrote:
I'd need well-reasoned explanations
My reasoning is essentially the same as what I've already said about "where". To summarise, "given" sounds like something an English-speaking mathematician would write, whereas ":=" doesn't even have an obvious pronunciation. Some variation on "given" just seems greatly more pythonic to me. -- Greg
+1 to this reasoning. One of the main reason python is popular is because code is easy to read, while ":=" would clearly not be as readable as "given". For me the difference between "given" and ":=" is the same as between python and C for loops. On Fri, 11 May 2018 at 09:06 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I dont really like "given". If we compare: if m given m = re.match(stuff): to if m := re.match(stuff) then I count 4+(name_length) more tokens and 2 more spaces. Since I believe := is perfectly clear, I don't see the reason for a far more verbose syntax. That all said, I would still prefer: if re.match(stuff) as m: which is exactly equal to the := in line length and parallels with. While that may -technically- be a different beast. For beginners the difference is really irrelevant, and you can just tell advanced people the full story(technically speaking the as in a with statement isn't an expression assignment, it's a part of the with statement, and it feeds in the value through the context manager machinery before binding it. Similar for the except statement.). But I've kind of given up on "as" (so no need to reply on that bit).
On 11/05/18 11:14, Jacco van Dorp wrote:
I respectfully disagree with your opinion (i.e. you're wrong :-) Consider: while (cmd := get_command()).token != CMD_QUIT: cmd.do_something() vs: while cmd.token != CMD_QUIT given cmd = get_command(): cmd.do_something() I find I write code like this[*] a fair bit, since my major use for Python is to write remote monitors for embedded kit, so it's pretty much a real world example. I don't find the first version using ":=" to be perfectly clear, in fact I think it's rather ugly. That may be partly the same reaction that many of us had to the asymmetry of assignment expressions in (over-)complicated comprehensions. The second version using "given" reads much more naturally to the mathematician in me, and not too badly to my English half either. [*] By "like this" I mean the clunky "while true:" spelling, obviously. -- Rhodri James *-* Kynesim Ltd
On Fri, May 11, 2018 at 9:37 PM, Rhodri James <rhodri@kynesim.co.uk> wrote:
Yes, I've considered it. And I don't like the fact that the evaluation is right-to-left. It isn't a problem when your condition is extremely simple, but I can guarantee you that people will use this with more complicated conditions. And when that happens, you have to be aware that the tail of the statement is actually evaluated before the primary expression. It's like with Perl: die("blah blah") unless some_condition Reverse-order evaluation is confusing and frequently annoying. It's not an instant failure of the proposal, but it's a serious cost, and I'd much rather avoid it by using := (which leaves the expression where it is). ChrisA
Actually, the first version is more readable. It's got a lot to do with what Chris said about order of operations, but IMO, even more with grouping. There's a couple of things you might want to learn from this statement. First, will the while check succeed? Well.... while......get_command()).token != CMD_QUIT: yup, looks clear. I can ignore the cmd := part easily, and the bonus paren I see there doesn't matter that much. Another thing i might be curious about, is what is the value of cmd after ? while (cmd := get_command())........................: Looks like it has the value of getcommand(). Hey, that was both clear and readable. I can just ignore half the line and learn stuff. Great. Lets see with the other notation. What's the value of cmd ? while ................................................... cmd = get_command(): That works. Bit of line I had to skip. Will the check succeed ? while cmd.token != CMD_QUIT .....................................: Wait, what's the value of cmd ? Lets look in the code in the preceding lines....oh, ok, it's at the end of the line. I actually have to mentally parse the entire line to get what the check will work. This, along with what Chris said about order of operations, reduce the readability of the "given" version.
On Fri, May 11, 2018 at 11:34 PM, Rhodri James <rhodri@kynesim.co.uk> wrote:
No, not a win. Do you read the entire source code for an entire project before trying to comprehend one part of it? I doubt it. Do you read an entire file before trying to comprehend a single function in that file? No. Do you even read an entire function before processing one line in that function? Unlikely. It's normal and correct to seek to understand one part of some code while ignoring other parts. That's why we have proper variable names, even inside functions - we could just use "slot0" and "slot1" and so on, since that's how they work to the interpreter. But we use good names, so that you can understand some code without having to first read the thing that created that variable. ChrisA
A while ago, we had this gem: 2018-04-06 8:19 GMT+02:00 Serhiy Storchaka <storchaka@gmail.com>:
Using currently supported syntax:
smooth_signal = [average for average in [0] for x in signal for average in [(1-decay)*average + decay*x]]
Go ahead and understand that line in 1 go. It's currently legal syntax for a running average for a smoothing signal, which remembers something about it. (Subject: Proposal: A Reduce-Map Comprehension and a "last" builtin) You're not allowed to work it out bit by bit, just understand the entire line or nothing. Any failure of yours proves my point.
[João] How do you read something like " while (cmd := get_command()).token != CMD_QUIT:" in plain english?
while open-paren cee em dee colon is call get-underscore-command close-paren dot token doesn't equal all-caps cee em dee underscore quit colon. Might be some dutch in there. But far more importantly, I can hold the concept into my head, or just the parts of it that I need. How we call it in english is actually not a good argument - whether we can easily mentally parse it is, since I tend not to code by voice command, but with a keyboard. Your mileage may vary, but I think we should optimize for keyboard coding over voice chat coding. And when I need to refer to it, I say "this bit here" or I copy paste it.
On 11/05/18 15:04, Jacco van Dorp wrote:
Personally I thought it proved the point that you shouldn't be trying to squash things like that into a list comprehension in the first place, because average = 0 smooth_signal = [] for x in signal: average = (1 - decay) * average + decay * x smooth_signal.append(average) is quite a bit more comprehensible. -- Rhodri James *-* Kynesim Ltd
On Sat, May 12, 2018 at 02:37:20AM +0100, Rob Cliffe via Python-ideas wrote:
Do you mean Serhiy's example of currently supported syntax? smooth_signal = [average for average in [0] for x in signal for average in [(1-decay)*average + decay*x]] It helps if you know the algorithm for exponential smoothing: for each value x (aside from the first), the average is equal to a mix of the current value x and the previous average A, split by some proportion P: A = (1-P)*A + P*x If P is 0.5, that is equivalent to taking the ordinary average between the current value and the previous average: A = (A+x)/2 # when P == 0.5 In the comprehension, P is called "decay" and A is called "average": average = (1-decay)*average + decay*x Writing the comprehension as a single line is hard to read. Let's give it some structure: smooth_signal = [average # append average to the results for average in [0] for x in signal for average in [(1-decay)*average + decay*x] ] Horrible as it is, it is perfectly legal Python right now. It uses for name in SINGLE_ITEM_LIST to perform an assignment. So that's equivalent to: average = 0 for x in signal average = (1-decay)*average + decay*x append average to the results Pull the initial value of average out of the comprehension, and use the PEP 572 syntax: average = 0 smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal] which is a huge improvement in my opinion. It would be more obvious if the expression being calculated came first: smooth_signal = [(1-decay)*average + decay*x as average for x in signal] but there are good reasons why the "as" syntax won't work. So it looks like we're stuck with needing to look ahead past the := to see the actual value being appended to the list. A minor inconvenience, equivalent to that in ternary if, where we have to look ahead to see the condition: [target := COMPREHENSION_VALUE for x in sequence] true_value if CONDITION else false_value So I expect that it will take me a little while to learn to look ahead and read binding-expressions fluently. (Like comprehensions themselves, really. It took me a few months to stop needing to pull them apart to understand them.) He's Nick's version, as best as I am able to tell: average = 0 smooth_signal = [(average given average = (1-decay)*average + decay*x) for x in signal] So we have the same look-ahead needed to see the expression we care about, but instead of merely having two characters := needed to do the binding, we need "given average =". -- Steve
I would write this using a for loop and the two-argument form of iter: for cmd in iter(get_command, ''): if cmd.token == CMD_QUIT: break cmd.do_something() or from itertools import take while for cmd in takewhile(lambda x: x.token != CMD_QUIT, iter(get_command, '')): cmd.do_something() Depending on what get_command actually returns, you might be able to construct a valid sentinel that doesn't require an explicit test of cmd.token. (This reminds that I wish ``iter`` could take a predicate instead of a sentinel as its second argument. Then you could just write for cmd in iter(get_command, lambda x: x.token == CMD_QUIT): cmd.do_something() ) -- Clint
2018-05-11 17:06 GMT+03:00 Clint Hepner <clint.hepner@gmail.com>:
But you can do it right now: class P: def __init__(self, key): self.key = key def __eq__(self, other): return self.key(other) for cmd in iter(get_command, P(lambda x: x.token == CMD_QUIT)): cmd.do_something() With kind regards, -gdg
On Fri, May 11, 2018 at 12:37:43PM +0100, Rhodri James wrote:
Okay, considered. I think the first is preferable. Much earlier in the PEP 572 discussion, I strongly argued in favour of the expr as name syntax on the basis that the most important part of the overall expression is "expr", not the assignment target, and therefore that should come first. Even though I have accepted that "as" is not viable, I still believe that it is preferable to have the expression first, or if not first, at least as close to the left as we can get it. This "given" syntax puts the expr part all the way to the far right of the line. A line which is made all the longer for needing to use "given" and redundantly state the target name. It's like we're trying to maximize the distance the eye has to travel back and forth when reading. I have to read to the end of the line before I have any idea where cmd has come from or what it is. The fact that it comes from a "given" expression comes as a surprise at the end of the line. Now obviously this doesn't matter if I'm reading lines of code in careful detail, but I don't do that all the time. I skim code far more than I read it in careful detail, and the closer things are to the left, the more likely I am to see them while skimming. The further out they are, the easier they are to miss. I think that "given" will *literally* make reading harder, in that the eye has to travel further to spot the relevant expression while skimming over code. As I said, I don't think it makes any difference when reading closely in detail. But most of my reading of code is skimming to find the relevant line or section, and then read closely. I would probably skim a hundred lines for every one I read closely. We read more code than we write, but writing is important too. I think the verbosity of "given" (six chars versus two) and the redundancy of needing to repeat the name of the target even if you only use it once will soon make using this syntax seem like a chore. -- Steve
On 2018-05-11 09:17, Steven D'Aprano wrote:
That is an interesting argument --- interesting to me because I agree with a lot of it, but it leads me to the opposite conclusion, and also because it highlights some of the relevant factors for me. The main part I disagree with is that the most important thing is the definition of expr. Rather, what I think is most important is the role of expr within the surrounding expression. For simple cases, it doesn't much matter which comes first: if x := do_some_stuff(): if x where x = do_some_stuff(): . . . and it's true the latter is a bit more verbose in that case for little extra benefit. But when the locally-defined value is used within a more complicated expression (like the quadratic formula example), I think readability goes down significantly. To appease Tim, instead of using the quadratic formula, though, I will use a more realistic example that comes up fairly often for me: wanting to do some kind of normalization on a piece of data for a comparison, while keeping the unnormalized data for use within the block: if some_condition and (stuff:= get_user_input()).lower().strip().replace('-', ''): versus if some_condition and stuff.lower().strip().replace('-', '') given stuff = get_user_input(): Now, the latter is still more verbose. But to me it is now more readable, because the assignment does not disrupt the flow of reading the surrounding expression. This benefit increases the more complicated the surrounding expression is. Your point about reading ease is well taken, but also relevant to me is that we only read a piece of code *for the first time* once. The advantage of the given-style assignment is that on multiple readings, it foregrounds how the assigned value is USED, not how it is DEFINED. This encourages a "top-down" understanding of the expression in which you first understand the overall picture, and then later "drill down" into definition of what the components are. I wonder if some of the disagreement about the relative merits of the two cases comes from people focusing on different kinds of examples. As I said, I think the advantages of "cleft assignment" (i.e., where the assignment is shunted to the end of the line) become more pronounced as the surrounding expression becomes more complex. They also become somewhat greater as the definition of the expression becomes more complex, because there is more to skip over when finding out how the value is used. But many of the examples we're seeing are very simple ones, and in particular have a trivial surrounding expression. (That is, in something like "m := re.match()" the assigned value is not being used in any larger expression; the assigned value constitutes the whole of the expression.) I'm also finding it useful to think of parallel situations in prose writing, particularly journalistic-style prose writing. The current Python behavior, in which assignments must be done before the block, I think of as akin to something like this: "The IAAEA (International Association of Assignment Expression Advocates) is an organization dedicated to the promotion of assignment expressions. Its president is Eddie McEquals. In a statement yesterday, McEquals called for a new syntax to bring assignment expressions to the masses." The inline-assignment syntax is akin to this: In a statement yesterday, Eddie McEquals, president of the International Association of Assignment Expression Advocates (IAAEA), an organization dedicated to the promotion of assignment expressions, called for a new syntax to bring assignment expressions to the masses. The cleft-assignment syntax is akin to this: In a statement yesterday, Eddie McEquals called for a new syntax to bring assignment expressions to the masses. McEquals is the president of the International Association of Assignment Expression Advocates (IAAEA), which is an organization dedicated to the promotion of assignment expressions. Now of course I'm fudging a bit on the details (like whether you have IAAEA in parentheses or its full expansion), but the point is that the last version foregrounds the "bottom line" or "predicate" --- what actually happened. The first one foregrounds the "participants", or who/what was involved, and saves what actually happened for the end. But the middle one, to my eye, foregrounds nothing. It stuffs everything into one big clause where the descriptions of the participants occur as asides that disrupt the flow of reading. By the time we get to the predicate, we have to jump back to the beginning to remember who it is we're talking about, Again, the difference in readability varies depending on the complexity of the different components. If we just have "Eddie McEquals, president of the International Association of Assignment Expression advocates, delivered a speech yesterday at the organization's convention", the inline appositive is not so disruptive. But the more complex the inline definitions become, and especially the more complex the expression in which they are embedded, the lower readability goes, for me. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On 2018-05-11 11:08, Brendan Barnwell wrote:
Ironically I weakened my argument by forgetting to finish my expression there. I intended that chain of method calls to be used in a comparison to make the surrounding expression more complex. So revise the above to if some_condition and (stuff := get_user_input()).lower().strip().replace('-', '') == existing_value: versus if some_condition and stuff.lower().strip().replace('-', '') == existing_value given stuff = get_user_input(): -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
[Brendan Barnwell]
[also Brendan]
Even more ironically, to my eyes the original more strongly supported your view than the rewrite ;-) "given stuff =" stuck out in the original because it was preceded by punctuation (a right parenthesis). I had to read the rewrite 3 times before i realized you were even _using_ "given", because there it's buried between two other names - "existing_value given stuff" - and visually looks more like it's actually the 3rd of 4 words (because of the underscore in "existing_value"). Of course that would have been obvious in a Python-aware editor that colored "given" differently, but as-is I found the original easy to read but the rewrite a puzzle to decode. Similarly, in the rewritten assignment expression spelling, it's obvious at a glance that the test is of the form some_condition and some_messy_expression == existing_value but in the rewritten "given" sample that's obscured because "existing_value" not only doesn't end the statement, it's not even followed by punctuation. Of course coloring "given" differently would remove that visual uncertainty too. For a dumb display, I'd write it if some_condition and ( stuff.lower().strip().replace('-', '') == existing_value) given stuff = get_user_input(): instead (added parens so that "existing_value" and "given" are separated by punctuation).
On May 11, 2018 1:45:27 PM Tim Peters <tim.peters@gmail.com> wrote:
There are some variants of tanks like 'if let' where the bindings come *first*, unlike 'given' where they come last (like Haskell's 'where').
Well, you've partly explained the reason: our eyes are drawn to what sticks out. In this case, the := stuck out in a section heavy on black letters. In a proper editor, it may even be the other way around: they tend to highlight keywords in a stronger manner (like bolding) than operators.
Apology for top post, but this is a general statement about Readability and not a response to an individual. it would be nice to list the objective parts separate from the "argument" (i.e. debate, not fight), perhaps list them then make a case for which metric is a more important, and which values produce better results. Key strokes, keyboard location, location of information vs where it is used, cognitive load (objective barring neuroscience changes) are all objective points (along with many other points raised). "Better" will be decided by Guido I guess, but listing objective points with explanatory examples gives a basis for that discussion. Legibility, for example, is not objective at all, it has nothing to do with syntax. This covers fonts, colors, monitors, lighting, chairs, lunch, etc. None of this is relevent to the symbols or their ordering in a file we all must read. Teachability likewise. My opinion here is learnability is far more important anyways, I am 90% self taught going back 25 years, but this is equally unquantifiable. Perhaps just trust students to learn as an author must trust a reader. Of course, let it not be lost that determining teachability and learnability for something which doesn't even exist yet is quite challenging. Any quantification will give more information than only naked impassioned pleas to Readability. Note Tim came up with a real metric: 2 * count(":=")/len(statement). It's objective. it's just unclear if a higher score is better or worse. However, one could say "a Tim of .3 is considered too high" as a guideline. Perhaps coders find these opportunities to express feelings and opinion cathartic after speaking to the most pedantic creature on Earth (compilier/computer) but I think exercising the high skill level available here to dissect and find objective statements is a worthy puzzle. On Fri, May 11, 2018, 11:14 Brendan Barnwell <brenbarn@brenbarn.net> wrote:
On Fri, May 11, 2018 at 11:52:02AM -0700, Matt Arcidy wrote:
I think Tim was making a joke about demanding objective measurements of subjective things. Certainly he hasn't done any research or study to justify that metric. He just plucked the formula out of thin air. Or at least no peer reviewed research. -- Steve
[Matt Arcidy]
[Steven D'Aprano]
It was the outcome of an intense 17-year research project.
Or at least no peer reviewed research.
Au contraire! My peers are here, and that message was reviewed by at least 3 people on this list. That said, I am a fan of objectively measuring subjective things, just not of taking the measurements seriously ;-) If people do want to take it seriously, check out prior Python art first: http://radon.readthedocs.io/en/latest/intro.html
On Fri, May 11, 2018, 17:04 Tim Peters <tim.peters@gmail.com> wrote:
apparently my joke was objectively not funny :-) I thought calling it a "Tim" was sufficient. Im not serious about actually ranking for the purposes of a PEP. I brought it up when I felt the subjectivity was making the debate worse. Reiterating my point, n long sub-threads about fonts, screens, etc are ridiculous when those exist outside the pyfile. I don't know why personal preference for a font would stop a useful tool. Hopefully those arguments are ignored. Likewise for googlability, teachability, cross-language similarity and familiarity. If the tool is useful, that's all that will matter with respect to these points, they solve themselves. I happen to be working on a ranking tool for code (not quality, just an ordering to find entry points for new devs), so i tossed the idea in. it seemed appropriate to remind people that the fact that not everyone uses green to highlight "+" doesn't make "+" somehow more or less _useful_ (people -1'd just for legibility alone because of their personal feelings) I'm not sure where people stand on usefulness, but it's clear this tool is a pattern. No counter example of "but I can already do this" is related to other counter examples in the way that named expressions solves all of them, and does it succinctly regardless of chosen syntax. Some required imports! Obviously making these decisions with the future unknown is nearly impossible and requires careful consideration of all points, but I don't think Hypothetical Bill's perscription glasses should determine syntax decisions. Best of luck with the hard parts, clearly I hope the PEP makes it.
If people do want to take it seriously, check out prior Python art first:
Awesome, thanks!
On Fri, May 11, 2018 at 03:47:05PM +0200, João Santos wrote:
How do you read something like " while (cmd := get_command()).token != CMD_QUIT:" in plain english?
I wouldn't if I could avoid it. I hardly ever program by talking about code in plain English. Often the lines are gobblydegook: zreplace = '%c%02d%02d' % (sign, h, m) # datetime.py and even when they are technically pronouncable English: # subprocess.py (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) my brain would glaze over by the second "p2c". I prefer to read and write code than speak it, and if I need to discuss it, I prefer to use a whiteboard so I can write things down. But if I really needed to, I'd probably start by reading it as: while open bracket command defined as get-command close bracket dot token is not equal to command-quit and then I'd probably drop the "defined" and swap the order around. Actually, that's not true. I probably wouldn't say that, not in a real conversation. What I'd probably say is, So, like, I've got this command object, see, which I get from calling get-command, right, and so I get the, um, token attribute, okay, and if that's not equal to the quit value, I loop until it is. Right? (And this is why I prefer *writing* code than *saying* code.) -- Steve
On Fri, May 11, 2018 at 11:40:51AM +0200, Jacco van Dorp wrote:
I agree with Jacco here. We have to name the variable twice, even if it is only used once, and we have a relatively long keyword, five characters, longer than average for all keywords, and only one char short of the maximum. I know the aim isn't to absolutely minimise the number of keystrokes, but it does seem strange to use such a long symbol which requires duplicating the target, unless the intent is to discourage people from using it. -- Steve
On Fri, May 11, 2018 at 11:43 AM Steven D'Aprano <steve@pearwood.info> wrote:
To be fair when counting the keystrokes, you should take into account that the colon and the parentheses that appear in the := syntax are the upper register keys that counting the shift require two keystrokes each.
On 11May2018 11:40, Jacco van Dorp <j.van.dorp@deonet.nl> wrote:
Yeah, this is my favorite also. Has the same feel and ordering as with, import and except. Could someone point me to a post which nicely describes the rationale behind its rejection? I'm sure there's one in the many in this discussion but I've not found it yet. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sun, May 13, 2018 at 2:05 PM, Cameron Simpson <cs@cskk.id.au> wrote:
https://www.python.org/dev/peps/pep-0572/#special-casing-conditional-stateme... https://www.python.org/dev/peps/pep-0572/#alternative-spellings I'm not sure which version you're looking at, so there's the rejections of both. ChrisA
On 13May2018 14:23, Chris Angelico <rosuav@gmail.com> wrote:
I meant the latter, but I'd already looked at that part of the PEP and found its explaination... unfulfilling. It says: EXPR as NAME: stuff = [[f(x) as y, x/y] for x in range(5)] Since EXPR as NAME already has meaning in except and with statements (with different semantics), this would create unnecessary confusion or require special-casing (eg to forbid assignment within the headers of these statements). All you need to disambiguate, say: with expr as expr_as as with_as: is to require parentheses for (expr as exp_as) if someone wanted that complications (assuming that is even necessary - it seems unambiguous to my eye already, unless the token lookahead requirement in Python's grammar prevents that. So I'd hoped for a post to the list discussing this aspect and outlining why it was considered unsatisfactory. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sun, May 13, 2018 at 2:58 PM, Cameron Simpson <cs@cskk.id.au> wrote:
There were a large number of posts, so I can't really point to one of them. The problem isn't the double-as case that you describe; it's that these two are ALMOST identical: with expr as target: with (expr as target): In fact, they are functionally identical for many situations - "with (open(...) as target):" is exactly the same as the form without the parens. It'd make for data-dependent bugs, which are a really REALLY bad idea. ChrisA
On 5/13/18 1:05 AM, Chris Angelico wrote:
A little more detail: The first one is partially: target = expr.__enter__() The second one is partially: target = expr For many objects, obj.__enter__() just returns obj, it often looks like these two statements do the same thing, but they do not. Chris's concern, which I share, is that users wouldn't know why these are different, and why the second on works for some objects but not others. I agree the PEP could use more detail in explaining this particular issue. Eric
On 13May2018 06:45, Eric V. Smith <eric@trueblade.com> wrote:
Ah, thank you! Technically I knew this; in practice I'd forgotten and was thus vulnerable in exactly the same way as you anticipate.
Chris's concern, which I share, is that users wouldn't know why these are different, and why the second on works for some objects but not others.
While I can see the issue, on a personal basis I'd accept it: import and except don't do this and they happily use "as". In my mind the problem lies with "with" (and is a perfectly acceptable inconsistency there, given what with does for us). I agree the change in semantics is not obvious. But we've got a similar thing with tuples as well: x = (1) x = (1,) with a different spelling.
I agree the PEP could use more detail in explaining this particular issue.
Yes. This issue certainly isn't clear to me from the PEP's wording. I'm no longer trying to push the "as" agenda here, BTW. I still like it, but that ship seems sailed. Cheers, Cameron Simpson <cs@cskk.id.au>
On Mon, May 14, 2018 at 7:00 AM, Cameron Simpson <cs@cskk.id.au> wrote:
The same problem happens with 'except', only less subtly. except Exception as e: # binds the caught exception to e except (Exception as e): # would bind the type Exception import doesn't put an expression on the left of 'as', so it's less likely to cause confusion; but all three of them do something special before binding to the target given with 'as'. By the way: do you know which of the three support arbitrary assignment targets and which support only names? No? Neither did I, till I checked the grammar. So there's no consistency there, other than a loose sense that "as" means "we're gonna toss something into somewhere". Not nearly enough to justify using that syntax for arbitrary name bindings, given how much hassle there is with 'with'. ChrisA
On Thu, May 10, 2018 at 9:44 AM, Guido van Rossum <guido@python.org> wrote:
I'm sorry, but unless there's a sudden landslide of support for 'given' in favor of ':=', I'm really not going to consider it.
How much support was there for ":="? Are you serious about bringing back Pascal and Algol from their comfortable resting places?
Probably going to completely lose this, but would it be possible to have a vote? +1 for either 'given' and/or ':='? On Thu, May 10, 2018 at 2:48 PM Guido van Rossum <guido@python.org> wrote:
-- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else https://refi64.com/
Another benefit of given compared with := that I just thought of is this. Suppose you have a generator like (expression(f(x), y, z) for x in xs for y in ys(x) for z in zs(y)) With given notation you can optimize: (expression(f_x, y, z) for x in xs given f_x = f(x) for y in ys(x) for z in zs(y)) whereas with :=, you can't. Best, Neil On Sat, May 12, 2018 at 10:56 PM David Mertz <mertz@gnosis.cx> wrote:
On Sat, May 12, 2018 at 11:04:45PM -0400, Neil Girdhar wrote:
Is that legal syntax? You're splitting the "given" expression by sticking the for clause in the middle of it. I don't think that will be legal. It would be trying to split an ternary if: (true_expr for x in xs if condition else false_expr for y in ys) (true_expr if condition for x in xs else false_expr for y in ys) But whether legal or not, Neil, you went on at length about how professionals don't write code like this and such overly dense comprehensions are only fit for competitions. Now you want your cake and to eat it too: "given is better, because it doesn't allow the awful unreadable code that := gives; oh, and it's also better, because it allows *this* awful unreadable code that := doesn't allow" In any case, of course you can write this with := binding expression. You just shouldn't do it: (expression(f_x, y, z) for x in xs for y in ys(x) for z in zs(y) if (f_x := f(x)) or True) ) That's fine for mucking about, but I wouldn't do it for serious code. Replacing the colon with "given f_x" doesn't change that. -- Steve
On Sat, May 12, 2018 at 11:48 PM Steven D'Aprano <steve@pearwood.info> wrote:
You're right that it's a different proposal, but I thought it would be a natural extension to this proposal since at the start of this discussion someone mentioned how this could be accomplished with something like (expression for x in xs for f_x in [f(x)]) The regular given proposal is an extension of (I think) "expr" into something like expr ["given" test annassign] I think a natural extension of that is to add it to "testlist_comp" and "dictorsetmaker" so that given name = value can be used a cleaner synonym of something like for name in [value] But your'e right, it's a different proposal and I'm getting ahead of things. But whether legal or not, Neil, you went on at length about how
My thought is that if each component is simple enough *and* if each component can be reasoned about independently of the others, then it's fine. The issue I had with the := code you presented was that it was impossible to reason about the components independently from the whole. Ultimately, this is one of feelings. I think every one of us has a unique set of experiences, and those experiences led us to having different programming aesthetics. I used to write code one way, and then after lots of code reviews, my experience has made me write code a different way. Best, Neil
On Sun, May 13, 2018 at 12:10:08AM -0400, Neil Girdhar wrote:
Actually, as you admitted, you can't, since this isn't part of the proposed syntax or semantics. But let's put that aside.
Except of course you can:
[Neil]
I said it was rubbish code which I wouldn't use for serious work. But "impossible" to reason about? That's pretty strong, and definite, words for something which is actually pretty easy to reason about. (expression(f_x, y, z) for x in xs for y in ys(x) for z in zs(y) if (f_x := f(x)) or True) ) I'm pretty sure you can reason about "expression(f_x, y, z)" on its own. After all, the above code was (apart from the := line) your own example. If you can't even reason about your own examples, you're in a bad place. I think you can reason about the three for-loops "for x in xs" etc on their own. Again, they were your idea in the first place. I expect you can reason about "if <clause> or True" on its own. It's a pretty basic Python technique, to call <clause> for its side-effect without caring about its return value. Not something I would use for serious work, but this is YOUR example, not mine, so if you hate this generator expression, you were the one who suggested it. Only the <clause> is new or different from ordinary Python code that works now: f_x := f(x) The semantics of := are pretty damn simple (at least from the high-level overview without worrying about precedence or possible scoping issues): - evaluate f(x) - assign that value to f_x - return the value of f_x which is EXACTLY THE SAME SEMANTICS OF "f_x given f_x = f(x)": - evaluate f(x) - assign that value to f_x - return the value of f_x And you would reason about them the same way. -- Steve
On Sun, May 13, 2018 at 10:56 AM Steven D'Aprano <steve@pearwood.info> wrote:
I was talking about this snippet you wrote: smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal] Not impossible to reason about, but not pretty either.
Not that it matters, but this doesn't work in general since this relies on f_x being evaluable to bool. You can always just write "for f_x in [f(x)]". This use of given was more of an aside.
target := expr expr as target expr -> target target given target = expr let target = expr : target expr ; Although in general "target:=exp" seems the most palatable of these to me, there is one nice benefit to the "given" syntax: Suppose you have a comprehension wherein you want to pass forward an internal "state" between iterations, but not return it as the output: In today's python, you'd to: outputs = [] state = initial_state for inp in inputs: out, state = my_update_func(state) outputs.append(state) This could not be neatly compacted into: state = initial_state outputs = [out given out, state = my_update_func(inp, state) for inp in inputs] Or maybe: outputs = [out given out, state = my_update_func(inp, state) for inp in inputs given state=initial_state] Though I agree for the much more common case of assigning a value inline "x given x=y" seems messily redundant. On Sat, May 12, 2018 at 10:37 PM, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
On 13.05.2018 11:23, Peter O'Connor wrote:
Question still stands if this type of code needs compaction in the first place? List comprehensions usually have some sort of declarative touch (set builder notation). Even though, striving for a more compacted version, I tend to think that using a declarative version of it doesn't serve it well in the long term. We recently came across the following code snippet in our source base (1st answer of https://stackoverflow.com/questions/480214/how-do-you-remove-duplicates-from...). It was absolutely not comprehensible. Your example is inherently imperative because the internal state changes from iteration to iteration; something unusual for set builder notation. Regards, Sven
2018-05-10 16:44 GMT+03:00 Guido van Rossum <guido@python.org>:
I think you do not quite objectively look at the current situation. Many just lost interest in attempts to move the topic at least a little bit in the other way, seeing how you and Tim so actively expresses support/protects this `:=` syntax, while ignoring or pushing out alternative opinions :-). Of course, the latter is partly due to the incredible number of different threads and messages on this topic. Briefly: Initially, the main argument in favor of `:=` was that this form is similar to the usual assignment statement, but can be used as an expression. Ok. Then everyone agreed with the idea that it's necessary to limit assignment target to name only. Although all this criticism was actually put forward in the first 50-100 messages on the topic. In the same first hundred it was actively discussed, that in fact, this idea gives a win only in `while` and `if` statemetns that probably will match 99%+ where it will be used for its intended purpose. At the same time, most of the criticism concerned exactly the use in generators and comprehenshions, they are already often overloaded for perception. And as you once said - "Language Design Is Not Just Solving Puzzles". There was also discussed the difference in perception between the `expr op name` and `name op expr`. Here the expression is something that is important, the name is only a convenient consequence. At the moment with all the constraints of `:=`, the discuscation is more like - trying to cram this syntax into the language. While for those who are familiar with Pascal, Icon and other languages that use this syntax, this - `:=` looks natural. For others and I believe such a majority among users, this syntax is, to put it mildly, not natural and ugly producing a line noise, the colon `:` symbol is already used in a lot of places. With all the changes, the limitations and magic with scopes. Is it now easier to explain all the differences between `=` and `:=`, than the difference between `if expr as name ...: ...` and `with expr as name:`? Therefore, I take Nick's insistent position as an attempt to at least somehow make an alternative look at this topic. With kind regards, -gdg
Kirill Balunov wrote:
As someone familiar with Pascal, I think the similarity to the Pascal assignment operator is actually an argument *against* it. Knowing what it means in Pascal is confusing, because Pascal's ":=" is equivalent to Python's "=" (it's strictly a statement, and can't be used in expressions). -- Greg
On 11.05.2018 09:38, Greg Ewing wrote:
Same here. It means something different. Also coding in Pascal was annoying from the beginning with its extremely verbose syntax like begin/end etc. So, ":=" also felt like "why the hell do we need a colon in front of the equal sign?" Absolutely unnecessary bloat, like almost everything in Pascal. Maybe that's also part, why I am -1 on the proposal. Who knows...
I love given compared with := mainly because Simpler is better than complex: * given breaks a complex statement into two simpler ones, which is putting people off in the simple examples shown here (some people are annoyed by the extra characters). However, when given is used in a list comprehension to prevent having to re-express it as a for loop, then two simple statements are easier to understand than one complex statement. This is a common difference between code written at programming contests versus code written by those same software engineers years later at big companies. Code that you write for yourself can be compact because you already understand it, but code you write professionally is read many many more times than it is written. Accessibility is much more important than concision. * Python has a reputation for being working pseudocode, and given reads like pseudocode. := needs to be deciphered by comparison especially in the complicated cases where multiple := operators are used on one line. * there are no difficult mental questions about evaluation order, e.g., in a bracketed expression having multiple assignments. Similarly, instead of (a.b(a) given a = c.d()) do I write (a.b(a := c.d())) or ((a := c.d()).b(a)) ? * it avoids the question of what happens when := is used in a switch: (a if (b := c) else d) Sometimes you want the assignment to happen unconditionally (a if (b:=c) else d) + b; sometimes you don't. How do you force one case or the other? given makes it obvious by separating assignment from the usage of its assignment target. Style: * it avoids the big style question of when to use and when not to use :=. (Even if you ask people not to, people are going to write the expression-statement a := b as a synonym for the statement a = b.) * it looks a lot like the existing Python "for" and "if" clauses, which also do in-expression assignments. This makes formatting the code obvious too: (expression given a = b) compared with expresion ( a := b ) rest of expression which quickly gets ugly. Best, Neil On Thursday, May 10, 2018 at 9:46:01 AM UTC-4, Guido van Rossum wrote:
On Sat, May 12, 2018 at 08:16:07AM -0700, Neil Girdhar wrote:
(Foreshadowing: this argument applies to augmented assignment. See below.) I don't see how you justify that statement about "given". I think that it is "given" which is more complex. Significantly so. Let's compare the syntax: target := expr That is a single, simple expression with a single side-effect: it assigns the value to <target>. That's it. Like all expressions, it returns a value, namely the result of "expr". Like all expressions, you can embed it in other expressions (possibly wrapping it in parens to avoid precedence issues), or not, as required. (That surrounding expression can be as simple or complex as you like.) Now here's Nick's syntax: target given target = expr Exactly like the := version above, we can say that this is a single expression with a single side-effect. Like all expressions, it returns a value, namely the result of "expr", and like all expressions, you can embed it in other expressions. So far the two are precisely the same. There is no difference in the complexity, because they are exactly the same except for the redundant and verbose "given" spelling. But actually, I lied. Nick's syntax is *much more complicated* than the := syntax. Any arbitrary expression can appear on the left of "given". It need not even involve the binding target! So to make a fair comparison, I ought to compare: target := expr which evaluates a single expression, binds it, and returns it, to: another_expr given target := expr which evaluates "expr", binds it to "target", evaluates a SECOND unrelated expression, and returns that. If you want to argue that this is more useful, then fine, say so. But to say that it is *simpler* makes no sense to me. Option 1: evaluate and bind a single expression Option 2: exactly the same as Option 1, and then evaluate a second expression How do you justify that Option 2 "given", which does everything := does PLUS MORE, is simpler than Option 1? That's not a rhetorical question.
I'd like to see an example of one of these list comprehensions that is simpler written with given. Here's an earlier example, an exponentially weighted running average: average = 0 smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal] assert average == smooth_signal[-1] I'm not even sure if "given" will support this. Nick is arguing strongly that bound targets should be local to the comprehension, and so I think you can't even write this example at all with Nick's scoping rule. But let's assume that the scoping rule is the same as the above. In that case, I make it: average = 0 smooth_signal = [average given average = (1-decay)*average + decay*x) for x in signal] Is it longer, requiring more typing? Absolutely. Does it contain a redundantly repetitious duplication of the repeated target name? Certainly. But is it simpler? I don't think so. If you don't like the exponential running average, here's a simple running total: total = 0 running_totals = [total := total + x for x in xs] versus total = 0 running_totals = [total given total = total + x for x in xs] If you don't like this example either, please show me an example of an actual list comp that given makes simpler.
Ah, nice rhetorical argument: "given" is for professionals, := is a hack for amateurs and programming contests. Seriously? Do you use augmented assignment? Your simple versus complex argument for "given" applies well to augmented assignment. Augmented assignment combines two conceptual operations: x = x + 1 - addition (for example) - assignment into a single operator: x += 1 By your argument, augmented assignment is more complex, and we ought to prefer splitting it into two separate operations x = x + 1 because that's simpler. I think I agree that x = x + 1 *is* simpler. We can understand it by understanding the two parts separately: x+1, followed by assignment. Whereas += requires us to understand that the syntax not only calls a dunder method __iadd__ (or __add__ if that doesn't exist), which potentially can operate in place, but it also does an assignment, all in one conceptual operation. There's a whole lot of extra complexity there. That doesn't mean I agree with your conclusion that we ought to prefer the simpler version, let alone that the complex case (augmented assignment) is fit only for programming contests and that professionals ought to choose the simpler one. -- Steve
Just clarifying a fine point here: [Steven D'Aprano <steve@pearwood.info>]
You can't under Nick's proposal(s), at least not directly (there are always "tricks"). But it also blows up with UnboundLocalError (for the "average" in "(1-decay)*average") under the current PEP 572 (the ":=" PEP). I've proposed to change 572's scoping rules for targets of assignment expressions appearing in comprehensions so that "it would just work" instead, but that's getting strong opposition too. My favorite so far was Nick's (Coghlan's) entertainingly hyperbolic: "Comprehension scopes are already confusing, so it's OK to dial their weirdness all the way up to 11" is an *incredibly* strange argument to be attempting :-) The scope issues are logically independent of assignment-expression spelling, but it's a pretty safe guess Nick is opposed to that example ever "just working" regardless of spelling, while PEP 572 doesn't currently support it anyway. Last I heard, Chris (Angelico - the PEP's author) didn't seem keen on changing it either.
On Sat, May 12, 2018 at 01:13:05PM -0500, Tim Peters wrote:
Yes, but I've sort of assumed that if PEP 572 has even a microscopic chance of being accepted, it will have to be changed, given that Guido has already stated that the only behaviour that makes sense is what you and I have been suggesting. Unless Guido has changed his mind, the relevant links are: https://mail.python.org/pipermail/python-ideas/2018-May/050411.html https://mail.python.org/pipermail/python-ideas/2018-May/050456.html So maybe this is a tiny bit naughty (or a lot...) but I've just been ignoring what the PEP currently says and going by what it ought to say :-) -- Steve
On 12 May 2018 at 14:13, Tim Peters <tim.peters@gmail.com> wrote:
I'm personally fine with that example working if there's an explicit nonlocal declaration on "average" in the nested scope - it's Guido that objected to requiring the explicit scoping declaration to access that behaviour. For the implicit version, my request is that any PEP proposing the idea of parent local scoping be held to the standard of *actually drafting the patch for the language specification*, rather than handwaving away the hard problems that it creates (i.e. what to do at class scope, what to do when multiple generators expressions reference the same nonlocal name, what to do with nested comprehensions, how to expand comprehensions using this kind of scoping to their statement form in a context independent way). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 11 May 2018 at 12:45, Tim Peters <tim.peters@gmail.com> wrote:
I was one of those core devs, and would personally prefer to require that folks spell the inline binding completely unambiguously as "if i given i = 1:". However, if the repetition of "i" is considered a deal breaker relative to ":=" (even though the status quo already requires repetition of the target name in the condition), then I'd prefer to add this shorthand (which folks can then opt to prohibit in favour of the more explicit form in their style guides) over adding the cognitive complexity of deciding when to use "i = 1" and when to use "i := 1". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Just noting some real code I typed today where `given` works great if it allows unpacking syntax, and assignment expressions don't: while True: head, matched, s = s.partition(sep) if not matched: break Using `given`: while matched given head, matched, s = s.partition(sep): Typing "matched " twice still sucks, though;-) It doesn't work as well with assignment expressions even if they were (re)generalized to allow unpacking syntax. In this specific case, I'd keep the original loop-and-a-half rather than do: while (t := s.partition(sep})[1]: head, matched, s = t With unpacking syntax restored, same answer: while (head, matched, s := s.partition(sep})[1]: or while [(head, matched, s := s.partition(sep}), matched][-1]: to combine the worst annoyances of everything and add another ;-)
[Tim]
[MRAB <python@mrabarnett.plus.com>]
If you're using .partition multiple times, you might as well use .split instead!
Possibly - depends on what the rest of the loop is doing. In this specific case, there are paths in the loop body that change what `sep` is bound to, so I'd have to pass `maxsplit=1` to get a clumsier way to do what `partition()` does directly, For example,
I'ts more convenient that `partition()` always returns a 3-tuple, not sometimes a 1-list and other times a 2-list.
[Steven D'Aprano <steve@pearwood.info>]
[Tim]
[Nick Coghlan <ncoghlan@gmail.com>]
I'm personally fine with that example working
"Just working" meant "exactly as written". "Regardless of spelling" meant whether binding is spelled via ":=", "given", "as", "where", "let ... in ...' "->", ..
I suspect, but don't know, that Guido would like that example to "just work" because it _looks like_ it should "just work". There's no visible function involved, and _needing_ to add scope declarations to make it work is pretty much inexplicable unless the user first learns more than most users "should" need to learn about how it's implemented. If so, no technical argument will change his mind - and especially not one based on "but the implementation today doesn't do that already". Recall that Python had no lexically nested scoping at first? If he wanted to _make_ users learn about nested lexical scopes to use Python features that don't _appear_ to use it, Python would have had it from the start ;-) I'd be happy enough with needing an explicit declaration too, but am _happiest_ with what I'm guessing Guido's view is.
Of course.
rather than handwaving away the hard problems that it creates (i.e. what to do at class scope,
I finally looked at class scope and quickly decided it makes no sense there (a comprehension at class scope has no access to the class scope, so "same scope in the comprehension as in its immediately containing block" is incoherent in that case).
what to do when multiple generators expressions reference the same nonlocal name,
Then, as you said, they access the same nonlocal name. What of it? What if two generator functions reference (or even rebind) the same nonlocal name? What's unclear about that? If you were happy with explicit scope declarations, then exactly the same thing would happen as if the user were forced to explicitly declare the scopes chosen for them.
what to do with nested comprehensions,
Again, if you were happy with explicit scope declarations, then exactly the same ... Where are the docs explaining how nested comprehensions work today? I haven't seen any. If any such exist, I'd bet nothing about them needs to be changed. If none such exist, I don't see a need to write any just for this PEP. How do nested expressions of any kind work? Same thing. The only thing the suggestion changes is the scope of assignment expression targets in synthetic functions created to implement comprehensions. That has nothing at all to do with the possibility of nesting, or with the structure of nesting. Why do you think it does - or might?
how to expand comprehensions using this kind of scoping to their statement form in a context independent way).
I've already explained why I view that as a non-issue (making a tedious manual process a relative handful of users undertake once in their life for self-education purposes slightly less tedious has approximately no value to me - and, to the contrary, the "context dependent" bits they may have to learn would make it _more_ educational). If that's a show-stopper for you, so be it.
Can someone explain to me why it was considered a bad thing that for-clauses leaked names in comprehensions, but it will be a good thing for inline assignments to leak names from them? -- Greg
[Greg Ewing <greg.ewing@canterbury.ac.nz>']
Can someone explain to me why it was considered a bad thing that for-clauses leaked names in comprehensions,
Because you can't write a list comprehension or generator expression AT ALL without specifying a `for` loop header, so whether its target(s) leaks is an issue in virtually every listcomp/genexp ever written, and that people tend to use short (typically 1-letter) names for for-targets, so unintentional stomping on names is exceptionally likely in this context.
but it will be a good thing for inline assignments to leak names from them?
Because you never _need_ to use an assignment expression to write a listcomp/genexp. You have to go out of your way to use it. Which will probably be rare in listcomps;genexps,, not virtually 100% of the time as with for-targets. Then you get what you went out of your way to explicitly ask for: a name for some subexpression result. Otherwise it's essentially impossible to explain why: total = 0 sums = [total := total + value for value in data] assert sums[-1] == total "blows up", despite that its intent is obvious, unless you first explain to a user how the listcomp is implemented via an invisible synthetic function created by magic, inside of which `total` has nothing to do with the `total` they see on the first line. UnboundLocalError - WTF? That's why leaking "is good". It works both directions: the outer name leaks _into_ the body too, not just _out_ of it. Things that "look like" they should obviously work do work then, and a user can remain blissfully unaware of the implementation.. Of course you can also find cases in which it's not wanted. If it doesn't leak the kind of use shown above can't be done at all via listcomps (at least not straightforwardly). If it does leak, the subset of cases where leaking is unwanted _of_ the subset of cases in which a listcomp//genexp uses an assignment expression at all are indeed inconvenienced. So - surprise! It's a tradeoff, something we've never faced before ;-)
I think we're approaching this from the wrong direction. My point is, expression assignments dont have the complex case as purpose - most coders wont try to maximize line information density. If you're doing magic, you might as well spell it out over multiple lines, because neither := nor given will be readable. However, in the simple case, it does matter a lot - and there := beats out given by a mile. If you're breaking your lines to avoid line length violations, you might as well put your assignments on a separate lines first. Im inclined to argue that if assignment expressions of any form forces to you make a multi-line statement, you're doing it wrong. (in the vast majority of cases). Consider it like the ternary operator - good to compact simple constructs, but while you can use it for black magic, you probably shouldn't. How about we just explicitly advice to keep it simple, stupid, and update PEP 8 to state that if assignment expressions take you to multi-line, split of the assignment and use statements instead. (truth be told, the general while (assignment expression isn't as clear cut as if-uses, but I think the idea holds.)) Jacco
Tim Peters wrote:
Because you never _need_ to use an assignment expression to write a listcomp/genexp.
This whole discussion started because someone wanted a way to bind a temporary result for use *within* a comprehension. Those use cases don't require leakage.
It's no harder to explain that than it is to explain why x = 42 y = [x * x for x in range(5)] print(x) prints 42 rather than whatever value was last bound to the x in the comprehension. Seems to me it would be easier to explain that *all* names bound within a comprehension are local to the comprehension, than to have to say that some are and some aren't. -- Greg
[Greg Ewing <greg.ewing@canterbury.ac.nz>']
This whole discussion started because someone wanted a way to bind a temporary result for use *within* a comprehension.
It's been noted several times recently that the example PEP 572 gives as _not_ working: total = 0 progressive_sums = [total := total + value for value in data] was the original use case that prompted work on the PEP. You gotta admit that's ironic ;-)
Those use cases don't require leakage.
I already said that - yes - some use cases don't want the leakage. Others do. And the vast majority of listcomps/genexps will likely never use an assignment expression anyway. If "no leak" wins, "want leak" can't do what they want at all. If "want leak" wins, the subset-of-a-subset "no leak" cases are inconvenienced. As I said, it's a tradeoff. You're giving 0 weight to one of the sides.
You're overlooking the most relevant point: The example blows with an UnboundLocalError, which can't possibly be explained by looking at the example as it stands, or by reference to shallow tricks. You _need_ to drag in stuff about invisible (in the code) synthesized scopes in which `total` is in fact an unbound local name. You don't need that heavy machinery to explain why for-target names don't leak; indeed, shallow implementation tricks were used to achieve that well before synthetic functions _were_ used to implement listcomps.
for-targets are local, assignment expression targets aren't. I agree that's harder to explain, but on a scale of 1 to 10? 1. It's a tradeoff. And I cheerfully just gave 1 point to the side you favor. By my accounting, "leak" is still wining by about 136 to 17 ;-)
On 14 May 2018 at 06:10, Tim Peters <tim.peters@gmail.com> wrote:
After pondering this case further, I think it's also worth noting that that *particular* example could also be addressed by: 1. Allowing augmented assignment *expressions* 2. Changing the scoping rules for augmented assignment operations in general such that they *don't change the scope of the referenced name* Writing "i += n" without first declaring the scope of "i" with "i = 0", "nonlocal i" or "global i" is one of the most common sources of UnboundLocalError after all, so I'd be surprised to find anyone that considered the current augmented assignment scoping rules to be outside the realm of reconsideration. The accumulation example would then be written: total = 0 progressive_sums = [total += value for value in data] if progressive_sums: assert total == progressive_sums[-1] The question would then turn to "What if you just want to bind the target name, without considering the old value?". And then *that's* where "NAME : = EXPR" would come in: as an augmented assignment operator that used augmented assignment scoping semantics, rather than regular local name binding semantics. That would mean *directly* overturning PEP 3099's rejection of the idea of using "NAME := EXPR" to imply "nonlocal NAME" at function scope, but that's effectively on the table for implicit functions anyway (and I'd prefer to have ":=" be consistent everywhere, rather than having to special case the implicit scopes). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
[Tim]
[Nick]
Yes, it's worth considering. In my experience, I don't believe I've ever had an UnboundLocalError for which a correct fix was to add `nonlocal`. Usually the correct fix was to add `global`, but that's mostly due to an old habit of using piles of globals to count trips through various code paths, used by a watchdog thread that periodically wakes up to display (the global) counts.
And the divide-out-small-primes example could be factor = -42 while any(n % (factor += p - factor) == 0 for p in small_primes): n //= factor Heh ;-)
Plain old ":=" would somehow be viewed as being an augmented assignment operator too? ... OK, the meaning is that augmented assignment _and_ "::=" would resolve the target's scope in the way the containing block resolves it.
Creating a local in a containing scope by magic is never done by Python today. Extending that beyond "need" seems potentially perilous. For example, it can already be tedious to figure out which names _are_ local to a function by staring at the function's code, but people quickly get better at that over time; change the rules so that they _also_ have to stare at all immediately contained functions too to figure it out, and it may become significantly harder (OK, I didn't declare `x`, and a contained function did `x := 3.14` but `x` isn't declared there either - I guess it's my `x` now). Then again, if they're doing that much function nesting they deserve whatever they get ;-) Restrict it to that only synthetically generated functions can pull off this trick by magic (where there are real use cases to motivate it), and they still don't have to look outside the body of a function's text to figure it out. Visually, there's no distinction between the code running in the function's scope and in scopes synthesized to implement comprehensions appearing in the function's text. The comprehensions aren't even indented more. So, offhand, I'm not sure that the right way to address something you view as a wart is to vastly expand its reach to 12 operators that impose it on everyone everywhere every time they're used ;-) Seriously, I do suspect that in def f(...): ... no instances of `s` ... s += f"START {time.time():.2f}" it's overwhelmingly more likely that they simply forgot to do s = "" earlier in `f` than they actually wanted to append to whatever `s` means in f's parent block.. That's a radical change to what people have come to expect `NAME +=` to do. OTOH, I don't (yet?) see a way the change could break code that currently works, so it remains worth thinking about. BTW, would def f(): x := 3.14 x = 3.14 be a compile-time error? Everyone agreed the analogous case would be in synthetic functions. Fine by me!
On 15 May 2018 at 01:53, Tim Peters <tim.peters@gmail.com> wrote:
More likely they'd get a compile time error complaining that the compiler couldn't figure out what they meant, and asking them to be clearer about the intended scoping. Restrict it to that only synthetically generated functions can pull
Once I reframed the idea as being like an augmented assignment, your proposed semantics seemed a lot less magical to me, since I was able to define them in terms of "find the assignment or declaration that already exists", rather than implicitly creating a new one. If the compiler can't find a suitable target scope, then it can throw AmbiguousTargetError (which would be an improvement over the runtime UnboundLocalError you typically get today).
I think this is the key argument in favour of only allowing the "implicitly nonlocal rebinding" behaviour in lambda expressions, generator expressions, and comprehensions, as that way the search for a target to bind would always terminate at the containing block (just as it does today). BTW, would
Yeah, I think that would be an AmbiguousTargetError, as when the compiler saw "x := 3.14", it wouldn't have seen "x = 3.14" yet. For other augmented assignments, it would be a DeprecationWarning for the time being, and become an AmbiguousTargetError at a later date. (This also relates to the previous point: if "x := 3.14" can be implicitly nonlocal, then I couldn't answer that question without knowing which names were defined in outer scopes. By contrast, if the implicit access to outer scopes is limited to inline scopes accessing their containing scope, then this example becomes precisely analagous to the current syntax error for binding a name as a local before declaring it as global or nonlocal. The statement of ambiguity would arise from the fact that when we see "TARGET := EXPR" at statement level, we don't know if the missing prior statement is a local variable assignment, a type declaration, or a global or nonlocal declaration) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
total = 0 progressive_sums = [total := total + value for value in data]
[Greg Ewing <greg.ewing@canterbury.ac.nz>]
I'm skeptical that it's a good idea to encourage this kind of thing in the first place.
Adding a feature is creating possibility for its use, not encouraging its use. I'd hate to, ,e.g., see people encouraging use of ternary-if either, _Sometimes_ ternary-if is exactly right, both shortening code and increasing clarity. But not often. Ditto for metaclasses, the ability to nest blocks 20 levels deep, on & on. That they can be abused, and sometimes are abused, is a very weak argument against them. On a scale of 1 to 1000000? Maybe 23 ;-) The specific use case that grabbed Guido's attention on this one wasn't the above, but: while any(n % p == 0 for p in small_primes): // divide p out of n - but which p succeeded? _General_ features turn out to have multiple good uses, not all of which are anticipated. I think this is an example of that. It's easy to miss that the argument to `any()` runs in an entirely different scope, so that its local `p` vanishes when `any()` completes. It's not proposed to change that either, because people overwhelmingly seem to want `for` targets not to leak. Fine by me. Under the proposal, it's straightforward to use an assignment expression to "export" p's last materialized value: while any(n % (factor := p) == 0 for p in small_primes): n //= factor To my eyes, the most surprising thing about that is why it's necessary to "export" p's winning value to begin with. "Because for-targets always vanish, so there's another form of explicit binding that doesn't" is a reasonably satisfying explanation. Nick would be happiest, and I'd be "happy enough", if all forms of binding remained "local" and instead we fiddled the syntax to allow explicitly declaring the desired scope. Then, e.g., while any(<nonlocal p> n % p == 0 for p in small_primes): n //= p wouldn't need an assignment expression at all. But Guido doesn't like that. I think he wants to hide, so far as humanly possible, that listcomps and genexps happen to implemented now by synthesizing functions.
On Mon, May 14, 2018 at 09:17:03PM +1200, Greg Ewing wrote:
To reiterate what Tim already pointed out, that original usecase required a way to feed values *into* the comprehension. https://mail.python.org/pipermail/python-ideas/2018-February/048971.html There's no need for dedicated syntax for that if we can just set an variable and have it show up in the comprehension. [...]
But it *is* harder to explain why comprehensions are their own scope in the first place. They don't look like they are in their own scope. They look like any other expression. No other expression (apart from lambda) runs in its own scope. Every other scope relates to a clear lexical separation: - modules - classes - functions except for comprehensions, which just returns a plain old list. (Generator expressions are a little fuzzier: they at least are equivalent to a lambda-with-yield, if such a thing existed.) It is sometimes useful to be able to reach into a generator expression and manipulate a variable between calls. That would make it a coroutine (to use the pre-async language), and that's why yield is an expression that returns a value, not just a statement. And it is sometimes useful to be able to see the value of comprehension variables after they have finished running. We have given up all of that to allow ease of implementation to drive the semantics, and that's okay. Its a trade-off. But we can decide the other way too, and choose more useful semantics for assignment expressions over an easier implementation.
Of course it would be easier to explain. It wouldn't be as useful. If we wanted "easy to explain", we'd be using BASIC circa 1974, we wouldn't have async, generators, comprehensions, exceptions, decorators, classes, metaclasses, descriptors, Unicode, or floating point numbers. -- Steve
Steven D'Aprano wrote:
Yes, but doing it that way also allows things to come *out* of the comprehension, which is something we previously thought was such a bad idea that we went to considerable lengths to stop it happening. Now we seem to be seriously considering going to further lengths to allow it again for *some* things. Seems to me we should have a pretty compelling reason for doing that. I don't feel very compelled by what I've seen so far. -- Greg
On 14/05/18 10:17, Greg Ewing wrote:
I still don't find that argument compelling. If you have a sufficiently complicated comprehension that you need a temporary result bound, IMHO you are well on the way to (if not far past) the point where converting it into a more traditional for-loop will be much clearer. I'm not sure we should be encouraging people to write less clear code. -- Rhodri James *-* Kynesim Ltd
Minor gripe: The ability to *articulate* Python is not the same as the ability to *type Python verbally*. Nobody articulates `def area(width, height): return width * height` as *def area, open paren, width, comma, space, height, closed paren..*. They would say something like *def area as a function of width and height, equal to width by height*. -- Carl Smith carl.input@gmail.com On 12 May 2018 at 18:24, Steven D'Aprano <steve@pearwood.info> wrote:
On 2018-05-12 18:24, Steven D'Aprano wrote:
Your interpretation of the argument you quote, as I understand it, is that "`given` is simpler than `:=`". I interpreted it as, and would argue myself, more like "`given` makes *the code that uses it* simpler". The reason the `given` syntax might be simpler has nothing to do with the number of syntactic elements (brainfuck is down the hall...). Rather, it is that `given` separates things: if m.group(2) in words given m = word_re.match(s): versus: if (m := word_re.match(s)).group(2) in words: In the `:=` version, the assignment is embedded in the expression. It's different. Most of the time it will save at least a few characters. But it's not obviously--and certainly not objectively--simpler. One somewhat concrete difference I can think of is that in expressions that refer to the result multiple times, the `:=` assignment must always be positioned such that it is the first to be evaluated, moving around should that ever change. In cases where evaluation depends on runtime information (if-else, and, or), I'm not actually sure what you would do. In general, I feel like you're focusing on the simplicity or otherwise of the operator itself, rather than whether it has a simplifying effect on code that uses it. As you say in part 2 (which arrived while I was still taking forever to write this; sorry if it reads a bit confused), we'll learn the syntax, eventually. What's not so explicitly spelled out is that we'll be reading new code that uses it forever.
average = 0 smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal] assert average == smooth_signal[-1]
Like, holy smoke, man. Sure, `:=` is probably better for cramming side-effects into list comprehensions. Please don't cram side-effects into list comprehensions.
What is it with the cramming side-effects into list comprehensions‽ Do you realise itertools.accumulate exists? See the OP: On 2018-05-04 13:06, Nick Coghlan wrote:
I don't have a more concrete example of this to hand, mostly because I can't do this today, and faking it is hard enough (I didn't even know about the `for value in [stuff]` hack until this thread) that I just write for loops. On 2018-05-12 18:24, Steven D'Aprano wrote:
While I'm not sorry that I can do `x += 1`, it does have a substantial cost--one that is perhaps justified by the fact that it's *not* equivalent to `x = x + 1`, and that the possibilities it opens up are both useful and easy to understand. It's not simpler in and of itself, but that's not really the issue. On 2018-05-12 19:27, Steven D'Aprano wrote:
I believe that that's missing the point: to wit, in x.method(y) is `x` or `y` evaluated first? I didn't know. I don't think the order is documented anywhere or guaranteed not to change. I don't know what other languages do in general, though I know in C it's explicitly unspecified. So, sure, you can do this fine with `:=`. But it forces your code to depend on what I'd regard as a subtlety of the implementation.
On Sun, May 13, 2018 at 7:33 AM, Ed Kellett <e+python-ideas@kellett.im> wrote:
It's documented. It's guaranteed not to change. https://docs.python.org/3/reference/expressions.html#evaluation-order ChrisA
On 2018-05-12 22:42, Chris Angelico wrote:
It's documented. It's guaranteed not to change.
https://docs.python.org/3/reference/expressions.html#evaluation-order
Thanks. That's good to know, though I continue to hope nobody makes a habit of writing code that depends on it.
On Sat, May 12, 2018 at 1:25 PM Steven D'Aprano <steve@pearwood.info> wrote:
It's simpler in the sense that each of the components is simpler, and that the components exhibit *separation of concerns*. The bindings of the expressions are separate from calculation of the returned expression. The separation is both conceptual since the bindings are all distinct pieces of the whole expression, and temporal since the bindings all happen in some order, and must happen first. It can be very hard to get the ordering right. What if you are implementing something using the visitor pattern and you have something like: (visitor.access(database, key=database.key) given database = kwargs.pop("database") given visitor = database.default_visitor()) The beauty of given is that in this pattern: EXPRESSION given A = B given C = D you can completely forget about * B and D when thinking about EXPRESSION, * D and EXPRESSION when thinking about B, and * B and EXPRESSION when thinking about C. That's the separation of concerns. Now with :=, it's tricky because you can't set the visitor without first setting the database, but the visitor shows up first in the expression! Even if you could do something like: (visitor := database.default_visitor()).access((database := kwargs.pop("database")), key=database.key) I think this is much more complicated because all of the pieces are in the same giant expression, and so the concerns aren't separate. It's not obvious how to temporally order the pieces. And it takes a couple minutes to work out which parts of the expression fit with which other parts of the expression. If anyone ever writes code like that, you can guarantee that the Google style guide will preclude := except for maybe its the simplest uses.
Back when I was on the ACM team, people wrote code like that. It's fine if you're a brilliant contestant who has a few hours to solve a bunch of problems on one computer. It makes sense to optimize for number of characters, and anyway, ideally you were the only one reading your code unless your team had to help you debug it. However, and I'm sorry to be so bold, but this code will never pass a code review at a big company. It may have been easy for you to write, but it is not easy to understand at all. You're using the binding of := to update a a value that's used in subsequent iterations of the loop. That's way too complicated for a one-liner. You can write code like this for yourself, but no one else should have to decipher that. This is what I meant when I mentioned the difference between code at programming contests versus code written by those same software engineers years later at big companies. In my professional career (most recently at Google), the main goal was writing clear, accessible code so that the next person looking at it (or even me months later) would not have to think about it at all. It is immediately obvious that simple code is right. Even if you're brilliant enough to understand the code above, it needs to be immediately obviously correct to next person looking at it. Compare it with the for loop: def leaky_integral(signals, decay): value = 0.0 for x in signals: value = value * decay + x * (1 - decay) yield value
Nick is right in my opinion to argue this. If you want the bound target to leak out, just write the for loop. Otherwise you're asking too much of the reader.
I showed one above where := isn't even possible.
Sorry, I didn't mean to offend you. I fleshed out what I meant in this email. Two separate parts of my life as a programmer were my contest life and my professional career, and each had different ideals.
I don't agree with that. += is simpler since in that case the number of elements is fewer. The reason I'm saying that := is not simpler is that it tangles together unrelated subelements, which I explained in my separation of concerns paragraphs above.
Part 2. On Sat, May 12, 2018 at 08:16:07AM -0700, Neil Girdhar wrote:
I love given compared with := mainly because
[...]
Until you learn and become familiar with a new syntax, there is generally going to be a period you have to decipher it. I spent a long time mentally translating list comprehensions into mathematical set builder notation before it became second nature to me. Even now, I know people who find decorators and comprehensions indecipherable. Or at least, so they claim, and they aren't motivated to bother learning them. Oh well, that's their loss. Binding-expressions aren't like asynchronous programming, where the entire coding paradigm is different, and you literally have to think about your algorithms in another way. Whether you spell the binding operation target := expr expr as target expr -> target target given target = expr let target = expr : target expr ; (that last one is stolen from Forth, and should not be taken as a serious suggestion) is just a matter of spelling. We'll get used to whatever spelling it is. Some may be more convenient than others, or more error-prone, or may be harder to parse, but they're secondary issues. (Important, but still secondary.) Fundamentally, the operation is the same regardless of the spelling: - evaluate an expression - bind that value to a name - return the value (Except of course Nick's "given" suggestion is more complex, since the returned value is not necessarily the same as the bound value.)
* there are no difficult mental questions about evaluation order, e.g., in a bracketed expression having multiple assignments.
Aren't there just? x = 1 print( x + x given x = 50 ) Will that print 100, or 51? Brackets ought to make it clearer: (x + x given x = 50) # this ought to return 100 x + (x given x = 50) # this ought to return 51 but if I leave the brackets out, I have no idea what I'll get.
I would expect that your first example is a NameError: a.b(a := c.d()) since Python evaluates arguments to a method *after* looking up the method. So that corresponds to: - look up "a" # NameError, unless you've already got an "a" - look up "a.b" - evaluate c.d() - assign that value to a - pass that to the a.b method we found earlier What you probably want is the second version: (a := c.d()).b(a) which of course looks like utter crap. It might look better if we use at least half-way sensible variable names and a more realistic looking example, instead of obfuscated one-letter names. (widget := widget_builder.new(*args)).method(widget)
I think that this argument is really weak. The obvious answer is, if you don't want the assignment to happen unconditionally, then don't do the assignment unconditionally. Where you do it depends on when you want the assignment to take place. There's no mystery here. # unconditionally pop from a list (spam if mylist.pop(idx) else eggs) + mylist # conditionally pop from a list (mylist.pop(idx) if condition else eggs) + mylist (spam if condition else mylist.pop(idx)) + mylist Take your choice of which you want: (spam if (mylist := expr) else eggs) + mylist ((mylist := expr) if condition else eggs) + mylist (spam if condition else (mylist := expr)) + mylist Of course, in the last two cases, you're going to get a NameError when you try to add mylist if the ternary operator took the wrong branch. Unless you already defined mylist earlier. If you want something else, you have to explain what you want.
given makes it obvious by separating assignment from the usage of its assignment target.
This is just a purely mechanical source transformation from one to the other. Just replace ":" with "mylist given mylist " and you are done. # unconditional version (spam if (mylist given mylist = expr) else eggs) + mylist # conditional versions ((mylist given mylist = expr) if condition else eggs) + mylist (spam if condition else (mylist given mylist = expr)) + mylist If you can write the "given" versions, then just do the reverse transformation and replace the redundant verbosity with a colon.
What if they do? Is it really the end of the world if some ex-Pascal coders or people with an over-developed desire for (foolish) consistency add a colon to their statement level assignments? Some people add semi-colons to their statements too. Do we care? No. But if it aggitates people so much, then I'm perfectly happy with Guido's suggestion that we simply ban top level binding expressions and require them to leave the colons out.
* it looks a lot like the existing Python "for" and "if" clauses, which also do in-expression assignments.
"if" clauses do an assignment? Have you borrowed the keys to Guido's time machine, and are writing from 2025 and Python 4.2? *wink* I don't think "given" expressions look even remotely similar to either. for target in iterable: if condition: another_expr given target = expr Aside from "all three use a keyword". -- Steve
On Sat, May 12, 2018 at 2:28 PM Steven D'Aprano <steve@pearwood.info> wrote:
It has to be 100, or else outer parentheses change how an expression is evaluated.
That's the problem I'm showing. This is impossible: (spam if (mylist := expr) else eggs) + mylist but just fine with given: ((spam if mylist else eggs) + mylist) given mylist = expr)
First of all, you cannot convert all := expressions to given expressions. Even if you could, the point is that they are separated. I went over the separation of concerns in my other mail.
You can't always do that. I've given you two examples now where you cannot go backwards.
I meant the clauses not the statements: (expression for x in it if x.y given z = x.z) All three clauses have the same format, and obvious temporal ordering. for and given both bind a name that is visible to the expression and to clauses below.
On Sun, May 13, 2018 at 10:06 AM, Neil Girdhar <mistersheik@gmail.com> wrote:
I don't understand. How is that impossible with the colon-equals form? The 'if' expression is always going to be evaluated, followed by exactly one of 'spam' and 'eggs'. So mylist will be properly assigned before you get to adding it onto the end. ChrisA
On Sat, May 12, 2018 at 08:06:02PM -0400, Neil Girdhar wrote:
On Sat, May 12, 2018 at 2:28 PM Steven D'Aprano <steve@pearwood.info> wrote:
Whether that it right or wrong, you're missing the point. You said that we don't have to care about evaluation order. But we do: even if your analysis is correct, not everyone will realise it. I didn't. And I still don't, because I think your analyse is wrong. "Outer parentheses" is a red herring. I should have written: (x + x) given x = 50 # this ought to return 100 x + (x given x = 50) # this ought to return 51 and now there are no outer parentheses to worry about. The question is now whether "given" binds more tightly than + or not. We get to choose the precedence, and whatever we choose, it will surprise (or annoy) some people, and more importantly, some people simply won't know, and will have to ponder the difficult question of what the evaluation order is. This is already true today with code that has side-effects. I can simulate "given" using exec. It only works in the module scope: # another_expr given target = expr exec("target = expr") or another_expr but now we can see how precedence makes a real difference: x = 1 x + (exec("x = 50") or x) versus: x = 1 exec("x = 50) or (x + x) Regardless of which behaviour we choose, some people won't know it and will have to deal with "difficult mental questions about evaluation order". That's unavoidable, regardless of how we spell it, := or "given". -- Steve
On 14May2018 02:04, Steven D'Aprano <steve@pearwood.info> wrote:
Just to this: as a general principle I think "words" should bind less tightly than "punctuationlike operators". Certainly for myself I read most code as "words surrounding expressions". While "given" and "for (comprehension)" and so forth are all words within expressions as far as the language grammar goes, to my intuition and _visually_ they are bulky separators between terse "mathy" things. Even the Boolean words "and", "or", "not"...) fit this role. So as a matter of egonomic design "given" ought also to bind less tightly than "+". Arithmetic before logic before declaration before control, if you like. Cheers, Cameron Simpson <cs@cskk.id.au>
04.05.18 15:06, Nick Coghlan пише:
Sorry, but these examples don't look as good examples for inline assignments to me. I think that all these cases can be written better without using the inline assignment.
This case can be better handled by combining patterns in a single regular expression. pattern = re.compile('(?P<foo>pattern1)|(?P<bar>pattern2)|...') m = pattern.search(data) if not m: # this can be omitted if the pattern is always found ... elif m.group('foo'): ... elif m.group('bar'): ... See for example gettext.py where this pattern is used.
This case can be better handled by re.finditer(). for m in pattern.finditer(remaining_data): ... In more complex cases it is handy to write a simple generator function and iterate its result. The large number of similar cases are covered by a two-argument iter().
There are a lot of ways of writing this. PEP 572 mentions them. Different ways are used in real code depending on preferences of the author. Actually the number of these cases is pretty low in comparison with the total number of comprehensions. It is possible to express an assignment in comprehensions with the "for var in [value]" idiom, and this idiom is more powerful than PEP 572 in this case because it allows to perform an assignment before the first 'for'. But really complex comprehensions could be better written as 'for' statements with explicit adding to the collection or yielding.
[Nick Coghlan <ncoghlan@gmail.com>]
That deserves more thought. I started my paying career working on a Fortran compiler, a language which, by design, had no reserved words (although plenty of keywords). The language itself (and vendor-specific extensions) never had to suffer "but adding a new keyword could break old code!" consequences. In practice that worked out very well, Yes, you _could_ write hard-to-read code using language keywords as, e.g., identifier names too, but, no, absolutely nobody did that outside of "stupid Fortran tricks" posts on Usenet ;-) It had the _intended_ effect in practice: no breakage of old code just because the language grew new constructs. It's no longer the case that Python avoided that entirely, since "async def", "async for", and "async with" statements were added _without_ making "async" a new reserved word. It may require pain in the parser, but it's often doable anyway. At this stage in Python's life, adding new _reserved_ words "should be" an extremely high bar - but adding new non-reserved keywords (like "async") should be a much lower bar. That said, I expect it's easier in general to add a non-reserved keyword introducing a statement (like "async") than one buried inside expressions ("given").
On Fri, May 4, 2018 at 11:11 AM, Tim Peters <tim.peters@gmail.com> wrote:
Do note that this was a temporary solution. In 3.5 we introduced this hack. In 3.6, other uses of `async` and `await` became deprecated (though you'd have to use `python -Wall` to get a warning). In 3.7, it's a syntax error.
I'd also say that the difficulty of Googling for the meaning of ":=" shouldn't be exaggerated. Currently you can search for "python operators" and get tons of sites that list all operators. I also note that Google seems to be getting smarter about non-alphabetic searches -- I just searched for "python << operator" and the first hit was https://wiki.python.org/moin/BitwiseOperators -- --Guido van Rossum (python.org/~guido)
On Fri, May 4, 2018, 11:35 Guido van Rossum <guido@python.org> wrote:
Without adding hits to the search algorithm, this will remain the case. Google must have clicks to rank up. Right now there is no page, nothing on a high "Google juice" page like python.org, no one searching for it, and no mass of people clicking on it. no SO questions, etc. there is a transient response for all change. uniqueness and length of search term is just a faster one. All python syntax is findable eventually due to popularity. plus a better search is "why would I use...in python" or similar. = python also doesn't bring up anything interesting that wouldn't be had because of just "python". The details are too mundane and/or technical and everyone knows already.
that being said, if := had been (theoretically) included from the beginning, would people continue to have issues with it? unlikely, but I can't know. familiarity will cure many of these issues of readability or symbolic disagreement no matter what is chosen (well, to a point). it's unfortunate that changes have to be made up front with so little information like that, so I'm not advocating anything based on this, just pointing it out. I do think post hoc assignment will cause a cognitive load, like trying to figure out which variable is the iterator, and having to keep two contexts till the end of a comp with one given statement. [f(x) + a for all a in blah given x=1] not worse than a double nested comp though.
On Sat, May 5, 2018 at 5:27 AM, Matt Arcidy <marcidy@gmail.com> wrote:
Did you try? I searched for 'python :=' and for 'python colon equals' and got this hit each time: https://stackoverflow.com/questions/26000198/what-does-colon-equal-in-python... Which, incidentally, now has a response to it citing PEP 572. Good ol' Stack Overflow. ChrisA
[Tim]
[Guido]
See my "that deserves more thought" at the start, but wrt future cases then ;-) In 3.5 and 3.6, "everything just works" for everyone. In 3.7 the implementation gets churned again, to go out of its way to break the handful of code using "async" as an identifier. It's obvious who that hurts, but who does that really benefit? My experience with Fortran convinces me nobody would _actually_ be confused even if they wrote code like: async def frobnicate(async=True): if async: async with ... But nobody would actually do that. Then again, "but people _could_ do that!" barely registers with me because the nobody-actually-does-it theoretical possibilities were so much worse in Fortran, so I tend to tune that kind of argument out reflexively. For example, whitespace was also irrelevant in Fortran, and these two statements mean radically different things: D O1 0I=1 00,30 0 D O1 0I=1 00.30 0 The first is like: for I in range(100, 301): # the block ends at the next statement with label 10 The seconds is like: DO10I = 100.300 All actual Fortran code spells them like this instead: DO 10 I = 100, 300 DO10I = 100.300 The differing intents are obvious at a glance then - although, yup, to the compiler the difference is solely due to that one uses a comma where the other a period. I'm not suggesting Python go anywhere near _that_ far ;-) Just as far as considering that there's no actual harm in Fortran allowing "DO" to be a variable name too. Nobody is even tempted to think that "DO" might mean "DO loop" in, e.g., DO = 4 X = FUNC(DO) X = DO(Y) IF (DO.OR.DONTDO) GOTO 10 etc. People generally _don't_ use Fortran keywords as identifiers despite that they can, but it's a real boon for the relatively rare older code that failed to anticipate keywords added after it was written.
I've noted before that people don't seem to have trouble finding the meaning of Python's "is", "and", and "or" either. But Googling for "is" (etc) on its own isn't the way to do it ;-)
Ya - most arguments are crap ;-)
On Fri, May 4, 2018 at 1:53 PM, Tim Peters <tim.peters@gmail.com> wrote:
IIUC, Javascript has also gone all-in on contextual keywords. The realities of browser deployment mean they simply cannot have flag days or break old code, ever, meaning that contextual keywords are really the only kind they can add at all. So their async/await uses the same kind of trick that Python 3.5 did, and I believe they plan to keep it that way forever. FWIW. -n -- Nathaniel J. Smith -- https://vorpus.org
I agree it would be useful to have new keywords without being reserved, and we could even go with mechanism like infix operators created by user. It would allow things like [given for x in range(5) given given = x+1] or even [given for given in range(given) given given = given + 1] haha, but as other pointed out, it would be called Bad practice ^^ By the way, I still prefer "where" to "given". Le sam. 5 mai 2018 à 10:52, Greg Ewing <greg.ewing@canterbury.ac.nz> a écrit :
I think that "expr as y" was discarded too quickly. It would be a syntax completely familiar to Python programmers, and the new semantics would be obvious. That choice would also be in line with the Zen of Python. The special cases that may arise over "except" and "with" can be worked out and documented. Cheers, On Sat, May 5, 2018 at 5:22 AM, Robert Vanden Eynde <robertve92@gmail.com> wrote:
-- Juancarlo *Añez*
On Sat, May 05, 2018 at 06:24:07AM -0400, Juancarlo Añez wrote:
I think that "expr as y" was discarded too quickly.
This discussion started back in *February*. I don't think "too quickly" applies to ANYTHING about it. https://mail.python.org/pipermail/python-ideas/2018-February/048971.html And Chris' first draft of the PEP: https://mail.python.org/pipermail/python-ideas/2018-February/049041.html I have been one of the main proponents of "as". See, for example: https://mail.python.org/pipermail/python-ideas/2018-April/049880.html At least, I *was*. I'm now satisfied that "as" is the wrong solution, and I don't think it was discarded too quickly. Even though it makes me sad that "as" is not suitable, I'm satisfied that the problems with it would require too high a price to solve. The major problem is that it will clash with "except as" and "with as" statements. Of course we *could* introduce some sort of special treatment, possibly as simple as simply banning the use of binding- assignments inside except/with statements, but such special rules add complexity, make the feature less useful, harder to learn, and more surprising. Allowing or disallowing particular expressions after a certain keywork ought to be a last resort. Even though this was my preferred solution, I've now come to change my mind and think this would have been a mistake. (Thanks Chris for sticking to your guns and rejecting "as".) You say:
The special cases that may arise over "except" and "with" can be worked out and documented.
but there's no "may" about this. Using "as" does clash, it's not a matter of whether or not it will clash, we know it will. And it's easy to say that it "can" be worked out, but unless you have a concrete proposal to work it out, that's not really an argument in favour for "as", it is just a hope. Guido has also correctly pointed out that will "as" is used to bind names in other contexts, it doesn't *quite* work the same as regular = assignment. Again, the "with" statement is especially relevant: with expression as name does not bind the value of the expression to name, except by coincidence. It actually binds the value of expession.__enter__() to name. I still, and probably always will, like the look of result = (expression as spam) + spam**2 but I'm realistic to realise that it isn't practical. -- Steve
On Fri, May 4, 2018 at 8:06 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I think this is a step in the right direction. I stayed away from the PEP 572 discussions because while intuitively it felt wrong, I could not formulate what exactly was wrong with the assignment expressions proposals. This proposal has finally made me realize why I did not like PEP 572. The strong expression vs. statement dichotomy is one of the key features that set Python apart from many other languages and it makes Python programs much easier to understand. Right from the title, "Assignment Expressions", PEP 572 was set to destroy the very feature that in my view is responsible for much of Python's success. Unlike PEP 572, Nick's proposal does not feel like changing the syntax of Python expressions, instead it feels like an extension to the if-, while- and for-statements syntax. (While comprehensions are expressions, for the purposes of this proposal I am willing to view them as for-statements in disguise.)
On Fri, May 4, 2018 at 6:56 PM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
This is what makes me uncomfortable too. As Dijkstra once wrote: "our intellectual powers are rather geared to master static relations and ... our powers to visualize processes evolving in time are relatively poorly developed. For that reason we should do (as wise programmers aware of our limitations) our utmost to shorten the conceptual gap between the static program and the dynamic process, to make the correspondence between the program (spread out in text space) and the process (spread out in time) as trivial as possible." [1] Normally, Python code strongly maps *time* onto *vertical position*: one side-effect per line. Of course there is some specific order-of-operations for everything inside an individual line that the interpreter has to keep track of, but I basically never have to care about that myself. But by definition, := involves embedding side-effects within expressions, so suddenly I do have to care after all. Except... for the three cases Nick wrote above, where the side-effect occurs at the very end of the evaluation. And these also seem to be the three cases that have the most compelling use cases anyway. So restricting to just those three cases makes it much more palatable to me. (I won't comment on Nick's actual proposal, which is a bit more complicated than those examples, since it allows things like 'if m.group(1) given m = ...'.) (And on another note, I also wonder if all this pent-up desire to enrich the syntax of comprehensions means that we should add some kind of multi-line version of comprehensions, that doesn't require the awkwardness of explicitly accumulating a list or creating a nested function to yield out of. Not sure what that would look like, but people sure seem to want it.) -n [1] This is from "Go to statement considered harmful". Then a few lines later he uses a sequence of assignment statements as an example, and says that the wonderful thing about this example is that there's a 1-1 correspondence between lines and distinguishable program states, which is also uncannily apropos. -- Nathaniel J. Smith -- https://vorpus.org
On Fri, May 04, 2018 at 09:56:10PM -0400, Alexander Belopolsky wrote:
I'm not so sure that the "expression versus statement" dichotomy is as strong or as useful as Alexander says. If it were, we'd be writing much more imperative code, as if it were 1970 and we were using BASIC. Some of the greatest Python successes have been to add expression forms of what used to be purely imperative statements: - comprehensions (for-loops); - ternary if (if...else statement). In addition, we have more overlap between statements and expressions: - an expression form of def (lambda); - a functional form of the import statement (importlib.import_module, before that people used to use __import__(); - expression forms of augmented assignment (ordinary + etc operators), which is a rare case where the expression form came first and the imperative command came afterwards. I may have missed some. Also notable is that Python does not have "procedures" (pure statement callables). We use functions instead, and simply ignore the returned result. If the difference between statements and expressions was really a dichotomy, we would only need One Way to do these things, not two. I also have seen many people disappointed that Python doesn't treat "everything as an expression". The lack of assignment-expressions is a turn-off for some people: https://mail.python.org/pipermail/python-list/2018-May/732890.html See also: https://stackoverflow.com/questions/50090868/why-are-assignments-not-allowed... -- Steve
[Nick Coghlan <ncoghlan@gmail.com>]
I'm not clear on what "these specific cases" are, specifically ;-) Conditions in "if", "elif", and "while" statement expressions? Restricted to one "given" clause, or can they chain? In a listcomp, is it one "given" clause per "if", or after at most one "if"? Or is an "if" even needed at all in a listcomp? For example, [(f(x)**2, f(x)**3) for x in xs] has no conditions, and [(fx := f(x))**2, fx**3) for x in xs] is one reasonable use for binding expressions. [(fx**2, fx**3) for x in xs given fx = f(x)] reads better, although it's initially surprising (to my eyes) to find fx defined "at the end". But no more surprising than the current: [(fx**2, fx**3) for x in xs for fx in [f(x)]] trick.
The problem with complex targets in general assignment expressions is that, despite trying, I found no plausible use case for (at least) unpacking syntax. As in, e.g., x, y = func_returning_twople() if x**2 + y**2 > 9: # i.e., distance > 3, but save expensive sqrt The names can be _unpacked_ in a general assignment expression, but there appears to be no sane way then to _use_ the names in the test. This may be as good as it gets: if [(x, y := func_returning_twople()). x**2 + y**2 > 9][-1]: That reminds me of the hideous (condition and [T] or [F])[0] idiom I "invented" long ago to get the effect (in all cases) of the current T if condition else F That was intended to be goofy fun at the time, but I was appalled to see people later use it ;-) It''s certain sanest as if x**2 + y**2 > 9 given x, y = func_returning_twople(): "given" really shines there!
Which is OK. The one-letter variable name obscures that it doesn't actually reduce _redundancy_, though. That is, in the current match = pattern.search(data) if match: it's obviously less redundant typing as: if match := pattern.search(data): In if match given match = pattern.search(data): the annoying visual redundancy (& typing) persists.
# This name is rebound on each trip around the loop while m given m = pattern.search(remaining_data):
Also fine, but also doesn't reduce redundancy.
# "f(x)" is only evaluated once on each iteration result = [(x, y, x/y) for x in data if y given y = f(x)]
As above, the potential usefulness of "given" in a listcomp doesn't really depend on having a conditional. Or on having a listcomp either, for that matter ;-) r2, r3 = fx**2, fx**3 given fx = f(x) One more, a lovely (to my eyes) binding expression simplification requiring two bindings in an `if` test, taken from real-life code I happened to write during the PEP discussion: diff = x - x_base if diff: g = gcd(diff, n) if g > 1: return g collapsed to the crisp & clear: if (diff := x - x_base) and (g := gcd(diff, n)) > 1: return g If only one trailing "given" clause can be given per `if` test expression, presumably I couldn't do that without trickery. If it's more general, if (diff given diff = x _ xbase) and g > 1 given g = gcd(diff, n): reads worse to my eyes (perhaps because of the "visual redundancy" thing again), while if diff and g > 1 given diff = x - x_base given g = gcd(diff, n): has my eyes darting all over the place, and wondering which of the trailing `given` clauses executes first.
...
On 5 May 2018 at 13:36, Tim Peters <tim.peters@gmail.com> wrote:
Exactly the 3 cases presented (if/elif/while conditions). The usage in comprehensions would mirror the usage in if statements, and avoid allowing name bindings in arbitrary locations due to the implicity nested scopes used by comprehensions. Conditional expressions would be initially omitted since including them would allow arbitrary name bindings in arbitrary locations via the quirky "x if x given x = expr else x" spelling, and because "else" isn't as distinctive an ending token as "given ... :", "given ... )", "given ... ]", or "given ... }".
There were a couple key reasons I left the "for x in y" case out of the initial proposal: 1. The "for x in y" header is already quite busy, especially when tuple unpacking is used in the assignment target 2. Putting the "given" clause at the end would make it ambiguous as to whether it's executed once when setting up the iterator, or on every iteration 3. You can stick in an explicit "if True" if you don't need the given variable in the filter condition [(fx**2, fx**3) for x in xs if True given fx = f(x)] And then once you've had an entire release where the filter condition was mandatory for the comprehension form, allowing the "if True" in "[(fx**2, fx**3) for x in xs given fx = f(x)]" to be implicit would be less ambiguous. [snip]
Yep, that's why I don't have the same immediate reaction of "It would need to be limited to simple names as targets" reaction as I do for assignment expressions. It might still be a good restriction to start out with, though (especially if we wanted to allow multiple name bindings in a single given clause). [snip] The one-letter variable name obscures that it doesn't
Right, but that's specific to the case where the desired condition really is just "bool(target)". That's certainly likely to be a *common* use case, but if we decide that it's *that* particular flavour of redundancy that really bothers us, then there's always the "if expr as name:" spelling (similar to the way that Python had "a and b" and "a or b" logical control flow operators long before it got "a if c else b"). One more, a lovely (to my eyes) binding expression simplification
I was actually thinking that if we did want to allow multiple assignments, and we limited targets to single names, we could just use a comma as a separator: if diff and g > 1 given diff = x - x_base, g = gcd(diff, n): return g Similar to import statements, optional parentheses could be included in the grammar, allowing the name bindings to be split across multiple lines: if diff and g > 1 given ( diff = x - x_base, g = gcd(diff, n), ): return g (Other potential separators would be ";", but that reads weirdly to me since my brain expects the semi-colon to end the entire statement, and "and", but that feels overly verbose, while also being overly different from its regular meaning)
I find that last effect is lessened when using the comma as a separator within the given clause rather than repeating the keyword itself. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
[Nick]
And some people claim ":=" would make Python harder to teach ;-) [Tim]
I contrived that specific "use case", of course - I actually didn't stumble into any real code where multiple targets would benefit to my eyes. Perhaps because, as you noted above of `"for x in y" headers`, multiple-target assignment statements are often quite busy already too (I have no interest in cramming as much logic as possible into each line - but "sparse is better than dense" doesn't also mean "almost empty is better than sparse" ;-) ).
(especially if we wanted to allow multiple name bindings in a single given clause).
Right, but that's specific to the case where the desired condition really is just "bool(target)".
Not only. If the result _needs_ to be used N times in total in the test, binding expressions allow for that, but `given` requires N+1 instances of the name (the "extra one" to establish the name to begin with). For example, where `probable_prime()` returns `True` or `False`, and `bool(candidate)` is irrelevant: # highbit is a power of 2 >= 2; create a random prime # whose highest bit is highbit while (not probable_prime(candidate) given candidate = highbit | randrange(1, highbit, 2)): pass versus while not probable_prime(candidate := highbit | randrange(1, highbit, 2)): pass There I picked a "long" name to make the redundancy visually annoying ;-)
That's certainly likely to be a *common* use case,
In all the code I looked at where I believe a gimmick like this would actually help, it was indeed by far _most_ common that the result only needed to be used once in the test. In all such cases, the binding expression spelling of the test requires one instance of the name, and the `given` spelling two.
Reducing each redundancy is a small win to me, but reaches "importance" because it's so frequent. Binding expressions have more uses than _just_ that, though. But I'm sure I don't know what they all are. When a _general_ feature is added, people find surprising uses for it. For example, at times I'd love to write code like this, but can't: while any(n % p == 0 for p in small_primes): # divide p out - but what is p? Generator expressions prevent me from seeing which value of `p` succeeded. While that's often "a feature", sometimes it's a PITA. I don't know whether this binding-expression stab would work instead (I'm not sure the PEP realized there's "an issue" here, about the intended scope for `thisp`): while any(n % (thisp := p) == 0 for p in small_primes): n //= thisp If that is made to work, I think that counts as "a surprising use" (capturing a witness for `any(genexp)` and a counterexample for `all(genexp)`, both of which are wanted at times, but neither of which `any()`/`all()` will ever support on their own).. I suppose I could do it with `given` like so: while p is not None given p = next( (p for p in small_primes if n % p == 0), None): n //= p but at that point I'd pay to go back to the original loop-and-a-half ;-)
I expect that 's bound to be confusing, because the assignment _statement_ diff = x - x_base, g = gcd(diff, n) groups very differently than intended: diff = (x - x_base, g) = gcd(diff, n) And that's a syntax error. With enclosing parens, expectations change, and then people would expect it to work like specifying keyword arguments instead:
Keyword arguments work as a syntactic model (group as intended), but not semantically: if they really were keyword arguments, `x - x_base` and `gcd(diff, n)` would both be evaluated _before_ any bindings occurred. So it's more quirky `given`-specific rules no matter how you cut it. The closest bit of Python syntax that captures the grouping (but only partially), and the "left-to-right, with each binding in turn visible to later expressions" semantics, is the semicolon. Which would create even weirder expectations :-(
Yup.
I find that last effect is lessened when using the comma as a separator within the given clause rather than repeating the keyword itself.
Definitely. I tend to believe Python has "slightly more than enough" meanings for commas already, though. But using commas and _requiring_ parens for more than one `given` binding seems least surprising to me overall. Then again, everyone already knows what ":=" means. They just dislike it because so many major languages already have it -)
On Sun, May 06, 2018 at 02:00:35AM +1000, Nick Coghlan wrote:
o_O I'm replying to an email which is a week old. I haven't seen anyone other than Tim comment on that "if True" boilerplate. Are you still proposing that? If not, you can ignore the following.
I'm sorry, perhaps I'm having another slow day, but I don't see the ambiguity in the first place. And I *certainly* do not see how adding in a logically superfluorous "if True" would make it unambiguous. For comparison sake, here is the PEP 572 proposal compared to yours: [((fx := f(x))**2, fx**3) for x in xs] I'd read that as for each x in xs, let fx be f of x, return fx squared and fx cube [(fx**2, fx**3) for x in xs if True given fx = f(x)] which I'd read as for each x in xs, if True, return fx squared and fx cube, given fx is f of x and then wonder why on earth the test is there. Explain to me again why this boilerplate is necessary, please. Especially since you're already suggesting that in a future release it could be dropped without changing the meaning. [Tim Peters]
That really isn't the case. if result.method() given result = compare(data, arg): versus the PEP 572 syntax: if (result := compare(data, arg)).method(): So it certainly isn't just the bool(target) case that is redundant. -- Steve
On 06May2018 02:00, Nick Coghlan <ncoghlan@gmail.com> wrote:
I'm well behind, but... this! This turns "given" into a +0.8 for me. That's really nice. It reads clearly too. I was hitherto in the "expression as name" camp, which I gather is already rejected. Cheers, Cameron Simpson <cs@cskk.id.au>
On 13May2018 07:07, Cameron Simpson <cs@cskk.id.au> wrote:
And if we're still worried about new keywords or reserved words, this: if diff and g > 1 with ( diff = x - x_base, g = gcd(diff, n), ): return g or: if diff and g > 1 with ( x - x_base as diff, gcd(diff, n) as g ): return g or even: if diff and g > 1 with ( x - x_base, gcd(diff, n), ) as diff, g: return g read nearly as well to my eye. My main point here is that "with" works as well as "given" in this form from an English prose point of view. Cheers, Cameron Simpson <cs@cskk.id.au>
My main point here is that "with" works as well as "given" in this form from an English prose point of view.
+1 for "with...as", -1 for ":=" About affecting existing contexts, it seems that "with..as" would create a new context just for the expression, and the control statement it is embedded in, similar to what the current "with" statement does. These are semantics that are really easy to explain. Cheers!
On Sun, May 13, 2018 at 8:47 AM, Juancarlo Añez <apalala@gmail.com> wrote:
The trouble with every variant involving 'with' is that the semantics LOOK similar, but are subtly different. The current 'with' statement doesn't create a subscope; the only "context" it creates is regarding the resource represented by the context manager. For instance, opening a file in a 'with' block will close the file at the end of the block - but you still have a (closed) file object. Using "with... as" for name bindings wouldn't call __enter__ or __exit__, so it won't create that kind of context; and whether it creates a subscope for the variable or not, it's not going to match the 'with' statement. ChrisA
That is all valid, but it still would be familiar, and easier to explain. Python already uses "in", which is used in other languages to introduce context. The statement structure of "with...as" seems desirable, just asking for a word that is not "with" or "given". I don't remember if "when" was already rejected. http://www.thesaurus.com/browse/with?s=t http://www.thesaurus.com/browse/given?s=t http://www.thesaurus.com/browse/considering?s=t http://www.thesaurus.com/browse/assume?s=t http://www.thesaurus.com/browse/when?s=t Cheers! -- Juancarlo *Añez*
On Sat, May 12, 2018 at 07:36:33PM -0400, Juancarlo Añez wrote:
Python already uses "in", which is used in other languages to introduce context.
Fortunately, we don't have to come up with syntax that works with other languages, only Python.
The statement structure of "with...as" seems desirable
But it's not a statement, its an expression.
just asking for a word that is not "with" or "given".
How about "antidisestablishmentarianism"? That's unlikely to be used in many programs, so we could make it a keyword. *wink* I jest, of course. But I don't think "with" reads well, and given doesn't really work for me either *as prose*. In my experience mathematicians put the given *before* the statement: Given a, b, c three sides of a triangle, then Area = sqrt(s*(s-a)*(s-b)*(s-c)) where s = (a + b + c)/2 is the semi-perimeter of the triangle. For the record, that is almost exactly what I wrote for a student earlier today, and its not just me, it is very similar to the wording used on both Wolfram Mathworld and Wikipedia's pages on Heron's Formula. http://mathworld.wolfram.com/HeronsFormula.html https://en.wikipedia.org/wiki/Heron%27s_formula Putting "given" after the expression is backwards. -- Steve
On 2018-05-13 04:23, Steven D'Aprano wrote:
Yes, but that's because we're ruling out the use of "where". At this point I would be fine with "snicklefritz" as the keyword. The point is that I want to put SOMETHING after the expression, and this is not at all unusual. See for instance Wikipedia pages on the Reimann zeta function (https://en.wikipedia.org/wiki/Riemann_zeta_function#Definition), gravitation equation (https://en.wikipedia.org/wiki/Gravity#Newton%27s_theory_of_gravitation), and compound interest (https://en.wikipedia.org/wiki/Compound_interest#Mathematics_of_interest_rate...). If we have to use the word "given" even though the word mathematicians would use in that position is "where", that's not such a big deal. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Sun, May 13, 2018, 11:28 Brendan Barnwell <brenbarn@brenbarn.net> wrote:
it is a big deal. postfix requires more cognitive load, we will have no idea up front what's going on except for trivial exames. more givens, more cognitive load. if you think spending that is fine for you, I can't argue, but to say it doesn't matter isn't correct. 2.exames which get far worse for complex cases. left for the for can be as complex.as.you wish. 1: [ x + y for t in range(10) ... ] 2: x = 10 y = 20 [ x + y for t in range(10) ...] up till you read ... you have no idea there even will be a substitution. The lower is even worse, you think you know, but then have to redo the whole problem with new information. also : mathematicians don't just put the _word_ "given", they put givens, things that are known or assumed to be true. Axioms and definitions, where definitions assign names to values. This is for formal arguements. reassigning values is handled in post fix occasionally once it is clear what x and y are. but that's not what we are talking about if the name doesn't exist already. again, you want to use given, that's fine, but the math argument is wrong, as is the "it doesn't matter" argument, assuming the current neurological model for working memory continues to hold. Maybe the difference is small, especially after familiarity sets in, but that doesn't mean the difference in load isn't there. it will only increase for more complex statements with more givens.
On 2018-05-13 11:53, Matt Arcidy wrote:
Sorry, but that's nonsense. In the message you're replying to, what I'm arguing "doesn't matter" is the WORD used to do the postfixing. Are you seriously saying that the "neurological model for working memory" means that "x + y where x = foo" is easier to understand, on a fundamental neurological basis, than "x + y given x = foo"? Since you're the one who was advocating for objective measures, I'm sure you'll understand if I want some solid objective evidence for that! If you're saying that the entire concept of postfixing the givens (rather than putting them before the expression) creates greater cognitive load, then how do you explain the use of this format in the Wikipedia articles I linked, not to mention various math papers, texts, etc. throughout the world? Finally, even if we allow that postfixed-givens does increase cognitive load on a single, initial reading, that's not sufficient. Part of what I'm saying is that on LATER readings it's faster to see the overall expression first, because you don't have to plow through the definitions of the givens. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Sun, May 13, 2018 at 2:54 PM Matt Arcidy <marcidy@gmail.com> wrote:
I usually read the vertical bar "given" in set-builder notation: {x | 10 < x < 100, x is prime} Conditional probabilities, expectations, entropy, etc.: P(X | Y), E(X| Y), H(X | Y), And, the so-called "evaluation bar": https://math.stackexchange.com/questions/52651/what-is-the-name-of-the-verti... In words, I agree that where is probably better in text because writing "given the gravitational constant G" puts the subject last, unlike "where G is the gravitational constant". However, the given proposal for Python is putting the subject first: "given x = y".
Most (if not all) uses of the vertical are read as "given" and they all put the givens to the right of it.
I think Peter tried to outline this earlier, but what he was laying out wasn't clear to me at first. There seem to be 4 variations when it comes to assignment expressions. I'm going to try to ignore exact keywords here since we can sort those out once we have settled on which variation we prefer. 1. infix: TARGET := EXPR 2. infix: EXPR as TARGET 3. prefix: let TARGET = EXPR in ANOTHER_EXPR 4. postfix: ANOTHER_EXPR given TARGET = EXPR Both 1 and 2 may appear in the context of a larger expression where TARGET may or may not be used: 1. 99 + (TARGET := EXPR) ** 2 + TARGET 2. 99 + (EXPR as TARGET) ** 2 + TARGET 3 and 4 require that TARGET appear in ANOTHER_EXPR, even if TARGET is the only thing contained in that expression, whereas with 1 and 2, TARGET need not be used again. Example I: 1. x := 10 2. 10 as x 3. let x = 10 in x 4. x given x = 10 In the simple case where the goal of the assignment expression is to bind the EXPR to the TARGET so that TARGET can be used in a future statement, 1 and 2 are clearly the most straightforward because they do not require ANOTHER_EXPR. # Please ignore that m.group(2) doesn't do anything useful here Example II: 1. if m := re.match(...): m.group(2) 2. if re.match(...) as m: res = m.group(2) 3. if let m = re.match(...) in m: m.group(2) 4. if m given m = re.match(...): m.group(2) I also think expressions that use "or" or "and" to make a compound expression benefit from the infix style, mostly because each sub-expression stands on its own and is only made longer with the repetition of TARGET: Example III: 1. if (diff := x - x_base) and (g := gcd(diff, n)) > 1: ... 2. if (x - x_base as diff) and (gcd(diff, n) as g) > 1: ... 3. if (let diff = x - x_base in diff) and (let g = gcd(diff, n) in g > 1): ... 4. if (diff given diff = x - x_base) and (g > 1 given g = gcd(diff, n)): ... In the more complex case where TARGET is reused in the expression, I find 3 and 4 to benefit as there is a separation of the binding from its usage. I can consider each expression separately and I don't have to deal with the assignment side effects at the same time. I believe this is what Neil is mostly arguing for. # Borrowing from Andre, please forgive any mathematical problems like division by 0 Example IV: 1: [(-b/(2*a) + (D := sqrt( (b/(2*a))**2 - c/a), -b/(2*a) - D) for a in range(10) for b in range(10) for c in range(10) if D >= 0] 2: [(-b/(2*a) + (sqrt( (b/(2*a))**2 - c/a as D), -b/(2*a) - D) for a in range(10) for b in range(10) for c in range(10) if D >= 0] 3. [let D = sqrt( (b/(2*a))**2 - c/a) in (-b/(2*a) + D, -b/(2*a) - D) for a in range(10) for b in range(10) for c in range(10) if D >= 0] 4. [(-b/(2*a) + D, -b/(2*a) - D) for a in range(10) for b in range(10) for c in range(10) if D >= 0 given D = sqrt( (b/(2*a))**2 - c/a)] Also in the case with multiple bindings I find that 3 and 4 benefit over 1 and 2: Example V: 1. [(x := f(y := (z := f(i) ** 2) + 1)) for i in range(10)] 2. [(f((f(i) ** 2 as z) + 1 as y) as x) for i in range(10)] 3. [let x = f(y), y = z + 1, z = f(i) ** 2 in x for i in range(10)] # maybe the order of the let expressions should be reversed? 4. [x given x = f(y) given y = z + 1 given z = f(i) ** 2 for i in range(10)] No matter which variation we prefer, there are plenty of arguments to be made that multiple assignment expressions in a single expression or usage of the TARGET later in the expression is harder to work with in most cases,. And since 1 and 2 (at least to me) are more difficult to parse in those situations, I'm more likely to push back on whoever writes that code to do it another way or split it into multiple statements. I feel that Steven prefers 1, mostly for the reason that it makes Examples I, II, and III easier to write and easier to read. Neil prefers 4 because Examples I, II, and II still aren't that bad with 4, and are easier to work with in Examples IV and V. If you feel that Examples IV and V should be written differently in the first place, you probably prefer infix (1 or 2). If you feel that Examples IV and V are going to be written anyway and you want them to be as readable as possible, you probably prefer prefix (3) or postfix (4). If you want to know what all the TARGETs are assigned to up front, you probably prefer 1 or 3 (for reading from left to right). If you want to see how the TARGET is used in the larger expression up front and are willing to read to the end to find out if or where the TARGET has been defined, you probably prefer 4. In my mind, all 4 variations have merit. I think I prefer prefix or postfix (postfix feels very natural to me) because I believe more complex expressions should be separateable (Neil argues better than I can for this). But Steven has gone a long way to convince me that the sky won't fall if we choose an infix variation because in practice our better angels will push us away from using expressions that are too complex. Prefix vs postfix is a discussion worth having if we decide that infix isn't the right choice. I would love to see us reach consensus (too optimistic?) or at least an acknowledgment of the explicit tradeoffs for whichever variation we ultimately choose. -- Nick ----- Original message ----- From: Matt Arcidy <marcidy@gmail.com> To: Brendan Barnwell <brenbarn@brenbarn.net> Cc: "python-ideas" <python-ideas@python.org> Subject: Re: [Python-ideas] Inline assignments using "given" clauses Date: Sun, 13 May 2018 11:53:20 -0700 On Sun, May 13, 2018, 11:28 Brendan Barnwell <brenbarn@brenbarn.net> wrote:
it is a big deal. postfix requires more cognitive load, we will have no idea up front what's going on except for trivial exames. more givens, more cognitive load. if you think spending that is fine for you, I can't argue, but to say it doesn't matter isn't correct. 2.exames which get far worse for complex cases. left for the for can be as complex.as.you wish. 1: [ x + y for t in range(10) ... ] 2: x = 10 y = 20 [ x + y for t in range(10) ...] up till you read ... you have no idea there even will be a substitution. The lower is even worse, you think you know, but then have to redo the whole problem with new information. also : mathematicians don't just put the _word_ "given", they put givens, things that are known or assumed to be true. Axioms and definitions, where definitions assign names to values. This is for formal arguements. reassigning values is handled in post fix occasionally once it is clear what x and y are. but that's not what we are talking about if the name doesn't exist already. again, you want to use given, that's fine, but the math argument is wrong, as is the "it doesn't matter" argument, assuming the current neurological model for working memory continues to hold. Maybe the difference is small, especially after familiarity sets in, but that doesn't mean the difference in load isn't there. it will only increase for more complex statements with more givens.
_________________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Sun, May 13, 2018 at 4:30 PM Nick Malaguti <python@fwdaddr.fastmail.fm> wrote:
Yes.
In my mind, your given clauses are upside-down. The way I see it, code like this: for a in range(10): if a != 5: for b in range(10): for c in range(10): D = sqrt((b/(2*a))**2 - c/a) if D >= 0: yield (-b/(2*a) + D, -b/(2*a) - D) should be cast into a generator like this: ((-b/(2*a) + D, -b/(2*a) - D) for a in range(10) if a != 5 for b in range(10) for c in range(10) given D = sqrt( (b/(2*a))**2 - c/a) if D >= 0) Just leave everything in the order you wrote it except the yield statement turns into a bare expression in front. Regarding prefix versus postfix, we already do postfix binding using "for" clauses and I want to be able to interleave given clauses with for clauses as above to prevent a bracketing mess that prefix would require.
On 13May2018 08:55, Chris Angelico <rosuav@gmail.com> wrote:
For myself, I'm not a fan of a narrow scope. I'd be happy with an inline assignment landing in the function scope (or whatever the current innermost scope is). Like normal assignments. In how many other places does Python adopt a narrower scope than the function/method/class/module? Consider: r = re.compile("bah(baz)") if m given m = r.match("foo barbaz"): thing = m.group(1) I _want_ to access "m" after the expression. My other dislike of narrow scopes is shadowing. I had an unpleasant experience last year working in Go, which has its own ":=" assignment. While Go's := semantics are a little different to the assignment expressions being considered here, it has an IMO dangerous shadowing effect. I'm going to digress into an explaination of this issue in Go here because I want to highlight my dislike of easy accidental shadowing, which is a problem in many areas, and I think that conflating inline assignment with an implied narrow scope in Python would make this worse. In Go you can declare variables 2 ways: var ok = True and: ok := True Also, because Go doesn't use exceptions a common idiom is to return a success/fail value and the task's result: ok, result := do_something(...) That handily declares "ok" and "result" and gives them values. Because the ":=" is so convenient, you might often declare variables as they get used: ok, result1 := do_something() ok, result2 := do_something_else() and so on. You're allowed to mix existing names with new (undeclared) names provided there's at least one new name left of the ":=". So far this is all just ordinary assignments, with convenient declaration included. But consider the function that calls other functions: func thing_outer(x) { ok, result1 := do_something() if ok { ok, result2 := do_something_else() if ok { fmt.Println("ok!") } } return ok, result } That's how it might go in Pythonic form. But Go lets you preceed the test condition with a statement, somewhat like the C "for (setup values; test; advance)" form: for (i=0; i<10; i++) So you may wish to write the function like this: func thing_outer(x) { if ok, result1 := do_something(); ok { if ok, result2 := do_something_else(); ok { fmt.Println("ok!") } } return ok, result } and here is where the scoping causes a disaster. In Go, the declarations _within_ the "if" statement have the "if" statement as their scope. In particular, the inner "ok" shadows the outer "ok", such that an inner failure (setting "ok" to false) doesn't affect the outer "ok" which was true. And so the overall function returns apparent success. And that is entirely enabled by the implied scoping in the "if" statement. The above example is subtly wrong because I'm doing this from memory, but I had a very simple real world example bite me this way and it was a PITA to debug because the effect was so surprising. So much so that from then on I pretty much eschewed the ":=" declaration as a dangerous construct and went with "var" all the time, effectively giving me _one_ scope within any function. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sat, May 12, 2018 at 5:54 PM Cameron Simpson <cs@cskk.id.au> wrote:
I love given, but that's the one thing I don't like. I prefer this: if (diff and g > 1 given diff = x - x_base given g = gcd(diff, n)): return g —just like for and if subexpressions. Doing this can also open up weirdness if someone tries to roll something like: a = f(), # Make a tuple of length 1 into a given statement. Now, where do you up the parentheses? given ( a = (f(),), b = whatever? ) Seems weird.
On 12/05/2018 23:52, Neil Girdhar wrote:
I don't like the consecutive "given"s. Reading it aloud in English suggests to me that the second "given" should be evaluated before the first, which I'm sure is not the intention. (Just as I think that multiple for-loops inside a comprehension sound the wrong way round. :-( But that ship has sailed.) Rob Cliffe
[attributions lost - sorry, but I can't get 'em back] ...
I'm well behind, but... this! This turns "given" into a +0.8 for me.
That's really nice. It reads clearly too.
Since that was my example to begin with, I think it's fair to point out that they all miss a key part of the original example: this code is working with multi-thousand bit integers, and calling gcd() is expensive. It was a key point that if (diff := x - x_base) and (g := gcd(diff, n)) > 1: return g didn't call gcd() _at all_ unless `diff` was non-zero. The original real-life code was: diff = x - x_base if diff: g = gcd(diff, n) if g > 1: return g
if (diff := x - x_base) and (g := gcd(diff, n)) > 1: return g
I don't see the advantage in that succinctness: g = special_gcd(x - x_base, n) if g: return g The code bases I work on constantly move towards having the next guy grok what's going on just by reading the code. It could also be: if special_gcd(x - x_base, n) as g: return g Cheers! Juancarlo *Añez*
On Sun, May 13, 2018 at 10:27 AM, Juancarlo Añez <apalala@gmail.com> wrote:
Now I have to go read elsewhere to figure out what "special_gcd" does. With Tim's original code, I could see the effect right there. It might not seem significant with a single function, but if you had this situation come up a dozen times, now you have a dozen functions, each one used only once. You have to go a LONG way out of line to find the meaning of this name. Remember: Giving a function a useful name means figuring out a name that means you do not need to read the function's body to understand what it does. It's easy to say "just make it a function", but that's utterly useless if the next reader has to grok the function's implementation to understand its usage. ChrisA
On Sat, May 12, 2018 at 08:27:53PM -0400, Juancarlo Añez wrote:
That's an excellent point. What's "special_gcd" and how does it differ from normal gcd? How am I supposed to grok that just from reading the code above? Do I have to dig into the source of special_gcd to understand it? What happens if the normal gcd would return zero? Is that going to lead to a bug?
No it can't, because "as" is not an option. -- Steven
On Mon, May 14, 2018 at 12:20 AM, Juancarlo Añez <apalala@gmail.com> wrote:
Okay, then. What happens to the succinctness? On Sun, May 13, 2018 at 10:27 AM, Juancarlo Añez <apalala@gmail.com> wrote:
It's actually this: def special_gcd(diff, n): return diff and gcd(diff, n) g = special_gcd(x - x_base, n) if g: return g Yes, very succinct. You can't have it both ways; either you need a name that completely defines the function, such that it can be placed out-of-line without needing to be read; or you're just creating an inline helper function, which doesn't shorten your code at all, and just adds another layer of wrapping around everything. Remember, you're contrasting with one of two options: either a nested if, or an inline 'and' with the ability to capture values. What you've created here is a multi-line way of capturing a value. That's all. ChrisA
On 13May2018 00:51, MRAB <python@mrabarnett.plus.com> wrote:
Yes, but in a slightly larger scope such as a cascaded if/elif/.../else sequence embedding the "given" into the expression has 2 advantages: it stops the if-else cascading across the terminal purely to embed assignments before the next test, and it lets one clearly associate these particular assignments as relevant to the test itself. Obviously there are circumstances for each, and I expect overuse of "given" degrades readability. However, I still find Nick's "if this given that" formulation quite compelling as a logical form. Cheers, Cameron Simpson <cs@cskk.id.au>
On Fri, May 4, 2018 at 3:06 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Well, I think it looks good. Nice job Nick! I like that it leaves the "=" alone finally (but it's maybe the reason why I am biased towards this syntax :) It has clear look so as not to confuse with conditionals and the variable is emphasized so it's well understood what it means. I wish I could like the whole idea of inline assignments more now - but well, it is just what it is. Something that can help on rare occasion. But as said, your variant feels right to me. Unlike with most other proposed constructs here my first impression - this is Python. If the new keyword is too much, there are some options too choose from. (though I've no idea whether this is plausible due to parsing ambiguities) if m given m = pattern.search(data): vs: if m with m = pattern.search(data): if m def m = pattern.search(data): if m as m = pattern.search(data): Yes, all these reads strange, but IMO understandable. BTW, using your keyword, in comprehensions vs the original idea of Chris then merely I suppose: result = [(x, y, x/y) given y = f(x) for x in data if y ] vs result = [(x, y, x/y) with y = f(x) for x in data if y ] result = [(x, y, x/y) def y = f(x) for x in data if y ] result = [(x, y, x/y) as y = f(x) for x in data if y ] vs result = [(x, y, x/y) for x in data if y given y = f(x)] vs result = [(x, y, x/y) for x in data if y with y = f(x)] result = [(x, y, x/y) for x in data if y def y = f(x)] result = [(x, y, x/y) for x in data if y as y = f(x)] I can't say for sure what is better but the former order seems to be slightly more appealing for some reson. [Tim]
I disagree, I think it is not redundant but it's just how it is - namely it does not introduce 'implicitness' in the first place, and that is what I like about Nick's variant. But of course it is less compact. PS: recently I have discovered interesting fact: it seems one can already use 'inline assignment' with current syntax. E.g.: if exec("m = input()") or m: print (m) It seem to work as inline assignment correctly. Yes it is just coincidence, BUT: this has the wanted effect! - hides one line in "if" and "while" - leaves the assignment statement (or is it expression now? ;) - has explicit appearance So we have it already? Mikhail
On 6 May 2018 at 00:33, Mikhail V <mikhailwas@gmail.com> wrote:
Not really, since: 1. exec with a plain string has a very high runtime cost (the compiler is pretty slow, since we assume the results will be cached) 2. exec'ed strings are generally opaque to static analysis tools 3. writing back to the local namespace via exec doesn't work consistently at function scope (and assuming PEP 558 is eventually accepted, will some day reliably *never* work) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 4 May 2018 at 22:06, Nick Coghlan <ncoghlan@gmail.com> wrote:
Since I genuinely don't think this idea is important enough to disrupt hypothesis's public API, I've been pondering potential synonyms that are less likely to be common in real world code, while still being comprehensible and useful mnemonics for the proposed functionality. The variant I've most liked is the word "letting" (in the sense of "while letting these names have these values, do ..."): # Exactly one branch is executed here if m letting m = pattern.search(data): ... elif m letting m = other_pattern.search(data)): ... else: ... # This name is rebound on each trip around the loop while m letting m = pattern.search(remaining_data): ... # "f(x)" is only evaluated once on each iteration result = [(x, y, x/y) for x in data if y letting y = f(x)] # Tim's "bind two expressions" example if diff and g > 1 letting diff = x - x_base, g = gcd(diff, n): return g # The "bind two expressions" example across multiple lines while diff and g > 1 letting ( diff = x - x_base, g = gcd(diff, n), ): ... # Do something with diff and g Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 6 May 2018 at 02:06, Nick Coghlan <ncoghlan@gmail.com> wrote:
I've also been pondering Tim's suggestion of instead enhancing the code generation pipeline's native support for pseudo-keywords in a way that can be explicitly represented in the Grammar and AST, rather than having to be hacked in specifically every time we want to introduce a new keyword without a mandatory __future__ statement (and without creating major compatibility headaches for code that uses those new keywords as attribute or variable names). While I haven't actually tried this out yet, the way I'm thinking that might look is to add the following nodes to the grammar: name_plus: NAME | pseudo_keyword pseudo_keyword: 'given' and the replace all direct uses of 'NAME' in the grammar with 'name_plus'. That way, to allow a new keyword to be used as a name in addition to its syntactic use case, we'd add it to the pseudo_keyword list in addition to adding it to . To avoid ambiguities in the grammar, this could only be done for keywords that *can't* be used to start a new expression or statement (so it wouldn't have been sufficient for the async/await case, since 'async' can start statements, and 'await' can start both statements and expressions). So if Guido's view on the out-of-order execution approach to inline name binding softens, I think this would be a better approach to pursue than making a more awkward keyword choice. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, 4 May 2018 at 05:07 Nick Coghlan <ncoghlan@gmail.com> wrote:
My brain wants to drop the variable name in front of 'given': if given m = pattern.search(data): while given m = pattern.search(remaining_data): Maybe it's because the examples use such a short variable name? if match given match = pattern.search(data): vs. if given match = pattern.search(data); Nope, I still like mine more. ;) -Brett
On 8 May 2018 at 04:19, Brett Cannon <brett@python.org> wrote:
Does that change if the condition isn't just "bool(name)"? For example: if y > 0 given y = f(x): ... That's the situation where I strongly prefer the postfix operator spelling, since it's pretty clear how I should pronounce it (i.e. "if y is greater than zero, given y is set to f-of-x, then ..."). By contrast, while a variety of plausible suggestions have been made, I still don't really know how to pronounce "if (y := f(x)) > 0:)" in a way that's going to be clear to an English-speaking listener (aside from pronouncing it the same way as I'd pronounce the version using "given", but that then raises the question of "Why isn't it written the way it is pronounced?"). I do agree with Tim that the name repetition would strongly encourage the use of short names rather than long ones (since you're always typing them at least twice), such that we'd probably see code like: while not probable_prime(n) given (n = highbit | randrange(1, highbit, 2)): pass Rather than the more explicit: while not probable_prime(candidate) given (candidate = highbit | randrange(1, highbit, 2)): pass However, I'd still consider both of those easier to follow than: while not probable_prime(candidate := highbit | randrange(1, highbit, 2)): pass since it's really unclear to me that "candidate" in the latter form is a positional argument being bound to a name in the local environment, and *not* a keyword argument being passed to "probable_prime". I've also been pondering what the given variant might look like as a generally available postfix operator, rather than being restricted to if/elif/while clauses, and I think that would have interesting implications for the flexibility of its usage in comprehensions, since there would now be *three* places where "given" could appear (as is already the case for the inline binding operator spelling): - in the result expression - in the iterable expression - in the filter expression That is: [(x, y, x - y) given y = f(x) for x in data] [(x, data) for x in data given data = get_data()] [(x, y, x/y) for x in data if y given y = f(x)] Rather than: [(x, y := f(x), x - y) for x in data] [(x, data) for x in data := get_data()] [(x, y, x/y) for x in data if y := f(x)] Opening it up that way would allow for some odd usages that might need to be discouraged in PEP 8 (like explicitly preferring "probable_prime(n) given n = highbit | randrange(1, highbit, 2)" to "probable_prime(n given n = highbit | randrange(1, highbit, 2))"), but it would probably still be simpler overall than attempting to restrict the construct purely to if/elif/while. Even as a generally available postfix keyword, "given" should still be amenable to the treatment where it could be allowed as a variable name in a non-operator context (since we don't allow two adjacent expressions to imply a function call, it's only prefix keywords that have to be disallowed as names to avoid ambiguity in the parser). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I'm sorry, but unless there's a sudden landslide of support for 'given' in favor of ':=', I'm really not going to consider it. I'd pronounce "if (x := y) > 0" as either "if y (assigned to x) is greater than zero" or "if x (assigned from y) is greater than zero". On Thu, May 10, 2018 at 6:39 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On 10/05/18 14:44, Guido van Rossum wrote:
I'm sorry, but unless there's a sudden landslide of support for 'given' in favor of ':=', I'm really not going to consider it.
OK, this is my ha'p'th in favour of 'given', for what little it's worth. The more I see of general assignment expressions, the less I like them. All I really want is a less clumsy way to write while true: thing_to_do = get_something_to_do() if thing_to_do == GET_OUT_OF_HERE: break # else do stuff -- Rhodri James *-* Kynesim Ltd
If it just needs a stream of +1s, I personally like the "given" approach much more than the ":=" approach, for all of the many reasons repeated many times in the various email chains. (I preferred it as "as", but that's been struck down already) (and if it's between ":=" and not having them at all, I would rather just not have them)
Please no, it's not that easy. I can easily generate a stream of +1s or -1s for any proposal. I'd need well-reasoned explanations and it would have to come from people who are willing to spend significant time writing it up eloquently. Nick has tried his best and failed to convince me. So the bar is high. (Also note that most of the examples that have been brought up lately were meant to illustrate the behavior in esoteric corner cases while I was working out the fine details of the semantics. Users should use this feature sparingly and stay very far away of those corner cases -- but they have to be specified in order to be able to implement this thing.) On Thu, May 10, 2018 at 10:26 AM, marky1991 . <marky1991@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On 2018-05-10 16:10, Guido van Rossum wrote:
Poor prospects, then, but I'll do my best. I think the most obvious argument (to me) favouring `given` over `:=` is that it separates the two things it's doing: if m.group(2) given m = pattern.search(data): as opposed to the more-nested := version: if (m := pattern.search(data)).group(2): which, at least to me, is more complicated to think about because it feels like it's making the .group() something to do with the assignment. Put another way, I think your use of parentheses when discussing the *pronunciation* of this thing is telling. It feels as though one needs to start in the middle and then go in both directions at once, first explaining the origin (or destination) of the operand in question in a parenthesized offshoot, and then switching context and describing what is done to it. It's midly mentally taxing. I'm sure we can all live with that, but I don't want to: Python's exceptionally-readable syntax is one of the bigger reasons I choose it. There's a striking parallel in C, where the well-known idiom: while ((c = getchar()) != EOF) ... has an obviously-nicer alternative: while (c = getchar(), c != EOF) ... Most people I show this to agree that it's nicer, despite the fact that it manages to repeat a variable name *and* use the comma operator. I don't have proof, but I'd suggest that unwrapping that layer of context for the reader imparts a significant benefit. The C example also provides a convenient test: if you think the former example is nicer, I can just give up now ;)
On Thu, 10 May 2018 at 16:49, Ed Kellett <e+python-ideas@kellett.im> wrote:
IMHO, all these toy examples don't translate well to the real world because they tend to use very short variable names while in real world [good written code] tends to select longer more descriptive variable names. Try replacing "c" with a longer name, like input_command, then it becomes: while ((input_command = getchar()) != EOF) ... while (input_command = getchar(), input_command != EOF) ... In the second example, having to type the variable name twice is an annoyance that adds almost nothing to readability, so I would definitely prefer the first one. The "given" proposals have the same issue. (a shame we can't use "as", for reasons already stated, it would have been perfect otherwise) -- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert
Gustavo Carneiro wrote:
I don't believe that's always true. It depends on the context. Sometimes, using long variable names can make code *harder* to read. I don't think there's anything unrealistic about this example: if m given m = pattern.match(the_string): nugget = m.group(2) Most people's short-term memory is good enough to remember that "m" refers to the match object while they read the next couple of lines. IMO, using a longer name would serve no purpose and would just clutter things up. -- Greg
On 11.05.2018 09:33, Greg Ewing wrote:
I gather we don't talk about interactive usage. So, I grepped through part of our code-base. Like it or not, it's almost always called "match" there. Like Gustavo, I also have the feeling that long-living, real-world code tends to have more descriptive names than all those toy examples/arguments. Cheers, Sven
On 11 May 2018 at 03:33, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I've been thinking about this problem, and I think for the If/elif/while cases it's actually possible to allow the "binding is the same as the condition" case to be simplified to: if command = pattern.match(the_string): ... elif command = other_pattern.match(the_string): ... while data = read_data(): ... Allowing this would be part of the definition of the if/elif/while statement headers, rather than a general purpose assignment expression. The restriction of the LHS to a simple name target would need to be in the AST generator rather than in the grammar, but it's hardly the only case where we do that kind of thing. Switching to the given expression form would then only be necessary in cases where the condition *wasn't* the same as the binding target. A similar enhancement could be made to conditional expressions (adjusting their grammar to permit "EXPR if NAME = EXPR else EXPR") and filter clauses in comprehensions (allowing "EXPR for TARGET in EXPR if NAME = EXPR"). In essence, "if", "elif", and "while" would all allow for an "implied given" clause in order to simplify the 90% case where the desired condition and the bound expression are the same. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 11.05.2018 13:43, Nick Coghlan wrote:
I can imagine that this will cover 80 to 90% of the usecases and it's readable as well.
Not sure if that is too much for now. List comprehensions tend to be longer than expected, the same goes for the ternary expression. Maybe, we could start with the 90% case and whether the need for more arises.
Exactly. Maybe, even here: let's do just the 90% case. And if a lot of people need finer control, we can reconsider :=/given/as. Regards, Sven
[Gustavo Carneiro]
[Greg Ewing]
[Nick Coghlan]
Unless there's some weird font problem on my machine, that looks like a single "equals sign". In which case we'd be reproducing C's miserable confusion about whether: if (i = 1) was a too-hastily-typed spelling of the intended: if (i == 1) or whether they were thinking "equals" and typed "=" by mistake. If so, that would get an instant -1 from any number of core devs, who have vivid painful memories of being burned by that in C. That's not just speculation - it came up a number of times in the PEP 572 threads.
Spell it ":=" (colon equals) instead, and a few core devs would stop objecting ;-)
On Fri, May 11, 2018 at 3:33 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Indeed. A thought just occurred to me. Maybe we need to instigate a cultural shift where people think about style guides as less dictated by hard-coded rules that were "passed down from the mountain" and more as derived from research that we can all understand about usability. A lot more is known about how human perception and various types of memory and learning work than it was when the "7 things plus/minus 2" rule was invented ( https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two). It would be fascinating to imagine a future where language designers could talk about such topic with as much confidence as they talk about the efficiency of hash tables. -- --Guido van Rossum (python.org/~guido)
On Fri, May 11, 2018 at 12:33:10PM -0400, Guido van Rossum wrote:
That would be fantastic, but it's a real uphill battle. How long have we known about the effects of low-contrast on readability? https://www.wired.com/2016/10/how-the-web-became-unreadable/ http://contrastrebellion.com/ And don't get me started on the awful, awful choices made for chart design and data visualisation. We know how to make good charts: https://en.wikipedia.org/wiki/Edward_Tufte and yet: https://i.redd.it/0w0y1r1ba8x01.jpg -- Steve
On 2018-05-10 17:10, Gustavo Carneiro wrote: literally hundreds of others do the same, and I've never seen anyone spell it `input_command`. I do the same in Python in a few contexts, usually where the variable's meaning is very clear from its usage, or where the code in question doesn't know or care what the variable is used for: for i in range(10): def get_const(self, x): for k in self.defaults.keys(): etc., and if we had `given` clauses, I'd probably do this there too. I think one of the advantages of that syntax is that it makes it extremely clear what's going on: if m given m = re.match(...): I don't need to try to stuff an English description of m into its name, because the `given` clause already describes it perfectly.
On 10 May 2018 at 11:10, Guido van Rossum <guido@python.org> wrote:
I raised this with some of the folks that were still here at the Education Summit (similar to what I did for data classes at the PyCon Australia education seminar last year), but whereas the reactions to data classes were "as easy or easier to teach as traditional classes", the reaction for this for the folks that I asked was almost entirely negative - the most positive reaction was "Yes, if it's as a wholesale replacement for the '=' spelling, since that sometimes gets confused with mathematical equality". As an *addition* to the existing spelling, and especially with the now proposed leaking semantics in comprehension scopes, it was "No, that would just confuse out students". It's one thing adding syntactic and semantic complexity for the sake of something that significantly increases the language's expressive power (which is what the original sublocal scopes proposal was aiming for: the ability to more readily express constrained scoping and name shadowing without explicit name aliasing and del statements), it's something else entirely to do it for the sake of purely cosmetic tweaks like flattening the occasional nested if-else chain or replacing a loop-and-a-half with an embedded assignment. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hi all. I've been lurking for a little while on this discussion and I thought I might contribute some thoughts. One of my hurdles for ":=" is understanding when I should use it rather than "=". Should I use it everywhere? Should I use it only where I can't use regular "="? Is it a personal choice? Will it become so common that I need to think harder because some people will use it really frequently or intermix them? I don't want to see "=" vs ":=" become like semicolons in JavaScript. When I work on a different codebase, am I going to have to follow an "always" or "never" for binding expressions? Maybe this is all overblown and PEP8 direction will keep everyone on the same page, but I guess I worry about there being 2 very similar, but not the same, ways to do it. What I really like about "given" is it makes it a lot clearer when I should use it. No one is going to want to write x given x = f() if they can just write x = f() If you need a binding expression in a comprehension or an if or while statement, you'll know the pattern of using "given" to save that loop and a half or to call a function and bind its result while iterating. Just like you know when to use a ternary if to save that extra temporary variable - there's little confusion about when to use a ternary, especially since a few if statements quickly prove clearer to read. 10 if x == 5 else 9 if x == 2 else 8 if x == 3 else 100 looks much better as: if x == 5: result = 10 elif x == 2: result = 9 elif x == 3: result = 8 else: result = 100 I feel the same way about given. If you feel tempted to go overboard with: x given x = y * 2 given y = z + 3 given z = f() Which should be equivalent to: x := (y := ((z := f()) + 3)) * 2 hopefully you'll think, "maybe I should just make 3 statements instead?" And also I have no trouble following what that statement actually does when using given. I didn't need any parenthesis to make sure I didn't bind the wrong expressions and I don't have to read it from the inside out. Each sub-expression is complete rather than being mixed together (even though I have to read it from right to left). I feel like the strongest argument for ":=" is for all the situations where someone will actually want a binding expression in real code, ":=" is more succinct. I'm just concerned that when given a new binding expression hammer, everything is going to look like a nail and all the places where someone could really benefit from a binding expression will be drowned out by the unnecessary usage of ":=" (and its side effects). -- Nick ----- Original message ----- From: Guido van Rossum <guido@python.org> To: "marky1991 ." <marky1991@gmail.com> Cc: "python-ideas" <python-ideas@python.org> Subject: Re: [Python-ideas] Inline assignments using "given" clauses Date: Thu, 10 May 2018 11:10:50 -0400 Please no, it's not that easy. I can easily generate a stream of +1s or -1s for any proposal. I'd need well-reasoned explanations and it would have to come from people who are willing to spend significant time writing it up eloquently. Nick has tried his best and failed to convince me. So the bar is high. (Also note that most of the examples that have been brought up lately were meant to illustrate the behavior in esoteric corner cases while I was working out the fine details of the semantics. Users should use this feature sparingly and stay very far away of those corner cases -- but they have to be specified in order to be able to implement this thing.) On Thu, May 10, 2018 at 10:26 AM, marky1991 . <marky1991@gmail.com> wrote:> If it just needs a stream of +1s, I personally like the "given"
-- --Guido van Rossum (python.org/~guido) _________________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, May 10, 2018 at 10:31:00PM -0400, Nick Malaguti wrote:
When should I use "x = x + 1" and when should I use "x += 1"? When should I write alist.sort() and when sorted(alist)? When should I use a list comp and when should I use a for-loop? I could give a dozen more examples. We always have a choice in writing code. There is almost never *only* one way to do it. Nevertheless, we manage, and I believe that most of the things which perplex us today will be so trivially easy to understand tomorrow that we'll wonder what the fuss was all about. It is sometimes humbling to remember back at the things that we swore were terrible, terrible mistakes. I hated augmented assignment, until I gave in and started using them. I hated != instead of <> and I actually seriously considered using "from __future__ import barry_as_FLUFL" for a while. Depending on whether you are a "glass half full" or "glass half empty" kind of guy, you could say either that: - we're irrationally risk adverse, and would rather miss out on something fantastic than risk something slightly not so good; - or like Stockholm Syndrome victims, we can get used to, and even learn to enjoy, the most awful crap if it goes on long enough. (Or possibly both at the same time.) Personally, I think that in hindsight my dislike of != was irrational and rather silly, rather than a prophetic realisation that the dropping of <> began the ruination of Python. YMMV.
I don't want to see "=" vs ":=" become like semicolons in JavaScript.
The difference between = and := is nothing like automatic semicolon insertion in Javascript. The Python interpreter is never going to (only sometimes) insert a colon to make your "name = expression" work and it is never going to turn two statements into a single broken expression because you used = instead of := or vice versa. Of course people can still screw up the precedence and get the wrong results, but the same applies to all operators and the same solution applies: when in doubt, add parentheses to make it clear. Some arguments against := are better than others: will it encourage people to write unreadable one-liners instead of using multiple statements? Maybe, but I think most people will show more sense and restraint. You haven't seen many quadruply-nested list comprehensions, or ternary if operators nested ten deep, have you? I haven't.
When I work on a different codebase, am I going to have to follow an "always" or "never" for binding expressions?
Yes, of course you are. Some places will be conservative and say Never, and some will be gung ho and say Always, but the majority will say "Use them when they make the code better". Just like when you work on some code bases you will have to always follow PEP 8, and when you work on other code bases, you will have to follow different rules.
You seem to have two contradictory opinions here. Paraphrasing: "I worry that the same people who never abuse ternary if by writing obfuscated one-liners will suddenly turn around and abuse := binding expressions by writing obfuscated one-liners." and "clearly nobody will abuse 'given' binding expressions by writing obfuscated one-liners, because they don't abuse ternary if to write obfuscated one-liners." I understand being pessimistic about the common-sense of my fellow programmers. I understand being optimistic about the common-sense of my fellow programmers. I don't understand doing both at the same time. If people will abuse := they will abuse "given". If they won't abuse "given", they surely won't abuse := either. Since we have over a quarter of a century of experience showing that the Python community, as a whole, tends not to abuse syntax to write unreadable one-liners, I think that fears about people abusing := are overblown. -- Steve
Guido van Rossum wrote:
I'd need well-reasoned explanations
My reasoning is essentially the same as what I've already said about "where". To summarise, "given" sounds like something an English-speaking mathematician would write, whereas ":=" doesn't even have an obvious pronunciation. Some variation on "given" just seems greatly more pythonic to me. -- Greg
+1 to this reasoning. One of the main reason python is popular is because code is easy to read, while ":=" would clearly not be as readable as "given". For me the difference between "given" and ":=" is the same as between python and C for loops. On Fri, 11 May 2018 at 09:06 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I dont really like "given". If we compare: if m given m = re.match(stuff): to if m := re.match(stuff) then I count 4+(name_length) more tokens and 2 more spaces. Since I believe := is perfectly clear, I don't see the reason for a far more verbose syntax. That all said, I would still prefer: if re.match(stuff) as m: which is exactly equal to the := in line length and parallels with. While that may -technically- be a different beast. For beginners the difference is really irrelevant, and you can just tell advanced people the full story(technically speaking the as in a with statement isn't an expression assignment, it's a part of the with statement, and it feeds in the value through the context manager machinery before binding it. Similar for the except statement.). But I've kind of given up on "as" (so no need to reply on that bit).
On 11/05/18 11:14, Jacco van Dorp wrote:
I respectfully disagree with your opinion (i.e. you're wrong :-) Consider: while (cmd := get_command()).token != CMD_QUIT: cmd.do_something() vs: while cmd.token != CMD_QUIT given cmd = get_command(): cmd.do_something() I find I write code like this[*] a fair bit, since my major use for Python is to write remote monitors for embedded kit, so it's pretty much a real world example. I don't find the first version using ":=" to be perfectly clear, in fact I think it's rather ugly. That may be partly the same reaction that many of us had to the asymmetry of assignment expressions in (over-)complicated comprehensions. The second version using "given" reads much more naturally to the mathematician in me, and not too badly to my English half either. [*] By "like this" I mean the clunky "while true:" spelling, obviously. -- Rhodri James *-* Kynesim Ltd
On Fri, May 11, 2018 at 9:37 PM, Rhodri James <rhodri@kynesim.co.uk> wrote:
Yes, I've considered it. And I don't like the fact that the evaluation is right-to-left. It isn't a problem when your condition is extremely simple, but I can guarantee you that people will use this with more complicated conditions. And when that happens, you have to be aware that the tail of the statement is actually evaluated before the primary expression. It's like with Perl: die("blah blah") unless some_condition Reverse-order evaluation is confusing and frequently annoying. It's not an instant failure of the proposal, but it's a serious cost, and I'd much rather avoid it by using := (which leaves the expression where it is). ChrisA
Actually, the first version is more readable. It's got a lot to do with what Chris said about order of operations, but IMO, even more with grouping. There's a couple of things you might want to learn from this statement. First, will the while check succeed? Well.... while......get_command()).token != CMD_QUIT: yup, looks clear. I can ignore the cmd := part easily, and the bonus paren I see there doesn't matter that much. Another thing i might be curious about, is what is the value of cmd after ? while (cmd := get_command())........................: Looks like it has the value of getcommand(). Hey, that was both clear and readable. I can just ignore half the line and learn stuff. Great. Lets see with the other notation. What's the value of cmd ? while ................................................... cmd = get_command(): That works. Bit of line I had to skip. Will the check succeed ? while cmd.token != CMD_QUIT .....................................: Wait, what's the value of cmd ? Lets look in the code in the preceding lines....oh, ok, it's at the end of the line. I actually have to mentally parse the entire line to get what the check will work. This, along with what Chris said about order of operations, reduce the readability of the "given" version.
On Fri, May 11, 2018 at 11:34 PM, Rhodri James <rhodri@kynesim.co.uk> wrote:
No, not a win. Do you read the entire source code for an entire project before trying to comprehend one part of it? I doubt it. Do you read an entire file before trying to comprehend a single function in that file? No. Do you even read an entire function before processing one line in that function? Unlikely. It's normal and correct to seek to understand one part of some code while ignoring other parts. That's why we have proper variable names, even inside functions - we could just use "slot0" and "slot1" and so on, since that's how they work to the interpreter. But we use good names, so that you can understand some code without having to first read the thing that created that variable. ChrisA
A while ago, we had this gem: 2018-04-06 8:19 GMT+02:00 Serhiy Storchaka <storchaka@gmail.com>:
Using currently supported syntax:
smooth_signal = [average for average in [0] for x in signal for average in [(1-decay)*average + decay*x]]
Go ahead and understand that line in 1 go. It's currently legal syntax for a running average for a smoothing signal, which remembers something about it. (Subject: Proposal: A Reduce-Map Comprehension and a "last" builtin) You're not allowed to work it out bit by bit, just understand the entire line or nothing. Any failure of yours proves my point.
[João] How do you read something like " while (cmd := get_command()).token != CMD_QUIT:" in plain english?
while open-paren cee em dee colon is call get-underscore-command close-paren dot token doesn't equal all-caps cee em dee underscore quit colon. Might be some dutch in there. But far more importantly, I can hold the concept into my head, or just the parts of it that I need. How we call it in english is actually not a good argument - whether we can easily mentally parse it is, since I tend not to code by voice command, but with a keyboard. Your mileage may vary, but I think we should optimize for keyboard coding over voice chat coding. And when I need to refer to it, I say "this bit here" or I copy paste it.
On 11/05/18 15:04, Jacco van Dorp wrote:
Personally I thought it proved the point that you shouldn't be trying to squash things like that into a list comprehension in the first place, because average = 0 smooth_signal = [] for x in signal: average = (1 - decay) * average + decay * x smooth_signal.append(average) is quite a bit more comprehensible. -- Rhodri James *-* Kynesim Ltd
On Sat, May 12, 2018 at 02:37:20AM +0100, Rob Cliffe via Python-ideas wrote:
Do you mean Serhiy's example of currently supported syntax? smooth_signal = [average for average in [0] for x in signal for average in [(1-decay)*average + decay*x]] It helps if you know the algorithm for exponential smoothing: for each value x (aside from the first), the average is equal to a mix of the current value x and the previous average A, split by some proportion P: A = (1-P)*A + P*x If P is 0.5, that is equivalent to taking the ordinary average between the current value and the previous average: A = (A+x)/2 # when P == 0.5 In the comprehension, P is called "decay" and A is called "average": average = (1-decay)*average + decay*x Writing the comprehension as a single line is hard to read. Let's give it some structure: smooth_signal = [average # append average to the results for average in [0] for x in signal for average in [(1-decay)*average + decay*x] ] Horrible as it is, it is perfectly legal Python right now. It uses for name in SINGLE_ITEM_LIST to perform an assignment. So that's equivalent to: average = 0 for x in signal average = (1-decay)*average + decay*x append average to the results Pull the initial value of average out of the comprehension, and use the PEP 572 syntax: average = 0 smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal] which is a huge improvement in my opinion. It would be more obvious if the expression being calculated came first: smooth_signal = [(1-decay)*average + decay*x as average for x in signal] but there are good reasons why the "as" syntax won't work. So it looks like we're stuck with needing to look ahead past the := to see the actual value being appended to the list. A minor inconvenience, equivalent to that in ternary if, where we have to look ahead to see the condition: [target := COMPREHENSION_VALUE for x in sequence] true_value if CONDITION else false_value So I expect that it will take me a little while to learn to look ahead and read binding-expressions fluently. (Like comprehensions themselves, really. It took me a few months to stop needing to pull them apart to understand them.) He's Nick's version, as best as I am able to tell: average = 0 smooth_signal = [(average given average = (1-decay)*average + decay*x) for x in signal] So we have the same look-ahead needed to see the expression we care about, but instead of merely having two characters := needed to do the binding, we need "given average =". -- Steve
I would write this using a for loop and the two-argument form of iter: for cmd in iter(get_command, ''): if cmd.token == CMD_QUIT: break cmd.do_something() or from itertools import take while for cmd in takewhile(lambda x: x.token != CMD_QUIT, iter(get_command, '')): cmd.do_something() Depending on what get_command actually returns, you might be able to construct a valid sentinel that doesn't require an explicit test of cmd.token. (This reminds that I wish ``iter`` could take a predicate instead of a sentinel as its second argument. Then you could just write for cmd in iter(get_command, lambda x: x.token == CMD_QUIT): cmd.do_something() ) -- Clint
2018-05-11 17:06 GMT+03:00 Clint Hepner <clint.hepner@gmail.com>:
But you can do it right now: class P: def __init__(self, key): self.key = key def __eq__(self, other): return self.key(other) for cmd in iter(get_command, P(lambda x: x.token == CMD_QUIT)): cmd.do_something() With kind regards, -gdg
On Fri, May 11, 2018 at 12:37:43PM +0100, Rhodri James wrote:
Okay, considered. I think the first is preferable. Much earlier in the PEP 572 discussion, I strongly argued in favour of the expr as name syntax on the basis that the most important part of the overall expression is "expr", not the assignment target, and therefore that should come first. Even though I have accepted that "as" is not viable, I still believe that it is preferable to have the expression first, or if not first, at least as close to the left as we can get it. This "given" syntax puts the expr part all the way to the far right of the line. A line which is made all the longer for needing to use "given" and redundantly state the target name. It's like we're trying to maximize the distance the eye has to travel back and forth when reading. I have to read to the end of the line before I have any idea where cmd has come from or what it is. The fact that it comes from a "given" expression comes as a surprise at the end of the line. Now obviously this doesn't matter if I'm reading lines of code in careful detail, but I don't do that all the time. I skim code far more than I read it in careful detail, and the closer things are to the left, the more likely I am to see them while skimming. The further out they are, the easier they are to miss. I think that "given" will *literally* make reading harder, in that the eye has to travel further to spot the relevant expression while skimming over code. As I said, I don't think it makes any difference when reading closely in detail. But most of my reading of code is skimming to find the relevant line or section, and then read closely. I would probably skim a hundred lines for every one I read closely. We read more code than we write, but writing is important too. I think the verbosity of "given" (six chars versus two) and the redundancy of needing to repeat the name of the target even if you only use it once will soon make using this syntax seem like a chore. -- Steve
On 2018-05-11 09:17, Steven D'Aprano wrote:
That is an interesting argument --- interesting to me because I agree with a lot of it, but it leads me to the opposite conclusion, and also because it highlights some of the relevant factors for me. The main part I disagree with is that the most important thing is the definition of expr. Rather, what I think is most important is the role of expr within the surrounding expression. For simple cases, it doesn't much matter which comes first: if x := do_some_stuff(): if x where x = do_some_stuff(): . . . and it's true the latter is a bit more verbose in that case for little extra benefit. But when the locally-defined value is used within a more complicated expression (like the quadratic formula example), I think readability goes down significantly. To appease Tim, instead of using the quadratic formula, though, I will use a more realistic example that comes up fairly often for me: wanting to do some kind of normalization on a piece of data for a comparison, while keeping the unnormalized data for use within the block: if some_condition and (stuff:= get_user_input()).lower().strip().replace('-', ''): versus if some_condition and stuff.lower().strip().replace('-', '') given stuff = get_user_input(): Now, the latter is still more verbose. But to me it is now more readable, because the assignment does not disrupt the flow of reading the surrounding expression. This benefit increases the more complicated the surrounding expression is. Your point about reading ease is well taken, but also relevant to me is that we only read a piece of code *for the first time* once. The advantage of the given-style assignment is that on multiple readings, it foregrounds how the assigned value is USED, not how it is DEFINED. This encourages a "top-down" understanding of the expression in which you first understand the overall picture, and then later "drill down" into definition of what the components are. I wonder if some of the disagreement about the relative merits of the two cases comes from people focusing on different kinds of examples. As I said, I think the advantages of "cleft assignment" (i.e., where the assignment is shunted to the end of the line) become more pronounced as the surrounding expression becomes more complex. They also become somewhat greater as the definition of the expression becomes more complex, because there is more to skip over when finding out how the value is used. But many of the examples we're seeing are very simple ones, and in particular have a trivial surrounding expression. (That is, in something like "m := re.match()" the assigned value is not being used in any larger expression; the assigned value constitutes the whole of the expression.) I'm also finding it useful to think of parallel situations in prose writing, particularly journalistic-style prose writing. The current Python behavior, in which assignments must be done before the block, I think of as akin to something like this: "The IAAEA (International Association of Assignment Expression Advocates) is an organization dedicated to the promotion of assignment expressions. Its president is Eddie McEquals. In a statement yesterday, McEquals called for a new syntax to bring assignment expressions to the masses." The inline-assignment syntax is akin to this: In a statement yesterday, Eddie McEquals, president of the International Association of Assignment Expression Advocates (IAAEA), an organization dedicated to the promotion of assignment expressions, called for a new syntax to bring assignment expressions to the masses. The cleft-assignment syntax is akin to this: In a statement yesterday, Eddie McEquals called for a new syntax to bring assignment expressions to the masses. McEquals is the president of the International Association of Assignment Expression Advocates (IAAEA), which is an organization dedicated to the promotion of assignment expressions. Now of course I'm fudging a bit on the details (like whether you have IAAEA in parentheses or its full expansion), but the point is that the last version foregrounds the "bottom line" or "predicate" --- what actually happened. The first one foregrounds the "participants", or who/what was involved, and saves what actually happened for the end. But the middle one, to my eye, foregrounds nothing. It stuffs everything into one big clause where the descriptions of the participants occur as asides that disrupt the flow of reading. By the time we get to the predicate, we have to jump back to the beginning to remember who it is we're talking about, Again, the difference in readability varies depending on the complexity of the different components. If we just have "Eddie McEquals, president of the International Association of Assignment Expression advocates, delivered a speech yesterday at the organization's convention", the inline appositive is not so disruptive. But the more complex the inline definitions become, and especially the more complex the expression in which they are embedded, the lower readability goes, for me. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On 2018-05-11 11:08, Brendan Barnwell wrote:
Ironically I weakened my argument by forgetting to finish my expression there. I intended that chain of method calls to be used in a comparison to make the surrounding expression more complex. So revise the above to if some_condition and (stuff := get_user_input()).lower().strip().replace('-', '') == existing_value: versus if some_condition and stuff.lower().strip().replace('-', '') == existing_value given stuff = get_user_input(): -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
[Brendan Barnwell]
[also Brendan]
Even more ironically, to my eyes the original more strongly supported your view than the rewrite ;-) "given stuff =" stuck out in the original because it was preceded by punctuation (a right parenthesis). I had to read the rewrite 3 times before i realized you were even _using_ "given", because there it's buried between two other names - "existing_value given stuff" - and visually looks more like it's actually the 3rd of 4 words (because of the underscore in "existing_value"). Of course that would have been obvious in a Python-aware editor that colored "given" differently, but as-is I found the original easy to read but the rewrite a puzzle to decode. Similarly, in the rewritten assignment expression spelling, it's obvious at a glance that the test is of the form some_condition and some_messy_expression == existing_value but in the rewritten "given" sample that's obscured because "existing_value" not only doesn't end the statement, it's not even followed by punctuation. Of course coloring "given" differently would remove that visual uncertainty too. For a dumb display, I'd write it if some_condition and ( stuff.lower().strip().replace('-', '') == existing_value) given stuff = get_user_input(): instead (added parens so that "existing_value" and "given" are separated by punctuation).
On May 11, 2018 1:45:27 PM Tim Peters <tim.peters@gmail.com> wrote:
There are some variants of tanks like 'if let' where the bindings come *first*, unlike 'given' where they come last (like Haskell's 'where').
Well, you've partly explained the reason: our eyes are drawn to what sticks out. In this case, the := stuck out in a section heavy on black letters. In a proper editor, it may even be the other way around: they tend to highlight keywords in a stronger manner (like bolding) than operators.
Apology for top post, but this is a general statement about Readability and not a response to an individual. it would be nice to list the objective parts separate from the "argument" (i.e. debate, not fight), perhaps list them then make a case for which metric is a more important, and which values produce better results. Key strokes, keyboard location, location of information vs where it is used, cognitive load (objective barring neuroscience changes) are all objective points (along with many other points raised). "Better" will be decided by Guido I guess, but listing objective points with explanatory examples gives a basis for that discussion. Legibility, for example, is not objective at all, it has nothing to do with syntax. This covers fonts, colors, monitors, lighting, chairs, lunch, etc. None of this is relevent to the symbols or their ordering in a file we all must read. Teachability likewise. My opinion here is learnability is far more important anyways, I am 90% self taught going back 25 years, but this is equally unquantifiable. Perhaps just trust students to learn as an author must trust a reader. Of course, let it not be lost that determining teachability and learnability for something which doesn't even exist yet is quite challenging. Any quantification will give more information than only naked impassioned pleas to Readability. Note Tim came up with a real metric: 2 * count(":=")/len(statement). It's objective. it's just unclear if a higher score is better or worse. However, one could say "a Tim of .3 is considered too high" as a guideline. Perhaps coders find these opportunities to express feelings and opinion cathartic after speaking to the most pedantic creature on Earth (compilier/computer) but I think exercising the high skill level available here to dissect and find objective statements is a worthy puzzle. On Fri, May 11, 2018, 11:14 Brendan Barnwell <brenbarn@brenbarn.net> wrote:
On Fri, May 11, 2018 at 11:52:02AM -0700, Matt Arcidy wrote:
I think Tim was making a joke about demanding objective measurements of subjective things. Certainly he hasn't done any research or study to justify that metric. He just plucked the formula out of thin air. Or at least no peer reviewed research. -- Steve
[Matt Arcidy]
[Steven D'Aprano]
It was the outcome of an intense 17-year research project.
Or at least no peer reviewed research.
Au contraire! My peers are here, and that message was reviewed by at least 3 people on this list. That said, I am a fan of objectively measuring subjective things, just not of taking the measurements seriously ;-) If people do want to take it seriously, check out prior Python art first: http://radon.readthedocs.io/en/latest/intro.html
On Fri, May 11, 2018, 17:04 Tim Peters <tim.peters@gmail.com> wrote:
apparently my joke was objectively not funny :-) I thought calling it a "Tim" was sufficient. Im not serious about actually ranking for the purposes of a PEP. I brought it up when I felt the subjectivity was making the debate worse. Reiterating my point, n long sub-threads about fonts, screens, etc are ridiculous when those exist outside the pyfile. I don't know why personal preference for a font would stop a useful tool. Hopefully those arguments are ignored. Likewise for googlability, teachability, cross-language similarity and familiarity. If the tool is useful, that's all that will matter with respect to these points, they solve themselves. I happen to be working on a ranking tool for code (not quality, just an ordering to find entry points for new devs), so i tossed the idea in. it seemed appropriate to remind people that the fact that not everyone uses green to highlight "+" doesn't make "+" somehow more or less _useful_ (people -1'd just for legibility alone because of their personal feelings) I'm not sure where people stand on usefulness, but it's clear this tool is a pattern. No counter example of "but I can already do this" is related to other counter examples in the way that named expressions solves all of them, and does it succinctly regardless of chosen syntax. Some required imports! Obviously making these decisions with the future unknown is nearly impossible and requires careful consideration of all points, but I don't think Hypothetical Bill's perscription glasses should determine syntax decisions. Best of luck with the hard parts, clearly I hope the PEP makes it.
If people do want to take it seriously, check out prior Python art first:
Awesome, thanks!
On Fri, May 11, 2018 at 03:47:05PM +0200, João Santos wrote:
How do you read something like " while (cmd := get_command()).token != CMD_QUIT:" in plain english?
I wouldn't if I could avoid it. I hardly ever program by talking about code in plain English. Often the lines are gobblydegook: zreplace = '%c%02d%02d' % (sign, h, m) # datetime.py and even when they are technically pronouncable English: # subprocess.py (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) = self._get_handles(stdin, stdout, stderr) my brain would glaze over by the second "p2c". I prefer to read and write code than speak it, and if I need to discuss it, I prefer to use a whiteboard so I can write things down. But if I really needed to, I'd probably start by reading it as: while open bracket command defined as get-command close bracket dot token is not equal to command-quit and then I'd probably drop the "defined" and swap the order around. Actually, that's not true. I probably wouldn't say that, not in a real conversation. What I'd probably say is, So, like, I've got this command object, see, which I get from calling get-command, right, and so I get the, um, token attribute, okay, and if that's not equal to the quit value, I loop until it is. Right? (And this is why I prefer *writing* code than *saying* code.) -- Steve
On Fri, May 11, 2018 at 11:40:51AM +0200, Jacco van Dorp wrote:
I agree with Jacco here. We have to name the variable twice, even if it is only used once, and we have a relatively long keyword, five characters, longer than average for all keywords, and only one char short of the maximum. I know the aim isn't to absolutely minimise the number of keystrokes, but it does seem strange to use such a long symbol which requires duplicating the target, unless the intent is to discourage people from using it. -- Steve
On Fri, May 11, 2018 at 11:43 AM Steven D'Aprano <steve@pearwood.info> wrote:
To be fair when counting the keystrokes, you should take into account that the colon and the parentheses that appear in the := syntax are the upper register keys that counting the shift require two keystrokes each.
On 11May2018 11:40, Jacco van Dorp <j.van.dorp@deonet.nl> wrote:
Yeah, this is my favorite also. Has the same feel and ordering as with, import and except. Could someone point me to a post which nicely describes the rationale behind its rejection? I'm sure there's one in the many in this discussion but I've not found it yet. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sun, May 13, 2018 at 2:05 PM, Cameron Simpson <cs@cskk.id.au> wrote:
https://www.python.org/dev/peps/pep-0572/#special-casing-conditional-stateme... https://www.python.org/dev/peps/pep-0572/#alternative-spellings I'm not sure which version you're looking at, so there's the rejections of both. ChrisA
On 13May2018 14:23, Chris Angelico <rosuav@gmail.com> wrote:
I meant the latter, but I'd already looked at that part of the PEP and found its explaination... unfulfilling. It says: EXPR as NAME: stuff = [[f(x) as y, x/y] for x in range(5)] Since EXPR as NAME already has meaning in except and with statements (with different semantics), this would create unnecessary confusion or require special-casing (eg to forbid assignment within the headers of these statements). All you need to disambiguate, say: with expr as expr_as as with_as: is to require parentheses for (expr as exp_as) if someone wanted that complications (assuming that is even necessary - it seems unambiguous to my eye already, unless the token lookahead requirement in Python's grammar prevents that. So I'd hoped for a post to the list discussing this aspect and outlining why it was considered unsatisfactory. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sun, May 13, 2018 at 2:58 PM, Cameron Simpson <cs@cskk.id.au> wrote:
There were a large number of posts, so I can't really point to one of them. The problem isn't the double-as case that you describe; it's that these two are ALMOST identical: with expr as target: with (expr as target): In fact, they are functionally identical for many situations - "with (open(...) as target):" is exactly the same as the form without the parens. It'd make for data-dependent bugs, which are a really REALLY bad idea. ChrisA
On 5/13/18 1:05 AM, Chris Angelico wrote:
A little more detail: The first one is partially: target = expr.__enter__() The second one is partially: target = expr For many objects, obj.__enter__() just returns obj, it often looks like these two statements do the same thing, but they do not. Chris's concern, which I share, is that users wouldn't know why these are different, and why the second on works for some objects but not others. I agree the PEP could use more detail in explaining this particular issue. Eric
On 13May2018 06:45, Eric V. Smith <eric@trueblade.com> wrote:
Ah, thank you! Technically I knew this; in practice I'd forgotten and was thus vulnerable in exactly the same way as you anticipate.
Chris's concern, which I share, is that users wouldn't know why these are different, and why the second on works for some objects but not others.
While I can see the issue, on a personal basis I'd accept it: import and except don't do this and they happily use "as". In my mind the problem lies with "with" (and is a perfectly acceptable inconsistency there, given what with does for us). I agree the change in semantics is not obvious. But we've got a similar thing with tuples as well: x = (1) x = (1,) with a different spelling.
I agree the PEP could use more detail in explaining this particular issue.
Yes. This issue certainly isn't clear to me from the PEP's wording. I'm no longer trying to push the "as" agenda here, BTW. I still like it, but that ship seems sailed. Cheers, Cameron Simpson <cs@cskk.id.au>
On Mon, May 14, 2018 at 7:00 AM, Cameron Simpson <cs@cskk.id.au> wrote:
The same problem happens with 'except', only less subtly. except Exception as e: # binds the caught exception to e except (Exception as e): # would bind the type Exception import doesn't put an expression on the left of 'as', so it's less likely to cause confusion; but all three of them do something special before binding to the target given with 'as'. By the way: do you know which of the three support arbitrary assignment targets and which support only names? No? Neither did I, till I checked the grammar. So there's no consistency there, other than a loose sense that "as" means "we're gonna toss something into somewhere". Not nearly enough to justify using that syntax for arbitrary name bindings, given how much hassle there is with 'with'. ChrisA
On Thu, May 10, 2018 at 9:44 AM, Guido van Rossum <guido@python.org> wrote:
I'm sorry, but unless there's a sudden landslide of support for 'given' in favor of ':=', I'm really not going to consider it.
How much support was there for ":="? Are you serious about bringing back Pascal and Algol from their comfortable resting places?
Probably going to completely lose this, but would it be possible to have a vote? +1 for either 'given' and/or ':='? On Thu, May 10, 2018 at 2:48 PM Guido van Rossum <guido@python.org> wrote:
-- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else https://refi64.com/
Another benefit of given compared with := that I just thought of is this. Suppose you have a generator like (expression(f(x), y, z) for x in xs for y in ys(x) for z in zs(y)) With given notation you can optimize: (expression(f_x, y, z) for x in xs given f_x = f(x) for y in ys(x) for z in zs(y)) whereas with :=, you can't. Best, Neil On Sat, May 12, 2018 at 10:56 PM David Mertz <mertz@gnosis.cx> wrote:
On Sat, May 12, 2018 at 11:04:45PM -0400, Neil Girdhar wrote:
Is that legal syntax? You're splitting the "given" expression by sticking the for clause in the middle of it. I don't think that will be legal. It would be trying to split an ternary if: (true_expr for x in xs if condition else false_expr for y in ys) (true_expr if condition for x in xs else false_expr for y in ys) But whether legal or not, Neil, you went on at length about how professionals don't write code like this and such overly dense comprehensions are only fit for competitions. Now you want your cake and to eat it too: "given is better, because it doesn't allow the awful unreadable code that := gives; oh, and it's also better, because it allows *this* awful unreadable code that := doesn't allow" In any case, of course you can write this with := binding expression. You just shouldn't do it: (expression(f_x, y, z) for x in xs for y in ys(x) for z in zs(y) if (f_x := f(x)) or True) ) That's fine for mucking about, but I wouldn't do it for serious code. Replacing the colon with "given f_x" doesn't change that. -- Steve
On Sat, May 12, 2018 at 11:48 PM Steven D'Aprano <steve@pearwood.info> wrote:
You're right that it's a different proposal, but I thought it would be a natural extension to this proposal since at the start of this discussion someone mentioned how this could be accomplished with something like (expression for x in xs for f_x in [f(x)]) The regular given proposal is an extension of (I think) "expr" into something like expr ["given" test annassign] I think a natural extension of that is to add it to "testlist_comp" and "dictorsetmaker" so that given name = value can be used a cleaner synonym of something like for name in [value] But your'e right, it's a different proposal and I'm getting ahead of things. But whether legal or not, Neil, you went on at length about how
My thought is that if each component is simple enough *and* if each component can be reasoned about independently of the others, then it's fine. The issue I had with the := code you presented was that it was impossible to reason about the components independently from the whole. Ultimately, this is one of feelings. I think every one of us has a unique set of experiences, and those experiences led us to having different programming aesthetics. I used to write code one way, and then after lots of code reviews, my experience has made me write code a different way. Best, Neil
On Sun, May 13, 2018 at 12:10:08AM -0400, Neil Girdhar wrote:
Actually, as you admitted, you can't, since this isn't part of the proposed syntax or semantics. But let's put that aside.
Except of course you can:
[Neil]
I said it was rubbish code which I wouldn't use for serious work. But "impossible" to reason about? That's pretty strong, and definite, words for something which is actually pretty easy to reason about. (expression(f_x, y, z) for x in xs for y in ys(x) for z in zs(y) if (f_x := f(x)) or True) ) I'm pretty sure you can reason about "expression(f_x, y, z)" on its own. After all, the above code was (apart from the := line) your own example. If you can't even reason about your own examples, you're in a bad place. I think you can reason about the three for-loops "for x in xs" etc on their own. Again, they were your idea in the first place. I expect you can reason about "if <clause> or True" on its own. It's a pretty basic Python technique, to call <clause> for its side-effect without caring about its return value. Not something I would use for serious work, but this is YOUR example, not mine, so if you hate this generator expression, you were the one who suggested it. Only the <clause> is new or different from ordinary Python code that works now: f_x := f(x) The semantics of := are pretty damn simple (at least from the high-level overview without worrying about precedence or possible scoping issues): - evaluate f(x) - assign that value to f_x - return the value of f_x which is EXACTLY THE SAME SEMANTICS OF "f_x given f_x = f(x)": - evaluate f(x) - assign that value to f_x - return the value of f_x And you would reason about them the same way. -- Steve
On Sun, May 13, 2018 at 10:56 AM Steven D'Aprano <steve@pearwood.info> wrote:
I was talking about this snippet you wrote: smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal] Not impossible to reason about, but not pretty either.
Not that it matters, but this doesn't work in general since this relies on f_x being evaluable to bool. You can always just write "for f_x in [f(x)]". This use of given was more of an aside.
target := expr expr as target expr -> target target given target = expr let target = expr : target expr ; Although in general "target:=exp" seems the most palatable of these to me, there is one nice benefit to the "given" syntax: Suppose you have a comprehension wherein you want to pass forward an internal "state" between iterations, but not return it as the output: In today's python, you'd to: outputs = [] state = initial_state for inp in inputs: out, state = my_update_func(state) outputs.append(state) This could not be neatly compacted into: state = initial_state outputs = [out given out, state = my_update_func(inp, state) for inp in inputs] Or maybe: outputs = [out given out, state = my_update_func(inp, state) for inp in inputs given state=initial_state] Though I agree for the much more common case of assigning a value inline "x given x=y" seems messily redundant. On Sat, May 12, 2018 at 10:37 PM, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
On 13.05.2018 11:23, Peter O'Connor wrote:
Question still stands if this type of code needs compaction in the first place? List comprehensions usually have some sort of declarative touch (set builder notation). Even though, striving for a more compacted version, I tend to think that using a declarative version of it doesn't serve it well in the long term. We recently came across the following code snippet in our source base (1st answer of https://stackoverflow.com/questions/480214/how-do-you-remove-duplicates-from...). It was absolutely not comprehensible. Your example is inherently imperative because the internal state changes from iteration to iteration; something unusual for set builder notation. Regards, Sven
2018-05-10 16:44 GMT+03:00 Guido van Rossum <guido@python.org>:
I think you do not quite objectively look at the current situation. Many just lost interest in attempts to move the topic at least a little bit in the other way, seeing how you and Tim so actively expresses support/protects this `:=` syntax, while ignoring or pushing out alternative opinions :-). Of course, the latter is partly due to the incredible number of different threads and messages on this topic. Briefly: Initially, the main argument in favor of `:=` was that this form is similar to the usual assignment statement, but can be used as an expression. Ok. Then everyone agreed with the idea that it's necessary to limit assignment target to name only. Although all this criticism was actually put forward in the first 50-100 messages on the topic. In the same first hundred it was actively discussed, that in fact, this idea gives a win only in `while` and `if` statemetns that probably will match 99%+ where it will be used for its intended purpose. At the same time, most of the criticism concerned exactly the use in generators and comprehenshions, they are already often overloaded for perception. And as you once said - "Language Design Is Not Just Solving Puzzles". There was also discussed the difference in perception between the `expr op name` and `name op expr`. Here the expression is something that is important, the name is only a convenient consequence. At the moment with all the constraints of `:=`, the discuscation is more like - trying to cram this syntax into the language. While for those who are familiar with Pascal, Icon and other languages that use this syntax, this - `:=` looks natural. For others and I believe such a majority among users, this syntax is, to put it mildly, not natural and ugly producing a line noise, the colon `:` symbol is already used in a lot of places. With all the changes, the limitations and magic with scopes. Is it now easier to explain all the differences between `=` and `:=`, than the difference between `if expr as name ...: ...` and `with expr as name:`? Therefore, I take Nick's insistent position as an attempt to at least somehow make an alternative look at this topic. With kind regards, -gdg
Kirill Balunov wrote:
As someone familiar with Pascal, I think the similarity to the Pascal assignment operator is actually an argument *against* it. Knowing what it means in Pascal is confusing, because Pascal's ":=" is equivalent to Python's "=" (it's strictly a statement, and can't be used in expressions). -- Greg
On 11.05.2018 09:38, Greg Ewing wrote:
Same here. It means something different. Also coding in Pascal was annoying from the beginning with its extremely verbose syntax like begin/end etc. So, ":=" also felt like "why the hell do we need a colon in front of the equal sign?" Absolutely unnecessary bloat, like almost everything in Pascal. Maybe that's also part, why I am -1 on the proposal. Who knows...
I love given compared with := mainly because Simpler is better than complex: * given breaks a complex statement into two simpler ones, which is putting people off in the simple examples shown here (some people are annoyed by the extra characters). However, when given is used in a list comprehension to prevent having to re-express it as a for loop, then two simple statements are easier to understand than one complex statement. This is a common difference between code written at programming contests versus code written by those same software engineers years later at big companies. Code that you write for yourself can be compact because you already understand it, but code you write professionally is read many many more times than it is written. Accessibility is much more important than concision. * Python has a reputation for being working pseudocode, and given reads like pseudocode. := needs to be deciphered by comparison especially in the complicated cases where multiple := operators are used on one line. * there are no difficult mental questions about evaluation order, e.g., in a bracketed expression having multiple assignments. Similarly, instead of (a.b(a) given a = c.d()) do I write (a.b(a := c.d())) or ((a := c.d()).b(a)) ? * it avoids the question of what happens when := is used in a switch: (a if (b := c) else d) Sometimes you want the assignment to happen unconditionally (a if (b:=c) else d) + b; sometimes you don't. How do you force one case or the other? given makes it obvious by separating assignment from the usage of its assignment target. Style: * it avoids the big style question of when to use and when not to use :=. (Even if you ask people not to, people are going to write the expression-statement a := b as a synonym for the statement a = b.) * it looks a lot like the existing Python "for" and "if" clauses, which also do in-expression assignments. This makes formatting the code obvious too: (expression given a = b) compared with expresion ( a := b ) rest of expression which quickly gets ugly. Best, Neil On Thursday, May 10, 2018 at 9:46:01 AM UTC-4, Guido van Rossum wrote:
On Sat, May 12, 2018 at 08:16:07AM -0700, Neil Girdhar wrote:
(Foreshadowing: this argument applies to augmented assignment. See below.) I don't see how you justify that statement about "given". I think that it is "given" which is more complex. Significantly so. Let's compare the syntax: target := expr That is a single, simple expression with a single side-effect: it assigns the value to <target>. That's it. Like all expressions, it returns a value, namely the result of "expr". Like all expressions, you can embed it in other expressions (possibly wrapping it in parens to avoid precedence issues), or not, as required. (That surrounding expression can be as simple or complex as you like.) Now here's Nick's syntax: target given target = expr Exactly like the := version above, we can say that this is a single expression with a single side-effect. Like all expressions, it returns a value, namely the result of "expr", and like all expressions, you can embed it in other expressions. So far the two are precisely the same. There is no difference in the complexity, because they are exactly the same except for the redundant and verbose "given" spelling. But actually, I lied. Nick's syntax is *much more complicated* than the := syntax. Any arbitrary expression can appear on the left of "given". It need not even involve the binding target! So to make a fair comparison, I ought to compare: target := expr which evaluates a single expression, binds it, and returns it, to: another_expr given target := expr which evaluates "expr", binds it to "target", evaluates a SECOND unrelated expression, and returns that. If you want to argue that this is more useful, then fine, say so. But to say that it is *simpler* makes no sense to me. Option 1: evaluate and bind a single expression Option 2: exactly the same as Option 1, and then evaluate a second expression How do you justify that Option 2 "given", which does everything := does PLUS MORE, is simpler than Option 1? That's not a rhetorical question.
I'd like to see an example of one of these list comprehensions that is simpler written with given. Here's an earlier example, an exponentially weighted running average: average = 0 smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal] assert average == smooth_signal[-1] I'm not even sure if "given" will support this. Nick is arguing strongly that bound targets should be local to the comprehension, and so I think you can't even write this example at all with Nick's scoping rule. But let's assume that the scoping rule is the same as the above. In that case, I make it: average = 0 smooth_signal = [average given average = (1-decay)*average + decay*x) for x in signal] Is it longer, requiring more typing? Absolutely. Does it contain a redundantly repetitious duplication of the repeated target name? Certainly. But is it simpler? I don't think so. If you don't like the exponential running average, here's a simple running total: total = 0 running_totals = [total := total + x for x in xs] versus total = 0 running_totals = [total given total = total + x for x in xs] If you don't like this example either, please show me an example of an actual list comp that given makes simpler.
Ah, nice rhetorical argument: "given" is for professionals, := is a hack for amateurs and programming contests. Seriously? Do you use augmented assignment? Your simple versus complex argument for "given" applies well to augmented assignment. Augmented assignment combines two conceptual operations: x = x + 1 - addition (for example) - assignment into a single operator: x += 1 By your argument, augmented assignment is more complex, and we ought to prefer splitting it into two separate operations x = x + 1 because that's simpler. I think I agree that x = x + 1 *is* simpler. We can understand it by understanding the two parts separately: x+1, followed by assignment. Whereas += requires us to understand that the syntax not only calls a dunder method __iadd__ (or __add__ if that doesn't exist), which potentially can operate in place, but it also does an assignment, all in one conceptual operation. There's a whole lot of extra complexity there. That doesn't mean I agree with your conclusion that we ought to prefer the simpler version, let alone that the complex case (augmented assignment) is fit only for programming contests and that professionals ought to choose the simpler one. -- Steve
Just clarifying a fine point here: [Steven D'Aprano <steve@pearwood.info>]
You can't under Nick's proposal(s), at least not directly (there are always "tricks"). But it also blows up with UnboundLocalError (for the "average" in "(1-decay)*average") under the current PEP 572 (the ":=" PEP). I've proposed to change 572's scoping rules for targets of assignment expressions appearing in comprehensions so that "it would just work" instead, but that's getting strong opposition too. My favorite so far was Nick's (Coghlan's) entertainingly hyperbolic: "Comprehension scopes are already confusing, so it's OK to dial their weirdness all the way up to 11" is an *incredibly* strange argument to be attempting :-) The scope issues are logically independent of assignment-expression spelling, but it's a pretty safe guess Nick is opposed to that example ever "just working" regardless of spelling, while PEP 572 doesn't currently support it anyway. Last I heard, Chris (Angelico - the PEP's author) didn't seem keen on changing it either.
On Sat, May 12, 2018 at 01:13:05PM -0500, Tim Peters wrote:
Yes, but I've sort of assumed that if PEP 572 has even a microscopic chance of being accepted, it will have to be changed, given that Guido has already stated that the only behaviour that makes sense is what you and I have been suggesting. Unless Guido has changed his mind, the relevant links are: https://mail.python.org/pipermail/python-ideas/2018-May/050411.html https://mail.python.org/pipermail/python-ideas/2018-May/050456.html So maybe this is a tiny bit naughty (or a lot...) but I've just been ignoring what the PEP currently says and going by what it ought to say :-) -- Steve
On 12 May 2018 at 14:13, Tim Peters <tim.peters@gmail.com> wrote:
I'm personally fine with that example working if there's an explicit nonlocal declaration on "average" in the nested scope - it's Guido that objected to requiring the explicit scoping declaration to access that behaviour. For the implicit version, my request is that any PEP proposing the idea of parent local scoping be held to the standard of *actually drafting the patch for the language specification*, rather than handwaving away the hard problems that it creates (i.e. what to do at class scope, what to do when multiple generators expressions reference the same nonlocal name, what to do with nested comprehensions, how to expand comprehensions using this kind of scoping to their statement form in a context independent way). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 11 May 2018 at 12:45, Tim Peters <tim.peters@gmail.com> wrote:
I was one of those core devs, and would personally prefer to require that folks spell the inline binding completely unambiguously as "if i given i = 1:". However, if the repetition of "i" is considered a deal breaker relative to ":=" (even though the status quo already requires repetition of the target name in the condition), then I'd prefer to add this shorthand (which folks can then opt to prohibit in favour of the more explicit form in their style guides) over adding the cognitive complexity of deciding when to use "i = 1" and when to use "i := 1". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Just noting some real code I typed today where `given` works great if it allows unpacking syntax, and assignment expressions don't: while True: head, matched, s = s.partition(sep) if not matched: break Using `given`: while matched given head, matched, s = s.partition(sep): Typing "matched " twice still sucks, though;-) It doesn't work as well with assignment expressions even if they were (re)generalized to allow unpacking syntax. In this specific case, I'd keep the original loop-and-a-half rather than do: while (t := s.partition(sep})[1]: head, matched, s = t With unpacking syntax restored, same answer: while (head, matched, s := s.partition(sep})[1]: or while [(head, matched, s := s.partition(sep}), matched][-1]: to combine the worst annoyances of everything and add another ;-)
[Tim]
[MRAB <python@mrabarnett.plus.com>]
If you're using .partition multiple times, you might as well use .split instead!
Possibly - depends on what the rest of the loop is doing. In this specific case, there are paths in the loop body that change what `sep` is bound to, so I'd have to pass `maxsplit=1` to get a clumsier way to do what `partition()` does directly, For example,
I'ts more convenient that `partition()` always returns a 3-tuple, not sometimes a 1-list and other times a 2-list.
[Steven D'Aprano <steve@pearwood.info>]
[Tim]
[Nick Coghlan <ncoghlan@gmail.com>]
I'm personally fine with that example working
"Just working" meant "exactly as written". "Regardless of spelling" meant whether binding is spelled via ":=", "given", "as", "where", "let ... in ...' "->", ..
I suspect, but don't know, that Guido would like that example to "just work" because it _looks like_ it should "just work". There's no visible function involved, and _needing_ to add scope declarations to make it work is pretty much inexplicable unless the user first learns more than most users "should" need to learn about how it's implemented. If so, no technical argument will change his mind - and especially not one based on "but the implementation today doesn't do that already". Recall that Python had no lexically nested scoping at first? If he wanted to _make_ users learn about nested lexical scopes to use Python features that don't _appear_ to use it, Python would have had it from the start ;-) I'd be happy enough with needing an explicit declaration too, but am _happiest_ with what I'm guessing Guido's view is.
Of course.
rather than handwaving away the hard problems that it creates (i.e. what to do at class scope,
I finally looked at class scope and quickly decided it makes no sense there (a comprehension at class scope has no access to the class scope, so "same scope in the comprehension as in its immediately containing block" is incoherent in that case).
what to do when multiple generators expressions reference the same nonlocal name,
Then, as you said, they access the same nonlocal name. What of it? What if two generator functions reference (or even rebind) the same nonlocal name? What's unclear about that? If you were happy with explicit scope declarations, then exactly the same thing would happen as if the user were forced to explicitly declare the scopes chosen for them.
what to do with nested comprehensions,
Again, if you were happy with explicit scope declarations, then exactly the same ... Where are the docs explaining how nested comprehensions work today? I haven't seen any. If any such exist, I'd bet nothing about them needs to be changed. If none such exist, I don't see a need to write any just for this PEP. How do nested expressions of any kind work? Same thing. The only thing the suggestion changes is the scope of assignment expression targets in synthetic functions created to implement comprehensions. That has nothing at all to do with the possibility of nesting, or with the structure of nesting. Why do you think it does - or might?
how to expand comprehensions using this kind of scoping to their statement form in a context independent way).
I've already explained why I view that as a non-issue (making a tedious manual process a relative handful of users undertake once in their life for self-education purposes slightly less tedious has approximately no value to me - and, to the contrary, the "context dependent" bits they may have to learn would make it _more_ educational). If that's a show-stopper for you, so be it.
Can someone explain to me why it was considered a bad thing that for-clauses leaked names in comprehensions, but it will be a good thing for inline assignments to leak names from them? -- Greg
[Greg Ewing <greg.ewing@canterbury.ac.nz>']
Can someone explain to me why it was considered a bad thing that for-clauses leaked names in comprehensions,
Because you can't write a list comprehension or generator expression AT ALL without specifying a `for` loop header, so whether its target(s) leaks is an issue in virtually every listcomp/genexp ever written, and that people tend to use short (typically 1-letter) names for for-targets, so unintentional stomping on names is exceptionally likely in this context.
but it will be a good thing for inline assignments to leak names from them?
Because you never _need_ to use an assignment expression to write a listcomp/genexp. You have to go out of your way to use it. Which will probably be rare in listcomps;genexps,, not virtually 100% of the time as with for-targets. Then you get what you went out of your way to explicitly ask for: a name for some subexpression result. Otherwise it's essentially impossible to explain why: total = 0 sums = [total := total + value for value in data] assert sums[-1] == total "blows up", despite that its intent is obvious, unless you first explain to a user how the listcomp is implemented via an invisible synthetic function created by magic, inside of which `total` has nothing to do with the `total` they see on the first line. UnboundLocalError - WTF? That's why leaking "is good". It works both directions: the outer name leaks _into_ the body too, not just _out_ of it. Things that "look like" they should obviously work do work then, and a user can remain blissfully unaware of the implementation.. Of course you can also find cases in which it's not wanted. If it doesn't leak the kind of use shown above can't be done at all via listcomps (at least not straightforwardly). If it does leak, the subset of cases where leaking is unwanted _of_ the subset of cases in which a listcomp//genexp uses an assignment expression at all are indeed inconvenienced. So - surprise! It's a tradeoff, something we've never faced before ;-)
I think we're approaching this from the wrong direction. My point is, expression assignments dont have the complex case as purpose - most coders wont try to maximize line information density. If you're doing magic, you might as well spell it out over multiple lines, because neither := nor given will be readable. However, in the simple case, it does matter a lot - and there := beats out given by a mile. If you're breaking your lines to avoid line length violations, you might as well put your assignments on a separate lines first. Im inclined to argue that if assignment expressions of any form forces to you make a multi-line statement, you're doing it wrong. (in the vast majority of cases). Consider it like the ternary operator - good to compact simple constructs, but while you can use it for black magic, you probably shouldn't. How about we just explicitly advice to keep it simple, stupid, and update PEP 8 to state that if assignment expressions take you to multi-line, split of the assignment and use statements instead. (truth be told, the general while (assignment expression isn't as clear cut as if-uses, but I think the idea holds.)) Jacco
Tim Peters wrote:
Because you never _need_ to use an assignment expression to write a listcomp/genexp.
This whole discussion started because someone wanted a way to bind a temporary result for use *within* a comprehension. Those use cases don't require leakage.
It's no harder to explain that than it is to explain why x = 42 y = [x * x for x in range(5)] print(x) prints 42 rather than whatever value was last bound to the x in the comprehension. Seems to me it would be easier to explain that *all* names bound within a comprehension are local to the comprehension, than to have to say that some are and some aren't. -- Greg
[Greg Ewing <greg.ewing@canterbury.ac.nz>']
This whole discussion started because someone wanted a way to bind a temporary result for use *within* a comprehension.
It's been noted several times recently that the example PEP 572 gives as _not_ working: total = 0 progressive_sums = [total := total + value for value in data] was the original use case that prompted work on the PEP. You gotta admit that's ironic ;-)
Those use cases don't require leakage.
I already said that - yes - some use cases don't want the leakage. Others do. And the vast majority of listcomps/genexps will likely never use an assignment expression anyway. If "no leak" wins, "want leak" can't do what they want at all. If "want leak" wins, the subset-of-a-subset "no leak" cases are inconvenienced. As I said, it's a tradeoff. You're giving 0 weight to one of the sides.
You're overlooking the most relevant point: The example blows with an UnboundLocalError, which can't possibly be explained by looking at the example as it stands, or by reference to shallow tricks. You _need_ to drag in stuff about invisible (in the code) synthesized scopes in which `total` is in fact an unbound local name. You don't need that heavy machinery to explain why for-target names don't leak; indeed, shallow implementation tricks were used to achieve that well before synthetic functions _were_ used to implement listcomps.
for-targets are local, assignment expression targets aren't. I agree that's harder to explain, but on a scale of 1 to 10? 1. It's a tradeoff. And I cheerfully just gave 1 point to the side you favor. By my accounting, "leak" is still wining by about 136 to 17 ;-)
On 14 May 2018 at 06:10, Tim Peters <tim.peters@gmail.com> wrote:
After pondering this case further, I think it's also worth noting that that *particular* example could also be addressed by: 1. Allowing augmented assignment *expressions* 2. Changing the scoping rules for augmented assignment operations in general such that they *don't change the scope of the referenced name* Writing "i += n" without first declaring the scope of "i" with "i = 0", "nonlocal i" or "global i" is one of the most common sources of UnboundLocalError after all, so I'd be surprised to find anyone that considered the current augmented assignment scoping rules to be outside the realm of reconsideration. The accumulation example would then be written: total = 0 progressive_sums = [total += value for value in data] if progressive_sums: assert total == progressive_sums[-1] The question would then turn to "What if you just want to bind the target name, without considering the old value?". And then *that's* where "NAME : = EXPR" would come in: as an augmented assignment operator that used augmented assignment scoping semantics, rather than regular local name binding semantics. That would mean *directly* overturning PEP 3099's rejection of the idea of using "NAME := EXPR" to imply "nonlocal NAME" at function scope, but that's effectively on the table for implicit functions anyway (and I'd prefer to have ":=" be consistent everywhere, rather than having to special case the implicit scopes). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
[Tim]
[Nick]
Yes, it's worth considering. In my experience, I don't believe I've ever had an UnboundLocalError for which a correct fix was to add `nonlocal`. Usually the correct fix was to add `global`, but that's mostly due to an old habit of using piles of globals to count trips through various code paths, used by a watchdog thread that periodically wakes up to display (the global) counts.
And the divide-out-small-primes example could be factor = -42 while any(n % (factor += p - factor) == 0 for p in small_primes): n //= factor Heh ;-)
Plain old ":=" would somehow be viewed as being an augmented assignment operator too? ... OK, the meaning is that augmented assignment _and_ "::=" would resolve the target's scope in the way the containing block resolves it.
Creating a local in a containing scope by magic is never done by Python today. Extending that beyond "need" seems potentially perilous. For example, it can already be tedious to figure out which names _are_ local to a function by staring at the function's code, but people quickly get better at that over time; change the rules so that they _also_ have to stare at all immediately contained functions too to figure it out, and it may become significantly harder (OK, I didn't declare `x`, and a contained function did `x := 3.14` but `x` isn't declared there either - I guess it's my `x` now). Then again, if they're doing that much function nesting they deserve whatever they get ;-) Restrict it to that only synthetically generated functions can pull off this trick by magic (where there are real use cases to motivate it), and they still don't have to look outside the body of a function's text to figure it out. Visually, there's no distinction between the code running in the function's scope and in scopes synthesized to implement comprehensions appearing in the function's text. The comprehensions aren't even indented more. So, offhand, I'm not sure that the right way to address something you view as a wart is to vastly expand its reach to 12 operators that impose it on everyone everywhere every time they're used ;-) Seriously, I do suspect that in def f(...): ... no instances of `s` ... s += f"START {time.time():.2f}" it's overwhelmingly more likely that they simply forgot to do s = "" earlier in `f` than they actually wanted to append to whatever `s` means in f's parent block.. That's a radical change to what people have come to expect `NAME +=` to do. OTOH, I don't (yet?) see a way the change could break code that currently works, so it remains worth thinking about. BTW, would def f(): x := 3.14 x = 3.14 be a compile-time error? Everyone agreed the analogous case would be in synthetic functions. Fine by me!
On 15 May 2018 at 01:53, Tim Peters <tim.peters@gmail.com> wrote:
More likely they'd get a compile time error complaining that the compiler couldn't figure out what they meant, and asking them to be clearer about the intended scoping. Restrict it to that only synthetically generated functions can pull
Once I reframed the idea as being like an augmented assignment, your proposed semantics seemed a lot less magical to me, since I was able to define them in terms of "find the assignment or declaration that already exists", rather than implicitly creating a new one. If the compiler can't find a suitable target scope, then it can throw AmbiguousTargetError (which would be an improvement over the runtime UnboundLocalError you typically get today).
I think this is the key argument in favour of only allowing the "implicitly nonlocal rebinding" behaviour in lambda expressions, generator expressions, and comprehensions, as that way the search for a target to bind would always terminate at the containing block (just as it does today). BTW, would
Yeah, I think that would be an AmbiguousTargetError, as when the compiler saw "x := 3.14", it wouldn't have seen "x = 3.14" yet. For other augmented assignments, it would be a DeprecationWarning for the time being, and become an AmbiguousTargetError at a later date. (This also relates to the previous point: if "x := 3.14" can be implicitly nonlocal, then I couldn't answer that question without knowing which names were defined in outer scopes. By contrast, if the implicit access to outer scopes is limited to inline scopes accessing their containing scope, then this example becomes precisely analagous to the current syntax error for binding a name as a local before declaring it as global or nonlocal. The statement of ambiguity would arise from the fact that when we see "TARGET := EXPR" at statement level, we don't know if the missing prior statement is a local variable assignment, a type declaration, or a global or nonlocal declaration) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
total = 0 progressive_sums = [total := total + value for value in data]
[Greg Ewing <greg.ewing@canterbury.ac.nz>]
I'm skeptical that it's a good idea to encourage this kind of thing in the first place.
Adding a feature is creating possibility for its use, not encouraging its use. I'd hate to, ,e.g., see people encouraging use of ternary-if either, _Sometimes_ ternary-if is exactly right, both shortening code and increasing clarity. But not often. Ditto for metaclasses, the ability to nest blocks 20 levels deep, on & on. That they can be abused, and sometimes are abused, is a very weak argument against them. On a scale of 1 to 1000000? Maybe 23 ;-) The specific use case that grabbed Guido's attention on this one wasn't the above, but: while any(n % p == 0 for p in small_primes): // divide p out of n - but which p succeeded? _General_ features turn out to have multiple good uses, not all of which are anticipated. I think this is an example of that. It's easy to miss that the argument to `any()` runs in an entirely different scope, so that its local `p` vanishes when `any()` completes. It's not proposed to change that either, because people overwhelmingly seem to want `for` targets not to leak. Fine by me. Under the proposal, it's straightforward to use an assignment expression to "export" p's last materialized value: while any(n % (factor := p) == 0 for p in small_primes): n //= factor To my eyes, the most surprising thing about that is why it's necessary to "export" p's winning value to begin with. "Because for-targets always vanish, so there's another form of explicit binding that doesn't" is a reasonably satisfying explanation. Nick would be happiest, and I'd be "happy enough", if all forms of binding remained "local" and instead we fiddled the syntax to allow explicitly declaring the desired scope. Then, e.g., while any(<nonlocal p> n % p == 0 for p in small_primes): n //= p wouldn't need an assignment expression at all. But Guido doesn't like that. I think he wants to hide, so far as humanly possible, that listcomps and genexps happen to implemented now by synthesizing functions.
On Mon, May 14, 2018 at 09:17:03PM +1200, Greg Ewing wrote:
To reiterate what Tim already pointed out, that original usecase required a way to feed values *into* the comprehension. https://mail.python.org/pipermail/python-ideas/2018-February/048971.html There's no need for dedicated syntax for that if we can just set an variable and have it show up in the comprehension. [...]
But it *is* harder to explain why comprehensions are their own scope in the first place. They don't look like they are in their own scope. They look like any other expression. No other expression (apart from lambda) runs in its own scope. Every other scope relates to a clear lexical separation: - modules - classes - functions except for comprehensions, which just returns a plain old list. (Generator expressions are a little fuzzier: they at least are equivalent to a lambda-with-yield, if such a thing existed.) It is sometimes useful to be able to reach into a generator expression and manipulate a variable between calls. That would make it a coroutine (to use the pre-async language), and that's why yield is an expression that returns a value, not just a statement. And it is sometimes useful to be able to see the value of comprehension variables after they have finished running. We have given up all of that to allow ease of implementation to drive the semantics, and that's okay. Its a trade-off. But we can decide the other way too, and choose more useful semantics for assignment expressions over an easier implementation.
Of course it would be easier to explain. It wouldn't be as useful. If we wanted "easy to explain", we'd be using BASIC circa 1974, we wouldn't have async, generators, comprehensions, exceptions, decorators, classes, metaclasses, descriptors, Unicode, or floating point numbers. -- Steve
Steven D'Aprano wrote:
Yes, but doing it that way also allows things to come *out* of the comprehension, which is something we previously thought was such a bad idea that we went to considerable lengths to stop it happening. Now we seem to be seriously considering going to further lengths to allow it again for *some* things. Seems to me we should have a pretty compelling reason for doing that. I don't feel very compelled by what I've seen so far. -- Greg
On 14/05/18 10:17, Greg Ewing wrote:
I still don't find that argument compelling. If you have a sufficiently complicated comprehension that you need a temporary result bound, IMHO you are well on the way to (if not far past) the point where converting it into a more traditional for-loop will be much clearer. I'm not sure we should be encouraging people to write less clear code. -- Rhodri James *-* Kynesim Ltd
Minor gripe: The ability to *articulate* Python is not the same as the ability to *type Python verbally*. Nobody articulates `def area(width, height): return width * height` as *def area, open paren, width, comma, space, height, closed paren..*. They would say something like *def area as a function of width and height, equal to width by height*. -- Carl Smith carl.input@gmail.com On 12 May 2018 at 18:24, Steven D'Aprano <steve@pearwood.info> wrote:
On 2018-05-12 18:24, Steven D'Aprano wrote:
Your interpretation of the argument you quote, as I understand it, is that "`given` is simpler than `:=`". I interpreted it as, and would argue myself, more like "`given` makes *the code that uses it* simpler". The reason the `given` syntax might be simpler has nothing to do with the number of syntactic elements (brainfuck is down the hall...). Rather, it is that `given` separates things: if m.group(2) in words given m = word_re.match(s): versus: if (m := word_re.match(s)).group(2) in words: In the `:=` version, the assignment is embedded in the expression. It's different. Most of the time it will save at least a few characters. But it's not obviously--and certainly not objectively--simpler. One somewhat concrete difference I can think of is that in expressions that refer to the result multiple times, the `:=` assignment must always be positioned such that it is the first to be evaluated, moving around should that ever change. In cases where evaluation depends on runtime information (if-else, and, or), I'm not actually sure what you would do. In general, I feel like you're focusing on the simplicity or otherwise of the operator itself, rather than whether it has a simplifying effect on code that uses it. As you say in part 2 (which arrived while I was still taking forever to write this; sorry if it reads a bit confused), we'll learn the syntax, eventually. What's not so explicitly spelled out is that we'll be reading new code that uses it forever.
average = 0 smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal] assert average == smooth_signal[-1]
Like, holy smoke, man. Sure, `:=` is probably better for cramming side-effects into list comprehensions. Please don't cram side-effects into list comprehensions.
What is it with the cramming side-effects into list comprehensions‽ Do you realise itertools.accumulate exists? See the OP: On 2018-05-04 13:06, Nick Coghlan wrote:
I don't have a more concrete example of this to hand, mostly because I can't do this today, and faking it is hard enough (I didn't even know about the `for value in [stuff]` hack until this thread) that I just write for loops. On 2018-05-12 18:24, Steven D'Aprano wrote:
While I'm not sorry that I can do `x += 1`, it does have a substantial cost--one that is perhaps justified by the fact that it's *not* equivalent to `x = x + 1`, and that the possibilities it opens up are both useful and easy to understand. It's not simpler in and of itself, but that's not really the issue. On 2018-05-12 19:27, Steven D'Aprano wrote:
I believe that that's missing the point: to wit, in x.method(y) is `x` or `y` evaluated first? I didn't know. I don't think the order is documented anywhere or guaranteed not to change. I don't know what other languages do in general, though I know in C it's explicitly unspecified. So, sure, you can do this fine with `:=`. But it forces your code to depend on what I'd regard as a subtlety of the implementation.
On Sun, May 13, 2018 at 7:33 AM, Ed Kellett <e+python-ideas@kellett.im> wrote:
It's documented. It's guaranteed not to change. https://docs.python.org/3/reference/expressions.html#evaluation-order ChrisA
On 2018-05-12 22:42, Chris Angelico wrote:
It's documented. It's guaranteed not to change.
https://docs.python.org/3/reference/expressions.html#evaluation-order
Thanks. That's good to know, though I continue to hope nobody makes a habit of writing code that depends on it.
On Sat, May 12, 2018 at 1:25 PM Steven D'Aprano <steve@pearwood.info> wrote:
It's simpler in the sense that each of the components is simpler, and that the components exhibit *separation of concerns*. The bindings of the expressions are separate from calculation of the returned expression. The separation is both conceptual since the bindings are all distinct pieces of the whole expression, and temporal since the bindings all happen in some order, and must happen first. It can be very hard to get the ordering right. What if you are implementing something using the visitor pattern and you have something like: (visitor.access(database, key=database.key) given database = kwargs.pop("database") given visitor = database.default_visitor()) The beauty of given is that in this pattern: EXPRESSION given A = B given C = D you can completely forget about * B and D when thinking about EXPRESSION, * D and EXPRESSION when thinking about B, and * B and EXPRESSION when thinking about C. That's the separation of concerns. Now with :=, it's tricky because you can't set the visitor without first setting the database, but the visitor shows up first in the expression! Even if you could do something like: (visitor := database.default_visitor()).access((database := kwargs.pop("database")), key=database.key) I think this is much more complicated because all of the pieces are in the same giant expression, and so the concerns aren't separate. It's not obvious how to temporally order the pieces. And it takes a couple minutes to work out which parts of the expression fit with which other parts of the expression. If anyone ever writes code like that, you can guarantee that the Google style guide will preclude := except for maybe its the simplest uses.
Back when I was on the ACM team, people wrote code like that. It's fine if you're a brilliant contestant who has a few hours to solve a bunch of problems on one computer. It makes sense to optimize for number of characters, and anyway, ideally you were the only one reading your code unless your team had to help you debug it. However, and I'm sorry to be so bold, but this code will never pass a code review at a big company. It may have been easy for you to write, but it is not easy to understand at all. You're using the binding of := to update a a value that's used in subsequent iterations of the loop. That's way too complicated for a one-liner. You can write code like this for yourself, but no one else should have to decipher that. This is what I meant when I mentioned the difference between code at programming contests versus code written by those same software engineers years later at big companies. In my professional career (most recently at Google), the main goal was writing clear, accessible code so that the next person looking at it (or even me months later) would not have to think about it at all. It is immediately obvious that simple code is right. Even if you're brilliant enough to understand the code above, it needs to be immediately obviously correct to next person looking at it. Compare it with the for loop: def leaky_integral(signals, decay): value = 0.0 for x in signals: value = value * decay + x * (1 - decay) yield value
Nick is right in my opinion to argue this. If you want the bound target to leak out, just write the for loop. Otherwise you're asking too much of the reader.
I showed one above where := isn't even possible.
Sorry, I didn't mean to offend you. I fleshed out what I meant in this email. Two separate parts of my life as a programmer were my contest life and my professional career, and each had different ideals.
I don't agree with that. += is simpler since in that case the number of elements is fewer. The reason I'm saying that := is not simpler is that it tangles together unrelated subelements, which I explained in my separation of concerns paragraphs above.
Part 2. On Sat, May 12, 2018 at 08:16:07AM -0700, Neil Girdhar wrote:
I love given compared with := mainly because
[...]
Until you learn and become familiar with a new syntax, there is generally going to be a period you have to decipher it. I spent a long time mentally translating list comprehensions into mathematical set builder notation before it became second nature to me. Even now, I know people who find decorators and comprehensions indecipherable. Or at least, so they claim, and they aren't motivated to bother learning them. Oh well, that's their loss. Binding-expressions aren't like asynchronous programming, where the entire coding paradigm is different, and you literally have to think about your algorithms in another way. Whether you spell the binding operation target := expr expr as target expr -> target target given target = expr let target = expr : target expr ; (that last one is stolen from Forth, and should not be taken as a serious suggestion) is just a matter of spelling. We'll get used to whatever spelling it is. Some may be more convenient than others, or more error-prone, or may be harder to parse, but they're secondary issues. (Important, but still secondary.) Fundamentally, the operation is the same regardless of the spelling: - evaluate an expression - bind that value to a name - return the value (Except of course Nick's "given" suggestion is more complex, since the returned value is not necessarily the same as the bound value.)
* there are no difficult mental questions about evaluation order, e.g., in a bracketed expression having multiple assignments.
Aren't there just? x = 1 print( x + x given x = 50 ) Will that print 100, or 51? Brackets ought to make it clearer: (x + x given x = 50) # this ought to return 100 x + (x given x = 50) # this ought to return 51 but if I leave the brackets out, I have no idea what I'll get.
I would expect that your first example is a NameError: a.b(a := c.d()) since Python evaluates arguments to a method *after* looking up the method. So that corresponds to: - look up "a" # NameError, unless you've already got an "a" - look up "a.b" - evaluate c.d() - assign that value to a - pass that to the a.b method we found earlier What you probably want is the second version: (a := c.d()).b(a) which of course looks like utter crap. It might look better if we use at least half-way sensible variable names and a more realistic looking example, instead of obfuscated one-letter names. (widget := widget_builder.new(*args)).method(widget)
I think that this argument is really weak. The obvious answer is, if you don't want the assignment to happen unconditionally, then don't do the assignment unconditionally. Where you do it depends on when you want the assignment to take place. There's no mystery here. # unconditionally pop from a list (spam if mylist.pop(idx) else eggs) + mylist # conditionally pop from a list (mylist.pop(idx) if condition else eggs) + mylist (spam if condition else mylist.pop(idx)) + mylist Take your choice of which you want: (spam if (mylist := expr) else eggs) + mylist ((mylist := expr) if condition else eggs) + mylist (spam if condition else (mylist := expr)) + mylist Of course, in the last two cases, you're going to get a NameError when you try to add mylist if the ternary operator took the wrong branch. Unless you already defined mylist earlier. If you want something else, you have to explain what you want.
given makes it obvious by separating assignment from the usage of its assignment target.
This is just a purely mechanical source transformation from one to the other. Just replace ":" with "mylist given mylist " and you are done. # unconditional version (spam if (mylist given mylist = expr) else eggs) + mylist # conditional versions ((mylist given mylist = expr) if condition else eggs) + mylist (spam if condition else (mylist given mylist = expr)) + mylist If you can write the "given" versions, then just do the reverse transformation and replace the redundant verbosity with a colon.
What if they do? Is it really the end of the world if some ex-Pascal coders or people with an over-developed desire for (foolish) consistency add a colon to their statement level assignments? Some people add semi-colons to their statements too. Do we care? No. But if it aggitates people so much, then I'm perfectly happy with Guido's suggestion that we simply ban top level binding expressions and require them to leave the colons out.
* it looks a lot like the existing Python "for" and "if" clauses, which also do in-expression assignments.
"if" clauses do an assignment? Have you borrowed the keys to Guido's time machine, and are writing from 2025 and Python 4.2? *wink* I don't think "given" expressions look even remotely similar to either. for target in iterable: if condition: another_expr given target = expr Aside from "all three use a keyword". -- Steve
On Sat, May 12, 2018 at 2:28 PM Steven D'Aprano <steve@pearwood.info> wrote:
It has to be 100, or else outer parentheses change how an expression is evaluated.
That's the problem I'm showing. This is impossible: (spam if (mylist := expr) else eggs) + mylist but just fine with given: ((spam if mylist else eggs) + mylist) given mylist = expr)
First of all, you cannot convert all := expressions to given expressions. Even if you could, the point is that they are separated. I went over the separation of concerns in my other mail.
You can't always do that. I've given you two examples now where you cannot go backwards.
I meant the clauses not the statements: (expression for x in it if x.y given z = x.z) All three clauses have the same format, and obvious temporal ordering. for and given both bind a name that is visible to the expression and to clauses below.
On Sun, May 13, 2018 at 10:06 AM, Neil Girdhar <mistersheik@gmail.com> wrote:
I don't understand. How is that impossible with the colon-equals form? The 'if' expression is always going to be evaluated, followed by exactly one of 'spam' and 'eggs'. So mylist will be properly assigned before you get to adding it onto the end. ChrisA
On Sat, May 12, 2018 at 08:06:02PM -0400, Neil Girdhar wrote:
On Sat, May 12, 2018 at 2:28 PM Steven D'Aprano <steve@pearwood.info> wrote:
Whether that it right or wrong, you're missing the point. You said that we don't have to care about evaluation order. But we do: even if your analysis is correct, not everyone will realise it. I didn't. And I still don't, because I think your analyse is wrong. "Outer parentheses" is a red herring. I should have written: (x + x) given x = 50 # this ought to return 100 x + (x given x = 50) # this ought to return 51 and now there are no outer parentheses to worry about. The question is now whether "given" binds more tightly than + or not. We get to choose the precedence, and whatever we choose, it will surprise (or annoy) some people, and more importantly, some people simply won't know, and will have to ponder the difficult question of what the evaluation order is. This is already true today with code that has side-effects. I can simulate "given" using exec. It only works in the module scope: # another_expr given target = expr exec("target = expr") or another_expr but now we can see how precedence makes a real difference: x = 1 x + (exec("x = 50") or x) versus: x = 1 exec("x = 50) or (x + x) Regardless of which behaviour we choose, some people won't know it and will have to deal with "difficult mental questions about evaluation order". That's unavoidable, regardless of how we spell it, := or "given". -- Steve
On 14May2018 02:04, Steven D'Aprano <steve@pearwood.info> wrote:
Just to this: as a general principle I think "words" should bind less tightly than "punctuationlike operators". Certainly for myself I read most code as "words surrounding expressions". While "given" and "for (comprehension)" and so forth are all words within expressions as far as the language grammar goes, to my intuition and _visually_ they are bulky separators between terse "mathy" things. Even the Boolean words "and", "or", "not"...) fit this role. So as a matter of egonomic design "given" ought also to bind less tightly than "+". Arithmetic before logic before declaration before control, if you like. Cheers, Cameron Simpson <cs@cskk.id.au>
participants (36)
-
Alexander Belopolsky -
Brendan Barnwell -
Brett Cannon -
Cameron Simpson -
Carl Smith -
Chris Angelico -
Clint Hepner -
David Mertz -
Ed Kellett -
Eric V. Smith -
Greg Ewing -
Guido van Rossum -
Gustavo Carneiro -
Jacco van Dorp -
João Santos -
Juancarlo Añez -
Kirill Balunov -
marky1991 . -
Matt Arcidy -
Mikhail V -
MRAB -
Nathaniel Smith -
Neil Girdhar -
Nick Coghlan -
Nick Malaguti -
Peter O'Connor -
Rhodri James -
Rob Cliffe -
Robert Vanden Eynde -
Ryan Gonzalez -
Serhiy Storchaka -
Stephen J. Turnbull -
Steven D'Aprano -
Sven R. Kunze -
Terry Reedy -
Tim Peters