Making the for statement work better with nested functions
Starting a new thread for this as suggested by Guido. On 18/11/20 7:20 pm, Guido van Rossum wrote:
On Tue, Nov 17, 2020 at 22:12 Greg Ewing <greg.ewing@canterbury.ac.nz <mailto:greg.ewing@canterbury.ac.nz>> wrote:
If there's anything I would change, it would be to have the for statement create a new binding on each iteration, so that capturing it with a def or lambda inside the loop works as expected. I even came up with a way to do that while still allowing the last-bound value to be seen afterwards, but the idea didn't gain any traction.
I wonder if we should try that idea again? The problem may be backwards compatibility — there’s always someone who depends on such a thing (even if by accident, it would be a pain to debug). Maybe we could invent new syntax to signal this behavior, though that might defeat the purpose.
I think it can be made close enough to backwards compatible, although I'm not greatly opposed to new syntax if it's thought desirable. Essentially the idea is this: If the loop body contains a def or lambda that captures the loop variable, then on each iteration, a new binding is created, instead of changing an existing binding. Note that this is *not* the same as introducing a new scope. All the bindings share the same name, and from the perspective of code outside the nested function, nothing changes -- the name is still bound to the most recent value, and can still be accessed after the loop has finished. In CPython, it would be implemented by creating a new cell for each iteration, instead of changing the value of an existing cell. For other implementations, I don't know -- it would depend on how those implementations currently deal with nested functions. Do we need new syntax? My feeling is, probably not. I believe that the vast majority of existing code, perhaps all of it, would be unaffected by this change. All the existing workarounds for the current behaviour would continue to work. The only things that would break would be code that somehow relies on capturing the same instance of the loop variable every time, and I find it hard to imagine code that needs to do that. If we do want new syntax, my suggestion would be for new x in some_iterator: ... The downside of requiring special syntax is that we would still regularly get people asking why their lambdas in for statements don't do what they expect. We would have a better answer for them, but the questions wouldn't go away. -- Greg
On Wed, Nov 18, 2020 at 9:54 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Essentially the idea is this: If the loop body contains a def or lambda that captures the loop variable, then on each iteration, a new binding is created, instead of changing an existing binding.
Note that this is *not* the same as introducing a new scope. All the bindings share the same name, and from the perspective of code outside the nested function, nothing changes -- the name is still bound to the most recent value, and can still be accessed after the loop has finished.
The problem I see is that nested functions referring to the same variable can be defined outside loops as well. If you define a helper before the loop that refers to the loop variable, it won't work inside the loop or ever again after the loop has terminated. Even if the helper has nothing to do with the loop and just uses the same generic variable name like "item", it'll break. It's fairly common to reuse the same identifier for different purposes inside functions. As much as I hate the current behavior, I feel like the proposed behavior is too confusing and weird. Another problem is that the capturing behavior would depend on whether the variable is in a function or module or class scope, since cells only exist in function scopes. Also, "while x := get_next_thing():" loops wouldn't behave consistently with for loops. for new x in some_iterator:
...
I don't think special syntax would help. I've been bitten by this problem fairly often, and 100% of the time I've been able to solve it by adding x=x to the argument list to capture by value. I can't imagine I'd ever want to capture a loop variable by reference, especially if the cell being referenced no longer has a name outside of the nested function when the nested function is called. I get in trouble because I forget to write x=x where it's necessary, but if I forget to write x=x then I'd forget to write "new x" too.
Hmm... In the end I think the language design issue is with functions (and how they capture variable references rather than values), and fixing it by changing the for-loop is still just a band-aid, with other problems and inconsistencies. Agreed that the fix 'x=x' essentially always works, and that having to write 'for new x in ...' is no better, because you'd still forget it. Maybe we should consider introducing a new kind of function that captures values instead? That would be a much more principled fix. --Guido On Thu, Nov 19, 2020 at 4:20 PM Ben Rudiak-Gould <benrudiak@gmail.com> wrote:
On Wed, Nov 18, 2020 at 9:54 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Essentially the idea is this: If the loop body contains a def or lambda that captures the loop variable, then on each iteration, a new binding is created, instead of changing an existing binding.
Note that this is *not* the same as introducing a new scope. All the bindings share the same name, and from the perspective of code outside the nested function, nothing changes -- the name is still bound to the most recent value, and can still be accessed after the loop has finished.
The problem I see is that nested functions referring to the same variable can be defined outside loops as well. If you define a helper before the loop that refers to the loop variable, it won't work inside the loop or ever again after the loop has terminated. Even if the helper has nothing to do with the loop and just uses the same generic variable name like "item", it'll break. It's fairly common to reuse the same identifier for different purposes inside functions.
As much as I hate the current behavior, I feel like the proposed behavior is too confusing and weird.
Another problem is that the capturing behavior would depend on whether the variable is in a function or module or class scope, since cells only exist in function scopes.
Also, "while x := get_next_thing():" loops wouldn't behave consistently with for loops.
for new x in some_iterator:
...
I don't think special syntax would help. I've been bitten by this problem fairly often, and 100% of the time I've been able to solve it by adding x=x to the argument list to capture by value. I can't imagine I'd ever want to capture a loop variable by reference, especially if the cell being referenced no longer has a name outside of the nested function when the nested function is called. I get in trouble because I forget to write x=x where it's necessary, but if I forget to write x=x then I'd forget to write "new x" too.
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VKHB72... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On 26/11/20 7:07 pm, Guido van Rossum wrote:
Hmm... In the end I think the language design issue is with functions (and how they capture variable references rather than values)
Well, every other language I know of that has a notion of closures captures variables exactly the same way that Python does. Either they're all doing it wrong, or there's something unusual about Python that makes the traditional approach inappropriate for it. The reason I think the fault is with the for loop rather than the functions is that there are other languages, such as Scheme and Haskell, where the problem doesn't arise. The reason is that the equivalent of a for loop in those languages is some kind of mapping construct. For example the Python loop for x in items: add_button(lambda: do_action(x)) would be written in Scheme something like (map (lambda (x) (add_button (lambda () (do_action x))))) Because the body is a function that gets called with the value of the loop variable, it naturally creates a new binding on each iteration, which then gets captured by the inner lambda.
having to write 'for new x in ...' is no better, because you'd still forget it.
I think it's still better (or at least more principled) than relying on default argument abuse. BTW, an advantage of requiring 'new' is that it could be applied to other things. If it's allowed for any lvalue then you could write things like while new x := something(): ...
Maybe we should consider introducing a new kind of function that captures values instead?
People would then have to remember to use the right kind of function. Also I'm not sure the function is the right level of granularity. Conceivably you could want a function to capture some names as variables and others as values. I've just been looking into the new lambda feature in C++11, and it seems they've gone down this route. You get to specify which variables from the enclosing scope get captured, and for each one, whether it's captured by reference or by value. If we were to add something like this to Python, variable capture would have to be the default, so that existing code still works. Then we just need a way to specify that particular names are captured by value. Not sure how to do that in a way that doesn't look ugly and/or obscure. I'm reminded of something related that was discussed a while back: a way for a function to have "constant" local names that are evaluated at definition time, like default arguments, but not part of the argument list. Kind of "default argument abuse without the abuse". Maybe we should be revisiting that idea? -- Greg
On Thu, Nov 26, 2020, 5:12 AM Greg Ewing
Then we just need a way to specify that particular names are captured by value. Not sure how to do that in a way that doesn't look ugly and/or obscure.
Isn't that EXACTLY what you call "default argument abuse"?! To me, that is concise, obvious, and easy to remember. So what you are asking for is a less obvious way to spell something that already exists.
On 25Nov2020 22:07, Guido van Rossum <guido@python.org> wrote:
Hmm... In the end I think the language design issue is with functions (and how they capture variable references rather than values), and fixing it by changing the for-loop is still just a band-aid, with other problems and inconsistencies. Agreed that the fix 'x=x' essentially always works, and that having to write 'for new x in ...' is no better, because you'd still forget it.
Maybe we should consider introducing a new kind of function that captures values instead? That would be a much more principled fix.
I was thinking in my previous post (where such thought was off topic:-) that block scopes for Python should be as clear as in C, without using braces. What about something along the lines of: with expression as new x, y: ... x and y local to this suite ... ... also x and y cannot shadow existing names ... I wouldn't want to let any "as" support a "new", but the one on "with" introduces a suite which nicely delineates a block. Cheers, Cameron Simpson <cs@cskk.id.au>
On 30Nov2020 04:00, sairamkumar2022@gmail.com <sairamkumar2022@gmail.com> wrote:
This is quite good but remove the new keyword, it's not necessary there.
The idea is to introduce names which are not previously present, and _expire_ and the end of the enclosed suite. Without the "new" is valid Python right now and does not mean that. So some indication of the special meaning would be required. Cheers, Cameron Simpson <cs@cskk.id.au>
Wouldn't `local` make sense as the opposite of `global` - personally I would rather see that than LET. -----Original Message----- From: Cameron Simpson <cs@cskk.id.au> Sent: 30 November 2020 07:13 To: sairamkumar2022@gmail.com Cc: python-ideas@python.org Subject: [Python-ideas] Re: Making the for statement work better with nested functions On 30Nov2020 04:00, sairamkumar2022@gmail.com <sairamkumar2022@gmail.com> wrote:
This is quite good but remove the new keyword, it's not necessary there.
The idea is to introduce names which are not previously present, and _expire_ and the end of the enclosed suite. Without the "new" is valid Python right now and does not mean that. So some indication of the special meaning would be required. Cheers, Cameron Simpson <cs@cskk.id.au> _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KIZGGX... Code of Conduct: http://python.org/psf/codeofconduct/
On 2020-11-25 22:07, Guido van Rossum wrote:
Hmm... In the end I think the language design issue is with functions (and how they capture variable references rather than values), and fixing it by changing the for-loop is still just a band-aid, with other problems and inconsistencies. Agreed that the fix 'x=x' essentially always works, and that having to write 'for new x in ...' is no better, because you'd still forget it.
Maybe we should consider introducing a new kind of function that captures values instead? That would be a much more principled fix.
I think something along those lines makes a lot more sense to me, After following this thread I'm still not convinced changing the for loop is really a good idea. I basically agree with Steven D'Aprano that the supposed benefits of "protecting" against accidental name shadowing are marginal at best and don't justify complicating matters. I also agree that the for loop isn't really the issue. For instance, if we "fix" for loops, the name-binding issue will still exist for functions in while loops. For me the only problem with the `x=x` default argument "fix" is that. . . well, it's in the argument signature. That means it effectively becomes part of the function's public API, and people can pass in other values, which maybe you don't want. Plus it just seems very unclean to me to have to shoehorn "I want to capture these variables from the enclosing scope at definition time" into the mechanism for "these are the arguments you can pass at call time". What we really want is a way to separate these into orthogonal choices. So what I've been wondering is, could we provide a syntax *other than default arguments* to explicitly indicate that a function wants to capture specific variables from the enclosing scope? Something like: def f(a, b, c) with x, y: # x and y here have whatever values they had in the enclosing scope at definition time The advantages I see to this are: 1) It makes explicit the separation between "capture this value at definition time" and "allow passing this at call time". 2) It maintains the convention that we don't "peek" inside the function body at definition time (i.e., all the variables to be captured are "outside" in the def line) 3) It narrowly targets specific variables. It's probably not a good idea for a function to just capture ALL variables from outer scopes (for instance, you usually don't want to capture values of global config variables that might change before call time). 4) It is backwards compatible, doesn't change any existing behavior It is new syntax but if we use "with" we could avoid grabbing a new keyword. I don't know enough about the internals of Python to know how this would be implemented, but my idea is that it would effectively create new local variables for x and y, but not put them in the argument list (so they'd be in co_varnames but not co_argcount). Basically what I'm saying here is that I don't want block scopes. Adding block scopes as a thing will just complicate the language as block-scoped variables could then be used for all manner of things besides "freezing" them into functions. For me the issue is only about how function scopes capture variables from enclosing scopes; we don't need new scopes, we just need new ways to control that capturing behavior. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Sun, Nov 29, 2020 at 8:28 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:
After following this thread I'm still not convinced changing the for loop is really a good idea. I basically agree with Steven D'Aprano that the supposed benefits of "protecting" against accidental name shadowing are marginal at best and don't justify complicating matters.
+100
def f(a, b, c) with x, y: # x and y here have whatever values they had in the enclosing scope at definition time
I don't *entirely* hate this. But it still feels like new syntax for something that is perfectly good as pure convention: def f(a, b, c=21, _foo=foo, _bar=bar): # "capture" foo and bar from enclosing scope foo, bar = _foo, _bar # optionally ... Of course I know that users COULD call `f(1, 2, _foo=something_else)`. But we have asked them not to politely, and "we're all consenting adults." What feels like a VASTLY more important issue is that foo and bar might have mutable types. Quite possibly, the underlying objects have changed before the calling point. That can be good or bad, but it's definitely something folks stumble over. On the other hand, I know it perfectly well, and the number of times I've written this is small: def g(a, b, _foo=deepcopy(foo)): ... -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On 30/11/20 9:46 am, David Mertz wrote:
def f(a, b, c=21, _foo=foo, _bar=bar):
Of course I know that users COULD call `f(1, 2, _foo=something_else)`. But we have asked them not to politely, and "we're all consenting adults."
This doesn't work so well if the function takes *args or **kwds arguments. -- Greg
On Sun, Nov 29, 2020, 11:52 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 30/11/20 9:46 am, David Mertz wrote:
def f(a, b, c=21, _foo=foo, _bar=bar):
Of course I know that users COULD call `f(1, 2, _foo=something_else)`. But we have asked them not to politely, and "we're all consenting adults."
This doesn't work so well if the function takes *args or **kwds arguments.
Why?! I cannot see any relevance there.
On Sun, Nov 29, 2020 at 12:28 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:
For me the only problem with the `x=x` default argument "fix" is that. . . well, it's in the argument signature. That means it effectively becomes part of the function's public API, and people can pass in other values, which maybe you don't want. Plus it just seems very unclean to me to have to shoehorn "I want to capture these variables from the enclosing scope at definition time" into the mechanism for "these are the arguments you can pass at call time". What we really want is a way to separate these into orthogonal choices.
But do we? The key to language design is to have as few features as you can that can be combined in as many ways as they can. *That* is what I consider orthogonality (of features). Using argument defaults for a novel purpose scores high according to this, while dedicated syntax as you show below (def f(a, b, c) with x, y: ...) scores low -- it's a new feature that can do only one thing. OTOH if we were to introduce 'let' or 'const' in the language, it would surely be usable to solve a whole bunch of other problems, in addition to giving us a cleaner way to solve the value capture problem. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Sun, 29 Nov 2020 at 23:34, Guido van Rossum <guido@python.org> wrote:
OTOH if we were to introduce 'let' or 'const' in the language, it would surely be usable to solve a whole bunch of other problems, in addition to giving us a cleaner way to solve the value capture problem.
Well, IMHO let could help the famous misspelled var problem: myvar = 5 [...] myvvar = 7 # Whops! but to work, if let is used somewhere in the scope, all the variables declared in the scope must be declared with let: let myvar = 5 myvvar = 7 # SyntaxError But I'm really unsure that this is what you want.
Hello, On Thu, 19 Nov 2020 18:53:01 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Starting a new thread for this as suggested by Guido.
On 18/11/20 7:20 pm, Guido van Rossum wrote:
On Tue, Nov 17, 2020 at 22:12 Greg Ewing <greg.ewing@canterbury.ac.nz <mailto:greg.ewing@canterbury.ac.nz>> wrote:
If there's anything I would change, it would be to have the for statement create a new binding on each iteration, so that capturing it with a def or lambda inside the loop works as expected. I even came up with a way to do that while still allowing the last-bound value to be seen afterwards, but the idea didn't gain any traction.
I wonder if we should try that idea again? The problem may be backwards compatibility — there’s always someone who depends on such a thing (even if by accident, it would be a pain to debug). Maybe we could invent new syntax to signal this behavior, though that might defeat the purpose.
I think it can be made close enough to backwards compatible, although I'm not greatly opposed to new syntax if it's thought desirable.
Essentially the idea is this: If the loop body contains a def or lambda that captures the loop variable, then on each iteration, a new binding is created, instead of changing an existing binding.
Note that this is *not* the same as introducing a new scope.
And that's very sad. That means instead of solving the root of the problem, adhoc workarounds are proliferated.
All the bindings share the same name, and from the perspective of code outside the nested function, nothing changes -- the name is still bound to the most recent value, and can still be accessed after the loop has finished.
In CPython, it would be implemented by creating a new cell for each iteration,
Million cells for million of iterations? Truly a CPython-style solution!
instead of changing the value of an existing cell. For other implementations, I don't know -- it would depend on how those implementations currently deal with nested functions.
Do we need new syntax? My feeling is, probably not. I believe that the vast majority of existing code, perhaps all of it, would be unaffected by this change. All the existing workarounds for the current behaviour would continue to work. The only things that would break would be code that somehow relies on capturing the same instance of the loop variable every time, and I find it hard to imagine code that needs to do that.
If we do want new syntax, my suggestion would be
for new x in some_iterator: ...
I'd suggest that it should be "for let", that it should introduce a new scope, and the scoping should be optimized to always treat "x" (as the "for" control variable) by-value, instead of breeding billions of cells. Using "let" will then give natural way to introduce proper block-level lexical scoping in Python: def foo(): let x = 1 if bar: let x = 2 ... # x is 1 again here Using "let" will also allow for a natural counterpart of "const" keyword syntax, vs annotation, like it has to be now (https://mail.python.org/archives/list/python-ideas@python.org/thread/SQTOWJ6...). Actually, thinking about it, the syntax should be "for const" in the first place: for const x in some_iterator: ... That will make it obvious that "x" is captured by value, and that: for const x in some_iterator: ... x = 1 Doesn't make sense (should lead to error - assigning to a const).
The downside of requiring special syntax is that we would still regularly get people asking why their lambdas in for statements don't do what they expect. We would have a better answer for them, but the questions wouldn't go away.
So, there shouldn't be "special syntax". There should be generic syntax to solve Python's block scoping and const'ness issues.
-- Greg
-- Best regards, Paul mailto:pmiscml@gmail.com
On Thu, Nov 26, 2020 at 02:11:10PM +0300, Paul Sokolovsky wrote:
Using "let" will then give natural way to introduce proper block-level lexical scoping in Python:
Why would we want to do that? Block-level scoping is too much of a good thing. The right level of scoping is the function, not the block. (I'm willing to accept at least one exception to this, comprehensions.) Block scoping adds complexity to the implementation *and* the semantics of code. Block scoping allows shadowing within a function. Some C-derived languages, such as Java and C#, considered that a big enough mistake that they corrected it by requiring variables within a block to be distinct from the variables outside of the block, which is conceptually weird. The point of scoping is that variables in one scope should be independent of those in another scope, yet here we have this action at a distance where names in one scope can prevent you from using the same name in a different scope. Python does have one example of a pseudo-block scope: the treatment of the except variable: try: ... except Exception as err: ... Effectively, "err" is treated as a "let" variable with a pseudo-block scope, limiting it to the except block. This is necessary, but an annoyance. I have come across many people suprised and annoyed that err is not available outside of the block. (Fortunately, it is rare for folks to need access to the err variable outside of the block, otherwise the annoyance factor would be much greater.) Block scoping adds semantic and implementation complexity and annoyance, while giving very little benefit. No thank you.
def foo(): let x = 1 if bar: let x = 2 ... # x is 1 again here
Is there a shortage of variable names that you have to reuse the same name "x" for two distinct variables? I know that "naming things" is classically one of the hard problems of computer science, but this is taking things to an extreme. -- Steve
On Fri, Nov 27, 2020 at 12:28 AM Steven D'Aprano <steve@pearwood.info> wrote:
Block scoping adds semantic and implementation complexity and annoyance, while giving very little benefit. No thank you.
def foo(): let x = 1 if bar: let x = 2 ... # x is 1 again here
Is there a shortage of variable names that you have to reuse the same name "x" for two distinct variables?
I know that "naming things" is classically one of the hard problems of computer science, but this is taking things to an extreme.
I spy the Blub Paradox here. Since you don't use lexical scoping, you don't like it, don't want it, and see no value in it. I use lexical scoping all the time, and it doesn't lead to the stupidities you're implying that it does; in fact, it can be used to make code much clearer and easier to reason about, because - quite the contrary to what you're saying here - you can use distinct variable names, and have a guarantee that the variable can't be misused outside of its intended scope. This wouldn't really apply as cleanly to Python, since the rest of the function could potentially use a global with the same name, but please, stop assuming and asserting that block scoping must inherently be both useless and confusing. (And there are a number of very VERY good - albeit unusual - uses for name reuse. It's not that we can't think of different names. It's that we specifically WANT the same name, since it is serving the same purpose. In the absence of sub-function scoping, we're forced to invent arbitrarily different names, and that doesn't actually improve clarity at all.) ChrisA
On Fri, Nov 27, 2020 at 01:05:45AM +1100, Chris Angelico wrote:
On Fri, Nov 27, 2020 at 12:28 AM Steven D'Aprano <steve@pearwood.info> wrote:
Block scoping adds semantic and implementation complexity and annoyance, while giving very little benefit. No thank you. [...] Is there a shortage of variable names that you have to reuse the same name "x" for two distinct variables?
I know that "naming things" is classically one of the hard problems of computer science, but this is taking things to an extreme.
I spy the Blub Paradox here. Since you don't use lexical scoping,
Um, actually I do use lexical scoping :-) Most languages, including Python, use lexical scoping :-) https://en.wikipedia.org/wiki/Scope_(computer_science)#Lexical_scope_vs._dyn... I think you mean *block* scoping, as opposed to function scoping. It's true that I don't often use block scoping, but come on Chris, this is *me* you are talking too. Do you really think I'm a knee-jerk Blubbite? :-) If anything, I sometimes have to struggle against the opposite tendency, the desire to shoe-horn arbitrary language features and paradigms into the language because they would be cool. (Thunks. I want thunks. I don't care what they get used for, or how they are spelled, or what they do, so long as I can say "Python has thunks".) *wink* [...]
it can be used to make code much clearer and easier to reason about, because - quite the contrary to what you're saying here - you can use distinct variable names, and have a guarantee that the variable can't be misused outside of its intended scope.
You say that as if you are coding in a hostile environment where your nemesis is sneaking around deliberately re-using (*misusing*) your variables after you have finished with them. You know how Python doesn't have constants? And we say, or at least we said, "if you don't want to re-bind a name, just *don't re-bind it*"? Surely the same applies to within a function. If you bind a value to a name inside a loop: for x in seq: y = expression and you don't want to re-use y outside of that loop, just don't re-use it. As I said in a previous post, if your variable names are so generic that they can be re-used, or your code so large and complicated that you might accidentally re-use a name, the problem isn't the lack of block scopes, the problem is your excessively complicated function and rubbish variable names. Now Chris, I do get it. In the universe of an effectively infinite number of computer programs, I daresay that there are one or two where there is a specific name that needs to represent two different things at the same time, without forcing them into separate functions: bow = Ribbon(colour='yellow') tie(bow, old_oak_tree) for archer in troop: let bow = get_weapon() archer.take(bow) assert bow.colour = 'yellow' and I'm sure you can come up with some actual genuine examples. Fine, I believe you. But I maintain that if we balance the negatives: - a more complex set of scoping rules means more to learn; - and more to misunderstand; - harder to implement; - knock on complexity (e.g. what happens to `locals()`?); - annoyance factor, e.g. treatment of `except ... as err`; against the positives, the balance comes out clearly against block scopes. You're welcome to try to persuade me otherwise by demonstrating that the positive uses of block-level scoping are not as unusual as I expect, and more useful than I think, but please don't assume that because I have come to the opposite conclusion as you, it can only be because I don't understand the feature and have therefore dismissed it as "this weird Blub language". We're not talking about Haskell Monads here :-) For just under 30 years Python has done without constants, not because constants are pointless, but because the additional complexity isn't worth the benefit. We can get 95% of the benefit of constants from naming conventions and self-control ("just don't re-bind it") with none of the cost. And my argument is that we can do the same for block scopes: we get 95% of the benefit, with none of the costs, by just *not* re-using names.
(And there are a number of very VERY good - albeit unusual - uses for name reuse. It's not that we can't think of different names. It's that we specifically WANT the same name, since it is serving the same purpose. In the absence of sub-function scoping, we're forced to invent arbitrarily different names, and that doesn't actually improve clarity at all.)
It's fine to re-use names. What we're talking about with block scoping is having two variables with the same name coexisting within the same function at the same time. That's quite different from sequentially re-using the same variable for two (related or not) purposes. -- Steve
bow = Ribbon(colour='yellow') tie(bow, old_oak_tree) for archer in troop: let bow = get_weapon() archer.take(bow) assert bow.colour = 'yellow'
I had just this problem yesterday, and on many other days. However, I don't think the opt-in block scoping would have done much to help. In my particular case, I had some geographic data in a file. It seemed like a name like 'geo_data' was a good way to refer to the filename with the data. Then it seemed like 'geo_data was a good way to refer to the file handle. Then it seemed like a name like 'geo_data' was a good way to refer to the DataFrame resulting from reading it. Then it seemed like 'geo_data' was a good way to refer to the reformatted data I created in memory. Then it seemed like 'geo_data' was a good name for the file handle to the new file I write in the new format. Naming things is hard. (I might have exaggerated the specific example). But not much of the problem had to do with scoping. I pretty much wanted to use all those different meanings in the same scope. And also not use absurdly long names like 'filehandle_for_json_geo_data'. Even if scoping *could* have separated all those uses, it would be kinda terrible code where that same name had all these different meanings nearby, even if in technically different scopes. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
Hello, On Fri, 27 Nov 2020 00:25:21 +1100 Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Nov 26, 2020 at 02:11:10PM +0300, Paul Sokolovsky wrote:
Using "let" will then give natural way to introduce proper block-level lexical scoping in Python:
Why would we want to do that?
Because without that, when you're doing something simple like turning: foo.bar1 = bar.bar1.bar2.bar3 into: # We know that Python doesn't do CSE, and don't want to evaluate # bar.bar1.bar2.bar3 twice, so we cache it in a temporary a = bar.bar1.bar2.bar3 foo.bar1 = a func(a) - you're in constant fear that some another guy already did something similar before, and you're overwriting "a" (or whatever other good name you (or they) have come up with, like "i", "tmp", etc.). And of course, that's not just your fears, you heard other fellow programmers talking about it too. The closest one is right higher in this thread: https://mail.python.org/archives/list/python-ideas@python.org/message/VKHB72... Quite for you: "Even if the helper has nothing to do with the loop and just uses the same generic variable name like "item", it'll break. It's fairly common to reuse the same identifier for different purposes inside functions."
Block-level scoping is too much of a good thing. The right level of scoping is the function, not the block.
This is getting old. It's neither practical nor realistic to refactor each breadcrumb (or vice-versa jumbo written by somebody too respected to teach them about a suggested function size) to a function. A trivial fan-out assignment above is a good example.
(I'm willing to accept at least one exception to this, comprehensions.)
Please meet me on the other side of the rainbow, where what we previously saw as an exception, is actually first shy, blindfolded steps to resolve the general underlying issue.
Block scoping adds complexity to the implementation *and* the semantics of code.
Python already has lexical scoping for functions, extending it to blocks within functions is something a notch above trivial. Semantics of the language re: scoping remains the same. Semantics of the actual code written using it improves, as it allows to reason more locally about how a variable is used.
Block scoping allows shadowing within a function. Some C-derived languages, such as Java and C#, considered that a big enough mistake that they corrected it by requiring variables within a block to be distinct from the variables outside of the block, which is conceptually weird.
Sorry, this is MISRA-style disinformation (https://en.wikipedia.org/wiki/MISRA_C#Criticism). Some adhoc, narrow-use, narrow-purpose codestyles indeed disallow shadowing of variable names. Otherwise, lexical scoping of name bindings is fundamental concept of the computation theory and practice.
The point of scoping is that variables in one scope should be independent of those in another scope, yet here we have this action at a distance where names in one scope can prevent you from using the same name in a different scope.
It's all the same as already for functions in Python. Again, doing it block-level allows for more local, more precise reasoning (e.g. if you declared a let-variable, you know you don't overwrite anything else).
Python does have one example of a pseudo-block scope: the treatment of the except variable:
try: ... except Exception as err: ...
Python does have one example of pseudo-block scope, then a few example of truly block (well, expression!) level scope (similarly implemented in hacky way), and sooner or later will get proper, generic block-level scope. The sooner the better of course. Let the usual denial/anger/bargaining/depression/acceptance cycle begin. (With pattern matching, we're at bargaining stage, don't you think? Actually, looking at https://discuss.python.org/t/gauging-sentiment-on-pattern-matching/5770 , maybe it's depression.) []
Is there a shortage of variable names that you have to reuse the same name "x" for two distinct variables?
But that's the whole point - I (and other people) don't want to know whether it's used 2nd, 3rd, or hundredth time. We want to use it as a generic variable in a very limited scope, knowing that it won't affect semantics of the rest of a program.
I know that "naming things" is classically one of the hard problems of computer science, but this is taking things to an extreme.
Not only hard, but also largely solved. That's why it's easy to predict that Python will get block-scoped things sooner or later (heck, it does, as we know). Saying otherwise is the same as saying "Python will forever lag behind other languages and will plug glaring conceptual holes with hilarious means like IDEs, etc." It's "Thanks but no" from me to such a perspective.
-- Steve []
-- Best regards, Paul mailto:pmiscml@gmail.com
On 27Nov2020 00:25, Steven D'Aprano <steve@pearwood.info> wrote:
Block scoping allows shadowing within a function.
Just to this: it needn't. You could forbid shadowing of the _static_ outer scope easily enough at parse/compile time. That would prevent a certain class of easy misuse. i = 9 { new scope here for "i" ==> parse/compile error, since "i" is in play } It depends what the scope's for: to define a variable lifetime (very desireable sometimes) or to _actually_ shadow something (I'm against that myself - there are better and less error prone ways). Anyway, I'm against proliferation of new scopes on the whole. That said, there _are_ times I wish I could mark out the lifetime of a variable, akin to C level: ... i does not exist ... { int i; ... use i ... } ... i now unknown, use is an error ... The nearest Python equivalent is: i = blah() ... use i del i which feels fragile - accidental assignment to "i" later is not forbidden. Of course, in C variables must be declared. Not so easy with Python, syntacticly. Cheers, Cameron Simpson <cs@cskk.id.au>
On Fri, Nov 27, 2020 at 08:32:04AM +1100, Cameron Simpson wrote:
On 27Nov2020 00:25, Steven D'Aprano <steve@pearwood.info> wrote:
Block scoping allows shadowing within a function.
Just to this: it needn't.
Yes, I'm aware of that, and discussed languages such as Java which prohibit name shadowing within a function. https://docs.oracle.com/javase/specs/jls/se7/html/jls-6.html#jls-6.4 Shadowing is a double-edged sword. It is sometimes useful, but often a source of hard to find bugs. One might argue that a language should allow *some* shadowing but not too much.
You could forbid shadowing of the _static_ outer scope easily enough at parse/compile time. That would prevent a certain class of easy misuse.
i = 9 { new scope here for "i" ==> parse/compile error, since "i" is in play }
Yes, that's precisely the sort of thing I discussed, and described as action-at-a-distance between two scopes, where the mere existence of a name in scope A prevents you from using the same name in scope B. The problem is, if your inner block scope must not reuse variable names in the outer function scope, well, what's the advantage to making them seperate scopes? Analogy: I think most of us would consider it *really weird* if this code was prohibited: a = None def func(): a = 1 # Local with the same name as the global prohibited. True, it might save newbies who haven't learned about the global keyword from making a few mistakes, but that's a pretty small, and transient, benefit. (Most coders are newbies for, what, one or two percent of their active coding life?) One possible advantage, I guess, is that if your language only runs the garbage collector when leaving a scope, adding extra scopes helps to encourage the timely collection of garbage. I don't think that's a big advantage to CPython with it's reference counting gc.
That said, there _are_ times I wish I could mark out the lifetime of a variable, akin to C level:
... i does not exist ... { int i; ... use i ... } ... i now unknown, use is an error ...
The nearest Python equivalent is:
i = blah() ... use i del i
which feels fragile - accidental assignment to "i" later is not forbidden.
Why do you care about the *name* "i" rather than whatever value is bound to that name? I completely get the idea of caring about the lifetime of an object, e.g. I understand the need to garbage collect the exception object when leaving `except` blocks. (At least by default.) But I don't get why I might care about the lifetime of a *name*. try: ... except Exception as e: pass e # Name is unbound, for good reasons. e = None # But why should this be an error? We don't generally take the position that reuse of a name in the same function is Considered Harmful, let alone *so harmful* that we need the compiler to protect us from doing so. If I am *accidentally* reusing names, my code has much bigger problems than just the name re-use: - my names are so generic and undescriptive that that can be re-used for unrelated purposes; - and the function is so large and/or complicated that I don't notice when I am re-using a name. Name re-use in the bad sense is a symptom of poor code, not a cause of it, and as such block scopes are covering up the problem. (That's my opinionated opinion :-) -- Steve
On 27Nov2020 21:13, Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Nov 27, 2020 at 08:32:04AM +1100, Cameron Simpson wrote:
On 27Nov2020 00:25, Steven D'Aprano <steve@pearwood.info> wrote:
Block scoping allows shadowing within a function.
Just to this: it needn't.
Yes, I'm aware of that, and discussed languages such as Java which prohibit name shadowing within a function. https://docs.oracle.com/javase/specs/jls/se7/html/jls-6.html#jls-6.4
Shadowing is a double-edged sword. It is sometimes useful, but often a source of hard to find bugs. One might argue that a language should allow *some* shadowing but not too much.
"mm, yeah. I'd think I'd be happier simply disallowing it. Got a nice example of somewhere where shadowing would be useful and hard to do some task otherwise?
You could forbid shadowing of the _static_ outer scope easily enough at parse/compile time. That would prevent a certain class of easy misuse.
i = 9 { new scope here for "i" ==> parse/compile error, since "i" is in play }
Yes, that's precisely the sort of thing I discussed, and described as action-at-a-distance between two scopes, where the mere existence of a name in scope A prevents you from using the same name in scope B.
Well, it prevents a lot of easy mistakes. I think for it to be "action at a distance" the function / code block would need to be quite long. Generally held to be a poor practice (not that I haven't some quite long functions of my own, of course, but I accept the cognitive burden there for other reasons).
The problem is, if your inner block scope must not reuse variable names in the outer function scope, well, what's the advantage to making them seperate scopes?
Lifetime, but in terms of referencing resources and also accidental (mis)use of something beyond where it was supposed to be used. I used to do short {...} scopes in C all the time for this purpose.
Analogy: I think most of us would consider it *really weird* if this code was prohibited:
a = None
def func(): a = 1 # Local with the same name as the global prohibited.
Yes, but a function is a clean new scope to my mind. (Yes, closures.) As opposed to this: f = None # define a lifespan for "f" using the fictitious "as new" syntax. with open("foo") as new f: process file f # f is None again I'd be all for forbidding that, and instead requiring a different name for the with-statement "f".
One possible advantage, I guess, is that if your language only runs the garbage collector when leaving a scope, adding extra scopes helps to encourage the timely collection of garbage. I don't think that's a big advantage to CPython with it's reference counting gc.
I'm not concerned directly with garbage collection so much as preventing use of a name beyond its intended range. Which is the next bit:
That said, there _are_ times I wish I could mark out the lifetime of a variable, akin to C level:
... i does not exist ... { int i; ... use i ... } ... i now unknown, use is an error ...
The nearest Python equivalent is:
i = blah() ... use i del i
which feels fragile - accidental assignment to "i" later is not forbidden.
Why do you care about the *name* "i" rather than whatever value is bound
Because of intent. Why do we encapsulate things in functions? It isn't just releasing resources and having convenient code reuse, but also to say "all these names inside the function are only relevant to it, and are _not_ of use after the function completes".
I completely get the idea of caring about the lifetime of an object, e.g. I understand the need to garbage collect the exception object when leaving `except` blocks. (At least by default.)
But I don't get why I might care about the lifetime of a *name*.
Maybe not. But I very much do.
try: ... except Exception as e: pass e # Name is unbound, for good reasons. e = None # But why should this be an error?
We don't generally take the position that reuse of a name in the same function is Considered Harmful, let alone *so harmful* that we need the compiler to protect us from doing so.
Indeed not. But Sometimes I _want_ do take that position, for _particular_ names.
If I am *accidentally* reusing names, my code has much bigger problems than just the name re-use: - my names are so generic and undescriptive that that can be re-used for unrelated purposes;
Wasn't one of your complaints above "Why do you care about the *name* "i" rather than whatever value is bound"? "i" is a _very_ generic and undescriptive name.
- and the function is so large and/or complicated that I don't notice when I am re-using a name.
Name re-use in the bad sense is a symptom of poor code, not a cause of it, and as such block scopes are covering up the problem.
(That's my opinionated opinion :-)
Aye. I think we may need to just disagree here. I think we agree that name "misreuse" is symptomatic of larger problems. But we seem to disagree about letting the compiler _aid_ us in defining the intended bounds of use, so that the compiler can be _told_ of the range, and warn us (or the coder who comes afterward) about this error. The more I _can_ say about names and their use, the better. How _much_ I then say is a matter of requirements, style and taste. Do you use linters? I do. And the very neat typeguard module which can check type annotations at runtime, and icontract which can vet preconditions and post conditions at runtime, etc. Cheers, Cameron Simpson <cs@cskk.id.au>
On Sat, Nov 28, 2020 at 12:57:12PM +1100, Cameron Simpson wrote:
Got a nice example of somewhere where shadowing would be useful and hard to do some task otherwise?
Mocking. For example: https://docs.python.org/3/library/unittest.mock.html#patch-builtins Monkey-patching. One nice trick I've used is if I have a function that is excessively verbose, printing oodles of unwanted output, I can monkey-patch the owner's module, without affecting other calls to print: import library library.print = lambda *args, **kw: None result = library.noisy_function() There are other ways to get the same result, of course (change sys.stdout, etc) but I find this works well. Another technique I've used occassionally: I need diagnostics to debug a function, but I don't want to insert debugging code into it. (Because I invariably forget to take it out again.) But I can sometimes monkey-patch a built-in to collect the diagnostics. I don't have to remove my debugging code from the source because it isn't in the source. I've even shadowed entire modules: - copy module A to a new directory; - modify the copy; - cd into the directory; - run your code as normal, with the sole difference that "import A" will give you the patched (experimental) A rather than the regular A. But most important of all: we must be able to shadow module level globals with locals with the same name, otherwise the addition of a global "x" will suddenly and without warning break any function that happens to also use "x".
The problem is, if your inner block scope must not reuse variable names in the outer function scope, well, what's the advantage to making them seperate scopes?
Lifetime, but in terms of referencing resources and also accidental (mis)use of something beyond where it was supposed to be used. I used to do short {...} scopes in C all the time for this purpose.
I keep hearing about "accidental misuse" of variables. Are people in the habit of coding by closing their eyes and mashing the keyboard until the code compiles? *wink* Seriously, how big and complex are your functions that you can't tell whether you are re-using a variable within a single function and are at risk of doing so *accidentally*? Martin Fowler suggests that (Ruby) functions with more than, say, half a dozen lines are a code smell: https://www.martinfowler.com/bliki/FunctionLength.html 75% of his functions are four lines or less. Even if you think this is a tad excessive, surely we're not in the habit of writing functions with hundreds of lines and dozens of variables, and consequently keep accidentally re-using variables? This seems especially ... odd ... in a language with declarations like C, where you ought to easily be able to tell what variables are in use by looking at the declarations. Unless of course you are in the habit of scattering your declarations all through the function, an anti-pattern that block-scopes encourages. "Only block-scopes can save us from the incomprehensible mess of code that uses block-scopes!" *wink*
Analogy: I think most of us would consider it *really weird* if this code was prohibited:
a = None
def func(): a = 1 # Local with the same name as the global prohibited.
Yes, but a function is a clean new scope to my mind. (Yes, closures.) As opposed to this:
f = None
# define a lifespan for "f" using the fictitious "as new" syntax. with open("foo") as new f: process file f
# f is None again
I'd be all for forbidding that, and instead requiring a different name for the with-statement "f".
This implies that in your mind, with statements are *not* "clean new scopes" like functions are. If they were, you would be quite comfortable with the with statement introducing a separate variable that happened to be called f but was distinct from the local variable "f" or global variable "f" outside of the block. I think that's telling. Functions make naturally distinct scopes, which is why we're comfortable with locals and globals with the same name. Likewise comprehensions: they look and feel self-contained, which is why people were surprised that they leaked. Nobody said that we ought to prohibit the comprehension from using the same names as the surrounding function. But blocks don't make natural scopes. We can force them to be separate scopes, but even (some) people who want this feature don't actually think of them as naturally distinct scopes like functions with independent sets of names. If a function and a block use the same name, they see that as a clash, rather than two distinct, independent, non-clashing scopes like functions or comprehensions.
One possible advantage, I guess, is that if your language only runs the garbage collector when leaving a scope, adding extra scopes helps to encourage the timely collection of garbage. I don't think that's a big advantage to CPython with it's reference counting gc.
I'm not concerned directly with garbage collection so much as preventing use of a name beyond its intended range. Which is the next bit: [...]
Why do you care about the *name* "i" rather than whatever value is bound
Because of intent. Why do we encapsulate things in functions? It isn't just releasing resources and having convenient code reuse, but also to say "all these names inside the function are only relevant to it, and are _not_ of use after the function completes".
That cannot possibly be true, because we use the same *names* after the function completes all the time. That's why functions introduce a new scope, so that distinct scopes can use the same names without stomping on each other! If we cared about the *names*, as you insist, we would insist that names in functions have to be globally unique. But we don't. Functions are their own distinct scopes so that we can use names inside a function without caring whether those names are used elsewhere. But I can't do the same for block scopes if names are forced to be unique. Instead of being independent, the scopes are strongly coupled: every time I use a name inside a block, I have to care whether it is already used outside of that block. So blocks are not actually independent scopes under the Java semantics. (They are independent in C.) [...]
But I don't get why I might care about the lifetime of a *name*.
Maybe not. But I very much do.
I think you actually care about the value bound to the name, but mistake it for the name itself. Consider: with open(...) as new f: # new scope, the *name* f is only allowed in this block process(f) g = f # smuggle the *value* of f outside the block do_something_with(g) If you care about the *name* f, then you will be fine with that. Nobody is reusing f, so it's all good! -- Steve
I don’t know if this has been discussed before. Similar to PEP 645 idea of writing "Optional[type]" as “type?”, I propose we write "Callable[[type1, type2, ...], type3]” as “[type1, type2, … -> type3]”. Look at the two examples below and see which one looks better to the eyes: def func1(f: typing.Callable[[str, int], str], arg1: str, arg2: int) -> str: return f(arg1, arg2) def func2(f: [str, int-> str], arg1: str, arg2: int) -> str: return f(arg1, arg2) There is less clutter especially if we have nested Callables. e.g., f: Callable[[str, int], Callable[[int,…], str]] becomes f: [str, int -> [int, ... -> str]] Callable without zero arguments.. f: Callable[[], str] would become f: [ -> str] Equivalent to Callable alone without caring about arguments and the return value would be [… -> typing.Any] or [… -> ] Let’s say we have a function that accepts a decorator as an argument. This might not be useful to do, but I want to show case how it would be easier to read. The old way would be: def decorator(f: Callable[…, int]) -> Callable[…, tuple[int, str]]: def wrapper(*args, **kwargs) -> tuple[int, str]: text = “some text” res = f(*args, **kwargs) return res, text return wrapper def function(decorator: Callable[[Callable[…, int]], Callable[…, tuple[int, str]]], decorated_on: Callable[…, int]) -> Callable[…, tuple[int, str]]: wrapper = decorator(decorated_on) return wrapper The new way is as follows: def decorator(f: [… -> int]) -> [… -> tuple[int, str]]: def wrapper(*args, **kwargs) -> tuple[int, str]: text = “some text” res = f(*args, **kwargs) return rest, text return wrapper def function(decorator: [ [… -> int] -> [… -> tuple[int, str]]], decorated_on: [… -> int]) -> [… -> tuple[int, str]]: wrapper = decorator(decorated_on) return wrapper I saw something similar in Pylance type checker (VSC extension) when you hover over an annotated function that has Callable as an argument or a return value, but they don’t seem to use brackets to mark the beginning and end of the callable, which could be hard to follow mentally (see screenshot below) Personally, I think it would be easier if Pylance wrote the hint like the following: (function) function: (decorator: [p0:[*args: Any, **kwargs: Any -> int ]] -> [*args: Any, **kwargs: Any -> tuple[int, str]], decorated_on: [*args: Any, **kwargs: Any -> int]) -> [*args: Any, **kwargs: Any -> tuple[int, str]]
I'm not so keen on the square brackets you propose. How about we write Callable[[x, y], z] as (x, y) -> z instead? Callable[[], z] would become () -> z, and Callable[..., z] could become (...) -> z, the latter being a bit of an anomaly, but manageable, and better than (*_, **_) -> z which looks like some kind of weird emoji. :-) On Sat, Nov 28, 2020 at 8:50 AM Abdulla Al Kathiri < alkathiri.abdulla@gmail.com> wrote:
I don’t know if this has been discussed before.
Similar to PEP 645 idea of writing "Optional[type]" as “type?”, I propose we write "Callable[[type1, type2, ...], type3]” as “[type1, type2, … -> type3]”. Look at the two examples below and see which one looks better to the eyes:
def func1(f: typing.Callable[[str, int], str], arg1: str, arg2: int) -> str: return f(arg1, arg2)
def func2(f: [str, int-> str], arg1: str, arg2: int) -> str: return f(arg1, arg2)
There is less clutter especially if we have nested Callables.
e.g., f: Callable[[str, int], Callable[[int,…], str]] becomes f: [str, int -> [int, ... -> str]]
Callable without zero arguments.. f: Callable[[], str] would become f: [ -> str]
Equivalent to Callable alone without caring about arguments and the return value would be [… -> typing.Any] or [… -> ]
Let’s say we have a function that accepts a decorator as an argument. This might not be useful to do, but I want to show case how it would be easier to read. The old way would be:
def decorator(f: Callable[…, int]) -> Callable[…, tuple[int, str]]: def wrapper(*args, **kwargs) -> tuple[int, str]: text = “some text” res = f(*args, **kwargs) return res, text return wrapper
def function(decorator: Callable[[Callable[…, int]], Callable[…, tuple[int, str]]], decorated_on: Callable[…, int]) -> Callable[…, tuple[int, str]]: wrapper = decorator(decorated_on) return wrapper
The new way is as follows:
def decorator(f: [… -> int]) -> [… -> tuple[int, str]]: def wrapper(*args, **kwargs) -> tuple[int, str]: text = “some text” res = f(*args, **kwargs) return rest, text return wrapper
def function(decorator: [ [… -> int] -> [… -> tuple[int, str]]], decorated_on: [… -> int]) -> [… -> tuple[int, str]]: wrapper = decorator(decorated_on) return wrapper
I saw something similar in Pylance type checker (VSC extension) when you hover over an annotated function that has Callable as an argument or a return value, but they don’t seem to use brackets to mark the beginning and end of the callable, which could be hard to follow mentally (see screenshot below)
Personally, I think it would be easier if Pylance wrote the hint like the following:
(function) function: (decorator: [p0:[*args: Any, **kwargs: Any -> int ]] -> [*args: Any, **kwargs: Any -> tuple[int, str]], decorated_on: [*args: Any, **kwargs: Any -> int]) -> [*args: Any, **kwargs: Any -> tuple[int, str]]
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LRUXIJ... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
Initially, when I wrote this, I had a similar syntax to what you wrote. I like it, I changed it to brackets so we could contain Callable that has multiple arguments or return value as Callables themselves. E.g., function that has two Callables as arguments and a Callable return looks like this.. f: ((int, str) -> int, (…) -> str) -> (str) -> int The arguments are clear to me but the callable return confuses me a little (the last two ->’s). However, the return value does’t have parenthesis, I still could see the consistency (arguments) -> return. But my mind is focusing on the last item, int, instead of the whole (str) -> int for some reason. [[int, str -> int], [… -> str] -> [str -> int]] looks uglier but the one before last “->" clearly communicates that the return value is callable. Regardless of how it’s done, the less we depend on the typing module for annotations, the better I think because it encourages python users to annotate their functions.
On Nov 28, 2020, at 9:31 PM, Guido van Rossum <guido@python.org> wrote:
I'm not so keen on the square brackets you propose. How about we write Callable[[x, y], z] as (x, y) -> z instead? Callable[[], z] would become () -> z, and Callable[..., z] could become (...) -> z, the latter being a bit of an anomaly, but manageable, and better than (*_, **_) -> z which looks like some kind of weird emoji. :-)
On Sat, Nov 28, 2020 at 8:50 AM Abdulla Al Kathiri <alkathiri.abdulla@gmail.com <mailto:alkathiri.abdulla@gmail.com>> wrote:
I don’t know if this has been discussed before.
Similar to PEP 645 idea of writing "Optional[type]" as “type?”, I propose we write "Callable[[type1, type2, ...], type3]” as “[type1, type2, … -> type3]”. Look at the two examples below and see which one looks better to the eyes:
def func1(f: typing.Callable[[str, int], str], arg1: str, arg2: int) -> str: return f(arg1, arg2)
def func2(f: [str, int-> str], arg1: str, arg2: int) -> str: return f(arg1, arg2)
There is less clutter especially if we have nested Callables.
e.g., f: Callable[[str, int], Callable[[int,…], str]] becomes f: [str, int -> [int, ... -> str]]
Callable without zero arguments.. f: Callable[[], str] would become f: [ -> str]
Equivalent to Callable alone without caring about arguments and the return value would be [… -> typing.Any] or [… -> ]
Let’s say we have a function that accepts a decorator as an argument. This might not be useful to do, but I want to show case how it would be easier to read. The old way would be:
def decorator(f: Callable[…, int]) -> Callable[…, tuple[int, str]]:
def wrapper(*args, **kwargs) -> tuple[int, str]: text = “some text” res = f(*args, **kwargs) return res, text
return wrapper
def function(decorator: Callable[[Callable[…, int]], Callable[…, tuple[int, str]]], decorated_on: Callable[…, int]) -> Callable[…, tuple[int, str]]: wrapper = decorator(decorated_on) return wrapper
The new way is as follows:
def decorator(f: [… -> int]) -> [… -> tuple[int, str]]:
def wrapper(*args, **kwargs) -> tuple[int, str]: text = “some text” res = f(*args, **kwargs) return rest, text
return wrapper
def function(decorator: [ [… -> int] -> [… -> tuple[int, str]]], decorated_on: [… -> int]) -> [… -> tuple[int, str]]: wrapper = decorator(decorated_on) return wrapper
I saw something similar in Pylance type checker (VSC extension) when you hover over an annotated function that has Callable as an argument or a return value, but they don’t seem to use brackets to mark the beginning and end of the callable, which could be hard to follow mentally (see screenshot below)
<Screen Shot 2020-11-28 at 8.30.09 PM.png>
Personally, I think it would be easier if Pylance wrote the hint like the following:
(function) function: (decorator: [p0:[*args: Any, **kwargs: Any -> int ]] -> [*args: Any, **kwargs: Any -> tuple[int, str]], decorated_on: [*args: Any, **kwargs: Any -> int]) -> [*args: Any, **kwargs: Any -> tuple[int, str]]
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org <mailto:python-ideas@python.org> To unsubscribe send an email to python-ideas-leave@python.org <mailto:python-ideas-leave@python.org> https://mail.python.org/mailman3/lists/python-ideas.python.org/ <https://mail.python.org/mailman3/lists/python-ideas.python.org/> Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LRUXIJ... <https://mail.python.org/archives/list/python-ideas@python.org/message/LRUXIJ...> Code of Conduct: http://python.org/psf/codeofconduct/ <http://python.org/psf/codeofconduct/>
-- --Guido van Rossum (python.org/~guido <http://python.org/~guido>) Pronouns: he/him (why is my pronoun here?) <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Sat, Nov 28, 2020 at 10:02 AM Abdulla Al Kathiri < alkathiri.abdulla@gmail.com> wrote:
Initially, when I wrote this, I had a similar syntax to what you wrote. I like it, I changed it to brackets so we could contain Callable that has multiple arguments or return value as Callables themselves.
E.g., function that has two Callables as arguments and a Callable return looks like this..
f: ((int, str) -> int, (…) -> str) -> (str) -> int
The arguments are clear to me but the callable return confuses me a little (the last two ->’s). However, the return value does’t have parenthesis, I still could see the consistency (arguments) -> return. But my mind is focusing on the last item, int, instead of the whole (str) -> int for some reason.
You could parenthesize the return value if you think it's not clear. (IIRC some functional languages like Haskell use the ... -> ... -> ... notation to indicate functions of multiple arguments, so it's probably good to avoid looking like that -- even if it'sconsistent it would mislead people into thinking that Python uses a similar idiom.
[[int, str -> int], [… -> str] -> [str -> int]] looks uglier but the one before last “->" clearly communicates that the return value is callable.
Regardless of how it’s done, the less we depend on the typing module for annotations, the better I think because it encourages python users to annotate their functions.
Indeed. Shantanu did some quick counting and found that after 'Any' and the types covered by PEP 585, Callable is by far the most used: https://bugs.python.org/issue42102#msg381155 -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Nov 29, 2020, at 3:46 AM, Guido van Rossum <guido@python.org> wrote:
You could parenthesize the return value if you think it's not clear
Yeah I agree. Parenthesizing the return should be optional because if we require it, the callable arguments with parenthesized returns and the parenthesized return of the return itself would look too much. Optional Return Parenthesis: ((int, str) -> int, (…) -> str) -> ((str) -> int) VS. Required Return Parenthesis: ((int, str) -> (int), (…) -> (str)) -> ((str) -> (int)) …. Consistent but too many parenthesis
Indeed. Shantanu did some quick counting and found that after 'Any' and the types covered by PEP 585, Callable is by far the most used: https://bugs.python.org/issue42102#msg381155 <https://bugs.python.org/issue42102#msg381155> Nice survey. Can we do something with “Any" once we are at it? Maybe we use “?”. But that might confuse people and think it’s None mirroring PEP 645’s “type?”.
On 11/28/20 9:31 AM, Guido van Rossum wrote:
I'm not so keen on the square brackets you propose. How about we write Callable[[x, y], z] as (x, y) -> z instead?
I also like the "(x, y) -> z" syntax a bit better than "[x, y -> z]". (+0.5) On 11/28/20 3:46 PM, Guido van Rossum wrote:
You could parenthesize the return value if you think it's not clear. (IIRC some functional languages like Haskell use the ... -> ... -> ... notation to indicate functions of multiple arguments, so it's probably good to avoid looking like that -- even if it'sconsistent it would mislead people into thinking that Python uses a similar idiom.
To avoid the "... -> ... -> ... " ambiguity, I'd suggest that kind of chaining be recognized and treated as a syntax error. If describing a Callable that itself returned another Callable, it would then be necessary to parenthesize the return value: A function that has two Callables as arguments and a Callable return type would then look like: f: ((int, str) -> int, (…) -> str) -> ((str) -> int) (It's worth noting that I don't think either the original "Callable[[x, y], z]" syntax OR the proposed "(x, y) -> z" syntax is particularly readable when lots of nesting is involved, but I suspect that's just because the type is just complicated. :) ) On 11/28/20 10:02 AM, Abdulla Al Kathiri wrote:
Regardless of how it’s done, the less we depend on the typing module for annotations, the better I think because it encourages python users to annotate their functions.
Indeed. (+1) -- David Foster | Seattle, WA, USA
On Sat, Nov 28, 2020 at 10:22 PM Abdulla Al Kathiri < alkathiri.abdulla@gmail.com> wrote:
Indeed. Shantanu did some quick counting and found that after 'Any' and the types covered by PEP 585, Callable is by far the most used: https://bugs.python.org/issue42102#msg381155
Nice survey. Can we do something with “Any" once we are at it? Maybe we use “?”. But that might confuse people and think it’s None mirroring PEP 645’s “type?”.
Please, no more proposals to use "?". :-) There's also PEP 640, and PEP 505. Maybe we should just make Any a builtin (but please start a new thread to debate that). On Sat, Nov 28, 2020 at 10:28 PM David Foster <davidfstr@gmail.com> wrote:
(It's worth noting that I don't think either the original "Callable[[x, y], z]" syntax OR the proposed "(x, y) -> z" syntax is particularly readable when lots of nesting is involved, but I suspect that's just because the type is just complicated. :) )
You could simplify it by judicious application of type aliases. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
I would see support of all argument kinds support in any proposal for a new callable: positional only args, named args, keyword-only, *args and **kwargs. The exact notation in probably less important than missing functionality. On Sat, Nov 28, 2020, 18:50 Abdulla Al Kathiri <alkathiri.abdulla@gmail.com> wrote:
I don’t know if this has been discussed before.
Similar to PEP 645 idea of writing "Optional[type]" as “type?”, I propose we write "Callable[[type1, type2, ...], type3]” as “[type1, type2, … -> type3]”. Look at the two examples below and see which one looks better to the eyes:
def func1(f: typing.Callable[[str, int], str], arg1: str, arg2: int) -> str: return f(arg1, arg2)
def func2(f: [str, int-> str], arg1: str, arg2: int) -> str: return f(arg1, arg2)
There is less clutter especially if we have nested Callables.
e.g., f: Callable[[str, int], Callable[[int,…], str]] becomes f: [str, int -> [int, ... -> str]]
Callable without zero arguments.. f: Callable[[], str] would become f: [ -> str]
Equivalent to Callable alone without caring about arguments and the return value would be [… -> typing.Any] or [… -> ]
Let’s say we have a function that accepts a decorator as an argument. This might not be useful to do, but I want to show case how it would be easier to read. The old way would be:
def decorator(f: Callable[…, int]) -> Callable[…, tuple[int, str]]: def wrapper(*args, **kwargs) -> tuple[int, str]: text = “some text” res = f(*args, **kwargs) return res, text return wrapper
def function(decorator: Callable[[Callable[…, int]], Callable[…, tuple[int, str]]], decorated_on: Callable[…, int]) -> Callable[…, tuple[int, str]]: wrapper = decorator(decorated_on) return wrapper
The new way is as follows:
def decorator(f: [… -> int]) -> [… -> tuple[int, str]]: def wrapper(*args, **kwargs) -> tuple[int, str]: text = “some text” res = f(*args, **kwargs) return rest, text return wrapper
def function(decorator: [ [… -> int] -> [… -> tuple[int, str]]], decorated_on: [… -> int]) -> [… -> tuple[int, str]]: wrapper = decorator(decorated_on) return wrapper
I saw something similar in Pylance type checker (VSC extension) when you hover over an annotated function that has Callable as an argument or a return value, but they don’t seem to use brackets to mark the beginning and end of the callable, which could be hard to follow mentally (see screenshot below)
Personally, I think it would be easier if Pylance wrote the hint like the following:
(function) function: (decorator: [p0:[*args: Any, **kwargs: Any -> int ]] -> [*args: Any, **kwargs: Any -> tuple[int, str]], decorated_on: [*args: Any, **kwargs: Any -> int]) -> [*args: Any, **kwargs: Any -> tuple[int, str]]
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LRUXIJ... Code of Conduct: http://python.org/psf/codeofconduct/
On Sat, Nov 28, 2020 at 9:32 AM Andrew Svetlov <andrew.svetlov@gmail.com> wrote:
I would see support of all argument kinds support in any proposal for a new callable: positional only args, named args, keyword-only, *args and **kwargs. The exact notation in probably less important than missing functionality.
Hm, but using Protocol you can already express every callable type. We could duplicate all of the complexity of 'def' parameter lists into the type notation, but it would be very complex. So this requirement is at least debatable (I'm not actually sure how I feel about it). -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Sun, Nov 29, 2020 at 1:49 AM Guido van Rossum <guido@python.org> wrote:
On Sat, Nov 28, 2020 at 9:32 AM Andrew Svetlov <andrew.svetlov@gmail.com> wrote:
I would see support of all argument kinds support in any proposal for a new callable: positional only args, named args, keyword-only, *args and **kwargs. The exact notation in probably less important than missing functionality.
Hm, but using Protocol you can already express every callable type. We could duplicate all of the complexity of 'def' parameter lists into the type notation, but it would be very complex. So this requirement is at least debatable (I'm not actually sure how I feel about it).
Don't get me wrong please. I'm not the typing system developer (while use typing every day in my code). You can consider this email as end-user feedback. I have a lot of code that uses `Callable[]` annotation; it is a natural way of annotating any callable, isn't it. Sometimes `Callable` doesn't work. For example, I cannot annotate a function that accepts a keyword-only argument. Sure, I can use Protocol but it doesn't feel natural and requires more code lines, especially if the Protocol is used once or twice. Also, Protocol adds `self` to methods which looks at least confusing when I annotate plain callables which naturally don't belong to any class: class MyFunc(typing.Protocol): def __call__(self, /, arg: int) -> str: ... Moreover, I recall a pull request to only of my libraries that had something similar to: class MyProto(typing.Protocol): def __call__(self_, /, self: Class, arg: int) -> str: ... The protocol actually had two 'self': one fake self for protocol itself and another one if the *real* self. The PR was refactored to use a more *natural* style but my first reaction was eyeballs :) The last thing that is unclear for me is mixing protocols with ParamSpec from PEP-612. Sure, I can live with all the mentioned limitations and learn all the required tricks if we have a consensus that Callable should operate with positional unnamed arguments only; but allowing to write such things in a manner similar to that Python supports for function definitions already makes life easier. -- Thanks, Andrew Svetlov
That's a fair criticism -- Protocols are usable for this, but the syntax is hardly the most intuitive, and most people would never think of this approach. I guess the alternative would be to duplicate the full syntax allowed in the parameter list of a 'def' statement (with some tweaks perhaps), so the following would all be valid types: (x: float, method: int|None = None) -> str (*args: int, **kwds: int) -> int (a: int, b: int, /) -> int As for tweaks, maybe we can change things so that (int, int) -> int is equivalent to the latter (since this is presumably still a very common case). The rule would be something like "if there's no /, a / is assumed at the end", and also "in positional args, the syntax is [name ':'] type_expr ['=' default_expr]." Gotta go, On Sun, Nov 29, 2020 at 12:02 AM Andrew Svetlov <andrew.svetlov@gmail.com> wrote:
On Sun, Nov 29, 2020 at 1:49 AM Guido van Rossum <guido@python.org> wrote:
On Sat, Nov 28, 2020 at 9:32 AM Andrew Svetlov <andrew.svetlov@gmail.com> wrote:
I would see support of all argument kinds support in any proposal for a new callable: positional only args, named args, keyword-only, *args and **kwargs. The exact notation in probably less important than missing functionality.
Hm, but using Protocol you can already express every callable type. We could duplicate all of the complexity of 'def' parameter lists into the type notation, but it would be very complex. So this requirement is at least debatable (I'm not actually sure how I feel about it).
Don't get me wrong please. I'm not the typing system developer (while use typing every day in my code). You can consider this email as end-user feedback.
I have a lot of code that uses `Callable[]` annotation; it is a natural way of annotating any callable, isn't it. Sometimes `Callable` doesn't work. For example, I cannot annotate a function that accepts a keyword-only argument.
Sure, I can use Protocol but it doesn't feel natural and requires more code lines, especially if the Protocol is used once or twice. Also, Protocol adds `self` to methods which looks at least confusing when I annotate plain callables which naturally don't belong to any class:
class MyFunc(typing.Protocol): def __call__(self, /, arg: int) -> str: ...
Moreover, I recall a pull request to only of my libraries that had something similar to:
class MyProto(typing.Protocol): def __call__(self_, /, self: Class, arg: int) -> str: ...
The protocol actually had two 'self': one fake self for protocol itself and another one if the *real* self. The PR was refactored to use a more *natural* style but my first reaction was eyeballs :)
The last thing that is unclear for me is mixing protocols with ParamSpec from PEP-612.
Sure, I can live with all the mentioned limitations and learn all the required tricks if we have a consensus that Callable should operate with positional unnamed arguments only; but allowing to write such things in a manner similar to that Python supports for function definitions already makes life easier.
-- Thanks, Andrew Svetlov
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
The proposed examples look awesome, thanks! On Sun, Nov 29, 2020 at 7:27 PM Guido van Rossum <guido@python.org> wrote:
That's a fair criticism -- Protocols are usable for this, but the syntax is hardly the most intuitive, and most people would never think of this approach.
I guess the alternative would be to duplicate the full syntax allowed in the parameter list of a 'def' statement (with some tweaks perhaps), so the following would all be valid types:
(x: float, method: int|None = None) -> str (*args: int, **kwds: int) -> int (a: int, b: int, /) -> int
As for tweaks, maybe we can change things so that (int, int) -> int is equivalent to the latter (since this is presumably still a very common case). The rule would be something like "if there's no /, a / is assumed at the end", and also "in positional args, the syntax is [name ':'] type_expr ['=' default_expr]."
Gotta go,
On Sun, Nov 29, 2020 at 12:02 AM Andrew Svetlov <andrew.svetlov@gmail.com> wrote:
On Sun, Nov 29, 2020 at 1:49 AM Guido van Rossum <guido@python.org> wrote:
On Sat, Nov 28, 2020 at 9:32 AM Andrew Svetlov <andrew.svetlov@gmail.com> wrote:
I would see support of all argument kinds support in any proposal for a new callable: positional only args, named args, keyword-only, *args and **kwargs. The exact notation in probably less important than missing functionality.
Hm, but using Protocol you can already express every callable type. We could duplicate all of the complexity of 'def' parameter lists into the type notation, but it would be very complex. So this requirement is at least debatable (I'm not actually sure how I feel about it).
Don't get me wrong please. I'm not the typing system developer (while use typing every day in my code). You can consider this email as end-user feedback.
I have a lot of code that uses `Callable[]` annotation; it is a natural way of annotating any callable, isn't it. Sometimes `Callable` doesn't work. For example, I cannot annotate a function that accepts a keyword-only argument.
Sure, I can use Protocol but it doesn't feel natural and requires more code lines, especially if the Protocol is used once or twice. Also, Protocol adds `self` to methods which looks at least confusing when I annotate plain callables which naturally don't belong to any class:
class MyFunc(typing.Protocol): def __call__(self, /, arg: int) -> str: ...
Moreover, I recall a pull request to only of my libraries that had something similar to:
class MyProto(typing.Protocol): def __call__(self_, /, self: Class, arg: int) -> str: ...
The protocol actually had two 'self': one fake self for protocol itself and another one if the *real* self. The PR was refactored to use a more *natural* style but my first reaction was eyeballs :)
The last thing that is unclear for me is mixing protocols with ParamSpec from PEP-612.
Sure, I can live with all the mentioned limitations and learn all the required tricks if we have a consensus that Callable should operate with positional unnamed arguments only; but allowing to write such things in a manner similar to that Python supports for function definitions already makes life easier.
-- Thanks, Andrew Svetlov
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
-- Thanks, Andrew Svetlov
On 27/11/20 2:25 am, Steven D'Aprano wrote:
Block scoping adds semantic and implementation complexity and annoyance, while giving very little benefit.
Yet in *certain situations* it seems that block scoping is what people subconsciously assume. For example, loop variables in list comprehensions, the leaking of which unsettled enough people that it got changed. -- Greg
On Fri, Nov 27, 2020 at 01:17:07PM +1300, Greg Ewing wrote:
On 27/11/20 2:25 am, Steven D'Aprano wrote:
Block scoping adds semantic and implementation complexity and annoyance, while giving very little benefit.
Yet in *certain situations* it seems that block scoping is what people subconsciously assume. For example, loop variables in list comprehensions, the leaking of which unsettled enough people that it got changed.
I was one of those people who *liked* the fact that list comp variables leaked :-) I've come around to accept that it's okay that they don't, and that conceptually comprehensions feel like a self-contained scope, more like a function than an ordinary expression. Especially since the walrus operator now allows me to leak variables out of a comprehension if I really need to. We can have too many scopes as well as too few: - a single process-wide global scope is too few; - every (sub-)expression being its own scope is too many; so we're just arguing about where the Goldilocks Zone is. My argument is that it is *function scope*, with comprehensions considered to be an honorary function for that purpose. -- Steve
Hello, On Fri, 27 Nov 2020 21:21:48 +1100 Steven D'Aprano <steve@pearwood.info> wrote: []
We can have too many scopes as well as too few:
- a single process-wide global scope is too few;
- every (sub-)expression being its own scope is too many;
so we're just arguing about where the Goldilocks Zone is. My argument is that it is *function scope*, with comprehensions considered to be an honorary function for that purpose.
So, the alternative opinion you hear is that we could allow *optional* block-level scoping. People who don't need it can continue to not have it. Nor it's supposed to be too-widely used feature. (Yeah, just the same as str.format(), annotations, f"", or dataclasses - all they should be strictly optional, adhoc-use features ;-) ). -- Best regards, Paul mailto:pmiscml@gmail.com
On Fri, Nov 27, 2020 at 01:36:00PM +0300, Paul Sokolovsky wrote:
So, the alternative opinion you hear is that we could allow *optional* block-level scoping.
No such thing. Even if I don't use it myself, I still have to read code that uses it, I need to learn about it, there will surely be times I'm required to maintain someone else's code that uses it. When I'm answering people's questions, or teaching newbies, I still have to deal with this. If I'm reviewing a colleague's code, I'll probably see it. If I ask for help on some web forum, I'll probably get solutions that involve this feature. If it has implementation costs, in the size of the interpreter, or runtime costs, I pay those costs whether I use the feature myself or not. There are opportunity costs: every minute a developer is implementing or maintaining this feature is a minute that they aren't working on something else. I can't opt out of the costs of new features, even if I don't use them myself. This is why we have a duty to try to ensure that new features have more benefits than costs. This might not be a particularly high bar to reach: if the implementation is simple, the maintenance needs negligible, the runtime costs small, and the concept turns out to be easier to learn and teach than I expect, then the required benefits needed to give a positive balance are likewise reduced. Or on the other hand, it might be that the benefits are small and rare, and the costs large and significant. The hard cases are when the pros and cons are of comparable size and it comes down to a personal judgement which is bigger. -- Steve
On Fri, Nov 27, 2020 at 10:33:17PM +1100, Steven D'Aprano wrote:
On Fri, Nov 27, 2020 at 01:36:00PM +0300, Paul Sokolovsky wrote:
So, the alternative opinion you hear is that we could allow *optional* block-level scoping.
No such thing.
That dismissal (which made sense in my head when I wrote it) fails to survive on re-reading it. Sorry. Of course it could be optional in the sense of being explicitly opt-in, e.g. with `let` assignments, rather than mandatory. But it won't be optional in the important sense that I went on to describe, namely that even if I personally don't use this feature, I'll still have to deal with it. -- Steve
On Thu, Nov 26, 2020 at 3:11 AM Paul Sokolovsky <pmiscml@gmail.com> wrote:
[...]
Hi Paul, I think there is a case to be made for adding "let" to Python. Would you like to write a PEP? I recommend that you find a co-author who can help you write a convincing motivation to sway the Steering Council (not me). (Why do I propose "let" and not "const"? It seems "let" has become the industry standard language. The other day I saw a reference to a LET() function in Excel. :-) -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
Hello Guido, On Thu, 26 Nov 2020 14:18:20 -0800 Guido van Rossum <guido@python.org> wrote:
On Thu, Nov 26, 2020 at 3:11 AM Paul Sokolovsky <pmiscml@gmail.com> wrote:
[...]
Hi Paul,
I think there is a case to be made for adding "let" to Python. Would you like to write a PEP? I recommend that you find a co-author who can help you write a convincing motivation to sway the Steering Council (not me).
Thanks for encouragement, but process-wise, I think it would be rejected just because it's written by me ;-). So, I'd prefer it were written by somebody else. That said, even rejected or permanently deferred PEPs still have their value, so I'll keep that in mind, for a case that nobody beats me on it.
(Why do I propose "let" and not "const"? It seems "let" has become the industry standard language. The other day I saw a reference to a LET() function in Excel. :-)
Note that many functional languages conflate the matter of introducing new variable binding (which are lexically nested per the computation theory best practices) and immutability. For an imperative language like Python we shouldn't fall into that trap. If anything, the mental model for "let" should be ol' good BASIC, where "LET", well, introduces a variable (well, it's semantically null, the point that there's no "immutability" connotation). JavaScript now has both "let" and "const", and I think there's a good case to have both in Python too. (As a disclaimer, I'm not a JavaScript user, nor avid JavaScript spec reader. I do the latter strictly on demand, "to not pick up bad ideas". And I'm currently at the stage of intuitive contemplation of why both "let" and "const" may be useful for languages like JavaScript and Python, and only then going to compare my intuition with what's actually implemented in JavaScript. Of course, doing this and many other homeworks are prerequisites for writing the PEP.) -- Best regards, Paul mailto:pmiscml@gmail.com
On 27/11/20 12:11 am, Paul Sokolovsky wrote:
On Thu, 19 Nov 2020 18:53:01 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Note that this is *not* the same as introducing a new scope.
And that's very sad. That means instead of solving the root of the problem, adhoc workarounds are proliferated.
I'd be open to the idea of a variation on the for loop that does introduce a new scope for the loop variable. It would need new syntax, though. I didn't suggest it initially because proposals for subscopes within functions typically meet with a lot of resistance, and I didn't want to get sidetracked arguing about that when it isn't strictly needed.
Million cells for million of iterations?
If your million iterations create a million nested functions that capture the variable and are all kept alive, then yes, you will get a million cells along with your million function objects. If the loop doesn't have any nested functions, or they don't capture the loop variable, there will be 0 cells. Note that the cells would still be needed even if there were a nested scope.
I'd suggest that it should be "for let"
That makes no sense as a phrase in English.
the scoping should be optimized to always treat "x" (as the "for" control variable) by-value,
That would make it very different from the way closures work anywhere else in the language.
for const x in some_iterator: ...
That will make it obvious that "x" is captured by value,
and that:
for const x in some_iterator: ... x = 1
Doesn't make sense (should lead to error - assigning to a const).
This is mixing up two different issues -- the scope of the loop variable, and whether it can be assigned to. I proposed "new" because it directly gets at the important thing, which is that x is a *different* x each time round the loop.
So, there shouldn't be "special syntax". There should be generic syntax to solve Python's block scoping and const'ness issues.
Which people would then have to remember to use. By "special" in that context I mean "something you have to do differently to get your for loop to work the way you expect". Not necessarily something that can only be used with for loops. -- Greg
On Fri, Nov 27, 2020 at 11:08 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I'd suggest that it should be "for let"
That makes no sense as a phrase in English.
Nor do lots of other constructs when they get combined. English doesn't really have good parallels for most computing concepts. How will this "new assignment target" work with different forms of assignment targets? For instance: for let k, stuff[k] in d.items(): ... ChrisA
On 27/11/20 1:23 pm, Chris Angelico wrote:
That makes no sense as a phrase in English.
Nor do lots of other constructs when they get combined. English doesn't really have good parallels for most computing concepts.
Python manages to not be wildly ungrammatical with the bits of English that it borrows, though.
How will this "new assignment target" work with different forms of assignment targets? For instance:
for let k, stuff[k] in d.items(): ...
I was thinking it could only be put before a bare name and would only apply to that name. So if there are multiple targets in the assignment, you would need to mark each one that you want treated. -- Greg
On Fri, Nov 27, 2020 at 4:34 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 27/11/20 1:23 pm, Chris Angelico wrote:
That makes no sense as a phrase in English.
Nor do lots of other constructs when they get combined. English doesn't really have good parallels for most computing concepts.
Python manages to not be wildly ungrammatical with the bits of English that it borrows, though.
How will this "new assignment target" work with different forms of assignment targets? For instance:
for let k, stuff[k] in d.items(): ...
I was thinking it could only be put before a bare name and would only apply to that name. So if there are multiple targets in the assignment, you would need to mark each one that you want treated.
That'd be my expectation, too. Which would mean you could have: for k, let v in d.items() As such, I don't think it needs to be read as "for let". It's more "let v". Maybe I'm weird for having spent a lot of time writing "for (int i = 0; i < n; ++i)" in C++, but I don't see inline variable tags as being overly problematic :) But if it can be done in the loop header, and it's actually a part of the assignment part (rather than being a keyword applying to the entire loop), there's going to be people expecting to be able to do this: for let i in collections.Counter(): let val = input("> ") def foo(): print(i, val) functions.append(foo) where the "let" could be equivalently applied to ANY variable name or assignment. IMO that will need to be addressed in the PEP (either for or against, but not silent). ChrisA
Hello, On Fri, 27 Nov 2020 18:32:54 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 27/11/20 1:23 pm, Chris Angelico wrote:
That makes no sense as a phrase in English.
Nor do lots of other constructs when they get combined. English doesn't really have good parallels for most computing concepts.
Python manages to not be wildly ungrammatical with the bits of English that it borrows, though.
I'm not a native English speaker, but for me, "for let something in sequence" doesn't sound too bad. If spoken as a natural language, it would need intonation and pauses, something like: "for... let something be each element in sequence ... repeat ..." And mind that "for let" was a "draft" proposal in my initial post, immediately replaced by "for const", which exactly both readable and intuitive for what it does (e.g. the fact that it uses/passes variable(s) by value, so no cells are needed).
How will this "new assignment target" work with different forms of assignment targets? For instance:
for let k, stuff[k] in d.items(): ...
I was thinking it could only be put before a bare name and would only apply to that name. So if there are multiple targets in the assignment, you would need to mark each one that you want treated.
When I wrote about it, my mental model was that "for let" or "for const" is an atomic syntactic construct akin to "async for". I also would like to remind that the backing idea behind both is that they introduce a new block scope ;-). Then it's clear that their arguments can be only bare variable names ("for let stuff[k] in ..." is a syntax error, there's normal "for stuff[k] in ..." remains for that). And both "for let" and "for const" are clearly not needed, and between them, "for const" is much better. (But "let" can be useful in other places, ditto for "const").
-- Greg
-- Best regards, Paul mailto:pmiscml@gmail.com
Hello, On Fri, 27 Nov 2020 13:06:56 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 27/11/20 12:11 am, Paul Sokolovsky wrote:
On Thu, 19 Nov 2020 18:53:01 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Note that this is *not* the same as introducing a new scope.
And that's very sad. That means instead of solving the root of the problem, adhoc workarounds are proliferated.
I'd be open to the idea of a variation on the for loop that does introduce a new scope for the loop variable. It would need new syntax, though.
Yes, that's what we talked about with "for let" (stroked thru) and "for const" ideas. Before responding, I decided to try to eat my own dogfood and prototype block-level scoping for "for". Good news is that I was able to hack up something. It still needs cleanup before posting though. (As a sneak preview, it's *of course* not written in C.)
I didn't suggest it initially because proposals for subscopes within functions typically meet with a lot of resistance, and I didn't want to get sidetracked arguing about that when it isn't strictly needed.
And I'm here because seemingly unrelated topics quite visibly converge on the (fundamental) topics of (finer-grained) scoping and const'ness. E.g. the previous one was just a couple of weeks ago in relation to pattern matching: https://mail.python.org/archives/list/python-dev@python.org/message/MUCUIUF5... (I kinda dropped the ball with answering it, but now hope to). Here again they both come up in relation to the issues with "for". It's time maybe to make following observations: 1. Python tried hard to avoid block-level scoping altogether. 2. It failed, the cases of comprehension and "except" block are the witnesses. 3. Those cases were implemented in adhoc way (in one, a whole new function scope and call to it introduced, in another, there's a flat-world emulation with just del'ing a variable without caring that it kills the variable in the overall scope). 4. On the other hand, block-level scoping is well-known, well-explored, and popular scoping discipline in other programming languages, with solid formal model behind it. (That being that each block is well, a function in lambda-calculus. But that's conceptual model, the implementation can be more efficient). 5. Maybe it's time to embrace proper block-level scoping as means to address Python semantic issues (up to enabling its explicit and generic usage)?
Million cells for million of iterations?
If your million iterations create a million nested functions that capture the variable and are all kept alive, then yes, you will get a million cells along with your million function objects.
If the loop doesn't have any nested functions, or they don't capture the loop variable, there will be 0 cells.
That's good answers for the "coding time". (I can even continue: "... And if you create inner functions in a loop, you probably don't run it million of times. And if you do, there's already plenty of overhead due to allocation of such an inner function object anyway"). But what I argue for is to not rush with coding yet another adhoc workaround, like already were for comprehensions/excepts, but spend more time in the "design" phase, trying to find a fundamental solution which could cover many cases.
Note that the cells would still be needed even if there were a nested scope.
I'm not sure if there was a better word to use somewhere in that phrase. Anyway, cells are needed *only* because the default semantics allows mutability of variables (and this semantics makes good sense for imperative language). But if we know that some variable is immutable, we don't need a cell, we can capture the *value* of the variable in the closure. That's what "for const var in ..." (and "const var = ...") syntax expresses.
I'd suggest that it should be "for let"
That makes no sense as a phrase in English.
Was covered in other replies.
the scoping should be optimized to always treat "x" (as the "for" control variable) by-value,
That would make it very different from the way closures work anywhere else in the language.
But that's exactly the main point of what I'm talking about. We should not go for random syntactic/semantic patchings here and there. Instead, we should consider what fundamental notions are missing in Python semantics, and that's: 1. Block-level scope. 2. Immutability - and then see how to apply them in general way to the language as a whole (in a sense "to allow them used 'everywhere'", not "force their usage on everyone"). In this regard: --- def foo(): for const x in seq: f = lambda: print(x) --- and --- def foo(): const x = 1 f = lambda: print(x) --- - will both capture "x" by value. In other words, "const" will consistently work everywhere in the language.
for const x in some_iterator: ...
That will make it obvious that "x" is captured by value,
and that:
for const x in some_iterator: ... x = 1
Doesn't make sense (should lead to error - assigning to a const).
This is mixing up two different issues -- the scope of the loop variable, and whether it can be assigned to.
There was no mix-up on my side, and neither seems there was on yours. Block-level scoping and const'ness are orthogonal, well composable features. In the case of "for" they exactly play well together.
I proposed "new" because it directly gets at the important thing, which is that x is a *different* x each time round the loop.
So, there shouldn't be "special syntax". There should be generic syntax to solve Python's block scoping and const'ness issues.
Which people would then have to remember to use. By "special" in that context I mean "something you have to do differently to get your for loop to work the way you expect". Not necessarily something that can only be used with for loops.
The way to not make it hard for people to remember is to make the features generic, fundamental, reusable, composable. "for const" on its own doesn't make much sense (and neither "for new", i.e. adhoc workaround for the "for" case). Implementation-wise, "for const" might be a pilot, but the general vision should be the availability of "const" (and eventually, block scoping ;-)) in the language in general.
-- Greg
[] -- Best regards, Paul mailto:pmiscml@gmail.com
On 29/11/20 4:14 am, Paul Sokolovsky wrote:
On Fri, 27 Nov 2020 13:06:56 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
There was no mix-up on my side, and neither seems there was on yours. Block-level scoping and const'ness are orthogonal, well composable features.
Someone (maybe not you) suggested "for const i = ..." as a way to address the problem of variable capture in for loops. The logic behind this seems to be: * If i is const, rebinding it each time round the loop is obviously impossible, therefore the loop body must be a new scope in which there is a different i for each iteration. * Since i is const, we don't need to capture it as a variable, so as an optimisation we can just capture its value. But this makes constness and new scopes non-orthogonal, because there's no way to have a new scope for a variable without making it const, and no way to make a variable const without creating a new scope for it. I also think that "const" is not an *obvious* solution to someone faced with the for-loop problem. They're not trying to reassign the variable anywhere, so there's no reason for constness to come to mind.
"for const" on its own doesn't make much sense (and neither "for new", i.e. adhoc workaround for the "for" case).
It's not so ad-hoc if "new" can be applied to any assignment. Also, if you think "for new" doesn't make sense, you're probably reading it incorrectly. The mental bracketing isn't "(for new) i", it's "for (new i)". This works with "new" and "const", because they're adjectives. But "let" isn't an adjective, so "for let i" doesn't make sense however you parse it. -- Greg
I'm definitely being nerd-sniped here, so take this with a grain of salt. The reason why Python currently doesn't have block scopes is the rule that assignment creates or updates a variable in the closest scope. This rule has taken Python far, because it means you can create a local variable by assigning to it without having to declare it. If we wanted to introduce something similar to C's block scopes, regardless of syntax, we'd have to answer the question where local variables will be declared. Suppose we had the (crummy) syntax block var1 = exp1, var2 = exp2, ...: <statements> as the moral equivalent to C's { int var1 = exp1, var2 = exp2, ...; <statements> } then we'd still have to decide what should happen to assignments to variables *other* than var1, var2, etc. Consider def f(x): block a = 1: x = a # Is this local to the block or to the function? return x print(f(0)) # 0 or 1? IMO, if 'x = a' were to create a new variable in the block, I think that would be confusing. It would also beg the question why we bother to adorn the block with variable declarations. There are other good reasons why any variable *not* explicitly declared as being part of a block should have function scope: otherwise it would be impossible for code in a block to have a side effect to be consumed by the code outside the block. That could be addressed by using nonlocal, but at that point we might as well use a nested class statement (at least that gives us a way to get the locals out, as attributes of said class). Certainly it would be a shame if we added a block scope concept and we couldn't upgrade the for-loop to make use of it, optionally (like you can do in C with "for (int i = 0; i < n; i++) { ... }"). And a for-loop that can't set other variables that survive the loop seems crippled. So maybe we should go with Paul's original gut feelings and introduce let var1 = exp1, var2 = exp2, ...: <statements> with the semantics that it doesn't really create a fully-fledged new scope but defines scoped constants. It then would make sense to add some other syntax extensions like for let i in ...: <statements> (makes i local to the loop, giving it the semantics needed by closures as well) and with contextmanager() as let x: <statements> Clearly we need to bikeshed more about the syntax. We could then declare that comprehensions implicitly use 'for let ...'. If we were to drive this through quickly enough we could even make it apply to pattern matching captures. (Or maybe pattern matching could get these semantics anyway.) My bus is here, so that's it, -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
Hello, On Sat, 28 Nov 2020 18:59:14 -0800 Guido van Rossum <guido@python.org> wrote:
I'm definitely being nerd-sniped here, so take this with a grain of salt.
The reason why Python currently doesn't have block scopes is the rule that assignment creates or updates a variable in the closest scope. This rule has taken Python far, because it means you can create a local variable by assigning to it without having to declare it.
And everybody loves Python for that. Nothing beats Python in being able to take a 15-line algorithm in pseudocode, and write it in a way that it still looks like a pseudocode but also actually runs. But there're also 100-line and 200-line algorithms. And Python isn't used just as a "pseudocode", but as a real implementation language for software systems of different scales. And such usage over long time, identified some, well, weaknesses. Some of them were addressed, but arguably, in somewhat adhoc ways. And now there's an idea that maybe it's time to address even wider scope of issues in a more general way.
If we wanted to introduce something similar to C's block scopes, regardless of syntax, we'd have to answer the question where local variables will be declared.
Perhaps, we can start with saying "there won't be any breakages to already existing scoping rules" (well, maybe rough-edge cleanups for cases where adhoc workarounds were already applied).
Suppose we had the (crummy) syntax
block var1 = exp1, var2 = exp2, ...: <statements>
as the moral equivalent to C's
{ int var1 = exp1, var2 = exp2, ...; <statements> }
then we'd still have to decide what should happen to assignments to variables *other* than var1, var2, etc.
Simple answer: nothing. They will work as before.
Consider
def f(x): block a = 1: x = a # Is this local to the block or to the function? return x
print(f(0)) # 0 or 1?
IMO, if 'x = a' were to create a new variable in the block, I think that would be confusing.
Not just "confusing", it would break backward compatibility. So, no go.
It would also beg the question why we bother to adorn the block with variable declarations.
Indeed! So, we would take hint from existing Python variable declarations, like "global" and "nonlocal", and would write it like (let's imagine for a moment that we use "let" to introduce block-local *mutable* variables): def f(x): if 1: # We can bikeshed how to make this more "beautiful" later. let a = 1 x = a # Is this local to the block or to the function? return x
There are other good reasons why any variable *not* explicitly declared as being part of a block should have function scope:
Backward compatibility is at the top of them, I'm sure.
otherwise it would be impossible for code in a block to have a side effect to be consumed by the code outside the block. That could be addressed by using nonlocal, but at that point we might as well use a nested class statement (at least that gives us a way to get the locals out, as attributes of said class). Certainly it would be a shame if we added a block scope concept and we couldn't upgrade the for-loop to make use of it, optionally (like you can do in C with "for (int i = 0; i < n; i++) { ... }"). And a for-loop that can't set other variables that survive the loop seems crippled.
So maybe we should go with Paul's original gut feelings and introduce
let var1 = exp1, var2 = exp2, ...: <statements>
with the semantics that it doesn't really create a fully-fledged new scope but defines scoped constants.
... except not constants, by *mutable* variables, ("const" would introduce constants). And making it a suite-starting statement also seems like bowing too much at the side of the functional languages (I mean, typical syntax used for let's in them). "let", "const", are really look similar to "global" and "const", except that they allow an assignment to follow. (It's debatable whether multiple vars per one "let" should be allowed.)
It then would make sense to add some other syntax extensions like
for let i in ...: <statements>
for const i in ...:
(makes i local to the loop, giving it the semantics needed by closures as well) and
with contextmanager() as let x: <statements>
Likely also const.
Clearly we need to bikeshed more about the syntax.
We could then declare that comprehensions implicitly use 'for let ...'. If we were to drive this through quickly enough we could even make it apply to pattern matching captures. (Or maybe pattern matching could get these semantics anyway.)
My bus is here, so that's it,
Thanks!
-- --Guido van Rossum (python.org/~guido)
-- Best regards, Paul mailto:pmiscml@gmail.com
Hello, On Sun, 29 Nov 2020 13:39:37 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 29/11/20 4:14 am, Paul Sokolovsky wrote:
On Fri, 27 Nov 2020 13:06:56 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
There was no mix-up on my side, and neither seems there was on yours. Block-level scoping and const'ness are orthogonal, well composable features.
Someone (maybe not you) suggested "for const i = ..." as a way to address the problem of variable capture in for loops.
I proposed "for const". I didn't propose to use equal sign "=" in the "for" ;-).
The logic behind this seems to be:
* If i is const, rebinding it each time round the loop is obviously impossible, therefore the loop body must be a new scope in which there is a different i for each iteration.
Please watch your terminology. Because it's exactly right, but implications you make doesn't seem to match the terms. If something is const, it means that you can't *assign* to it. However, the semantics of "for" is that it executes the loop body with "i" *bound* to a new value each time. This is pretty much corresponds to what you said a few messages ago. Except you proposed to implement that literally, and create a new cell for each new value of "i". What I propose is to take advantage that binding of "i" is constant (indeed, formally const'ness applies to binding, not to value bound to), which allows to optimize away the need to create multiple cells.
* Since i is const, we don't need to capture it as a variable, so as an optimisation we can just capture its value.
Right.
But this makes constness and new scopes non-orthogonal, because there's no way to have a new scope for a variable without making it const, and no way to make a variable const without creating a new scope for it.
That's why we don't limit our discussion to just the case of the "for", and consider it in wider language scope, to avoid temptation of choosing a non-generic solution for just "for". But for the situation of "for", we can "strike out" those generic cases which don't apply (well, don't make much sense) to it. So, let's see the full matrix of the "plausible world": Case 1: x = 123 for x in seq: print(x) # Possible, doesn't make much sense, but always has been there x = 1 # Prints last item in "seq" print(x) Case 2: x = 123 for let x in seq: print(x) # Possible, doesn't make much sense x = 1 # Prints 123 print(x) Case 3: x = 123 for const x in seq: print(x) # Error # Prints 123 print(x) x = 1 So, in the plausible domain, there would be "for let", which would introduce a loop variable as mutable (just like classical "for" does). But again, it doesn't make much sense to assign to a loop control var (it will be overwritten on the next loop iter). To emphasize the fact that loop variable belongs to the loop statement (and not to a user), we put our full bet on the "for const" syntax, and don't implement "for let" at all. But it's only part of the big picture, because we have (even if eventually): Case 4: let x = 1 # Possible x = 2 Case 5: const x = 1 # Error x = 2
I also think that "const" is not an *obvious* solution to someone faced with the for-loop problem. They're not trying to reassign the variable anywhere, so there's no reason for constness to come to mind.
It will be much more obvious if there's a general (standalone) "const", that's why I'm saying we can't really consider "for const" without just "const" (even though implementation-wise, one could appear a version or 2 earlier). And it's "pretty obvious" to someone who considered various choices and saw pieces falling into their places. Also might be pretty obvious for someone who used other languages. It's certainly not a replacement for reading docs, tutorials, and changelogs.
"for const" on its own doesn't make much sense (and neither "for new", i.e. adhoc workaround for the "for" case).
It's not so ad-hoc if "new" can be applied to any assignment.
So assuming conceptual part is clear(er), let's speak of raw words. Among 3 of "new", "const", "let", considering the use as a *standalone* keyword (as in "applied to any assignment") - "new" seems to be the worst choice, sorry. Sorry but even I (a cautious guy) use "new" as a variable name *often*. Yeah, we might address that with over-modern parsers with exponential performance and memory usage, but I hope the point is clear. Extra nag is that "new" is the well-known keyword in other languages, with a different meaning (creating a new object). I bet I never used "const" as a variable name, but on a bad day, I could. Well, we'll need to deal with that (I vote for non-NP solution!!1). There're not many viable alternatives. I guess we won't take a pun from Rust for example with something like "immut". "let" should be as safe as an introduction of a new keyword can be. In no way I could use it as a variable name (because it's verb, not a noun), though I bet grepping around will show us a few "let" functions/methods. Another issue is that "let" may confuse people regarding its (im)mutability, like it clearly does in the case of BDFL. Well, I again don't see a viable alternative. A common "var" is also widely used variable name in Python. Overall, this is a rare case when I hang my hopes on JavaScript to save us from this dilemma.
Also, if you think "for new" doesn't make sense, you're probably reading it incorrectly. The mental bracketing isn't "(for new) i", it's "for (new i)".
I'm much less tied to words than to concepts and implementation details. So, when I say "for new", I mean "implementation which creates a new cell on each iteration".
This works with "new" and "const", because they're adjectives. But "let" isn't an adjective, so "for let i" doesn't make sense however you parse it.
So, stroke of luck for us that we don't need "for let", only "for const".
-- Greg
-- Best regards, Paul mailto:pmiscml@gmail.com
On 29/11/20 11:02 pm, Paul Sokolovsky wrote:
It will be much more obvious if there's a general (standalone) "const",
I don't think it will. There's nothing about the problem that points towards constness as a solution, so it doesn't matter how many other places in the language "const" appears. And even if you're told about it, you need two or three steps of reasoning to understand *why* it solves the problem.
that's why I'm saying we can't really consider "for const" without just "const"
I agree with that.
And it's "pretty obvious" to someone who considered various choices and saw pieces falling into their places. Also might be pretty obvious for someone who used other languages.
I strongly suspect it's something that's obvious only in hindsight. -- Greg
This is a massively less ambitious idea, and doesn't solve the original problem, but: I'd like to see a scope for names that are "obviously ;-)" meant to be short lived. At the moment that's the for loop "counters" and those created by context managers: for i in something: # use i as usual more_code # now i is not defined This would be a lot like comprehensions, and for the same reasons (though less compelling) And: with something as a_name: # use a_name here more_code # a_name is not defined. This just plain makes "natural" sense to me -- a context manager creates, well, a context. Various cleanup is done at the end of the block. The variable was created specifically for that context. It's really pretty odd that it hangs around after the context manager is done -- deleting that name would seem a natural part of the cleanup a context manager does. This one seems more compelling than the for loop example. Consider probably the most commonly used context manager: with open(a_path, 'w') as output_file: output_file.write(some-stuff) # whatever else now we have a closed, useless, file object hanging around -- all keeping that name around does is keep that file object from being freed. The challenge, of course, is that this would be a backward incompatible change. I don't think it would break much code (though a lot more for for than with) but maybe it could be enabled with a __future__ import. In any case, I would much rather have to opt-in to keep that variable around than opt out with an extra let or something. -CHB On Sun, Nov 29, 2020 at 3:59 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 29/11/20 11:02 pm, Paul Sokolovsky wrote:
It will be much more obvious if there's a general (standalone) "const",
I don't think it will. There's nothing about the problem that points towards constness as a solution, so it doesn't matter how many other places in the language "const" appears.
And even if you're told about it, you need two or three steps of reasoning to understand *why* it solves the problem.
that's why I'm saying we can't really consider "for const" without just "const"
I agree with that.
And it's "pretty obvious" to someone who considered various choices and saw pieces falling into their places. Also might be pretty obvious for someone who used other languages.
I strongly suspect it's something that's obvious only in hindsight.
-- Greg _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KKMR7Z... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
Hello, On Sun, 29 Nov 2020 18:53:57 -0800 Christopher Barker <pythonchb@gmail.com> wrote:
This is a massively less ambitious idea, and doesn't solve the original problem, but:
I'd like to see a scope for names that are "obviously ;-)" meant to be short lived. At the moment that's the for loop "counters" and those created by context managers:
for i in something: # use i as usual more_code # now i is not defined
What other language(s) implement such a scoping discipline? I know of none. On the other hand, block-scoped variables are implemented in: * C * C++ * Java * Rust * Lua * JavaScript (not by default, as opt-in) * Scheme * Common Lisp (as opt-in) * ML * Ocaml * Haskell * very long list of other languages... The aim of the block scoping proposal is to make Python *not worse* than these other languages, instead of adding funny workarounds again. [] -- Best regards, Paul mailto:pmiscml@gmail.com
On Mon, 30 Nov 2020 at 08:03, Paul Sokolovsky <pmiscml@gmail.com> wrote:
On the other hand, block-scoped variables are implemented in:
* C * C++ * Java * Rust * Lua * JavaScript (not by default, as opt-in) * Scheme * Common Lisp (as opt-in) * ML * Ocaml * Haskell * very long list of other languages...
How many of these languages don't require any sort of variable declaration for all but a tiny minority of variables?
The aim of the block scoping proposal is to make Python *not worse* than these other languages, instead of adding funny workarounds again.
One of the ways in which Python is *better* than these languages is in not requiring variables to be declared. Let's not make the proposed Python *worse* than the current version of Python, by making variable declarations common. Paul
Hello, On Mon, 30 Nov 2020 08:11:07 +0000 Paul Moore <p.f.moore@gmail.com> wrote:
On Mon, 30 Nov 2020 at 08:03, Paul Sokolovsky <pmiscml@gmail.com> wrote:
On the other hand, block-scoped variables are implemented in:
* C * C++ * Java * Rust * Lua * JavaScript (not by default, as opt-in) * Scheme * Common Lisp (as opt-in) * ML * Ocaml * Haskell * very long list of other languages...
How many of these languages don't require any sort of variable declaration for all but a tiny minority of variables?
I don't see to what your question applies. The proposal to introduce block-scope variables in Python relies on a special keyword to introduce them. So, any language with support for block-scoped vars would require "declaration", Python including. (But Python and some other languages keep non-block-scoped vars without any decls.)
The aim of the block scoping proposal is to make Python *not worse* than these other languages, instead of adding funny workarounds again.
One of the ways in which Python is *better* than these languages is in not requiring variables to be declared. Let's not make the proposed Python *worse* than the current version of Python, by making variable declarations common.
They won't be common, if people don't find common need for them (they shouldn't). If they do... oh, people!
Paul
-- Best regards, Paul mailto:pmiscml@gmail.com
On Mon, 30 Nov 2020 at 08:29, Paul Sokolovsky <pmiscml@gmail.com> wrote:
Hello,
On Mon, 30 Nov 2020 08:11:07 +0000 Paul Moore <p.f.moore@gmail.com> wrote:
On Mon, 30 Nov 2020 at 08:03, Paul Sokolovsky <pmiscml@gmail.com> wrote:
On the other hand, block-scoped variables are implemented in:
* C * C++ * Java * Rust * Lua * JavaScript (not by default, as opt-in) * Scheme * Common Lisp (as opt-in) * ML * Ocaml * Haskell * very long list of other languages...
How many of these languages don't require any sort of variable declaration for all but a tiny minority of variables?
I don't see to what your question applies. The proposal to introduce block-scope variables in Python relies on a special keyword to introduce them. So, any language with support for block-scoped vars would require "declaration", Python including. (But Python and some other languages keep non-block-scoped vars without any decls.)
To cover some of the languages I know, C, C++ and Java require *every* variable to be declared. Rust requires every variable to be declared, at least to the point of using "let". You can't introduce a variable just by using it. In Lua, variables introduced by assignment are global, even function-local variables need "local". Conversely, in Python, x=12 introduces a function-local name x, if used in a function. Without any sort of "declaration", or keyword. You seem to be treating "block-scoped" variables as different from function-local variables. I know of *no* other language that makes such a distinction, much less requiring different syntax for them.
The aim of the block scoping proposal is to make Python *not worse* than these other languages, instead of adding funny workarounds again.
One of the ways in which Python is *better* than these languages is in not requiring variables to be declared. Let's not make the proposed Python *worse* than the current version of Python, by making variable declarations common.
They won't be common, if people don't find common need for them (they shouldn't). If they do... oh, people!
So you're saying this proposal is to add new syntax and semantics to Python for an uncommon situation? You can't have it both ways. Is this proposal useful to a lot of people (and hence worth new syntax/semantics) or is it uncommon (and as a result, likely not worth the disruption). I'm ambivalent, with a bias towards saying that we don't need this, to the proposal. But I'm finding your arguments confusing and inconsistent. It feels like it's almost impossible to discuss the specifics of the proposal, as you're not taking a consistent position to debate against. Paul
Hello, On Mon, 30 Nov 2020 08:48:21 +0000 Paul Moore <p.f.moore@gmail.com> wrote: []
On the other hand, block-scoped variables are implemented in: * C * C++ * Java * Rust * Lua * JavaScript (not by default, as opt-in) * Scheme * Common Lisp (as opt-in) * ML * Ocaml * Haskell * very long list of other languages...
How many of these languages don't require any sort of variable declaration for all but a tiny minority of variables?
I don't see to what your question applies. The proposal to introduce block-scope variables in Python relies on a special keyword to introduce them. So, any language with support for block-scoped vars would require "declaration", Python including. (But Python and some other languages keep non-block-scoped vars without any decls.)
To cover some of the languages I know,
C, C++ and Java require *every* variable to be declared. Rust requires every variable to be declared, at least to the point of using "let". You can't introduce a variable just by using it. In Lua, variables introduced by assignment are global, even function-local variables need "local".
Conversely, in Python, x=12 introduces a function-local name x, if used in a function. Without any sort of "declaration", or keyword.
That feature stays.
You seem to be treating "block-scoped" variables as different from function-local variables.
Literally, block-scoped vars are block-scoped, and require special syntax to introduce them. Do *you* treat them much differently beyond that?
I know of *no* other language that makes such a distinction, much less requiring different syntax for them.
Well, Python is special ;-). More seriously, JavaScript have had the same scoping discipline as Python - variables are function-local, though their introduction requires "var" keyword. But recently, that changed. Well, how recently - a few years ago, now unalienable part of JavaScript, with Python falling behind its biggest competitor. So what they did: So, any language is in dire need of immutable variables (some have such by default, some only such). JavaScript needed them too (dynamic languages actually need them more than static). So, they introduced it. And they introduced it. But per the modern best practices, they introduced it block-scoped. So, they introduced block-scoped immutable variables. Ant to not leave accidental gap (https://en.wikipedia.org/wiki/Accidental_gap) in the language, they also introduced mutable block-scoped, with the "let" keyword. Now, the "const" is the great hit in JavaScript (heck, it was in the *dire* need). What about "let"? Well, it's there, so that the *current* JavaScript designers aren't laughed at for designing it in 2 weeks while on holiday drinking things. It has its usages too, in great demand for egghead phd's, laid off haskell developers who couldn't find better job than to code javascript, all that crowd, you know. Also, scientific studies show that much less people leave JavaScript for Haskell now in the first place. So, if anything, we're re-tracing JavaScript steps. We're in dire need of const'ness in the language. We'll introduce it as annotation first, but likely it will be so useful, that we'll want to make it a keyword. Then a question of scope for variables declared with it will raise. We should not make mistakes then. The stage should be setting already now. []
They won't be common, if people don't find common need for them (they shouldn't). If they do... oh, people!
So you're saying this proposal is to add new syntax and semantics to Python for an uncommon situation? You can't have it both ways. Is this proposal useful to a lot of people (and hence worth new syntax/semantics) or is it uncommon (and as a result, likely not worth the disruption).
I'm ambivalent, with a bias towards saying that we don't need this, to the proposal. But I'm finding your arguments confusing and inconsistent. It feels like it's almost impossible to discuss the specifics of the proposal, as you're not taking a consistent position to debate against.
So, the quest is for the fundamental, "atomic", orthogonal yet well-composable features which can be (re)used to address various (ideally, as many as possible) Python language design challenges and issues. This necessary makes the discourse topic wide, and people constantly bring up fringe issues like "but C has mandatory variable declarations!", "what about debuggers?", "what about locals()?". All that makes following the mailing list discussion hard, yes. I guess, for now we reached its limits. People interested in the topic should find enough food for reading and thought. But those fundamental, orthogonal language features under scrutiny are clear and simple: 1. Const'ness aka variable immutability. 2. Block-level scope. -- Best regards, Paul mailto:pmiscml@gmail.com
On Mon, Nov 30, 2020, 5:20 AM Paul Sokolovsky
So, if anything, we're re-tracing JavaScript steps. We're in dire need of const'ness in the language.
Somehow I've used Python for 22 years without experiencing that direness. I've taught thousands of students already experienced in other languages (most with constants) without anyone noting the need). I've been *read* by MILLIONS of readers using Python without mentioning the need to me. Somehow "dire" doesn't strike me as the right word.... Maybe you were looking for "conceivably useful in niche cases."?
On Mon, 30 Nov 2020 at 23:26, David Mertz <mertz@gnosis.cx> wrote:
Somehow "dire" doesn't strike me as the right word.... Maybe you were looking for "conceivably useful in niche cases."?
Well, I think const can be useful for: * multiprocessing. Now, for example, dict is passed between processes using MappingProxyType, which is slow. * avoid side effects. I expect that my object will not change and I want to be sure I'll not change it by mistake. Mistake that I made a lot of times. * contract. A function marks a parameter as const will guarantee that the object will not be changed. It's something complementar to annotations. * possible future speed improvements. For example, if an iterable is const, you can skip a lot of checks about mutability on iteration and make it more fast.
On Tue, Dec 1, 2020 at 10:25 AM Marco Sulla <Marco.Sulla.Python@gmail.com> wrote:
On Mon, 30 Nov 2020 at 23:26, David Mertz <mertz@gnosis.cx> wrote:
Somehow "dire" doesn't strike me as the right word.... Maybe you were looking for "conceivably useful in niche cases."?
Well, I think const can be useful for: * multiprocessing. Now, for example, dict is passed between processes using MappingProxyType, which is slow. * avoid side effects. I expect that my object will not change and I want to be sure I'll not change it by mistake. Mistake that I made a lot of times. * contract. A function marks a parameter as const will guarantee that the object will not be changed. It's something complementar to annotations. * possible future speed improvements. For example, if an iterable is const, you can skip a lot of checks about mutability on iteration and make it more fast.
Are you assuming that "const" means "will not be rebound" or "is immutable"? Or both? ChrisA
"Immutable", like C. Not rebound has another keyword, in Java for example (final). If we want to only avoid rebounding, I think const will be a bit confusing for C/C++ people. (I wrote "immutable" because in C constness can be removed) On Tue, 1 Dec 2020 at 01:12, Chris Angelico <rosuav@gmail.com> wrote:
On Tue, Dec 1, 2020 at 10:25 AM Marco Sulla <Marco.Sulla.Python@gmail.com> wrote:
On Mon, 30 Nov 2020 at 23:26, David Mertz <mertz@gnosis.cx> wrote:
Somehow "dire" doesn't strike me as the right word.... Maybe you were looking for "conceivably useful in niche cases."?
Well, I think const can be useful for: * multiprocessing. Now, for example, dict is passed between processes using MappingProxyType, which is slow. * avoid side effects. I expect that my object will not change and I want to be sure I'll not change it by mistake. Mistake that I made a lot of times. * contract. A function marks a parameter as const will guarantee that the object will not be changed. It's something complementar to annotations. * possible future speed improvements. For example, if an iterable is const, you can skip a lot of checks about mutability on iteration and make it more fast.
Are you assuming that "const" means "will not be rebound" or "is immutable"? Or both?
ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YFDQCV... Code of Conduct: http://python.org/psf/codeofconduct/
Hello, On Mon, 30 Nov 2020 17:26:00 -0500 David Mertz <mertz@gnosis.cx> wrote:
On Mon, Nov 30, 2020, 5:20 AM Paul Sokolovsky
So, if anything, we're re-tracing JavaScript steps. We're in dire need of const'ness in the language.
Somehow I've used Python for 22 years without experiencing that direness.
I've taught thousands of students already experienced in other languages (most with constants) without anyone noting the need). I've been *read* by MILLIONS of readers using Python without mentioning the need to me.
I suspect someone will eventually write a drama along the lines of "Best-kept Secret from the Python Community", second only to Oedipus Rex. Perhaps, you were talking to the wrong people? Perhaps you should teach students like https://www.youtube.com/watch?v=x2IQP8iug3c ? That guy reached only 0.3M, not millions like you, but spread the word that Python is slow. And the const is the cheapest way to make Python a tad faster (for sure open up possibilities for further optimizations), which should be accessible even to such an old clumsy behemoth as CPython. (Alternative, recently quoted, is to spend $1+M on that. https://mail.python.org/archives/list/python-dev@python.org/message/RDXLCH22...) And I'm mentioning just that usecase with a bow to my fixation on Python JITting, other people already mentioned more.
Somehow "dire" doesn't strike me as the right word.... Maybe you were looking for "conceivably useful in niche cases."?
Perhaps we can bargain on "really useful in many cases". -- Best regards, Paul mailto:pmiscml@gmail.com
On Tue, Dec 1, 2020 at 10:38 AM Paul Sokolovsky <pmiscml@gmail.com> wrote:
And the const is the cheapest way to make Python a tad faster (for sure open up possibilities for further optimizations), which should be accessible even to such an old clumsy behemoth as CPython.
This is at least the 50th idea I've seen that will "with absolute certainty make Python faster" ... where the first 49 failed to do so, often despite significant institutional investment in money and skilled developers. Obviously, there *have* been actual speed improvements. But they rarely seem to follow the "obvious intuitions" of speculation on python-ideas.
Somehow "dire" doesn't strike me as the right word.... Maybe you were looking for "conceivably useful in niche cases."?
Perhaps we can bargain on "really useful in many cases".
I've still yet to see an example that is very compelling. Not speed, I'll treat that with skepticism. Just in terms of notably improved code clarity. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
Hello, On Tue, 1 Dec 2020 16:41:36 +0000 David Mertz <mertz@gnosis.cx> wrote:
On Tue, Dec 1, 2020 at 10:38 AM Paul Sokolovsky <pmiscml@gmail.com> wrote:
And the const is the cheapest way to make Python a tad faster (for sure open up possibilities for further optimizations), which should be accessible even to such an old clumsy behemoth as CPython.
This is at least the 50th idea I've seen that will "with absolute certainty make Python faster"
That's why I don't make such claims, and instead making a very different one: that idea with absolute certainty will remove *one* of 50 problems which keep Python slow. Hard-boiled pragmatists among us won't get it, but it's a true intellectual pleasure to be left with 49 problems instead of 50. Besides, if every advanced Python programmer peeled one of such problems, before waving bye-bye and flying over to Haskell, we'd already have a fast Python. In that regard, I continue a well-established Python tradition. (And indeed, we have many fast Pythons, working under various conditions. We still need to pull further as a community to go over the summit where "a Python" is by default "fast", not "slow").
... where the first 49 failed to do so, often despite significant institutional investment in money and skilled developers.
Obviously, there *have* been actual speed improvements. But they rarely seem to follow the "obvious intuitions" of speculation on python-ideas.
Challenge accepted - let's on python-ideas speculate on actual speed improvements, not wait with popcorn for "significant institutional investment" (can do both).
Somehow "dire" doesn't strike me as the right word.... Maybe you were looking for "conceivably useful in niche cases."?
Perhaps we can bargain on "really useful in many cases".
I've still yet to see an example that is very compelling. Not speed, I'll treat that with skepticism. Just in terms of notably improved code clarity.
That's in the eye of the beholder, I'm afraid. So, if you don't want to be convinced of those clarity improvements, you'll never be. -- Best regards, Paul mailto:pmiscml@gmail.com
On Tue, Dec 1, 2020 at 5:43 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
That's in the eye of the beholder, I'm afraid. So, if you don't want to be convinced of those clarity improvements, you'll never be.
It's hard to be convinced by something that neither you nor anyone else on the thread has actually presented. Other than posturing about "dire need" and vague hand-waving about potential speed-ups, there's absolutely nothing that makes this desirable. Not even a small toy example, let alone a more real world need. Yes, some other languages have block level scopes. And some other languages have constants. But you have not even *suggested* why this would be good for Python itself (I guess other than "gotta keep up with the Javascript programmers"). It's not especially common, but I've sometimes written: for i in big_collection: if condition(): break # other stuff ... intervening code ... if i+1 < len(big_collection): something_about_early_stop() On the other hand, I can count on one hand, without using any fingers, the number of times when I would have used: for const i in big_collection: # whatever It's just not a need that arises. There exist TWO highly successful, widely used, JIT compilers for Python. PyPy and Numba. Neither one of them would have any use whatsoever for this constantness. Or if you believe otherwise, get a developer of one of those to comment so. JIT'd Python simply is not slow, even compared to compiled languages. Searching for maybe-possibly-someday optimizations while ignoring the actual speed paths, is silly. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
On 2/12/20 6:43 am, Paul Sokolovsky wrote:
That's why I don't make such claims, and instead making a very different one: that idea with absolute certainty will remove *one* of 50 problems which keep Python slow.
But without an implementation demonstrating an actual speed improvement, you're only speculating that the problem is, in fact, keeping Python slow. I'm also not convinced that the proposed solution is necessary to remove the problem. A sufficiently smart JIT could notice, for example, that a particular function doesn't mutate its arguments and pass them in an optimised way, without needing any declarations. -- Greg
Hello, On Wed, 02 Dec 2020 12:36:23 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 2/12/20 6:43 am, Paul Sokolovsky wrote:
That's why I don't make such claims, and instead making a very different one: that idea with absolute certainty will remove *one* of 50 problems which keep Python slow.
But without an implementation demonstrating an actual speed improvement, you're only speculating that the problem is, in fact, keeping Python slow.
That summarizes it well, yes. Just as PEP266/PEP267/PEP280, I'm speculating on how to optimize namespace lookups. I however aspire to improve on them, providing wider, more coherent picture on how to achieve that. I also speculate in the same way as PEP509, which was accepted, and actually implemented in CPython, without any improvements on CPython speed (on its own). But it lays a framework to "maybe do it somewhere, somewhen". My proposal is very similar, but explores a different direction of changes. https://www.python.org/dev/peps/pep-0266/ https://www.python.org/dev/peps/pep-0267/ https://www.python.org/dev/peps/pep-0280/ https://www.python.org/dev/peps/pep-0509/
I'm also not convinced that the proposed solution is necessary to remove the problem. A sufficiently smart JIT could notice,
A sufficiently smart JIT is sufficiently hard to develop. As an example, a most well-known and most-used Python implementation, CPython, doesn't have any JIT at all, not only "sufficiently advanced", but even "simple". But simple would be much easier to add (to any project). And my proposal explores how to get specific advantages from even simple JIT techniques.
for example, that a particular function doesn't mutate its arguments and pass them in an optimised way, without needing any declarations.
-- Greg
[] -- Best regards, Paul mailto:pmiscml@gmail.com
On Wed, Dec 2, 2020 at 6:37 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
A sufficiently smart JIT is sufficiently hard to develop. As an example, a most well-known and most-used Python implementation, CPython, doesn't have any JIT at all, not only "sufficiently advanced", but even "simple". But simple would be much easier to add (to any project). And my proposal explores how to get specific advantages from even simple JIT techniques.
If all you're doing is exploring, why a PEP? Just create a "Python with constness" variant, and go to town. What's the advantage of having CPython and Jython and PyPy and everyone else synchronize on your proposed syntax? ChrisA
Hello, On Wed, 2 Dec 2020 18:39:42 +1100 Chris Angelico <rosuav@gmail.com> wrote:
On Wed, Dec 2, 2020 at 6:37 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
A sufficiently smart JIT is sufficiently hard to develop. As an example, a most well-known and most-used Python implementation, CPython, doesn't have any JIT at all, not only "sufficiently advanced", but even "simple". But simple would be much easier to add (to any project). And my proposal explores how to get specific advantages from even simple JIT techniques.
If all you're doing is exploring, why a PEP?
Where's PEP? I informally once called my stuff "pseudo-PEP", to emphasize that it aspires to cover a topic in-depth, like expected from a PEP. But it's not fully written up to PEP standards, nor intended to be it.
Just create a "Python with constness" variant, and go to town. What's the advantage of having CPython and Jython and PyPy and everyone else synchronize on your proposed syntax?
Because all those things are "Pythons", and there should be exchange of ideas and cross-pollination between implementations, and what can be a better place for that, than a list called "python-ideas"? (Short of it being renamed to "cpython-ideas").
ChrisA
[] -- Best regards, Paul mailto:pmiscml@gmail.com
On Wed, Dec 2, 2020 at 7:18 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
Hello,
On Wed, 2 Dec 2020 18:39:42 +1100 Chris Angelico <rosuav@gmail.com> wrote:
On Wed, Dec 2, 2020 at 6:37 PM Paul Sokolovsky <pmiscml@gmail.com> wrote:
A sufficiently smart JIT is sufficiently hard to develop. As an example, a most well-known and most-used Python implementation, CPython, doesn't have any JIT at all, not only "sufficiently advanced", but even "simple". But simple would be much easier to add (to any project). And my proposal explores how to get specific advantages from even simple JIT techniques.
If all you're doing is exploring, why a PEP?
Where's PEP? I informally once called my stuff "pseudo-PEP", to emphasize that it aspires to cover a topic in-depth, like expected from a PEP. But it's not fully written up to PEP standards, nor intended to be it.
My bad, you wrote it up in the same style and I assumed your intention was to go that route. Generalizing a bit: "Why a python-ideas proposal?"
Just create a "Python with constness" variant, and go to town. What's the advantage of having CPython and Jython and PyPy and everyone else synchronize on your proposed syntax?
Because all those things are "Pythons", and there should be exchange of ideas and cross-pollination between implementations, and what can be a better place for that, than a list called "python-ideas"? (Short of it being renamed to "cpython-ideas").
But you can make your own private research project without asking anyone else for information. Why try to synchronize with anyone else? Why not just make your own thing and find out what constness can do for Python? ChrisA
On Wed, 2 Dec 2020 at 09:27, Chris Angelico <rosuav@gmail.com> wrote:
But you can make your own private research project without asking anyone else for information. Why try to synchronize with anyone else? Why not just make your own thing and find out what constness can do for Python?
I agree. I think it will be very interesting some macro-benchmarks (read: pyperformance) before proposing this big change. Anyway, personally I think a "const" can be useful for developers to avoid changing or rebinding the object by mistake, not primarily for speeding up things.
Hello, On Wed, 2 Dec 2020 19:26:22 +1100 Chris Angelico <rosuav@gmail.com> wrote: []
A sufficiently smart JIT is sufficiently hard to develop. As an example, a most well-known and most-used Python implementation, CPython, doesn't have any JIT at all, not only "sufficiently advanced", but even "simple". But simple would be much easier to add (to any project). And my proposal explores how to get specific advantages from even simple JIT techniques.
If all you're doing is exploring, why a PEP?
Where's PEP? I informally once called my stuff "pseudo-PEP", to emphasize that it aspires to cover a topic in-depth, like expected from a PEP. But it's not fully written up to PEP standards, nor intended to be it.
My bad, you wrote it up in the same style and I assumed your intention was to go that route. Generalizing a bit: "Why a python-ideas proposal?"
But it was answered in the very the same mail that you reply to, below. []
Because all those things are "Pythons", and there should be exchange of ideas and cross-pollination between implementations, and what can be a better place for that, than a list called "python-ideas"? (Short of it being renamed to "cpython-ideas").
But you can make your own private research project without asking anyone else for information. Why try to synchronize with anyone else?
Not only I can make private research project, that's what I have been doing for years. And many other people have been doing that. But the worst part of that is that many of them never actually went out to share the results and/or experience. It's really frustrating to think that you may be duplicating somebody's steps from long ago, only because they missed to post about, or were shy to do that because of the expected feedback. That's not right thing. And I can't see what can stop ideas exchange, short of banning Python ideas on the python-ideas mailing list. And in all fairness, what I posted isn't worse than other content posted here, like "let's stuff more stuff into builtin namespace" or "let's make list indexes silently wrap around behind your back".
Why not just make your own thing and find out what constness can do for Python?
That's exactly what I have been doing. And I like what I see and I would like to share with whoever may be interested, and get a second opinion. And I'm surprised that discussing such points are good topic for this list, I thoughts that's the background of why this list exists. If anything to learn form it, is that I posted too late. Because when you post something simple and low-effort, there's usually enough people wanting to contemplate and speculate what can be done about it. But when you post something well worked out and detailed, there's that surprised stare and question: "Why did you post that?".
ChrisA
[] -- Best regards, Paul mailto:pmiscml@gmail.com
Hello, On Mon, 30 Nov 2020 12:56:59 +1300 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 29/11/20 11:02 pm, Paul Sokolovsky wrote:
It will be much more obvious if there's a general (standalone) "const",
I don't think it will. There's nothing about the problem that points towards constness as a solution, so it doesn't matter how many other places in the language "const" appears.
As was mentioned, there's no replacement for reading docs/tutorials. And all that applies the same to "for new".
And even if you're told about it, you need two or three steps of reasoning to understand *why* it solves the problem.
that's why I'm saying we can't really consider "for const" without just "const"
I agree with that.
Good.
And it's "pretty obvious" to someone who considered various choices and saw pieces falling into their places. Also might be pretty obvious for someone who used other languages.
I strongly suspect it's something that's obvious only in hindsight.
The same for "for new". But at least "for const" fits better with other usages of "const", including in other languages (so much less of NIH syndrome).
-- Greg
[] -- Best regards, Paul mailto:pmiscml@gmail.com
participants (17)
-
Abdulla Al Kathiri -
Andrew Svetlov -
Ben Rudiak-Gould -
Brendan Barnwell -
Cameron Simpson -
Chris Angelico -
Christopher Barker -
David Foster -
David Mertz -
Greg Ewing -
Guido van Rossum -
Marco Sulla -
Paul Moore -
Paul Sokolovsky -
sairamkumar2022＠gmail.com -
Steve Barnes -
Steven D'Aprano