On Mon, Apr 06, 2020 at 11:54:54AM -0700, Guido van Rossum wrote:
(In an early version of the PEG parser, all keywords were context-sensitive, and there were only very few places in the grammar where this required us to insert negative lookaheads to make edge cases parse correctly. The rest was taken care by careful ordering of rules, e.g. the rule for `del_stmt` must be tried before the rule for `expression_stmt` since `del *x` would match the latter.)
I think, on first glance, I'd rather have all keywords context-sensitive than just some. But I haven't put a great deal of thought into that aspect of it, and I reserve the right to change my mind :-)
Personally, I would not like to have to explain to newcomers why `match` is a keyword but you can still use it as a function or variable, but not other keywords like `raise`, `in`, `def` etc.
match expression: match = True
What kind of newcomers do you have that they even notice that, unless you were to draw attention to it?
It didn't take me 25 years to try using "of" and "if" for "output file" and "input file", so I guess my answer to your question is ordinary newcomers :-) "Newcomers" doesn't just including beginners to programming, it can include people experienced in one or more other language coming to Python for the first time. But if we're talking about complete beginners, the concept of what is and isn't a keyword is not always clear. Why is the first of these legal but not the second? Both words are highlighted in my editor: str = "Hello world" class = "wizard" People are going to learn that `match` is a keyword, and then they are going to come across code using it as a variable or method, and while the context-sensitive rule might be obvious to us, it won't be obvious to them precisely because they are still learning the language rules. I think that `match` would be an especially interesting case because I can easily see someone starting off with a variable `match`, that they handle in an `if` statement, and then as the code evolves they shift it to a `match` statement: match match: and not bother to refactor the name because they are familiar enough with is that the meaning is obvious. On the other hand there are definitely a few keywords that collide with useful names. Apart from `if`, I have wanted to use these as variables, parameters or functions: class, raise, in, from, except, while, lambda (off the top of my head, there may be others). There's at least one place in the random module where a parameter is misspelled "lambd" because lambda is a keyword. So there is certainly something to be said for getting rid of keywords. On the third hand, keywords don't just make it easier for the interpreter, they also make it easier for the human reader. You don't need to care about context, `except` is `except` wherever you see it. That makes it a dead-simple rule for anyone to learn, because there are no exceptions, pun intended. (I guess inside strings and comments are exceptions, but they are well-understood and *simple* exceptions.) I just can't help feeling at this point that while there are pros and cons to making things a keyword, having some keywords be context sensitive but not others is going to combine the worst of both and end up be confusing and awkward.
I'm serious -- from the kind of questions I've seen in user forums, most newcomers are having a hard enough time learning more fundamental concepts and abstractions than the precise rules for reserved words.
That's because the precise rules for reserved words are dead-simple to learn. You can't use them anywhere except in the correct context. If we start adding exceptions to that, that reserved words are only sometimes reserved, I think that will make them harder to learn. If it's only some reserved words but not others, that's even harder because we have three classes of words: * words that are never reserved * words that are sometimes reserved, depending on what is around them * words that are always reserved I had thought that "no context-sensitive keywords" was a hard rule, so I was surprised that you are now re-considering it. -- Steven