On Mon, Apr 06, 2020 at 11:54:54AM -0700, Guido van Rossum wrote:

> (In an early version of the PEG parser, all keywords were
> context-sensitive, and there were only very few places in the grammar where
> this required us to insert negative lookaheads to make edge cases parse
> correctly. The rest was taken care by careful ordering of rules, e.g. the
> rule for `del_stmt` must be tried before the rule for `expression_stmt`
> since `del *x` would match the latter.)

I think, on first glance, I'd rather have all keywords context-sensitive
than just some. But I haven't put a great deal of thought into that
aspect of it, and I reserve the right to change my mind :-)


> > Personally, I would not like to have to explain to newcomers why `match`
> > is a keyword but you can still use it as a function or variable, but not
> > other keywords like `raise`, `in`, `def` etc.
> >
> >     match expression:
> >         match = True
> >
>
> What kind of newcomers do you have that they even notice that, unless you
> were to draw attention to it?

It didn't take me 25 years to try using "of" and "if" for "output file"
and "input file", so I guess my answer to your question is ordinary
newcomers :-)

"Newcomers" doesn't just including beginners to programming, it can
include people experienced in one or more other language coming to
Python for the first time.

But if we're talking about complete beginners, the concept of what is
and isn't a keyword is not always clear. Why is the first of these legal
but not the second? Both words are highlighted in my editor:

    str = "Hello world"
    class = "wizard"

People are going to learn that `match` is a keyword, and then they are
going to come across code using it as a variable or method, and while
the context-sensitive rule might be obvious to us, it won't be obvious
to them precisely because they are still learning the language rules.

I think that `match` would be an especially interesting case because I
can easily see someone starting off with a variable `match`, that they
handle in an `if` statement, and then as the code evolves they shift it
to a `match` statement:

    match match:

and not bother to refactor the name because they are familiar enough
with is that the meaning is obvious.

On the other hand there are definitely a few keywords that collide with
useful names. Apart from `if`, I have wanted to use these as variables,
parameters or functions:

    class, raise, in, from, except, while, lambda

(off the top of my head, there may be others). There's at least one
place in the random module where a parameter is misspelled "lambd"
because lambda is a keyword. So there is certainly something to be said
for getting rid of keywords.

On the third hand, keywords don't just make it easier for the
interpreter, they also make it easier for the human reader. You don't
need to care about context, `except` is `except` wherever you see it.
That makes it a dead-simple rule for anyone to learn, because there are
no exceptions, pun intended.

(I guess inside strings and comments are exceptions, but they are
well-understood and *simple* exceptions.)

I just can't help feeling at this point that while there are pros and
cons to making things a keyword, having some keywords be context
sensitive but not others is going to combine the worst of both and end
up be confusing and awkward.


> I'm serious -- from the kind of questions
> I've seen in user forums, most newcomers are having a hard enough time
> learning more fundamental concepts and abstractions than the precise rules
> for reserved words.

That's because the precise rules for reserved words are dead-simple to
learn. You can't use them anywhere except in the correct context. If we
start adding exceptions to that, that reserved words are only sometimes
reserved, I think that will make them harder to learn. If it's only some
reserved words but not others, that's even harder because we have three
classes of words:

* words that are never reserved
* words that are sometimes reserved, depending on what is around them
* words that are always reserved

I had thought that "no context-sensitive keywords" was a hard rule, so I
was surprised that you are now re-considering it.


--
Steven
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NNMJB7QIRAMROQU5XRM6YV7K4NXBGXBT/
Code of Conduct: http://python.org/psf/codeofconduct/