PEG parser and raw strings

Hi. Just a question about the new PEG parser: will it support lone slash in raw strings? the fact that r'\' doesn't work as expected it's a strange exception to remember.

That's the business of the tokenizer, not the parser. And no, nothing is changing there. Please start a discussion on this topic here -- use python-ideas if you feel the need to vent. On Thu, May 28, 2020 at 1:20 PM Vito De Tullio <vito.detullio@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

On 5/28/2020 3:26 PM, Vito De Tullio wrote:
Just a question about the new PEG parser: will it support lone slash in raw strings?
In 3.9, the new parser is intended to be a drop-in replacement. That is still being worked on. For 3.10, user-visible changes can be considered, but I have not seen any concrete proposals.
the fact that r'\' doesn't work as expected it's a strange exception to remember.
That said, this seems like a scanner, not a compiler issue.
print(repr(r'\')) SyntaxError: EOL while scanning string literal
The python syntax highlighter marks all of "r'\')) as part of the string, ... but
exec(compile("print(repr(r'\'))", '', 'exec')) ''
-- Terry Jan Reedy

On 29/05/20 7:26 am, Vito De Tullio wrote:
Even if it could be made to, it presumably won't, because that would be a language change (e.g. the meaning of r'foo\'bar' would change). In any case, this is a function of the tokeniser, which I understand isn't being replaced at this stage. -- Greg

28.05.20 22:26, Vito De Tullio пише:
As others said, this is a part of the tokenizer. I once experimented with changing the tokenizer to make a backslash always inert in raw strings. This change breaks code in ~15 files in the stdlib and tests, so it would likely break a large amount of user code. The cost is too high. https://github.com/python/cpython/pull/15217

That's the business of the tokenizer, not the parser. And no, nothing is changing there. Please start a discussion on this topic here -- use python-ideas if you feel the need to vent. On Thu, May 28, 2020 at 1:20 PM Vito De Tullio <vito.detullio@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>

On 5/28/2020 3:26 PM, Vito De Tullio wrote:
Just a question about the new PEG parser: will it support lone slash in raw strings?
In 3.9, the new parser is intended to be a drop-in replacement. That is still being worked on. For 3.10, user-visible changes can be considered, but I have not seen any concrete proposals.
the fact that r'\' doesn't work as expected it's a strange exception to remember.
That said, this seems like a scanner, not a compiler issue.
print(repr(r'\')) SyntaxError: EOL while scanning string literal
The python syntax highlighter marks all of "r'\')) as part of the string, ... but
exec(compile("print(repr(r'\'))", '', 'exec')) ''
-- Terry Jan Reedy

On 29/05/20 7:26 am, Vito De Tullio wrote:
Even if it could be made to, it presumably won't, because that would be a language change (e.g. the meaning of r'foo\'bar' would change). In any case, this is a function of the tokeniser, which I understand isn't being replaced at this stage. -- Greg

28.05.20 22:26, Vito De Tullio пише:
As others said, this is a part of the tokenizer. I once experimented with changing the tokenizer to make a backslash always inert in raw strings. This change breaks code in ~15 files in the stdlib and tests, so it would likely break a large amount of user code. The cost is too high. https://github.com/python/cpython/pull/15217
participants (5)
-
Greg Ewing
-
Guido van Rossum
-
Serhiy Storchaka
-
Terry Reedy
-
Vito De Tullio