[Python-ideas] Possible method of distinguishing between set-literals, dict-literals, and odict-literals

Benjamin Peterson benjamin at python.org
Wed Jun 17 02:22:54 CEST 2009

Guido van Rossum <guido at ...> writes:
> On Tue, Jun 16, 2009 at 11:24 AM, Terry Reedy<tjreedy at ...> wrote:
> > I consider b"" and r"" to be hacks, necessary but not to be imitated without
> > such necessity. I believe both affect how the lexer interprets code
> > characters when it constructs literal value tokens to be fed to the parser.
> Not quite -- the lexer passes the entire string literal uninterpreted
> to the parser (including quotes). It doesn't even need the 'b' or 'r'
> prefix to tell where the literal ends (though it does need the opening
> quote for that, and whether it's tripled). The code generation stage,
> however, looks at the prefix and interprets the contents of the
> literal differently based on the presence of 'b', and creates a
> different object depending on the presence of 'b'. (Or 'u' in Python
> 2.)

Technically, string decoding is done during the building of the AST not code
generation. See parsestrplus in ast.c if you want to know all the gory details.


More information about the Python-ideas mailing list