[Python-ideas] Possible method of distinguishing between set-literals, dict-literals, and odict-literals

[Benjamin Peterson]

Guido van Rossum <guido at ...> writes:
> On Tue, Jun 16, 2009 at 11:24 AM, Terry Reedy<tjreedy at ...> wrote:
> > I consider b"" and r"" to be hacks, necessary but not to be imitated without
> > such necessity. I believe both affect how the lexer interprets code
> > characters when it constructs literal value tokens to be fed to the parser.
> 
> Not quite -- the lexer passes the entire string literal uninterpreted
> to the parser (including quotes). It doesn't even need the 'b' or 'r'
> prefix to tell where the literal ends (though it does need the opening
> quote for that, and whether it's tripled). The code generation stage,
> however, looks at the prefix and interprets the contents of the
> literal differently based on the presence of 'b', and creates a
> different object depending on the presence of 'b'. (Or 'u' in Python
> 2.)

<nitpicking>
Technically, string decoding is done during the building of the AST not code
generation. See parsestrplus in ast.c if you want to know all the gory details.
</nitpicking>

Regards,
Benjamin