
On 9/20/2021 11:19 AM, Terry Reedy wrote:
On 9/20/2021 7:18 AM, Pablo Galindo Salgado wrote:
there are some interesting things we **could** (emphasis on could) get out of this and I wanted to discuss what people think about them.
* The parser will allow nesting quote characters. This means that we **could** allow reusing the same quote type in nested expressions like this:
f"some text { my_dict["string1"] } more text"
I believe that this will disable regex-based processing, such as syntax highlighters, as in IDLE. I also think that it will be sometimes confusing to human readers.
When I initially wrote f-strings, it was an explicit design goal to be just like existing strings, but with a new prefix. That's why there are all of the machinations in the parser for scanning within f-strings: the parser had already done its duty, so there needed to be a separate stage to decode inside the f-strings. Since they look just like regular strings, most tools could add the lowest possible level of support just by adding 'f' to existing prefixes they support: 'r', 'b', 'u'. The upside is that if you don't care about what's inside an f-string, your work is done. I definitely share your concern about making f-strings more complicated to parse for tool vendors: basically all editors, alternative implementations, etc.: really anyone who parses python source code. But maybe we've already crossed this bridge with the PEG parser. Although I realize there's a difference between lexing and parsing. While the PEG parser just makes parsing more complicated, this change would make what was lexing into a more sophisticated parsing problem. In 2018 or 2019 at PyCon in Cleveland I talked to several tool vendors. It's been so long ago that I don't remember who, but I'm pretty sure it was PyCharm and 2 or 3 other editors. All of them supported making this change, even understanding the complications it would cause them. I don't recall if I talked to anyone who maintains an alternative implementation, but we should probably discuss it with MicroPython, Cython, PyPy, etc., and understand where they stand on it. In general I'm supportive of this change, because as Pablo points out there are definite benefits. But I think if we do accept it we should understand what sort of burden we're putting on tool and implementation authors. It would probably be a good idea to discuss it at the upcoming dev sprints. Eric