[Python-Dev] Python Language Reference has no mention of list comÃprehensions

[Andrew Barnert]

On Dec 4, 2015, at 00:38, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 4 December 2015 at 12:48, Andrew Barnert via Python-Dev
> <python-dev at python.org> wrote:
>> On Dec 3, 2015, at 17:25, Steven D'Aprano <steve at pearwood.info> wrote:
>>>> On Thu, Dec 03, 2015 at 09:25:53AM -0800, Andrew Barnert via Python-Dev wrote:
>>>> I've seen people saying that before, but I don't know where they get
>>>> that. It's certainly not the way, say, C++ or JavaScript use the term.
> 
> I'm one of the folks that use it that way, but I learned that
> terminology *from* the Python language reference.

If that's the usual case, then isn't it almost certainly more true for "display" than for "literal"? I doubt most Python users came in with a pre-existing notion of "display" from another language, or from programming in general--or, if they did, it's probably one of the senses that's irrelevant enough to not confuse anyone (like a repr, or a string formatting template). So if you want to redefine one of our terms to allow a new distinction, why not that one?

More importantly, as I said in my other message: do we actually need to be able to make this distinction? The problem this thread set out to solve is that "comprehension" doesn't have a docs section because it's just a subset of displays, so you can't search for it. Making it a subset of dynamic literals, which is a subset of literals, seems like it gets us farther from a solution. Right now, we could easily change the section title to "list displays (including comprehensions)" and we're done.

>>> I wouldn't take either of those two languages as examples of best
>>> practices in language design :-)
>> 
>> No, but they seem to be the languages (along with C and Java) that people usually appeal to.
>> 
>> You also found "literal" used the same way as JavaScript in Ruby, one of three languages in your quick survey. It's also used similarly in ML and Haskell. In Lisp, it has a completely different meaning (a quoted list).
>> 
>> But as I said before, we can't use the word "literal" to contrast with comprehensions, because a large segment of the Python community (including you) would find that use of the word confusing and/or annoying because you intuitively think of the C/FORTRAN/etc. definition rather than the C++/Ruby/JS/Haskell definition. It doesn't matter whether that's a peculiar quirk of the Python community or not, whether there's a good reason for it or not, etc.; all that matters is that it's true.
> 
> Even though it's true, I'm not sure it's sufficient to rule out a
> switch to "there are two kinds of literal" as the preferred
> terminology.
> 
> The recent case that comes to mind is the new format string literals -
> those can include arbitrary subexpressions, like container displays
> and comprehensions, but the conclusion from the PEP 498 discussion was
> that it makes the most sense to still consider them a kind of string
> literal.
> 
> There's also a relatively straightforward way of defining the key
> semantic different between a literal and a normal constructor call:
> with a literal, there's no way to override the type of the resulting
> object, while a constructor call can be monkeypatched like any other
> callable.

Is that an important distinction to anyone but people who write Python implementations? If some library I'm using chooses to monkeypatch or shadow a type name, the objects are still going to quack the way I expect (otherwise, I'm going to stop using that library pretty quickly). And meanwhile, why do I need to distinguish between libraries that monkeypatch the stdlib for me and libraries that install an import hook to patch my code?

It's certainly not meaningless or completely useless (e.g., the discussion about whether f-strings are literals would have been shorter, and had more of a point), but it doesn't seem useful enough to be worth redefining existing terminology.

> The distinction that arises for containers is then the one that Chris
> Angelico pointed out: a container literal may have constant content,
> *or* it may have dynamic content.

Well, yes, but, again, both forms of container literal can have dynamic content: [f(), g()] is just as dynamic as [x() for x in (f, g)]. So we still don't have the contrast we were looking for.

Also, [1, 2] is literal, and not dynamic, but it's not a constant value, so calling it a constant literal seems likely to be more confusing than helpful.

One more thing: we don't have to worry about whether def and class are literal constructs because they're not expressions, but what about lambda? It fits your definition of literal. JS calls them literals (Ruby is a bit confusing because it splits the notion of function into three entirely independent things), as do some functional languages. And this means you can write a function-table dict that's all literals. As for whether it's constant--the code object obviously can be compiled into the bytecode, because that's what CPython does, and the function object itself could be when there are no free variables, even if it isn't today. (In fact, even with free variables, it could be done, but not the way CPython implements closures.)

> If we switched from calling things
> "displays" to calling them "dynamic literals", I'd be surprised if too
> many folks that were able to figure out what "display" meant struggled
> to make the transition.
> 
> Summarising that idea:
> 
> * literals: any of the dedicated expressions that produce an instance
> of a builtin type
> * constant literal: literals that produce a constant object that can
> be cached in the bytecode
> * dynamic literal: literals containing dynamic subexpressions that
> can't be pre-calculated
> * display: legacy term for a dynamic literal (originally inherited from ABC)

That doesn't seem right--f-strings are dynamic literals, but they aren't displays. (And lambdas, too.) And (1, 2) is a constant literal but it is a display.

> * comprehension: a dynamic literal that creates a new container from
> an existing iterable
> * lexical literal: constant literals and dynamic string literals [1]
> 
> The ast.literal_eval() docs would need a slight adjustment to refer to
> "literals (excluding container comprehensions and generator
> expressions)", rather than the current "literals and container
> displays".

It seems like it already need a change with or without your suggestion, if it uses "container displays" to rule out comprehensions, which are a kind of container display under today's definition.