[Python-ideas] Proposal: Tuple of str with w'list of words'

[Stephen J. Turnbull]

Random832 writes:

 > I don't understand what this "after splitting" you're talking about
 > is.  It would be a single pass through the characters of the token,

Which may as well be thought of as a string (not a str).  Although you
can implement this process in one pass, you can also think of it in
terms of two passes that give the same result.  I suspect many people
will think in terms of two passes, and I certainly do.  Steven
d'Aprano appears to, as well (he also used the "before splitting"
terminology).  Of course, he may find "the implementation will be
single pass" persuasive, even though I don't.

 > You and I have very different definitions of the word "compact". In
 > fact, this is *so obviously* non-compact

I used \u notation to ensure that people would understand that the
separator is a non-character.  (Emacs allows me to enter it, and with
my current font it displays an empty box.  I could fiddle with my
PYTHONIOENCODING to use some sort of escape error handler to make it
convenient, but I won't use w"" anyway so the point is sort of moot.)

 > (The point of returning a tuple is to avoid the disadvantage that
 > the list returned by split must be built at runtime and can't be
 > loaded as a constant, or perhaps turned into a frozenset constant
 > by the optimizer in cases like "if x in w'foo bar baz':".

That's true, but where's the use case where that optimization matters?