Random832 writes:
I don't understand what this "after splitting" you're talking about is. It would be a single pass through the characters of the token,
Which may as well be thought of as a string (not a str). Although you can implement this process in one pass, you can also think of it in terms of two passes that give the same result. I suspect many people will think in terms of two passes, and I certainly do. Steven d'Aprano appears to, as well (he also used the "before splitting" terminology). Of course, he may find "the implementation will be single pass" persuasive, even though I don't.
You and I have very different definitions of the word "compact". In fact, this is *so obviously* non-compact
I used \u notation to ensure that people would understand that the separator is a non-character. (Emacs allows me to enter it, and with my current font it displays an empty box. I could fiddle with my PYTHONIOENCODING to use some sort of escape error handler to make it convenient, but I won't use w"" anyway so the point is sort of moot.)
(The point of returning a tuple is to avoid the disadvantage that the list returned by split must be built at runtime and can't be loaded as a constant, or perhaps turned into a frozenset constant by the optimizer in cases like "if x in w'foo bar baz':".
That's true, but where's the use case where that optimization matters?