We have been implementing LiteralString (PEP 675) support in PyCharm and even though the PEP is written wonderfully and most of the proposal seems straightforward to implement, I just can't wrap my head around how this feature should work with generics without breaking existing code. There is a brief section about it in the PEP but it doesn't cover my concerns, so I decided to bring it here. Sorry if it has been discussed already, I couldn't find anything relevant in the typing-sig archives.
The main question is "how far" inferred LiteralStrings should be propagated through substitution of type parameters during the inference.
Consider the following example
def expects_literal_string(s: LiteralString) -> None:
pass
s: str = ...
xs = ['foo', 'bar'] # inferred type: list[LiteralString]
expects_literal_string(''.join(xs)) # ok!
it's appealing to infer the type list[LiteralString] for xs here right away to make the result of ''.join(xs) be LiteralString as well (there is an overload for str.join in Typeshed)
however doing so will trigger a typing error for the following unassuming code
xs.append(s) # type error: expected LiteralString, got str
Of course, the same reasoning applies to user-defined generics, not having their own literal syntax
T = TypeVar('T')
class Box(Generic[T]):
def __init__(self, x: T) -> None:
self.value = x
def set(self, x: T) -> None:
self.value = x
box = Box('foo')
box.set(s) # type error: expected LiteralString, got str
One approach (though very limiting) might be to infer list[LiteralString] only in cases where a collection literal is used directly, not assigned to a name, e.g.
expects_literal_string(''.join(['foo', 'bar'])) # ok!
but even then it might cause some problems, such as here, where a type containing LiteralString "over-constraints" possible values for other parameters sharing the same TypeVar
def couple(first: T, second: T) -> tuple[T, T]:
return first, second
couple(['foo'], [s]) # type error: expecting list[LiteralString], got list[str]
I would argue that even if the first argument was explicitly annotated here (for instance to pass it to str.join), it would still be a surprise for a user that an existing call to couple now leads to a type checker error.
xs: list[LiteralString] = ['foo']
expects_literal_string(''.join(xs))
couple(xs, [s]) # type error: expecting list[LiteralString], got list[str]
I see that Mypy support of LiteralString is still in progress. I'm curious how other type checkers (pyright, Pyre) handle such cases gracefully. I guess there are a number of different strategies to allow gradually introducing LiteralString to existing type hinted code base. For instance:
Do you upcast LiteralString to str, "erasing" it, whenever it's captured in a type parameter? Do you require all generic types possibly containing LiteralString values to be explicitly type hinted?
Thanks for sharing the knowledge!
-- Mikhail Golubev
Software Developer
JetBrains
The drive to develop