[Python-ideas] Lessons from typing hinting Whoosh (PEP484)

[Matt Chaput]

Since PyCharm added support for pep484 I've been working on adding type hints to the Whoosh code base. It makes it a lot more enjoyable to work with the code in the IDE. As I add more and more hints, it's already revealed bugs where arguments were accidentally swapped, or interfaces weren't updated everywhere. I feel much more confident working in the (quite large) codebase with typing.

Basically, I'm loving the idea of PEP484. For me, it's a huge improvement in the development experience. However, I've already run into a few issues. I'm not sure if these were discussed during the development of PEP484.

1. The biggest problem, most perplexing problem for me is circular imports. That type hints must actually be imported at the top level is a MASSIVE limitation, at least in a big codebase. I started off importing the types I needed, then switched to the "import the module, specify the type in a string" trick, but even that has failed.

In a large/complex codebase, the more things you have to import at the top level (to use for typing), the more potential import circles you create. Now I've hit an impasse where every step in the circle is using the module/string trick, but it doesn't help because I need to subclass something in the circle.

For now, I'm going to have to workaround this, either by trying to move "interface" base classes out to their own side package, or just not type hinting some things. Neither is appetizing at all.

I think a good solution might be if the typing system could have its own special "imports" that are only used by the typing system, e.g. something like this at the top of a file:

    # "real" import
    from foo import bar

    # imports just for type checking
    __typeimports__ = """
    from baz import Qux
    """

2. It would be really nice if we could have "type aliasing" (or whatever it's called). I have a lot of types that are just something like "Tuple[int, int]", so type checking doesn't help much. It would be much more useful if I have a value that Python sees as (e.g.) an int, but have the type system track it as something more specific. Something like this:

    DocId = typing.TypeAlias(int)
    DocLength = typing.TypeAlias(int)

    def id_and_length() -> Tuple[DocId, Length]:
        docid = 5  # type: DocId
        length = 10  # type: Length
        return docid, length


3. I can't tell from the PEP484 documentation... If I have a fully hinted base class, and subclass it with the same signature for every method, am I still supposed to hint all the arguments and returns in the subclass? That's what I've been doing, but it's pretty tedious.

Cheers,