Mailman 3 PEP 646: It Probably Doesn't Do What Everyone Wants It To Do - Typing-sig

13 Apr 2021

      PEP 646: It Probably Doesn't Do What Everyone Wants It To Do

Hi, all! Warmest greetings on this drizzly Spring evening from the squalid
Canadian wetlands, where even the mosquitoes think it's really too cold.

I'm Cecil Curry (@leycec), the plucky author of beartype
(https://github.com/beartype/beartype) – the pure-Python O(1) runtime type
checker that pretends to do alot while actually doing not too much at all.
Thus O(1). "You gets what you pays for," right?

My wife and I also co-authored a several million-line pure-Python multiphysics
biology simulator (https://gitlab.com/betse/betse) heavily leveraging the
standard scientific stack (e.g., NumPy, SciPy, Matplotlib, NetworkX). Those
were good years. I still had hair back then, so they had to have been good.

I was kindly redirected here by Ryan Soklaski (@rsokl), the brilliant @beartype
user I never knew I had and head instructor of the CogWorks course at the MIT
Beaver Works Summer Institute. Ryan accurately predicted that PEP 646
specifically and the vigorous discussion here surrounding tensor typing in
general would be right up my lakeside wheelhouse. Boy, is it! We're all over
this sort of deep structural science-oriented QA.

In fact, @beartype's userbase (https://github.com/beartype/beartype/stargazers)
is predominated by machine learning, computer vision, and data scientists.
We're poised on the awkward precipice between typing and tensors, because what
everyone (including my wife and I) always wanted from @beartype was runtime
type checking of scientific data structures.

PEP 646 promises all this... and more! But does it deliver? Does @leycec ever
regain his hair? Do Canadian brown bears groom themselves on spindly trees on
our front yard? The answers to all these questions and more follow shortly.

First, I'd like to profusely thank all the PEP 646 authors – Mark Mendoza,
Matthew Rahtz, Pradeep Kumar Srinivasan, Vincent Siles, and Guido van Rossum – 
for their profuse work and dedication to the good cause.

Second, I'd like to profusely apologize a priori for any untoward argumentation
or firework shows I am about to generate. That's not the intention. Really! PEP
646 is currently at the steering committee stage. Nobody wants to see that
derailed, because hard work deserves its just reward and GitHub karma.

All that said... let's do this.

The real-world motivation of PEP 646 is to convince the various authors of
various popular numerical frameworks to adopt, embrace, and extend PEP 646.
This is a noble effort. But can this effort actually be realized?

The PEP 646 API assumes a Python-centric worldview. Specifically, PEP 646
assumes that numerical frameworks will subclass their data structures from
"typing.Generic", a magical pure-Python class. Those assumptions derive from
the first example in PEP 646:

    from typing import TypeVar, TypeVarTuple

    DType = TypeVar('DType')
    Shape = TypeVarTuple('Shape')

    class Array(Generic[DType, *Shape]):

        def __abs__(self) -> Array[DType, *Shape]: ...

        def __add__(self, other: Array[DType, *Shape]) -> Array[DType, *Shape]: ...

But this assumption may not necessarily hold. Most popular numerical frameworks
worth supporting (including numerical frameworks explicitly referenced in PEP
646, which means NumPy and TensorFlow) are implemented with at least two
layers: a low-level mostly Python-agnostic layer implemented in some
combination of C, C++, Fortran, or Rust referred to as the "core API" and one
or more high-level layers providing bindings to high-level languages. Python is
only one of many high-level languages supported by these frameworks.

This includes popular numerical frameworks one might erroneously assume from
their names to be Python-specific like PyTorch. Despite the name, PyTorch is
also implemented as a low-level Python-agnostic C++ layer
(https://pytorch.org/tutorials/advanced/cpp_frontend.html). The same goes for:

* The entirety of TensorFlow's core C++ API
  (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/framewo...).
* *Most* but not all of NumPy's core C API
  (https://numpy.org/doc/stable/reference/c-api/coremath.html), which NumPy
  developers are desperately refactoring to be Python-agnostic. I believe the
  core "numpy.core._multiarray_umath.ndarray" type is still Python-specific,
  but that's likely to change sometime this decade. (Don't quote me on that.
  Please. My GitHub karma can only take so many body blows.)

Is this a problem? It might very well be. Of course, it should be admitted at
this crucial juncture that my idea of a fun time includes scrambling over
tick-infested shrubland in an inhospitable wasteland for eight hours yesterday,
which only goes to show that you can't trust anyone who voluntarily lives in a
poorly insulated cottage plagued with mosquitoes. </ahem>

"typing.Generic" is magical, because it defines the magical PEP 487
__init_subclass__() dunder method. Despite banging my balding head against
CPython documentation and my ketchup-stained keyboard for several hours, I'm
kinda unconvinced that a C[++]-based class can even subclass one or more
magical or non-magical pure-Python classes. I've never seen anyone do either.
Then again, I've never seen a circumpolar double rainbow on Christmas Eve
composed of flying winged bugbears. Who's to say that can't happen? I'd sure
like to see that someday.

So let's assume for the sake of debate that's feasible – that C[++]-based
classes can in fact subclass one or more magical pure-Python classes. Under
that generous assumption, it's still unlikely that numerical frameworks that
were either Python-agnostic from the get-go (like TensorFlow) or are rapidly
refactoring their core APIs away from Python (like NumPy) would revert those
fundamental decisions just to appease our small but vocal cadre of Python
type-checking aficionados. (That's us: the good guys in the distinct minority.)

Now I *know* what you're thinking. "Ah-ha! You poor sad Canadian hillbilly. All
we need to do is strenuously encourage the authors of these numerical
frameworks to define entirely new Python-specific classes in their Python
bindings that comply with PEP 646 by subclassing "typing.Generic" in the exact
way we want them to! It's so blindingly simple! If only you still had luxurious
hair, @leycec. Then you might have thought of that. Alas. Here we are."

In the dark ruminations of your monitor-lit devcave, you may even be
dangerously considering unspeakable horrors like adding a new type-checking
shim like this into the perfidious bowels of the NumPy codebase:

    from numpy.core._multiarray_umath import ndarray
    from typing import TypeVar, TypeVarTuple

    DType = TypeVar('DType')
    Shape = TypeVarTuple('Shape')

    class NDArray(ndarray, Generic[DType, *Shape]):

        def __abs__(self) -> Array[DType, *Shape]:
            return super().__abs__()

        def __add__(self, other: Array[DType, *Shape]) -> Array[DType, *Shape]:
            return super().__add__(other)

Right? Something unmentionable like that, right? Of course, we can all admit to
ourselves that's horrible, because that imposes one additional stack frame per 
method call. So, nobody will do that.

Now I *know* what you're thinking. "Ah-ha! You poor sad Canadian person of
indeterminate mental stability. All we need to do is define .pyi-formatted stub
files instead expressing the same type semantics! If only your limbs weren't
painfully arthritic, @leycec. Then you might have thought of that. Alas. Here
we are."

Right? Something inexpressible like that, right? Of course, we can all admit to
ourselves that's horrible too, because .pyi-formatted stub files aren't sanely
accessible at runtime. And we are back where we started in a cursed reenactment
of "Blair Witch Project: Tensor Typing Edition."

You're now thinking of either bolting type hints onto C[++]-based callables or
wholly reimplementing "typing.Generic" in C, aren't you? I knew it! You are!
And at this point, everyone should be thinking: "This doesn't quite seem right
anymore."

The core issue here is that PEP 646 requires someone who is not us to change
their workflow for us. It's not necessarily clear that that will ever happen.
Sure, it might. I'm an optimistic man, which means I'm not a betting man.

Even if that utopic Xanadu were to become manifest, PEP 646 still seems
problematic. My PEP 484 isn't strong, but it's unclear how you would:

* Type tensor methods returning new tensors satisfying different constraints
  (especially of differing shapes).
* Type tensor constraints in a forward-compatible manner. Tensor constraints
  declared by PEP 646 are crazy fragile. They lock any numerical framework that
  complies into a specific preordained ordering of tensor constraints. A tensor
  typed as "class Array(Generic[DType, *Shape])" can *never* be typed as
  anything else for all time, which is a pretty long time. Want to add another
  constraint like, say, PyTorch's newly introduced "layout" tensor attribute
  that fundamentally changes how one uses passed or returned
  tensors? (https://pytorch.org/docs/stable/tensor_attributes.html#torch.torch.layout)
  **Sorry.** No one can ever do that, because the original specification of
  "class Array(Generic[DType, *Shape])" prematurely locked everyone into
  checking only those two constraints.

Surely we have sane alternatives? We do. Many! Oodles! Bucket-loads! And other
colourful idioms for "lots." They all share one attribute in common: they don't
require someone who is not us to change their fundamental workflow for us.

The PEP 593 "typing.Annotated" class offers an illustrative example:

    import numpy as np
    from tensorflow.types.core import Tensor
    from typing import Annotated

    PixellatedRetroImage = Annotated[Tensor, np.float32, [640, 480]]

    def upscale_image(image: PixellatedRetroImage) -> PixellatedRetroImage: ...

That has the benefit of working now under Python >= 3.9 without necessitating
that external authors, open-source communities, or for-profit corporations do
unlikely (and perhaps infeasible) stuff for us. We'll be pursuing a similar
solution to that in @beartype, because it works now and satisfies everyone's
downstream concerns.

Of course, "typing" wouldn't want to do something like that. "typing" would
want to do something even more robust, extensible, and absurdly awesome like:

    import numpy as np
    from tensorflow.types.core import Tensor
    from typing import Structure

    PixellatedRetroImage = Structure[
        Tensor, {dtype=np.float32, shape=[640, 480]}]

    def upscale_image(image: PixellatedRetroImage) -> PixellatedRetroImage: ...

Even better, right? To avoid conflicts with user-defined "typing.Annotated"
type hints that already exist in the wild, we define a new "typing.Structure"
class suitable for type-hinting data structure constraints. We then define two
constraints in an extensible manner supporting forward-compatible changes,
improvements, and modifications to third-party APIs.

Of course, that's probably not quite "typing" enough. Right? Not quite enough
square braces above and more than a little hostile to static analysis, which is
fair enough. So let's delve one step deeper into the cavernous labyrinth of
awesomeness that solves all tensor typing concerns:

    import numpy as np
    from tensorflow.types.core import Tensor
    from typing import Structure, Dtype, Shape

    PixellatedRetroImage = Structure[
        Tensor, Dtype[np.float32], Shape[640, 480]]

    def upscale_image(image: PixellatedRetroImage) -> PixellatedRetroImage: ...

Yup. It doesn't get more "typing" than that. You've got square braces all over
the place; you've got well-named constraints generically applicable to the
gamut of popular frameworks; you've got order-invariant extensibility, because
it no longer matters what order users declare "Dtype[...]" and "Shape[...]"
constraints in (or whether users even declare those constraints); you've got
built-in support for both static and runtime analysis. And you've got all that
without burdening third-party vendors with a Python-specific C[++]-unfriendly
API they're unlikely to ever adopt. In short, you've got everything you could
probably, possibly, or likely ever want.

...which means it'll never happen. PEP 646 is all but a fait accompli at this
eleventh hour. That's fine, I guess. But that's also kinda *not* fine, you
know? We can and should do substantially better.

I reckon we can.

In closing, thanks for all the Pythonic goodness over the past several decades.
Everyone's amazing! Regardless of where and how Python type-checking intersects
with scientific computing in the decades to come, I'm indefatigably proud of
everything everyone's accomplished here. Keep on rocking the type-safe world.

PEP 646: It Probably Doesn't Do What Everyone Wants It To Do

Cecil Curry

Brandt Bucher

Stephan Hoyer

Matthew Rahtz

S Pradeep Kumar

tags

participants (5)