Thanks for the comments, Cecil! To summarize, it sounds like the issues are:

1. It's unclear how the non-Python backends of various libraries would utilise (or perhaps even implement) support for PEP 646.
2. Runtime type checkers can't use stubs in .pyi files.
3. PEP 646 is relatively inflexible in the kinds of constraints on tensors it can encode.
4. That inflexibility might cause problems down the line: if we want to add new kinds of constraints, it's going to require such a major upheaval as to break backwards-compatibility.
5. In general, it would be better to aim for an approach that doesn't require TensorFlow, NumPy etc to do things differently to get good things happening.

As a particular alternative, we're talking about using either `Annotated` from PEP 593, or thinking about instead creating a tensor-like-thing-specific `Structure` thing.

---

Before responding to these individually, I want to speak to a higher-level crux. I think there are actually two questions here:

1. Is PEP 646 a good thing for Python in general?
2. Should we endorse PEP 646 as the answer to tensor typing in particular, and therefore try to encourage libraries like TensorFlow and NumPy to adopt it?

I believe the answer to the first question is "Yes". Variadic generics have plenty of other use cases (https://github.com/python/typing/issues/193), and although we wrote PEP 646 with tensor typing in mind, PEP 646 should still be usable for all these other cases too. PEP 646 will (hopefully) benefit folks other than 'us'

For the second question, though - I agree it's less clear whether the answer should be "Yes". To be clear, I'm not going to be advocating to the authors of TensorFlow etc that they should adopt PEP 646 any time soon (as grateful as I am for Stephan's endorsement of the PEP). I think it's too soon to say whether the way we suggest in the examples of PEP 646 is the way that we should lock ourselves into. My personal motivation in pushing this forward is: hey, it's plausible it could be the right way (or at least a good enough way), so let's get things to a stage where the friction is low enough that enough people try it that we can get feedback on what works and what doesn't for tensor typing. And we can get feedback even if TensorFlow etc don't adopt it, through stub libraries like TensorAnnotations.

(Is it the best way to get feedback? Couldn't we get useful feedback in a much easier way with approaches like Beartype and torchtyping? I think it depends how much you care about static analysis. Personally I do care quite a lot about it. But that's not to say I'm not excited about runtime methods too - and in fact I'll be adding a runtime checker to TensorAnnotations soon as well.)

(A reasonable counterargument is: we don't really need PEP 646 to test this hypothesis with e.g. TensorAnnotations. We can make do with type aliases for different ranks, like `Tensor2`, `Tensor3` and so on. I'm sympathetic to this counterargument. My original thinking was that the need for aliases is likely to create too much friction/general feelings of ickiness among users, such that we wouldn't be able to "run the experiment" properly. But I could be over-estimating the effect here.)

---

> 1. It's unclear how the non-Python backends of various libraries would utilise (or perhaps even implement) support for PEP 646.

This I'm actually confused about. Are classes like `Tensor` and `ndarray` really defined in C or C++? I assumed the classes themselves were defined in Python. (Though admittedly I don't know whether it would be as simple as just changing the class definition - e.g. https://github.com/tensorflow/tensorflow/issues/12345 suggests there might be some blockers to making `Tensor` generic in TensorFlow.)

> 2. Runtime type checkers can't use stubs in .pyi files.

Interesting - I hadn't thought about this.

I'm expecting that most of what runtime checkers care about will be the types of the tensor objects we're passing around - say, if a function is annotated as taking a float32 tensor, is the tensor's dtype really float32? The information in stubs - e.g. that some library function is supposed to permute the axes in a certain way - will be less important. But am I missing something here - what do you think?

> 3. PEP 646 is relatively inflexible in the kinds of constraints on tensors it can encode.

> 4. That inflexibility might cause problems down the line: if we want to add new kinds of constraints, it's going to require such a major upheaval as to break backwards-compatibility.

These are fair points, and part of why I think it'd be good to gather more feedback before advocating for uptake by e.g. TensorFlow and NumPy.

Having said that, I also don't want us to be too perfectionist about it. We could easily spend a long time trying to come up with something that was perfectly future-compatible.

> As a particular alternative, we're talking about using either `Annotated` from PEP 593, or thinking about instead creating a tensor-like-thing-specific `Structure` thing.

Right; we (some colleagues and I who were talking about it last year) did consider `Annotated` at first - but iirc I changed my mind because, man, we're so close to being able to do cool stuff with standard type checkers - whereas `Annotated` or the like would require new tooling if we wanted static analysis.

Having said that, I would be very excited to see the `Annotated` approach explored more thoroughly - e.g. if someone proposed a standard set of annotations to get us going, and implemented a runtime checker for them. I'd also be excited to see someone write a more detailed proposal for something like `Structure`.

---

So I think in summary, the two key points are:

• We should absolutely take our time to see how PEP 646-like tensor typing works in the real world before locking ourselves into anything, and in particular before advocating adoption by big libraries.
• Whether you like the PEP 646 way of doing things probably depends a lot on whether you like static analysis. If you don't care so much about static analysis, then I totally agree, it's inflexible compared to what you can do if you're doing your verification with a runtime checker. If you do care about static analysis, then it's a yay, because you get to use existing tooling.

On Tue, 13 Apr 2021 at 14:58, Cecil Curry <leycec@gmail.com> wrote:

PEP 646: It Probably Doesn't Do What Everyone Wants It To Do

Hi, all! Warmest greetings on this drizzly Spring evening from the squalid
Canadian wetlands, where even the mosquitoes think it's really too cold.

I'm Cecil Curry (@leycec), the plucky author of beartype
(https://github.com/beartype/beartype) – the pure-Python O(1) runtime type
checker that pretends to do alot while actually doing not too much at all.
Thus O(1). "You gets what you pays for," right?

My wife and I also co-authored a several million-line pure-Python multiphysics
biology simulator (https://gitlab.com/betse/betse) heavily leveraging the
standard scientific stack (e.g., NumPy, SciPy, Matplotlib, NetworkX). Those
were good years. I still had hair back then, so they had to have been good.

I was kindly redirected here by Ryan Soklaski (@rsokl), the brilliant @beartype
user I never knew I had and head instructor of the CogWorks course at the MIT
Beaver Works Summer Institute. Ryan accurately predicted that PEP 646
specifically and the vigorous discussion here surrounding tensor typing in
general would be right up my lakeside wheelhouse. Boy, is it! We're all over
this sort of deep structural science-oriented QA.

In fact, @beartype's userbase (https://github.com/beartype/beartype/stargazers)
is predominated by machine learning, computer vision, and data scientists.
We're poised on the awkward precipice between typing and tensors, because what
everyone (including my wife and I) always wanted from @beartype was runtime
type checking of scientific data structures.

PEP 646 promises all this... and more! But does it deliver? Does @leycec ever
regain his hair? Do Canadian brown bears groom themselves on spindly trees on
our front yard? The answers to all these questions and more follow shortly.

First, I'd like to profusely thank all the PEP 646 authors – Mark Mendoza,
Matthew Rahtz, Pradeep Kumar Srinivasan, Vincent Siles, and Guido van Rossum –
for their profuse work and dedication to the good cause.

Second, I'd like to profusely apologize a priori for any untoward argumentation
or firework shows I am about to generate. That's not the intention. Really! PEP
646 is currently at the steering committee stage. Nobody wants to see that
derailed, because hard work deserves its just reward and GitHub karma.

All that said... let's do this.

The real-world motivation of PEP 646 is to convince the various authors of
various popular numerical frameworks to adopt, embrace, and extend PEP 646.
This is a noble effort. But can this effort actually be realized?

The PEP 646 API assumes a Python-centric worldview. Specifically, PEP 646
assumes that numerical frameworks will subclass their data structures from
"typing.Generic", a magical pure-Python class. Those assumptions derive from
the first example in PEP 646:

    from typing import TypeVar, TypeVarTuple

    DType = TypeVar('DType')
    Shape = TypeVarTuple('Shape')

    class Array(Generic[DType, *Shape]):

        def __abs__(self) -> Array[DType, *Shape]: ...

        def __add__(self, other: Array[DType, *Shape]) -> Array[DType, *Shape]: ...

But this assumption may not necessarily hold. Most popular numerical frameworks
worth supporting (including numerical frameworks explicitly referenced in PEP
646, which means NumPy and TensorFlow) are implemented with at least two
layers: a low-level mostly Python-agnostic layer implemented in some
combination of C, C++, Fortran, or Rust referred to as the "core API" and one
or more high-level layers providing bindings to high-level languages. Python is
only one of many high-level languages supported by these frameworks.

This includes popular numerical frameworks one might erroneously assume from
their names to be Python-specific like PyTorch. Despite the name, PyTorch is
also implemented as a low-level Python-agnostic C++ layer
(https://pytorch.org/tutorials/advanced/cpp_frontend.html). The same goes for:

* The entirety of TensorFlow's core C++ API
  (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/framework).
* *Most* but not all of NumPy's core C API
  (https://numpy.org/doc/stable/reference/c-api/coremath.html), which NumPy
  developers are desperately refactoring to be Python-agnostic. I believe the
  core "numpy.core._multiarray_umath.ndarray" type is still Python-specific,
  but that's likely to change sometime this decade. (Don't quote me on that.
  Please. My GitHub karma can only take so many body blows.)

Is this a problem? It might very well be. Of course, it should be admitted at
this crucial juncture that my idea of a fun time includes scrambling over
tick-infested shrubland in an inhospitable wasteland for eight hours yesterday,
which only goes to show that you can't trust anyone who voluntarily lives in a
poorly insulated cottage plagued with mosquitoes. </ahem>

"typing.Generic" is magical, because it defines the magical PEP 487
__init_subclass__() dunder method. Despite banging my balding head against
CPython documentation and my ketchup-stained keyboard for several hours, I'm
kinda unconvinced that a C[++]-based class can even subclass one or more
magical or non-magical pure-Python classes. I've never seen anyone do either.
Then again, I've never seen a circumpolar double rainbow on Christmas Eve
composed of flying winged bugbears. Who's to say that can't happen? I'd sure
like to see that someday.

So let's assume for the sake of debate that's feasible – that C[++]-based
classes can in fact subclass one or more magical pure-Python classes. Under
that generous assumption, it's still unlikely that numerical frameworks that
were either Python-agnostic from the get-go (like TensorFlow) or are rapidly
refactoring their core APIs away from Python (like NumPy) would revert those
fundamental decisions just to appease our small but vocal cadre of Python
type-checking aficionados. (That's us: the good guys in the distinct minority.)

Now I *know* what you're thinking. "Ah-ha! You poor sad Canadian hillbilly. All
we need to do is strenuously encourage the authors of these numerical
frameworks to define entirely new Python-specific classes in their Python
bindings that comply with PEP 646 by subclassing "typing.Generic" in the exact
way we want them to! It's so blindingly simple! If only you still had luxurious
hair, @leycec. Then you might have thought of that. Alas. Here we are."

In the dark ruminations of your monitor-lit devcave, you may even be
dangerously considering unspeakable horrors like adding a new type-checking
shim like this into the perfidious bowels of the NumPy codebase:

    from numpy.core._multiarray_umath import ndarray
    from typing import TypeVar, TypeVarTuple

    DType = TypeVar('DType')
    Shape = TypeVarTuple('Shape')

    class NDArray(ndarray, Generic[DType, *Shape]):

        def __abs__(self) -> Array[DType, *Shape]:
            return super().__abs__()

        def __add__(self, other: Array[DType, *Shape]) -> Array[DType, *Shape]:
            return super().__add__(other)

Right? Something unmentionable like that, right? Of course, we can all admit to
ourselves that's horrible, because that imposes one additional stack frame per
method call. So, nobody will do that.

Now I *know* what you're thinking. "Ah-ha! You poor sad Canadian person of
indeterminate mental stability. All we need to do is define .pyi-formatted stub
files instead expressing the same type semantics! If only your limbs weren't
painfully arthritic, @leycec. Then you might have thought of that. Alas. Here
we are."

Right? Something inexpressible like that, right? Of course, we can all admit to
ourselves that's horrible too, because .pyi-formatted stub files aren't sanely
accessible at runtime. And we are back where we started in a cursed reenactment
of "Blair Witch Project: Tensor Typing Edition."

You're now thinking of either bolting type hints onto C[++]-based callables or
wholly reimplementing "typing.Generic" in C, aren't you? I knew it! You are!
And at this point, everyone should be thinking: "This doesn't quite seem right
anymore."

The core issue here is that PEP 646 requires someone who is not us to change
their workflow for us. It's not necessarily clear that that will ever happen.
Sure, it might. I'm an optimistic man, which means I'm not a betting man.

Even if that utopic Xanadu were to become manifest, PEP 646 still seems
problematic. My PEP 484 isn't strong, but it's unclear how you would:

* Type tensor methods returning new tensors satisfying different constraints
  (especially of differing shapes).
* Type tensor constraints in a forward-compatible manner. Tensor constraints
  declared by PEP 646 are crazy fragile. They lock any numerical framework that
  complies into a specific preordained ordering of tensor constraints. A tensor
  typed as "class Array(Generic[DType, *Shape])" can *never* be typed as
  anything else for all time, which is a pretty long time. Want to add another
  constraint like, say, PyTorch's newly introduced "layout" tensor attribute
  that fundamentally changes how one uses passed or returned
  tensors? (https://pytorch.org/docs/stable/tensor_attributes.html#torch.torch.layout)
  **Sorry.** No one can ever do that, because the original specification of
  "class Array(Generic[DType, *Shape])" prematurely locked everyone into
  checking only those two constraints.

Surely we have sane alternatives? We do. Many! Oodles! Bucket-loads! And other
colourful idioms for "lots." They all share one attribute in common: they don't
require someone who is not us to change their fundamental workflow for us.

The PEP 593 "typing.Annotated" class offers an illustrative example:

    import numpy as np
    from tensorflow.types.core import Tensor
    from typing import Annotated

    PixellatedRetroImage = Annotated[Tensor, np.float32, [640, 480]]

    def upscale_image(image: PixellatedRetroImage) -> PixellatedRetroImage: ...

That has the benefit of working now under Python >= 3.9 without necessitating
that external authors, open-source communities, or for-profit corporations do
unlikely (and perhaps infeasible) stuff for us. We'll be pursuing a similar
solution to that in @beartype, because it works now and satisfies everyone's
downstream concerns.

Of course, "typing" wouldn't want to do something like that. "typing" would
want to do something even more robust, extensible, and absurdly awesome like:

    import numpy as np
    from tensorflow.types.core import Tensor
    from typing import Structure

    PixellatedRetroImage = Structure[
        Tensor, {dtype=np.float32, shape=[640, 480]}]

    def upscale_image(image: PixellatedRetroImage) -> PixellatedRetroImage: ...

Even better, right? To avoid conflicts with user-defined "typing.Annotated"
type hints that already exist in the wild, we define a new "typing.Structure"
class suitable for type-hinting data structure constraints. We then define two
constraints in an extensible manner supporting forward-compatible changes,
improvements, and modifications to third-party APIs.

Of course, that's probably not quite "typing" enough. Right? Not quite enough
square braces above and more than a little hostile to static analysis, which is
fair enough. So let's delve one step deeper into the cavernous labyrinth of
awesomeness that solves all tensor typing concerns:

    import numpy as np
    from tensorflow.types.core import Tensor
    from typing import Structure, Dtype, Shape

    PixellatedRetroImage = Structure[
        Tensor, Dtype[np.float32], Shape[640, 480]]

    def upscale_image(image: PixellatedRetroImage) -> PixellatedRetroImage: ...

Yup. It doesn't get more "typing" than that. You've got square braces all over
the place; you've got well-named constraints generically applicable to the
gamut of popular frameworks; you've got order-invariant extensibility, because
it no longer matters what order users declare "Dtype[...]" and "Shape[...]"
constraints in (or whether users even declare those constraints); you've got
built-in support for both static and runtime analysis. And you've got all that
without burdening third-party vendors with a Python-specific C[++]-unfriendly
API they're unlikely to ever adopt. In short, you've got everything you could
probably, possibly, or likely ever want.


...which means it'll never happen. PEP 646 is all but a fait accompli at this
eleventh hour. That's fine, I guess. But that's also kinda *not* fine, you
know? We can and should do substantially better.

I reckon we can.

In closing, thanks for all the Pythonic goodness over the past several decades.
Everyone's amazing! Regardless of where and how Python type-checking intersects
with scientific computing in the decades to come, I'm indefatigably proud of
everything everyone's accomplished here. Keep on rocking the type-safe world.
_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/
Member address: mrahtz@google.com

_______________________________________________
Typing-sig mailing list -- typing-sig@python.org
To unsubscribe send an email to typing-sig-leave@python.org
https://mail.python.org/mailman3/lists/typing-sig.python.org/
Member address: gohanpra@gmail.com