Re: [Python-ideas] Runtime types vs static types
For some background on the removal of __instancecheck__, check the linked issues here: https://github.com/python/typing/issues/135 -- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone elsehttp://refi64.com On Jun 25, 2017 at 8:11 AM, <Koos Zevenhoven <k7hoven@gmail.com>> wrote: On Sat, Jun 24, 2017 at 11:30 PM, Lucas Wiman <lucas.wiman@gmail.com> wrote:
On Sat, Jun 24, 2017 at 12:42 PM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
There has been some discussion here and there concerning the differences between runtime types and static types (mypy etc.). What I write below is not really an idea or proposal---just a perspective, or a topic that people may want to discuss. Since the discussion on this is currently very fuzzy and scattered and not really happening either AFAICT (I've probably missed many discussions, though). Anyway, I thought I'd give it a shot:
[...]
Regarding runtime types and isinstance, let's look at the Iterable[int]
example. For this case, there are a few options:
1) Don't implement isinstance
This is problematic for runtime uses of annotations.
2) isinstance([1, '2', 'three'], Iterable[int]) returns True
This is in fact now the case. This is ok for many runtime situations, but lacks precision compared to the static version. One may want to distinguish between Iterable[int] and Iterable[str] at runtime (e.g. the multidispatch example above).
3) Check as much as you can at runtime
There could be something like Reiterable, which means the object is not consumed by iterating over it, so one could actually check if all elements are instances of int. This would be useful in some situations, but not available for every object. Furthermore, the check could take an arbitrary amount of time so it is not really suitable for things like multidispatch or some matching constructs etc., where the performance overhead of the type check is really important.
4) Do a deeper check than in (2) but trust the annotations
For example, an instance of a class that has a method like
def __iter__(self) -> Iterator[int]: some code
could be identified as Iterable[int] at runtime, even if it is not guaranteed that all elements are really integers.
On the other hand, an object returned by
def get_ints() -> Iterable[int]: some code
does not know its own annotations, so the check is difficult to do at runtime. And of course, there may not be annotations available.
5) Something else?
And what about PEP544 (protocols), which is being drafted? The PEP seems to aim for having type objects that represent duck-typing protocols/interfaces. Checking whether a protocol is implemented by an object or type is clearly a useful thing to do at runtime, but it is not really clear if isinstance would be a guaranteed feature for PEP544 Protocols.
So one question is, is it possible to draw the lines between what works with isinstance and what doesn't, and between what details are checked by isinstance and what aren't? -- Or should insinstance be reserved for a more limited purpose, and add another check function, say `implements(...)`, which would perhaps guarantee some answer for all combinations of object and type?
I'm guessing to implement PEP 544, many of the `__instancecheck__` and `__subclasscheck__` methods in `typing.py` would need to be updated to check the `__annotations__` of the class of the object it's passed against its own definition, (covered in this section <https://www.python.org/dev/peps/pep-0544/#runtime-decorator-and-narrowing-ty...> of the PEP).
I may have missed something, but I believe PEP544 is not suggesting that annotations would have any effect on isinstance. Instead, isinstance would by default not work.
I've been somewhat surprised that many of the `__instancecheck__` implementations do not work at runtime, even when the implementation would be trivial (e.g. for `Union`), or would not have subtle edge cases due to immutability (e.g. for `Tuple`, which cannot be used for checking parameterized instances). This seems like counterintuitive behavior that would be straightforward to fix, unless there are subtleties & edge cases I'm missing.
Tuple is an interesting case, because for small tuples (say 2- or 3-tuples), it makes perfect sense to check the types of all elements for some runtime purposes. Regarding Union, I believe the current situation has a lot to do with the fact that the relation between type annotations and runtime behavior hasn't really settled yet. If people are amenable to updating those cases, I'd be interested in
submitting a patch to that effect.
Thanks for letting us know. (There may not be an instant decision on this particular case, though, but who knows :) -- Koos -- + Koos Zevenhoven + http://twitter.com/k7hoven + _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
For some background on the removal of __instancecheck__, check the linked issues here:
from typing import * isinstance(0, Union[int, float]) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/lucaswiman/.pyenv/versions/3.6/lib/python3.6/typing.py",
Thanks for the reference (the most relevant discussion starts here <https://github.com/python/typing/issues/136#issuecomment-104698674>). That said, I think I totally disagree with the underlying philosophy of throwing away a useful and intuitive feature (having `is_instance(foo, Union[Bar, Baz])` just work as you'd naively expect) in the name of making sure that people *understand* there's a distinction between types and classes. This seems opposed to the "zen" of python that there should be exactly one obvious way to do it, since (1) there isn't a way to do it without a third party library, and (2) the obvious way to do it is with `isinstance` and `issubclass`. Indeed, the current implementation makes it somewhat nonobvious even how to implement this functionality yourself in a third-party library (see this gist <https://gist.github.com/lucaswiman/21373bea33ccd2c5e868ec52b6eff412>). One of the first things I did when playing around with the `typing` module was to fire up the REPL, and try runtime typechecks: line 767, in __instancecheck__ raise TypeError("Unions cannot be used with isinstance().") TypeError: Unions cannot be used with isinstance(). I think the natural reaction of naive users of the library is "That's annoying. Why? What is this library good for?", not "Ah, I've sagely learned a valuable lesson about the subtle-and-important-though-unmentioned distinction between types and classes!" The restriction against runtime type checking makes `typing` pretty much *only* useful when used with the external library `mypy` (or when writing a library with the same purpose as `mypy`), which is a pretty unusual situation for a standard library module. Mark Shannon's example also specifically does not apply to the types I'm thinking of for the reasons I mentioned:
For example, List[int] and List[str] and mutually incompatible types, yet isinstance([], List[int]) and isinstance([], List[str)) both return true.
There is no corresponding objection for `Union`; I can't think of any* inconsistencies or runtime type changes that would result from defining `_Union.__instancecheck__` as `any(isinstance(obj, t) for t in self.__args__`. For `Tuple`, it's true that `()` would be an instance of `Tuple[X, ...]` for all types X. However, the objection for the `List` case (IIUC; extrapolating slightly) is that the type of the object could change depending on what's added to it. That's not true for tuples since they're immutable, so it's not *inconsistent* to say that `()` is an instance of `Tuple[int, ...]` and `Tuple[str, ...]`, it's just applying a sensible definition to the base case of an empty tuple.
That said, it sounds like the decision has already been made, and this would be quite useful functionality to have in *some* form. What do people think about implementing `__contains__` (for `__instancecheck__`) and `__lt__` (for `__subclasscheck__`) for these cases? Then there would still be a convenient syntax for doing runtime type checking/analysis, but wouldn't violate Mark Shannon's objections. Best, Lucas * Counterexamples welcomed, of course! <http://www.dictionary.com/browse/more-things-in-heaven-and-earth--horatio> On Sun, Jun 25, 2017 at 6:21 AM, rymg19@gmail.com <rymg19@gmail.com> wrote:
For some background on the removal of __instancecheck__, check the linked issues here:
https://github.com/python/typing/issues/135
-- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone elsehttp://refi64.com
On Jun 25, 2017 at 8:11 AM, <Koos Zevenhoven <k7hoven@gmail.com>> wrote:
On Sat, Jun 24, 2017 at 11:30 PM, Lucas Wiman <lucas.wiman@gmail.com> wrote:
On Sat, Jun 24, 2017 at 12:42 PM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
There has been some discussion here and there concerning the differences between runtime types and static types (mypy etc.). What I write below is not really an idea or proposal---just a perspective, or a topic that people may want to discuss. Since the discussion on this is currently very fuzzy and scattered and not really happening either AFAICT (I've probably missed many discussions, though). Anyway, I thought I'd give it a shot:
[...]
Regarding runtime types and isinstance, let's look at the Iterable[int]
example. For this case, there are a few options:
1) Don't implement isinstance
This is problematic for runtime uses of annotations.
2) isinstance([1, '2', 'three'], Iterable[int]) returns True
This is in fact now the case. This is ok for many runtime situations, but lacks precision compared to the static version. One may want to distinguish between Iterable[int] and Iterable[str] at runtime (e.g. the multidispatch example above).
3) Check as much as you can at runtime
There could be something like Reiterable, which means the object is not consumed by iterating over it, so one could actually check if all elements are instances of int. This would be useful in some situations, but not available for every object. Furthermore, the check could take an arbitrary amount of time so it is not really suitable for things like multidispatch or some matching constructs etc., where the performance overhead of the type check is really important.
4) Do a deeper check than in (2) but trust the annotations
For example, an instance of a class that has a method like
def __iter__(self) -> Iterator[int]: some code
could be identified as Iterable[int] at runtime, even if it is not guaranteed that all elements are really integers.
On the other hand, an object returned by
def get_ints() -> Iterable[int]: some code
does not know its own annotations, so the check is difficult to do at runtime. And of course, there may not be annotations available.
5) Something else?
And what about PEP544 (protocols), which is being drafted? The PEP seems to aim for having type objects that represent duck-typing protocols/interfaces. Checking whether a protocol is implemented by an object or type is clearly a useful thing to do at runtime, but it is not really clear if isinstance would be a guaranteed feature for PEP544 Protocols.
So one question is, is it possible to draw the lines between what works with isinstance and what doesn't, and between what details are checked by isinstance and what aren't? -- Or should insinstance be reserved for a more limited purpose, and add another check function, say `implements(...)`, which would perhaps guarantee some answer for all combinations of object and type?
I'm guessing to implement PEP 544, many of the `__instancecheck__` and `__subclasscheck__` methods in `typing.py` would need to be updated to check the `__annotations__` of the class of the object it's passed against its own definition, (covered in this section <https://www.python.org/dev/peps/pep-0544/#runtime-decorator-and-narrowing-ty...> of the PEP).
I may have missed something, but I believe PEP544 is not suggesting that annotations would have any effect on isinstance. Instead, isinstance would by default not work.
I've been somewhat surprised that many of the `__instancecheck__` implementations do not work at runtime, even when the implementation would be trivial (e.g. for `Union`), or would not have subtle edge cases due to immutability (e.g. for `Tuple`, which cannot be used for checking parameterized instances). This seems like counterintuitive behavior that would be straightforward to fix, unless there are subtleties & edge cases I'm missing.
Tuple is an interesting case, because for small tuples (say 2- or 3-tuples), it makes perfect sense to check the types of all elements for some runtime purposes. Regarding Union, I believe the current situation has a lot to do with the fact that the relation between type annotations and runtime behavior hasn't really settled yet.
If people are amenable to updating those cases, I'd be interested in
submitting a patch to that effect.
Thanks for letting us know. (There may not be an instant decision on this particular case, though, but who knows :)
-- Koos
-- + Koos Zevenhoven + http://twitter.com/k7hoven + _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/ mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/ codeofconduct/
On Sun, Jun 25, 2017 at 7:13 PM, Lucas Wiman <lucas.wiman@gmail.com> wrote: [...]
from typing import * isinstance(0, Union[int, float]) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/lucaswiman/.pyenv/versions/3.6/lib/python3.6/typing.py", line 767, in __instancecheck__ raise TypeError("Unions cannot be used with isinstance().") TypeError: Unions cannot be used with isinstance().
I think the natural reaction of naive users of the library is "That's annoying. Why? What is this library good for?", not "Ah, I've sagely learned a valuable lesson about the subtle-and-important-though-unmentioned distinction between types and classes!" The restriction against runtime type checking makes `typing` pretty much *only* useful when used with the external library `mypy` (or when writing a library with the same purpose as `mypy`), which is a pretty unusual situation for a standard library module.
Mark Shannon's example also specifically does not apply to the types I'm thinking of for the reasons I mentioned:
For example, List[int] and List[str] and mutually incompatible types, yet isinstance([], List[int]) and isinstance([], List[str)) both return true.
There is no corresponding objection for `Union`; I can't think of any* inconsistencies or runtime type changes that would result from defining `_Union.__instancecheck__` as `any(isinstance(obj, t) for t in self.__args__`.
One thing is that, as long as not all types support isinstance, then also isinstance(obj, Union[x, y, z]) will fail for some x, y, z. From some perspective, that may not be an issue, but on the other hand, it may invite people to think that all types do support isinstance. For `Tuple`, it's true that `()` would be an instance of `Tuple[X, ...]`
for all types X. However, the objection for the `List` case (IIUC; extrapolating slightly) is that the type of the object could change depending on what's added to it. That's not true for tuples since they're immutable, so it's not *inconsistent* to say that `()` is an instance of `Tuple[int, ...]` and `Tuple[str, ...]`, it's just applying a sensible definition to the base case of an empty tuple.
Yes, but then `isinstance(tuple(range(1000000)), Tuple[int, ...])` would be a difficult case. I think it all comes down to how much isinstance should pretend to be able to do with *types* (as opposed to only classes). Maybe isinstance is not the future of runtime type checking ;). Or should isinstance have a third return value like `Possibly`? -- Koos
Yes, but then `isinstance(tuple(range(1000000)), Tuple[int, ...])` would be a difficult case.
Yes, many methods on million-element tuples are slow. :-) Python usually chooses the more intuitive behavior over the faster behavior when there is a choice. IIUC, the objections don't have anything to do with speed. - Lucas On Sun, Jun 25, 2017 at 10:11 AM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
On Sun, Jun 25, 2017 at 7:13 PM, Lucas Wiman <lucas.wiman@gmail.com> wrote: [...]
from typing import * isinstance(0, Union[int, float]) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/lucaswiman/.pyenv/versions/3.6/lib/python3.6/typing.py", line 767, in __instancecheck__ raise TypeError("Unions cannot be used with isinstance().") TypeError: Unions cannot be used with isinstance().
I think the natural reaction of naive users of the library is "That's annoying. Why? What is this library good for?", not "Ah, I've sagely learned a valuable lesson about the subtle-and-important-though-unmentioned distinction between types and classes!" The restriction against runtime type checking makes `typing` pretty much *only* useful when used with the external library `mypy` (or when writing a library with the same purpose as `mypy`), which is a pretty unusual situation for a standard library module.
Mark Shannon's example also specifically does not apply to the types I'm thinking of for the reasons I mentioned:
For example, List[int] and List[str] and mutually incompatible types, yet isinstance([], List[int]) and isinstance([], List[str)) both return true.
There is no corresponding objection for `Union`; I can't think of any* inconsistencies or runtime type changes that would result from defining `_Union.__instancecheck__` as `any(isinstance(obj, t) for t in self.__args__`.
One thing is that, as long as not all types support isinstance, then also isinstance(obj, Union[x, y, z]) will fail for some x, y, z. From some perspective, that may not be an issue, but on the other hand, it may invite people to think that all types do support isinstance.
For `Tuple`, it's true that `()` would be an instance of `Tuple[X, ...]`
for all types X. However, the objection for the `List` case (IIUC; extrapolating slightly) is that the type of the object could change depending on what's added to it. That's not true for tuples since they're immutable, so it's not *inconsistent* to say that `()` is an instance of `Tuple[int, ...]` and `Tuple[str, ...]`, it's just applying a sensible definition to the base case of an empty tuple.
Yes, but then `isinstance(tuple(range(1000000)), Tuple[int, ...])` would be a difficult case.
I think it all comes down to how much isinstance should pretend to be able to do with *types* (as opposed to only classes). Maybe isinstance is not the future of runtime type checking ;). Or should isinstance have a third return value like `Possibly`?
-- Koos
On Sun, Jun 25, 2017 at 8:47 PM, Lucas Wiman <lucas.wiman@gmail.com> wrote:
Yes, but then `isinstance(tuple(range(1000000)), Tuple[int, ...])` would be a difficult case.
Yes, many methods on million-element tuples are slow. :-)
Python usually chooses the more intuitive behavior over the faster behavior when there is a choice. IIUC, the objections don't have anything to do with speed.
Sure, performance cannot dictate everything. But if you look at, for example, that multidispatch example I wrote in the OP, it would not be wise to do that expensive check every time. Also, people don't expect an isinstance check to be expensive. But looking at annotations, it is often possible to get the desired answer without checking all elements in a tuple (or especially without consuming generators etc. to check the types). -- Koos
After rereading Koos' OP, I can now see that he was referring to a different kind of runtime type checking than I am interested in. There was a distinction I was unaware the core typing devs make between "typing-style types" and "classes" that I'll discuss further in the typing repo itself. Apologies for the digression. - Lucas On Sun, Jun 25, 2017 at 11:13 AM, Koos Zevenhoven <k7hoven@gmail.com> wrote:
On Sun, Jun 25, 2017 at 8:47 PM, Lucas Wiman <lucas.wiman@gmail.com> wrote:
Yes, but then `isinstance(tuple(range(1000000)), Tuple[int, ...])` would be a difficult case.
Yes, many methods on million-element tuples are slow. :-)
Python usually chooses the more intuitive behavior over the faster behavior when there is a choice. IIUC, the objections don't have anything to do with speed.
Sure, performance cannot dictate everything. But if you look at, for example, that multidispatch example I wrote in the OP, it would not be wise to do that expensive check every time. Also, people don't expect an isinstance check to be expensive. But looking at annotations, it is often possible to get the desired answer without checking all elements in a tuple (or especially without consuming generators etc. to check the types).
-- Koos
Sorry, I was not able to completely digest the OP, but I think there are some points I need to clarify. 1. Distinction between runtime classes and static types is quite sane and a simple idea. A runtime class is something associated with an actual object, while static type is something associated with an AST node. Mixing them would be misleading, since they "live in parallel planes". Although it is true that there is a type that corresponds to every runtime class. 2. Currently isinstance(obj, List[int]) fails with TypeError, ditto for issubclass and for user defined generic classes: class C(Generic[T]): ... isinstance(obj, C) # works, returns only True or False isinstance(obj, C[int]) # TypeError issubclass(cls, C) # works issubclass(cls, C[int]) # raisesTypeError 3. User defined protocols will by default raise TypeError with isinstance(), but the user can opt-in (using @runtime decorator) for the same behavior as normal generics, this is how typing.Iterable currently works: class MyIter: def __iter__(self): return [42] isinstance(MyIter(), Iterable) # True isinstance(MyIter(), Iterable[int]) # TypeError class A(Protocol[T]): x: T isinstance(obj, A) # TypeError @runtime class B(Protocol[T]): y: T isinstance(obj, B) # True or False depending on whether 'obj' has attribute 'y' isinstance(obj, B[int]) # Still TypeError -- Ivan
On Thu, Jun 29, 2017 at 12:40 PM, Ivan Levkivskyi <levkivskyi@gmail.com> wrote:
Sorry, I was not able to completely digest the OP, but I think there are some points I need to clarify.
1. Distinction between runtime classes and static types is quite sane and a simple idea. A runtime class is something associated with an actual object, while static type is something associated with an AST node.
Here, I'm more concerned about *types* at runtime vs *types* for static checking. Some of these may be normal classes, but those are not the problematic ones.
Mixing them would be misleading, since they "live in parallel planes". Although it is true that there is a type that corresponds to every runtime class.
It's not clear to me what you mean by mixing them. They are already partly mixed, right? I think you are speaking from the static-checker point of view, where there are only types, and runtime behavior is completely separate (at least in some sense). This view works especially well when types are in comments or stubs. But when the types are also present at runtime, I don't think this view is completely realistic. Some of the objects that represent types are regular classes, and some of them may be only types (like Union[str, bytes] or Sequence[int]), but not normal Python classes that you instantiate. Even if they represent types, not classes, they exist at runtime, and there should at least *exist* a well-defined answer to whether an object is in 'instance' of a given type. (Not sure if 'instance' should be world used here) Ignoring that *types* are also a runtime concept seems dangerous to me.
2. Currently isinstance(obj, List[int]) fails with TypeError, ditto for issubclass and for user defined generic classes:
class C(Generic[T]): ...
isinstance(obj, C) # works, returns only True or False isinstance(obj, C[int]) # TypeError issubclass(cls, C) # works issubclass(cls, C[int]) # raisesTypeError
I suppose that's the best that isinstance can do in these cases. But I'm not sure if isinstance and issubclass should try to do as much as reasonable, or if it they should just handle the normal classes, and let some new function, say implements(), take care of the other *types*. Anyway, this whole concept of two 'parallel universes' is problematic, because the universes overlap in at least two different ways. -- Koos 3. User defined protocols will by default raise TypeError with isinstance(),
but the user can opt-in (using @runtime decorator) for the same behavior as normal generics, this is how typing.Iterable currently works:
class MyIter: def __iter__(self): return [42]
isinstance(MyIter(), Iterable) # True isinstance(MyIter(), Iterable[int]) # TypeError
class A(Protocol[T]): x: T isinstance(obj, A) # TypeError
@runtime class B(Protocol[T]): y: T isinstance(obj, B) # True or False depending on whether 'obj' has attribute 'y' isinstance(obj, B[int]) # Still TypeError
-- Ivan
-- + Koos Zevenhoven + http://twitter.com/k7hoven +
@ Koos Zevenhoven
and there should at least *exist* a well-defined answer to whether an object is in 'instance' of a given type. (Not sure if 'instance' should be world used here)
Let me illustrate why being an "instance" (or any other word) does not apply well to runtime objects. Consider a list [1, 2, 3], then is it an "instance" of List[int]? Probably yes. Is it an "instance" of List[Union[str, int]]? Probably also yes. However, List[int] and List[Union[str, int]] are mutually incompatible i.e. the latter is not a subtype of the former and the former is not a subtype of the latter. (This is due to lists being mutable and therefore invariant in its type variable.) The next important point is that static type checkers never decide (or at least I have never seen this) whether a given literal (since there are no objects before runtime) is an "instance" of a type. Static type checkers (roughly speaking) verify that the semantics represented by an AST is consistent with declared/inferred types. Concerning the above example with [1, 2, 3], static type checkers can infer List[int] for such literal, or refuse to do so and require an explicit annotation, or a user can overrule the inference by an explicit annotation. This decision (whether to use List[int] or any other acceptable type for this literal) will influence type checking outcome (i.e. are there errors or not) even _earlier_ in the program, this is something that is not possible at runtime.
Ignoring that *types* are also a runtime concept seems dangerous to me.
It is not ignored. Annotations are structured type metadata accessible both at static and runtime, there is even typing_inspect module on PyPI designed to provide some runtime introspection of types (it is at an early stage of development) and some elements of it might end up in typing. Also checking subtyping between two types (without mixing them with classes) at runtime is entirely possible, but this is very complicated task with lots of corner cases, therefore I don't think it will be in stdlib. stdlib was always kept simple and easy to maintain. -- Ivan
On Sun, Jun 25, 2017 at 09:13:44AM -0700, Lucas Wiman wrote:
For some background on the removal of __instancecheck__, check the linked issues here:
Thanks for the reference (the most relevant discussion starts here <https://github.com/python/typing/issues/136#issuecomment-104698674>). That said, I think I totally disagree with the underlying philosophy of throwing away a useful and intuitive feature (having `is_instance(foo, Union[Bar, Baz])` just work as you'd naively expect) in the name of making sure that people *understand* there's a distinction between types and classes.
Yes... I agree. I think Mark Shannon has an exaggerated preference for "purity over practicality" when he writes: Determining whether a class is a subclass of a type is meaningless as far I'm concerned. https://github.com/python/typing/issues/136#issuecomment-217386769 That implies that runtime types ("classes") and static types are completely unrelated. I don't think that's true, and I think that would make static types pointless if it were true. I'd put it this way... runtime types are instantations of static types (not *instances*). https://en.wiktionary.org/wiki/instantiation If we didn't already use the terms for something else, I'd say that static types are *abstract types* and runtime types ("classes") are *concrete types*. But that clashes with the existing use of abstract versus concrete types. I can see that there are actual problems to be solved, and *perhaps* Mark's conclusion is the right one (even if for the wrong reasons). For example: isinstance([], List[int]) isinstance([], List[str]) How can a single value be an instance of two mutually incompatible types? (But see below, for an objection.) On the other hand, just because a corner case is problematic, doesn't mean that the vast majority of cases aren't meaningful. It just seems perverse to me to say that it is "meaningless" (in Mark's words) to ask whether isinstance(['a', 'b'], List[int]) isinstance(123, List[str]) for example). If static type checking has any meaning at all, then the answers to those two surely have to be False.
This seems opposed to the "zen" of python that there should be exactly one obvious way to do it, since (1) there isn't a way to do it without a third party library, and (2) the obvious way to do it is with `isinstance` and `issubclass`. Indeed, the current implementation makes it somewhat nonobvious even how to implement this functionality yourself in a third-party library (see this gist <https://gist.github.com/lucaswiman/21373bea33ccd2c5e868ec52b6eff412>).
I think that the current status is that the MyPy folks, including Guido, consider that it *is* reasonable to ask these questions for the purpose of introspection, but issubclass and isinstance are not the way to do it.
One of the first things I did when playing around with the `typing` module was to fire up the REPL, and try runtime typechecks:
from typing import * isinstance(0, Union[int, float]) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/lucaswiman/.pyenv/versions/3.6/lib/python3.6/typing.py", line 767, in __instancecheck__ raise TypeError("Unions cannot be used with isinstance().") TypeError: Unions cannot be used with isinstance().
I think the natural reaction of naive users of the library is "That's annoying. Why? What is this library good for?", not "Ah, I've sagely learned a valuable lesson about the subtle-and-important-though-unmentioned distinction between types and classes!"
Indeed! Why shouldn't isinstance(x, Union[A, B]) isinstance(x, (A, B)) be treated as equivalent? [...]
Mark Shannon's example also specifically does not apply to the types I'm thinking of for the reasons I mentioned:
For example, List[int] and List[str] and mutually incompatible types, yet isinstance([], List[int]) and isinstance([], List[str)) both return true.
There is no corresponding objection for `Union`; I can't think of any* inconsistencies or runtime type changes that would result from defining `_Union.__instancecheck__` as `any(isinstance(obj, t) for t in self.__args__`.
Or just isinstance(x, tuple(self.__args__)) as above.
For `Tuple`, it's true that `()` would be an instance of `Tuple[X, ...]` for all types X. However, the objection for the `List` case (IIUC; extrapolating slightly) is that the type of the object could change depending on what's added to it. That's not true for tuples since they're immutable, so it's not *inconsistent* to say that `()` is an instance of `Tuple[int, ...]` and `Tuple[str, ...]`, it's just applying a sensible definition to the base case of an empty tuple.
I'm not even completely convinced that the List example really is a problem. Well, it may be a problem for applying the theory of types, which in turn may make actually programming a type-checker more difficult. But to the human reader, why is is a problem that an empty list can be considered both a list of strings and a list of ints? That's just the vacuous truth! An empty bag can be equally well described as a bag containing no apples or a bag containing no oranges. They're both true, and if the theory of types cannot cope with that fact, that's a weakness in the theory, not the fact. (That's analogous to the Circle-Ellipse problem for the the theory behind object oriented code.) https://en.wikipedia.org/wiki/Circle-ellipse_problem -- Steve
On Sun, Jul 2, 2017 at 4:54 AM, Steven D'Aprano <steve@pearwood.info> wrote:
I think that the current status is that the MyPy folks, including Guido, consider that it *is* reasonable to ask these questions for the purpose of introspection, but issubclass and isinstance are not the way to do it.
That may have once been the viewpoint of the developers of typing/MyPy, though it seems to have changed. The current view is that this should be implemented in a third party library. Further discussion is here <https://github.com/ilevkivskyi/typing_inspect/issues/2>. - Lucas
Steven D'Aprano wrote:
isinstance([], List[int]) isinstance([], List[str])
How can a single value be an instance of two mutually incompatible types?
I don't think there's any contradiction there, because the compatibility rules are different for static and runtime types. Statically, when you assign something of type A to a variable of type B, you're assering that *all* values of type A are compatible with B. But at runtime, you're only asserting that one *particular* value is compatible with B.
It just seems perverse to me to say that it is "meaningless" (in Mark's words) to ask whether
isinstance(['a', 'b'], List[int]) isinstance(123, List[str])
for example). If static type checking has any meaning at all, then the answers to those two surely have to be False.
I doubt whether Mark meant "separate" to imply "unrelated". Static and runtime types are clearly related, although the relationship is not one-to-one and involves complicated overlaps. To my mind, the question isn't whether tests like that are meaningful -- clearly they are. The question is whether we should attempt to support answering them at run time, given that doing so in the general case requires unbounded amounts of computation. -- Greg
participants (6)
-
Greg Ewing -
Ivan Levkivskyi -
Koos Zevenhoven -
Lucas Wiman -
rymg19＠gmail.com -
Steven D'Aprano