[Python-3000] Third-party annotation libraries vs the stdlib

[Tony Lownds]

On Jun 11, 2006, at 5:00 AM, Collin Winter wrote:

> In working on the annotations PEP, I've run into more issues
> concerning the balance of responsibility between third-party libraries
> and the stdlib.
>
> So far, the trend has been to push responsibility for manipulating and
> interpreting annotations into libraries, keeping core Python free from
> any built-in semantics for the annotation expressions. However, nearly
> all the issues that have been discussed on this list go against the
> flow: the proposed Function() and Generator() classes, used for
> expressing higher-order functions and generator functions,
> respectively; type operations, like "T1 & T2" or "T1 | T2"; and the
> type parameterisation mechanism.
>
> Shipping any of these things with Python raises a number of other
> issues/questions that would need to be dealt with:
>
> 1. If Function() and Generator() ship in the stdlib, where do they go?
> In types? In a new module?

The types module seems like a decent place.

> Also, if Function() and Generator() come with Python, how do we make
> sure that third-party libraries can use them with minimal extra
> overhead (e.g., wrapping layers to make the shipped Function() and
> Generator() objects compatible with the library's internal
> architecture)?
>

Thats an issue for third party libraries.

> 2. If "T1 & T2" is possible with core Python (ie, no external
> libraries), what does "type(T1 & T2)" return? Is "type(T1 & T2)" the
> same as "type(T1 | T2)"?
>

These operations could return objects that describe the types and  
nothing else.
It doesn't make sense for the result of T1 | T2 to be a type object.

class TypeUnion:
   def __init__(self, *types):
     self.types = types

   def __repr__(self):
     return '(%s)' % ' | '.join(map(repr, self.types))

   def __or__(self, other):
      ...


> What can you do with these objects in core Python? Can you subclass
> from "T1 & T2"? Does "issubclass(T1, T1 | T2)" return True? What about
> "isinstance(5, int | dict)"?

isinstance could be extended to work with TypeUnion objects. Only  
type objects are
sensible for issubclass. It makes more sense for another predicate to  
determine subtype
relationships.

And I think it makes more sense for third party packages to provide  
the specific
definitions and mechanisms for determining subtype relationships.  
Coming to a common
and usable definition would be too difficult otherwise.

I wanted to suggest that core Python's isinstance be extended to work  
with types and
subtyping definitions be left up to third party packages but that  
won't work. You can't
tell whether a given callable is a valid instance of a Function()  
without a subtype predicate.

> Are "T1 & T2" and "T1 | T2" the only defined operations? What about  
> xor or not?
>

I can't think of any useful semantics for this.

> 3. Similar questions are raised by having the "T1[x, y, z]"
> parameterisation method present in core Python: what is the type of
> "tuple[int, int]"? What can you do with it?

It could be a type object that is a subclass of tuple. It could also  
be an object that describes
the type, like TypeUnion above.

> Does "isinstance((5, 6,
> 7), tuple[int, int, int])" return True?

For new style classes, isinstance(obj, T) is roughly equivalent to  
issubclass(type(obj), T).
Lets say tuple[int, int, int] is a subclass of tuple. The result of  
type((5, 6, 7)) won't change -- it's
the tuple type object. So isinstance((5, 6, 7), tuple[int, int, int])  
would return False.

That is misleading. I think it would be better if the tuple[int, int]  
would return something that isn't
a type so that uses of isinstance are not misleading.

Another idea would be to provide a different way to check that an  
instance is a valid member of a type.
I bet this would get rejected quickly.

 >>> int.ismember(5)
True
 >>> (int | dict).ismember(5)
True

> Do they have the same & and | operations as other built-in types?

Sure, why not.

> What happens when you mix
> parameterised types and non-parameterised types, e.g., "tuple[int,
> (int, int)]"?

Is the question is whether the parameterization mechanism should enforce
that it's parameters are valid types?

> Based on the complexity involved in specifying all of these issues, I
> say we punt: let the third-party libraries handle this.
[...]
> To sum up: I propose that -- to combat these issues -- I limit the PEP
> to discussing how to supply annotations (the annotation syntax and C
> API) and how to read them back later (via __signature__).

+1

I think the annotations PEP should definitely punt on this and also  
punt on definitions
of And(), Function(), Generator(), etc. Unless those are what is  
returned by __signature__?
Annotations syntax and __signature__ object API could also be  
independent PEPs.

It would be really nice to have a common language for type annotations.
This lets authors of type-annotated code use the same annotations  
with a variety
of third-party packages. From the issues above this seems hard to  
accomplish
in a way that integrates well with the rest of Python.

-Tony