[Python-ideas] Was: Annotations (and static typing), Now:Sharing __annotations__

[Cem Karan]

On Aug 20, 2014, at 11:08 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> Sigh. I go away for a week and come back to a mega-thread I can never
> hope to catch up on :-)
> 
> TL; DR; - Although mypy looks interesting, I think it's too soon to
> close the door on all other uses of annotations. Let's find a solution
> that allows exploration of alternative uses for a while longer.
> 
> OK, can I just make some points regarding the static typing thread.
> First of all, I have no issue with the idea of static typing, and in
> fact I look forward to seeing what benefits it might have (if nothing
> else, the pointer to mypy, which I'd never heard of before, is
> appreciated). It won't be something I use soon (see below) but that's
> fine.
> 
> But Guido seems to be saying (on a number of occasions) that nobody is
> really using annotations, so he wants to focus on the static typing
> use case alone. I think this is a mistake. First of all, I see no
> reason why functions using typing annotations could not be introduced
> with a decorator[1]. So why insist that this is the *only* use of
> annotations, when it's pretty easy to allow others to co-exist?
> 
> Also, the "nobody is using annotations" argument? Personally, I know
> of a few other uses:
> 
> 1. Argument parsers - at least 3 have been mentioned in the thread.
> 2. Structure unpacking - I think there is a library that uses
> annotations for this, although I may be wrong.
> 3. FFI bindings. I know I've seen this discussed, although I can't
> find a reference just now.
> 
> There are probably other ideas around as well (GUI bindings,
> documentation generation, ...) None are particularly mature, and most
> are just at the "ideas" stage, but typically the ideas I have seen are
> the sort of thing you'd write a library for, and Python 3 only
> libraries *really* aren't common yet.
> 
> The problem for people wanting to experiment with annotations, is that
> they need to be writing Python 3 only code. While Python 3 adoption is
> growing rapidly, I suspect that large applications are typically still
> focused on either going through, or tidying up after, a 2-3 migration.
> And new projects, while they may be developed from the ground up using
> Python 3, will typically be using programmers skilled in Python 2, to
> whom Python 3 features are not yet an "instinctive" part of the
> toolset. Apart from large standalone applications, there are smaller
> scripts (which are typically going to be too small to need
> programming-in-the-large features like annotations) and libraries
> (which really aren't yet in a position to drop Python 2.x totally,
> unless they have a fairly small user base).
> 
> So I don't see it as compelling that usage of annotations in the wild
> is not yet extensive.
> 
> Rather than close the door on alternative uses of annotations, can I suggest:
> 
> 1. By all means bless mypy syntax as the standard static typing
> notation - this seems like a good thing.
> 2. Clarify that static typing annotations should be introduced with a
> decorator. Maybe reserve a decorator name ("@typed"?) that has a dummy
> declaration in the stdlib, and have a registration protocols for tools
> to hook their own implementation into it.[2]
> 3. Leave the door open for other uses of decorators, at least until
> some of the more major libraries drop Python 2.x support completely
> (and hence can afford to have a dependency on a Python 3 only module
> that uses annotations). See at that stage if annotations take off.
> 4. If we get to a point where even libraries that *could* use
> annotations don't, then revisit the idea of restricting usage to just
> type information.
> 
> Paul
> 
> [1] Also, a decorator could allow a Python 2 compatible form by using
> decorator arguments as an alternative to annotations.
> [2] I've yet to see a clear explanation of how "a tool using type
> annotations" like an linter, editor, IDE or Python compiler would use
> them in such a way that precludes decoration as a means of signalling
> the annotation semantics.

A long while back I proposed a mechanism for sharing __annotations__ between multiple, non-cooperating projects.  The basic idea is that each annotation becomes a dictionary.  Each project (and 'project' is a very loosely defined concept here) chooses a UUID that it uses as key into the dictionary.  The value is up to the project.

The advantage to this is manifold:

- Annotations can still have multiple uses by different groups without stepping on each other's toes.
- If someone wants to make a standard, all they have to do is publish the UUID associated with their standard.  For example, we might choose UUID('2cca6238-9fca-4053-aa3d-db9050e6b26b') as the official type information UUID.  All projects that want to develop linters, documentation generators, etc., will use that UUID for all annotations, and the PEP will require it.
- de facto standards can become de jure standards by blessing a particular UUID.
- Guessing if this method is being used is relatively easy; if its a dictionary, and if every key is a UUID, it probably follows this standard.  We can tighten it further by requiring some key-value pair be in every dictionary (i.e., {UUID('1ad60d50-8237-4b98-b2b1-69fd08ed575c'):"PEPXXXX"} is always in the dictionary).  This makes it fairly simple to add without stomping on what people are already doing. 
- Finding the standard on the web should also be easy; while you might not find the PEP instantly, you'll probably zoom into it fairly fast.

Disadvantages:

- Typing out UUIDs is PAINFUL.  I highly recommend using decorators instead.
- Reading the __annotations__ dictionary will be difficult.  pprint() should make this easier.

I have working proof-of-concept code at https://github.com/oranguman/annotizer that defines a decorator class that handles the UUID for you.  It needs to be extended to parse out information, but it handles the 'other use cases' problem fairly well.

Thanks,
Cem Karan