[Python-ideas] Suggestion for standardized annotations

[Cem Karan]

On Mar 8, 2014, at 6:38 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Sun, Mar 9, 2014 at 10:24 AM, Cem Karan <cfkaran2 at gmail.com> wrote:
>> There are a number of reasons I'm suggesting UUIDs instead of simple strings:
> 
> But against that is that they're extremely long. Most other places
> where long hex strings are used, humans won't use the full thing. With
> git and hg, revisions are identified by hash - but you can use a
> prefix, and in most repos, 6-8 hex digits will uniquely identify
> something. When humans look at SSH key fingerprints, how many actually
> read the whole thing? That's why randart was produced. Tell me, which
> of these is different?
> 
> def f(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
>    pass
> 
> def asdf(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
>    pass
> 
> def hello_world(x:"f0e4c2f76c58916ec258f246851bea891d14d4247a2fc3e18694461b1816e13b"):
>    pass
> 
> def testing(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
>    pass
> 
> def longer_function_name(x:"f0e4c2f76c58916ec258f246851bea091d14d4247a2fc3e18694461b1816e13b"):
>    pass
> 
> As uniqueness guarantors, they're a bit unwieldy, and that makes them
> pretty unhelpful.

Agreed... for human beings.  However, the purpose isn't just for people, its for automated systems that use annotations in possibly incompatible ways.  My original example was the problem of docstrings between projects like PLY (http://www.dabeaz.com/ply/) and Sphinx (http://sphinx-doc.org/).  For this pair of projects, you must choose to either document your code, or use the parser; not both.  My proposal circumvents the problem.  Nick Coghlan suggested using decorators instead, which I implemented.  In my test code (posted below), you can stack decorators, which completely hide the UUID from human eyes.  If an automated tool requires the annotations, it can look for its own UUID.

BTW, my apologies if the test code isn't 100% perfect; I threw it together to make sure that the decorator idea would work well.  Also, I'm using actual UUID instances instead of strings; it was easier to see what I was doing that way, but would need to be changed in real code.

Thanks,
Cem Karan

#!/usr/bin/env python
# -*- coding: utf-8 -*-

__docformat__ = "restructuredtext en"

import inspect
import uuid
import pprint

##############################################################################
##############################################################################
### Helper classes
##############################################################################
##############################################################################


class _parameter_decorator(object):
    """
    This class adds a documentation string to the annotations of a function or
    class instance's parameter.  You can add multiple documentation strings,
    or you can add one at a time.  Note that it is intended to only be used by
    the annotizer class.
    """
    def __init__(self, ID, *args, **kwargs):
        self._ID = ID
        self._args = args
        self._kwargs = kwargs
    # End of __init__()

    def __call__(self, f):
        sig = inspect.signature(f)

        func_args = {x for x in sig.parameters}
        decorator_args = {x for x in self._kwargs}
        keys = func_args.intersection(decorator_args)

        for key in keys:
            if key not in f.__annotations__:
                f.__annotations__[key] = dict()
            if self._ID not in f.__annotations__[key]:
                f.__annotations__[key][self._ID] = dict()
            f.__annotations__[key][self._ID]["doc"] = self._kwargs[key]

        return f
    # End of __call__()
# End of class _parameter_decorator

class _return_decorator(object):
    """
    This class adds a documentation string to the annotations of a function or
    class instance's return value.
    """
    def __init__(self, ID, *args, **kwargs):
        self._ID = ID
        self._args = args
        self._kwargs = kwargs
    # End of __init__()

    def __call__(self, f):
        sig = inspect.signature(f)

        key = 'return'
        if sig.return_annotation == inspect.Signature.empty:
            f.__annotations__[key] = dict()
        if self._ID not in f.__annotations__[key]:
            f.__annotations__[key][self._ID] = dict()
        f.__annotations__[key][self._ID]["doc"] = self._args[0]

        return f
    # End of __call__()
# End of class _return_decorator

##############################################################################
##############################################################################
### annotizer
##############################################################################
##############################################################################


class annotizer(object):
    """
    This assists in making annotations by providing decorators that are aware
    of your project's internal `UUID <http://en.wikipedia.org/wiki/UUID>`_.
    Using this class will ensure that your annotations are separate from the
    annotations that others use, even if the keys they use are the same as
    your keys.  Thus, projects **A** and **B** can both use the key ''doc''
    while annotating the same function, without accidentally overwriting the
    other project's use.
    """
    def __init__(self, ID,
                 parameter_decorator_class=_parameter_decorator,
                 return_decorator_class=_return_decorator):
        """
        :param ID: This is your project's UUID.  The easiest way to generate
            this is to use the
            `uuid <http://docs.python.org/3/library/uuid.html`_ module, and
            then store the ID somewhere convenient.
        :type ID: ``str`` that can be used to initialize a
            `UUID <http://docs.python.org/3/library/uuid.html#uuid.UUID`_
            instance, or a
            `UUID <http://docs.python.org/3/library/uuid.html#uuid.UUID`_
            instance
        """
        if isinstance(ID, uuid.UUID):
            self._ID = ID
        else:
            self._ID = uuid.UUID(ID)

        self.parameter_decorator_class = parameter_decorator_class
        self.return_decorator_class = return_decorator_class
    # End of __init__()

    def ID():
        doc = ("This is the ID of your project.  It is a " +
               "`UUID <http://docs.python.org/3/library/uuid.html#uuid.UUID`_" +
               "instance.")
        def fget(self):
            return self._ID
        return locals()
    ID = property(**ID())

    def parameter_decorator_class():
        doc = ("Instances of this class may be used to decorate")
        def fget(self):
            return self._parameter_decorator_class
        def fset(self, value):
            self._parameter_decorator_class = value
        def fdel(self):
            self._parameter_decorator_class = _parameter_decorator
        return locals()
    parameter_decorator_class = property(**parameter_decorator_class())

    def return_decorator_class():
        doc = "The return_decorator_class property."
        def fget(self):
            return self._return_decorator_class
        def fset(self, value):
            self._return_decorator_class = value
        def fdel(self):
            self._return_decorator_class = _return_decorator
        return locals()
    return_decorator_class = property(**return_decorator_class())

    def parameter_decorator(self, *args, **kwargs):
        decorator = self.parameter_decorator_class(self.ID, *args, **kwargs)
        return decorator
    # End of parameter_decorator()

    def return_decorator(self, *args, **kwargs):
        decorator = self.return_decorator_class(self.ID, *args, **kwargs)
        return decorator
    # End of return_decorator()
# End of class annotizer

##############################################################################
##############################################################################
### Main
##############################################################################
##############################################################################

if __name__ == "__main__":
    ID1 = uuid.uuid1()
    an1 = annotizer(ID1)
    ID2 = uuid.uuid1()
    an2 = annotizer(ID2)

    @an1.parameter_decorator(a="a", b="b", c="c")
    @an2.parameter_decorator(a="A", b="B", c="C")
    @an1.return_decorator("Doesn't return anything of value")
    @an2.return_decorator("Does not return a value")
    def func(a,b,c):
        print("a = {0!s}, b = {1!s}, c = {2!s}".format(a,b,c))

    pprint.pprint(func.__annotations__)

The output is:

$ ./annotizer.py 
{'a': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': 'a'},
       UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'A'}},
 'b': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': 'b'},
       UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'B'}},
 'c': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': 'c'},
       UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'C'}},
 'return': {UUID('f2e91c2c-a721-11e3-9535-d49a20c52ef2'): {'doc': "Doesn't return anything of value"},
            UUID('f2ec5608-a721-11e3-b494-d49a20c52ef2'): {'doc': 'Does not return a value'}}}