Conventions for Function Annotations
This isn't really a proposal for any sort of language/stdlib change, rather I felt like discussing the potential for informal standard conventions with annotations. Probably for the better, there is no special syntax for annotating raised exceptions or yields from a generator. In line with how the "->" syntax sets the 'return' key, I suggest an informal standard of representing keywords this way, for example in a decorator for raise/yield annotations: # third-party decorator @raises(ValueError) def foo():pass assert foo.__annotations__['raise'] == ValueError This might be an obvious solution, but I just wanted to "put it out there" up front, before inconsistent workarounds emerge. This convention should work because it is primarily needed for control structures that are reserved keywords anyway. The one exception I can think of is the inverse of yield: generator.send() - "send" could conflict with a function argument, and should therefore not be put in __annotations__. (A hack could be to use a different but semantically related keyword like 'import', or an otherwise invalid identifier like 'send()', but it might be best to simply not use __annotations__ for this.)
What if you want to say that it raises both ValueError and ArithmeticError? Annotations never did support multiple annotations, it's a big drawback. This is a decorator though, you can do what you want. How about making __annotations__['raise'] a list? Otherwise sounds fine to me. The use of keywords in __annotations__ is pretty clever. :) Devin On Sun, Aug 7, 2011 at 7:27 AM, dag.odenhall@gmail.com <dag.odenhall@gmail.com> wrote:
On 7 August 2011 14:59, Devin Jeanpierre <jeanpierreda@gmail.com> wrote:
Neither does return support multiple values, and yet we can do "return x, y". :) Annotations are bound to the evaluation of an expression and you're free to put anything in them. I expect a convention of special-casing tuples will emerge, especially as annotations are particularly suitable for different sorts of "type hints" where tuples are already used for issubclass/isinstance, in "except" clauses and for listing base classes with type() etc. I think annotations should be used sparingly: there's little need for any sort of interoperability between multiple unrelated uses of annotations. It would be rather complicated and unwieldy to say, both put type hints and argument documentation strings in annotations. Decorators are better suited for composability, as are docstrings for documentation. If you're doing something like mapping a view function to a template with the return annotation, it is unlikely that you need to annotate it with type hints as well, for example.
Neither does return support multiple values, and yet we can do "return x, y". :)
That's neither here nor there. That is a single return value, a tuple. Not different kinds of return values, such as returning something that might be annotated as a "sequence" as well as, in other circumstances, returning something that might be annotated as a "dict". Since few to no sanely written functions return incompatible types, there isn't as much reason to complain about this. On the other hand, almost every single function in the stdlib can raise multiple different exceptions. A "raises" annotation is not very useful unless you can annotate multiple exceptions that may be raised.
The decorator you have provided is _not_ composable. In fact, that was my complaint. I never said anything about docstrings or mixing and matching annotations. Devin On Sun, Aug 7, 2011 at 10:18 AM, dag.odenhall@gmail.com <dag.odenhall@gmail.com> wrote:
On 7 August 2011 16:26, Devin Jeanpierre <jeanpierreda@gmail.com> wrote:
That was exactly the point I was trying to demonstrate. It doesn't make sense to put any sort of special support for multiple values in annotations, because you're really just reimplementing conventional use of tuples.
Actually I (quite intentionally) didn't provide any decorator. If really desired, the fictional @raises decorator could check if 'raise' is already set and then combine them into a tuple. I'd argue that this would be inconsistent with syntactical annotations, and @raises should either override, fail, or silently do nothing, in case of the key already being set. More consistently, if you wanted to set a tuple, well then you pass a tuple to the decorator. This could perhaps work via unpacking as well: @raises(ValueError, ArithmeticError). My intent with this thread was to consider conventions for mimicing syntactical annotations in the cases not covered by syntax. If you want to do something more domain-specific using annotations, then by all means go right ahead, but consider that you might be better of using a custom function attribute as well.
On 08/07/2011 01:27 PM, dag.odenhall@gmail.com wrote:
I wrote something using the foo.__annotations__ convention a while back, but it wasn't received well on this mailinglist. Here it is again now with an added `annotations` decorator: """
from functools import wraps def annotations(**annots): def deco(obj): if hasattr(obj,'__annotations__'): obj.__annotations__.update(annots) else: obj.__annotations__ = annots return obj return deco _NONE = object() def getannot(obj, key, default=_NONE): if hasattr(obj, '__annotations__'): if default is _NONE: return obj.__annotations__[key] else: return obj.__annotations__.get(key, default) elif default is _NONE: raise KeyError(key) else: return default def setannot(obj, key, value): if hasattr(obj, '__annotations__'): obj.__annotations__[key] = value else: obj.__annotations__ = {key: value} def hasannot(obj, key): if hasattr(obj, '__annotations__'): return key in obj.__annotations__ else: return False def annotation(annotfunc): if hasattr(annotfunc, '__name__'): key = annotfunc.__name__ else: key = annotfunc.func_name @wraps(annotfunc) def params(*args,**kwargs): def deco(obj): setannot(obj, key, annotfunc(*args,**kwargs)) return obj return deco return params
On Sun, Aug 7, 2011 at 7:27 AM, dag.odenhall@gmail.com <dag.odenhall@gmail.com> wrote:
Hi Dag, Are you currently using annotations? Could you post some of the cool usages that you are making of annotations? The explicit plan with annotations (read PEP 3107) was that significant use should precede the creation of conventions for use. So please don't wait until a convention has been established -- go ahead and have fun with them, and let us know what you are doing with them! Re: declaring raised exceptions, IIUC Java is pretty much the only language supporting such a feature, and even there the current view is that they have not lived up to the expectation when the feature was designed. So I would rather do nothing in that area. -- --Guido van Rossum (python.org/~guido)
I'm toying with them for adaptation, interfaces and dependency injection from a component registry. Each use is about type constants, but not necessarily in the vein of static type checking, which I think stands to show their strengths and "Pythonicity". I do kinda think there's some need of informal conventions, examplified by Mathias post: his decorator sets the 'raises' key, effectively making "raises" a reserved keyword in function arguments using the decorator!
Probably true, though probably in part because it isn't optional in Java. In Pythonland it's probably more useful to make i part of the documentation á la the :raises: field of Sphinx. Much of the benefits of exceptions are that they can be ... unexpected, and bubble up: doing something like reraising unexpected exceptions as a TypeError would not likely be very useful (well, maybe a little useful with the new __cause__/__context__) and neither do exceptions fit in well with my other uses (adaptation, dependency injection) of annotations. Some convention for annotating 'yield' may still be useful though, although one alternative convention could use some form of "parametrized types" and the return annotation: foo() -> Iterator[tuple]. Now we just need to add this to ABCMeta. *cough* ;)
On Mon, Aug 8, 2011 at 11:03 AM, dag.odenhall@gmail.com <dag.odenhall@gmail.com> wrote:
So far, the general approach has been for annotations to be paired with decorator APIs, such that there is a cleaner less repetitive syntax that relies on function annotations and a more general (but more verbose) approach that uses arguments to a decorator factory. That approach seems to work well, with the latter API used to handle cases where a developer wants to use more than one annotation based decorator on a single function. The general principle is that any decorator that can use argument annotations should have an alternate decorator factory based API that can be used when necessary (usually either because the annotations are being used for something else or because the function being decorated is an existing one from another library that doesn't have any relevant annotations at all). Using the annotation namespace to store arbitrary metadata doesn't seem like a good idea at all. It is better to use the function attribute namespace for that kind of thing - don't forget about the old tools just because there is a shiny new tool to play with.
Why not just use the return field on the generator function as is? The return type of calling something for which 'inspect.isgeneratorfunction(x)' is true is always going to be 'generator', so the return annotation is unlikely to be referring directly to that. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 8 August 2011 05:01, Nick Coghlan <ncoghlan@gmail.com> wrote:
Or, not even an explicit decorator: it's often useful to embed instructions without runtime side-effects, such as with venusian and its use in Pyramid for @view_config and config.scan(). I use annotations like that for functions as adapter factories without a global registry. Similarly an interface definition might treat all methods as abstract with type hints without requiring a decorator like @abstractmethod.
I think Armin Ronacher said generator detection is unreliable, but in any case, it seems perhaps more Pythonic to rely only on the iterator/iterable protocol rather than specifically generators. "Iterator[tuple]" for example would match dict.items().
What if you want to say that it raises both ValueError and ArithmeticError? Annotations never did support multiple annotations, it's a big drawback. This is a decorator though, you can do what you want. How about making __annotations__['raise'] a list? Otherwise sounds fine to me. The use of keywords in __annotations__ is pretty clever. :) Devin On Sun, Aug 7, 2011 at 7:27 AM, dag.odenhall@gmail.com <dag.odenhall@gmail.com> wrote:
On 7 August 2011 14:59, Devin Jeanpierre <jeanpierreda@gmail.com> wrote:
Neither does return support multiple values, and yet we can do "return x, y". :) Annotations are bound to the evaluation of an expression and you're free to put anything in them. I expect a convention of special-casing tuples will emerge, especially as annotations are particularly suitable for different sorts of "type hints" where tuples are already used for issubclass/isinstance, in "except" clauses and for listing base classes with type() etc. I think annotations should be used sparingly: there's little need for any sort of interoperability between multiple unrelated uses of annotations. It would be rather complicated and unwieldy to say, both put type hints and argument documentation strings in annotations. Decorators are better suited for composability, as are docstrings for documentation. If you're doing something like mapping a view function to a template with the return annotation, it is unlikely that you need to annotate it with type hints as well, for example.
Neither does return support multiple values, and yet we can do "return x, y". :)
That's neither here nor there. That is a single return value, a tuple. Not different kinds of return values, such as returning something that might be annotated as a "sequence" as well as, in other circumstances, returning something that might be annotated as a "dict". Since few to no sanely written functions return incompatible types, there isn't as much reason to complain about this. On the other hand, almost every single function in the stdlib can raise multiple different exceptions. A "raises" annotation is not very useful unless you can annotate multiple exceptions that may be raised.
The decorator you have provided is _not_ composable. In fact, that was my complaint. I never said anything about docstrings or mixing and matching annotations. Devin On Sun, Aug 7, 2011 at 10:18 AM, dag.odenhall@gmail.com <dag.odenhall@gmail.com> wrote:
On 7 August 2011 16:26, Devin Jeanpierre <jeanpierreda@gmail.com> wrote:
That was exactly the point I was trying to demonstrate. It doesn't make sense to put any sort of special support for multiple values in annotations, because you're really just reimplementing conventional use of tuples.
Actually I (quite intentionally) didn't provide any decorator. If really desired, the fictional @raises decorator could check if 'raise' is already set and then combine them into a tuple. I'd argue that this would be inconsistent with syntactical annotations, and @raises should either override, fail, or silently do nothing, in case of the key already being set. More consistently, if you wanted to set a tuple, well then you pass a tuple to the decorator. This could perhaps work via unpacking as well: @raises(ValueError, ArithmeticError). My intent with this thread was to consider conventions for mimicing syntactical annotations in the cases not covered by syntax. If you want to do something more domain-specific using annotations, then by all means go right ahead, but consider that you might be better of using a custom function attribute as well.
On 08/07/2011 01:27 PM, dag.odenhall@gmail.com wrote:
I wrote something using the foo.__annotations__ convention a while back, but it wasn't received well on this mailinglist. Here it is again now with an added `annotations` decorator: """
from functools import wraps def annotations(**annots): def deco(obj): if hasattr(obj,'__annotations__'): obj.__annotations__.update(annots) else: obj.__annotations__ = annots return obj return deco _NONE = object() def getannot(obj, key, default=_NONE): if hasattr(obj, '__annotations__'): if default is _NONE: return obj.__annotations__[key] else: return obj.__annotations__.get(key, default) elif default is _NONE: raise KeyError(key) else: return default def setannot(obj, key, value): if hasattr(obj, '__annotations__'): obj.__annotations__[key] = value else: obj.__annotations__ = {key: value} def hasannot(obj, key): if hasattr(obj, '__annotations__'): return key in obj.__annotations__ else: return False def annotation(annotfunc): if hasattr(annotfunc, '__name__'): key = annotfunc.__name__ else: key = annotfunc.func_name @wraps(annotfunc) def params(*args,**kwargs): def deco(obj): setannot(obj, key, annotfunc(*args,**kwargs)) return obj return deco return params
On Sun, Aug 7, 2011 at 7:27 AM, dag.odenhall@gmail.com <dag.odenhall@gmail.com> wrote:
Hi Dag, Are you currently using annotations? Could you post some of the cool usages that you are making of annotations? The explicit plan with annotations (read PEP 3107) was that significant use should precede the creation of conventions for use. So please don't wait until a convention has been established -- go ahead and have fun with them, and let us know what you are doing with them! Re: declaring raised exceptions, IIUC Java is pretty much the only language supporting such a feature, and even there the current view is that they have not lived up to the expectation when the feature was designed. So I would rather do nothing in that area. -- --Guido van Rossum (python.org/~guido)
I'm toying with them for adaptation, interfaces and dependency injection from a component registry. Each use is about type constants, but not necessarily in the vein of static type checking, which I think stands to show their strengths and "Pythonicity". I do kinda think there's some need of informal conventions, examplified by Mathias post: his decorator sets the 'raises' key, effectively making "raises" a reserved keyword in function arguments using the decorator!
Probably true, though probably in part because it isn't optional in Java. In Pythonland it's probably more useful to make i part of the documentation á la the :raises: field of Sphinx. Much of the benefits of exceptions are that they can be ... unexpected, and bubble up: doing something like reraising unexpected exceptions as a TypeError would not likely be very useful (well, maybe a little useful with the new __cause__/__context__) and neither do exceptions fit in well with my other uses (adaptation, dependency injection) of annotations. Some convention for annotating 'yield' may still be useful though, although one alternative convention could use some form of "parametrized types" and the return annotation: foo() -> Iterator[tuple]. Now we just need to add this to ABCMeta. *cough* ;)
On Mon, Aug 8, 2011 at 11:03 AM, dag.odenhall@gmail.com <dag.odenhall@gmail.com> wrote:
So far, the general approach has been for annotations to be paired with decorator APIs, such that there is a cleaner less repetitive syntax that relies on function annotations and a more general (but more verbose) approach that uses arguments to a decorator factory. That approach seems to work well, with the latter API used to handle cases where a developer wants to use more than one annotation based decorator on a single function. The general principle is that any decorator that can use argument annotations should have an alternate decorator factory based API that can be used when necessary (usually either because the annotations are being used for something else or because the function being decorated is an existing one from another library that doesn't have any relevant annotations at all). Using the annotation namespace to store arbitrary metadata doesn't seem like a good idea at all. It is better to use the function attribute namespace for that kind of thing - don't forget about the old tools just because there is a shiny new tool to play with.
Why not just use the return field on the generator function as is? The return type of calling something for which 'inspect.isgeneratorfunction(x)' is true is always going to be 'generator', so the return annotation is unlikely to be referring directly to that. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 8 August 2011 05:01, Nick Coghlan <ncoghlan@gmail.com> wrote:
Or, not even an explicit decorator: it's often useful to embed instructions without runtime side-effects, such as with venusian and its use in Pyramid for @view_config and config.scan(). I use annotations like that for functions as adapter factories without a global registry. Similarly an interface definition might treat all methods as abstract with type hints without requiring a decorator like @abstractmethod.
I think Armin Ronacher said generator detection is unreliable, but in any case, it seems perhaps more Pythonic to rely only on the iterator/iterable protocol rather than specifically generators. "Iterator[tuple]" for example would match dict.items().
participants (5)
-
dag.odenhall＠gmail.com -
Devin Jeanpierre -
Guido van Rossum -
Mathias Panzenböck -
Nick Coghlan