[Python-3000] Draft pre-PEP: function annotations
Collin Winter
collinw at gmail.com
Thu Aug 10 02:32:19 CEST 2006
After letting the discussions from the Spring stew in my head for a
few months, here's my first draft of the proto-PEP for function
annotations. This is intended to lay out in a single document the
basic ideas for function annotations, to get community feedback on the
fundamentals before proceeding to the nitty-gritty. As such, the
implementation section isn't filled out; that's still in progress.
Also, the list of references is incomplete. Both of these will be
completed before the initial submission to the PEP editors.
Without further ado...
PEP: 3XXX
Title: Function Annotations
Version: $Revision: 43251 $
Last-Modified: $Date: 2006-03-23 09:28:55 -0500 (Thu, 23 Mar 2006) $
Author: Collin Winter <collinw at gmail period com>
Discussions-To: python-3000 at python.org
Status: Draft
Type: Standards Track
Requires: 3XXX (Brett Cannon's __signature__ PEP)
Content-Type: text/x-rst
Created: 03-Aug-2006
Python-Version: 3.0
Post-History:
Abstract
========
This PEP introduces a syntax for adding annotations to Python
functions [#func-term#]_. In addition to annotations for function
parameters, the syntax includes support for annotating a function's
return value(s).
In section one, I outline the "philosophy" and fundamentals needed
to understand function annotations before launching into an
in-depth discussion.
In section two, the syntax for function annotations is presented,
including a full explanation of the changes needed in Python's
grammar.
In section three, I discuss how user code will be able to access
the annotation information.
Section four describes a possible implementation of function
annotations for Python 3.0.
In section five, a C-language API for use by extension modules is
discussed.
Lastly, section six lists a number of ideas that were considered for
inclusion but were ultimately rejected.
Rationale
=========
Because Python's 2.x series lacks a standard way of annotating a
function's parameters and return values (e.g., with information about
a what type a function's return value should be), a variety
of tools and libraries have appeared to fill this gap [#tail-examp#]_.
Some utilise the decorators introduced in "PEP 318", while others
parse a function's doctext strings, looking for annotations
there.
This PEP aims to provide a single, standard way of specifying this
information, reducing the confusion caused by the wide variation in
mechanism and syntax that has existed until this point.
Fundamentals of Function Annotations
====================================
Before launching into a discussion of the precise ins and outs of
Python 3.0's function annotations, let's first talk broadly about
what annotations are and are not:
1. Function annotations, both for parameters and return values, are
completely optional.
2. Function annotations are nothing more than a way of associating
arbitrary Python expressions with various parts of a function at
compile-time.
Re-read that. Once more.
By itself, Python does not attach any particular meaning or
significance to annotations. Left to its own, Python simply
takes these expressions and uses them as the values in some
theoretical parameter-name-to-annotation-expression mapping.
The only way that annotations take on meaning is when they
are interpreted by third-party libraries. These third-party,
annotation-interpreting libraries (TAILs, for short) can do
anything they want with a function's annotations. For
example, one library might use string-based annotations to provide
improved help messages, like so:
::
def compile(source: "something compilable",
filename: "where the compilable thing comes from",
mode: "is this a single statement or a suite?"):
...
Another library might be used to provide typechecking for Python
functions and methods. This library could use annotations to
indicate the function's expected input and return types, possibly
something like
::
def sum(*vargs: Number) -> Number:
...
where ``Number`` is some description of the protocol for numeric
types.
However, neither the strings in the first example nor the
type information in the second example have any meaning on their
own; meaning comes from third-party libraries alone.
3. Following from point 2, this PEP makes no attempt to introduce
any kind of standard semantics, even for the built-in types.
This work will be left to third-party libraries.
There is no worry that these libraries will assign semantics at
random, or that a variety of libraries will appear, each with varying
semantics and interpretations of what, say, a tuple of strings
means. The difficulty inherent in writing annotation interpreting
libraries will keep their number low and their authorship in the
hands of people who, frankly, know what they're doing.
Syntax
======
Parameters
----------
Annotations for parameters take the form of optional expressions
that follow the parameter name. This example indicates that
parameters 'a' and 'c' should both be a ``Number``, while parameter
'b' should both be a ``Mapping``:
::
def foo(a: Number, b: Mapping, c: Number = 5):
...
In pseudo-grammar, parameters now look like
``identifier [: expression] [= expression]``. That is, type
annotations always precede a parameter's default value and both type
annotations and default values are optional. Just like how equal
signs are used to indicate a default value, colons are used to mark
annotations. All annotation expressions are evaluated at the time
the function is compiled.
Annotations for excess parameters (i.e., *vargs and **kwargs)
are indicated similarly. In the follow function definition,
``*vargs`` is flagged as a list of ``Number``s, and ``**kwargs`` is
marked as a dict whose keys are strings and whose values are
``Sequence``s.
::
def foo(*vargs: Number, **kwargs: Sequence):
...
Note that, depending on what annotation-interpreting library you're
using, the following might also be a valid spelling of the above:
::
def foo(*vargs: [Number], **kwargs: {str: Sequence}):
...
Only the first, however, has the BDFL's blessing [#blessed-excess#]_
as the One Obvious Way.
Return Values
-------------
The examples thus far have omitted examples of how to annotate the
type of a function's return value. This is done like so:
::
def sum(*vargs: Number) -> Number:
...
The parameter list can now be followed by a literal ``->`` and
a Python expression. Like the annotations for parameters, this
expression will be evaluated when the function is compiled.
The pseudo-grammar for function definition is now something like
::
vargs = '*' identifier [':' expression]
kwargs = '**' identifier [':' expression]
parameter = identifier [':' expression] ['=' expression]
funcdef = 'def' identifier '(' [parameter ',']*
[vargs ',']
[kwargs]
')' ['->' expression] ':' suite
For a complete discussion of the changes to Python's grammar, see
the section `Grammar Changes`_.
Accessing Function Annotations
==============================
Once compiled, a function's annotations are available via the
function's ``__signature__`` attribute, introduced by PEP 3XXX.
Signature objects include an attribute just for annotations,
appropriately called ``annotations``. This attribute is a
dictionary, mapping parameter names to an object representing
the evaluated annotation expression.
There is a special key in the ``annotations`` mapping, ``"return"``.
This key is present only if an annotation was supplied for the
function's return value.
For example, the following annotation:
::
def foo(a: Number, b: 5 + 6, c: list) -> String:
...
would result in a ``__signature__.annotations`` mapping of
::
{'a': Number,
'b': 11,
'c': list,
'return': String}
The ``return`` key was chosen because it cannot conflict with
the name of a parameter; any attempt to use ``return`` as a
parameter name would result in a ``SyntaxError``.
Implementation
==============
XXX This is all very much TODO. Beyond the obvious changes to Python's
grammar, the eventual implementation will probably involve a change to
the MAKE_FUNCTION opcode, though the details haven't been fully worked
out yet.
I'm still working on a sample implementation that works separately from
the __signature__ mechanism.
API for Annotations in C-language Extension Modules
===================================================
XXX TODO
This will probably involve macros around CPython API calls to set
and fetch the annotation expression for a given parameter.
Rejected Proposals
==================
+ The BDFL rejected the author's idea for a special syntax for adding
annotations to generators as being "too ugly" [#reject-gen-syn]_.
+ Though discussed early on ([#thread-gen#]_, [#thread-hof#]_),
including special objects in the stdlib for annotating generator
functions and higher-order functions was ultimately rejected as
being more appropriate for third-party libraries: including them
in the standard library raised too many thorny issues.
+ Despite considerable discussion about a standard type parameterisation
syntax, it was decided that this should also be left to third-party
libraries. ([#thread_imm-list#]_, [#thread-mixing#]_,
[#emphasis-tpls#]_)
Footnotes
=========
.. _[#func-term#] - Unless specifically stated, "function" is
generally used as a synonym for "callable" throughout this
document.
.. _[#tail-examp#] - The author's typecheck_ library makes use of
decorators, while `Maxime Bourget's own typechecker`_ utilises parsed
doctext strings.
References
##########
.. _[#blessed-excess#] -
http://mail.python.org/pipermail/python-3000/2006-May/002173.html
.. _[#reject-gen-syn#] -
http://mail.python.org/pipermail/python-3000/2006-May/002103.html
.. _typecheck -
http://oakwinter.com/code/typecheck/
.. _Maxime Bourget's own typechecker -
http://maxrepo.info/taxonomy/term/3,6/all
.. _[#thread-gen#] -
http://mail.python.org/pipermail/python-3000/2006-May/002091.html
.. _[#thread-hof#] -
http://mail.python.org/pipermail/python-3000/2006-May/001972.html
.. _[#thread-imm-list#] -
http://mail.python.org/pipermail/python-3000/2006-May/002105.html
.. _[#thread-mixing#] -
http://mail.python.org/pipermail/python-3000/2006-May/002209.html
.. _[#emphasis-tpls#] -
http://mail.python.org/pipermail/python-3000/2006-June/002438.html
More information about the Python-3000
mailing list