I'd like to offer my perspective(and objections) on this: I author and maintain a reflective argument parser for python, clize[1], possibly in the same vein as Ethan's. While it originally did not support any Python 3 features, it has come to accept keyword-only parameters and annotations for alternate parameter names and conversion functions(which may look like type annotations to a reader looking for them). The upcoming version will completely deprecate the old "pass a bunch of unsightly arguments to a decorators[2]" API, and use annotations exclusively[3], using backports[4][5] to make them and keyword-only parameters available to Python 2 users. Clize has a handful of users, with 100 stars on GitHub, and ~75 downloads per day according to PyPI[1]. Needless to say, the idea of deprecating uses of parameter and return annotations that aren't this particular way of typechecking is upsetting to me. On to the most formal of formal complaints: PEP 3107[6] rejected standardizing typing-related annotations. What has changed since then, or what has otherwise invalidated that conclusion, that it should be backtracked on and changed? GvR says he knows of little practical use of function annotations in mainstream code. Here's the (unfortunate) reason: Mainstream code is either incompatible with Python 3 or maintaining compatibility between both Python 2 and Python 3. Standard library code which had no such concern for the most part stuck with what stood in PEP 3107, which was not to teach annotations as typing info. Short of resorting to tools like sigtools.modifiers.annotate[4], *mainstream code can't use function annotations*. You will argue that people are already coming up with type-checking libraries en masse, and that little alternative uses have come up. That's because type-checking is an idea that people have seen and had in mind well before now, PEP 3107, or Python itself. It is even one of the two examples in the PEP, the other being fairly irrelevant given docstrings, IMO. Mainstream code can't use annotations. New brains are mostly taught to use python 2[7] or keep compatibility with it, so no new ideas there(save for the endless stream of "I've used Python for a day and it needs static typing" coming from statically-typed languages.) IMO there simply hasn't been enough time for new uses to show up. There's the argument parsers me and a couple others have mentioned, and although the idea can be ported to other interfaces(web forms, GUI forms?), this is only one use yet, but I think it is a good one, one that is also fairly unique to Python. Do we want to close this window of opportunity by trying to imitate the 80's? Even if you don't deprecate other uses, putting this in the standard library sends a message to the mob: "Annotations are meant to express types and other uses should conform." It's already slightly against-current to propose reflective tools because "reflection is weird and unreliable," I'd hate to see where this would take us. Worse than shutting down a potential area of innovation, promoting type-checking in such a way would alienate what many experienced Python programmers view as a core tenet of Python: duck-typing. Ironically, GvR's example illustrates it perfectly: from typing import List, Dict def word_count(input: List[str]) -> Dict[str, int]: result = {} #type: Dict[str, int] for line in input: for word in line.split(): result[word] = result.get(word, 0) + 1 return result First, there's the obvious mistake others have noted: this thing does not need a list, it needs an iterable, and that iterable merely needs to yield objects that have a split method, which yields... hashable objects. That's it. This function could work just as well if line.split yielded integers. It's what's awesome about things like collections.Counter. You can give it whatever iterable you like, whether it yields characters, strings, integers, the types of objects tracked by the garbage collector, you name it. This is awesome. Declaring and casting between type templates is confusing, verbose, scary and not awesome in general. It just makes me think of C++ (sorry). Let's say that this function needs to operate on strings, because at some point in the future it will replace line.split() with something that accounts for punctuation and whatnot, and ignore that this would really mean it actually became a different function. (Do you expect str.split to begin handling punctuation overnight?) Now the only mistake is that it pretends it is specific to lists. This mistake is easily realized, so what happens when you write your fancy library that fully declares its typing, declaring generic interfaces where appropriate, and one of your dependencies finally makes the switch over and gets it wrong? Now your typechecker says your functions are making incorrect calls to that API because you passed the result of a generator function to something that expects a list. How often do you think this will happen? How many people eager to type-check everything will not know how duck-typing matters in their library or program and specify overly restrictive interfaces for their functions? How many will assume checking declared types is a substitute for unit testing? This is speculative, but being a regular participant of #python on FreeNode, I already dread this. Finally, if I abstract this proposal, it boils down to: The proposal wants to take what has been so far a fully free-for-all attribute of functions, and make it free-for-all-but-only-for-this-purpose. That's a bit... weird? Maybe the mypy project could split its declaration and checking components, sort of like I split clize(CLI argument parsing) and sigtools(high-level signature operations and reflection improvements)? Isn't the way those libraries declare types-to-be-checked the main way they will distinguish each other for developers? Is there a problem preventing IDEs from implementing a runner for mypy, much like eg. the way you can use pyflakes as your compiler in vim? Sorry for the long rant. [1] Clize on PyPI: https://pypi.python.org/pypi/clize [2] Old clize API: https://github.com/epsy/clize/blob/e84637a631574e793719114ad7e40d0b36df1a78/... [3] New clize API involving annotations: http://clize.readthedocs.org/en/latest/basics.html#converting-arguments [4] Inspect.signature-compatible backport of annotations: http://sigtools.readthedocs.org/en/latest/#sigtools.modifiers.annotate [5] Backport of inspect.signature: https://pypi.python.org/pypi/funcsigs/0.4 [6] Rejected proposals from Function Annotations PEP: http://legacy.python.org/dev/peps/pep-3107/#rejected-proposals [7] Warnings for Beginners - Learn Python the Hard Way: http://learnpythonthehardway.org/book/ex0.html#warnings-for-beginners -Yann Kaiser On 14 August 2014 13:36, Andrey Vlasovskikh <andrey.vlasovskikh@gmail.com> wrote:

On Thu, Aug 14, 2014 at 2:28 PM, Manuel Cerón <ceronman@gmail.com> wrote:

...

One interesting feature of TypeScript is that it allows you to annotate existing code without modifying it, by using external definition files. In the JavaScript world, many people have contributed TypeScript annotation files for popular JS libraries (http://definitelytyped.org/).

I think this is possible in Python as well doing something like this:

@annotate('math.ciel') def ciel(x: float) -> int: pass

I think this should be the way to go for annotating the stdlib. It has the advantage that if the type syntax changes, it's possible to provide new type annotations without changing the libraries at all, and even supporting older versions. In this way the code and type annotations can evolve separately.

Does mypy support something like this?

We use something quite similar to TypeScript's repository of annotations in PyCharm. Here is our external annotations proposal and stubs for some stdlib modules (https://github.com/JetBrains/python-skeletons) that uses a custom type syntax in docstrings due to lack of a better standard, see README. We state in our proposal that we would like the standard to emerge. The idea being discussed here about using Mypy's type system is not new to us. As I've mentioned in the original thread, we have discussed it with Jukka Lehtosalo, the author of Mypy. Some initial ideas are listed here (https://github.com/pytypes/pytypes).

-- Andrey Vlasovskikh Web: http://pirx.ru/ _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Aug 14, 2014 at 12:28:26PM +0200, Manuel Cerón wrote:

One interesting feature of TypeScript is that it allows you to annotate existing code without modifying it, by using external definition files. In the JavaScript world, many people have contributed TypeScript annotation files for popular JS libraries (http://definitelytyped.org/).

I think this is possible in Python as well doing something like this:

@annotate('math.ciel') def ciel(x: float) -> int: pass

I'm afraid I don't understand what the annotate decorator is doing here. Can you explain please? -- Steven

On Thu, Aug 14, 2014 at 8:55 PM, Steven D'Aprano <steve@pearwood.info> wrote:

On Thu, Aug 14, 2014 at 12:28:26PM +0200, Manuel Cerón wrote:

...

One interesting feature of TypeScript is that it allows you to annotate existing code without modifying it, by using external definition files. In the JavaScript world, many people have contributed TypeScript annotation files for popular JS libraries (http://definitelytyped.org/).

I think this is possible in Python as well doing something like this:

@annotate('math.ciel') def ciel(x: float) -> int: pass

I'm afraid I don't understand what the annotate decorator is doing here. Can you explain please?

The idea is to add type annotations to modules without modifying them. For example, in this case, the stdlib math module is defined and implemented in C, but you still want to have annotations for it so that if you write math.ciel('foo'), the static type analyzer gives you a error or warning. By defining a new module, for example math_annotations.py with empty functions with annotated signatures, you can let the static analyzer know what are the annotations for another module. In this example, the annotate decorator is just a way of telling the static analyzer that these annotations apply to the math.ceil function, not math_annotations.ceil. This is what TypeScript does to annotate popular libraries written in plain JavaScript with zero type information. Manuel.

-- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Aug 14, 2014 at 1:44 PM, Ethan Furman <ethan@stoneleaf.us> wrote:

On 08/14/2014 01:29 PM, Manuel Cerón wrote:

...

On Thu, Aug 14, 2014 at 8:55 PM, Steven D'Aprano wrote:

...

On Thu, Aug 14, 2014 at 12:28:26PM +0200, Manuel Cerón wrote:

...

@annotate('math.ciel') def ciel(x: float) -> int: pass

I'm afraid I don't understand what the annotate decorator is doing here. Can you explain please?

The idea is to add type annotations to modules without modifying them. For example, in this case, the stdlib math module is defined and implemented in C, but you still want to have annotations for it so that if you write math.ciel('foo'), the static type analyzer gives you a error or warning. By defining a new module, for example math_annotations.py with empty functions with annotated signatures, you can let the static analyzer know what are the annotations for another module. In this example, the annotate decorator is just a way of telling the static analyzer that these annotations apply to the math.ceil function, not math_annotations.ceil.

To make sure I understand: The above snippet is located in a file named 'math_annotations.py', and the annotate decorator says "the following function annotation should be stored against the 'ceil' function in the 'math' module, not the 'ceil' function in this current module" ?

mypy has existing infrastructure in the form of stub modules to support this use case: https://github.com/JukkaL/mypy/blob/master/stubs/3.2/math.py I'm not sure if stubs can easily be pulled in from outside of mypy, but that would be straightforward to support if it doesn't work already. -bob