[Python-ideas] Proposal to extend PEP 484 (gradual typing) to support Python 2.7

[Pavol Lisy]

2016-01-09 0:04 GMT+01:00, Guido van Rossum <guido at python.org>:
> At Dropbox we're trying to be good citizens and we're working towards
> introducing gradual typing (PEP 484) into our Python code bases (several
> million lines of code). However, that code base is mostly still Python 2.7
> and we believe that we should introduce gradual typing first and start
> working on conversion to Python 3 second (since having static types in the
> code can help a big refactoring like that).
>
> Since Python 2 doesn't support function annotations we've had to look for
> alternatives. We considered stub files, a magic codec, docstrings, and
> additional `# type:` comments. In the end we decided that `# type:`
> comments are the most robust approach. We've experimented a fair amount
> with this and we have a proposal for a standard.
>
> The proposal is very simple. Consider the following function with Python 3
> annotations:
>
>     def embezzle(self, account: str, funds: int = 1000000, *fake_receipts:
> str) -> None:
>         """Embezzle funds from account using fake receipts."""
>         <code goes here>
>
> An equivalent way to write this in Python 2 is the following:
>
>     def embezzle(self, account, funds=1000000, *fake_receipts):
>         # type: (str, int, *str) -> None
>         """Embezzle funds from account using fake receipts."""
>         <code goes here>
>
> There are a few details to discuss:
>
> - Every argument must be accounted for, except 'self' (for instance
> methods) or 'cls' (for class methods). Also the return type is mandatory.
> If in Python 3 you would omit some argument or the return type, the Python
> 2 notation should use 'Any'.
>
> - If you're using names defined in the typing module, you must still import
> them! (There's a backport on PyPI.)
>
> - For `*args` and `**kwds`, put 1 or 2 starts in front of the corresponding
> type annotation. As with Python 3 annotations, the annotation here denotes
> the type of the individual argument values, not of the tuple/dict that you
> receive as the special argument value 'args' or 'kwds'.
>
> - The entire annotation must be one line. (However, see
> https://github.com/JukkaL/mypy/issues/1102.)
>
> We would like to propose this as a standard (either to be added to PEP 484
> or as a new PEP) rather than making it a "proprietary" extension to mypy
> only, so that others in a similar situation can also benefit.
>
> A brief discussion of the considered alternatives:
>
> - Stub files: this would complicate the analysis in mypy quite a bit,
> because it would have to parse both the .py file and the .pyi file and
> somehow combine the information gathered from both, and for each function
> it would have to use the types from the stub file to type-check the body of
> the function in the .py file. This would require a lot of additional
> plumbing. And if we were using Python 3 we would want to use in-line
> annotations anyway.
>
> - A magic codec was implemented over a year ago (
> https://github.com/JukkaL/mypy/tree/master/mypy/codec) but after using it
> for a bit we didn't like it much. It slows down imports, it requires a `#
> coding: mypy` declaration, it would conflict with pyxl (
> https://github.com/dropbox/pyxl), things go horribly wrong when the codec
> isn't installed and registered, other tools would be confused by the Python
> 3 syntax in Python 2 source code, and because of the way the codec was
> implemented the Python interpreter would occasionally spit out confusing
> error messages showing the codec's output (which is pretty bare-bones).
>
> - While there are existing conventions for specifying types in docstrings,
> we haven't been using any of these conventions (at least not consistently,
> nor at an appreciable scale), and they are much more verbose if all you
> want is adding argument annotations. We're working on a tool that
> automatically adds type annotations[1], and such a tool would be
> complicated by the need to integrate the generated annotations into
> existing docstrings (which, in our code base, unfortunately are wildly
> incongruous in their conventions).
>
> - Finally, the proposed comment syntax is easy to mechanically translate
> into standard Python 3 function annotations once we're ready to let go of
> Python 2.7.
>
> __________
> [1] I have a prototype of such a tool, i mplemented as a 2to3 fixer. It's a
> bit over 200 lines. It's not very interesting yet, since it sets the types
> of nearly all arguments to 'Any'. We're considering building a much more
> advanced version that tries to guess much better argument types using some
> form of whole-program analysis. I've heard that Facebook's Hack project got
> a lot of mileage out of such a tool. I don't yet know how to write it yet
> -- possibly we could use a variant of mypy's type inference engine, or
> alternatively we might be able to use something like Jedi (
> https://github.com/davidhalter/jedi).
>
> --
> --Guido van Rossum (python.org/~guido)
>

Could not something like this ->

    def embezzle(self, account, funds=1000000, *fake_receipts):
        # def embezzle(self, account: str, funds: int = 1000000,
*fake_receipts: str) -> None:
        """Embezzle funds from account using fake receipts."""
        <code goes here>

make
1. transition from python2 to python3 more simple?
2. python3 checkers more easily changeable to understand new python2 standard?
3. simpler impact to documentation (means also simpler knowledbase to
be learn) about annotations?