[Python-ideas] Proposal to extend PEP 484 (gradual typing) to support Python 2.7

[Guido van Rossum]

On Wed, Jan 20, 2016 at 4:54 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On Wednesday, January 20, 2016 4:11 PM, Guido van Rossum <guido at python.org>
> wrote:
>
> >On Wed, Jan 20, 2016 at 9:42 AM, Andrew Barnert via Python-ideas <
> python-ideas at python.org> wrote:
> >
> >On Jan 20, 2016, at 06:27, Agustín Herranz Cecilia <
> agustin.herranz at gmail.com> wrote:
> >>>
> >>> - GVR proposal includes some kind of syntactic sugar for function type
> comments (" # type: (t_arg1, t_arg2) -> t_ret "). I think it's good but
> this must be an alternative over typing module syntax (PEP484), not the
> preferred way (for people get used to typehints). Is this syntactic sugar
> compatible with generators? The type analyzers could be differentiate
> between a Callable and a Generator?
> >>
> >>I'm pretty sure Generator is not the type of a generator function, bit
> of a generator object. So to type a generator function, you just write
> `(int, int) -> Generator[int]`. Or, the long way, `Function[[int, int],
> Generator[int]]`.
> >
>
> >There is no 'Function' -- it existed in mypy before PEP 484 but was
> replaced by 'Callable'. And you don't annotate a function def with '->
> Callable' (unless it returns another function).
>
> Sorry about getting the `Function` from the initial proposal instead of
> the current PEP.
>
> Anyway, I don't think the OP was suggesting that. If I interpreted his
> question right:
>
> He was expecting that the comment `(int, int) -> int` was a way to
> annotate a function so it comes out as type `Callable[[int, int], int]`,
> which is correct.


Not really. I understand that you're saying that after:

def foo(a, b):
    # type: (int, int) -> str
    return str(a+b)

the type of 'foo' is 'Callable[[int, int], int]'.

But it really isn't. The type checker (e.g. mypy) knows more at this point:
it knows that foo has arguments named 'a' and 'b' and that e.g. calls like
'foo(1, b=2)' are valid. There's no way to express that using Callable.
Also Callable doesn't support argument defaults.


> And he wanted to know how to instead write a comment for a generator
> function of type `GeneratorFunction[[int, int], int]`, and the answer is
> that you don't. There is no type needed for generator functions; they're
> just functions that return generators.
>

Aha. No wonder I didn't get the question. :-(


> You're right that he doesn't need to know the actual type; you're never
> going to write that, you're just going to annotate the arguments and return
> value, or use the 2.x comment style:
>
>     def f(arg1: int, arg2: int) -> Iterator[int]
>
>     def f(arg1, arg2):
>
>         # type: (int, int) -> Iterator[int]
>
> Either way, the type checker will determine that type of the function is
> `Callable[[int, int], Iterator[int]]`, and the only reason you'll ever care
> is if that type shows up in an error message.
>

I don't think you can the word 'Callable' to show up in an error message
unless it's part of the type as written somewhere. A name defined with
'def' is special and it shows up differently. (And so is a lambda.)


> >Regarding using Iterator[T] instead of Generator[..., ..., T], you are
> correct.
> >
>
> >Note that you *cannot* define a generator function as returning a
> *subclass* of Iterator/Generator;
>
> But you could define it as returning the superclass `Iterable`, right?


Yes.


> As I understand it, it's normal type variance, so any superclass will
> work; the only reason `Iterator` is special is that it happens to be
> simpler to specify than Generator and it's plausible that it isn't going to
> matter whether you've written a generator function or, say, a function that
> returns a list iterator.
>

Right.


> > there is no way to have a generator function instantiate some other
> class as its return value.
>
> If you really want that, you could always write a wrapper that forwards
> __next__, and a decorator that applies the wrapper. Can MyPy infer the type
> of the decorated function from the wrapped function and the decorator?
>

I think that's an open question. Your example below is complicated because
of the **args, *kw pattern.


>     # Can I leave this annotation off? And get one specialized to the
> actual
>     # argument types of the wrapped function? That would be cool.
>

You can't -- mypy never infers a function's type from its inner workings.

However, some Googlers are working on a tool that does infer types:
https://github.com/google/pytype

It's early days though (relatively speaking), and I don't think it handles
this case yet.


>     def my_iterating(func: Callable[Any, Iterator]) -> Callable[Any,
> MyIterator]
>

Alas, PEP 484 is not powerful enough to describe the relationship between
the input and output functions. You'd want to do something that uses a type
variable to capture all arguments together, so you could write something
like

T = TypeVar('T')
S = TypeVar('S')
def my_iterating(func: Callable[T, Iterator[S]]) -> Callable[T,
MyIterator[S]]:


>         @wraps(func)
>         def wrapper(*args, **kw):
>             return MyIterator(func(*args, **kw))
>         return wrapper
>
>     @my_iterating
>     def foo() -> Iterator[int]:
>         yield
>
>     x = foo()
>     x.bar()
>

The only reasonable way to do something like this without adding more
sophistication to PEP 484 would be to give up on the decorator and just
hardcode it using a pair of functions:

# API
def foo() -> MyIterator[int]:
    return MyIterator(_foo())

# Implementation
def _foo() -> Iterator[int]:
    yield 0

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160120/e4efbc7b/attachment-0001.html>