<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Jan 20, 2016 at 9:42 AM, Andrew Barnert via Python-ideas <span dir="ltr"><<a href="mailto:python-ideas@python.org" target="_blank">python-ideas@python.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>On Jan 20, 2016, at 06:27, Agustín Herranz Cecilia <<a href="mailto:agustin.herranz@gmail.com" target="_blank">agustin.herranz@gmail.com</a>> wrote:<br>
><br>
> - GVR proposal includes some kind of syntactic sugar for function type comments (" # type: (t_arg1, t_arg2) -> t_ret "). I think it's good but this must be an alternative over typing module syntax (PEP484), not the preferred way (for people get used to typehints). Is this syntactic sugar compatible with generators? The type analyzers could be differentiate between a Callable and a Generator?<br>
<br>
</span>I'm pretty sure Generator is not the type of a generator function, bit of a generator object. So to type a generator function, you just write `(int, int) -> Generator[int]`. Or, the long way, `Function[[int, int], Generator[int]]`.<br></blockquote><div><br></div><div>There is no 'Function' -- it existed in mypy before PEP 484 but was replaced by 'Callable'. And you don't annotate a function def with '-> Callable' (unless it returns another function). The Callable type is only needed in the signature of higher-order functions, i.e. functions that take functions for arguments or return a function. For example, a simple map function would be written like this:<br><br></div><div><span style="font-family:monospace,monospace">def map(f: Callable[[T], S], a: List[T]) -> List[S]:<br> ...</span><br></div><div><br></div><div>As to generators, we just improved how mypy treats generators (<a href="https://github.com/JukkaL/mypy/commit/d8f72279344f032e993a3518c667bba813ae041a">https://github.com/JukkaL/mypy/commit/d8f72279344f032e993a3518c667bba813ae041a</a>). The Generator type has *three* parameters: the "yield" type (what's yielded), the "send" type (what you send() into the generator, and what's returned by yield), and the "return" type (what a return statement in the generator returns, i.e. the value for the StopIteration exception). You can also use Iterator if your generator doesn't expect its send() or throw() messages to be called and it isn't returning a value for the benefit of `yield from'.<br><br>For example, here's a simple generator that iterates over a list of strings, skipping alternating values:<br></div><div><br></div><div><span style="font-family:monospace,monospace">def skipper(a: List[str]) -> Iterator[str]:<br></span></div><div><span style="font-family:monospace,monospace"> for i, s in enumerate(a):<br></span></div><div><span style="font-family:monospace,monospace"> if i%2 == 0:<br></span></div><div><span style="font-family:monospace,monospace"> yield s</span><br></div><div> <br></div><div>and here's a coroutine returning a string (I know, it's pathetic, but it's an example :-):<br><br></div><div><span style="font-family:monospace,monospace">@asyncio.coroutine<br></span></div><div><span style="font-family:monospace,monospace">def readchar() -> Generator[Any, None, str]:<br></span></div><div><span style="font-family:monospace,monospace"> # Implementation not shown<br></span></div><div><span style="font-family:monospace,monospace">@asyncio.coroutine<br></span></div><div><span style="font-family:monospace,monospace">def readline() -> Generator[Any, None, str]:<br></span></div><div><span style="font-family:monospace,monospace"> buf = ''<br></span></div><div><span style="font-family:monospace,monospace"> while True:<br></span></div><div><span style="font-family:monospace,monospace"> c = yield from readchar()<br></span></div><div><span style="font-family:monospace,monospace"> if not c: break<br></span></div><div><span style="font-family:monospace,monospace"> buf += c<br></span></div><div><span style="font-family:monospace,monospace"> if c == '\n': break<br></span></div><div><span style="font-family:monospace,monospace"> return buf</span><br><br></div><div>Here, in Generator[Any, None, str], the first parameter ('Any') refers to the type yielded -- it actually yields Futures, but we don't care about that (it's an asyncio implementation detail). The second parameter ('None') is the type returned by yield -- again, it's an implementation detail and we might just as well say 'Any' here. The third parameter (here 'str') is the type actually returned by the 'return' statement.<br><br>It's illustrative to observe that the signature of readchar() is exactly the same (since it also returns a string). OTOH the return type of e.g. asyncio.sleep() is Generator[Any, None, None], because it doesn't return a value.<br><br></div><div>This business is clearly still suboptimal -- we would like to introduce a new type, perhaps named Coroutine, so that you can write Coroutine[T] instead of Generator[Any, None, T]. But that would just be a shorthand. The actual type of a generator object is always some parametrization of Generator.<br><br></div><div>In any case, whatever we write after the -> (i.e., the return type) is still the type of the value you get when you call the function. If the function is a generator function, the value you get is a generator object, and that's what the return type designates.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
(Of course you can use Callable instead of the more specific Function, or Iterator (or even Iterable) instead of the more specific Generator, if you want to be free to change the implementation to use an iterator class or something later, but normally you'd want the most specific type, I think.)<span><br></span></blockquote><div><br></div><div>I don't know where you read about Callable vs. Function.<br><br></div><div>Regarding using Iterator[T] instead of Generator[..., ..., T], you are correct.<br><br></div><div>Note that you *cannot* define a generator function as returning a *subclass* of Iterator/Generator; there is no way to have a generator function instantiate some other class as its return value. Consider (ignoring generic types):<br><br></div><div><span style="font-family:monospace,monospace">class MyIterator:<br></span></div><div><span style="font-family:monospace,monospace"> def __next__(self): ...<br></span></div><div><span style="font-family:monospace,monospace"> def __iter__(self): ...<br></span></div><div><span style="font-family:monospace,monospace"> def bar(self): ...<br><br></span></div><div><span style="font-family:monospace,monospace">def foo() -> MyIterator:<br></span></div><div><span style="font-family:monospace,monospace"> yield<br><br></span></div><div><span style="font-family:monospace,monospace">x = foo()<br></span></div><div><span style="font-family:monospace,monospace">x.bar() # Boom!</span><br><br></div><div>The type checker would assume that x has a method bar() based on the declared return type for foo(), but it doesn't. (There are a few other special cases, in addition to Generator and Iterator; declaring the return type to be Any or object is allowed.)<br></div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>
> - As this is intended to gradual type python2 code to port it to python 3 I think it's convenient to add some sort of import that only be used for type checking, and be only imported by the type analyzer, not the runtime. This could be achieve by prepending "#type: " to the normal import statement, something like:<br>
> # type: import module<br>
> # type: from package import module<br>
<br>
</span>That sounds like a bad idea. If the typing module shadows some global, you won't get any errors, but your code will be misleading to a reader (and even worse if you from package.module import t). If the cost of the import is too high for Python 2, surely it's also too high for Python 3. And what other reason do you have for skipping it?<span><br></span></blockquote><div><br></div><div>Exactly. Even though (when using Python 2) all type annotations are in comments, you still must write real imports. (This causes minor annoyances with linters that warn about unused imports, but there are ways to teach them.)<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>
> - Also there must be addressed how it work on a python2 to python3 environment as there are types with the same name, str for example, that works differently on each python version. If the code is for only one version uses the type names of that version.<br>
<br>
</span>That's the same problem that exists at runtime, and people (and tools) already know how to deal with it: use bytes when you mean bytes, unicode when you mean unicode, and str when you mean whatever is "native" to the version you're running under and are willing to deal with it. So now you just have to do the same thing in type hints that you're already doing in constructors, isinstance checks, etc.<br></blockquote><div><br></div><div>This is actually still a real problem. But it has no bearing on the choice of syntax for annotations in Python 2 or straddling code.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Of course many people use libraries like six to help them deal with this, which means that those libraries have to be type-hinted appropriately for both versions (maybe using different stubs for py2 and py3, with the right one selected at pip install time?), but if that's taken care of, user code should just work.<br></blockquote><div><br></div><div>Yeah, we could use help. There are some very rudimentary stubs for a few things defined by six (<a href="https://github.com/python/typeshed/tree/master/third_party/3/six">https://github.com/python/typeshed/tree/master/third_party/3/six</a>, <a href="https://github.com/python/typeshed/tree/master/third_party/2.7/six">https://github.com/python/typeshed/tree/master/third_party/2.7/six</a>) but we need more. There's a PR but it's of bewildering size (<a href="https://github.com/python/typeshed/pull/21">https://github.com/python/typeshed/pull/21</a>).<br><br></div><div>PS. I have a hard time following the rest of Agustin's comments. The comment-based syntax I proposed for Python 2.7 does support exactly the same functionality as the official PEP 484 syntax; the only thing it doesn't allow is selectively leaving out types for some arguments -- you must use 'Any' to fill those positions. It's not a problem in practice, and it doesn't reduce functionality (omitted argument types are assumed to be Any in PEP 484 too). I should also remark that mypy supports the comment-based syntax in Python 2 mode as well as in Python 3 mode; but when writing Python 3 only code, the non-comment version is strongly preferred. (We plan to eventually produce a tool that converts the comments to standard PEP 484 syntax).<br></div><div> </div></div>-- <br><div>--Guido van Rossum (<a href="http://python.org/~guido" target="_blank">python.org/~guido</a>)</div>
</div></div>