(I prefer to start a new thread, the previous one is too long for me :-)) Hi, I'm still trying to understand how the PEP 3152 would impact asyncio. Guido suggests to replace "yield from fut" with "cocall fut()" (add parenthesis) and so add a __cocall__() method to asyncio.Future. Problem: PEP 3152 says "A cofunction (...) does not contain any yield or yield from expressions". Currently, Future.__iter__() is implemented using yield: def __iter__(self): if not self.done(): self._blocking = True yield self # This tells Task to wait for completion. assert self.done(), "yield from wasn't used with future" return self.result() # May raise too.
From my understanding, the PEP 3151 simply does not support asyncio.Future. Am I right?
How is it possible to suspend a cofunction if it's not possible to use yield? If waiting for a future in a cofunction is not supported, the PEP 3151 is useless for asyncio, since asyncio completly relies on futures. Victor
I can live with `cocall fut()` but the difference between `data = yield from loop.sock_recv(sock, 1024)` and `data = cocall (loop.sock_recv(sock, 1024))()` frustrates me very much. — Sent from Mailbox On Thu, Apr 23, 2015 at 4:09 PM, Victor Stinner <victor.stinner@gmail.com> wrote:
On 2015-04-23 11:26 AM, Victor Stinner wrote:
I can do that. class Future: .... def __iter__(self): .... __cocall__ = __iter__ I've outlined the problem with this approach in parallel email: https://mail.python.org/pipermail/python-dev/2015-April/139456.html Yury
Victor Stinner wrote:
The implementation of a __cocall__ method is *not* a cofunction, it's an *ordinary* function that returns an iterator. In the case of Future, what it needs to do is identical to Future.__iter__. So the code can be just def __cocall__(self): return iter(self) or equivalent. -- Greg
On Apr 23, 2015, at 8:22 AM, andrew.svetlov@gmail.com wrote:
I can live with `cocall fut()` but the difference between `data = yield from loop.sock_recv(sock, 1024)` and `data = cocall (loop.sock_recv(sock, 1024))()` frustrates me very much.
This is unacceptable. None of the existing async/await implementations in other languages force you to write stuff like this. -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev
Greg, On 2015-04-24 4:13 AM, Greg Ewing wrote:
I'm sorry, but this is ridiculous. You haven't addressed the issues. We raise issues, saying that your PEP isn't backwards compatible and is breaking existing idioms. You say - there is a workaround for that; or that we can rewrite that and make it like that; or something else that doesn't make any sense for existing asyncio developers. Your PEP isn't backwards compatible. Period. It *will* be harder for people to get, as it *does* introduce a new calling grammar that isn't obvious for at least some people. We, asyncio developers, who write asyncio code, *don't* want to write 'cocall fut()'. I don't understand *why* I'm required to put parentheses there (besides someone just requiring me to do so, because they failed to solve some problem in backwards compatible way). You avoid confusion in one place, but you introduce it in other places. I'm sorry, but your current way of handling the discussion isn't really productive. You don't listen to arguments by Victor Stinner, Andrew Svetlov, and me. At this point, this whole PEP 3152 related discussion isn't helping anyone. Yury P.S. I'm sorry if this sounded harsh, this wasn't my intent.
Hi Victor, On 2015-04-23 4:43 AM, Victor Stinner wrote: [...]
From my understanding, the PEP 3151 simply does not support asyncio.Future. Am I right?
Greg wants to implement __cocall__ on futures. This way you'll be able to write cocall fut() # instead of "await fut" So you *will have to* use "()" Another problem is functions that return future: def do_something(): ... return fut With Greg's idea to call it you would do: cocall (do_something())() That means that you can't refactor your "do_something" function and make it a coroutine. Yury
Hi, 2015-04-23 17:54 GMT+02:00 Yury Selivanov <yselivanov.ml@gmail.com>:
Oh, I missed something in the PEP 3152: a obj__cocall__() method can be an iterator/generator, it can be something different than a cofunction. So a __cocall__() *can* use yield and yield from. But to use cocall, it must be a cofunction. It's not easy to understand the whole puzzle. IMO the PEP 492 better explains how pieces are put together ;-) Victor
Victor Stinner wrote:
In fact, it *can't* be cofunction. It's part of the machinery for implementing cofunctions.
It's not easy to understand the whole puzzle. IMO the PEP 492 better explains how pieces are put together ;-)
Yes, it's written in a rather minimal style, sorry about that. -- Greg
Yury Selivanov wrote:
There's no fundamental problem with a cofunction returning another cofunction: codef do_something(): return fut f = cocall do_something() result = cocall f() Combining those last two lines into one would require some extra parenthesisation, but I don't think that's something you're going to be doing much in practice. If you're just going to immediately call the result, there's no point in returning a future -- just do it all in do_something(). -- Greg
Yury Selivanov wrote:
Hmmm. So you have an ordinary function that returns a future, and you want to turn it into a coroutine function, but still have it return a future in order to keep the API the same, is that right? Turning it into a coroutine means you're going to have to change every site that calls it, so its API has already changed. Given that, I'm not sure what advantage there is in keeping the future- returning part of the API. However, if we use the await()-cofunction idea, then a call to the initial version looks like cocall await(f(x)) and after the refactoring it becomes cocall await(cocall f(x)) That doesn't look so bad to me. -- Greg
On 24 April 2015 at 09:34, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I've not been following this discussion (and coroutines make my head hurt) but this idiom looks like it's bound to result in people getting the idea that you scatter "cocall" throughout an expression until you get it to work. Paul
Paul Moore wrote:
They won't need to do that, because they'll get told exactly where they've left one out, or put one in that they shouldn't have. Also, the places you need to put cocall are exactly the same as the places you need yield-from currently, or await under PEP 492. -- Greg
On Fri, Apr 24, 2015 at 11:34 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
No. In asyncio there is no difference between coroutine and regular function returning future.
From caller site next both are equal:
@asyncio.coroutine def f(): return 1 def g(): fut = asyncio.Future() fut.set_result(1) return fut Both may be called via `yield from`: ret1 = yield from f() ret2 = yield from g()
-- Thanks, Andrew Svetlov
Victor Stinner wrote:
A __cocall__ method doesn't have to be implemented with a cofunction. Any method that returns an iterator will do, including a generator. So a Future.__cocall__ that just invokes Future.__iter__ should work fine.
How is it possible to suspend a cofunction if it's not possible to use yield?
The currently published version of PEP 3152 is not really complete. A few things would need to be added to it, one of them being a suspend() builtin that has the same effect as yield in a generator-based coroutine. -- Greg
I can live with `cocall fut()` but the difference between `data = yield from loop.sock_recv(sock, 1024)` and `data = cocall (loop.sock_recv(sock, 1024))()` frustrates me very much. — Sent from Mailbox On Thu, Apr 23, 2015 at 4:09 PM, Victor Stinner <victor.stinner@gmail.com> wrote:
On 2015-04-23 11:26 AM, Victor Stinner wrote:
I can do that. class Future: .... def __iter__(self): .... __cocall__ = __iter__ I've outlined the problem with this approach in parallel email: https://mail.python.org/pipermail/python-dev/2015-April/139456.html Yury
Victor Stinner wrote:
The implementation of a __cocall__ method is *not* a cofunction, it's an *ordinary* function that returns an iterator. In the case of Future, what it needs to do is identical to Future.__iter__. So the code can be just def __cocall__(self): return iter(self) or equivalent. -- Greg
On Apr 23, 2015, at 8:22 AM, andrew.svetlov@gmail.com wrote:
I can live with `cocall fut()` but the difference between `data = yield from loop.sock_recv(sock, 1024)` and `data = cocall (loop.sock_recv(sock, 1024))()` frustrates me very much.
This is unacceptable. None of the existing async/await implementations in other languages force you to write stuff like this. -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev
Greg, On 2015-04-24 4:13 AM, Greg Ewing wrote:
I'm sorry, but this is ridiculous. You haven't addressed the issues. We raise issues, saying that your PEP isn't backwards compatible and is breaking existing idioms. You say - there is a workaround for that; or that we can rewrite that and make it like that; or something else that doesn't make any sense for existing asyncio developers. Your PEP isn't backwards compatible. Period. It *will* be harder for people to get, as it *does* introduce a new calling grammar that isn't obvious for at least some people. We, asyncio developers, who write asyncio code, *don't* want to write 'cocall fut()'. I don't understand *why* I'm required to put parentheses there (besides someone just requiring me to do so, because they failed to solve some problem in backwards compatible way). You avoid confusion in one place, but you introduce it in other places. I'm sorry, but your current way of handling the discussion isn't really productive. You don't listen to arguments by Victor Stinner, Andrew Svetlov, and me. At this point, this whole PEP 3152 related discussion isn't helping anyone. Yury P.S. I'm sorry if this sounded harsh, this wasn't my intent.
Hi Victor, On 2015-04-23 4:43 AM, Victor Stinner wrote: [...]
From my understanding, the PEP 3151 simply does not support asyncio.Future. Am I right?
Greg wants to implement __cocall__ on futures. This way you'll be able to write cocall fut() # instead of "await fut" So you *will have to* use "()" Another problem is functions that return future: def do_something(): ... return fut With Greg's idea to call it you would do: cocall (do_something())() That means that you can't refactor your "do_something" function and make it a coroutine. Yury
Hi, 2015-04-23 17:54 GMT+02:00 Yury Selivanov <yselivanov.ml@gmail.com>:
Oh, I missed something in the PEP 3152: a obj__cocall__() method can be an iterator/generator, it can be something different than a cofunction. So a __cocall__() *can* use yield and yield from. But to use cocall, it must be a cofunction. It's not easy to understand the whole puzzle. IMO the PEP 492 better explains how pieces are put together ;-) Victor
Victor Stinner wrote:
In fact, it *can't* be cofunction. It's part of the machinery for implementing cofunctions.
It's not easy to understand the whole puzzle. IMO the PEP 492 better explains how pieces are put together ;-)
Yes, it's written in a rather minimal style, sorry about that. -- Greg
Yury Selivanov wrote:
There's no fundamental problem with a cofunction returning another cofunction: codef do_something(): return fut f = cocall do_something() result = cocall f() Combining those last two lines into one would require some extra parenthesisation, but I don't think that's something you're going to be doing much in practice. If you're just going to immediately call the result, there's no point in returning a future -- just do it all in do_something(). -- Greg
Yury Selivanov wrote:
Hmmm. So you have an ordinary function that returns a future, and you want to turn it into a coroutine function, but still have it return a future in order to keep the API the same, is that right? Turning it into a coroutine means you're going to have to change every site that calls it, so its API has already changed. Given that, I'm not sure what advantage there is in keeping the future- returning part of the API. However, if we use the await()-cofunction idea, then a call to the initial version looks like cocall await(f(x)) and after the refactoring it becomes cocall await(cocall f(x)) That doesn't look so bad to me. -- Greg
On 24 April 2015 at 09:34, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I've not been following this discussion (and coroutines make my head hurt) but this idiom looks like it's bound to result in people getting the idea that you scatter "cocall" throughout an expression until you get it to work. Paul
Paul Moore wrote:
They won't need to do that, because they'll get told exactly where they've left one out, or put one in that they shouldn't have. Also, the places you need to put cocall are exactly the same as the places you need yield-from currently, or await under PEP 492. -- Greg
On Fri, Apr 24, 2015 at 11:34 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
No. In asyncio there is no difference between coroutine and regular function returning future.
From caller site next both are equal:
@asyncio.coroutine def f(): return 1 def g(): fut = asyncio.Future() fut.set_result(1) return fut Both may be called via `yield from`: ret1 = yield from f() ret2 = yield from g()
-- Thanks, Andrew Svetlov
Victor Stinner wrote:
A __cocall__ method doesn't have to be implemented with a cofunction. Any method that returns an iterator will do, including a generator. So a Future.__cocall__ that just invokes Future.__iter__ should work fine.
How is it possible to suspend a cofunction if it's not possible to use yield?
The currently published version of PEP 3152 is not really complete. A few things would need to be added to it, one of them being a suspend() builtin that has the same effect as yield in a generator-based coroutine. -- Greg
participants (10)
-
Andrew Svetlov -
andrew.svetlov＠gmail.com
-
Antoine Pitrou -
Greg Ewing -
Guido van Rossum -
Paul Moore -
Ryan Gonzalez -
Victor Stinner -
Yury Selivanov -
Łukasz Langa