<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Guido and Andrew,<br>
<br>
Thank you for your prompt responses!<br>
<br>
On 5.5.2015 21:48, Guido van Rossum wrote:<br>
<blockquote
cite="mid:CAP7+vJ+ZsdCmDf+Hns2FXzo1stEX-pfeUZpxFBoin4GbzpPZZw@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>Quick notes:<br>
</div>
- I don't think it's really possible to write
realistic async code independently from an async
framework.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
And since there is asyncio in the standard library, I would assume
there typically is no reason to do that either(?) However, as a side
effect of my proposal, there would still be a way to use an if
statement to pick the right async code to match the framework, along
with matching the non-async version :).<br>
<br>
Speaking of side effects, I think the same "__async__" variable
might also naturally provide this:<br>
<br>
<a class="moz-txt-link-freetext" href="https://mail.python.org/pipermail/python-ideas/2015-April/033152.html">https://mail.python.org/pipermail/python-ideas/2015-April/033152.html</a><br>
<br>
By the way, if I understand your first note, it might be the same as
my "The Y and L ends need to be compatible with each other for the
code to work." Sorry about the terminology. I hope the explanations
of Y and L are somewhat understandable.<br>
<br>
<blockquote
cite="mid:CAP7+vJ+ZsdCmDf+Hns2FXzo1stEX-pfeUZpxFBoin4GbzpPZZw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>- For synchronous code that wants to use some async
code, the pattern is simple:<br>
</div>
asyncio.get_event_loop().run_until_complete(some_async_call(args,
etc))<br>
</div>
- We can probably wrap this in a convenience helper function
so you can just write:<br>
</div>
asyncio.sync_wait(some_async_call(args, etc))<br>
</div>
</div>
</blockquote>
<br>
This is what is keeping me from using asyncio. Ignoring performance
overhead, if in any synchronous script (or interactive prompt or
ipython notebook) all calls to my library would look like that, I
will happily use my 2.7 version that uses threads. Well, I admit
that the part about "happily" is not completely true in my case. <br>
<br>
Instead, I would be quite happy typing "await
<function_call>", since awaiting the function call (to
finish/return a value) is exactly what I would be doing, regardless
of whether there is an event loop or not.<br>
<br>
<blockquote
cite="mid:CAP7+vJ+ZsdCmDf+Hns2FXzo1stEX-pfeUZpxFBoin4GbzpPZZw@mail.gmail.com"
type="cite">
<div dir="ltr">- Note that this will fail (and rightly so!) if
called when the event loop is already running.<br>
</div>
<div class="gmail_extra"><br>
</div>
</blockquote>
<br>
Regarding my proposal, there would still be a way for libraries to
provide this functionality, if desired :).<br>
<br>
Please see also the comments below.<br>
<br>
<blockquote
cite="mid:CAP7+vJ+ZsdCmDf+Hns2FXzo1stEX-pfeUZpxFBoin4GbzpPZZw@mail.gmail.com"
type="cite">
<div class="gmail_extra">
<div class="gmail_quote">On Tue, May 5, 2015 at 11:00 AM, Andrew
Barnert via Python-ideas <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:python-ideas@python.org" target="_blank">python-ideas@python.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
style="color:#000;background-color:#fff;font-family:HelveticaNeue,Helvetica
Neue,Helvetica,Arial,Lucida
Grande,sans-serif;font-size:16px">
<div dir="ltr"><span>It seems like it might be a lot
easier to approach this from the other end: Is it
possible to write a decorator that takes an async
coroutine function, strips out all the awaits, and
returns a regular sync function? If so, all you need
to do is write everything as async, and then users
can "from spam import sync as spam" or "from spam
import async as spam" (where async just imports all
the real functions, while sync imports them and
calls the decorator on all of them).</span></div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
Interesting idea. If this is possible, it would solve part of the
issue, but the "Y end" (sorry) of the chain may still need to be
done by hand.<br>
<br>
<blockquote
cite="mid:CAP7+vJ+ZsdCmDf+Hns2FXzo1stEX-pfeUZpxFBoin4GbzpPZZw@mail.gmail.com"
type="cite">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
style="color:#000;background-color:#fff;font-family:HelveticaNeue,Helvetica
Neue,Helvetica,Arial,Lucida
Grande,sans-serif;font-size:16px">
<div dir="ltr"><span><br>
</span></div>
<div dir="ltr"><span>That also avoids the need to have
all the looking up the event loop, switching between
different code branches, etc. inside every function
at runtime. (Not that it matters for the performance
of sleep(1), but it might matter for the performance
of other functions—and, more importantly, it might
make the implementation of those functions simpler
and easier to debug through.)</span></div>
<div>
<div class="h5"> <br>
<div><br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
This could indeed save some if statements at runtime. <br>
<br>
Note that the if statements would not be inside every function, but
only in the ones that do the actual IO. For instance, some 3rd-party
library might use wrappers around socket send and socket recv to
choose between sync and async versions, and that might be all the IO
it needs to build several layers of async code. Even better, had
someone taken the time to provide these if statements inside the
standard library, the whole 3rd-party async library would just
magically work also in synchronous code :).<br>
<br>
Best regards,<br>
Koos<br>
<br>
<blockquote
cite="mid:CAP7+vJ+ZsdCmDf+Hns2FXzo1stEX-pfeUZpxFBoin4GbzpPZZw@mail.gmail.com"
type="cite">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
style="color:#000;background-color:#fff;font-family:HelveticaNeue,Helvetica
Neue,Helvetica,Arial,Lucida
Grande,sans-serif;font-size:16px">
<div>
<div class="h5">
<div><br>
</div>
<div style="display:block">
<div style="font-family:HelveticaNeue,Helvetica
Neue,Helvetica,Arial,Lucida
Grande,sans-serif;font-size:16px">
<div style="font-family:HelveticaNeue,Helvetica
Neue,Helvetica,Arial,Lucida
Grande,sans-serif;font-size:16px">
<div dir="ltr"> <font face="Arial" size="2">
On Tuesday, May 5, 2015 7:01 AM, Koos
Zevenhoven <<a moz-do-not-send="true"
href="mailto:koos.zevenhoven@aalto.fi"
target="_blank">koos.zevenhoven@aalto.fi</a>>
wrote:<br>
</font> </div>
<blockquote style="border-left:2px solid
rgb(16,16,255);margin-left:5px;margin-top:5px;padding-left:5px">
<br>
<br>
<div>Hi all!<br>
<br>
I am excited about seeing what's going on
with asyncio and PEP492 etc. I <br>
really like that Python is becoming more
suitable for the increasing <br>
amount of async code and that the
distinction between async functions <br>
and generators is increasing.<br>
<br>
In addition, however, I would also like to
see the async functions and <br>
methods come even closer to regular
functions and methods. This is <br>
something that is keeping me from using
asyncio at the moment even if I <br>
would like to. Below I'll try to explain
what and why, and a little bit <br>
of how. If it is not clear, please ask :)<br>
<br>
Motivation:<br>
<br>
One of the best things about asyncio and
coroutines/async functions is <br>
that you can write asynchronous code as if
it were synchronous, the <br>
difference in many places being just the
use of "await" ("yield from") <br>
when calling something that may end up
doing IO (somewhere down the <br>
function call chain) and that the code is
run from an event loop.<br>
<br>
When writing a package that does IO, you
have the option to make it <br>
either synchronous or asynchronous.
Regardless of the choice, the code <br>
will look roughly the same. But what if
you want to be able to do both? <br>
Should you maintain two versions, one with
"async" and "await" <br>
everywhere and one without?<br>
<br>
Besides the keywords "async" and "await",
async code of course differs <br>
from synchronous code by the
functions/coroutines that are used for IO
<br>
at the end of the function call chain.
Here, I mean the end (close to) <br>
where the "yield" expressions are hidden
in the async versions. At the <br>
other end of the calling chain, async code
needs the event loop and <br>
associated framework (almost always
asyncio?) which hides all the async <br>
scheduling fanciness etc. I'm not sure
about the terminology, but I will <br>
use "L end" and "Y end" to refer to the
two ends here. (L for event <br>
Loop; Y for Yield)<br>
<br>
The Y and L ends need to be compatible
with each other for the code to <br>
work. While asyncio and the standard
library might provide both ends in <br>
many cases, there can also be situations
where a package would want to <br>
work with different combinations of L and
Y end, or completely without <br>
an event loop, i.e. synchronously.<br>
<br>
In a very simple example, one might want
to wrap different <br>
implementations of sleep() in a function
that would pick the right one <br>
depending on the context. Perhaps
something like this:<br>
<br>
async def any_sleep(seconds):<br>
if __async__.framework is None:<br>
time.sleep(1)<br>
elif __async__.framework is asyncio:<br>
await asyncio.sleep(1)<br>
else:<br>
raise RuntimeError("Was called
with an unsupported async <br>
framework.")<br>
<br>
[You could of course replace sleep() with
socket IO or whatever, but <br>
sleep is nice and simple. Also, a larger
library would probably have a <br>
whole chain of async functions and methods
before calling something like <br>
this]<br>
<br>
But if await is only allowed inside "async
def", then how can <br>
any_sleep() be conveniently run in
non-async code? Also, there is <br>
nothing like __async__.framework. Below, I
describe what I think a <br>
potential solution might look like.<br>
<br>
<br>
<br>
Potential solution:<br>
<br>
This is simplified version; for instance,
as "awaitables", I consider <br>
only async function objects here. I
describe the idea in three parts:<br>
<br>
(1) next(...):<br>
<br>
Add a keyword argument "async_framework"
(or whatever) to next(...) with <br>
a default value of None. When an async
framework, typically asyncio, <br>
starts an async function object
(coroutine) with a call to next(...), it <br>
would do something like next(coro,
async_framework = asyncio). Here, <br>
asyncio could of course be replaced with
any object that identifies the <br>
framework. This information would then be
somehow attached to the async <br>
function object.<br>
<br>
<br>
(2) __async__.framework or something
similar:<br>
<br>
Add something like __async__ that has an
attribute such as .framework <br>
that allows the code inside the async
function to access the information <br>
passed to next(...) by the framework (L
end) using the keyword argument <br>
of next [see (1)].<br>
<br>
(3) Generalized "await":<br>
<br>
[When the world is ready:] Allow using
"await" anywhere, not just within <br>
async functions. Inside async functions,
the behavior of "await" would <br>
be the same as in PEP492, with the
addition that it would somehow <br>
propagate the __async__.framework value to
the awaited coroutine. <br>
Outside async functions, "await" would do
roughly the same as this function:<br>
<br>
def await(async_func_obj):<br>
try:<br>
next(async_func_obj) # same as
next(async_func_obj, <br>
async_framework = None)<br>
except StopIteration as si:<br>
return si.value<br>
raise RuntimeError("The function
does not support synchronous <br>
execution")<br>
<br>
(This function would, of course, work in
Python 3.4, but it would be <br>
mostly useless because the async functions
would not know that they are <br>
being called in a 'synchronous program'.
IIUC, this *function* would be <br>
valid even with PEP492, but having this as
a function would be ugly in <br>
the long run.)<br>
<br>
<br>
Some random thoughts:<br>
<br>
With this addition to Python, one could
write libraries that work both <br>
async and non-async. When await is not
inside async def, one would <br>
expect it to potentially do blocking IO,
just like an await inside async <br>
def would suggest that there is a
yield/suspend somewhere in there.<br>
<br>
For testing, I tried to see if there is a
reasonable way to make a hack <br>
with __async__.framework that could be set
by next(), but did not find <br>
an obvious way. For instance,
coro.gi_frame.f_locals is read-only, I <br>
believe.<br>
<br>
An alternative to this approach could be
that await would implicitly <br>
start a temporary event loop for running
the coroutine, but how would it <br>
know which event loop? This might also
have a huge performance overhead.<br>
<br>
Relation to PEP492:<br>
<br>
This of course still needs more thinking,
but I wanted to post it here <br>
now in case there is desire to prepare for
something like this already <br>
in PEP492. It is not completely clear
if/how this would need to affect <br>
PEP492, but some things come to mind. For
example, this could <br>
potentially remove the need for
__aenter__, __aiter__, etc. or even <br>
"async for" and "async with". If
__aenter__ is defined as "async def", <br>
then a with statement would do an "await"
on it, and the context manager <br>
would have __async__.framework (or
whatever it would be called) <br>
available, for determining what behavior
is appropriate.<br>
<br>
Was this clear enough to understand which
problem(s) this would be <br>
solving and how? I'd be happy to hear
about any thoughts on this :).<br>
<br>
<br>
Best regards,<br>
Koos<br>
<br>
_______________________________________________<br>
Python-ideas mailing list<br>
<a moz-do-not-send="true"
href="mailto:Python-ideas@python.org"
target="_blank">Python-ideas@python.org</a><br>
<a moz-do-not-send="true"
href="https://mail.python.org/mailman/listinfo/python-ideas"
target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br>
Code of Conduct: <a
moz-do-not-send="true"
href="http://python.org/psf/codeofconduct/"
target="_blank">http://python.org/psf/codeofconduct/</a><br>
<br>
<br>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Python-ideas mailing list<br>
<a moz-do-not-send="true"
href="mailto:Python-ideas@python.org">Python-ideas@python.org</a><br>
<a moz-do-not-send="true"
href="https://mail.python.org/mailman/listinfo/python-ideas"
target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br>
Code of Conduct: <a moz-do-not-send="true"
href="http://python.org/psf/codeofconduct/"
target="_blank">http://python.org/psf/codeofconduct/</a><br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div class="gmail_signature">--Guido van Rossum (<a
moz-do-not-send="true" href="http://python.org/%7Eguido"
target="_blank">python.org/~guido</a>)</div>
</div>
</blockquote>
<br>
</body>
</html>