<div dir="ltr"><div>Yarko, I think your vision is too far out. Maybe something like that could become a reality in Python 5 -- it would require all extensions to become aware of the async stuff (adding it to Python doesn't automatically add it to C!). Also the GIL has nothing to do with this, async tasks all run in the same thread, and if there was no GIL it would not be any different (else two cooperating tasks could be run on different threads and you'd be back on pre-emptive scheduling and the ensuing race conditions). (Note that I refer to Python 4 as Python after the Gilectomy -- it needs to be a new major version since the C API changes dramatically as C extensions will no longer have the protection of the GIL.)<br><br></div>--Guido<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jun 9, 2017 at 2:08 AM, Yarko Tymciurak <span dir="ltr"><<a href="mailto:yarkot1@gmail.com" target="_blank">yarkot1@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><br><div class="gmail_quote"><div><div class="h5"><div>On Fri, Jun 9, 2017 at 3:57 AM Alex Grönholm <<a href="mailto:alex.gronholm@nextday.fi" target="_blank">alex.gronholm@nextday.fi</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
Yarko Tymciurak kirjoitti 09.06.2017 klo 11:49:<br>
<blockquote type="cite">
<div><br>
<div class="gmail_quote">
<div>On Fri, Jun 9, 2017 at 3:05 AM Alex Grönholm <<a href="mailto:alex.gronholm@nextday.fi" target="_blank">alex.gronholm@nextday.fi</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> Yarko Tymciurak
kirjoitti 09.06.2017 klo 09:19:<br>
<blockquote type="cite">
<div><br>
<div class="gmail_quote">
<div>On Fri, Jun 9, 2017 at 12:48 AM Nathaniel Smith
<<a href="mailto:njs@pobox.com" target="_blank">njs@pobox.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On
Thu, Jun 8, 2017 at 3:32 PM, manuel miranda <<a href="mailto:manu.mirandad@gmail.com" target="_blank">manu.mirandad@gmail.com</a>>
wrote:<br>
> Hello everyone,<br>
><br>
> After using asyncio for a while, I'm
struggling to find information about<br>
> how to support both synchronous and
asynchronous use cases for the same<br>
> library.<br>
><br>
> I.e. imagine you have a package for http
requests and you want to give the<br>
> user the choice to use a synchronous or an
asynchronous interface. Right now<br>
> the approach the community is following is
creating separate libraries one<br>
> for each version. This is far from ideal for
several reasons, some I can<br>
> think of:<br>
><br>
> - Code duplication, most of the functionality
is the same in both libraries,<br>
> only difference is the sync/async behaviors<br>
> - Some new async libraries lack functionality
compared to their sync<br>
> siblings. Others will introduce bugs that the
sync version already solved<br>
> long ago, etc.<br>
> - Different interfaces for the user for the
same exact functionality.<br>
><br>
> In summary, in some cases it looks like
reinventing the wheel. So now comes<br>
> the question, is there any documentation,
guide on what would be best<br>
> practice supporting this kind of duality?<br>
<br>
I would say that this is something that we as a
community are still<br>
figuring out. I really like the Sans-IO approach,
and it's a really<br>
valuable piece of the solution, but it doesn't
solve the whole problem<br>
by itself - you still need to actually do I/O, and
this means things<br>
like error handling and timeouts that aren't
obviously a natural fit<br>
to the Sans-IO approach, and this means you may
still have some tricky<br>
code that can end up duplicated. (Or maybe the
Sans-IO approach can be<br>
extended to handle these things too?) There are
active discussions<br>
happening in projects like urllib3 [1] and
packaging [2] about what<br>
the best strategy to take is. And the options vary
a lot depending on<br>
whether you need to support python 2 etc.<br>
<br>
If you figure out a good approach I think everyone
would be interested<br>
to hear it :-)<br>
</blockquote>
<div><br>
</div>
<div>Just to leave this breadcrumb here - I've said
this before, but not thought in depth about it a
lot, but pretty sure that in something like
Python4, async needs to become "first class
citizen," that is from the inside out, right in
the bowels of the repl loop.</div>
<div><br>
</div>
</div>
</div>
</blockquote>
</div>
<div text="#000000" bgcolor="#FFFFFF"> Python 4 will be
nothing more than the next minor release after 3.9.
Because Guido hates double digit minor versions :)</div>
<div text="#000000" bgcolor="#FFFFFF"><br>
<blockquote type="cite">
<div>
<div class="gmail_quote">
<div>If async is the default, and synchronous calls
just a special case (e.g. single-task async), then
I'd expect two things (at least): developers would
have an easier time, make fewer mistakes in async
programming (the language would handle more), and
libraries would be unified as async & sync
would be the same.</div>
</div>
</div>
</blockquote>
</div>
<div text="#000000" bgcolor="#FFFFFF"> Are you suggesting
the removal of the "await", "async with" and "async for"
structures? Those were added deliberately so developers
can spot the yield points in a coroutine function. Not
having them would give us something like gevent where you
can never tell when your task is going to be adjourned in
favor of another.</div>
</blockquote>
<div><br>
</div>
<div>actually I was bot thinking of that... but I was
thinking of processing in the language, rather than a
library... </div>
<div><br>
</div>
<div>In any case, I don't have answers, only a vision which
keeps coming up. My interest is not in providing "a
solution", rather generating a reasoned discussion...</div>
</div>
</div>
</blockquote></div><div text="#000000" bgcolor="#FFFFFF">
Then explain what you mean by making async a first class citizen in
Python. In my mind it already is, by courtesy of having the "async
def", "await" et al added to the language syntax itself and the
inclusion of the asyncio module in the standard library. The only
other thing that could've been done is to tie the language syntax to
a single event loop implementation but that was deliberately left
out.</div><div text="#000000" bgcolor="#FFFFFF"><br>
<blockquote type="cite">
<div>
<div class="gmail_quote">
<div></div></div></div></blockquote></div></blockquote></div></div><div> i'm sorry - I thought that was clear by saying it would be in the repl loop itself and not in a library.</div><div><br></div><div> and those it wouldn't require two versions of every library. That's what I meant.</div><div><br></div><div>that is right now it's coming from the outside in, that is to say from applications, closer in, to an attempt at a common library. i'm suggesting it start from the inside of the language out so that all things have that support and that it is not just a library thus any code can take advantage of either single or multiple async tasks, goal being that there only need be on version of libraries. at least that's the discussion I'm calling for. </div><div><br></div><div> does that help? </div><div><div class="h5"><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><div><div class="gmail_quote"><div><br>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><br>
<blockquote type="cite">
<div>
<div class="gmail_quote">
<div><br>
</div>
<div>Maybe there's something that would make this
not make sense, but I'd be really surprised.
Larry's gil removal work intuitively seems an
enabler for this kind of (potential) work...</div>
<div><br>
</div>
<div>-y</div>
<div><br>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
-n<br>
<br>
[1] <a href="https://github.com/shazow/urllib3/pull/1068#issuecomment-294422348" rel="noreferrer" target="_blank">https://github.com/shazow/<wbr>urllib3/pull/1068#<wbr>issuecomment-294422348</a><br>
<br>
[2] Here's the same API implemented three
different ways:<br>
Using deferreds: <a href="https://github.com/pypa/packaging/pull/87" rel="noreferrer" target="_blank">https://github.com/pypa/<wbr>packaging/pull/87</a><br>
"traditional" sans-IO: <a href="https://github.com/pypa/packaging/pull/88" rel="noreferrer" target="_blank">https://github.com/pypa/<wbr>packaging/pull/88</a><br>
Using the "effect" library: <a href="https://github.com/dstufft/packaging/pull/1" rel="noreferrer" target="_blank">https://github.com/dstufft/<wbr>packaging/pull/1</a><br>
<br>
--<br>
Nathaniel J. Smith -- <a href="https://vorpus.org" rel="noreferrer" target="_blank">https://vorpus.org</a><br>
______________________________<wbr>_________________<br>
Async-sig mailing list<br>
<a href="mailto:Async-sig@python.org" target="_blank">Async-sig@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/async-sig" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/async-sig</a><br>
Code of Conduct: <a href="https://www.python.org/psf/codeofconduct/" rel="noreferrer" target="_blank">https://www.python.org/psf/<wbr>codeofconduct/</a><br>
</blockquote>
</div>
</div>
<br>
<fieldset class="m_5535695605849400252m_-7535290103303869586m_3511873895968407553mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
Async-sig mailing list
<a class="m_5535695605849400252m_-7535290103303869586m_3511873895968407553moz-txt-link-abbreviated" href="mailto:Async-sig@python.org" target="_blank">Async-sig@python.org</a>
<a class="m_5535695605849400252m_-7535290103303869586m_3511873895968407553moz-txt-link-freetext" href="https://mail.python.org/mailman/listinfo/async-sig" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/async-sig</a>
Code of Conduct: <a class="m_5535695605849400252m_-7535290103303869586m_3511873895968407553moz-txt-link-freetext" href="https://www.python.org/psf/codeofconduct/" target="_blank">https://www.python.org/psf/<wbr>codeofconduct/</a>
</pre>
</blockquote>
<br>
</div>
______________________________<wbr>_________________<br>
Async-sig mailing list<br>
<a href="mailto:Async-sig@python.org" target="_blank">Async-sig@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/async-sig" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/async-sig</a><br>
Code of Conduct: <a href="https://www.python.org/psf/codeofconduct/" rel="noreferrer" target="_blank">https://www.python.org/psf/<wbr>codeofconduct/</a><br>
</blockquote>
</div>
</div>
</blockquote>
<br>
</div></blockquote></div></div></div></div>
<br>______________________________<wbr>_________________<br>
Async-sig mailing list<br>
<a href="mailto:Async-sig@python.org">Async-sig@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/async-sig" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/async-sig</a><br>
Code of Conduct: <a href="https://www.python.org/psf/codeofconduct/" rel="noreferrer" target="_blank">https://www.python.org/psf/<wbr>codeofconduct/</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">--Guido van Rossum (<a href="http://python.org/~guido" target="_blank">python.org/~guido</a>)</div>
</div>