[Python-ideas] async/await in Python (C API ?)
Ludovic Gasc
gmludo at gmail.com
Mon Apr 20 21:39:06 CEST 2015
+1 for C Accelerators for AsyncIO.
I don't have the level to implement that, nevertheless I can contribute
freely with two other manners: benchmarks and launch that on several
daemons to track bugs.
--
Ludovic Gasc (GMLudo)
http://www.gmludo.eu/
2015-04-20 20:46 GMT+02:00 M.-A. Lemburg <mal at egenix.com>:
> On 20.04.2015 20:16, Andrew Svetlov wrote:
> > Marc-Andre, I see your need but we should write C Accelerators for
> > asyncio first.
> > The task is not related to the PEP directly, while I like to working on
> it soon.
> > The problem is: accelerator should be backward compatibility and
> > optional (no problem with it).
> > But making C API requires much more: we need to implement the most
> > critical parts of asyncio in C and then invent C API for that.
> > Anyway, thank you for inspiration for desired C Async API.
>
> Understood, and thanks for pushing all this. Looking forward to
> some future PEP which outlines the C API for async/await :-)
>
> BTW: If you need some financial backup for these things, please
> consider writing a grant application to the PSF:
>
> https://www.python.org/psf/grants/
>
> Cheers,
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source (#1, Apr 20 2015)
> >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/
> >>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
> ________________________________________________________________________
>
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>
> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> Registered at Amtsgericht Duesseldorf: HRB 46611
> http://www.egenix.com/company/contact/
>
> > On Mon, Apr 20, 2015 at 7:15 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> >> On 20.04.2015 17:17, Yury Selivanov wrote:
> >>>
> >>> On 2015-04-20 10:49 AM, M.-A. Lemburg wrote:
> >>>> On 20.04.2015 15:40, Yury Selivanov wrote:
> >>>>> Hi Marc-Andre,
> >>>>>
> >>>>> On 2015-04-20 9:18 AM, M.-A. Lemburg wrote:
> >>>>>> Hi Yury,
> >>>>>>
> >>>>>> do you have any plans of also exposing an easy to use C API
> >>>>>> for async and await ?
> >>>>>>
> >>>>>> I think this would be useful to e.g. implement asynchronous
> >>>>>> database access. Most of those adapters are written in C and
> >>>>>> need so a C API would help a lot with this.
> >>>>> I think one way to have a convenient C API is to implement
> >>>>> a Future-like object in C -- an object with __await__ method
> >>>>> that should return an iterator, which should also be
> >>>>> implemented in C and provide a way to attach callback
> >>>>> or to *pipeline* C functions.
> >>>>>
> >>>>> This way C code should seamlessly integrate with asyncio.
> >>>>>
> >>>>> I doubt that we have time to include this kind of API in
> >>>>> the PEP, but it can be developed as a separate library.
> >>>>>
> >>>>> Your thoughts?
> >>>> I'm just sketching here, since I don't have much experience
> >>>> with writing async code (I'm using threads and multiple
> >>>> processes instead) and it probably shows ;-)
> >>>>
> >>>> From the C side of things, I think using callbacks
> >>>> and a way to register/unregister polling functions
> >>>> would be great to have.
> >>>>
> >>>> The C code could then run like this in polling mode:
> >>>>
> >>>> long_running_query(...) {
> >>>> /* Start long running operation... creating a handle for it */
> >>>> PyAsync_RegisterPoll(mypolling_fct, handle, polling_interval);
> >>>> PyAsync_ReturnAwait();
> >>>> }
> >>>>
> >>>> mypolling_fct(int handle) {
> >>>> /* Check completion of operation */
> >>>> if (completed) {
> >>>> /* Fetch results into value */
> >>>> PyAsync_UnregisterPoll(mypolling_fct, handle);
> >>>> PyAsync_ReturnReady(value);
> >>>> }
> >>>> else
> >>>> PyAsync_ReturnContinue();
> >>>> }
> >>>>
> >>>> and like this in callback mode:
> >>>>
> >>>> long_running_query(...) {
> >>>> /* Create a handle for a long running operation... */
> >>>> PyAsync_RegisterCallback(myquery_done, handle, timeout);
> >>>> /* Start long running operation... */
> >>>> PyAsync_ReturnAwait();
> >>>> }
> >>>>
> >>>> myquery_done(int handle, PyObject *value) {
> >>>> PyAsync_UnregisterCallback(myquery_done, handle);
> >>>> PyAsync_ReturnReady(value);
> >>>> }
> >>>>
> >>>> Both mechanisms will likely have to move the long running
> >>>> operation into a separate (blocking) thread, unless there's also
> >>>> a way to register an event loop selector somewhere and the C lib
> >>>> your interfacing to provides this mechanism as well.
> >>>>
> >>>
> >>> I see what you want to do and why now.
> >>>
> >>> However, I can't imagine how this could be implemented
> >>> without asyncio event loop being implemented in C.
> >>> Coroutines from PEP 492 aren't an indepenedant concept,
> >>> they require a library/framework with an event loop
> >>> to make them work.
> >>>
> >>> The only way to go right now is to get 'get_event_loop()'
> >>> and 'loop.call_soon()' working in C using Python C API.
> >>>
> >>> From that point you can implement Future object in C,
> >>> see 'asyncio/futures.py'; Future._schedule_callbacks,
> >>> Future.set_result and Future.__iter__ methods in
> >>> particular.
> >>>
> >>> With all that, you can easily return your future object
> >>> from C methods and everything should work just fine.
> >>>
> >>> I know this isn't ideal, and we actually should start a
> >>> separate discussion on how we can improve things on
> >>> our asyncio C side.
> >>
> >> I'm not expecting this to be ready anytime soon,
> >> just wanted to bring this up, since it may be something
> >> that has to be considered in the async/await API design
> >> early rather than later to not make it too complex to
> >> add later on.
> >>
> >> I'm not sure, but going through the Python API from C
> >> can potentially result in the code to reenter which sounds
> >> like it could cause some unwanted race conditions. A C API
> >> would avoid this.
> >>
> >> Thanks,
> >> --
> >> Marc-Andre Lemburg
> >> eGenix.com
> >>
> >> Professional Python Services directly from the Source (#1, Apr 20 2015)
> >>>>> Python Projects, Coaching and Consulting ... http://www.egenix.com/
> >>>>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/
> >>>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
> >> ________________________________________________________________________
> >>
> >> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> >>
> >> eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
> >> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> >> Registered at Amtsgericht Duesseldorf: HRB 46611
> >> http://www.egenix.com/company/contact/
> >> _______________________________________________
> >> Python-ideas mailing list
> >> Python-ideas at python.org
> >> https://mail.python.org/mailman/listinfo/python-ideas
> >> Code of Conduct: http://python.org/psf/codeofconduct/
> >
> >
> >
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150420/7c487bb6/attachment-0001.html>
More information about the Python-ideas
mailing list