[Python-ideas] async/await in Python (C API ?)

[Yury Selivanov]

On 2015-04-20 10:49 AM, M.-A. Lemburg wrote:
> On 20.04.2015 15:40, Yury Selivanov wrote:
>> Hi Marc-Andre,
>>
>> On 2015-04-20 9:18 AM, M.-A. Lemburg wrote:
>>> Hi Yury,
>>>
>>> do you have any plans of also exposing an easy to use C API
>>> for async and await ?
>>>
>>> I think this would be useful to e.g. implement asynchronous
>>> database access. Most of those adapters are written in C and
>>> need so a C API would help a lot with this.
>> I think one way to have a convenient C API is to implement
>> a Future-like object in C -- an object with __await__ method
>> that should return an iterator, which should also be
>> implemented in C and provide a way to attach callback
>> or to *pipeline* C functions.
>>
>> This way C code should seamlessly integrate with asyncio.
>>
>> I doubt that we have time to include this kind of API in
>> the PEP, but it can be developed as a separate library.
>>
>> Your thoughts?
> I'm just sketching here, since I don't have much experience
> with writing async code (I'm using threads and multiple
> processes instead) and it probably shows ;-)
>
>  From the C side of things, I think using callbacks
> and a way to register/unregister polling functions
> would be great to have.
>
> The C code could then run like this in polling mode:
>
> long_running_query(...) {
>      /* Start long running operation... creating a handle for it */
>      PyAsync_RegisterPoll(mypolling_fct, handle, polling_interval);
>      PyAsync_ReturnAwait();
> }
>
> mypolling_fct(int handle) {
>      /* Check completion of operation */
>      if (completed) {
>          /* Fetch results into value */
>          PyAsync_UnregisterPoll(mypolling_fct, handle);
>          PyAsync_ReturnReady(value);
>      }
>      else
>          PyAsync_ReturnContinue();
> }
>
> and like this in callback mode:
>
> long_running_query(...) {
>      /* Create a handle for a long running operation... */
>      PyAsync_RegisterCallback(myquery_done, handle, timeout);
>      /* Start long running operation... */
>      PyAsync_ReturnAwait();
> }
>
> myquery_done(int handle, PyObject *value) {
>      PyAsync_UnregisterCallback(myquery_done, handle);
>      PyAsync_ReturnReady(value);
> }
>
> Both mechanisms will likely have to move the long running
> operation into a separate (blocking) thread, unless there's also
> a way to register an event loop selector somewhere and the C lib
> your interfacing to provides this mechanism as well.
>

I see what you want to do and why now.

However, I can't imagine how this could be implemented
without asyncio event loop being implemented in C.
Coroutines from PEP 492 aren't an indepenedant concept,
they require a library/framework with an event loop
to make them work.

The only way to go right now is to get 'get_event_loop()'
and 'loop.call_soon()' working in C using Python C API.

 From that point you can implement Future object in C,
see 'asyncio/futures.py'; Future._schedule_callbacks,
Future.set_result and Future.__iter__ methods in
particular.

With all that, you can easily return your future object
from C methods and everything should work just fine.

I know this isn't ideal, and we actually should start a
separate discussion on how we can improve things on
our asyncio C side.

Thanks,
Yury