[Python-ideas] The async API of the future: yield-from

[Calvin Spealman]

On Mon, Oct 15, 2012 at 10:10 PM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Oct 15, 2012 at 1:14 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Nick Coghlan wrote:
>>
>>> The main primitive I personally want out of an async API is a
>>> task-based equivalent to concurrent.futures.as_completed() [1]. This
>>> is what I meant about iteration being a bit of a mess: the way the
>>> as_completed() works, the suspend/resume channel of the iterator
>>> protocol is being used to pass completed future objects back to the
>>> calling iterator. That means that channel *can't* be used to talk
>>> between the coroutine and the scheduler,
>>
>>
>> I had to read this a couple of times before I figured out
>> what you're talking about, but I get it now.
>>
>> This is an instance of a general problem that was noticed
>> back when I was discussing my cofunctions idea: using
>> generator-based coroutines, it's not possible to have a
>> "suspendable iterator", because that would require "yield"
>> to have two conflicting meanings: "suspend this coroutine"
>> on one hand, and "provide a value to my caller" on the
>> other.
>>
>> Unfortunately, I suspect that a truly elegant solution to this
>> problem will require yet another language addition -- something
>> like
>>
>>    yield for item in subtask():
>>       ...
>>
>> which would run a slightly different version of the iterator
>> protocol in which values to be yield are wrapped somehow
>> (I haven't figured out all the details yet).
>
> I think I ran into a similar issue with NDB when defining iteration
> over an asynchronous query. My solution:
>
>   q = <some query specification>
>   it = q.iter()  # Fire off the query to the datastore
>   while (yield it.has_next_async()):  # Block until one result
>     emp = it.next()  # Get the result that was buffered on the iterator
>     print emp.name, emp.age  # Use it

Crazy Idea I Probably Don't Actually Want:

for yield emp in q:
  print emp.name, emp.age

Turns into something like:

_it = iter(q)
for _emp in _it:
    emp = yield _emp
    print emp.name, emp.age

> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy