[Python-ideas] Async API

[Yury Selivanov]

Hi Guido,

On 2012-10-24, at 6:43 PM, Guido van Rossum <guido at python.org> wrote:

> On Wed, Oct 24, 2012 at 2:30 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Yury Selivanov wrote:
>> 
>>> It's not about try..finally nesting, it's about Scheduler being aware
>>> that a coroutine is in its 'finally' block and thus shouldn't be
>>> interrupted
>>> at the moment
>> 
>> 
>> It would be a bad idea to make a close() method, or anything else
>> that might be needed for cleanup purposes, be a 'yield from' call.
>> If it's an ordinary function, it can't be interrupted in the world
>> we're talking about, so the PEP 419 problem doesn't apply.
>> 
>> If I were feeling in a radical mood, I might go as far as suggesting
>> that 'yield' and 'yield from' be syntactically forbidden inside
>> a finally clause. That would force you to design your cleanup
>> code to be safe from interruptions.
> 
> What's the problem with just letting the cleanup take as long as it
> wants to and do whatever it wants? That's how try/finally works in
> regular Python code.

The problem appears when you add timeouts support.

Let me show you an abstract example (I won't use yield_froms, but I'm
sure that the problem is the same with them):

   @coroutine
   def fetch_comments(app):
       session = yield app.new_session()
       try:
            return (yield session.query(...))
       finally:
            yield session.close()

and now we execute that with:

   #: Get a list of comments; throw a TimeoutError if it
   #: takes more than 1 second
   comments = yield fetch_comments(app).with_timeout(1.0)

Now, scheduler starts with 'fetch_comments', then executes
'new_session', then executes 'session.query' in a round-robin fashion.

Imagine, that database query took a bit less than a second to execute,
scheduler pushes the result in coroutine, and then a timeout event occurs.
So scheduler throws a 'TimeoutError' in the coroutine, thus preventing
the 'session.close' to be executed.  There is no way for a scheduler to
understand, that there is no need in pushing the exception right now,
as the coroutine is in its finally block.

And this situation is a pretty common when you have such timeouts 
mechanism in place and widely used.

-
Yury