[Python-ideas] Async API
Guido van Rossum
guido at python.org
Thu Oct 25 01:12:00 CEST 2012
On Wed, Oct 24, 2012 at 4:03 PM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> Hi Guido,
> On 2012-10-24, at 6:43 PM, Guido van Rossum <guido at python.org> wrote:
>> What's the problem with just letting the cleanup take as long as it
>> wants to and do whatever it wants? That's how try/finally works in
>> regular Python code.
> The problem appears when you add timeouts support.
> Let me show you an abstract example (I won't use yield_froms, but I'm
> sure that the problem is the same with them):
> def fetch_comments(app):
> session = yield app.new_session()
> return (yield session.query(...))
> yield session.close()
> and now we execute that with:
> #: Get a list of comments; throw a TimeoutError if it
> #: takes more than 1 second
> comments = yield fetch_comments(app).with_timeout(1.0)
> Now, scheduler starts with 'fetch_comments', then executes
> 'new_session', then executes 'session.query' in a round-robin fashion.
> Imagine, that database query took a bit less than a second to execute,
> scheduler pushes the result in coroutine, and then a timeout event occurs.
> So scheduler throws a 'TimeoutError' in the coroutine, thus preventing
> the 'session.close' to be executed. There is no way for a scheduler to
> understand, that there is no need in pushing the exception right now,
> as the coroutine is in its finally block.
> And this situation is a pretty common when you have such timeouts
> mechanism in place and widely used.
Ok, I can understand. But still, this is a problem with timeouts in
general, not just with timeouts in a yield-based environment. How does
e.g. Twisted deal with this?
As a work-around, I could imagine some kind of with-statement that
tells the scheduler we're already in the finally clause (it could
still send you a timeout if your cleanup takes way too long):
yield <regular code>
yield <cleanup code>
Of course this could be abused, but at your own risk -- the scheduler
only gives you a fixed amount of extra time and then it's quits.
--Guido van Rossum (python.org/~guido)
More information about the Python-ideas