[stdlib-sig] futures - a new package for asynchronous execution

[Brian Quinlan]

On Feb 26, 2010, at 2:26 PM, Jeffrey Yasskin wrote:

> On Thu, Feb 25, 2010 at 7:10 PM, Brian Quinlan <brian at sweetapp.com>  
> wrote:
>> On Feb 26, 2010, at 4:27 AM, Jeffrey Yasskin wrote:
>>> Heh. If you're going to put that in the pep, at least make it  
>>> correct
>>> (sleeping is not synchronization):
>>
>> I can't tell if you are joking or not. Was my demonstration of a  
>> possible
>> deadlock scenario really unclear?
>
> It's clear; it's just wrong code, even if the futures weren't a cycle.
> Waiting using sleep in any decently-sized system is guaranteed to
> cause problems. Yes, this example will work nearly every time
> (although if you get your load high enough, you'll still see
> NameErrors), but it's not the kind of thing we should be showing
> users. (For that matter, communicating between futures using globals
> is also a bad use of them, but it's not outright broken.)

Hey Jeff,

I'm trying to demonstrate a pattern of executor usage that is likely  
to lead to deadlock.

If, looking at the example, people are clear that this may lead to  
deadlock then I don't think that is necessary to write an example that  
provably always leads to deadlock.

In fact, I think that all of the extra locking code required really  
distracts from the core of the problem being demonstrated.


>>> Thanks. I still disagree, and think users are much more likely to be
>>> surprised by occasional deadlocks due to cycles of executors than  
>>> they are
>>> about guaranteed deadlocks from cycles of futures, but I don't  
>>> want to be
>>> the only one holding up the PEP by insisting on this.
>>
>> Cycles of futures are not guaranteed to deadlock. Remove the sleeps  
>> from my
>> example and it will deadlock a small percentage of the time.
>
> It only fails to deadlock when it fails to create a cycle of futures.
>
> It sounds like Antoine also wants you to either have the threaded
> futures steal work or detect executor cycles and raise an exception.


I really don't like the idea of work stealing.

Do you have a concrete proposal on how to detect cycles?

Cheers,
Brian