From jnoller at gmail.com Fri Mar 5 04:08:09 2010 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 4 Mar 2010 22:08:09 -0500 Subject: [stdlib-sig] futures - a new package for asynchronous execution In-Reply-To: <5d44f72f1002252012s10a7174et73076172c3a9eb69@mail.gmail.com> References:

<5d44f72f1002211937s3359b93ev8466676de818e2d4@mail.gmail.com> <598B9770-5129-41D7-9C14-BF151CB48103@sweetapp.com> <5d44f72f1002231204w32a2ebdi5952abf645005680@mail.gmail.com> <07A4B971-3691-4DBF-B62E-75560E9B817F@sweetapp.com> <5d44f72f1002250927r66d0fd4ei34d79cef7c0af54d@mail.gmail.com> <50C77B57-0C81-49B9-99ED-6CA3449D7AE6@sweetapp.com> <5d44f72f1002251926y497fc15bge94bb6b3eacb9897@mail.gmail.com> <5d44f72f1002252012s10a7174et73076172c3a9eb69@mail.gmail.com> <4222a8491003041908q2412b3baia29859b0e5fe0ffc@mail.gmail.com> Message-ID: <2014DE5A-E058-49A2-85F6-ADE4953DA181@sweetapp.com> Wow, timing is everything - I sent Guido an e-mail asking the same thing < 30 seconds ago :-) Cheers, Brian On Mar 5, 2010, at 2:08 PM, Jesse Noller wrote: > *mega snip* > > Jeffrey/Brian/all - Do you think we are ready to move this to the > grist mill of python-dev? Or should we hold off until I get off my > rump and do the concurrent.* namespace PEP? > > jesse From jnoller at gmail.com Fri Mar 5 04:12:44 2010 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 4 Mar 2010 22:12:44 -0500 Subject: [stdlib-sig] futures - a new package for asynchronous execution In-Reply-To: <2014DE5A-E058-49A2-85F6-ADE4953DA181@sweetapp.com> References: <598B9770-5129-41D7-9C14-BF151CB48103@sweetapp.com> <5d44f72f1002231204w32a2ebdi5952abf645005680@mail.gmail.com> <07A4B971-3691-4DBF-B62E-75560E9B817F@sweetapp.com> <5d44f72f1002250927r66d0fd4ei34d79cef7c0af54d@mail.gmail.com> <50C77B57-0C81-49B9-99ED-6CA3449D7AE6@sweetapp.com> <5d44f72f1002251926y497fc15bge94bb6b3eacb9897@mail.gmail.com> <5d44f72f1002252012s10a7174et73076172c3a9eb69@mail.gmail.com> <4222a8491003041908q2412b3baia29859b0e5fe0ffc@mail.gmail.com> <2014DE5A-E058-49A2-85F6-ADE4953DA181@sweetapp.com> Message-ID: <4222a8491003041912r20238390w53bd0424205f6401@mail.gmail.com> On Thu, Mar 4, 2010 at 10:09 PM, Brian Quinlan wrote: > Wow, timing is everything - I sent Guido an e-mail asking the same thing < > 30 seconds ago :-) > > Cheers, > Brian > Well, I'd like to make sure Jeffrey's concerns have been addressed. Once he's happy, I'm ok with pushing it towards it's inevitable end. I think the namespacing is secondary to the futures PEP though jesse From brian at sweetapp.com Fri Mar 5 05:02:11 2010 From: brian at sweetapp.com (Brian Quinlan) Date: Fri, 5 Mar 2010 15:02:11 +1100 Subject: [stdlib-sig] futures - a new package for asynchronous execution In-Reply-To: <5d44f72f1002251926y497fc15bge94bb6b3eacb9897@mail.gmail.com> References: <3BB0755A-2D85-46AF-89A6-CFC4A56744AF@sweetapp.com> <5d44f72f1002201941o76f80045ofa25238be43f455c@mail.gmail.com> <5d44f72f1002211937s3359b93ev8466676de818e2d4@mail.gmail.com> <598B9770-5129-41D7-9C14-BF151CB48103@sweetapp.com> <5d44f72f1002231204w32a2ebdi5952abf645005680@mail.gmail.com> <07A4B971-3691-4DBF-B62E-75560E9B817F@sweetapp.com> <5d44f72f1002250927r66d0fd4ei34d79cef7c0af54d@mail.gmail.com> <50C77B57-0C81-49B9-99ED-6CA3449D7AE6@sweetapp.com> <5d44f72f1002251926y497fc15bge94bb6b3eacb9897@mail.gmail.com> Message-ID: On Feb 26, 2010, at 2:26 PM, Jeffrey Yasskin wrote: > On Thu, Feb 25, 2010 at 7:10 PM, Brian Quinlan > wrote: >> On Feb 26, 2010, at 4:27 AM, Jeffrey Yasskin wrote: >>> Heh. If you're going to put that in the pep, at least make it >>> correct >>> (sleeping is not synchronization): >> >> I can't tell if you are joking or not. Was my demonstration of a >> possible >> deadlock scenario really unclear? > > It's clear; it's just wrong code, even if the futures weren't a cycle. > Waiting using sleep in any decently-sized system is guaranteed to > cause problems. Yes, this example will work nearly every time > (although if you get your load high enough, you'll still see > NameErrors), but it's not the kind of thing we should be showing > users. (For that matter, communicating between futures using globals > is also a bad use of them, but it's not outright broken.) Hey Jeff, I'm trying to demonstrate a pattern of executor usage that is likely to lead to deadlock. If, looking at the example, people are clear that this may lead to deadlock then I don't think that is necessary to write an example that provably always leads to deadlock. In fact, I think that all of the extra locking code required really distracts from the core of the problem being demonstrated. >>> Thanks. I still disagree, and think users are much more likely to be >>> surprised by occasional deadlocks due to cycles of executors than >>> they are >>> about guaranteed deadlocks from cycles of futures, but I don't >>> want to be >>> the only one holding up the PEP by insisting on this. >> >> Cycles of futures are not guaranteed to deadlock. Remove the sleeps >> from my >> example and it will deadlock a small percentage of the time. > > It only fails to deadlock when it fails to create a cycle of futures. > > It sounds like Antoine also wants you to either have the threaded > futures steal work or detect executor cycles and raise an exception. I really don't like the idea of work stealing. Do you have a concrete proposal on how to detect cycles? Cheers, Brian From guido at python.org Fri Mar 5 05:18:40 2010 From: guido at python.org (Guido van Rossum) Date: Thu, 4 Mar 2010 20:18:40 -0800 Subject: [stdlib-sig] futures - a new package for asynchronous execution In-Reply-To: <2014DE5A-E058-49A2-85F6-ADE4953DA181@sweetapp.com> References: <598B9770-5129-41D7-9C14-BF151CB48103@sweetapp.com> <5d44f72f1002231204w32a2ebdi5952abf645005680@mail.gmail.com> <07A4B971-3691-4DBF-B62E-75560E9B817F@sweetapp.com> <5d44f72f1002250927r66d0fd4ei34d79cef7c0af54d@mail.gmail.com> <50C77B57-0C81-49B9-99ED-6CA3449D7AE6@sweetapp.com> <5d44f72f1002251926y497fc15bge94bb6b3eacb9897@mail.gmail.com> <5d44f72f1002252012s10a7174et73076172c3a9eb69@mail.gmail.com> <4222a8491003041908q2412b3baia29859b0e5fe0ffc@mail.gmail.com> <2014DE5A-E058-49A2-85F6-ADE4953DA181@sweetapp.com> Message-ID: And yes, go ahead and bring it up on python-dev. Don't bother with c.l.py unless you are particularly masochistic. --Guido On Thu, Mar 4, 2010 at 7:09 PM, Brian Quinlan wrote: > Wow, timing is everything - I sent Guido an e-mail asking the same thing < > 30 seconds ago :-) > > Cheers, > Brian > > On Mar 5, 2010, at 2:08 PM, Jesse Noller wrote: > >> *mega snip* >> >> Jeffrey/Brian/all - Do you think we are ready to move this to the >> grist mill of python-dev? Or should we hold off until I get off my >> rump and do the concurrent.* namespace PEP? -- --Guido van Rossum (python.org/~guido) From jyasskin at gmail.com Fri Mar 5 06:55:45 2010 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Thu, 4 Mar 2010 21:55:45 -0800 Subject: [stdlib-sig] futures - a new package for asynchronous execution In-Reply-To: References: