[stdlib-sig] futures - a new package for asynchronous execution

[Brett Cannon]

Do you guys mind taking this discussion off-list? As of right now
neither of your projects are old enough or well known enough to be
considered for inclusion in the stdlib at this time so this is really
not relevant to the stdlib SIG to continue here.

-Brett

On Sat, Jan 16, 2010 at 20:53, Anh Hai Trinh <anh.hai.trinh at gmail.com> wrote:
> On Sun, Jan 17, 2010 at 5:22 AM, Brian Quinlan <brian at sweetapp.com> wrote:
>
>>>> db_future = executor.submit(setup_database, host, port)
>>>> data_future = executor.submit(parse_big_xml_file, data)
>>>> # Maybe do something else here.
>>>> wait(
>>>>   [db_future, data_future],
>>>>   timeout=10,
>>>>   # If either function raises then we can't complete the operation so
>>>>   # there is no reason to make the user wait.
>>>>   return_when=FIRST_EXCEPTION)
>>>>
>>>> db = db_future.result(timeout=0)
>>>> data = data.result(timeout=0)
>>>> save_data_in_db(data, db)
>>
>> It is definitely true that you can roll your own implementation using
>> threads but the purpose of the futures library is to make that unnecessary.
>>
>> I don't understand your doubts. To me the example that I gave is simple and
>> useful.
>
> What I mean is that your example is simple enough to do with threads. Here:
>
> [...]
>
> def setup_db():
>  nonlocal db;
>  db = setup_database(host, port)
>
> def parse_xml():
>  nonlocal data;
>  data = parse_big_xml(file)
>
> db_thread = threading.Thread(target=setup_db)
> db_thread.start()
>
> parse_thread = threading.Thread(target=parse_xml)
> parse_thread.start()
>
> [...] # Do something else here.
>
> db_thread.join()
> parse_thread.join()
> save_data_in_db(data, db)
>
> I used "nonlocal" here but you usually do this within a method and
> refer to self.db, self.data.
>
>
>> I don't understand your doubts. To me the example that I gave is simple and useful.
>
> My doubt is about the usefulness of futures' constructs for the kind
> of code that "Do several different operations on different data". I
> think ThreadPool/ProcessPool is really useful when you do
>
> 1. Same operation on different data
> 2. Different operations on same datum
>
> But
>
> 3. Different operations on different data
>
> is perhaps misusing it. It is a too general use case because
> dependency comes into play. What if the different operations depend on
> each other? A useful thread pool construct for this would be at a more
> fundamental level, e.g. Grand Central Dispatch.
>
> Perhaps you would give another example?
>
> Cheers,
> --
> // aht
> http://blog.onideas.ws
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>