Do you guys mind taking this discussion off-list? As of right now neither of your projects are old enough or well known enough to be considered for inclusion in the stdlib at this time so this is really not relevant to the stdlib SIG to continue here. -Brett On Sat, Jan 16, 2010 at 20:53, Anh Hai Trinh <anh.hai.trinh@gmail.com> wrote:
On Sun, Jan 17, 2010 at 5:22 AM, Brian Quinlan <brian@sweetapp.com> wrote:
db_future = executor.submit(setup_database, host, port) data_future = executor.submit(parse_big_xml_file, data) # Maybe do something else here. wait( [db_future, data_future], timeout=10, # If either function raises then we can't complete the operation so # there is no reason to make the user wait. return_when=FIRST_EXCEPTION)
db = db_future.result(timeout=0) data = data.result(timeout=0) save_data_in_db(data, db)
It is definitely true that you can roll your own implementation using threads but the purpose of the futures library is to make that unnecessary.
I don't understand your doubts. To me the example that I gave is simple and useful.
What I mean is that your example is simple enough to do with threads. Here:
[...]
def setup_db(): nonlocal db; db = setup_database(host, port)
def parse_xml(): nonlocal data; data = parse_big_xml(file)
db_thread = threading.Thread(target=setup_db) db_thread.start()
parse_thread = threading.Thread(target=parse_xml) parse_thread.start()
[...] # Do something else here.
db_thread.join() parse_thread.join() save_data_in_db(data, db)
I used "nonlocal" here but you usually do this within a method and refer to self.db, self.data.
I don't understand your doubts. To me the example that I gave is simple and useful.
My doubt is about the usefulness of futures' constructs for the kind of code that "Do several different operations on different data". I think ThreadPool/ProcessPool is really useful when you do
1. Same operation on different data 2. Different operations on same datum
But
3. Different operations on different data
is perhaps misusing it. It is a too general use case because dependency comes into play. What if the different operations depend on each other? A useful thread pool construct for this would be at a more fundamental level, e.g. Grand Central Dispatch.
Perhaps you would give another example?
Cheers, -- // aht http://blog.onideas.ws _______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig