[Python-Dev] futures API
raymond.hettinger at gmail.com
Thu Dec 9 23:39:38 CET 2010
On Dec 9, 2010, at 9:02 AM, Brian Quinlan wrote:
> On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote:
>> I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86) by the futures module which was announced in python 3.2 beta. I am a bit stuck with it, so I have a few questions about the futures:
>> 1. Is the futures API frozen?
Yes, unless the current API is defective in some way. A beta1 release is a chance for everyone to exercise the new API and discover whether it is problematic in any real world applications.
>> 2. How hard would it be to return the tasks processed in an output queue to process/consume the results while they are returned? The code does not seem to be very open for monkey patching.
> You can associate a callback with a submitted future. That callback could add the future to your queue.
The would be a good example for the docs.
>> 3. How hard would it be to add new tasks dynamically (after a task is executed) and have the futures object never complete?
> I'm not sure that I understand your question. You can submit new work to an Executor at until time until it is shutdown and a work item can take as long to complete as you want. If you are contemplating tasks that don't complete then maybe you could be better just scheduling a thread.
>> 4. Is there a performance evaluation of the futures code (execution overhead) ?
> No. Scott Dial did make some performance improvements so he might have a handle on its overhead.
FWIW, the source code is short and readable. From my quick read, it looks to be a relatively thin wrapper/adapter around existing tools. Most of the work still gets done by the threads or processes themselves. Think of this as a cleaner, more centralized API around the current toolset -- there is no deep new technology under the hood.
More information about the Python-Dev