[stdlib-sig] futures - PEP and API
brian at sweetapp.com
Sat Nov 7 23:41:56 CET 2009
On 8 Nov 2009, at 03:13, Antoine Pitrou wrote:
>> The PEP is right here:
>> I'm interested in hearing specific complaints about the API in the
>> context of what it's trying to *do*. The only thing which jumped out
>> at me was the number of methods on FutureList;
> Yes that would be the first complaint. Then many of those methods
> are as
> (un)trustable as, say, Queue.qsize().
> An example :
> Return an iterator over all `Future` instances that completed
> were cancelled."""
> First, it claims to return an iterator but the internal container
> mutate while iterating (since it can be mutated when a task terminates
> in another thread). So the API looks broken with respect to what the
> semantics dictate. It should probably return a distinct container
> or set) instead.
> Second, by the time the result is processed by the caller, there's no
> way to know if the information is still valid or not. It's entirely
> speculative, which makes it potentially deceiving -- and should be
> mentioned in the doc.
> Return `True` if any `Future` in the list has completed or was
> successfully cancelled."""
> Same problem. Please note that it can be removed if `done_futures()`
> returns a container, since you then just have to do a boolean check on
> the container (that would remove 5 methods :-)).
> Then about the Future API itself. I would argue that if we want it
> to be
> a simple helper, it should be as simple to use as a weakref.
> That is, rather than :
> Return the value returned by the call.
> Return the exception raised by the call."""
> Make the object callable, such as `future(timeout=None)` either
> the computed result (if successful), raises an exception (if failed)
> raises a TimeoutError.
> Then about the Executor API. I don't understand why we have the
> possibility to wait on a FutureList *and* on the Executor's
> run_to_results() method. I think all wait-type methods should be
> in to the Future or FutureList API, and the Executor should only
> generate that Future(List).
> Practically, there should be two ways to wait for multiple results,
> depending on whether you need the results ordered or not. In the web
> crawling situation given as example, it is silly to wait for the
> in order rather than process each result as soon as it gets available.
> I don't understand why the Executor seems to be used as a context
> manager in the examples. Its resources are still alive after the
> since the tasks are still executin, so it can't possibly have
> cleaned up
> anything, has it?
The executor needs to know when no more futures will be scheduled so
it can shutdown its thread/process pool after it has finished the
current work items.
Let me think about your other comments for a while.
> (*) And, of course, you start to understand why a callback-based API
> such as Deferreds makes a lot of sense...
> stdlib-sig mailing list
> stdlib-sig at python.org
More information about the stdlib-sig