A matter of queues, tasks and multiprocessing

John Nagle nagle at animats.com
Fri Nov 12 21:18:07 CET 2010


On 11/10/2010 9:25 AM, Emanuele D'Arrigo wrote:
> Greetings everybody,
>
> I've tried to come up with this message for a couple of weeks now and
> it doesn't look like I'm getting any clearer in my thoughts so I
> decided that it's probably best to take the plunge and ask you guys to
> kindly throw me a rope...
>
> What I'm trying to come up with is some kind of simple, dynamically
> scalable, domain-agnostic, multi-processor capable, tasking system.
> Maybe I'm already giving myself too many constraints here, i.e. it
> might all be possible except... the "simple" part.

    No, the problem is vagueness.  We can't tell from your description
what you want to do.  A real-time desktop application? Warehouse-scale
computing?  Your own "cloud"?

    Incidentally, bear in mind that once you scale beyond tens of
machines, error detection and recovery starts to dominate
the problem.

				John Nagle



More information about the Python-list mailing list