A matter of queues, tasks and multiprocessing

[John Nagle]

On 11/12/2010 12:18 PM, John Nagle wrote:
> On 11/10/2010 9:25 AM, Emanuele D'Arrigo wrote:
>> Greetings everybody,
>>
>> I've tried to come up with this message for a couple of weeks now and
>> it doesn't look like I'm getting any clearer in my thoughts so I
>> decided that it's probably best to take the plunge and ask you guys to
>> kindly throw me a rope...
>>
>> What I'm trying to come up with is some kind of simple, dynamically
>> scalable, domain-agnostic, multi-processor capable, tasking system.
>> Maybe I'm already giving myself too many constraints here, i.e. it
>> might all be possible except... the "simple" part.
>
> No, the problem is vagueness. We can't tell from your description
> what you want to do. A real-time desktop application? Warehouse-scale
> computing? Your own "cloud"?
>
> Incidentally, bear in mind that once you scale beyond tens of
> machines, error detection and recovery starts to dominate
> the problem.
>
> John Nagle

    Here's how Google does it:

	http://www.stanford.edu/class/ee380/Abstracts/101110-slides.pdf

				John Nagle