distributed queue?

[Bjoern Schliessmann]

Paul Rubin wrote:

> Does anyone have an implementation of a distributed queue?  I.e. I
> have a long running computation f(x) and I'd like to be able to
> evaluate it (for different values of x) on a bunch of different
> computers simultaneously, the usual "worker thread" pattern except 
> distributed across a network.  

Doesn't sound difficult to implement.

> I guess this is pretty easy to write with a centralized socket
> listener that dispatches requests through a Queue to multiple
> threads on the same machine, each talking synchronously to a
> server socket.  

(Why does everyone think that "concurrency" equals "usage of
multiple threads"?)

> I wonder if something like it already exists.  I see a little bit
> of discussion in the newsgroup archive but no obvious pointers to
> code.

Try Twisted for your networking needs. With it it's quite easy to
develop an own little protocol (e. g. on top of TCP). When you have
this ready, writing a server and client using it is a matter of
minutes.

http://twistedmatrix.com/projects/core/documentation/howto/servers.html
http://twistedmatrix.com/projects/core/documentation/howto/clients.html

For bigger needs, Twisted also has RPC features.

Regards,


Björn


-- 
BOFH excuse #281:

The co-locator cannot verify the frame-relay gateway to the ISDN
server.