remote multiprocessing, shared object

[Kushal Kumaran]

On Thu, Apr 8, 2010 at 3:04 AM, Norm Matloff <matloff at doe.com> wrote:
> Should be a simple question, but I can't seem to make it work from my
> understanding of the docs.
>
> I want to use the multiprocessing module with remote clients, accessing
> shared lists.  I gather one is supposed to use register(), but I don't
> see exactly how.  I'd like to have the clients read and write the shared
> list directly, not via some kind of get() and set() functions.  It's
> clear how to do this in a shared-memory setting, but how can one do it
> across a network, i.e. with serve_forever(), connect() etc.?
>
> Any help, especially with a concrete example, would be much appreciated.
> Thanks.
>

There's an example in the multiprocessing documentation.
http://docs.python.org/library/multiprocessing.html#using-a-remote-manager

It creates a shared queue, but it's easy to modify for lists.

For example, here's your shared list server:
from multiprocessing.managers import BaseManager
shared_list = []
class ListManager(BaseManager): pass
ListManager.register('get_list', callable=lambda:shared_list)
m = ListManager(address=('', 50000), authkey='abracadabra')
s = m.get_server()
s.serve_forever()

A client that adds an element to your shared list:
import random
from multiprocessing.managers import BaseManager
class ListManager(BaseManager): pass
ListManager.register('get_list')
m = ListManager(address=('localhost', 50000), authkey='abracadabra')
m.connect()
l = m.get_list()
l.append(random.random())

And a client that prints out the shared list:
from multiprocessing.managers import BaseManager
class ListManager(BaseManager): pass
ListManager.register('get_list')
m = ListManager(address=('localhost', 50000), authkey='abracadabra')
m.connect()
l = m.get_list()
print str(l)

-- 
regards,
kushal