[Numpy-discussion] xml-rpc with numpy arrays

[Robert Kern]

On Tue, Sep 30, 2008 at 12:53, Brian Blais <bblais at bryant.edu> wrote:
> Hello,
> I am trying to use xml-rpc to be able to run some simulations remotely.  I
> am running into a problem with the transfer of numpy arrays.  my server code
> looks like:
> #!/usr/bin/env python
> def again(x):  # test out the sending of data
>     return [x,x]
>
> from SimpleXMLRPCServer import SimpleXMLRPCServer
> SimpleXMLRPCServer.allow_reuse_address = 1
> server = SimpleXMLRPCServer(("", 8000))
> server.register_function(again)
> try:
>     print "Serving..."
>     server.serve_forever() # Start the server
> finally:
>     print "done."
>     server.server_close()
>
>
> my client code looks like:
> import numpy
> from xmlrpclib import ServerProxy
> server=ServerProxy('http://localhost:8000')
> server.again(5)  # this works
> b=numpy.random.rand(5,5)
> server.again(b)  # this gives an error
> this gives the error: <type 'exceptions.TypeError'>: cannot marshal <type
> 'numpy.ndarray'> objects
> which seems to be a deficiency of the marshal library, or perhaps I am doing
> something wrong.  Is there a way to fix this?  Is there another approach
> that I should be using?

The marshal module *only* handles builtin Python types. It explicitly
does not handle anything from third parties like numpy. As Uwe
suggests, you can use tostring()/fromstring() yourself and also pass
along the dtype and shape information. With numpy 1.2.0, there is a
somewhat better alternative. I have defined a small file format that
should represent all numpy arrays with dtype and shape information
embedded in its header. That way, you just have to pass the string.


from cStringIO import StringIO
from numpy.lib import format

def from_string(s):
    f = StringIO(s)
    arr = format.read_array(f)
    return arr

def to_string(arr):
    f = StringIO()
    format.write_array(f, arr)
    s = f.getvalue()
    return s

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco