[Baypiggies] Help ! How to exchange python objects between two python processes on the same machine ?

Riaz Rizvi riazrizvi at gmail.com
Fri Jul 10 20:35:00 CEST 2009


Here's an XMLRPC quick start:

1. Type this in a new python script called pyserve.py

from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler

# Restrict to a particular path.
class RequestHandler(SimpleXMLRPCRequestHandler):
     rpc_paths = ('/RPC2',)

# Create server
server = SimpleXMLRPCServer(("localhost", 8001),
                             requestHandler=RequestHandler)
server.register_introspection_functions()

x_to_y_buffer=[]
def send_fromxtoy(x):
	global x_to_y_buffer
	x_to_y_buffer.append(x)
	return len(x_to_y_buffer)

server.register_function(send_fromxtoy, 'send_fromxtoy')

def receive_fromxtoy():
	global x_to_y_buffer
	if len(x_to_y_buffer):
		return True,x_to_y_buffer.pop(0)
	else:
		return False,0

server.register_function(receive_fromxtoy, 'receive_fromxtoy')

# Run the server's main loop
server.serve_forever()

2. Run that script from your machine's command prompt:
	>>> python pyserve.py

3. Start interactive sessions to test your data exchanger:
 From a second command prompt run ipython and type
	In [1]: import xmlrpclib
  	In [2]: xmlrpclib.ServerProxy('http://localhost: 
8001').send_fromxtoy('hello1')
  	In [3]: xmlrpclib.ServerProxy('http://localhost: 
8001').send_fromxtoy('hello2')

 From a third command prompt run ipython and type
	In [1]: import xmlrpclib
  	In [2]: xmlrpclib.ServerProxy('http://localhost: 
8001').receive_fromxtoy()
  	In [3]: xmlrpclib.ServerProxy('http://localhost: 
8001').receive_fromxtoy()
  	In [4]: xmlrpclib.ServerProxy('http://localhost: 
8001').receive_fromxtoy()

(To be clear an xmlrpc 'server' is just another process whose only job  
is to send and receive stuff. You will put the send/receive calls, of  
step 3, in the two processes you have already written).

See also http://docs.python.org/library/simplexmlrpcserver.html

Riaz

On Jul 10, 2009, at 10:40 AM, baypiggies-request at python.org wrote:

> Send Baypiggies mailing list submissions to
> 	baypiggies at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://mail.python.org/mailman/listinfo/baypiggies
> or, via email, send a message with subject or body 'help' to
> 	baypiggies-request at python.org
>
> You can reach the person managing the list at
> 	baypiggies-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Baypiggies digest..."
>
>
> Today's Topics:
>
>   1. Re:  Timing: OSCON meeting (Aahz)
>   2. Re:  Help ! How to exchange python objects between two	python
>      processes on the same machine ? (Bill Janssen)
>   3. Re:  Help ! How to exchange python objects between two	python
>      processes on the same machine ? (Harish Mallipeddi)
>   4. Re:  Help ! How to exchange python objects between	two	python
>      processes on the same machine ? (Aahz)
>   5. Re:  Help ! How to exchange python objects between two	python
>      processes on the same machine ? (Harish Mallipeddi)
>   6. Re:  Help ! How to exchange python objects between two	python
>      processes on the same machine ? (David Berthelot)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 10 Jul 2009 09:35:42 -0700
> From: Aahz <aahz at pythoncraft.com>
> To: baypiggies at python.org
> Subject: Re: [Baypiggies] Timing: OSCON meeting
> Message-ID: <20090710163542.GB16853 at panix.com>
> Content-Type: text/plain; charset=us-ascii
>
> On Thu, Jul 09, 2009, Aahz wrote:
>>
>> Unless someone objects by midnight tonight (Thursday), I'll tell the
>> OSCON people to schedule BayPIGgies for 8-9:30pm Thurs 7/23 in the  
>> same
>> room as Wes's BoF -- that should give us enough time to make
>> announcements.
>
> Posted to c.l.py and c.l.py.announce -- would someone please update  
> the
> website and include a note about Wes's BoF in addition to the meeting
> itself?
> -- 
> Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/
>
> "as long as we like the same operating system, things are cool." -- 
> piranha
>
>
> ------------------------------
>
> Message: 2
> Date: Fri, 10 Jul 2009 09:58:45 PDT
> From: Bill Janssen <janssen at parc.com>
> To: David Berthelot <d_berthelot at yahoo.com>
> Cc: baypiggies at python.org
> Subject: Re: [Baypiggies] Help ! How to exchange python objects
> 	between two	python processes on the same machine ?
> Message-ID: <43480.1247245125 at parc.com>
>
> David Berthelot <d_berthelot at yahoo.com> wrote:
>
>> Ok, I'm trying to send some python objects from one Python process  
>> to another, on the same machine.
>
> Try "mmap", for inter-process, and "Queue", for intra-process (between
> threads).  Both in the standard library.  Probably want to pickle the
> data before writing it -- use a Pickler on the write side, and an
> Unpickler on the read side.
>
> Bill
>
>
> ------------------------------
>
> Message: 3
> Date: Fri, 10 Jul 2009 22:51:31 +0530
> From: Harish Mallipeddi <harish.mallipeddi at gmail.com>
> To: Bill Janssen <janssen at parc.com>
> Cc: baypiggies at python.org
> Subject: Re: [Baypiggies] Help ! How to exchange python objects
> 	between two	python processes on the same machine ?
> Message-ID:
> 	<e01b80590907101021m7d55d860m461f3f2d32b2f3a6 at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On Fri, Jul 10, 2009 at 10:28 PM, Bill Janssen <janssen at parc.com>  
> wrote:
>
>> David Berthelot <d_berthelot at yahoo.com> wrote:
>>
>>> Ok, I'm trying to send some python objects from one Python process  
>>> to
>> another, on the same machine.
>>
>> Try "mmap", for inter-process, and "Queue", for intra-process  
>> (between
>> threads).  Both in the standard library.  Probably want to pickle the
>> data before writing it -- use a Pickler on the write side, and an
>> Unpickler on the read side.
>>
>
> You could also use "processing" to exchange objects between processes.
>
> http://pypi.python.org/pypi/processing
>
> -- 
> Harish Mallipeddi
> http://blog.poundbang.in
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://mail.python.org/pipermail/baypiggies/attachments/20090710/2ee581ad/attachment-0001.htm 
> >
>
> ------------------------------
>
> Message: 4
> Date: Fri, 10 Jul 2009 10:29:23 -0700
> From: Aahz <aahz at pythoncraft.com>
> To: baypiggies at python.org
> Subject: Re: [Baypiggies] Help ! How to exchange python objects
> 	between	two	python processes on the same machine ?
> Message-ID: <20090710172923.GA11768 at panix.com>
> Content-Type: text/plain; charset=us-ascii
>
> On Fri, Jul 10, 2009, Harish Mallipeddi wrote:
>>
>> You could also use "processing" to exchange objects between  
>> processes.
>>
>> http://pypi.python.org/pypi/processing
>
> That's the same thing as the module multiprocessing, for versions of
> Python prior to 2.6.
> -- 
> Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/
>
> "as long as we like the same operating system, things are cool." -- 
> piranha
>
>
> ------------------------------
>
> Message: 5
> Date: Fri, 10 Jul 2009 23:07:43 +0530
> From: Harish Mallipeddi <harish.mallipeddi at gmail.com>
> To: Aahz <aahz at pythoncraft.com>
> Cc: baypiggies at python.org
> Subject: Re: [Baypiggies] Help ! How to exchange python objects
> 	between two	python processes on the same machine ?
> Message-ID:
> 	<e01b80590907101037k235acd6fw69229d8bbe8be895 at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On Fri, Jul 10, 2009 at 10:59 PM, Aahz <aahz at pythoncraft.com> wrote:
>
>> On Fri, Jul 10, 2009, Harish Mallipeddi wrote:
>>>
>>> You could also use "processing" to exchange objects between  
>>> processes.
>>>
>>> http://pypi.python.org/pypi/processing
>>
>> That's the same thing as the module multiprocessing, for versions of
>> Python prior to 2.6.
>> --
>> Aahz (aahz at pythoncraft.com)           <*>
>> http://www.pythoncraft.com/
>>
>
> My bad - didn't read the entire thread before answering :)
>
> -- 
> Harish Mallipeddi
> http://blog.poundbang.in
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://mail.python.org/pipermail/baypiggies/attachments/20090710/b2f3164a/attachment-0001.htm 
> >
>
> ------------------------------
>
> Message: 6
> Date: Fri, 10 Jul 2009 10:40:49 -0700 (PDT)
> From: David Berthelot <d_berthelot at yahoo.com>
> To: Aahz <aahz at pythoncraft.com>, baypiggies at python.org
> Subject: Re: [Baypiggies] Help ! How to exchange python objects
> 	between two	python processes on the same machine ?
> Message-ID: <592466.96345.qm at web57614.mail.re1.yahoo.com>
> Content-Type: text/plain; charset="us-ascii"
>
> Thanks for all the answers, I forgot to mention that the speed of  
> transfer between processes is an import factor, so hard-disk files  
> are out of the question.
>
> So I'm going to look into 3 proposed solutions:
> 1. mmap [+ cpickle]
> 2. processing
> 3. xmlrpc (this one is least clear to me, it looks like it needs  
> some server to run, I just need to look more into it).
>
> Thanks,
>
> David
>
>
>
>
> ________________________________
> From: Aahz <aahz at pythoncraft.com>
> To: baypiggies at python.org
> Sent: Friday, July 10, 2009 10:29:23 AM
> Subject: Re: [Baypiggies] Help ! How to exchange python objects  
> between two python processes on the same machine ?
>
> On Fri, Jul 10, 2009, Harish Mallipeddi wrote:
>>
>> You could also use "processing" to exchange objects between  
>> processes.
>>
>> http://pypi.python.org/pypi/processing
>
> That's the same thing as the module multiprocessing, for versions of
> Python prior to 2.6.
> -- 
> Aahz (aahz at pythoncraft.com)           <*>        http://www.pythoncraft.com/
>
> "as long as we like the same operating system, things are cool." -- 
> piranha
> _______________________________________________
> Baypiggies mailing list
> Baypiggies at python.org
> To change your subscription options or unsubscribe:
> http://mail.python.org/mailman/listinfo/baypiggies
>
>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://mail.python.org/pipermail/baypiggies/attachments/20090710/d6dd97d1/attachment.htm 
> >
>
> ------------------------------
>
> _______________________________________________
> Baypiggies mailing list
> Baypiggies at python.org
> To change your subscription options or unsubscribe:
> http://mail.python.org/mailman/listinfo/baypiggies
>
> End of Baypiggies Digest, Vol 45, Issue 14
> ******************************************

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/baypiggies/attachments/20090710/f0691574/attachment-0001.htm>


More information about the Baypiggies mailing list