multiprocessing managers and socket connection.

[bouncyinc at gmail.com]

Is this "server" something you wrote from scratch or something that is just being used. Some details are left out

----------


------Original Message------
From: Chris <chris0wj at gmail.com>
To: <python-list at python.org>
Date: Fri, 18 Sep 2009 09:10:15 AM -0700
Subject: Re: multiprocessing managers and socket connection.

On Aug 31, 10:41 pm, Terry <terry.yin... at gmail.com> wrote:
> On Aug 26, 7:25 pm, Chris <chris... at gmail.com> wrote:
>
>
>
> > On Aug 25, 9:11 pm, Terry <terry.yin... at gmail.com> wrote:
>
> > > On Aug 25, 10:14 pm, Chris <chris... at gmail.com> wrote:
>
> > > > I've been using multiprocessing managers and I really like the
> > > > functionality.
>
> > > > I have a question about reconnecting to a manager. I have a situation
> > > > where I start on one machine (A) a manager that is listening and then
> > > > on another machine (B) connects to that manager and uses its proxy
> > > > object to call functions on the manager's computer; this all works as
> > > > expected. But, if the manager from A shuts down, B's application won't
> > > > notice because in the MP code it ignores socket error
> > > > errno.ECONNREFUSED. If A becomes available again or is restarted, B
> > > > doesn't automatically reconnect either and continue its operation.
> > > > It's function is basically stopped.
>
> > > > Here is the code from connection.py:
> > > > while 1:
> > > >         try:
> > > >             s.connect(address)
> > > >         except socket.error, e:
> > > >             if e.args[0] != errno.ECONNREFUSED: # connection refused
> > > >                 debug('failed to connect to address %s', address)
> > > >                 raise
> > > >             time.sleep(0.01)
> > > >         else:
> > > >             break
>
> > > > How can I have B automatically reconnect to A and continue its work
> > > > once A is available again?
>
> > > I think you need to retry repeatedly until successfully connected.
>
> > > br, Terry
>
> > I'm having issue after once connected. If the server goes down during
> > a long-running connection. I would want to be notified so I could try
> > to reconnect. I'm doing more experimenting now and will try to post an
> > example.
>
> Hi Chris,
>
> Are you sure that the proxy object keeps a permanent connection to the
> server?
>
> br, Terry

Sorry for the delay in response. I was able to find a solution by NOT
using sockets, but using PIPES instead. The socket connections to the
managers don't disconnect properly.

Here's how to reproduce the situation (yes it's a stupid example, but
that's the point):

mp_manager_test.py --> http://python.pastebin.com/m3d10e343
mp_manager_test_client.py --> http://python.pastebin.com/m7a8fda4c

run the following sequence which requires 3 shells.
shell1> python mp_manager_test.py start_server
shell2> python mp_manager_test_client.py
shell3> python mp_manager_test.py stop_server

Run the 3rd line while the client is accessing the server.
When you use the socket for a connection, it hangs.
When you use a PIPE, it throws the exception and actually exits; so I
can catch the exception and handle it properly instead of having a
hanging program. I am not sure how well a remote pipe will work though
as I have yet to try it over a network between machines.

To switch from socket to pipe, just switch the comments at the top in
the source files:
ADDY = ("127.0.0.1",9000)
ADDY2 = ("127.0.0.1",9001)
#ADDY = r'\\.\pipe\PipeName'
#ADDY2 = r'\\.\pipe\PipeName2'

Chris
-- 
http://mail.python.org/mailman/listinfo/python-list