multiprocessing managers and socket connection.

Chris chris0wj at gmail.com
Fri Sep 18 18:59:45 CEST 2009


On Sep 18, 12:42 pm, "bouncy... at gmail.com" <bouncy... at gmail.com>
wrote:
> Is this "server" something you wrote from scratch or something that is just being used. Some details are left out
>
> ----------
>
> ------Original Message------
> From: Chris <chris... at gmail.com>
> To: <python-l... at python.org>
> Date: Fri, 18 Sep 2009 09:10:15 AM -0700
> Subject: Re: multiprocessing managers and socket connection.
>
> On Aug 31, 10:41 pm, Terry <terry.yin... at gmail.com> wrote:
> > On Aug 26, 7:25 pm, Chris <chris... at gmail.com> wrote:
>
> > > On Aug 25, 9:11 pm, Terry <terry.yin... at gmail.com> wrote:
>
> > > > On Aug 25, 10:14 pm, Chris <chris... at gmail.com> wrote:
>
> > > > > I've been using multiprocessing managers and I really like the
> > > > > functionality.
>
> > > > > I have a question about reconnecting to a manager. I have a situation
> > > > > where I start on one machine (A) a manager that is listening and then
> > > > > on another machine (B) connects to that manager and uses its proxy
> > > > > object to call functions on the manager's computer; this all works as
> > > > > expected. But, if the manager from A shuts down, B's application won't
> > > > > notice because in the MP code it ignores socket error
> > > > > errno.ECONNREFUSED. If A becomes available again or is restarted, B
> > > > > doesn't automatically reconnect either and continue its operation.
> > > > > It's function is basically stopped.
>
> > > > > Here is the code from connection.py:
> > > > > while 1:
> > > > >         try:
> > > > >             s.connect(address)
> > > > >         except socket.error, e:
> > > > >             if e.args[0] != errno.ECONNREFUSED: # connection refused
> > > > >                 debug('failed to connect to address %s', address)
> > > > >                 raise
> > > > >             time.sleep(0.01)
> > > > >         else:
> > > > >             break
>
> > > > > How can I have B automatically reconnect to A and continue its work
> > > > > once A is available again?
>
> > > > I think you need to retry repeatedly until successfully connected.
>
> > > > br, Terry
>
> > > I'm having issue after once connected. If the server goes down during
> > > a long-running connection. I would want to be notified so I could try
> > > to reconnect. I'm doing more experimenting now and will try to post an
> > > example.
>
> > Hi Chris,
>
> > Are you sure that the proxy object keeps a permanent connection to the
> > server?
>
> > br, Terry
>
> Sorry for the delay in response. I was able to find a solution by NOT
> using sockets, but using PIPES instead. The socket connections to the
> managers don't disconnect properly.
>
> Here's how to reproduce the situation (yes it's a stupid example, but
> that's the point):
>
> mp_manager_test.py -->http://python.pastebin.com/m3d10e343
> mp_manager_test_client.py -->http://python.pastebin.com/m7a8fda4c
>
> run the following sequence which requires 3 shells.
> shell1> python mp_manager_test.py start_server
> shell2> python mp_manager_test_client.py
> shell3> python mp_manager_test.py stop_server
>
> Run the 3rd line while the client is accessing the server.
> When you use the socket for a connection, it hangs.
> When you use a PIPE, it throws the exception and actually exits; so I
> can catch the exception and handle it properly instead of having a
> hanging program. I am not sure how well a remote pipe will work though
> as I have yet to try it over a network between machines.
>
> To switch from socket to pipe, just switch the comments at the top in
> the source files:
> ADDY = ("127.0.0.1",9000)
> ADDY2 = ("127.0.0.1",9001)
> #ADDY = r'\\.\pipe\PipeName'
> #ADDY2 = r'\\.\pipe\PipeName2'
>
> Chris
> --http://mail.python.org/mailman/listinfo/python-list
>
>

No, that's a full example posted in those 2 files right there; there
is nothing else needed to run it.



More information about the Python-list mailing list