[execnet-dev] Close remote gateway?

[Charles Solar]

I did see those examples and I remember that I had then wished that
gateway.exit() had the timeout option that I could use.
I would use gateway groups, but I am afraid its too restrictive to my
design.  Currently I setup a bunch of gateways to the target machines, about
10 or so, and hold onto the gateway instance in a list along with other
params like hostname, unique id, etc.  I then allow the user to close
subsets of remote hosts so something like this works:

server = RemoteServer()
server.open( [ 'server1', 'server2', 'server3' ] )
server.close( ['server1','server2'] )

which would be much harder if I opened all the servers under one group.

I went to the location you suggested and changed the value to 1.0 and it did
in fact help.  The instances fell down just 1 second after I called exit.
So yes, being able to do gateway.exit( timeout=0.5 ) or something would be
very nice.  I guess my remote clients are doing busy work that is preventing
the clean shutdown for 10 seconds, which would explain what I am seeing.

Thanks

On Tue, Jun 15, 2010 at 2:59 PM, holger krekel <holger at merlinux.eu> wrote:

> Hi Charles,
>
> i am interestd to resolve the issue you are seeing.
> i wonder if you looked into these examples:
>
>
> http://codespeak.net/execnet/example/test_group.html#robust-termination-of-ssh-popen-processes
>
> If this did not help your case, you could maybe change the "10.0" parameter
> in execnet/gateway_base.py:687 which tries to terminate execution.
> If this helps we could see to make this parameter API-changeable.
>
> cheers,
> holger
>
> On Tue, Jun 15, 2010 at 14:35 -0500, Charles Solar wrote:
> > Thanks for the pointers.  execnet.gateway does not have a close method,
> but
> > I assume you mean exit because it does have that.
> > Unfortunately after testing it out the last couple of days this method
> does
> > not do what I want either.  It just orphans the remote processes and they
> > eventually get cleaned up 5-10 seconds later by the os.
> >
> >
> > As a result of further testing I am now planning on shifting my design
> away
> > from many python interpreters in favor one interpreter per remote machine
> so
> > this is not really a problem anymore for me.  However it would still be
> nice
> > to be able to call exit on a remote gateway and have it close
> immediately.
> >
> > Also I have been referring to pytest-xdist for some things, but the way I
> > need to test my app requires a different sort of framework capable of
> > handling tests that communicate with other machines.  xdist only handles
> > running unit tests on other machines in a closed environment.
> >
> > Thanks
> >
> > On Sat, Jun 12, 2010 at 7:54 AM, Ronny Pfannschmidt <
> > Ronny.Pfannschmidt at gmx.de> wrote:
> >
> > > On Fri, 2010-06-11 at 15:42 -0500, Charles Solar wrote:
> > > > I am attempting to use execnet to start remote python sessions for a
> > > > testing framework I am working on.  This framework will spawn a lot
> of
> > > > remote gateways since I want each test to be run on its own process,
> a
> > > > simpler and more clean approach imo.
> > > >
> > > you might want to take a look into py.test's forked test execution
> > > (the --boxed param) and the pytest-xdist plugin for inspiration
> > > (or even a reason to contribute to pytest instead of making a own
> > > framework >:-) )
> > >
> > > > Things work great for the first few tests but I have been noticing
> > > > that python interpreters are being left around.  Well not really left
> > > > around, but zombified and left for the OS to clean up.  This is
> > > > undesirable since in the course of several seconds this app could
> > > > potentially spawn 20 or so python interpreters, each one only 'alive'
> > > > for about half a second.  I cannot wait around for the OS to kill the
> > > > zombies.
> > > >
> > > > Is there someway I can remotely terminate a python interpreter
> through
> > > > execnet?
> > >
> > > afair gateway.close() will clean up and should be used to ensure
> propper
> > >  cleanup
> > >
> > > >
> > > > Currently, when the remote interpreter finishes his test I delete the
> > > > gateway and channel instances from the host and I was hoping that
> > > > would clean up the python instances on the remote server.  But that
> > > > does not seem to be the case.
> > > >
> > > > Thanks for any guidance.
> > >
> > > -- ronny
> > >
> > >
>
> > _______________________________________________
> > execnet-dev mailing list
> > execnet-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/execnet-dev
>
>
> --
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/execnet-dev/attachments/20100615/16eea4d6/attachment.html>