On Wed, 28 Mar 2007 16:38:45 -0700, Brett Cannon <brett@python.org> wrote:

On 3/28/07, Facundo Batista <facundo@taniquetil.com.ar> wrote:

...

There's this bug (#451607) about the needing of tests for socket SSL...

Last interesting update in the tracker is five years ago, and since a lot of work has been done in test_socket_ssl.py (Brett, Neal, Tim, George Brandl).

Do you think is useful to leave this bug opened?

Having a bug left open because a module needs more test is not really needed. It's rather obvious when a module needs more tests. =)

I say close it. I just wish we had a more reliable web site to connect to for SSL tests.

How about something even better? Take a look at "openssl s_server". This is still a pretty terrible way to test the SSL functionality, but it's loads better than connecting to a site on the public internet. Jean-Paul

On Thu, 29 Mar 2007 00:22:23 +0000 (UTC), Facundo Batista <facundo@taniquetil.com.ar> wrote:

Jean-Paul Calderone wrote:

...

Take a look at "openssl s_server". This is still a pretty terrible way to test the SSL functionality, but it's loads better than connecting to a site on the public internet.

How would you deal with the deployment and maintenance of this server in all buildbot's machines?

Or we just can ask to see if we have the server available, and then run the tests if yes?

If the openssl binary is available, when the test starts, launch it in a child process, talk to it for the test, then kill it when the test is done. Jean-Paul

Jean-Paul Calderone wrote:

If the openssl binary is available, when the test starts, launch it in a child process, talk to it for the test, then kill it when the test is done.

Ok, I have a demo of this. Right now, I face this problem. I launch openssl through subprocess, but I do *not* find a way to tell him to quit serving, so all I can do is to kill the process (through the pid from the Popen object). The problem is that os.kill only works in Unix and Macintosh. So, there's a better way to do this? Or I shall check if I'm in one of those both platforms and only execute the tests there? Another question, this one more operative. For openssl I'll need both cert.pem and key.pem files. I have them, but in which directory I shall commit them? ./Lib/test? Or create another directory inside that? Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

...

The problem is that os.kill only works in Unix and Macintosh. So, there's a better way to do this? Or I shall check if I'm in one of those both platforms and only execute the tests there?

If you have a compilation of pywin32 (isn't it shipped by default in Python 2.5+?), you can kill the process with win32process.TerminateProcess() .

On Win32, you also have subprocess.TerminateProcess, if you have the subprocess module in the first place. Regards, Martin

Martin v. Löwis wrote:

On Win32, you also have subprocess.TerminateProcess, if you have the subprocess module in the first place.

The problem of TerminateProcess is that I need the handle of the process. I don't like the idea of rely on the private _handle and do: process = subprocess.Popen(...) ... subprocess.TerminateProcess(int(process._handle), -1) so, I'll end doing this: -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

[sorry if you see 2 copies of this] Facundo Batista wrote:

The problem of TerminateProcess is that I need the handle of the process.

I don't like the idea of rely on the private _handle and do:

process = subprocess.Popen(...) ... subprocess.TerminateProcess(int(process._handle), -1)

so, I'll end doing this:

process = subprocess.Popen(...) ... handle =ctypes.windll.kernel32.OpenProcess(1, False, process.pid) ctypes.windll.kernel32.TerminateProcess(handle, -1) ctypes.windll.kernel32.CloseHandle(handle)

Is this ok?

Would it not be better to put a platform-independent version of this into subprocess, so that this code doesn't have to be duplicated all over the place? Maybe a method on a Popen object called terminate()? That way you wouldn't even need ctypes, since you could look at the _handle without feeling bad about it! Eric.

Facundo Batista wrote:

Eric V. Smith wrote:

...

Would it not be better to put a platform-independent version of this into subprocess, so that this code doesn't have to be duplicated all over the place? Maybe a method on a Popen object called terminate()?

Yes. But I'm not up to that task. Really don't know how to kill processes in *all* the platforms where Python's subprocess is present.

I'd be willing to look at adding it, if the group thinks it's the right thing to do. Eric.

On 4/3/07, Christian Heimes <lists@cheimes.de> wrote:

...

I'd be willing to look at adding it, if the group thinks it's the right thing to do.

I like the idea and I'm proposing to add two more methods to subprocess Popen.

class Popen(...): ... def signal(self, signal): """Send a signal to the process (UNIX only)

signal is constant from the signal module """

def terminate(self, force=False): """Terminate the process

On UNIX terminate(False) is equivalent to signal(SIGTERM) and terminate(True) to signal(SIGKILL).

On Windows ... (does Windows support a forced terminate?) """

Another difference I believe is that TerminateProcess on Windows doesn't kill the tree of processes like kill would. It would be nice if Popen.terminate() did the same thing on both Unix and Windows. (I assume that would mean making *all* the appropriate TerminateProcess calls.) Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy

Steven Bethard wrote:

Another difference I believe is that TerminateProcess on Windows doesn't kill the tree of processes like kill would. It would be nice if Popen.terminate() did the same thing on both Unix and Windows. (I assume that would mean making *all* the appropriate TerminateProcess calls.)

Possibly that should be kept separate and handled via eventual handling for process jobs: http://msdn2.microsoft.com/en-us/library/ms684161.aspx

A job object allows groups of processes to be managed as a unit. Job objects are namable, securable, sharable objects that control attributes of the processes associated with them. Operations performed on the job object affect all processes associated with the job object.

... To terminate all processes currently associated with a job object, use the TerminateJobObject function.

Trent -- Trent Mick trentm at activestate.com

Christian Heimes wrote:

...

I'd be willing to look at adding it, if the group thinks it's the right thing to do.

I like the idea and I'm proposing to add two more methods to subprocess Popen.

class Popen(...): ... def signal(self, signal): """Send a signal to the process (UNIX only)

signal is constant from the signal module """

def terminate(self, force=False): """Terminate the process

On UNIX terminate(False) is equivalent to signal(SIGTERM) and terminate(True) to signal(SIGKILL).

On Windows ... (does Windows support a forced terminate?) """

Here is what my process.py [1] does for termination (not suggesting you follow the API here, just showing how it handles termination on Windows). Some of the Windows logic is borrowed from PyWin32's winprocess.py [2]

def kill(self, exitCode=0, gracePeriod=1.0, sig=None): """Kill process.

"exitCode" [deprecated, not supported] (Windows only) is the code the terminated process should exit with. "gracePeriod" (Windows only) is a number of seconds the process is allowed to shutdown with a WM_CLOSE signal before a hard terminate is called. "sig" (Unix only) is the signal to use to kill the process. Defaults to signal.SIGKILL. See os.kill() for more information.

Windows: Try for an orderly shutdown via WM_CLOSE. If still running after gracePeriod (1 sec. default), terminate. """ if sys.platform.startswith("win"): import win32gui # Send WM_CLOSE to windows in this process group. win32gui.EnumWindows(self._close_, 0)

# Send Ctrl-Break signal to all processes attached to this # console. This is supposed to trigger shutdown handlers in # each of the processes. try: win32api.GenerateConsoleCtrlEvent(CTRL_BREAK_EVENT, self._processId) except AttributeError: log.warn("The win32api module does not have "\ "GenerateConsoleCtrlEvent(). This may mean that "\ "parts of this process group have NOT been killed.") except win32api.error, ex: if ex.args[0] not in (6, 87): # Ignore the following: # api_error: (87, 'GenerateConsoleCtrlEvent', 'The parameter is incorrect.') # api_error: (6, 'GenerateConsoleCtrlEvent', 'The handle is invalid.') # Get error 6 if there is no console. raise

# Last resort: call TerminateProcess if it has not yet. retval = 0 try: self.wait(gracePeriod) except ProcessError, ex: log.info("[%s] Process.kill: calling TerminateProcess", id(self)) win32process.TerminateProcess(self._hProcess, -1) win32api.Sleep(100) # wait for resources to be released

else: if sig is None: sig = signal.SIGKILL try: os.kill(self._pid, sig) except OSError, ex: if ex.errno != 3: # Ignore: OSError: [Errno 3] No such process raise

Links: [1] http://trentm.com/projects/process/ [2] http://pywin32.cvs.sourceforge.net/pywin32/pywin32/win32/Demos/winprocess.py?revision=1.2&view=markup#l_121 -- Trent Mick trentm at activestate.com

I don't like the idea of rely on the private _handle and do:

process = subprocess.Popen(...) ... subprocess.TerminateProcess(int(process._handle), -1)

so, I'll end doing this:

process = subprocess.Popen(...) ... handle =ctypes.windll.kernel32.OpenProcess(1, False, process.pid) ctypes.windll.kernel32.TerminateProcess(handle, -1) ctypes.windll.kernel32.CloseHandle(handle)

Is this ok?

I don't like it. I would rather rely on the private _handle member. If that ever gets changed, the test fails. Regards, Martin

On 4/2/07, Facundo Batista <facundo@taniquetil.com.ar> wrote:

I launch openssl through subprocess, but I do *not* find a way to tell him to quit serving, so all I can do is to kill the process (through the pid from the Popen object).

The problem is that os.kill only works in Unix and Macintosh. So, there's a better way to do this? Or I shall check if I'm in one of those both platforms and only execute the tests there?

I just checked the man page for s_server (on linux) and it mentions something about exiting. "Certain single letter commands are also recognized which perform special operations: these are listed below. q end the current SSL connection but still accept new connections. Q end the current SSL connection and exit." Can a command "Q" be sent to the server once testing is complete? Thanks, Raghu.