Re: [Python-checkins] python/dist/src/Lib subprocess.py, NONE, 1.1
effbot@users.sourceforge.net writes:
Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24287/Lib
Added Files: subprocess.py Log Message: Added Peter Astrand's subprocess module.
I suggest to remove the imports of the pywin32 stuff, it's not needed and confuses tools like freeze or py2exe. if mswindows: import threading import msvcrt try: from _subprocess import * class STARTUPINFO: dwFlags = 0 hStdInput = None hStdOutput = None hStdError = None class pywintypes: error = IOError except ImportError: import pywintypes from win32api import GetStdHandle, STD_INPUT_HANDLE, \ STD_OUTPUT_HANDLE, STD_ERROR_HANDLE from win32api import GetCurrentProcess, DuplicateHandle, \ GetModuleFileName, GetVersion from win32con import DUPLICATE_SAME_ACCESS from win32pipe import CreatePipe from win32process import CreateProcess, STARTUPINFO, \ GetExitCodeProcess, STARTF_USESTDHANDLES, \ CREATE_NEW_CONSOLE from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0
Thomas Heller wrote:
I suggest to remove the imports of the pywin32 stuff, it's not needed and confuses tools like freeze or py2exe.
That's really a problem with the tools, not with the module. Demanding that users of a dynamic language use only static linking is really silly. Wouldn't it be better to invent a (pragma-level) syntax so that a module can inform the tool about its intentions? (removing the pywin stuff from the module would hamper development of new features, and also make it harder to verify that the _subprocess module works as it should). </F>
"Fredrik Lundh" <fredrik@pythonware.com> writes:
Thomas Heller wrote:
I suggest to remove the imports of the pywin32 stuff, it's not needed and confuses tools like freeze or py2exe.
That's really a problem with the tools, not with the module.
Yep.
Demanding that users of a dynamic language use only static linking is really silly.
Wouldn't it be better to invent a (pragma-level) syntax so that a module can inform the tool about its intentions?
Yes. But I don't think I have the energy to discuss that here, and don't feel it would be easy to get this into the core.
(removing the pywin stuff from the module would hamper development of new features, and also make it harder to verify that the _subprocess module works as it should).
Hm, that what the tests are for, and not code that is never executed. A working solution would be to comment out this stuff, or put it into an 'if 0:' block, in both cases one could easily activate it again for experimentation. Thomas
Thomas Heller wrote:
(removing the pywin stuff from the module would hamper development of new features, and also make it harder to verify that the _subprocess module works as it should).
Hm, that what the tests are for, and not code that is never executed.
if you remove the _subprocess driver, it is executed. _subprocess is just a win32all emulator...
A working solution would be to comment out this stuff, or put it into an 'if 0:' block, in both cases one could easily activate it again for experimentation.
does py2exe support "if 0" blocks? </F>
"Fredrik Lundh" <fredrik@pythonware.com> writes:
Thomas Heller wrote:
(removing the pywin stuff from the module would hamper development of new features, and also make it harder to verify that the _subprocess module works as it should).
Hm, that what the tests are for, and not code that is never executed.
if you remove the _subprocess driver, it is executed. _subprocess is just a win32all emulator...
To be honest, I'm against code in the core that depends on whether pywin32 is installed or not.
A working solution would be to comment out this stuff, or put it into an 'if 0:' block, in both cases one could easily activate it again for experimentation.
does py2exe support "if 0" blocks?
py2exe uses modulefinder to scan the byte code of compiled modules, and Python doesn't generate code for 'if 0' blocks, so, yes. Thomas
Thomas Heller wrote:
To be honest, I'm against code in the core that depends on whether pywin32 is installed or not.
does py2exe support "if 0" blocks?
py2exe uses modulefinder to scan the byte code of compiled modules, and Python doesn't generate code for 'if 0' blocks, so, yes.
just one more question: is there a specific problem with win32all? if this is a general problem, how do you deal with other optional modules in the standard library (e.g. _xmlrpclib) ? </F>
"Fredrik Lundh" <fredrik@pythonware.com> writes:
Thomas Heller wrote:
To be honest, I'm against code in the core that depends on whether pywin32 is installed or not.
does py2exe support "if 0" blocks?
py2exe uses modulefinder to scan the byte code of compiled modules, and Python doesn't generate code for 'if 0' blocks, so, yes.
just one more question: is there a specific problem with win32all? if this is a general problem, how do you deal with other optional modules in the standard library (e.g. _xmlrpclib) ?
If _xmlrpclib is installed, py2exe will find it if the script uses xmlrpclib. I guess that is what the user expects. If _xmlrpclib is not installed, py2exe will warn that it is missing (this warning is probably not very useful, and it seems Bob has supressed this warning in py2app). The point with win32all (now pywin32) is that most windows users will have it installed, for good reasons. py2exe will find it, and include it if the subprocess module is pulled in - whether _subprocess is available or not. But subprocess, the Python 2.4 version, will *never* use it, because _subprocess is always available (isn't this now even a builtin module, in python24.dll?). Hope that answers your question, Thomas
On OSX 10.3.4, test_subprocess is failing intermittently when run from regrtest: test_subprocess this bit of output is from a test of stdout in a different process ... test test_subprocess failed -- Traceback (most recent call last): File "/Users/anthonybaxter/pyhead/src/Lib/test/test_subprocess.py", line 434, in test_close_fds self.assertEqual(p.stdout.read(), "3") AssertionError: '4' != '3' Running just the test by itself works. I'm guessing it could be part of the test suite leaking a FD or something? Anyone know of a tool on OSX that I can run to examine open FDs for a process? -- Anthony Baxter <anthony@interlink.com.au> It's never too late to have a happy childhood.
On Thu, 14 Oct 2004, Anthony Baxter wrote:
On OSX 10.3.4, test_subprocess is failing intermittently when run from regrtest:
test_subprocess this bit of output is from a test of stdout in a different process ... test test_subprocess failed -- Traceback (most recent call last): File "/Users/anthonybaxter/pyhead/src/Lib/test/test_subprocess.py", line 434, in test_close_fds self.assertEqual(p.stdout.read(), "3") AssertionError: '4' != '3'
Running just the test by itself works. I'm guessing it could be part of the test suite leaking a FD or something?
That shouldn't matter, since when you are using close_fds=True, the subprocess module will close all filedescriptors. For example, you can insert additional lines with os.pipe() at the beginning of test_close_fds, and the test (should) still succeed(s). /Peter Åstrand <astrand@lysator.liu.se>
On Oct 13, 2004, at 8:01 AM, Thomas Heller wrote:
"Fredrik Lundh" <fredrik@pythonware.com> writes:
Thomas Heller wrote:
To be honest, I'm against code in the core that depends on whether pywin32 is installed or not.
does py2exe support "if 0" blocks?
py2exe uses modulefinder to scan the byte code of compiled modules, and Python doesn't generate code for 'if 0' blocks, so, yes.
just one more question: is there a specific problem with win32all? if this is a general problem, how do you deal with other optional modules in the standard library (e.g. _xmlrpclib) ?
If _xmlrpclib is installed, py2exe will find it if the script uses xmlrpclib. I guess that is what the user expects. If _xmlrpclib is not installed, py2exe will warn that it is missing (this warning is probably not very useful, and it seems Bob has supressed this warning in py2app).
I have disabled all missing module warnings, because I've found that are more of a nuisance than anything else. However, I have made it possible to generate a GraphViz DOT graph of everything that did get included, which is useful when you are having problems with too many or too few dependencies being included. Unlike modulefinder, py2app.modulegraph keeps track of why modules have been included, so it's easy enough to say that "pydoc should not depend on Tkinter", and Tkinter won't be included unless something else needs it. For cases like xmlrpclib->_xmlrpclib or xml -> _xmlplus, with a semi-standalone build (depends on an existing Python installation), a dotted edge will be drawn between the module that uses xmlrpclib and _xmlrpclib. The graph output only includes modules that ended up in the application bundle, so the edge is dotted to represent that the dependency exists, but was indirect (since xmlrpclib would not have been shown). That said, if you really wanted to display missing modules, you could find them pretty easily from the ModuleGraph instance, just walk it and keep instances of MissingModule. -bob
[Thomas Heller] ...
But subprocess, the Python 2.4 version, will *never* use it, because _subprocess is always available (isn't this now even a builtin module, in python24.dll?).
Yes, it is. I haven't had time to read the PEP, but I assumed some "backward compatibility" constraint is at work here too, since, e.g., subprocess.py has try: False except NameError: False = 0 True = 1 and that doesn't make sense for 2.4 either.
On Wed, 2004-10-13 at 11:31, Tim Peters wrote:
Yes, it is. I haven't had time to read the PEP, but I assumed some "backward compatibility" constraint is at work here too, since, e.g., subprocess.py has
try: False except NameError: False = 0 True = 1
and that doesn't make sense for 2.4 either.
If so, then it should be documented in PEP 291, otherwise backward compatibility stuff is fair game for ripping out. -Barry
Yes, it is. I haven't had time to read the PEP, but I assumed some "backward compatibility" constraint is at work here too, since, e.g., subprocess.py has
If so, then it should be documented in PEP 291, otherwise backward compatibility stuff is fair game for ripping out.
Below is a trivial patch to PEP 291. I'm stepping forward as a maintainer, if noone else wants to do the job. diff -u -r1.11 pep-0291.txt --- pep-0291.txt 20 Aug 2004 07:32:06 -0000 1.11 +++ pep-0291.txt 13 Oct 2004 18:00:14 -0000 @@ -90,6 +90,7 @@ modulefinder Thomas Heller 2.2 Just van Rossum platform Marc-Andre Lemburg 1.5.2 + subprocess Peter Astrand 2.2 Tool Maintainer(s) Python Version /Peter Åstrand <astrand@lysator.liu.se>
On Wed, 13 Oct 2004, Tim Peters wrote:
But subprocess, the Python 2.4 version, will *never* use it, because _subprocess is always available (isn't this now even a builtin module, in python24.dll?).
Yes, it is. I haven't had time to read the PEP, but I assumed some "backward compatibility" constraint is at work here too, since, e.g., subprocess.py has
try: False
The PEP doesn't say anything about this, but my intention is that the module should work on Python 2.2 and newer. My employer really needs this. If the subprocess module in the Python CVS won't work with 2.2, I'm forced to keep a separate "fork". I would be happy if this could be avoided. /Peter Åstrand <astrand@lysator.liu.se>
On Wed, 2004-10-13 at 13:56, Peter Astrand wrote:
The PEP doesn't say anything about this, but my intention is that the module should work on Python 2.2 and newer. My employer really needs this. If the subprocess module in the Python CVS won't work with 2.2, I'm forced to keep a separate "fork". I would be happy if this could be avoided.
There's no reason to fork it. (Reasonable) backward compatibility in the 2.4 library can be supported as long as it's documented, which is now is. -Barry
participants (7)
-
Anthony Baxter -
Barry Warsaw -
Bob Ippolito -
Fredrik Lundh -
Peter Astrand -
Thomas Heller -
Tim Peters