deprecate commands.getstatus()
I'd like to deprecate commands.getstatus() in 2.6. Reason: there is getoutput() and getstatusoutput(). In the latter, "status" means the exit code. getstatus(), however, returns the output of "ls -ld <file>" which is completely nonobvious and confusing. Perhaps the whole commands module can be deprecated in favor of subprocess. Georg
On 3/13/07, Georg Brandl <g.brandl@gmx.net> wrote:
I'd like to deprecate commands.getstatus() in 2.6.
Reason: there is getoutput() and getstatusoutput(). In the latter, "status" means the exit code. getstatus(), however, returns the output of "ls -ld <file>" which is completely nonobvious and confusing.
+1.
Perhaps the whole commands module can be deprecated in favor of subprocess.
-1. Reason (for both voteS): I have learned that the getoutput() and getstatusoutput() functions in the commands module are exceedingly popular amongst Googlers who write simple Python scripts that occasionally invoke an external command. It's much simpler than using os.popen() or the subprocess module (which is also very popular BTW). But I have found no uses of commands.getstatus() in our entire codebase so I'm okay with seeing that mistake finally eradicated (and it was *my* mistake :-). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum schrieb:
On 3/13/07, Georg Brandl <g.brandl@gmx.net> wrote:
I'd like to deprecate commands.getstatus() in 2.6.
Reason: there is getoutput() and getstatusoutput(). In the latter, "status" means the exit code. getstatus(), however, returns the output of "ls -ld <file>" which is completely nonobvious and confusing.
+1.
Done in rev. 54361. Georg
On Tue, Mar 13, 2007 at 02:16:21PM -0700, Guido van Rossum wrote: -> On 3/13/07, Georg Brandl <g.brandl@gmx.net> wrote: -> > I'd like to deprecate commands.getstatus() in 2.6. -> > -> > Reason: there is getoutput() and getstatusoutput(). In the latter, "status" -> > means the exit code. getstatus(), however, returns the output of "ls -ld -> > <file>" which is completely nonobvious and confusing. -> -> +1. -> -> > Perhaps the whole commands module can be deprecated in favor of subprocess. -> -> -1. -> -> Reason (for both voteS): I have learned that the getoutput() and -> getstatusoutput() functions in the commands module are exceedingly -> popular amongst Googlers who write simple Python scripts that -> occasionally invoke an external command. It's much simpler than using -> os.popen() or the subprocess module (which is also very popular BTW). What about reimplementing commands.* using subprocess? Or providing a commands.*-compatible interface in the subprocess module? cheers, --titus
On 3/13/07, Titus Brown <titus@caltech.edu> wrote:
What about reimplementing commands.* using subprocess? Or providing a commands.*-compatible interface in the subprocess module?
What does that buy us? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tue, Mar 13, 2007 at 04:55:07PM -0700, Guido van Rossum wrote: -> On 3/13/07, Titus Brown <titus@caltech.edu> wrote: -> > What about reimplementing commands.* using subprocess? Or providing a -> > commands.*-compatible interface in the subprocess module? -> -> What does that buy us? The simplicity of the commands interface on top of the more functional subprocess interface, no? subprocess is very powerful but there isn't a simple way to get the output. See http://docs.python.org/lib/node530.html, and see the docs comparing os.system() with subprocess, http://docs.python.org/lib/node537.html So, if you added 'getstatusoutput' and 'getoutput'-style commands to the subprocess module, you would (a) be able to deprecate a module in the stdlib, simplifying it a bit, and (b) provide simple commands implementing a common use case for subprocess, "run this command and give me the output". (You can already do 'getstatus' with a 'Popen(cmd).wait()'.) cheers, --titus
On Tue, Mar 13, 2007 at 04:55:07PM -0700, Guido van Rossum wrote:
What does that buy us?
subprocess offers better error-trapping, I think, and additional features such as closing all file descriptors after forking. At work, I found this fixed a number of bugs in our daemons code because if you were starting something long-running, it might keep a socket open and cause problems later. --amk
Alright already! It wasn't a thetorical question, and I'm convinced! :-) On 3/14/07, Fredrik Lundh <fredrik@pythonware.com> wrote:
Guido van Rossum wrote:
What about reimplementing commands.* using subprocess? Or providing a commands.*-compatible interface in the subprocess module?
What does that buy us?
multi-platform support? (commands is Unixoid only, right?)
</F>
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
-> > >> What about reimplementing commands.* using subprocess? Or providing a -> > >> commands.*-compatible interface in the subprocess module? OK, so as I understand it, the next step would be for me to provide a patch implementing this, right? Or is this PEP-required (please no...)? What do people think of this basic interface? (status, output) = subprocess.get_status_output(cmd) output = subprocess.get_output(cmd) Here 'status' is the 'returncode' from subprocess.Popen, and 'output' would be the combined stdout/stderr. 'commands.getstatus' would be removed entirely [0]. This mimics 'commands' fairly closely, while adhering to PEP 8 guidelines; it's a simple API; and it should be dead easy to implement. It will also have the various advantages people have mentioned: * better error trapping; * better post-fork behavior; * multi-platform support; If this sort of thing goes in, I guess commands.* would then be deprecated with a note saying "go look at these similar commands in subprocess", right? An additional function that I would personally like is: (status, output, errout) = subprocess.get_status_output_err(cmd) although the name is hideous. I'd like to change 'get_status_output' to return a triple, but I'm not sure how big a change that would be relative to the expected behavior from the ancestral commands function. cheers, --titus [0] As per GvR, http://mail.python.org/pipermail/python-dev/2007-March/071926.html
On 3/16/07, Titus Brown <titus@caltech.edu> wrote:
-> > >> What about reimplementing commands.* using subprocess? Or providing a -> > >> commands.*-compatible interface in the subprocess module?
OK, so as I understand it, the next step would be for me to provide a patch implementing this, right? Or is this PEP-required (please no...)?
No pep.
What do people think of this basic interface?
(status, output) = subprocess.get_status_output(cmd)
output = subprocess.get_output(cmd)
Here 'status' is the 'returncode' from subprocess.Popen, and 'output' would be the combined stdout/stderr. 'commands.getstatus' would be removed entirely [0].
This mimics 'commands' fairly closely, while adhering to PEP 8 guidelines; it's a simple API; and it should be dead easy to implement.
Right. Does it also match the style of the API provided by the subprocess module?
It will also have the various advantages people have mentioned:
* better error trapping; * better post-fork behavior; * multi-platform support;
If this sort of thing goes in, I guess commands.* would then be deprecated with a note saying "go look at these similar commands in subprocess", right?
Yes. Another intermediate step might be to rewrite the commands module to call the new APIs in the subprocess module.
An additional function that I would personally like is:
(status, output, errout) = subprocess.get_status_output_err(cmd)
although the name is hideous. I'd like to change 'get_status_output' to return a triple, but I'm not sure how big a change that would be relative to the expected behavior from the ancestral commands function.
Call it get_status_output_errors and you have my blessings. If at all possible, get_status_output should *not* just concatenate the output and errors from this API, but attempt to really *merge* the stdout and stderr stream so that if they are interleaved they come out in the right order. That's what the old getstatusoutput() did by putting '2>&1' around the command. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Fri, Mar 16, 2007 at 09:50:29AM -0700, Guido van Rossum wrote: -> > What do people think of this basic interface? -> > -> > (status, output) = subprocess.get_status_output(cmd) -> > -> > output = subprocess.get_output(cmd) -> > -> > Here 'status' is the 'returncode' from subprocess.Popen, and 'output' -> > would be the combined stdout/stderr. 'commands.getstatus' would be -> > removed entirely [0]. -> > -> > This mimics 'commands' fairly closely, while adhering to PEP 8 -> > guidelines; it's a simple API; and it should be dead easy to implement. -> -> Right. Does it also match the style of the API provided by the -> subprocess module? Yes; subprocess has only two "convenience" functions apart from the main class, in fact, and their names are quite short and simple. -> > It will also have the various advantages people have mentioned: -> > -> > * better error trapping; -> > * better post-fork behavior; -> > * multi-platform support; -> > -> > If this sort of thing goes in, I guess commands.* would then be -> > deprecated with a note saying "go look at these similar commands in -> > subprocess", right? -> -> Yes. Another intermediate step might be to rewrite the commands module -> to call the new APIs in the subprocess module. My preference would be to push people towards subprocess, and the more capable & complex API there, by deprecating commands completely. I'm happy to do it either way. -> > An additional function that I would personally like is: -> > -> > (status, output, errout) = subprocess.get_status_output_err(cmd) -> > -> > although the name is hideous. I'd like to change 'get_status_output' -> > to return a triple, but I'm not sure how big a change that would be -> > relative to the expected behavior from the ancestral commands function. -> -> Call it get_status_output_errors and you have my blessings. If at all -> possible, get_status_output should *not* just concatenate the output -> and errors from this API, but attempt to really *merge* the stdout and -> stderr stream so that if they are interleaved they come out in the -> right order. That's what the old getstatusoutput() did by putting -> '2>&1' around the command. I believe that this is what subprocess.Popen(..., stdout=subprocess.PIPE, stderr=subprocess.STDOUT) does, but I will check to be sure. --titus
On 3/16/07, Titus Brown <titus@caltech.edu> wrote:
-> Yes. Another intermediate step might be to rewrite the commands module -> to call the new APIs in the subprocess module.
My preference would be to push people towards subprocess, and the more capable & complex API there, by deprecating commands completely. I'm happy to do it either way.
Yeah, but the deprecation rules require that the commands module has to stay around for at least one release; the quickest we can go is to deprecate it in 2.6 and remove it in 2.7. However, it would be fine to reimplement it in 2.6 using calls to subprocess. Anyway, don't sweat it. Either way will be fine. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Hi all, I posted about adding 'get_output', 'get_status_output', and 'get_status_output_errors' to subprocess here, http://ivory.idyll.org/blog/mar-07/replacing-commands-with-subprocess and got some interesting responses. Briefly, my original proposal was to add these three functions: output = get_output(cmd, input=None, cwd=None, env=None) (status, output) = get_status_output(cmd, input=None, cwd=None, env=None) (status, output, errout) = get_status_output_errors(cmd, input=None, cwd=None, env=None) Commenters convinced me to propose a few additions. In order of estimated plausibility, * first, all sensical keyword args to subprocess.Popen (everything but universal_newlines, stdout, stderr, and bufsize, which don't make much sense in the context of the proposed functions) should be accepted by the 'get_' functions. This complicates the function signatures but does make them potentially much more useful. * second, I'd like to add a 'require_success' bool keyword, that is by default False (and does nothing in that case). However, when True, the functions would emulate check_call, i.e. they would raise CalledProcessError when the returncode was not zero. * third, the 'popen2' module should be deprecated for 2.6. I don't see that it has anything in it that subprocess doesn't have. Thoughts? --titus p.s. This has been a fun learning process... ;)
Sounds good to me. In 3.0 we should probably not have os.popen*(), nor the popen2 module at all, and do everything via the subprocess module. I wonder if we should even get rid of os.system(); then there should be a subprocess.system() instead. And do we even need os.fork(), os.exec*(), os.spawn*()? On 3/22/07, Titus Brown <titus@caltech.edu> wrote:
Hi all,
I posted about adding 'get_output', 'get_status_output', and 'get_status_output_errors' to subprocess here,
http://ivory.idyll.org/blog/mar-07/replacing-commands-with-subprocess
and got some interesting responses.
Briefly, my original proposal was to add these three functions:
output = get_output(cmd, input=None, cwd=None, env=None)
(status, output) = get_status_output(cmd, input=None, cwd=None, env=None) (status, output, errout) = get_status_output_errors(cmd, input=None, cwd=None, env=None)
Commenters convinced me to propose a few additions. In order of estimated plausibility,
* first, all sensical keyword args to subprocess.Popen (everything but universal_newlines, stdout, stderr, and bufsize, which don't make much sense in the context of the proposed functions) should be accepted by the 'get_' functions.
This complicates the function signatures but does make them potentially much more useful.
* second, I'd like to add a 'require_success' bool keyword, that is by default False (and does nothing in that case). However, when True, the functions would emulate check_call, i.e. they would raise CalledProcessError when the returncode was not zero.
* third, the 'popen2' module should be deprecated for 2.6. I don't see that it has anything in it that subprocess doesn't have.
Thoughts?
--titus
p.s. This has been a fun learning process... ;) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum schrieb:
Sounds good to me. In 3.0 we should probably not have os.popen*(), nor the popen2 module at all, and do everything via the subprocess module. I wonder if we should even get rid of os.system(); then there should be a subprocess.system() instead. And do we even need os.fork(), os.exec*(), os.spawn*()?
I don't know about about *os*.fork; I surely like to have posix.fork. The posix module exposes many OS functions as-is. This has the advantage that their semantics are crystal-clear: they do whatever the system call does (which, ideally, is what POSIX specifies for it). So you can do systems programming in Python, and only need good knowledge of the underlying system calls (i.e. using Python as a better C). For the subsystem module, the semantics is not so clear, in border cases. Regards, Martin
"\"Martin v. Löwis\"" <martin@v.loewis.de> wrote:
And do we even need os.fork(), os.exec*(), os.spawn*()?
I don't know about about *os*.fork; I surely like to have posix.fork. The posix module exposes many OS functions as-is. This has the advantage that their semantics are crystal-clear: they do whatever the system call does (which, ideally, is what POSIX specifies for it). So you can do systems programming in Python, and only need good knowledge of the underlying system calls (i.e. using Python as a better C).
I definitely agree. Removing the POSIX system call mappings would make Python less useful and general-purpose. Yes it's nice to have high-level utility functions like those in the subprocess module, but I think it's very important for the low-level functions to be there too when you need them.
On 3/22/07, Jon Ribbens <jon+python-dev@unequivocal.co.uk> wrote:
"\"Martin v. Löwis\"" <martin@v.loewis.de> wrote:
And do we even need os.fork(), os.exec*(), os.spawn*()?
I don't know about about *os*.fork; I surely like to have posix.fork. The posix module exposes many OS functions as-is. This has the advantage that their semantics are crystal-clear: they do whatever the system call does (which, ideally, is what POSIX specifies for it). So you can do systems programming in Python, and only need good knowledge of the underlying system calls (i.e. using Python as a better C).
I definitely agree. Removing the POSIX system call mappings would make Python less useful and general-purpose.
Yes it's nice to have high-level utility functions like those in the subprocess module, but I think it's very important for the low-level functions to be there too when you need them.
Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(), that abomination invented by Microsoft? I also hear no opposition against killign os.system() and os.popen(). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
On 3/22/07, Jon Ribbens <jon+python-dev@unequivocal.co.uk> wrote:
"\"Martin v. Löwis\"" <martin@v.loewis.de> wrote:
And do we even need os.fork(), os.exec*(), os.spawn*()?
I don't know about about *os*.fork; I surely like to have posix.fork. The posix module exposes many OS functions as-is. This has the advantage that their semantics are crystal-clear: they do whatever the system call does (which, ideally, is what POSIX specifies for it). So you can do systems programming in Python, and only need good knowledge of the underlying system calls (i.e. using Python as a better C).
I definitely agree. Removing the POSIX system call mappings would make Python less useful and general-purpose.
Yes it's nice to have high-level utility functions like those in the subprocess module, but I think it's very important for the low-level functions to be there too when you need them.
Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(), that abomination invented by Microsoft? I also hear no opposition against killign os.system() and os.popen()
Except that 'os.system' is really easy to use and I use it rarely enough that I *always* have to RTFM for subprocess which makes you jump through a few more (albeit simple) hoops. Additionally, AFAIK subprocess is still broken for py2exe'd applications which is a problem. All the best, Michael Foord
On Thu, Mar 22, 2007 at 09:34:46PM +0000, Michael Foord wrote: -> Guido van Rossum wrote: -> > Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(), -> > that abomination invented by Microsoft? I also hear no opposition -> > against killign os.system() and os.popen() -> -> Except that 'os.system' is really easy to use and I use it rarely enough -> that I *always* have to RTFM for subprocess which makes you jump through -> a few more (albeit simple) hoops. Hopefully the patch I'm making will change that, no? I could add in a 'system'-alike call easily enough; that was suggested. But I think returncode = subprocess.call("program") is pretty simple, isn't it? -> Additionally, AFAIK subprocess is still broken for py2exe'd applications -> which is a problem. Explain? cheers, --titus
Titus Brown wrote:
On Thu, Mar 22, 2007 at 09:34:46PM +0000, Michael Foord wrote: -> Guido van Rossum wrote: -> > Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(), -> > that abomination invented by Microsoft? I also hear no opposition -> > against killign os.system() and os.popen() -> -> Except that 'os.system' is really easy to use and I use it rarely enough -> that I *always* have to RTFM for subprocess which makes you jump through -> a few more (albeit simple) hoops.
Hopefully the patch I'm making will change that, no?
I could add in a 'system'-alike call easily enough; that was suggested. But I think
returncode = subprocess.call("program")
is pretty simple, isn't it?
Probably. I can just never remember it (I didn't remember it when typing that email). My fault, os.system is just easier to remember - make an os system call. :-)
-> Additionally, AFAIK subprocess is still broken for py2exe'd applications -> which is a problem.
Explain?
Programs created with py2exe (or frozen in other ways I believe - including the Wing IDE for example), fail with an error similar to the following : Traceback (most recent call last): File "main.py", line 133, in <module> main() File "main.py", line 125, in main launch_launcher() File "main.py", line 98, in launch_launcher subprocess.Popen([path], stderr = childstderr) File "subprocess.pyc", line 586, in ___init___ File "subprocess.pyc", line 681, in _get_handles File "subprocess.pyc", line 722, in _make_inheritable TypeError: an integer is required The problem is detailed here : http://www.py2exe.org/index.cgi/Py2ExeSubprocessInteractions "if you used py2exe to create a Windows program as opposed to a console program, and you launched the py2exe program by clicking rather than by running if from a command window, then the parent process has no handle to inherit. subprocess.Popen will throw an exception (TypeError: an integer is required) because GetStdHandle returns None" (Although the workaround as listed there is reported not to work and I haven't put the effort into experimenting.) If subprocess is supposed to be the main way that users launch sub-processes it would be nice if it worked for py2exe without the users (who are often newbies) having to put a workaround in place. Currently of course 'os.system' is often sufficient workaround. :-) All the best, Michael Foord
cheers, --titus
Titus Brown wrote:
I could add in a 'system'-alike call easily enough; that was suggested. But I think
returncode = subprocess.call("program")
is pretty simple, isn't it?
Something to keep in mind is that system() doesn't directly launch a process running the command, it uses a shell. So it's not just simple sugar for some subprocess.* call. -- Greg
On 3/22/07, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Titus Brown wrote:
I could add in a 'system'-alike call easily enough; that was suggested. But I think
returncode = subprocess.call("program")
is pretty simple, isn't it?
Something to keep in mind is that system() doesn't directly launch a process running the command, it uses a shell. So it's not just simple sugar for some subprocess.* call.
subprocess.call("ls | grep tmp", shell=True) svn-commit.2.tmp svn-commit.tmp
The more important difference is the encoding of the return value: system() has magic to encode signal-related termination of the child process. -Mike
On 3/22/07, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Guido van Rossum wrote:
Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(), that abomination invented by Microsoft? I also hear no opposition against killign os.system() and os.popen()
Except that 'os.system' is really easy to use and I use it rarely enough that I *always* have to RTFM for subprocess which makes you jump through a few more (albeit simple) hoops.
So let's add subprocess.system() which takes care of the hoops (but still allows you more flexibility through optional keyword parameters).
Additionally, AFAIK subprocess is still broken for py2exe'd applications which is a problem.
That doesn't sound like an API design concern -- it's a bug that just needs to be fixed somewhere. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Thu, Mar 22, 2007 at 02:47:58PM -0700, Guido van Rossum wrote: -> On 3/22/07, Michael Foord <fuzzyman@voidspace.org.uk> wrote: -> > Guido van Rossum wrote: -> > > Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(), -> > > that abomination invented by Microsoft? I also hear no opposition -> > > against killign os.system() and os.popen() -> > -> > Except that 'os.system' is really easy to use and I use it rarely enough -> > that I *always* have to RTFM for subprocess which makes you jump through -> > a few more (albeit simple) hoops. -> -> So let's add subprocess.system() which takes care of the hoops (but -> still allows you more flexibility through optional keyword -> parameters). How would this differ from subprocess.call()? http://docs.python.org/lib/node530.html (And should I add this in my patch, or is this for Py3k?) --titus
* Titus Brown wrote:
On Thu, Mar 22, 2007 at 02:47:58PM -0700, Guido van Rossum wrote: -> On 3/22/07, Michael Foord <fuzzyman@voidspace.org.uk> wrote: -> > Guido van Rossum wrote: -> > > Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(), -> > > that abomination invented by Microsoft? I also hear no opposition -> > > against killign os.system() and os.popen() -> > -> > Except that 'os.system' is really easy to use and I use it rarely enough -> > that I *always* have to RTFM for subprocess which makes you jump through -> > a few more (albeit simple) hoops. -> -> So let's add subprocess.system() which takes care of the hoops (but -> still allows you more flexibility through optional keyword -> parameters).
How would this differ from subprocess.call()?
It doesn't implement the system() spec: <http://opengroup.org/onlinepubs/007908799/xsh/system.html> nd -- Winnetous Erbe: <http://pub.perlig.de/books.html#apache2>
On Thu, Mar 22, 2007 at 11:12:26PM +0100, Andr? Malo wrote: -> * Titus Brown wrote: -> -> > On Thu, Mar 22, 2007 at 02:47:58PM -0700, Guido van Rossum wrote: -> > -> On 3/22/07, Michael Foord <fuzzyman@voidspace.org.uk> wrote: -> > -> > Guido van Rossum wrote: -> > -> > > Sure. os.fork() and the os.exec*() family can stay. But -> > os.spawn*(), -> > > that abomination invented by Microsoft? I also hear -> > no opposition -> > > against killign os.system() and os.popen() -> > -> > -> > -> > Except that 'os.system' is really easy to use and I use it rarely -> > enough -> > that I *always* have to RTFM for subprocess which makes you -> > jump through -> > a few more (albeit simple) hoops. -> > -> -> > -> So let's add subprocess.system() which takes care of the hoops (but -> > -> still allows you more flexibility through optional keyword -> > -> parameters). -> > -> > How would this differ from subprocess.call()? -> > -> > http://docs.python.org/lib/node530.html -> -> It doesn't implement the system() spec: -> <http://opengroup.org/onlinepubs/007908799/xsh/system.html> -> -> nd OK, but I'm still confused. This isn't about moving os.system into subprocess, it's about reimplementing os.system *with* subprocess.Popen, right? And how would that be substantially different from call()? Different defaults? (like shell=True, close_fds=False?) --titus
Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(), that abomination invented by Microsoft?
Right, I personally would not miss it. It's also not a system call, but a library function on both Windows and Unix (the equivalent of exposing fork would be to expose CreateProcessEx - something that I think Python should do out of the box, and not just when PythonWin is installed - but you can now get it through ctypes).
I also hear no opposition against killign os.system() and os.popen().
Both are library functions; I can implement them in Python on top of what is there (plus popen is based on stdio, which we declared evil). So yes, the can go. Regards, Martin
On 22 Mar, 09:37 pm, martin@v.loewis.de wrote:
Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(), that abomination invented by Microsoft?
Right, I personally would not miss it. It's also not a system call, but a library function on both Windows and Unix (the equivalent of exposing fork would be to expose CreateProcessEx - something that I think Python should do out of the box, and not just when PythonWin is installed - but you can now get it through ctypes).
I also hear no opposition against killign os.system() and os.popen().
Both are library functions; I can implement them in Python on top of what is there (plus popen is based on stdio, which we declared evil). So yes, the can go.
In the long term (read: 3k) I think I agree completely. It seems that this is a clear-cut case of TATMWTDI ("there are too many ways to do it") and the subprocess module should satisfy all of these use-cases. I also like Martin's earlier suggestion of calling the remaining OS process-manipulation functions "posix.fork", etc. I think it would be a lot clearer to read and maintain the implementation of subprocess (and asynchronous equivalents, like Twisted's process support) if the platform back-ends were explicitly using APIs in platform-specific modules. The current Twisted implementation misleadingly looks like the UNIX implementation is cross-platform because it uses functions in the "os" module, whereas the Windows implementation uses win32all.
"Guido van Rossum" <guido@python.org> wrote:
Sure. os.fork() and the os.exec*() family can stay. But os.spawn*(), that abomination invented by Microsoft? I also hear no opposition against killign os.system() and os.popen().
As long as os.system migrates to subprocess.system (as you originally suggested), I'm +1 with everything mentioned in this thread. - Josiah
On 3/22/07, Guido van Rossum <guido@python.org> wrote:
Sounds good to me. In 3.0 we should probably not have os.popen*(), nor the popen2 module at all, and do everything via the subprocess module. I wonder if we should even get rid of os.system(); then there should be a subprocess.system() instead. And do we even need os.fork(), os.exec*(), os.spawn*()?
popen2 is already slated for removal in PEP 3108 based on subprocess making is obsolete (as is commands). -Brett
On 3/22/07, Titus Brown <titus@caltech.edu> wrote:
Hi all,
I posted about adding 'get_output', 'get_status_output', and 'get_status_output_errors' to subprocess here,
http://ivory.idyll.org/blog/mar-07/replacing-commands-with-subprocess
and got some interesting responses.
Briefly, my original proposal was to add these three functions:
output = get_output(cmd, input=None, cwd=None, env=None)
(status, output) = get_status_output(cmd, input=None, cwd=None, env=None) (status, output, errout) = get_status_output_errors(cmd, input=None, cwd=None, env=None)
Commenters convinced me to propose a few additions. In order of estimated plausibility,
* first, all sensical keyword args to subprocess.Popen (everything but universal_newlines, stdout, stderr, and bufsize, which don't make much sense in the context of the proposed functions) should be accepted by the 'get_' functions.
This complicates the function signatures but does make them potentially much more useful.
* second, I'd like to add a 'require_success' bool keyword, that is by default False (and does nothing in that case). However, when True, the functions would emulate check_call, i.e. they would raise CalledProcessError when the returncode was not zero.
* third, the 'popen2' module should be deprecated for 2.6. I don't see that it has anything in it that subprocess doesn't have.
Thoughts?
--titus
p.s. This has been a fun learning process... ;) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
Guido van Rossum wrote:
I wonder if we should even get rid of os.system(); then there should be a subprocess.system() instead.
That sounds okay, since system() isn't actually a system call, despite its name.
And do we even need os.fork(), os.exec*(), os.spawn*()?
Since fork() and exec() are directly wrapping actual system calls, I think they should stay in os. I'm not sure about spawn() -- on Unix it's not a direct system call, but it might be on Windows. -- Greg
On 22 Mar, 08:38 pm, guido@python.org wrote:
And do we even need os.fork(), os.exec*(), os.spawn*()?
Maybe not os.spawn*, but Twisted's spawnProcess is implemented (on UNIX) in terms of fork/exec and I believe it should stay that way. The subprocess module isn't really usable for asynchronous subprocess communication.
On Thu, 2007-03-22 at 13:38 -0700, Guido van Rossum wrote:
Sounds good to me. In 3.0 we should probably not have os.popen*(), nor the popen2 module at all, and do everything via the subprocess module. I wonder if we should even get rid of os.system(); then there should be a subprocess.system() instead. And do we even need os.fork(), os.exec*(), os.spawn*()?
Please don't remove os.fork and os.exec*. Some people need to fine-tune process creation and don't need portability to non-Unix OS'es. For them, the functions that call the underlying system API and little or nothing else are a god-send.
On 3/23/07, Hrvoje Nikšić <hrvoje.niksic@avl.com> wrote:
On Thu, 2007-03-22 at 13:38 -0700, Guido van Rossum wrote:
Sounds good to me. In 3.0 we should probably not have os.popen*(), nor the popen2 module at all, and do everything via the subprocess module. I wonder if we should even get rid of os.system(); then there should be a subprocess.system() instead. And do we even need os.fork(), os.exec*(), os.spawn*()?
Please don't remove os.fork and os.exec*. Some people need to fine-tune process creation and don't need portability to non-Unix OS'es. For them, the functions that call the underlying system API and little or nothing else are a god-send.
Right, but if you're really using only Posix, you can simply use ``posix.fork`` and ``posix.exec*`` and then you're even being explicit about the fact. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On Fri, Mar 23, 2007 at 10:30:37AM -0600, Steven Bethard wrote: -> On 3/23/07, Hrvoje Nik??i?? <hrvoje.niksic@avl.com> wrote: -> > On Thu, 2007-03-22 at 13:38 -0700, Guido van Rossum wrote: -> > > Sounds good to me. In 3.0 we should probably not have os.popen*(), nor -> > > the popen2 module at all, and do everything via the subprocess module. -> > > I wonder if we should even get rid of os.system(); then there should -> > > be a subprocess.system() instead. And do we even need os.fork(), -> > > os.exec*(), os.spawn*()? -> > -> > Please don't remove os.fork and os.exec*. Some people need to fine-tune -> > process creation and don't need portability to non-Unix OS'es. For -> > them, the functions that call the underlying system API and little or -> > nothing else are a god-send. -> -> Right, but if you're really using only Posix, you can simply use -> ``posix.fork`` and ``posix.exec*`` and then you're even being explicit -> about the fact. Yes, but: http://docs.python.org/lib/module-posix.html """Do not import this module directly.""" Unless people want me to try to extract something coherent from the recent discussion, I'm going to avoid doing anything with os.* functions. That can be done separately from the contemplated subprocess patch, anyway. The whole thread is here: http://www.gossamer-threads.com/lists/engine?post=553519;list=python and I will finish up a patch to do this: http://www.gossamer-threads.com/lists/python/dev/555743#555743 (add get_*output* functions to subprocess, modify docs appropriately, add 'require_success', and put in a docs deprecation for popen2/commands). I'll post again when I have a patch ready so there's something concrete to complain about ;). thanks, --titus
Titus Brown wrote:
* second, I'd like to add a 'require_success' bool keyword, that is by default False (and does nothing in that case). However, when True, the functions would emulate check_call, i.e. they would raise CalledProcessError when the returncode was not zero.
By the no-constant-parameters principle, I think it would be better to have another set of functions for this. They should be named without 'status' in the names, since if the function returns at all, the status is always going to be zero. -- Greg
participants (16)
-
"Martin v. Löwis" -
A.M. Kuchling -
André Malo -
Brett Cannon -
Fredrik Lundh -
Georg Brandl -
glyph＠divmod.com -
Greg Ewing -
Guido van Rossum -
Hrvoje Nikšić -
Jon Ribbens -
Josiah Carlson -
Michael Foord -
Mike Klaas -
Steven Bethard -
Titus Brown