[Python-checkins] python/nondist/peps pep-0000.txt, 1.291, 1.292 pep-0324.txt, 1.2, 1.3

goodger at users.sourceforge.net goodger at users.sourceforge.net
Fri Oct 8 15:03:25 CEST 2004


Update of /cvsroot/python/python/nondist/peps
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25124

Modified Files:
	pep-0000.txt pep-0324.txt 
Log Message:
update from Peter Astrand; title change

Index: pep-0000.txt
===================================================================
RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v
retrieving revision 1.291
retrieving revision 1.292
diff -u -d -r1.291 -r1.292
--- pep-0000.txt	7 Oct 2004 01:33:55 -0000	1.291
+++ pep-0000.txt	8 Oct 2004 13:03:22 -0000	1.292
@@ -116,7 +116,7 @@
  S   319  Python Synchronize/Asynchronize Block        Pelletier
  S   321  Date/Time Parsing and Formatting             Kuchling
  S   323  Copyable Iterators                           Martelli
- S   324  process - New POSIX process module           Astrand
+ S   324  subprocess - New process module              Astrand
  S   325  Resource-Release Support for Generators      Pedroni
  S   330  Python Bytecode Verification                 Pelletier
  S   331  Locale-Independent Float/String conversions  Reis
@@ -354,7 +354,7 @@
  S   321  Date/Time Parsing and Formatting             Kuchling
  SF  322  Reverse Iteration                            Hettinger
  S   323  Copyable Iterators                           Martelli
- S   324  process - New POSIX process module           Astrand
+ S   324  subprocess - New POSIX process module        Astrand
  S   325  Resource-Release Support for Generators      Pedroni
  SR  326  A Case for Top and Bottom Values             Carlson, Reedy
  SF  327  Decimal Data Type                            Batista

Index: pep-0324.txt
===================================================================
RCS file: /cvsroot/python/python/nondist/peps/pep-0324.txt,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -d -r1.2 -r1.3
--- pep-0324.txt	3 Aug 2004 13:13:43 -0000	1.2
+++ pep-0324.txt	8 Oct 2004 13:03:22 -0000	1.3
@@ -1,5 +1,5 @@
 PEP: 324
-Title: process - New POSIX process module
+Title: subprocess - New process module
 Version: $Revision$
 Last-Modified: $Date$
 Author: Peter Astrand <astrand at lysator.liu.se>
@@ -13,7 +13,7 @@
 Abstract
 
     This PEP describes a new module for starting and communicating
-    with processes on POSIX systems.
+    with processes.
 
 
 Motivation
@@ -31,9 +31,9 @@
       over-complicated shell scripts.
 
     Currently, Python has a large number of different functions for
-    process creation. This makes it hard for developers to choose.
+    process creation.  This makes it hard for developers to choose.
 
-    The process module provides the following enhancements over
+    The subprocess module provides the following enhancements over
     previous functions:
 
     - One "unified" module provides all functionality from previous
@@ -56,7 +56,7 @@
       and redirect stderr, but not stdout.  This is not possible with
       current functions, without using temporary files.
 
-    - With the process module, it's possible to control if all open
+    - With the subprocess module, it's possible to control if all open
       file descriptors should be closed before the new program is
       executed.
 
@@ -78,22 +78,22 @@
 
     The following points summarizes the design:
 
-    - process was based on popen2, which is tried-and-tested.
+    - subprocess was based on popen2, which is tried-and-tested.
 
     - The factory functions in popen2 have been removed, because I
       consider the class constructor equally easy to work with.
 
     - popen2 contains several factory functions and classes for
-      different combinations of redirection.  process, however,
-      contains one single class.  Since the process module supports 12
-      different combinations of redirection, providing a class or
+      different combinations of redirection.  subprocess, however,
+      contains one single class.  Since the subprocess module supports
+      12 different combinations of redirection, providing a class or
       function for each of them would be cumbersome and not very
       intuitive.  Even with popen2, this is a readability problem.
       For example, many people cannot tell the difference between
       popen2.popen2 and popen2.popen4 without using the documentation.
 
-    - Two small utility functions are provided: process.call() and
-      process.callv().  These aims to be an enhancement over
+    - Two small utility functions are provided: subprocess.call() and
+      subprocess.callv().  These aims to be an enhancement over
       os.system(), while still very easy to use:
 
         - It does not use the Standard C function system(), which has
@@ -106,8 +106,8 @@
         - The return value is easier to work with.
 
       The call() utility function accepts an 'args' argument, just
-      like the Popen class constructor. It waits for the command to
-      complete, then returns the returncode attribute. The
+      like the Popen class constructor.  It waits for the command to
+      complete, then returns the returncode attribute.  The
       implementation is very simple:
 
       def call(*args, **kwargs):
@@ -117,30 +117,30 @@
       process and wait for it to finish is a common task.
 
       The callv() function is identical to call(), except that each
-      non-keyword argument is treated as a program argument. This
-      gives a slightly nicer syntax. The drawback is that callv() does
-      not allow specifying the program and it's arguments as a
+      non-keyword argument is treated as a program argument.  This
+      gives a slightly nicer syntax.  The drawback is that callv()
+      does not allow specifying the program and it's arguments as a
       whitespace-separated string: The entire (first) string would be
-      intepreted as the executable. The implementation of callv() is
+      interpreted as the executable.  The implementation of callv() is
       also very simple:
 
       def callv(*args, **kwargs):
           return Popen(args, **kwargs).wait()
 
       While Popen supports a wide range of options, many users have
-      simple needs. Many people are using os.system() today, mainly
-      because it provides a simple interface. Consider this example:
+      simple needs.  Many people are using os.system() today, mainly
+      because it provides a simple interface.  Consider this example:
 
           os.system("stty sane -F " + device)
 
-      With process.call(), this would look like:
+      With subprocess.call(), this would look like:
 
-          process.call(["stty", "sane", "-F", device])
+          subprocess.call(["stty", "sane", "-F", device])
 
-      Some people feel that the list brackets are clumsy. With
+      Some people feel that the list brackets are clumsy.  With
       callv(), they are not needed:
 
-          process.callv("stty", "sane", "-F", device)
+          subprocess.callv("stty", "sane", "-F", device)
 
     - The "preexec" functionality makes it possible to run arbitrary
       code between fork and exec.  One might ask why there are special
@@ -156,6 +156,13 @@
         - env and cwd are considered quite cross-platform: They make
           sense even on Windows.
 
+     - On POSIX platforms, no extension module is required: the module
+       uses os.fork(), os.execvp() etc.
+
+     - On Windows platforms, the module requires either Mark Hammond's
+       Windows extensions[5], or a small extension module called
+       _subprocess.
+
 
 Specification
 
@@ -163,24 +170,24 @@
 
         class Popen(args, bufsize=0, executable=None,
                     stdin=None, stdout=None, stderr=None,
-                    preexec_fn=None, close_fds=False,
+                    preexec_fn=None, close_fds=False, shell=False,
                     cwd=None, env=None, universal_newlines=False,
                     startupinfo=None, creationflags=0):
-        
-      
+
+
       Arguments are:
-      
+
     - args should be a sequence of program arguments.  The program to
       execute is normally the first item in the args sequence, but can
       be explicitly set by using the executable argument.
-      
+
     - On UNIX: the Popen class uses os.execvp() to execute the child
       program, which operates on sequences.  If args is a string, it
       will be converted to a sequence using the cmdline2list method.
       Please note that syntax for quoting arguments is different from
       a typical UNIX shell.  See the documentation of the cmdline2list
       method for more information.
-      
+
     - On Windows: the Popen class uses CreateProcess() to execute the
       child program, which operates on strings.  If args is a
       sequence, it will be converted to a string using the
@@ -188,7 +195,7 @@
       applications interpret the command line the same way: The
       list2cmdline is designed for applications using the same rules
       as the MS C runtime.
-      
+
     - bufsize, if given, has the same meaning as the corresponding
       argument to the built-in open() function: 0 means unbuffered, 1
       means line buffered, any other positive value means use a buffer
@@ -213,6 +220,13 @@
     - If close_fds is true, all file descriptors except 0, 1 and 2
       will be closed before the child process is executed.
 
+    - If shell is true, the specified command will be executed through
+      the shell.  Note: When executing through the shell on UNIX
+      systems and the args argument is a sequence, only the first
+      element in the sequence will be passed as a command string to
+      the shell.  The remaining arguments can be used to specify
+      additional shell options.
+
     - If cwd is not None, the current directory will be changed to cwd
       before the child is executed.
 
@@ -233,66 +247,74 @@
       the underlying CreateProcess() function.  They can specify
       things such as appearance of the main window and priority for
       the new process.  (Windows only)
-      
-      
+
+
       This module also defines two shortcut functions:
-      
+
     - call(*args, **kwargs):
-          Run command with arguments.  Wait for command to complete, then
-          return the returncode attribute.
-      
-          The arguments are the same as for the Popen constructor.  Example:
-      
+          Run command with arguments.  Wait for command to complete,
+          then return the returncode attribute.
+
+          The arguments are the same as for the Popen constructor.
+          Example:
+
           retcode = call(["ls", "-l"])
-      
-      
+
+
     - callv(*args, **kwargs):
-          Run command with arguments.  Wait for command to complete, then
-          return the returncode attribute.
-      
-          This function is identical to call(), except that each non-keyword
-          argument is treated as a program argument.  Example:
-      
+          Run command with arguments.  Wait for command to complete,
+          then return the returncode attribute.
+
+          This function is identical to call(), except that each
+          non-keyword argument is treated as a program argument.
+          Example:
+
           retcode = callv("ls", "-l")
-      
+
           This is equivalent to:
-      
+
           retcode = call(["ls", "-l"])
-          
-      
+
+
     Exceptions
     ----------
+
     Exceptions raised in the child process, before the new program has
     started to execute, will be re-raised in the parent.
     Additionally, the exception object will have one extra attribute
     called 'child_traceback', which is a string containing traceback
-    information from the childs point of view.
-    
+    information from the child's point of view.
+
     The most common exception raised is OSError.  This occurs, for
     example, when trying to execute a non-existent file.  Applications
     should prepare for OSErrors.
-    
-    A ValueError will be raised if Popen is called with invalid arguments.
-    
-    
+
+    A ValueError will be raised if Popen is called with invalid
+    arguments.
+
+
     Security
     --------
-    Unlike some other popen functions, this implementation will never call
-    /bin/sh implicitly.  This means that all characters, including shell
-    metacharacters, can safely be passed to child processes.
-    
-    
+
+    Unlike some other popen functions, this implementation will never
+    call /bin/sh implicitly.  This means that all characters,
+    including shell meta-characters, can safely be passed to child
+    processes.
+
+
     Popen objects
     -------------
+
     Instances of the Popen class have the following methods:
 
     poll()
         Check if child process has terminated.  Returns returncode
         attribute.
-    
+
     wait()
-        Wait for child process to terminate.  Returns returncode attribute.
-    
+        Wait for child process to terminate.  Returns returncode
+        attribute.
+
     communicate(input=None)
         Interact with process: Send data to stdin.  Read data from
         stdout and stderr, until end-of-file is reached.  Wait for
@@ -306,21 +328,22 @@
         method if the data size is large or unlimited.
 
     The following attributes are also available:
-    
+
     stdin
         If the stdin argument is PIPE, this attribute is a file object
-        that provides input to the child process.  Otherwise, it is None.
-    
-    stdout
-        If the stdout argument is PIPE, this attribute is a file object
-        that provides output from the child process.  Otherwise, it is
+        that provides input to the child process.  Otherwise, it is
         None.
-    
+
+    stdout
+        If the stdout argument is PIPE, this attribute is a file
+        object that provides output from the child process.
+        Otherwise, it is None.
+
     stderr
-        If the stderr argument is PIPE, this attribute is file object that
-        provides error output from the child process.  Otherwise, it is
-        None.
-    
+        If the stderr argument is PIPE, this attribute is file object
+        that provides error output from the child process.  Otherwise,
+        it is None.
+
     pid
         The process ID of the child process.
 
@@ -328,14 +351,200 @@
         The child return code.  A None value indicates that the
         process hasn't terminated yet.  A negative value -N indicates
         that the child was terminated by signal N (UNIX only).
-          
-      
+
+
+Replacing older functions with the subprocess module
+
+    In this section, "a ==> b" means that b can be used as a
+    replacement for a.
+
+    Note: All functions in this section fail (more or less) silently
+    if the executed program cannot be found; this module raises an
+    OSError exception.
+
+    In the following examples, we assume that the subprocess module is
+    imported with "from subprocess import *".
+
+
+    Replacing /bin/sh shell backquote
+    ---------------------------------
+
+    output=`mycmd myarg`
+    ==>
+    output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
+
+
+    Replacing shell pipe line
+    -------------------------
+
+    output=`dmesg | grep hda`
+    ==>
+    p1 = Popen(["dmesg"], stdout=PIPE)
+    p2 = Popen(["grep", "hda"], stdin=p1.stdout)
+    output = p2.communicate()[0]
+
+
+    Replacing os.system()
+    ---------------------
+
+    sts = os.system("mycmd" + " myarg")
+    ==>
+    p = Popen(["mycmd" + " myarg"], shell=True)
+    sts = os.waitpid(p.pid, 0)
+
+    Note:
+
+    * Calling the program through the shell is usually not required.
+
+    * It's easier to look at the returncode attribute than the
+      exit status.
+
+    A more real-world example would look like this:
+
+    try:
+        retcode = callv("mycmd", "myarg")
+        if retcode < 0:
+            print >>sys.stderr, "Child was terminated by signal", -retcode
+        else:
+            print >>sys.stderr, "Child returned", retcode
+    except OSError, e:
+        print >>sys.stderr, "Execution failed:", e
+
+
+    Replacing os.spawn*
+    -------------------
+
+    P_NOWAIT example:
+
+    pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
+    ==>
+    pid = Popen(["/bin/mycmd", "myarg"]).pid
+
+
+    P_WAIT example:
+
+    retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
+    ==>
+    retcode = callv("/bin/mycmd", "myarg")
+
+
+    Vector example:
+
+    os.spawnvp(os.P_NOWAIT, path, args)
+    ==>
+    Popen([path] + args[1:])
+
+
+    Environment example:
+
+    os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
+    ==>
+    Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
+
+
+    Replacing os.popen*
+    -------------------
+
+    pipe = os.popen(cmd, mode='r', bufsize)
+    ==>
+    pipe = Popen([cmd], shell=True, bufsize=bufsize, stdout=PIPE).stdout
+
+    pipe = os.popen(cmd, mode='w', bufsize)
+    ==>
+    pipe = Popen([cmd], shell=True, bufsize=bufsize, stdin=PIPE).stdin
+
+
+    (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
+    ==>
+    p = Popen([cmd], shell=True, bufsize=bufsize,
+              stdin=PIPE, stdout=PIPE, close_fds=True)
+    (child_stdin, child_stdout) = (p.stdin, p.stdout)
+
+
+    (child_stdin,
+     child_stdout,
+     child_stderr) = os.popen3(cmd, mode, bufsize)
+    ==>
+    p = Popen([cmd], shell=True, bufsize=bufsize,
+              stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
+    (child_stdin,
+     child_stdout,
+     child_stderr) = (p.stdin, p.stdout, p.stderr)
+
+
+    (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
+    ==>
+    p = Popen([cmd], shell=True, bufsize=bufsize,
+              stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
+    (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
+
+
+    Replacing popen2.*
+    ------------------
+
+    Note: If the cmd argument to popen2 functions is a string, the
+    command is executed through /bin/sh.  If it is a list, the command
+    is directly executed.
+
+    (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
+    ==>
+    p = Popen(["somestring"], shell=True, bufsize=bufsize
+              stdin=PIPE, stdout=PIPE, close_fds=True)
+    (child_stdout, child_stdin) = (p.stdout, p.stdin)
+
+
+    (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
+    ==>
+    p = Popen(["mycmd", "myarg"], bufsize=bufsize,
+              stdin=PIPE, stdout=PIPE, close_fds=True)
+    (child_stdout, child_stdin) = (p.stdout, p.stdin)
+
+    The popen2.Popen3 and popen3.Popen4 basically works as
+    subprocess.Popen, except that:
+
+    * subprocess.Popen raises an exception if the execution fails
+    * the capturestderr argument is replaced with the stderr argument.
+    * stdin=PIPE and stdout=PIPE must be specified.
+    * popen2 closes all file descriptors by default, but you have to
+      specify close_fds=True with subprocess.Popen.
+
+
 Open Issues
 
-    Currently, the reference implementation requires the "win32all"
-    extensions when running on the Windows platform. This dependency
-    could probably be eliminated by providing a small "glue" module
-    written in C, just like the _winreg module. 
+    Some features have been requested but is not yet implemented.
+    This includes:
+
+    * Support for managing a whole flock of subprocesses
+
+    * Support for managing "daemon" processes
+
+    * Built-in method for killing subprocesses
+
+    While these are useful features, it's expected that these can be
+    added later without problems.
+
+    * expect-like functionality, including pty support.
+
+    pty support is highly platform-dependent, which is a
+    problem.  Also, there are already other modules that provides this
+    kind of functionality[6].
+
+
+Backwards Compatibility
+
+    Since this is a new module, no major backward compatible issues
+    are expected.  The module name "subprocess" might collide with
+    other, previous modules[3] with the same name, but the name
+    "subprocess" seems to be the best suggested name so far.  The
+    first name of this module was "popen5", but this name was
+    considered too unintuitive.  For a while, the module was called
+    "process", but this name is already used by Trent Mick's
+    module[4].
+
+    The functions and modules that this new module is trying to
+    replace (os.system, os.spawn*, os.popen*, popen2.*, commands.*)
+    are expected to be available in future Python versions for a long
+    time, to preserve backwards compatibility.
 
 
 Reference Implementation
@@ -352,6 +561,14 @@
     [2] Python Dialog
         http://pythondialog.sourceforge.net/
 
+    [3] http://www.iol.ie/~padraiga/libs/subProcess.py
+
+    [4] http://starship.python.net/crew/tmick/
+
+    [5] http://starship.python.net/crew/mhammond/win32/
+
+    [6] http://www.lysator.liu.se/~ceder/pcl-expect/
+
 
 Copyright
 



More information about the Python-checkins mailing list