spawning a process with subprocess

[Ove Svensson]

bhunter <brian.p.hunter at gmail.com> writes:

> Hi,
> 
> I've used subprocess with 2.4 several times to execute a process, wait
> for it to finish, and then look at its output.  Now I want to spawn
> the process separately, later check to see if it's finished, and if it
> is look at its output.  I may want to send a signal at some point to
> kill the process.  This seems straightforward, but it doesn't seem to
> be working.
> 
> Here's my test case:
> 
> import subprocess, time
> 
> cmd = "cat somefile"
> thread = subprocess.Popen(args=cmd.split(), shell=True,
> stdout=subprocess.PIPE, stdin=subprocess.PIPE,
> stderr=subprocess.STDOUT, close_fds=True)
> 
> while(1):
>       time.sleep(1)
>       if(thread.returncode):
>          break
>       else:
>          print thread.returncode
> 
> print "returncode = ", thread.returncode
> for line in thread.stdout:
>    print "stdout:\t",line
> 
> 
> This will just print the returncode of None forever until I Ctrl-C it.
> 
> Of course, the program works fine if I call thread.communicate(), but
> since this waits for the process to finish, that's not what I want.
> 
> Any help would be appreciated.

Reading documentation for subprocess, it mentions that

    On UNIX, with shell=False (default): In this case, the Popen class
    uses os.execvp() to execute the child program.  args should normally
    be a sequence.  A string will be treated as a sequence with the string
    as the only item (the program to execute).
    
    On UNIX, with shell=True: If args is a string, it specifies the
    command string to execute through the shell.  If args is a sequence,
    the first item specifies the command string, and any additional items
    will be treated as additional shell arguments.

Since you have specified shell = True, and since you pass a sequence as
args, you will efficiently invoke the cat process through the shell and
then pass somefile as an extra argument to she shell (not the cat command)
That is probably not what you intended.

This can be solved by either
- Not splitting the cmd, in which case you will pass the whole cmd
  string to the shell for execution
- Or setting shell to False. This is what I would have done, since
  I can't see any reason for going via the shell. Please note that
  if setting shell to False, you must then split the cmd.

Please also note that your test for the returncode might not work
since a normal returncode is 0. Your code will only detect non-0
values.

Also, it is good practice to call wait() on the subprocess in order
to avoid zombie-processes.

Finally, I find it somewhat misleading to use the name thread for
the variable used to represent a sub-process. Threads and processes
are not exactly the same

Hence, the following code should works as expected

cmd = "cat somefile"
proc = subprocess.Popen(
    args      = cmd.split(),
    shell     = False,
    stdin     = None,
    stdout    = subprocess.PIPE,
    stderr    = subprocess.STDOUT,
    close_fds = True)

while True:
    rc = proc.poll()
    if rc != None: break
    print rc
    time.sleep(1)

lno = 1
for lin in proc.stdout:
    print '%i: %s' % (lno,lin.rstrip('\n'))
    lno += 1

rc = proc.wait()
print "rc = %i" % rc


/Ove