[Baypiggies] Subprocess: Common problem/common pattern

Minesh B. Amin mamin at mbasciences.com
Sat Oct 2 19:23:40 CEST 2010


Hi Glen,

The are three general ways:

+ commands.getstatusoutput(...) 
     when you:
        * do not mind waiting for the command to finish
        * are interested in the stdout+stderr at the end (when the
command is finished)

+ pexpect < http://www.noah.org/wiki/Pexpect#Overview >
     when you:
        * want to interact with the launched command; e.g. enter login,
password, answer Y/N

     Thanks to pexpect, most of these interactions may be automated;
i.e.
     the python script may take over responsibility of processing
"login", "password" requests.

     Any interaction not automated may be passed on to the "human".

+ The solution "Capturing the Output and Error Streams from a 
     Unix Shell Command" from the book "Python Cookbook"

    when you:
      * want to simply consume the stderr/stdout of the launched command
in real-time

Hope this helps.

Cheers!
Minesh

On Sat, 2010-10-02 at 09:44 -0700, Glen Jarvis wrote:

> I've now seen a common problem come up several times. And, I imagine
> there is a common solution to this problem that I don't know.
> 
> 
> 
> About a year ago, I wrote an automatic script that would automatically
> do an 'svn export' of certain branches of a tree depending upon what
> menu option the customer chose. When I did the svn export, I used
> subprocess.Popen.
> 
> 
> The pattern was similar to the following:
> 
> 
> print """This output is being buffered so I can read the version
> number.
>     ....  I'm not stuck, just busy exporting files....
> 
> """
> 
> .....
> 
> process = subprocess.Popen(['svn', 'export', repository,
> repo_name], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> 
> 
> stdoutdata, stderrdata = process.communicate()
> 
> 
> 
> 
> I printed "please wait" and then printed the data given when the
> process was done (stdoutdata). It wasn't ideal but, it was sufficient
> for the time. If I were to have gone for the best fix, I would
> probably have learned the API for subversion to integrate directly
> into python.
> 
> 
> However, another BayPIGgie is having the same issue. He is using a
> command to start a CD burner from the command line and wants to print
> the output as it is being created from the command line.
> 
> 
> I can see that to solve this we wouldn't use the communicate()
> convenience function. A few 'hackish' ways that may solve this, but
> I'm looking for the common pattern that is used when other pythonista
> run up against this problem. I also want to ensure that I don't have a
> 'hack' that causes a deadlock only to discover this much later after
> I've implemented the pattern a few times.
> 
> 
> To help keep the conversation more focused, I've created two tiny test
> programs for a test case:
> 1) A C command line program that we have no control over changing
> within python, and
> 2) A Python program that calls that the c-program (baypiggies_demo):
> 
> 
> Compile command line so output is same as expected by Python program:
> gcc -pedantic baypiggies_demo.c -o baypiggies_demo
> 
> 
> 
> 
> ---- start of c program: File: baypiggies_demo.c ---
> #include <stdio.h>
> #include <unistd.h>
> 
> 
> int
> main()
> {
>     int             i = 0;
> 
> 
>     printf("Silly output for monitoring...\n");
>     for (i = 0; i <= 10; i++) {
>         printf("Counting... %d\n", i);
>         sleep(1);
>     }
> }
> --- end of c program ---
> 
> 
> 
> 
> 
> 
> --- start of python program to demonstrate. File baypiggies.py ---
> import subprocess
> 
> 
> print "Just waiting...."
> 
> 
> process = subprocess.Popen(['./baypiggies_demo'],
>                     stdout=subprocess.PIPE)
> 
> 
> stdoutdata, stderrdata = process.communicate()
> 
> 
> print "Well, NOW I get it.. :( "
> print stdoutdata
> --- end baypiggies.py --
> 
> 
> 
> 
> 
> 
> Has anyone else ran into this before? What's the classic pattern used?
> 
> 
> 
> 
> Thanks in advance,
> 
> 
> 
> 
> Glen
> 
> -- 
> Whatever you can do or imagine, begin it;
> boldness has beauty, magic, and power in it.
> 
> -- Goethe 
> 
> 
> _______________________________________________
> Baypiggies mailing list
> Baypiggies at python.org
> To change your subscription options or unsubscribe:
> http://mail.python.org/mailman/listinfo/baypiggies


-- 
MBA Sciences, Inc (www.mbasciences.com)
Tel #: 650-938-4306
Email: mamin at mbasciences.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/baypiggies/attachments/20101002/8f9be849/attachment-0001.html>


More information about the Baypiggies mailing list