What is needed is the input as it's being generated.<div><br></div><div>For example, the C-program counts to 10, with a second delay between each count.</div><div><br></div><div>The python program, when it calls the C program, will have a 10 second delay before it sees anything. Ideally, the python program can be printing the results of the C program as they are being generated.</div>
<div><br></div><div><br></div><div>Thanks for asking for clarification...</div><div><br></div><div><br></div><div>Cheers,</div><div><br></div><div><br></div><div><br></div><div>Glen<br><br><div class="gmail_quote">On Sat, Oct 2, 2010 at 10:05 AM, joshua kaderlan <span dir="ltr"><<a href="mailto:jkaderlan@yahoo.com">jkaderlan@yahoo.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div style="font-family:times new roman,new york,times,serif;font-size:12pt">Reading this, it's not entirely clear what the problem is. Is it that subprocess.Popen buffers output, rather than returning it immediately the way os.system() does?<br>
<div><br></div><blockquote style="border-left:2px solid rgb(16, 16, 255);margin-left:5px;padding-left:5px"><div style="font-family:times new roman,new york,times,serif;font-size:12pt"><br><div style="font-family:times new roman,new york,times,serif;font-size:12pt">
<font face="Tahoma" size="2"><b><span style="font-weight:bold">From:</span></b> Glen Jarvis <<a href="mailto:glen@glenjarvis.com" target="_blank">glen@glenjarvis.com</a>><br><b><span style="font-weight:bold">To:</span></b> Baypiggies <<a href="mailto:baypiggies@python.org" target="_blank">baypiggies@python.org</a>><br>
<b><span style="font-weight:bold">Sent:</span></b> Sat, October 2, 2010 9:44:05 AM<br><b><span style="font-weight:bold">Subject:</span></b> [Baypiggies] Subprocess: Common problem/common pattern<br></font><div><div></div>
<div class="h5"><br>
I've now seen a common problem come up several times. And, I imagine there is a common solution to this problem that I don't know.<div><br></div><div>About a year ago, I wrote an automatic script that would automatically do an 'svn export' of certain branches of a tree depending upon what menu option the customer chose. When I did the svn export, I used subprocess.Popen.</div>
<div><br></div><div>The pattern was similar to the following:</div><div><br></div><div>print """This output is being buffered so I can read the version number.</div><div>
<p> .... I'm not stuck, just busy exporting files....</p><p>"""</p>
<p>.....</p>
<p>process = subprocess.Popen(['svn', 'export', repository, repo_name], stdout=subprocess.PIPE, stderr=subprocess.PIPE)</p></div><div><div>stdoutdata, stderrdata = process.communicate()</div></div>
<div><br></div><div><br></div><div>I printed "please wait" and then printed the data given when the process was done (stdoutdata). It wasn't ideal but, it was sufficient for the time. If I were to have gone for the best fix, I would probably have learned the API for subversion to integrate directly into python.</div>
<div><br></div><div>However, another BayPIGgie is having the same issue. He is using a command to start a CD burner from the command line and wants to print the output as it is being created from the command line.</div><div>
<br></div><div>I can see that to solve this we wouldn't use the communicate() convenience function. A few 'hackish' ways that may solve this, but I'm looking for the common pattern that is used when other pythonista run up against this problem. I also want to ensure that I don't have a 'hack' that causes a deadlock only to discover this much later after I've implemented the pattern a few times.</div>
<div><br></div><div>To help keep the conversation more focused, I've created two tiny test programs for a test case:</div><div>1) A C command line program that we have no control over changing within python, and</div>
<div>2) A Python program that calls that the c-program (baypiggies_demo):</div><div><br></div><div>Compile command line so output is same as expected by Python program:</div><div>gcc -pedantic baypiggies_demo.c -o baypiggies_demo</div>
<div><br></div><div><br><div>---- start of c program: File: baypiggies_demo.c ---</div><div><div>#include <stdio.h></div><div>#include <unistd.h></div><div><br></div><div>int</div><div>main()</div><div>{</div>
<div> int i = 0;</div>
<div><br></div><div> printf("Silly output for monitoring...\n");</div><div> for (i = 0; i <= 10; i++) {</div><div> printf("Counting... %d\n", i);</div><div> sleep(1);</div><div>
}</div>
<div>}</div></div><div>--- end of c program ---</div><div><br></div><div><br></div><div><br></div><div>--- start of python program to demonstrate. File <a href="http://baypiggies.py" target="_blank">baypiggies.py</a> ---</div>
<div><div>import subprocess</div><div><br></div>
<div>print "Just waiting...."</div><div><br></div><div>process = subprocess.Popen(['./baypiggies_demo'],</div><div> stdout=subprocess.PIPE)</div><div><br></div><div>stdoutdata, stderrdata = process.communicate()</div>
<div><br></div><div>print "Well, NOW I get it.. :( "</div><div>print stdoutdata</div><div>--- end baypiggies.py --</div><div><br></div><div><br></div><div><br></div><div>Has anyone else ran into this before? What's the classic pattern used?</div>
<div><br></div><div><br></div><div>Thanks in advance,</div><div><br></div><div><br></div><div>Glen</div>-- <br>Whatever you can do or imagine, begin it;<br>
boldness has beauty, magic, and power in it.<br><br>-- Goethe <br>
</div>
</div>
</div></div></div></div></blockquote>
</div><br>
</div></blockquote></div><br><br clear="all"><br>-- <br>Whatever you can do or imagine, begin it;<br>boldness has beauty, magic, and power in it.<br><br>-- Goethe <br>
</div>