[Tutor] pstree cgi-script

Danny Yoo dyoo at hkn.eecs.berkeley.edu
Mon Jun 21 14:49:08 EDT 2004



On Fri, 18 Jun 2004, Conrad Gavin wrote:

> NB: With regard the previous post, I am finding similar things on google
> about python's seemingly dogey co-exhistence with Apache.
>
> http://lists.debian.org/debian-user/2002/05/msg00967.html
>
> Seems to be more or less what I am experiencing ...


Hi Conrad,

What they're running into is a tricky interaction between the buffering
that Apache and Python is doing, combined with the behavior of
os.system().  Let's go back to the script that isn't working:

###
#!/usr/bin/python
import sys
import os
import cgitb; cgitb.enable()

sys.stdout.write("Content-type: text/plain\n\n")
cmd = "/usr/bin/pstree"
os.system(cmd)
###


The reason it doesn't work is subtle, and I have to admit that I don't
quite understand it fully yet.  But I'll try saying something reasonable.
*grin*

It appears that Python keeps its own buffered standard output stream, and
because it's buffered, Python won't write it out until it triggered to do
so.  I assumed that any writes of '\n' newlines will do it, but it appears
that later versions of Apache do something funny to the STDOUT file handle
so that even '\n' doesn't flush it automatically.


Usually, this shouldn't matter much... except that the script also calls
os.system(), which accesses the underlying STDOUT file handle as a side
effect.  What Apache ends up seeing, ultimately, is the output of 'pstree'
first, followed by the Content-type header.


The following is one way to address the problem:

###
#!/usr/bin/python
import sys
import os
import cgitb; cgitb.enable()

sys.stdout.write("Content-type: text/plain\n\n\n")
sys.stdout.flush()              ## line change here
cmd = "/usr/bin/pstree"
os.system(cmd)
###

The explicit call to flush() guarantees that Apache will first see the
Content-type header.  And after that, Apache should be happy.


But a better way to fix this is to avoid depending on the 'side effect' of
os.system(), and to actually capture its output with os.popen() instead:

###
#!/usr/bin/python
import sys
import os
import cgitb; cgitb.enable()

sys.stdout.write("Content-type: text/plain\n\n\n")
cmd = "/usr/bin/pstree"
print os.popen(cmd).read()
###

That way, we avoid all the weird buffer issues that your script is running
into.  In summary: os.system() isn't meant to be used for output purposes,
and it gives no guarantee when you'll see its output on screen.
os.popen() is more appropriate for the task of capturing the output.


There are some other weird things with Apache's CGI sytem that have very
little to do with Python.  It appears that in some versions of Apache,
printing to STDERR was not a good idea:

    http://nagoya.apache.org/bugzilla/show_bug.cgi?id=22030

And this just sounded insane to me.  *grin* But I'm glad the Apache folks
are getting this fixed, finally.


Anyway, hope this helps!




More information about the Tutor mailing list