[Fwd: Re: [Tutor] What am I missing?]

[Danny Yoo]

On Mon, 27 Sep 2004, Tom Tucker wrote:

> I think I found the issue.  My "for" and "if" statements were never
> reached.  The system call "logfile = os.system('tail -f
> /var/log/messages')" doesn't appears to be communicating outside of
> itself.  It was just printing to STDOUT.  Back to the drawing board!


Hi Tom,

Ah!  Use os.popen(), not os.system().  The problem is that os.system()
waits until the command is finished.  But 'tail -f' never terminates.
*grin*


There are a few more issues with something like filtering 'tail -f'
through Python efficiently.  The big one is "blocking": if we do something
like:

    print os.popen("tail -f foobar").read()

we may actually end up waiting forever.  *grin*

read(), by default, is a "blocking" call that waits until the file is
closed.  But we can't afford to wait that long.  The trick is to make it
"unblocked".  Here's a small example that might help you on your way:


###
"""A little demonstration of the 'select' module with 'popen'."""

import os
import select
import sys
import fcntl

def unblock(f):
    """Given file 'f', sets its unblock flag to true."""
    fcntl.fcntl(f.fileno(), fcntl.F_SETFL, os.O_NONBLOCK)


if __name__ == '__main__':
    f = os.popen("tail -f %r" % sys.argv[1])
    unblock(f)

    while True:
        ins, outs, excs = select.select([f], [], [])
        sys.stdout.write("Got another bit of information:\n")
        sys.stdout.write(ins[0].read())
###

There's a little voodoo at the beginning to turn on the "nonblock" flag on
a file.  (Does anyone know a more obvious way to do this?)


Anyway, the code is also using a 'select' event loop to avoid spinning
around until there's actually new content in the files.  We could have
written the loop like this:

###
    while True:
        try:
            newText = f.read()
            sys.stdout.write(newText)
        except IOError:
            ## ignore IOErrors due to reading on a file that doesn't
            ## have anything yet
            pass
###


This is a "busy-waiting" loop that will cause the system to spiral through
the loop fast, regardless if there's actual content in the unblocked file
or not.  So this is acutally not a good thing to do.  Using
select.select() allows us to be more sympathetic with our machine.
*grin*


See:

    http://www.python.org/doc/lib/module-select.html

for more details about select.  I'm actually not so sure where to point to
about the nonblocking IO stuff though.


Hope this helps!