[Tutor] execute an OS command, get the output

[Terry Carroll]

On Sat, 11 Mar 2006, Terry Carroll wrote:

> I gotta admit, this took me by surprise, too, but my guess is that once 
> the head command is done, it closes the pipe it's reading from, which is 
> being filled by grep; grep takes the hint and terminates, closing the pipe 
> it's reading from, which is being filled by strings; and strings takes the 
> hint and terminates, even though it hasn't gotten through the entire file.

Just for the heack of it, I wrote a tiny Python echo program, and 
interposed it in the pipe between the strings and grep command:

 while 1:
     line = raw_input()
     print line

The command line now looks like this:

 strings 00003193.DAT | python echo.py | grep Newsgroups: | head

(the .DAT file is an Agent newsgroup file; the idea here is that by 
grepping for the first few lines with "Newsgroups:", I can tell what 
newsgroup the .DAT file is associated with.)

I get:

> strings 00003193.DAT | python echo.py | grep Newsgroups: | head -10
Newsgroups: comp.lang.python.announce
Newsgroups: comp.lang.python.announce
Newsgroups: comp.lang.python.announce
Newsgroups: comp.lang.python.announce
Newsgroups: comp.lang.python.announce
Newsgroups: comp.lang.python.announce
Newsgroups: comp.lang.python.announce
Newsgroups: comp.lang.python.announce
Newsgroups: comp.lang.python.announce
Newsgroups: comp.lang.python.announce
Traceback (most recent call last):
  File "echo.py", line 3, in ?
    print line
IOError: [Errno 22] Invalid argument
close failed: [Errno 22] Invalid argument

My guess is that the "Invalid argument" here was a write attempting to 
write to a closed file, the pipe.  The string and grep commands (or IO 
routines called by them) probably detect this and close gracefully; the 
end result being that a set of piped commands only lasts as long as the 
shortest-lived consumer, and the upstream producers shut down when they 
can no longer write to the pipe.