os.popen and broken pipes

[Antoon Pardon]

On 2007-02-09, Philipp Pagel <pDOTpagel at gsf.de> wrote:
>
> 	Hi Pythoneers,
>
> I need to process a large number of files which have been packed by the
> UNIX compress tool (*.Z files). As I am not aware of a compress
> equivalent of the gzip, zipfile or bzip2 modules, I thought I'd use the
> uncompress or zcat commands directly to deal with the files:
>
> for filename in file_list:
>     file = os.popen('uncompress -c '+filename, 'r')
>     do_something(file)
>     file.close()
>
>
> This works fine for some files but results in
>
> 'write error onstdout: Broken pipe'
>
> emitted by uncompress for others. Using zcat instead of uncompress
> changes the wording of the error message but not the result.
>
> I tried to give popen a large bufsize argument but that didn't really
> help.
>
> Probably I'm overlooking something obvious here but right now I can't
> see it. Any hints?

As far as I can tell, your do_something doesn't consume the entire file.
So you close the file prematurly, which results in the uncompress/zcat
program trying to write to a pipe that is closed on the otherside,
giving you the above message.

-- 
Antoon Pardon