os.popen hangs mutt

[Jason R. Mastaler]

I need some help debugging a strange problem I'm having with the mutt
mail client on certain BSD UNIX platforms.  I think it might be caused
by a bug in os.popen.

Basically, I use a custom Python "injection agent" which sits between
mutt and the mail transport agent (qmail in my case).  It accepts the
message from mutt as stdin, does some header manipulation, and hands
it off to qmail-inject for delivery.

mutt --> injection-agent --> qmail-inject --> recipient

The problem is that on certain operating systems (FreeBSD, NetBSD,
OpenBSD), this exchange hangs before reaching qmail-inject so the
message is never delivered.  The injection-agent is left hanging
around in the process table until manually killed.  For example,

(jason at mail1):(~)$ echo | mutt jason

(this command hangs, and the `ps' output looks like this)
jason      737  0.0  1.1   900  1424 ??  I     10:44AM    0:00.03 python /home/jason/pyinject -- jason

The strange thing is that if I implement the injection-agent using the
popen2 module instead of os.popen, the problem goes away.  I've tried
several different Python versions from 1.5.2 to 2.1 with the same
results.  So the question is: why does os.popen hang while popen2
doesn't?

Here is a simple prototype of the injection-agent that I use to
reproduce the problem.  The program simply accepts the message on
stdin, extracts the "To:" header, opens a pipe to qmail-inject, and
writes the message to it.

============================================

#!/usr/bin/env python

import os
import sys
import rfc822

message = sys.stdin
message_headers = rfc822.Message(message)

to = message_headers['to']
message_body = message.read()

inject = '/var/qmail/bin/qmail-inject ' + to
pipeline = os.popen(inject, 'w')
pipeline.write(str(message_headers))
pipeline.write(message_body)
pipeline.close()

============================================

If I replace the last chunk of the above program with popen2, the
problem goes away.  I.e,

[...]

import popen2
pipeline = popen2.popen2(inject)[1]
pipeline.write(str(message_headers))
pipeline.write('\n')
pipeline.write(message_body)
pipeline.close()

P.S - if anyone is brave enough, I have a 400k kernel process trace
log of a hung mutt delivery attempt that might contain the answer.

-- 
(TMDA - http://tmda.sourceforge.net)
(Python-based SPAM reduction system)