[Spambayes] sb_filter.py eats mail when used in postfix content_filter

[EMF]

On 4/24/2013 11:43 PM, Fred Smith wrote:
> On Wed, Apr 24, 2013 at 10:15:22PM -0400, EMF wrote:
>> OS:  CentOS 6.4 x86_64
>> Postfix package:  postfix-2.6.6-2.2.el6_1.x86_64
>> Spambayes package:  spambayes-1.1-0.2.a6.el6.noarch
>>
>> All right, I've hit a brick wall with my integration of spambayes
>> and postfix.  I based my build combining Jonathan St. Andre's
>> instructions at http://spambayes.sourceforge.net/server_side.html,
>> with a few tweaks from the Postfix side in
>> http://www.postfix.org/FILTER_README.html --- mostly because I'm
>> going to extend the logic.
>>
>> I've successfully trained spambayes with about 12,000 each spam and
>> ham, creating a database in /var/spambayes/filter.db.
>>
>> Running a captured message through sb_filter.py from the command
>> line works flawlessly:
>> -----------------------------------------
>> $ whoami
>> filter
>> $ cat /tmp/msg.7440 | /usr/bin/sb_filter.py -d /var/spambayes/filter.db -f
>> Return-Path: <emf at amhran.net>
>> Received: from amhran.net (3wg23q1.amhran.net [192.168.1.160])
>>          by courier.amhran.net (Postfix) with ESMTP id CFF223E
>>          for <emf at amhran.net>; Wed, 24 Apr 2013 21:28:34 -0400 (EDT)
>> From: "EMF" <emf at amhran.net>
>> To: "EMF" <emf at amhran.net>
>> Subject: Testing
>> X-Spambayes-Classification: ham; 0.17
>>
>> SB Test
>> $
>> -----------------------------------------
>>
>> Here are the lines from postfix's master.cf (which are very
>> straightforward):
>> -----------------------------------------
>> smtp      inet  n       -       n       -       -       smtpd
>>       -o content_filter=filter:dummy
>> filter unix  -   n   n   -   -   pipe
>>       flags=Rq user=filter
>>       argv=/usr/local/bin/spamfilter.sh -f ${sender} -- ${recipient}
>> -----------------------------------------
>>
>> Spamfilter.sh sends the mail just fine if it consists of this:
>> -----------------------------------------
>>      cat > /tmp/msg.$$
>>      /usr/sbin/sendmail -i $@ < /tmp/msg.$$
>> -----------------------------------------
>> but not if it contains this:
>> -----------------------------------------
>>      cat > /tmp/msg.$$
>>      cat /tmp/msg.$$ | /usr/bin/sb_filter.py -d
>> /var/spambayes/filter.db -f > /tmp/new.$$
>>      /usr/sbin/sendmail -i $@ < /tmp/new.$$
>> -----------------------------------------
>>
>> In the latter case --- msg.$$ contains the message, but new.$$ is a
>> zero-length file, and thus what comes through on the far end is an
>> empty message.  Interestingly, I get this in syslog.log:
>> -----------------------------------------
>> Apr 24 21:28:35 courier abrt: detected unhandled Python exception in
>> '/usr/bin/sb_filter.py'
>> Apr 24 21:28:35 courier abrt: can't communicate with ABRT daemon, is
>> it running? [Errno 13] Permission denied
>> -----------------------------------------
>>
>> How can I capture the exception?  What's being called by
>> sb_filter.py at the command line that isn't available when run
>> inside the content_filter?
> Well, I'm not intimate with the innards of the spambayes components,
> but if a program behaves differently on commandline than it does
> in its own normal environment, I'd guess there's something different
> about the system environment: PATH, LD_LIBRARY_PATH, etc., etc.,
> etc.
>
>
Here's the sorted diff on "set" output, cleaned up of stuff that is 
obviously extraneous like PS1.  ">" is command-line, "<" is scripted 
environment.  The only thing that seemed like it would affect anything 
was PATH, so I tried it again having duplicated the PATH into the 
script; still blank message and error in syslog.

-----------------------------------------

 > 
BASHOPTS=cmdhist:expand_aliases:extquote:force_fignore:hostcomplete:interactive_comments:login_shell:progcomp:promptvars:sourcepath
 > G_BROKEN_FILENAMES=1
 > HOME=/var/spool/filter
 > CVS_RSH=ssh
 > LANG=en_US.UTF-8
 > LESSOPEN='|/usr/bin/lesspipe.sh %s'
 > LINES=24
 > LOGNAME=filter
 > MAIL=/var/spool/mail/filter
 > MAILCHECK=60
 > 
PATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin
 > PWD=/var/spool/filter
 > QTDIR=/usr/lib64/qt-3.3
 > QTINC=/usr/lib64/qt-3.3/include
 > QTLIB=/usr/lib64/qt-3.3/lib
 > 
SHELLOPTS=braceexpand:hashall:histexpand:history:interactive-comments:monitor
 > TERM=xterm

< 
BASHOPTS=cmdhist:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath
< LANG=C
< MAIL_CONFIG=/etc/postfix
< PATH=/usr/bin:/bin
< PWD=/tmp
< SENDMAIL='/usr/sbin/sendmail -i'
< SHELLOPTS=braceexpand:hashall:interactive-comments
< SPOOL_DIR=/tmp
< TERM=dumb

-----------------------------------------

I don't believe sb_filter.py has any calls into