[Spambayes] sb_filter.py eats mail when used in postfix content_filter
Fred Smith
fredex at fcshome.stoneham.ma.us
Thu May 2 15:08:16 CEST 2013
On Wed, May 01, 2013 at 11:21:34PM -0400, E M F wrote:
> On 4/25/2013 12:02 AM, EMF wrote:
> >On 4/24/2013 11:43 PM, Fred Smith wrote:
> >>On Wed, Apr 24, 2013 at 10:15:22PM -0400, EMF wrote:
> >>>OS: CentOS 6.4 x86_64
> >>>Postfix package: postfix-2.6.6-2.2.el6_1.x86_64
> >>>Spambayes package: spambayes-1.1-0.2.a6.el6.noarch
> >>>
> >>>All right, I've hit a brick wall with my integration of spambayes
> >>>and postfix. I based my build combining Jonathan St. Andre's
> >>>instructions at http://spambayes.sourceforge.net/server_side.html,
> >>>with a few tweaks from the Postfix side in
> >>>http://www.postfix.org/FILTER_README.html --- mostly because I'm
> >>>going to extend the logic.
> >>>
> >>>I've successfully trained spambayes with about 12,000 each spam and
> >>>ham, creating a database in /var/spambayes/filter.db.
> >>>
> >>>Running a captured message through sb_filter.py from the command
> >>>line works flawlessly:
> >>>-----------------------------------------
> >>>$ whoami
> >>>filter
> >>>$ cat /tmp/msg.7440 | /usr/bin/sb_filter.py -d
> >>>/var/spambayes/filter.db -f
> >>>Return-Path: <emf at amhran.net>
> >>>Received: from amhran.net (3wg23q1.amhran.net [192.168.1.160])
> >>> by courier.amhran.net (Postfix) with ESMTP id CFF223E
> >>> for <emf at amhran.net>; Wed, 24 Apr 2013 21:28:34 -0400 (EDT)
> >>>From: "EMF" <emf at amhran.net>
> >>>To: "EMF" <emf at amhran.net>
> >>>Subject: Testing
> >>>X-Spambayes-Classification: ham; 0.17
> >>>
> >>>SB Test
> >>>$
> >>>-----------------------------------------
> >>>
> >>>Here are the lines from postfix's master.cf (which are very
> >>>straightforward):
> >>>-----------------------------------------
> >>>smtp inet n - n - - smtpd
> >>> -o content_filter=filter:dummy
> >>>filter unix - n n - - pipe
> >>> flags=Rq user=filter
> >>> argv=/usr/local/bin/spamfilter.sh -f ${sender} -- ${recipient}
> >>>-----------------------------------------
> >>>
> >>>Spamfilter.sh sends the mail just fine if it consists of this:
> >>>-----------------------------------------
> >>> cat > /tmp/msg.$$
> >>> /usr/sbin/sendmail -i $@ < /tmp/msg.$$
> >>>-----------------------------------------
> >>>but not if it contains this:
> >>>-----------------------------------------
> >>> cat > /tmp/msg.$$
> >>> cat /tmp/msg.$$ | /usr/bin/sb_filter.py -d
> >>>/var/spambayes/filter.db -f > /tmp/new.$$
> >>> /usr/sbin/sendmail -i $@ < /tmp/new.$$
> >>>-----------------------------------------
> >>>
> >>>In the latter case --- msg.$$ contains the message, but new.$$ is a
> >>>zero-length file, and thus what comes through on the far end is an
> >>>empty message. Interestingly, I get this in syslog.log:
> >>>-----------------------------------------
> >>>Apr 24 21:28:35 courier abrt: detected unhandled Python exception in
> >>>'/usr/bin/sb_filter.py'
> >>>Apr 24 21:28:35 courier abrt: can't communicate with ABRT daemon, is
> >>>it running? [Errno 13] Permission denied
> >>>-----------------------------------------
> >>>
> >>>How can I capture the exception? What's being called by
> >>>sb_filter.py at the command line that isn't available when run
> >>>inside the content_filter?
> >>Well, I'm not intimate with the innards of the spambayes components,
> >>but if a program behaves differently on commandline than it does
> >>in its own normal environment, I'd guess there's something different
> >>about the system environment: PATH, LD_LIBRARY_PATH, etc., etc.,
> >>etc.
> >>
> Okay, I caught the exception in sb_filter.py with a traceback:
>
> ------------------------------------------------------------
> Traceback (most recent call last):
> File "/usr/bin/sb_filter_debug.py", line 279, in <module>
> main()
> File "/usr/bin/sb_filter_debug.py", line 255, in main
> h.newdb()
> File "/usr/bin/sb_filter_debug.py", line 179, in newdb
> self.open('n')
> File "/usr/bin/sb_filter_debug.py", line 167, in open
> self.h = hammie.open(self.dbname, self.usedb, self.mode)
> File "/usr/lib/python2.6/site-packages/spambayes/hammie.py", line 272, in open
> return Hammie(storage.open_storage(filename, useDB, mode), mode)
> File "/usr/lib/python2.6/site-packages/spambayes/storage.py", line 998, in open_storage
> return klass(data_source_name, mode)
> File "/usr/lib/python2.6/site-packages/spambayes/storage.py", line 154, in __init__
> self.load()
> File "/usr/lib/python2.6/site-packages/spambayes/storage.py", line 180, in load
> self.dbm = dbmstorage.open(self.db_name, self.mode)
> File "/usr/lib/python2.6/site-packages/spambayes/dbmstorage.py", line 70, in open
> return f(db_name, mode)
> File "/usr/lib/python2.6/site-packages/spambayes/dbmstorage.py", line 40, in open_best
> return f(*args)
> File "/usr/lib/python2.6/site-packages/spambayes/dbmstorage.py", line 20, in open_dbhash
> return bsddb.hashopen(*args)
> File "/usr/lib64/python2.6/bsddb/__init__.py", line 361, in hashopen
> d.open(file, db.DB_HASH, flags, mode)
> DBAccessError: (13, 'Permission denied')
>
> ------------------------------------------------------------
>
> So, it's getting a permission denied on the hash database. I added some logic to capture self.dbname, self.usedb, and self.mode --- and I found a difference. When run from the command line, self.mode is "r", but when run as part of the filter, it's set to "n". It's trying to create a new hash database instead of reading the one it's been told to use!
>
> After digging back through the code and disabling parts, it's choking on "os.path.exists". For some reason, when run as part of the postfix filter, it believes /var/spambayes/filter.db doesn't exist. As shown in the master.cf lines, there's no chroot involved, so I didn't know why it couldn't find the file.
>
> And then, the lightbulb hits, and I proceed to start thumping my head on the table:
> [root at courier-b tmp] # getenforce
> Enforcing
>
> Curse you, SElinux, curse you. Setting the correct context on the filter.db file resolves the problem.
>
> NOTABUG.
>
Good troubleshooting!
It's unfortunate that SpamBayes won't fail gracefully, i.e., make sure
the mail gets delivered even if the filter fails.
Fred
--
-------------------------------------------------------------------------------
.---- Fred Smith /
( /__ ,__. __ __ / __ : /
/ / / /__) / / /__) .+' Home: fredex at fcshome.stoneham.ma.us
/ / (__ (___ (__(_ (___ / :__ 781-438-5471
-------------------------------- Jude 1:24,25 ---------------------------------
More information about the SpamBayes
mailing list