I want to set up spambayes to work with procmail on my mail server. Does anyone have experience with that? If not, will someone please discuss it with me? I'm particularly interested in what the model for getting new spam/ham classifications to procmail might be. My last query of 24 February went completely unanswered, which is a little discouraging. I have quite a learning curve to overcome, having no experience with procmail and little with IMAP. If someone who knows a little about SpamBayes could at least help me figure out which questions I need to answer in order to get started, that would be a big help. Thanks! -- Dave Abrahams Boost Consulting www.boost-consulting.com
3/31/2003 10:07:52 AM, David Abrahams <dave@boost-consulting.com> wrote:
I want to set up spambayes to work with procmail on my mail server. Does anyone have experience with that?
You should start by reading http://spambayes.sourceforge.net/applications.html. There is a link to a page called "guide to integrating hammie with your mailer" on that page that should give you some good starting points. The subject of integrating with procmail has been discussed relatively extensively in the mailing list. You might check out the archives, searching on procmail. Again, start at http://spambayes.sourceforge.net If after that you're having trouble, please be sure to drop us a line! c'est moi - TimS http://www.fourstonesExpressions.com http://wecanstopspam.org There are 10 kinds of people in the world: those who understand binary, and those who don't.
Dave> I want to set up spambayes to work with procmail on my mail Dave> server. Does anyone have experience with that? Dave, I use spambayes with procmail. The major issue is generally not one of getting messages classified, but of getting them trained. Here are the relevant bits out of my procmailrc file: PYCKSUM=$HOME/local/bin/pycksum HAMMIE=$HOME/local/bin/hammiefilter.py BAYESCUSTOMIZE=$HOME/hammie.opt :0 fw:hamlock | $HAMMIE -d $HOME/hammie.db :0 * ^X-Spambayes-Classification: spam { ### this recipe gobbles items with matching body checksums (taken ### loosely to try and avoid obvious tricks) :0 W: cksum.lock | $PYCKSUM -v $HOME/tmp/cksum.cache ### spam scores come in two flavors - equal to 1.00 and less than ### 1.00 scores are much more likely to be real spam, so require ### less sifting - therefore keep them separate :0: * ^X-Spambayes-Classification: spam; 1.00 $SPAM1 :0: $SPAM } :0 * ^X-Spambayes-Classification: unsure unsure ... You can dispense with the PYCKSUM stuff, though I find it does delete a fair number of duplicate spams. I get email for a large number of aliases at the same address however. YMMV. I've attached the version of the script which I use. It's similar to the loosecksum.py script in the Spambayes utilities directory, but incorporates the ideas Justin Mason detailed about the SpamAssassin checksummer. Skip
Skip, thanks for replying! Skip Montanaro <skip@pobox.com> writes:
I use spambayes with procmail. The major issue is generally not one of getting messages classified, but of getting them trained.
I figured it would be; I think that's what I meant by "classified". I do have a folder full of accumulated spam. What has been your strategy for training? -- Dave Abrahams Boost Consulting www.boost-consulting.com
>> I use spambayes with procmail. The major issue is generally not one >> of getting messages classified, but of getting them trained. Dave> I figured it would be; I think that's what I meant by Dave> "classified". I do have a folder full of accumulated spam. What Dave> has been your strategy for training? Here's what I do. It's sensitive to my particular mail setup, so you can probably only use this as a rough guide. My mail reader is VM inside XEmacs. VM has a "l"abel command prefix. I added two new keys to its keymap, "h" and "s" (which were fortuitously unused) to copy messages to spam and ham folders: (defun copy-to-spam () (interactive) (vm-save-message (expand-file-name "~/tmp/newspam")) (vm-undelete-message 1)) (defun copy-to-nonspam () (interactive) (vm-save-message (expand-file-name "~/tmp/newham")) (vm-undelete-message 1)) (define-key vm-mode-map "ls" 'copy-to-spam) (define-key vm-summary-mode-map "ls" 'copy-to-spam) (define-key vm-mode-map "lh" 'copy-to-nonspam) (define-key vm-summary-mode-map "lh" 'copy-to-nonspam) ~/tmp/new{ham,spam} are then processed using a fairly simple shell script: #!/bin/bash export BAYESCUSTOMIZE=$HOME/hammie.opt cd ~/tmp base=new db=hammie.db # touch the messages up a bit to avoid spurious "clues" if [ -f ${base}ham -a -f ${base}spam ] ; then unheader.py -p 'X-VM|X-Hammie|X-Spam' ${base}ham > ${base}ham.clean unheader.py -p 'X-VM|X-Hammie|X-Spam' ${base}spam > ${base}spam.clean # do the deed hammie.py -d -p $db -g ${base}ham.clean -s ${base}spam.clean # save the files for later retraining cat ${base}ham.clean >> ${base}ham.clean.save echo "" >> ${base}ham.clean.save rm ${base}ham ${base}ham.clean cat ${base}spam.clean >> ${base}spam.clean.save echo "" >> ${base}spam.clean.save rm ${base}spam ${base}spam.clean else echo Missing ${base}ham and/or ${base}spam files fi I run the train script periodically to train on new ham and spam, then copy the resulting hammie.db file to where it's really used: % train Training ham (newham.clean): 12 Training spam (newspam.clean): 29 % cp -p hammie.db ~ This setup works fine for me, though probably won't be as attractive for people who aren't as addicted to the shell prompt as I am. Skip
Skip Montanaro <skip@pobox.com> writes:
>> I use spambayes with procmail. The major issue is generally not one >> of getting messages classified, but of getting them trained.
Dave> I figured it would be; I think that's what I meant by Dave> "classified". I do have a folder full of accumulated spam. What Dave> has been your strategy for training?
Here's what I do. It's sensitive to my particular mail setup, so you can probably only use this as a rough guide.
My mail reader is VM inside XEmacs.
I'm using GNUs, FWIW.
VM has a "l"abel command prefix. I added two new keys to its keymap, "h" and "s" (which were fortuitously unused) to copy messages to spam and ham folders:
Here's what I've never understood about this system: shouldn't it be enough to label spam? GNUs gives me a key to label a message as spam. If I collect all of those, shouldn't I be able to tell spambayes that everything in my INBOX that's been read and isn't in my SpamBox is ham?
(defun copy-to-spam () (interactive) (vm-save-message (expand-file-name "~/tmp/newspam")) (vm-undelete-message 1))
(defun copy-to-nonspam () (interactive) (vm-save-message (expand-file-name "~/tmp/newham")) (vm-undelete-message 1))
(define-key vm-mode-map "ls" 'copy-to-spam) (define-key vm-summary-mode-map "ls" 'copy-to-spam) (define-key vm-mode-map "lh" 'copy-to-nonspam) (define-key vm-summary-mode-map "lh" 'copy-to-nonspam)
~/tmp/new{ham,spam} are then processed using a fairly simple shell script:
#!/bin/bash
export BAYESCUSTOMIZE=$HOME/hammie.opt cd ~/tmp
base=new db=hammie.db
# touch the messages up a bit to avoid spurious "clues" if [ -f ${base}ham -a -f ${base}spam ] ; then unheader.py -p 'X-VM|X-Hammie|X-Spam' ${base}ham > ${base}ham.clean unheader.py -p 'X-VM|X-Hammie|X-Spam' ${base}spam > ${base}spam.clean
# do the deed hammie.py -d -p $db -g ${base}ham.clean -s ${base}spam.clean
# save the files for later retraining cat ${base}ham.clean >> ${base}ham.clean.save echo "" >> ${base}ham.clean.save rm ${base}ham ${base}ham.clean
cat ${base}spam.clean >> ${base}spam.clean.save echo "" >> ${base}spam.clean.save rm ${base}spam ${base}spam.clean else echo Missing ${base}ham and/or ${base}spam files fi
I run the train script periodically to train on new ham and spam, then copy the resulting hammie.db file to where it's really used:
% train Training ham (newham.clean): 12 Training spam (newspam.clean): 29 % cp -p hammie.db ~
This setup works fine for me, though probably won't be as attractive for people who aren't as addicted to the shell prompt as I am.
Well, I'm not sure I understand it yet, but I think I'll get there. Thanks! -- Dave Abrahams Boost Consulting www.boost-consulting.com
Dave> Here's what I've never understood about this system: shouldn't it Dave> be enough to label spam? GNUs gives me a key to label a message Dave> as spam. If I collect all of those, shouldn't I be able to tell Dave> spambayes that everything in my INBOX that's been read and isn't Dave> in my SpamBox is ham? I suspect you can use or adapt Neil Schemenauer's mboxtrain.py script to do what you want. I started doing things this way before that was an option though. >> This setup works fine for me, though probably won't be as attractive >> for people who aren't as addicted to the shell prompt as I am. Dave> Well, I'm not sure I understand it yet, but I think I'll get Dave> there. Yeah, it will probably take awhile. If you fetch your email via POP you might find the pop3proxy a better fit. It provides a web-based training interface. Skip
Skip Montanaro <skip@pobox.com> writes:
Dave> Here's what I've never understood about this system: shouldn't it Dave> be enough to label spam? GNUs gives me a key to label a message Dave> as spam. If I collect all of those, shouldn't I be able to tell Dave> spambayes that everything in my INBOX that's been read and isn't Dave> in my SpamBox is ham?
I suspect you can use or adapt Neil Schemenauer's mboxtrain.py script to do what you want. I started doing things this way before that was an option though.
Excellent! Thank you.
>> This setup works fine for me, though probably won't be as attractive >> for people who aren't as addicted to the shell prompt as I am.
Dave> Well, I'm not sure I understand it yet, but I think I'll get Dave> there.
Yeah, it will probably take awhile. If you fetch your email via POP you might find the pop3proxy a better fit. It provides a web-based training interface.
Nope; I'm using IMAP. Thanks, -- Dave Abrahams Boost Consulting www.boost-consulting.com
i think this speaks well to the point that training is a individual and manual proceess! :o) b Skip Montanaro wrote:
Here's what I do. It's sensitive to my particular mail setup, so you can probably only use this as a rough guide.
My mail reader is VM inside XEmacs. VM has a "l"abel command prefix. I added two new keys to its keymap, "h" and "s" (which were fortuitously unused) to copy messages to spam and ham folders:
(defun copy-to-spam () (interactive) (vm-save-message (expand-file-name "~/tmp/newspam")) (vm-undelete-message 1))
(defun copy-to-nonspam () (interactive) (vm-save-message (expand-file-name "~/tmp/newham")) (vm-undelete-message 1))
(define-key vm-mode-map "ls" 'copy-to-spam) (define-key vm-summary-mode-map "ls" 'copy-to-spam) (define-key vm-mode-map "lh" 'copy-to-nonspam) (define-key vm-summary-mode-map "lh" 'copy-to-nonspam)
~/tmp/new{ham,spam} are then processed using a fairly simple shell script:
#!/bin/bash
export BAYESCUSTOMIZE=$HOME/hammie.opt cd ~/tmp
base=new db=hammie.db
# touch the messages up a bit to avoid spurious "clues" if [ -f ${base}ham -a -f ${base}spam ] ; then unheader.py -p 'X-VM|X-Hammie|X-Spam' ${base}ham > ${base}ham.clean unheader.py -p 'X-VM|X-Hammie|X-Spam' ${base}spam > ${base}spam.clean
# do the deed hammie.py -d -p $db -g ${base}ham.clean -s ${base}spam.clean
# save the files for later retraining cat ${base}ham.clean >> ${base}ham.clean.save echo "" >> ${base}ham.clean.save rm ${base}ham ${base}ham.clean
cat ${base}spam.clean >> ${base}spam.clean.save echo "" >> ${base}spam.clean.save rm ${base}spam ${base}spam.clean else echo Missing ${base}ham and/or ${base}spam files fi
I run the train script periodically to train on new ham and spam, then copy the resulting hammie.db file to where it's really used:
% train Training ham (newham.clean): 12 Training spam (newspam.clean): 29 % cp -p hammie.db ~
This setup works fine for me, though probably won't be as attractive for people who aren't as addicted to the shell prompt as I am.
Skip
_______________________________________________ Spambayes mailing list Spambayes@python.org http://mail.python.org/mailman/listinfo/spambayes
You can dispense with the PYCKSUM stuff, though I find it does delete a fair number of duplicate spams. I get email for a large number of aliases at the same address however. YMMV. I've attached the version of the script which I use. It's similar to the loosecksum.py script in the Spambayes utilities directory, but incorporates the ideas Justin Mason detailed about the SpamAssassin checksummer.
Maybe you can update integration.txt with these pertinent bits? Also, perhaps check in your checksummer? c'est moi - TimS http://www.fourstonesExpressions.com http://wecanstopspam.org There are 10 kinds of people in the world: those who understand binary, and those who don't.
participants (4)
-
bill parducci -
David Abrahams -
Skip Montanaro -
Tim Stone - Four Stones Expressions