Mailman 3 June 2003 - spambayes-dev

Re: [Spambayes] Database cleaning?
by Matthew Dixon Cowles June 1, 2003

June 1, 2003

[Alex Popiel on nonsense words in spam] > Yes, those words cause database pollution, and yes, they can be > weeded out with just a handful of lines of code... but it's hard to > tell which hapax legomena will be useless, and which will soon get > reinforced by other occurences, so it's (IMNSHO) generally not > worth the hassle. With an eye toward reducing the size of the database, I instrumented the classifier a while ago and found a very strong indication that that's true. Indeed, hapaxes often figured in scoring. I didn't bother to calculate exact numbers because the results were strong enough to persuade me that removing hapaxes wasn't a useful strategy. I tore that code out and instead hacked the classifier so that I could determine how soon after a word figures in scoring that it's used again. I think that the results are at least slightly interesting. Note that the histogram below is log scaled. Unique tokens used for scoring 60627 Used Once 17388 Days prev Count Histogram is log scaled 0 903644 ************************************************** 1 27694 ************************************* 2 15121 *********************************** 3 7024 ******************************** 4 4694 ******************************* 5 3634 ****************************** 6 3134 ***************************** 7 2443 **************************** 8 1697 *************************** 9 1340 ************************** 10 982 ************************* 11 801 ************************ 12 671 ************************ 13 871 ************************* 14 630 ************************ 15 494 *********************** 16 374 ********************** 17 343 ********************* 18 227 ******************** 19 216 ******************** 20 199 ******************* 21 226 ******************** 22 126 ****************** 23 114 ***************** 24 55 *************** 25 22 *********** 26 49 ************** My mail may not be representative in ways that exaggerate the slope here. Specifically, I read postmaster, webmaster, etc addresses for several domains so it's common for me to get multiple copies of the same spam. Regards, Matt

2 2

[ spambayes-Bugs-747107 ] SpamBayes leaves duplicate messages
by SourceForge.net June 1, 2003

June 1, 2003

Bugs item #747107, was opened at 2003-06-01 11:28 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=747107&group_i… Category: Outlook Group: None Status: Open Resolution: None Priority: 5 Submitted By: Eric Brown (ericbr) Assigned to: Mark Hammond (mhammond) Summary: SpamBayes leaves duplicate messages Initial Comment: I am subscribed to a number of different mailing lists, and I have a number of Outlook rules (set up via Rules Wizard) to route the different mailing lists into different folders. When the rules are of the form: Apply this rule after the message arrives with <words> in the subject move it to the <specified> folder and a matching message comes in, SpamBayes seems to leave multiple copies of that message in the inbox. This is very frustrating. If the rule is of the form: Apply this rule after the message arrives sent to <email address> move it to the <specified> folder and a matching message comes in, duplicate messages do not occur. So it seems to be some sort of odd interaction with Outlook Rules. I'm running Outlook 2002 SP 2 on Windows XP SP 1a (with current hotfixes). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=747107&group_i…

1 0

[ spambayes-Bugs-746909 ] Conflicting with Message Rules
by SourceForge.net June 1, 2003

June 1, 2003

Bugs item #746909, was opened at 2003-06-01 00:39 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=746909&group_i… Category: Outlook Group: None Status: Open Resolution: None Priority: 5 Submitted By: Henry Ho (cornflakeboy) Assigned to: Mark Hammond (mhammond) Summary: Conflicting with Message Rules Initial Comment: If you have Outlook message rules defined (i.e. moving messages automatically into different folders), and a message comes in, Spambayes will move it into spam or possible spam folders as it should. But Outlook will move that message to the folder defined by the rule, which subsequently makes Spambayes think it's good and will train itself accordingly. An example rule is: Apply this rule after the message arrives through the myemail(a)mydomain.com account, move it to the My Personal folder The above rule was created to sort multiple email accounts into proper folders. I have Spambayes configured to watch my Inbox and my "My Personal" folder. Perhaps we can make Spambayes process messages after Outlook's done running its rules? I am using Outlook 2002 SP-1 Thanks! Great program! I have been recommending it to everyone who owns a PC! :) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=746909&group_i…

1 0