Mailman 3 November 2004 - SpamBayes

Re: [spambayes-dev] A query on Frequently Asked Question 4.6
by Remi Ricard Nov. 1, 2004

Nov. 1, 2004

Hi Harry, > I'll try and answer your queries as well as I can. > > Good >"SpamBayes POP3 Proxy Version 1.0 (July 2004) >Spambayes Web Interface, 19:32 on Monday November 01 2004. Spambayes.org > > > I don't know what you mean when you say, "If your use sb_server they should be...". Until I set the filter in 'Virtual Access' all of the spam had landed into my Inbox. Then with the filter in place they all wound up in that 'Spam' folder which was fine. > **** I've just noticed in the Advanced Configuration section... > > With this info I can say that you are using a recent version of sb_server (This is the command line name for the Spambayes POP3 proxy). > > >>>Train when filtering: Train when filtering? After filtering a message, hammie can then train itself on the judgement (ham or spam). This can speed things up with a procmail-based solution. If you do enable this, please make sure to retrain any mistakes. Otherwise, your word database will slowly become >>> >>> >useless.<< > ...I left that set at its default that is with the radio button at 'No'; which still reads as being the correct position but perhaps it should be at 'Yes'. Perhaps I'll try running it on 'yes' a bit later not at the moment till I hear from you. > > The problem with this is if you receive more spam than ham your database will be more and more unbalance. I think you should enable this after you have train on some spam and ham then turn this options off after you find SpamBayes to be accurate enough. >>> If so how do I go about accessing this that I might reduce the amount of >>>spam to something near the same 'ham' figure of 1,725? >>> >>> >>> >>> >>You can do it simply by not training on all spam that comes in. >>The problem now is you can't reduce the number of spam you can only add >>info to the database. >>So you can train on more ham to have a balanced database. >> >> > > Oh, my goodness, by "...simply not training on all..."; so does the mean I should have been using the 'discard' radio buttons on lots, if not all of the spam that came in around the time that, "Warning: you have much more spam than ham - SpamBayes works best with approximately even numbers of ham and >spam." message appeared? As you said, 'simply' - dumb. > > Usually if there is a button in the web interface the button is there for something ;-) >=============== > > >>But I think, it might be better to start from scratch >> >> > > Will that be accomplished simply by an uninstall using Add/Remove programs and then a re-install? > As Tony said in another mail you can save/rename the old databases and restart from scratch. If something breaks you will have the old ones. Look for the files hammie.db and spambayes.messageinfo.db in c:\Documents and Settings\your_user_id\Application Data\Spambays\Proxy > Why I'm asking that is, won't there be some data files that have to be cleared out too? Or will the Add/Remove action get rid of any such files? > > I don't know since I'm using an old version installed from cvs (If it works don't fix it) Remi

1 0

RE: [Spambayes] Problems
by Reed Odeneal Nov. 1, 2004

Nov. 1, 2004

We tried: Renamed the database, as instructed Re-trained from junk suspects/ham After trying this, the problem still exists, Recovering junk suspects to the inbox takes 30 seconds or more for the email to show up into the inbox. It does the same thing if I do a drag and drop or by using the toolbar. The database size was right around 1.6 megs before it was renamed/recreated. Ran a detect and repair on outlook2k3, and the problem still exists. Any more recommendations? -----Original Message----- From: Tony Meyer [mailto:tameyer@ihug.co.nz] Sent: Thursday, October 28, 2004 8:03 PM To: Reed Odeneal; spambayes(a)python.org Subject: RE: [Spambayes] Problems > The problem that we are having is when you click to > recover an email from spam, out of the Junk Suspects > folder, it takes AT LEAST 15 seconds for the email to > show up in the inbox. It did not use to do this. How many messages are in your database? (The SpamBayes Manager dialog shows this). If you move a message manually (e.g. drag and drop), is it quick? If so, then it's the training that's taking a long time (which is the mostly likely explanation). You could try renaming the database files ("Show Data Directory" on the Advanced tab of the Manager dialog will show you them - they are default_bayes_database.db and default_message_database.db), and retraining, and see if that fixes the problem. If it does, we would be interested to know, as there have been a couple of other 'slow training' reports recently, which indicates that there might be a problem to fix. =Tony.Meyer -- Please always include the list (spambayes(a)python.org) in your replies (reply-all), and please don't send me personal mail about SpamBayes. http://www.massey.ac.nz/~tameyer/writing/reply_all.html explains this.

1 0

SpamBayes feature suggestion
by C B Nov. 1, 2004

Nov. 1, 2004

Am so far impressed with SpamBayes' intuitiveness, ease of use, and - though it's not quite near perfect yet - its accuracy... actually I have a suggestion and a half. 1) Would be great if SpamBayes could suggest and implement a settings change at install that lets a person right-click on their junk e-mail folder and see a menu selection for 'empty junk e-mail folder' - just like the trash folder. Maybe similar functionality for the junk suspects folder. 2) It might get prohibitively bulky to start processing individual blacklists - or for that matter whitelists - as part of SpamBayes (which after all is supposed to be Bayesian)... but if there's a way to do that fast, it would help. I find Microsoft Outlook's message rules processing way too slow to want to use it on incoming e-mail... no matter how few things I'm asking it to find. Carry on. :) Great work... so far better than a commercial packaged- with-updates anti-spam consumer product I've tried. Care2 make the world greener! Support Care2 Email: 1,400 whales may be killed this year. Take action! http://www.care2.com/go/z/17954

1 0

Re: [spambayes-dev] A query on Frequently Asked Question 4.6
by Remi Ricard Nov. 1, 2004

Nov. 1, 2004

Hi Harry, > The FAQ item 4.6 above addressed something that I've wondered about since >I got SB going around July this year, that is, "Where does all the 'saved' spam >go?" <smile>. > I do have a lot of spam now, > 16,054 spams and 1,725 hams. > The thing is where do I go to cut down on the amount of spam? > I see the comment, "Warning: you have much more spam than ham - SpamBayes >works best with approximately even numbers of ham and spam." in the 'Status and >Configuration' section. > I'll have another look and see if I can figure out where all the spam is >located... > > It could help if you tell us which version of Spambayes and how you use it. If you use sb_server they should be in your inbox or if you have set a filter rule then you should not. If you use something else they should be in one of your folder since spambayes don't delete anything. >================ >4.3 How do I train SpamBayes (forward/bounce method)? > >Alternatively, when you receive an incorrectly classified message, you can >forward it to the SMTP proxy for training. If the message should have been >classified as spam, forward or bounce the message to spambayes_spam@localhost, >and if the message should have been classified as ham, forward it to >spambayes_ham@localhost. You can still review the training through the web >interface, if you wish to do so. You should ensure that the "lookup message in >cache" option is set to True/Yes before you use this. > >Note that some mail clients (particularly Outlook Express) do not forward all >headers when you bounce, forward or redirect mail. We do not recommend using >the SMTP proxy with these clients. >=============== > ...I saw from that item 4.3 that there are locations at... > spambayes_spam@localhost > ...and at... > spambayes_ham@localhost > ...are these locations on this my computer or are they located at my ISP's >server? > > It is "on" your computer what really append is your email program will send the message with spambayes_spam@localhost to the port x (in think it is 25) on your computer then spambayes (working as a proxy) will see the message and grab it to train on it and then discard it. > This is a standalone machine with a cable-modem broadband connection to >NTL... > I've just used XP's Search facility to see if, spambayes_spam@localhost , >exists as a file on this machine it didn't find such and I didn't expect that >it would. > > I think you can find the info in the file options.py but the info might be split in 2 part "spambayes_spam" and localhost since the email address for the spam is configurable (spambayes _spam) > Are these thousands of spam actually located on my ISP's server, that is >on NTL's server? > > No But this could be yes if your email program is set to keep email on server. > If so how do I go about accessing this that I might reduce the amount of >spam to something near the same 'ham' figure of 1,725? > > You can do it simply by not training on all spam that comes in. The problem now is you can't reduce the number of spam you can only add info to the database. So you can train on more ham to have a balanced database. But I think, it might be better to start from scratch > > Excuse the long explanation. > > It is always better to have more info so don't excuse yourself ;-) See the start of the email (We need more info......) Remi

1 0

RE: [Spambayes] IMAP Error: Must specify server details first
by Tony Meyer Nov. 1, 2004

Nov. 1, 2004

> Wow, Tony, such service! Do you find time for anything else? Well, a great many other people also work on the project, of course :) And sometimes (like now) replies can take some time (sorry) - and even then, there are still 212 spambayes(a)python.org messages that I would like time to address after this one... > However, I'm still having trouble. So here's the dumb newbie > question of the day: HOW do I stop the script? Ctrl-C is a perfectly valid way of stopping it - however, try not to do this when it's in the midst of a training session. I'm happy to accept suggestions of alternative methods of shutting it down. > Now, tonight, when I start the script: > > sb_imapfilter.py -b -v > > I'm not getting a browser window. The verbose output gives me > three messages, which look okay, with no errors, but no > browser. And opening a browser window and entering the URL > manually gives me an error. > > I'd send you the log, but I don't know where to find it. imapfilter doesn't have a log, just whatever gets printed to the output window. (This is the same if you run any of the scripts from source). It's very strange that you don't get an error, and can't access the web interface. Has anything else changed in this time? If you telnet to port 8880 when imapfilter is running can you connect? (Or you could use netstat to see if it's listed). =Tony Meyer --- Please always include the list (spambayes(a)python.org) in your replies (reply-all), and please don't send me personal mail about SpamBayes. This way, you get everyone's help, and avoid a lack of replies when I'm busy.

3 2

spam with 100% score not moved to junk ++
by Cecconi, Enrico (E.) Nov. 1, 2004

Nov. 1, 2004

Hi, the following 2 emails have been filtered and score 100% as spam but were not moved to my designated junk folder. <<cheap oem software shipping worldwide. Andrew butane>> <<Heighten sensation>> any ides why? rgds, Enrico Cecconi M.Sc. Systems Engineer (Audio) Visteon (UK) Ltd. GB-28/H Electronics Tech. Centre Dunton Engineering Centre Laindon SS15 6EE * +44-1268-497813 email: ececconi(a)visteon.com > CONFIDENTIALITY NOTICE: This e-mail message including attachments, if any, > is intended only for the person or entity to which it is addressed and may > contain confidential and /or privileged material. Any unauthorized review, > use, disclosure or distribution is prohibited. If you are not the intended > recipient, please contact the sender by reply e-mail and destroy all > copies of the original message. If you are the intended recipient but do > not wish to receive communications through this medium, please do advise > the sender immediately. forgot to say am using Windows 98SE, Outlook 2000 (9.0.0.2711), Spambayes outlook addin Bin Ver 1.0rc2 (june2004) <<spambayes1.log>>

1 0

spam with 100% score not moved to junk
by Cecconi, Enrico (E.) Nov. 1, 2004

Nov. 1, 2004

Hi, the following 2 emails have been filtered and score 100% as spam but were not moved to my designated junk folder. <<cheap oem software shipping worldwide. Andrew butane>> <<Heighten sensation>> any ides why? rgds, Enrico Cecconi M.Sc. Systems Engineer (Audio) Visteon (UK) Ltd. GB-28/H Electronics Tech. Centre Dunton Engineering Centre Laindon SS15 6EE * +44-1268-497813 email: ececconi(a)visteon.com > CONFIDENTIALITY NOTICE: This e-mail message including attachments, if any, > is intended only for the person or entity to which it is addressed and may > contain confidential and /or privileged material. Any unauthorized review, > use, disclosure or distribution is prohibited. If you are not the intended > recipient, please contact the sender by reply e-mail and destroy all > copies of the original message. If you are the intended recipient but do > not wish to receive communications through this medium, please do advise > the sender immediately. > > <<spambayes1.log>>

1 0

RE: [spambayes-dev] A query on Frequently Asked Question 4.6
by Tony Meyer Nov. 1, 2004

Nov. 1, 2004

> The FAQ item 4.6 above addressed something that I've > wondered about since I got SB going around July this year, > that is, "Where does all the 'saved' spam go?" <smile>. The messages that end up in your mailer stay there, unless you tell it to do something with it (with the Outlook plug-in, this means they stay in the designated spam folder). The copies of the messages that sb_server makes are moved to a folder where they are stored for a set amount of time (by default 7 days) and then are permanently deleted (like if you choose "discard" from the review page). The information that SpamBayes collects from messages that it's trained on is stored in a database (called hammie.db or default_bayes_database.db by default). This is just a list of all the tokens ('words') that have been in all of those messages and a count of how many good & how many bad messages each token has been seen. It's not possible (with an unpatched SpamBayes) to go backwards from the database to a particular message - to 'untrain', you have to have a copy of the original message around. > I do have a lot of spam now, > 16,054 spams and 1,725 hams. > The thing is where do I go to cut down on the amount of spam? > I see the comment, "Warning: you have much more spam > than ham - SpamBayes > works best with approximately even numbers of ham and spam." > in the 'Status and > Configuration' section. Firstly, the golden rule is "if it's working, don't change anything" (where working would be defined as classifying well enough that you are happy with the results). All sets of mail are different, but it does appear that results are better when the balance is reasonably even - and certainly that large imbalances (eg 50::1) can cause weird results. You're at 9.3::1, which isn't extremely bad, although it wouldn't be surprising if results weren't optimum. That's quite a large database, too, really. Many of the developers have databases with less than a thousand messages in total, with good results. However, at the moment, it's only possible to add to the db, not remove from it. The general practice is to retrain from scratch when that's desired - since SpamBayes learns quickly, that's not usually a problem. I'd suggest reading (if you haven't already) the information about training on the wiki: <http://entrian.com/sbwiki>. In particular, doing 'mistake based training' or 'nonedge training' may help keep the db small, *may* help the imbalance, and may help the results. We're trying to figure out ways to help with the imbalance issue for future releases, although there's no clear solution as yet. (You could put aside the current databases, try out, with fresh ones, a new training regime and see if that works. If it doesn't, you can just put the old database back!). > 4.3 How do I train SpamBayes (forward/bounce method)? [...] > ...I saw from that item 4.3 that there are locations at... > spambayes_spam@localhost > ...and at... > spambayes_ham@localhost > ...are these locations on this my computer or are they > located at my ISP's server? These aren't locations, they are special email addresses. The idea is: All your outgoing mail goes via a SMTP server (at your ISP, probably). Rather than have to use the review page, you can bounce/forward messages to SpamBayes to indicate that they need to be trained. The way this works is to have all outgoing mail go through a proxy on its way to the SMTP server (like incoming mail goes through a proxy on its way from the POP server). This SpamBayes SMTP proxy examines the recipient list of each outgoing message and if it is going to one of those two special addresses (which don't exist), then they are intercepted and the messages are used for training instead. Some people use this method for training, but it's not particularly popular, and does have some flaws. It's better to use the review page in most cases. (There's another, unfinished, script that offers all-in-the-mail-client training, which I have more hope for). =Tony.Meyer -- Please always include the list (spambayes(a)python.org) in your replies (reply-all), and please don't send me personal mail about SpamBayes. http://www.massey.ac.nz/~tameyer/writing/reply_all.html explains this.

1 0

spambay
by Joseph Paonessa Nov. 1, 2004

Nov. 1, 2004

I can't uninstall spambays. I have XP home and using microsoft access 2003..I tried going to add or remove...It was removed but its still in my outlook...then I right clicked start and tried to 1 by 1 get rid of programs..but its still there. I mean here... please advise

1 0