Mailman 3 September 2003 - SpamBayes

Outlook plugin - training
by Moore, Paul Dec. 3, 2003

Dec. 3, 2003

When the Outlook plugin filters mails, it classifies them as either spam or potential spam, and can put them in appropriate folders. In the spam/potential spam folders, there is a "Recover from Spam" button available, and in other folders there is a "Delete as spam" button. These buttons add the message to the training database as well as taking the appropriate action. One thing I don't see, however, is a means of confirming the classifier's decisions as correct. A "yes, that is spam" button for the spam folder, and a "yes, that's ham" button in non-spam folders. As I'm starting from a very small message base, I worry that correct classifications are still somewhat based on "luck", and training based on correct decisions would help to increase both my and the classifier's confidence level. I can do this by regular retraining, but that has 2 disadvantages: it's much clumsier than simply clicking on a "clever boy!" button, and it relies on me not deleting messages until I do a training run. Much of the ham I get is "read and forget", so I'd rather delete immediately. When I get a chance to dive into the code, I'll see how hard this would be to implement. Paul.

7 8

RE: [Spambayes] Outlook plugin - training
by Piers Haken Dec. 3, 2003

Dec. 3, 2003

I don't believe you need this. I think that the classifier automatically trains on messages as they arrive (or at least on messages that it's sure about). You only need to retrain if it has made a mistake, or if it's unsure. Piers. > -----Original Message----- > From: Moore, Paul [mailto:Paul.Moore@atosorigin.com] > Sent: Wednesday, November 06, 2002 2:09 AM > To: Spambayes (E-mail) > Subject: [Spambayes] Outlook plugin - training > > > When the Outlook plugin filters mails, it classifies them as > either spam or potential spam, and can put them in > appropriate folders. > > In the spam/potential spam folders, there is a "Recover from > Spam" button available, and in other folders there is a > "Delete as spam" button. These buttons add the message to the > training database as well as taking the appropriate action. > > One thing I don't see, however, is a means of confirming the > classifier's decisions as correct. A "yes, that is spam" > button for the spam folder, and a "yes, that's ham" button in > non-spam folders. > > As I'm starting from a very small message base, I worry that > correct classifications are still somewhat based on "luck", > and training based on correct decisions would help to > increase both my and the classifier's confidence level. > > I can do this by regular retraining, but that has 2 > disadvantages: it's much clumsier than simply clicking on a > "clever boy!" button, and it relies on me not deleting > messages until I do a training run. Much of the ham I get is > "read and forget", so I'd rather delete immediately. > > When I get a chance to dive into the code, I'll see how hard > this would be to implement. > > Paul. > > _______________________________________________ > Spambayes mailing list > Spambayes(a)python.org http://mail.python.org/mailman/listinfo/spambayes >

10 18

Re: Spambayes Digest, Vol 59, Issue 9
by erik＠sky.cz Nov. 6, 2003

Nov. 6, 2003

Hi all, Mark replied to my recent message on installation problems in private, and I replied back in private... since there's been activity from Mark on the list but no reply to that e-mail, I'll take the liberty of continuing this one on the list. The basics again: - I first discovered the problem in the 002 binary, but it does repro in the 003. - both repro machines run Outlook 2K CZ - one runs W98SE CZ and the other W2K CZ - Reproduced in W2K under both CZ-CZ locale and EN-US regional settings - Addin joins the list in Outlook after installation, but cannot be successfully activated - each return to the COM add-ins dialog finds it unchecked. - The original report was for W2K CZ / binary 2; see the list a few days back for details. - Here is the behavior for W98SE/CZ regional settings/binary 3: Error connecting to Outlook! Traceback (most recent call last): File "out1.pyz/addin", line 800, in OnConnection File "out1.pyz/manager", line 682, in GetManager File "out1.pyz/manager", line 226, in __init__ File "out1.pyz/manager", line 487, in LoadConfig File "out1.pyz/ntpath", line 101, in join UnicodeDecodeError: 'ascii' codec can't decode byte 0xed in position 9: ordinal not in range(128) ERROR: 'There was an error initializing the SpamBayes addin\r\n\r\nPlease re- start Outlook and try again.' Traceback (most recent call last): File "out1.pyz/addin", line 800, in OnConnection File "out1.pyz/manager", line 682, in GetManager File "out1.pyz/manager", line 226, in __init__ File "out1.pyz/manager", line 487, in LoadConfig File "out1.pyz/ntpath", line 101, in join UnicodeDecodeError: 'ascii' codec can't decode byte 0xed in position 9: ordinal not in range(128) SpamBayes - Disconnecting from Outlook Addin terminating: 0 COM client and 1 COM servers exist. - the error with 003 on W2K was, as far as I can recall, mostly the same, except for the "can't decode byte XXXX" - a different byte was cited, AFAICR. Erik Piper

5 8

spamBayes is great, thank you all!
by AndreasK Oct. 6, 2003

Oct. 6, 2003

What a great program! Many many thanks ...! SpamBayes works just fine, removes 80%-90% SPAM automatically and learns with each manually deleted spam mail. Wow. Great. Only 4 non-spam mails (out of hundreds) were wrongly treated as spam so far - and they actually looked like Spam, I have to admit. The SPAM catastrophy started in ~1996, when mainstream capitalism discovered the web, culminated in about 800 spam mails last week, and ended when I found SpamBayes, actually by reading a web-article which compared SpamBayes with some other software. The one drawback is that all mails are still fetched from the POP3-box which is really annoying with these 116kB MS-Update.exe-Mails. But still, that only costs connection-money, not my precious time... The Outlook plugin is good, too. Seamless integration is a must. If you have German speaking friends, send them my signature below. It explains WHAT and HOW TO - but also shows one urgent need, because many Dummies might not be able to do all 4 steps ;-) Publish an installer which includes all 4 programs: Python, Python-Win32, SpamBayes and SpamBayes-Outlook-Setup ! Would that be a good idea? Obviously you have to ask Python.org - but they'd gain a lot by being installed on many more computers... ciao, Andreas ---- Hey, super: Ich hab das Unglaubliche geschafft, nämlich SPAM loszuwerden! Endlich, es war zuletzt die absolute Hölle geworden - und nun ist sie scheinbar überstanden... Es gibt ein kostenloses Python-Script "SpamBayes", das sich nahtlos in Outlook, etc. einbaut und mittels einer trainierbaren Bayes-Routine alle eMails, die nach SPAM aussehen, automatisch in zwei Ordner wegsortiert - "wahrscheinlich" (15%) und "fast sicher"(90%). Du musst allerdings einiges downloaden, da Dein Rechner vermutlich die moderne Programmiersprache Python noch nicht spricht, deshalb erstmal Python ("current production version"): http://www.python.org/download/ und die Python-Win32-Routinen: http://starship.python.net/crew/mhammond/win32/Downloads.html danach dann von Sourceforge (die wichtigste OpenSource-Community) https://sourceforge.net/project/showfiles.php?group_id=61702 das Hauptprogramm spambayes-1.XXX.zip und das Outlook-Addin SpamBayes-Outlook-Setup-0XXX.exe für andere POP-Clients als Outlook gäbe es wohl auch Unterstützung: http://spambayes.sourceforge.net/windows.html Tja und dann einfach alles in obiger Reihenfolge installieren. Am besten Du hast schon zwei vorsortierte Ordner (zB 400 gewünschte und 1300 SPAM-Mails), um das Ersttraining vorzunehmen - wenn Du bisher immer alle SPAMs sofort gelöscht hast, ist das auch nicht so schlimm, dann drückst Du einfach bei jeder ankommenden SPAM mail den neuen gelben Button in Outlook "Delete as SPAM" - und nach und nach wird SpamBayes immer mehr von alleine erkennen. Toll, nech?

5 7

Compiled proxy as exe
by Stefan Grönberg Oct. 1, 2003

Oct. 1, 2003

Could it be any chance to get the pop3proxy compiled as an exe? Because i tried to get it running under Windows Xp Home Edition, and it just complains about something (forgot to copy that) and is there a way compleatle being able to remove spambayes outlook-addin from..outlook? because it makes the loading/start-time get around 20 secs instead of 1-3 secs to start. /stefan

3 2

RE: Need more training messages
by Coe, Bob Oct. 1, 2003

Oct. 1, 2003

> From: Skip Montanaro [mailto:skip@pobox.com] > Sent: Tuesday, September 30, 2003 10:22 AM > To: Coe, Bob > Cc: spambayes(a)python.org > Subject: RE: [Spambayes] RE: Need more training messages > > > Bob> Well, either... > > Bob> - There are users of Spambayes in orthographically diverse > Bob> languages, in which case those users should be able to > Bob> contribute some ham samples, as well as their experience > Bob> regarding the accuracy of Spambayes's classification in > Bob> their languages, or > > Bob> - There aren't, in which case Spambayes's performance on ham > Bob> written in such languages is (at least for the time being) > Bob> irrelevant. > > I agree those are the two cases (<wink>), but don't agree with your > conclusions. If we distribute SpamBayes with a default db that > performs miserably on Asian ham, we're not likely to win a lot of > support. I would prefer that the default database process what > current users normally encounter in a reasonable way. ... I think I detect an a priori confidence that the same version of the Spambayes classifier, if properly trained, can work effectively on both European and Asian languages. I wonder if that confidence isn't unduly optimistic. For example, ... I presume that the Spambayes classifier tokenizes the incoming character stream according to an algorithm that depends heavily on clearly defined word markers (spaces and punctuation marks) that are largely absent, or at least less prominent, in Chinese. But if you try to tokenize the individual characters of written Chinese, you'll find that they're much more context sensitive than the words of an English sentence are. To put it another way, many Chinese "words" consist of two or three characters, and the information theoretical redundancy of a given character is rather low (not as low as in spoken Chinese, but low enough to be a problem). Which means, I suspect, that the tokenizer, if it's to be effective on Chinese text, will require somewhat more lookahead capability than it probably has now. The problem isn't insurmountable, of course, but I think it casts doubt on the "One size fits all" approach. Such problems as this probably don't affect the classifier's ability to differentiate Chinese text from English ham. And if you're a user for whom all Chinese text is spam, that should be good enough. But differentiating Chinese spam from Chinese ham may be beyond the capability of the current classifier. (By way of disclaimer, I don't speak Chinese and I haven't seen the internals of the Spambayes classifier. But I've written compilers and other text processing software, so I'm not a total novice either.) Bob MIS Department, City of Cambridge 831 Massachusetts Ave, Cambridge MA 02139 · 617-349-4217 · fax 617-349-6165

3 2

Error from pop3proxy
by Alan Campbell Oct. 1, 2003

Oct. 1, 2003

I got this error from pop3proxy running on my W2K server :- error: uncaptured python exception, closing channel <sb_smtpproxy.BayesSMTPProxy connected 192.168.1.100:1571 at 0x1833aa8> (bsddb._db.DBRunRecoveryError:(-3098 2, 'DB_RUNRECOVERY: Fatal error, run database recovery -- fatal region error det ected; run recovery') [C:\Python23\lib\asyncore.py|read|69] [C:\Python23\lib\asy ncore.py|handle_read_event|387] [C:\Python23\lib\asynchat.py|handle_read|136] [C :\spambayes-1.0a6\scripts\sb_smtpproxy.py|found_terminator|222] [C:\spambayes-1. 0a6\scripts\sb_smtpproxy.py|onProcessData|318] [C:\spambayes-1.0a6\scripts\sb_sm tpproxy.py|train|433] [C:\spambayes-1.0a6\spambayes\classifier.py|learn|211] [C: \spambayes-1.0a6\spambayes\classifier.py|_add_msg|346] [C:\spambayes-1.0a6\spamb ayes\storage.py|_wordinfoset|270] [C:\Python23\lib\shelve.py|__setitem__|130] [C :\Python23\lib\bsddb\__init__.py|__setitem__|90]) I was attempting to forward the following spam for training (it had been clasified as ham) :- Hey, just got this new toy. Saves me 30% on my gas, proven to work and seen all over the place. Mine paid for itself in 2 weeks. A worthwhile investment, and guaranteed to work. Can't go wrong, go see what I mean. I'll be saving 30% for lfe at the pumps now. Later. http://www.d3kn.org/default.asp?axel=1465 take me 0ff please. http://www.vnmc.org/out1v/pre-rem2e.asp g dfg fd g dfg dg iqydcpnqwsh xhdd lrvxe rad pcw j dljuaxayf sxq ngq vltc uk ksnjqaplfmptwvwlblgma --- mailto:alan@mullen.demon.co.uk http://www.mullen.demon.co.uk/

3 3

RE: [Spambayes] help
by Meyer, Tony Sept. 30, 2003

Sept. 30, 2003

>> In general, you should >> be fine (although using 1.0a5 rather than 1.0a6 might be advisable). > Why the warning about 1.0a6? There's a bug in it that means that you need to restart SpamBayes after every time you change your options via the web interface. Not serious, just annoying, but it could be worth using 1.0a5 and then getting 1.0a7/1.0b1 when it comes out. There has been talk on -dev about putting 1.0a7 out soon to address this, although I'm not sure when we'll find a chance to do it. =Tony Meyer

1 0

RE: [Spambayes] help
by Meyer, Tony Sept. 30, 2003

Sept. 30, 2003

> I purchase last week and Norton AntiSpam arrived on Saturday. > there is a problem with it on OE6 on windows XP. See below > from Samantec site. Is any of this about spambayes? It reads like it's all about Norton AntiSpam. If it is, you're really asking in the wrong place! I doubt anyone here will be able to help. > Question is are there any Spambayes problems with OE6 and XP? As far as we know there aren't any problems with SpamBayes specific to Outlook Express. If you want to read about the bugs we do know about, take a look at the list on <http://sf.net/projects/spamabyes>. In general, you should be fine (although using 1.0a5 rather than 1.0a6 might be advisable). =Tony Meyer

2 1

RE: [Spambayes] POP3 Training Question
by Meyer, Tony Sept. 30, 2003

Sept. 30, 2003

> On the developer's list I've raised the issue of having a > configurable default for each section. (Now it defaults to > Defer/Ham/Spam, I would prefer Defer/Discard/Discard so I > would normally only be training on errors I correct on a > per-message basis.) We'll see if that shows up in the future. This should have showed up by now for anyone using latest cvs, even given the sourceforge delay. It won't make it into 1.0a7/1.0b1/1.0, though, since the focus there is stability rather than better features :) =Tony Meyer

1 0