Mailman 3 August 2006 - spambayes-dev

Latest CVS update, Ocrad for Windows
by skip＠pobox.com Sept. 4, 2006

Sept. 4, 2006

I updated the OCR capabilities a bit more today. I added more intelligent assembly of split images into a single image after noticing that the spammers don't simply chop up multi-part GIF images horizontally. I also added a couple extra options (ocrad_scale and ocrad_charset) which control the image scaling factor (default is 2) and character set (default is "ascii") Ocrad uses. Scaling the image by a factor of 2 was a pretty obvious win: false positive percentages 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied 0.000 0.000 tied won 0 times tied 5 times lost 0 times total unique fp went from 0 to 0 tied mean fp % went from 0.0 to 0.0 tied false negative percentages 4.213 4.213 tied 1.404 0.843 won -39.96% 3.371 2.809 won -16.67% 2.528 2.247 won -11.12% 4.213 3.652 won -13.32% won 4 times tied 1 times lost 0 times total unique fn went from 56 to 49 won -12.50% mean fn % went from 3.14606741573 to 2.75280898876 won -12.50% Scaling by a factor of three was even better in the false negative department but regressed a bit in the false positive category so I checked Options.py in with a default scaling factor of 2. A couple things could stand to be further tested: * I have no idea how good Ocrad's scaling algorithm is. It's possible that PIL or NetPBM's scaling code is better. If so, it would make sense to scale the images before feeding to Ocrad. * The images I've see so far were all plain English, so I blindly made ascii the default charset. The other choices were iso-8859-9 and iso-8859-15. I simply assumed ascii would be the most appropriate default, but didn't test it. Finally, I put together a really simpleminded Ocrad-for-Windows release based upon the ocrad.exe binary that Tony built. Check the Files section of the SpamBayes project site: http://sourceforge.net/project/showfiles.php?group_id=61702 and grab ocrad-cygwin. There are a few caveats: 1. I don't do Windows. (No, really, I don't, strange as that may seem.) This is no fancy-schmancy point-and-shoot Windows installer. It's just a simple zip file with the Ocrad 0.15 distribution, Tony's .exe file and the patch he applied to the source. 2. I don't do Windows. The code I've written so far has been done entirely on my Mac. I've made no obvious concessions to portability. That said, I hope portability issues won't be daunting for any early adopters. 3. I don't do Windows. If you have problems it won't do you any good to mail me directly. Post about problems on the SpamBayes bug tracker: http://sourceforge.net/tracker/?group_id=61702&atid=498103 4. If you do Windows you will need PIL to take advantage of the recent changes: http://www.pythonware.com/products/pil/ (unless you want to put hair on your chest and build NetPBM on Windows). Fredrik Lundh provides prebuilt Windows versions of PIL. Grab the one appropriate for the version of Python you have installed. 5. If you do Windows (or any other platform for that matter), feedback to the lists about successes and failures would be helpful. Cheers, Skip

4 7

Need slightly better logic for blinking gifs
by skip＠pobox.com Aug. 31, 2006

Aug. 31, 2006

It didn't take long for the spammers to start with the blinking GIF images. Now I think they are using blinkers where the first image in the sequence is pretty much empty. The real content is in the second frame. I need to handle that. I don't think I can just blindly overwrite one frame with the next since they could just make the last one the blankish image. The way I concatenate images left-to-right and top-to-bottom makes it impossible to just concatenate the frames together either. Ideas? The code is in spambayes/ImageStripper.py in the distribution. Look at PIL_decode_parts. Skip

2 3

Integrate SpamBayes into a Win32 application
by Vilmos Schnedarek Aug. 28, 2006

Aug. 28, 2006

1 0

Re: [spambayes-dev] [Spambayes-checkins] spambayes/windows/py2exe setup_all.py, 1.26, 1.27
by Tony Meyer Aug. 25, 2006

Aug. 25, 2006

On 25/08/2006, at 1:18 AM, Mark Hammond wrote: > Update of /cvsroot/spambayes/spambayes/windows/py2exe > In directory sc8-pr-cvs8.sourceforge.net:/tmp/cvs-serv6540 > > Modified Files: > setup_all.py > Log Message: > Ship with PIL (but no Tkinter) and pyDNS > > > [...] > ! excludes = "Tkinter," # side-effect of PIL and markh doesn't > have it :) > ! "win32ui,pywin,pywin.debugger," # *sob* - these > still appear > ! # Keep zope out else outlook users lose training. > ! # (sob - but some of these may still appear!) > ! > "ZODB,_zope_interface_coptimizations,_OOBTree,cPersistence", I don't care about this for 1.1a3, but is this right? Outlook users (any users, really) would only lose training if they chose not to convert the database on installation and didn't change their configuration to continue to use bsddb. =Tony.Meyer

2 1

SpamBayes 1.1a3
by skip＠pobox.com Aug. 25, 2006

Aug. 25, 2006

The SpamBayes team is pleased to announce release 1.1a3 of SpamBayes. As is now usual, this is both a release of the source code and of an installation program for all Microsoft Windows users. This is an *ALPHA* release. It should only be installed by users willing to try out experimental software, and almost certainly contains new bugs. If you don't know what an alpha release is, please stick with 1.0.4 for the moment. The 1.1 release has been worked on since May of 2004, so contains a vast number of improvements over the 1.0.x line. These include, but are not limited to: * New database backends, including ZODB and ZOE. * Internationalisation support, including partial translations into French and Spanish. * Improved statistics reporting. * The ability to set audio notifications with the Outlook plug-in. * The ability to set the Outlook plug-in to move/copy ham, as well as spam/unsures. * Partial POP3 over SSL support for sb_server. * A vastly improved sb_imapfilter. * Several new experimental options, include one designed to help extract text content from image-based spams. Suggestions about what to try out can be found here: http://entrian.com/sbwiki/TryOutThePreRelease This release, like the ill-fated 1.0.2 and 1.0.3, is built with Python 2.4. We believe that the remaining incompatibilities with Python 2.4 have been resolved, and so this release should also include superior email parsing to the 1.0.x line. Details about the changes in this release can be found at http://sourceforge.net/project/shownotes.php?release_id=442102 You can get the release via the 'Download' page at http://spambayes.org/download.html Enjoy the new release and your spam-free mailbox As always, thanks to everyone involved in this release! Skip Montanaro. (on behalf of the SpamBayes team) --- What is SpamBayes? --- The SpamBayes project is working on developing a Bayesian (of sorts) anti-spam filter (in Python), initially based on the work of Paul Graham, but since modified with ideas from Robinson, Peters, et al. The project includes a number of different applications, all using the same core code, ranging from a plug-in for Microsoft Outlook, to a POP3 proxy, to various command-line tools and a command-line-based framework for testing new anti-spam techniques. The Windows installation program will install either the Outlook add-in (for Microsoft Outlook users), the SpamBayes server program (for all other POP3 mail client users, including Microsoft Outlook Express), or the SpamBayes IMAP filter (for all IMAP mail client users). All Windows users (including existing users of the Outlook add-in) are encouraged to use the installation program. If you wish to use the source-code version, you will also need to install Python - see README.txt in the source tree for more information.

1 0

My Humble Thanks To You
by Greg Aug. 24, 2006

Aug. 24, 2006

Guys, I run a small corporate network with around 100 e-mail users. We've been using Sophos PureMessage as an anti-SPAM solution, but it doesn't work very well. It's just too simplistic and easy for the spammers to work around. After some online research I decided to give SpamBayes a go. I downloaded the Outlook plugin but didn't know what to expect. I don't get any SPAM, but I have access to all the e-mail inboxes and can see that some users get around 50-60 per day, so I was pleased when I discovered I could get the plug-in to look at any folder I had access to !! I trialled it on 4 users, 2 who get heavy amounts of SPAM and 2 who get light amounts. Day 1, I had to go through a lot of e-mail and tell it what was SPAM and what was HAM. Day 2, I only had to tell it about a few. Day 3, I think there was 1. Day 4, well.... you get the picture. The users were delighted. They think I am a god now!! I now have the plug-in filtering every Inbox on the system and it doesn't miss a beat. We, at last, have a clean e-mail system. And the truth is that you guys are gods! My thanks for all your efforts. This has made everyone's work life here much easier. Greg.

2 1

How about a 1.1a3 release?
by skip＠pobox.com Aug. 24, 2006

Aug. 24, 2006

Any thought on making a 1.1a3 release? I'd like to get the image spam stuff into more peoples' hands. (Has anyone tried it yet?) Tony is extremely busy, and doesn't have the requisite Win2K setup to create a widely runnable Windows installer. Can someone else do that? While I have the ocrad-cygwin zipfile available for people to download, what do people think about bundling the ocrad.exe file and the source patch as part of a SpamBayes Windows installer? I confirmed with the Ocrad author that all we need to distribute in the (very small) patch, not the entire distribution as I originally did. Thx, Skip

5 7

Re: [spambayes-dev] [Spambayes] Posting problems
by skip＠pobox.com Aug. 20, 2006

Aug. 20, 2006

>>>>> "Dave" == David Abrahams <dave(a)boost-consulting.com> writes: Dave> David Abrahams <dave(a)boost-consulting.com> writes: >> I've posted several messages to this list through GMane, and wondered >> why nobody answered them. Well as it turned out, I wasn't subscribed, >> and you don't seem to be accepting posts from nonsubscribers (totally >> understandable). But I got no clue that subscription was needed, and >> GMane shows my posts anyway: >> >> http://thread.gmane.org/gmane.mail.spam.spambayes.devel/3613/focus=3613 >> >> They just don't show up in your email archive. Furthermore, for a >> little extra weirdness, I've posted successfully here before: >> >> http://mail.python.org/pipermail/spambayes/2003-December/author.html >> >> I dunno what's going on here. Dave> Um, I guess I was confusing the -dev list (on which nobody is Dave> answering me) with this one. Sorry for the noise. Dave> But if someone could get back to me on the -dev list I'd really Dave> appreciate it! Even just an ACK would be useful at this point! I saw no pending moderator requests on either spambayes or spambayes-dev. I saw nothing in the Mailman config for spambayes-dev that would prevent you from posting. Perhaps GMane isn't actually posting the messages. Skip

4 10

Patch for ocrad to run on Windows?
by skip＠pobox.com Aug. 13, 2006

Aug. 13, 2006

Mark Hammond and Sean True both said they had an ocrad.exe executable built under cygwin. (Hopefully it doesn't require cygwin runtime?) Was the only change you made to the source the "std::fprintf" -> "fprintf" replacement? Ocrad is GPL'd, so all we have to do to make it available is also distribute the modified source. If you can stick the .exe file somewhere and let me know if there are any Windows version restrictions, I'll put together the requisite modified Ocrad source distribution and place both the distribution and the executable on the SpamBayes website for Windows users to try out. I'll also send a (second) note to Antonio Diaz Diaz, the Ocrad author, letting him know where it is. Thx, Skip

2 1

Re: [spambayes-dev] [Spambayes-checkins] spambayes/spambayes dnscache.py, 1.2, 1.3
by skip＠pobox.com Aug. 13, 2006

Aug. 13, 2006

Tony> Remove reference to Skip, probably left there by mistake :) Yes, probably... Thanks for catching it. S

1 0