Mailman 3 September 2003 - spambayes-dev

About Bug 814322
by Remi Ricard Oct. 1, 2003

Oct. 1, 2003

Hi, Tony added a patch for the bug 814322 This is his comment: > Basically the review page would die if one of the cached messages was moved from > the cache directory by someone other than spambayes (for example a virus > protection program). Instead, we ignore the problem and just don't present that message > for review. I don't use a virus protection program or anything else and I see the error message often. It is possible that this error can be caused by the fact that spambayes is not able to separate the mime part of a message (to generate file 123456 and 123456-1) so when training there is a error since the file 123456-1 (the mime part) is not found ?? Remi papaDoc(a)videotron.ca

2 1

Re: [Spambayes] Error from pop3proxy
by Skip Montanaro Oct. 1, 2003

Oct. 1, 2003

Alan> I got this error from pop3proxy running on my W2K server :- Alan> error: uncaptured python exception, closing channel Alan> <sb_smtpproxy.BayesSMTPProxy connected 192.168.1.100:1571 at Alan> 0x1833aa8> (bsddb._db.DBRunRecoveryError:(-3098 2, Alan> 'DB_RUNRECOVERY: Fatal error, run database recovery -- fatal Alan> region error detected; run recovery') A number of people have seen this. It appears to be related to incorrect locking of the database in multi-threaded applications. If you're not a programmer that may not make sense to you. The simplest way to work around that may be to switch to the Pickle form of the database. However, before attempting that switch, let us know what version of SpamBayes you are running. Some bugs in the configuration saving code were fixed only recently. Tony's the expert in this area. For the spambayes-dev folks: If we're not properly mediating access to the on-disk version of the DBDictClassifier, we're probably not mediating access to the in-memory PickledClassifier either. Maybe all we need to do is add a little bit of locking to the Classifier class or its subclasses. Skip

4 3

Re: [spambayes-dev] Logging
by T. Alexander Popiel Sept. 30, 2003

Sept. 30, 2003

In message: <16249.46740.636638.581709(a)montanaro.dyndns.org> Skip Montanaro <skip(a)pobox.com> writes: > > Alex> It should be noted that 2.2.1 is the most recent python packaged > Alex> for Debian stable (which at least some of your users (me! me!) > Alex> run). Are there some simple tests that I can do to determine how > Alex> much logging support exists there? > >I suspect > > import logging > >will fail. Aye, this breaks. >You might try grabbing the 2.3 logging module and tests from CVS >and see if they run okay. If so, I'd be all for including the logging >module with the SpamBayes release. I'll try this... - Alex

1 0

Re: [spambayes-dev] Logging
by T. Alexander Popiel Sept. 30, 2003

Sept. 30, 2003

In message: <1ED4ECF91CDED24C8D012BCF2B034F13026F2976(a)its-xchg4.massey.ac.nz> "Tony Meyer" <ta-meyer(a)ihug.co.nz> writes: >A while back I mentioned that it would be nice to have various levels of >logging rather than just the True/False globals:verbose setting that >there is now. Skip mentioned that if we're going to do that then we >might as well use the logging module, although it would break 2.2 >compatibility. Mark already has some logging code in both the Outlook >plug-in and pop3proxy_service/tray for when they are run as binaries/a >service. > >Would anyone be opposed if we did start using the logging module in >places? How incompatible is it with 2.2? I've used it with 2.2.3 (but >only logging to stdout, to a file, and to a rotating file), but not >earlier, and we currently support 2.2 (or so we say ;). > >(Obviously for the 2.2.3 compatibility, there needs to be a >compatLogging module like the compatSets one we already have.) > >=Tony Meyer It should be noted that 2.2.1 is the most recent python packaged for Debian stable (which at least some of your users (me! me!) run). Are there some simple tests that I can do to determine how much logging support exists there? - Alex

3 2

Logging
by Tony Meyer Sept. 30, 2003

Sept. 30, 2003

A while back I mentioned that it would be nice to have various levels of logging rather than just the True/False globals:verbose setting that there is now. Skip mentioned that if we're going to do that then we might as well use the logging module, although it would break 2.2 compatibility. Mark already has some logging code in both the Outlook plug-in and pop3proxy_service/tray for when they are run as binaries/a service. Would anyone be opposed if we did start using the logging module in places? How incompatible is it with 2.2? I've used it with 2.2.3 (but only logging to stdout, to a file, and to a rotating file), but not earlier, and we currently support 2.2 (or so we say ;). (Obviously for the 2.2.3 compatibility, there needs to be a compatLogging module like the compatSets one we already have.) =Tony Meyer

2 1

Slightly new tactic
by T. Alexander Popiel Sept. 29, 2003

Sept. 29, 2003

I just noticed a new variation on an old spammer tactic in my unsure folder: in HTML mail, text colors which almost but not quite match the background color. I know that matching the background exactly is an old tactic, but this spam seemed to have a random adjustment to the font color, by plus or minus 3 quanta on the blue scale. The mail still showed up as high unsure, and I don't think we bother with trying to determine which text in a multipart or HTML message is actually visible; however, if we ever do try to figure that out, then we may want to put some wiggle room on the color-matching. - Alex

2 1

Web Interface Help
by Tony Meyer Sept. 29, 2003

Sept. 29, 2003

I've checked in a basic structure for some 'help' pages for the web interface. Basically there's a link in the footer which will display either the standard help page, or a page specific to the one on which the user clicked. There's a rudimentary standard page now, plus a different one for the review page, just as an example, but pretty much all the help needs to be written yet (or borrowed from other docs). There's a 'help' gif that goes with it, but it doesn't really match the other ones. If whoever made those (Richie?) could make a matching help one, that would be great :) Anyway, the point is, if anyone would like to write something to go in the help page for a particular part of the interface, please feel free :) At the moment the onHelp function in UserInterface.py serves up the correct help text given the topic, but this should be moved out (into a set of .ht files?) at some point. Comments & criticsm welcome as usual, of course ;) =Tony Meyer

2 1

RE: [spambayes-dev] spambayes.org
by Tony Meyer Sept. 29, 2003

Sept. 29, 2003

[Me] > Did spambayes.org die after all? I haven't been able to connect > to it for a few days now (although spambayes.sf.net works fine). [Skip, Rob and Kenny assert that it works for them] Looks like it's just me then, I suppose. Odd. A tracert dies at gandi-gw.cdg4.fr.mfnx.net [62.4.77.238]. The "gandi" bit was in addresses that Skip and Rob mentioned, so it looks like it's getting close. Presumably something that will sort itself out, then. Thanks for confirming that it's me, though :) =Tony Meyer

3 2

RE: [spambayes-dev] Re: [Spambayes] spam error...
by Meyer, Tony Sept. 29, 2003

Sept. 29, 2003

> There's also the problem that the setup.py script tries to > import __version__ from the spambayes package. I think we > concluded that it was probably pilot error when operating > WinZip and not an error in SpamBayes proper, but I'd like to > get confirmation of that. It would also be nice if setup.py > could sniff that problem out before trying that import (or > catching the ImportError and then sniff around a bit). Would this work? Is there anywhere else we could look for it? """ try: from spambayes.Version import __version__ except ImportError: # File should be there, maybe the user expanded it all # into one directory? try: from Version import __version__ except ImportError: print "Cannot find Version.py - something appears to " \ "have gone wrong unpacking the archive. Please " \ "try again, ensuring that you expand the complete " \ "archive, and retain the directory structure." sys.exit() """ =Tony Meyer

1 0

artificially tweaking the spam/ham to deal with n-way scoring
by Skip Montanaro Sept. 29, 2003

Sept. 29, 2003

I'm horsing around with my n-way script a bit and have written a small training script to help maintain the many training databases. Here's how it works. Suppose I have four input mailboxes: python, personal, cars and music, each with a different number of messages (450, 50, 1400, and 50 messages, respectively). I run sb_mboxtrain.py over each one calling all messages "good", e.g.: sb_mboxtrain.py -d python-ham.db -g python sb_mboxtrain.py -d personal-ham.db -g personal sb_mboxtrain.py -d cars-ham.db -g cars sb_mboxtrain.py -d music-ham.db -g music I then create python.db by initializing it with the contents of python-ham.db, but swap all the counts (treating python-ham.db as all "spam"), then merge in the other three unchanged. When I'm finished, I have these figures in the four databases: db spam ham python.db 450 1500 personal.db 50 1900 cars.db 1400 550 music.db 50 1900 My nway script then scores messages against those four databases. As a result of the way I'm building the databases, I can have very extreme ratios. Again, in real life I want to cluster with more mailboxes (I have 15 at the moment). I haven't thought of a good way to truly balance the scores and only run sb_mboxtrain once against each mbox file. I'm thinking I should fudge things. When I merge the multiple smaller databases into a single daatbase I was thinking I would try scaling the larger values by enough to satisfy this relationship: 0.5 <= ham count/spam count <= 2.0 Using the python database as an example, I would scale all ham counts by 900/1500, giving a 2-to-1 ham:spam ratio. Is that roughly what the now defunct imbalance ratio did? Skip

2 1