Mailman 3 July 2004 - spambayes-dev

DNS TTL clues OK, but not very good
by Matthew Dixon Cowles July 28, 2004

July 28, 2004

A few days ago, someone on the NANOG list mentioned that spammers routinely use very low time-to-live values for their DNS records. Once I'd heard that, it seemed obvious; a spammer would want to make his spamvertized URL resolve to new a new server quickly if the original server was shut down. Since I'm already using the IP address of the host part of a spamvertized URL as a synthetic token, it was easy enough to generate tokens for the TTL value. As an easy way of putting the values into buckets, I used the log (I tried both base 10 and base e) of the value, rounded to the nearest integer. Alas, it didn't work all that well. Still, I thought I'd post the results here in case someone else had the same idea. On my mail, it was somewhat better than all-defaults and almost as good as turning on mine_received_headers. But it wasn't as good as using the IP address of the server (the address itself and the value masked at /8, /16, /24). Using both the address and the TTL wasn't any better than using the address alone. Regards, Matt

1 0

untested idea for calculating message lengths
by Skip Montanaro July 27, 2004

July 27, 2004

One of the problems Spambayes seems to still have on occasion is properly classifying very short messages. I came up with the attached simple way to compute a message's effective length, but have yet to test it. I think the number of tokens will be better than the actual size of the message simply because that's what the classifier munches on. I will try to test it this week, but I don't have a large corpus at this point and thought I'd toss it out there in case others have some time to look at it. It just counts tokens as they are generated and tacks on a "token length:N" token to the end of the stream where N is the base two log of the number of tokens in the message. Skip

2 2

RE: [spambayes-dev] sb_unheader.py bug?
by Tony Meyer July 24, 2004

July 24, 2004

> I thought I saw some checkins related to > a change in Python 2.4. Could this be another manifestation > of that problem? Any clues appreciated. I realise that you've found the problem already, but in case it's of use to anyone: I've done a little bit of 2.4a1 testing with the Outlook plug-in and sb_server. I found a few problems (mostly with SpamBayes, but one with Python, which already has a fix checked in) with sb_server, but it appears to work ok now. I'll do a bit more testing when I get the chance - for one, I'm interested in seeing if the new FeedParser's invalid information (for malformed messages) can be used in the tokenizer, and whether this helps results. The only problems I've found so far have been with the web interface or with spambayes.Message, so if the script doesn't use that, then I have no idea :) =Tony Meyer

1 0

RE: [spambayes-dev] sb_unheader.py bug?
by Tony Meyer July 24, 2004

July 24, 2004

[Skip Montanaro] > When run with Python 2.3 it works just fine. I thought I saw some > checkins related to a change in Python 2.4. Could this be another > manifestation of that problem? Any clues appreciated. [Neil Schemenauer] > I believe the 'email' package is significantly changed in > 2.4. On mail.python.org I noticed that it did not parse > messages that used '\r\n' as line endings. I notified Barry > but I guess I should file a bug too. Was this pre a1? 2.4a1 seems to parse \r\n endings ok for simple messages, at least: Python 2.4a1 (#54, Jul 8 2004, 11:30:13) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import email >>> msg = email.message_from_string("Subject: Test\r\n\r\nTest Message.\r\n") >>> msg.as_string() 'Subject: Test\n\nTest Message.\r\n' >>> msg["Subject"] 'Test' >>> msg.get_payload() 'Test Message.\r\n' Although it has converted the header endings to '\n' (and since messages sent are meant to use \r\n, that means you need to do an appropriate replace before sending out a msg.as_string() - but sb_server and sb_imapfilter do this already, anyway). If there is a \r\n problem (with multipart, for example), then this would cause SpamBayes lots of strife, because we ensure that all the line endings *are* \r\n. I haven't noticed it with 2.4a1, but have done very limited testing. If you can find a problem with a1, then definitely +1 to filing a bug, too. =Tony Meyer

1 0

sb_unheader.py bug?
by Skip Montanaro July 23, 2004

July 23, 2004

Sb_unheader.py seems to completely lose messages when run with Python 2.4. When I run it as sb_unheader.py -p 'X-VM' -p 'X-Spam' -p 'X-Hammie' mbox all I get on stdout is a stream of lines like From nobody Fri Jul 23 00:13:14 2004 From nobody Fri Jul 23 00:13:14 2004 From nobody Fri Jul 23 00:13:14 2004 From nobody Fri Jul 23 00:13:14 2004 From nobody Fri Jul 23 00:13:14 2004 apparently one per input message. When run with Python 2.3 it works just fine. I thought I saw some checkins related to a change in Python 2.4. Could this be another manifestation of that problem? Any clues appreciated. Thanks, Skip

2 2

Modelos de cartas comerciais: http://www.gueb.de/cartascomerciais
by Malu Mendes July 23, 2004

July 23, 2004

As cartas comerciais, têm grande importância na administração de qualquer empreendimento, pois uma parte significativa das transações mundiais se realiza por esse meio. A carta é o instrumento que faz a conexão entre os negociantes. http://www.gueb.de/cartascomerciais Estamos lançando o CD MODELOS DE CARTAS COMERCIAIS, que sana suas dúvidas na elaboração de todos os tipos de cartas e documentos empresariais: agradecimentos, atestados e declarações, avisos, cartas de cobrança, cartas em inglês, comunicados, convites, contratos, propostas, empregos, solicitações e pedidos, telegramas, cartas por e-mail, etc. http://www.gueb.de/cartascomerciais O CD contém mais de 400 modelos de Cartas Comerciais e inúmeras técnicas de Redação Comercial. Indicado para: secretárias em geral, gerências, Rh, executivos, estudantes, empresas de toda ordem, etc. O custo é ínfimo em relação ao que poderá gerar no aperfeiçoamento da comunicação de sua empresa. http://www.gueb.de/cartascomerciais

1 0

Bug?: assert hamcount <= nham
by Titus von der Malsburg July 20, 2004

July 20, 2004

Hi, I observed, that the following error happens sometimes when a message is being classified while sb_mboxtrain.py is running: procmail: Assigning "MAILDIR=/home/mitarb/malsburg/Maildir" procmail: Assigning "DEFAULT=/home/mitarb/malsburg/Maildir/" procmail: Assigning "ARCHIVDIR=/home/mitarb/malsburg/.procmail/archiv" procmail: Assigning "BAYESCUSTOMIZE=/home/mitarb/malsburg/.spambayesrc" procmail: Locking "hamlock" procmail: Executing "/home/mitarb/malsburg/usr/bin/sb_filter.py" Traceback (most recent call last): File "/home/mitarb/malsburg/usr/bin/sb_filter.py", line 257, in ? main() File "/home/mitarb/malsburg/usr/bin/sb_filter.py", line 248, in main action(msg) File "/home/mitarb/malsburg/usr/bin/sb_filter.py", line 180, in filter return self.h.filter(msg) File "/home/mitarb/malsburg/usr/lib/python2.3/site-packages/spambayes/hammie.py", line 109, in filter prob, clues = self._scoremsg(msg, True) File "/home/mitarb/malsburg/usr/lib/python2.3/site-packages/spambayes/hammie.py", line 38, in _scoremsg return self.bayes.spamprob(tokenize(msg), evidence) File "/home/mitarb/malsburg/usr/lib/python2.3/site-packages/spambayes/classifier.py", line 190, in chi2_spamprob clues = self._getclues(wordstream) File "/home/mitarb/malsburg/usr/lib/python2.3/site-packages/spambayes/classifier.py", line 493, in _getclues tup = self._worddistanceget(word) File "/home/mitarb/malsburg/usr/lib/python2.3/site-packages/spambayes/classifier.py", line 508, in _worddistanceget prob = self.probability(record) File "/home/mitarb/malsburg/usr/lib/python2.3/site-packages/spambayes/classifier.py", line 308, in probability assert hamcount <= nham AssertionError procmail: [19899] Tue Jul 20 14:43:10 2004 procmail: Program failure (1) of "/home/mitarb/malsburg/usr/bin/sb_filter.py" procmail: Rescue of unfiltered data succeeded procmail: [19899] Tue Jul 20 14:43:10 2004 procmail: Unlocking "hamlock" procmail: No match on "^X-SpamBayes-Classification: spam" procmail: No match on "^Subject: Floss2004" procmail: No match on "^Subject:.*HzG" procmail: No match on "From: newsalerts-noreply(a)google.com" procmail: No match on "^X-Mailman-Version:.*" procmail: Assigning "LASTFOLDER=/home/mitarb/malsburg/Maildir/new/1090327388.19899_1.janus" procmail: Notified comsat: "malsburg@0:/home/mitarb/malsburg/Maildir/new/1090327388.19899_1.janus" >From malsburg(a)cl.uni-heidelberg.de Tue Jul 20 14:43:08 2004 Subject: Test5 Folder: /home/mitarb/malsburg/Maildir/new/1090327388.19899_1.janus 639 I configured spambayes and procmail just like it is described in README.txt. The error occured in spambayes-1.0a7 and spambayes-1.0rc2. Titus

2 1

RE: [Spambayes-checkins] spambayes/spambayes Dibbler.py, 1.13, 1.14
by Tony Meyer July 19, 2004

July 19, 2004

> Update of /cvsroot/spambayes/spambayes/spambayes > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6773/spambayes > > Modified Files: > Dibbler.py > Log Message: > Apply Richie's test for [ 990700 ] Changes to asyncore in > Python 2.4 break ServerLineReader > which seems to work for me w/ Python 2.4a1 And by "test", I mean "fix"... =Tony Meyer

1 0

Windows implementation using Command line (for SpamBayes)
by Rob McEwen July 14, 2004

July 14, 2004

RE: Windows implementation using Command line (for SpamBayes) Is there an implementation for SpamBayes for Windows as a Command line Utility? For example, some anti-virus programs (like McAfee VirusScan, F-Prot Antivirus for DOS, Grisoft AVG, etc.) can be invoked in a separate process, fed the path to the text file of the message to be process, return an "exit code" based on the results of scanning that message file. (or, possibly, add the appropriate header to the message). Also, training could be done by occasionally passing this command line tool a known-good "ham" folder and/or a known-bad "spam" folder. Couldn't this same concept could be applied to spambayes? (Remember, I'm specifically looking for something that will do this in Windows.) Also, it is my understanding that PROCMAIL is ONLY available on the Unix platform?? (If so, another way to solve this problem might be to find a 3rd party app which emulates PROCMAIL in windows?? ...but this sounds far-fetched to me!) Finally, could this be done as a windows service so that the program doesn't have to initialize and die with each individual request? Any suggestions are appreciated! Thanks, Rob McEwen PowerView Systems rob(a)PowerViewSystems.com (478) 475-9032

4 5

RE: [spambayes-dev] Windows implementation using Command line(forSpamBayes)
by Tony Meyer July 14, 2004

July 14, 2004

> Take a look at the sb_filter.py script. We don't build it as > a Windows executable so you'll have to install Python and run > from source to get it, but I see no reason why it wouldn't > work on Windows. I can confirm that it does. For training, you're after the sb_mboxtrain.py script. I don't recall whether I've run that or not, but it should be fine (assuming that your mail is in mbox format). If it fails to run, open a bug and include the traceback and assign to me (Anadelonbrin) and I should be able to fix it. > You might be able to use the sb_bnfilter.py and > sb_bnserver.py scripts, which I believe are intended to do > something similar to this. sb_bnserver.py is definitely *nix only at the moment. I don't see any reason why it would have to be, though, so it could probably be changed to be more cross platform. Note that there's a CVS branch that has these scripts in C (for speed), which I think is the intended final version. Toby is the one to talk to about them. =Tony Meyer

4 3