Mailman 3 July 2003 - spambayes-dev

RE: [Spambayes] Question (or possibly a bug report)
by Meyer, Tony Aug. 31, 2003

Aug. 31, 2003

> So when the marshalled representation of 0.001 is loaded under > "german" LC_NUMERIC here, we get back exactly 0.0. I'm not > sure why. When I call "marshal.dumps(0.1)" from AsyncDialog (or anywhere in the Outlook code) I get "f\x030.0", which fits with what you have. > So the obvious <wink> answers are: (Glad you posted this - I was wading through the progress of marshalling (PyOS_snprintf etc) and getting rapidly lost). > 1. When LC_NUMERIC is "german", MS C's atof() stops at the first > period it sees. This is the case: """ #include <locale.h> #include <stdio.h> #include <stdlib.h> int main() { float f; setlocale(LC_NUMERIC, "german"); f = atof("0.1"); printf("%f\n", f); } """ Gives me with gcc version 3.2 20020927 (prerelease): 0.100000 Gives me with Microsoft C++ Builder (I don't have Visual C++ handy, but I suppose it would be the same): 0,00000 The help file for Builder does say that this is the correct behaviour - it will stop when it finds an unrecognised character - here '.' is unrecognised (because we are in German), so it stops. Does this then mean that this is a Python bug? Or because Python tells us not to change the c locale and we (Outlook) are, it's our fault/problem? Presumably what we'll have to do for a solution is just what Mark is doing now - find the correct place to put a call that (re)sets the c locale to English. =Tony Meyer

8 16

Re: [spambayes-dev] I took a big step Tuesday...
by T. Alexander Popiel Aug. 4, 2003

Aug. 4, 2003

In message: <LNBBLJKPBEHFEDALKOLCGEHBFAAB.tim.one(a)comcast.net> "Tim Peters" <tim.one(a)comcast.net> writes: > >Something that would be fine: Add a config option mapping score ranges to >tokens. For example, > >> I currently have my ham and spam cutoffs set at 0.15 and 0.80, >> respectively. > >could be described by > > Ham 0.15 Unsure 0.80 Spam > >and > >> As I mentioned in a recent message, I consider 0.80 to 0.90 to be >> "low spam" and 0.91 to 1.00 to be "high spam". > >by > > Ham 0.15 Unsure 0.80 LowSpam 0.91 HighSpam +1 - Alex

5 6

RE: [spambayes-dev] Making the FAQ
by Meyer, Tony July 31, 2003

July 31, 2003

> You said it was emitting all the instructions. That suggests > it's not a make problem. Make's job is to fire off the > correct programs in the correct order. You indicated that it > did that. Ok, now I've figured out what it does. I should have probably done this first :) I ran the html.py faq.txt | sed ... command manually and looked at the files. The >> at the end of that command *appends* onto the end of faq.ht, so faq.ht ends up having the original copy and then a new copy. When the html file is made, it also is a whole heap of html files all appended together. The browser, of course, stops after the first </html>. This is the correct behaviour for >>, isn't it? Shouldn't the makefile then be deleting the faq.ht file before recreating it? Again, does it actually work for anyone else? Thanks, Tony Meyer

3 3

RE: [spambayes-dev] Making the FAQ
by Meyer, Tony July 31, 2003

July 31, 2003

> [Tony is baffled by make] Never a truer word was typed ;) > Check to see that your Makefile and scripts/make.rules files are > up-to-date. I can't seem to cvs-up at the moment, but it exactly matches the v1.8 that viewcvs gives me. Looking at the check-ins list there doesn't seem to be anything more recent. > Your Makefile should have a target to make > faq.ht from faq.txt and another to make faq.html from faq.ht. Yup, have both of those. > Also, make sure the html.py > script from the docutils distribution is on your PATH. Well, I fiddled the makefile so that it was an explicit PATH, but this isn't the problem, because if it isn't, I end up with a blank faq.ht, and that's not what is happening here. > Running make when faq.txt is newer than the others should > generate this output: > > echo "Title: SpamBayes FAQ" > faq.ht Shouldn't this be >>, BTW? [output snipped] I get all of this. > If you still can't get it to work and everything seems to be > in order, send me the output of "make -d faq.html". Attached. Note that it does say that it's remaking it, but the faq.ht file does not change. > P.S. Everyone seems to hate make and I just can't figure out why. I don't hate it, I'm just baffled by it (particularly in this case)! (Look at how many questions Tim has answered about Python that I have asked - the same could be said... ;) Thanks, Tony P.S. Did you try my process? You are positive that this works for you, right?

2 1

RE: [spambayes-dev] Making the FAQ
by Meyer, Tony July 31, 2003

July 31, 2003

[Foolish person] > This is the correct behaviour for >>, isn't it? Shouldn't the > makefile then be deleting the faq.ht file before recreating it? [Skip] > Not necessary. The first echo command uses '>'. Ah. Caught by my own alertness. I saw the ">" and thought it was a typo (I even mentioned this!) and 'corrected' it to ">>", which broke everything. <sigh>. A lesson learned. Thanks for helping me figure out what I did wrong. =Tony Meyer

1 0

Re: [spambayes-dev] Making the FAQ
by Skip Montanaro July 31, 2003

July 31, 2003

Tony, You said it was emitting all the instructions. That suggests it's not a make problem. Make's job is to fire off the correct programs in the correct order. You indicated that it did that. Skip

1 0

Making the FAQ
by Meyer, Tony July 31, 2003

July 31, 2003

I'm having difficulty (re)making the FAQ - could someone please check whether this is something I'm doing, or a problem with the make rules? Process: 1. Run make so that you have copies of faq.ht and faq.html 2. Change the FAQ somehow, so that you will be able to recognise the change (add a question or something). 3. Run make again so that you (should have) updated copies of faq.ht and faq.html 4. Check that your changes are in the new faq.html This doesn't work for me. I get the same faq.html that I had before. To get around this I rm faq.ht and *then* run make, which works. Thanks! =Tony Meyer

2 1

RE: [spambayes-dev] Script to find which db is used
by Meyer, Tony July 29, 2003

July 29, 2003

> This is a little script to find out which database is used by > pop3proxy since it is better to avoid using dumbdb. The script is based > on a mail sent by Skip to me. > > Lots of people (At least one... i.e. me) did not know which database > they use. This can help them. This is a nice little utility. I'll check it in and put a link to it on the website. > Since this is my first python script I have a question > I think this script should be in > c:/a_dir/spambayes/utilities > but how can you import the options from spambayes.Options > when your are in c:/a_dir/spambayes/utilities. If spambayes is on your PYTHONPATH then it will just work. If it's not, then you need to add the spambayes directory to sys.path. Some of the testing scripts do this (timtest.py and timcv.py do, I think). For example: """ import os import sys sys.path.insert(-1, os.getcwd()) # add this directory sys.path.insert(-1, os.path.dirname(os.getcwd())) # add the parent directory """ =Tony Meyer

1 0

RE: [spambayes-dev] SQL classifier
by Meyer, Tony July 29, 2003

July 29, 2003

> I added a trivial PGClassifier class which uses PostgreSQL as > the database to spambayes/storage.py. If you check this in, I might as well check in my mySQL patch with the modifications that you suggested a while back (in particular having a base generic SQLClassifier class). > 1. The hammie command line interface only allows two different save > modes, dbm-style or pickle-style. I faked it for the moment by > changing DBDictClassifier to PGClassifier. I faked it by allowing a "-S" option to be passed to pop3proxy (which I used for testing). > Has anyone considered either of these issues? I'd like to > solve the first one now so Spambayes can cleanly support more > than just two types of classifiers. I think it will require > a change to the command line args of the various (Unix-based) > tools. Unix based is perhaps not right since windows people can just as easily use pop3proxy/imapfilter. > They're getting pretty baroque as it is now. Maybe > it's time for a cleanup (or maybe they should get more > baroque in the interests of backward-compatibility). This is one of the changes that Greg was arguing for, and no-one really had anything against it. I'm happy if you make the changes and break backwards compatibility - we're in alpha after all! > Right now with hammie you give database args of either > -d -p FILE use dbm store found in FILE > -D -p FILE use pickle store found in FILE > On the other hand, hammiefilter, pop3proxy and imapfilter use > -d FILE use dbm store found in FILE > -D FILE use pickle store found in FILE > It seems to me that we should probably change hammie's > command line interface to match the other Unix-style apps. I would make one more change suggested by Greg - use "-p" instead of "-D" for a pickle. (And this means also changing all the other apps, plus fixing dbExpImp's bizarre -D dbm -d pickle case). What about those that have their storage options in their config file? At the moment we have: [Storage] persistent_use_database: True|False # True for dbm, False for pickle persistent_storage_file: hammie.db # dbm/pickle file "persistent_storage_file" is probably close enough to "data_store_name" that it could be used to hold the SQL information. We could change persistent_use_database to a multi-choice of ("DBM", "Pickle", "SQL") or something similar. This would probably break more people's setups than changing the command line options - we could have a (temporary) situation where "True" is an alias for "DBM" and "False" is an alias for "Pickle". > We could then add > -S dsn [...] > (which would require some parsing to mash into the form > MySQLdb likes). +1 to all of this. I'm happy to put in the parsing to get mySQLdb to work. =Tony Meyer

1 0

SQL classifier
by Skip Montanaro July 29, 2003

July 29, 2003

I added a trivial PGClassifier class which uses PostgreSQL as the database to spambayes/storage.py. I haven't messed with it much yet, but a couple things come to mind: 1. The hammie command line interface only allows two different save modes, dbm-style or pickle-style. I faked it for the moment by changing DBDictClassifier to PGClassifier. 2. Since real databases can do commit and rollback it would be nice if this was exposed at the classfier level (maybe begin, commit and rollback methods). Commits or rollbacks could be performed on a per-message basis, giving extra resilience in the face of errors. Has anyone considered either of these issues? I'd like to solve the first one now so Spambayes can cleanly support more than just two types of classifiers. I think it will require a change to the command line args of the various (Unix-based) tools. They're getting pretty baroque as it is now. Maybe it's time for a cleanup (or maybe they should get more baroque in the interests of backward-compatibility). Right now with hammie you give database args of either -d -p FILE use dbm store found in FILE -D -p FILE use pickle store found in FILE On the other hand, hammiefilter, pop3proxy and imapfilter use -d FILE use dbm store found in FILE -D FILE use pickle store found in FILE It seems to me that we should probably change hammie's command line interface to match the other Unix-style apps. We could then add -S dsn to specify that an SQL connection be made with the given dsn (data source name, e.g., "host=localhost dbname=skip"). The dsn could be suitably mangled to select appropriate databases, for instance: -S "PGSQL:host=localhost dbname=skip" or -S "MYSQL:host=remote dbname=bayes" (which would require some parsing to mash into the form MySQLdb likes). Thoughts? Skip P.S. No, I haven't forgotten about my trim_email_parameters idea. This occurred to me as I was waiting for about 20 minutes for a database save on a full retrain (obviously this wouldn't be a problem with an SQL database). Maybe it's time for me to trim my training database.

1 0