-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I'm about to merge my SQLAlchemy branch to the trunk. I'm happy
enough with where this is going to commit to this approach going
forward. I wrote about it a bit in a news article on wiki.list.org
and I'll try to elaborate there when things settle down.
This stuff could definitely use more testing, but the web u/i and
sending mail through the system work for me. If you're using the
trunk regularly now, you will notice that there's no migration path
yet for moving from the pickle-based storage to SQLAlchemy based
storage. This is what the import/export scripts are for. You notice
though that only bin/export is written yet. bin/import is tops on my
list after the merge.
I've also committed bin/export.py to the Mailman 2.1 branch. This
will be how you migrate MM2.1 lists to the MM trunk. Our next
release of MM2.1 will have this, so my plan for migration will be to
install the latest 2.1.x (x > 9), run bin/export.py, then feed the
resulting XML to the trunk's bin/import. For now, you can start from
scratch with bin/newlist or creating a list through the web. From
there the trunk should look and act just like you're used to.
As always, let us know what you find. I'm eager to fix problems
quickly. Please don't use the SF bug tracker for problems you
encounter on the trunk. It would also be interesting to know if you
try a SQLAlchemy backend other than SQLite.
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin)
iQCVAwUBRZWUFHEjvBPtnXfVAQIEJwP/WkB2jY3Ww2YR96oDejC/ZSSWwbct8ENb k/kpcoYm9V2ra36WUiHt0pfFLyhMk0pRXtBh/oScooLiZJu9AA2wnD046I5AJZLt 5DS+YJI2B5c4pn4pO80w5JeAUgQfiYIhafs6u/Oy3fDzag6Rp698Dt/Bv62V8Mly hteE9Iu8xcs= =plDI -----END PGP SIGNATURE-----
On 12/29/06 2:17 PM, "Barry Warsaw" <barry@python.org> wrote:
I'm about to merge my SQLAlchemy branch to the trunk. I'm happy enough with where this is going to commit to this approach going forward.
[Loud cheering from the sidelines!!]
(An upcoming rewrite of our mail handling system will also be eliminating (at least most) pickles. At present, these are cleverly stored in MySQL, and getting the data out in ad hoc queries is remarkably difficult.)
--John
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Dec 29, 2006, at 5:30 PM, John W. Baxter wrote:
On 12/29/06 2:17 PM, "Barry Warsaw" <barry@python.org> wrote:
I'm about to merge my SQLAlchemy branch to the trunk. I'm happy enough with where this is going to commit to this approach going forward.
[Loud cheering from the sidelines!!]
:)
BTW, I'm hoping there will be some adventurous folks willing to test
out the trunk. Don't use it for anything you care about, but at this
point you should be able to export your MM2.1 lists and import them
into the trunk.
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin)
iQCVAwUBRZqXAnEjvBPtnXfVAQKWuQQAs+gy4zibdmOKNFVn/qqtlJ6JEdJj+iR1 E7NO834fDCZAW65fWqPuTDxXwX5U7yOAlbg4QJutq5XOrwctlG3G5IvwNPfdf4ie wf0CVhuAuc07TuIqsylIEiAL5ooZWeqLWqL7TN/7IaUVJjEFLB2GsDZ0V0sKpD2N jHUrV6XJSQw= =zqBP -----END PGP SIGNATURE-----
BTW, I'm hoping there will be some adventurous folks willing to test
out the trunk. Don't use it for anything you care about, but at this
point you should be able to export your MM2.1 lists and import them
into the trunk.
A mini how-to would help me become adventurous again :)
How do I fetch the dev branch, what set of commands do I use to install it and migrate a few lists. Nothing fancy, but precise urls and commands.
-- Fil
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jan 2, 2007, at 12:40 PM, Fil wrote:
BTW, I'm hoping there will be some adventurous folks willing to test out the trunk. Don't use it for anything you care about, but at this point you should be able to export your MM2.1 lists and import them into the trunk.
A mini how-to would help me become adventurous again :)
How do I fetch the dev branch, what set of commands do I use to
install it and migrate a few lists. Nothing fancy, but precise urls and
commands.
Good point. I'll flesh things out in the wiki and will follow up
here when that's ready.
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin)
iQCVAwUBRZqcInEjvBPtnXfVAQIQ6QQAh3pS9mh8VhdpYNyb8QtI5pMEEhsDQ2m0 SiGI9Ko8GPQFcrrnpB+AEna7FKsyE8fuEFLEOzF82ZRIX1Z3HlDHm5ag6NUFtVMp j4BiN++EsDUW+L/hEVJnS5WOsCdm+KO8Vg4nXQAmeefbYJPEZ1s4I2S+2/YzS8pl MXHo8Fbzqvs= =nXoB -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jan 2, 2007, at 12:40 PM, Fil wrote:
BTW, I'm hoping there will be some adventurous folks willing to test out the trunk. Don't use it for anything you care about, but at this point you should be able to export your MM2.1 lists and import them into the trunk.
A mini how-to would help me become adventurous again :)
How do I fetch the dev branch, what set of commands do I use to
install it and migrate a few lists. Nothing fancy, but precise urls and
commands.
Take a look at
and let me know what you think.
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin)
iQCVAwUBRZrrI3EjvBPtnXfVAQJ6bwP+Lqh04drPra1xW6EmgP8itZEmTKqGFbcr 36QS94mTYzWlPfNPSUZZXnYqZ6zuEh+YOV97UgEpg5GU2qQj70bC/UbbKEo4b+Yi kWRLJhZkDAkJ42kbxikRQAzc3gxSbhR5/fSQNjK0DUNK8C7nC1PPdk/dulLT+FTM zj+g3MPd9xQ= =H3/m -----END PGP SIGNATURE-----
How do I fetch the dev branch, what set of commands do I use to install it and migrate a few lists. Nothing fancy, but precise urls and commands.
Take a look at
and let me know what you think.
OK; I was able to follow the instructions to the point of doing configure; make; make install. Here are my first glance/field notes:
I had to tweak permisions on logs/* to be able to launch bin/mailmanctl start, as they had been created with g-w (the logs/ dir is correctly g+sxw)
then i was able to create a list by using bin/newlist; it did create a directory in lists/listname@server/ (empty directory)
I found a sqlite db in data/mailman.db (I understand that to use a MySQL db I just need to create it and change the SQLAlchemy variable to reflect this db??)
the web server launched ok on port 2580 (I guess I'll have to ProxyPass to it from apache, which is fine).
It broke however when I tried to connect to it (put the list admin passwd, and got : UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 207: ordinal not in range(128)
maybe because I chose "french" as default language (I then switched to english, created another list, and got to the web ui)
tried to modify the list options thu the web ui and got
SQLError: (OperationalError) attempt to write a readonly database 'UPDATE "List data" SET bounce_processing=? WHERE "Listdata".list_id = ?' [0, 2]
I haven't tried yet to convert an older list to the new system.
-- Fil
Hi Fil,
Fil wrote:
tried to modify the list options thu the web ui and got
SQLError: (OperationalError) attempt to write a readonly database 'UPDATE "List data" SET bounce_processing=? WHERE "Listdata".list_id = ?' [0, 2]
Who and what is the owner and permission of data/mailman.db. Are you accessing web ui via wsgi (HTTPRunner) or classic cgi?
I haven't tried yet to convert an older list to the new system.
-- Fil
-- Tokio Kikuchi, tkikuchi@is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/
tried to modify the list options thu the web ui and got
SQLError: (OperationalError) attempt to write a readonly database 'UPDATE "List data" SET bounce_processing=? WHERE "Listdata".list_id = ?' [0, 2]
Who and what is the owner and permission of data/mailman.db.
fil@infora:~$ ls -al /usr/local/mailman22/data/ total 24 drwxrwsr-x 2 root mailman 4096 2007-01-03 15:38 . drwxrwsr-x 21 mailman mailman 4096 2007-01-03 15:02 .. -rw-r--r-- 1 root mailman 9216 2007-01-03 15:38 mailman.db -rw-rw---- 1 mailman mailman 6 2007-01-03 15:33 master-qrunner.pid
so i guess it was the same persmission problem I had on logs/*
Are you accessing web ui via wsgi (HTTPRunner) or classic cgi?
wsgi on localhost
-- Fil
Fil wrote:
tried to modify the list options thu the web ui and got
SQLError: (OperationalError) attempt to write a readonly database 'UPDATE "List data" SET bounce_processing=? WHERE "Listdata".list_id = ?' [0, 2] Who and what is the owner and permission of data/mailman.db.
fil@infora:~$ ls -al /usr/local/mailman22/data/ total 24 drwxrwsr-x 2 root mailman 4096 2007-01-03 15:38 . drwxrwsr-x 21 mailman mailman 4096 2007-01-03 15:02 .. -rw-r--r-- 1 root mailman 9216 2007-01-03 15:38 mailman.db -rw-rw---- 1 mailman mailman 6 2007-01-03 15:33 master-qrunner.pid
so i guess it was the same persmission problem I had on logs/*
logs and data and other writable directiory and files should be owned by mailman. We should be able to check them with bin/check_perms.
Are you accessing web ui via wsgi (HTTPRunner) or classic cgi?
wsgi on localhost
Group permission problems should be going away in wsgi and LMTP environment.
-- Fil
-- Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jan 3, 2007, at 9:37 AM, Fil wrote:
How do I fetch the dev branch, what set of commands do I use to
install it and migrate a few lists. Nothing fancy, but precise urls and
commands.Take a look at
and let me know what you think.
Fil, thanks for taking a look.
OK; I was able to follow the instructions to the point of doing
configure; make; make install. Here are my first glance/field notes:I had to tweak permisions on logs/* to be able to launch bin/ mailmanctl start, as they had been created with g-w (the logs/ dir is
correctly g+sxw)
I didn't notice this before because my shell umask was already set to
002, but if yours is set to e.g. 022, you'll get this problem. I'm
adding an explicit "os.umask(2)" to the Mailman initialization stuff
so that by default everything will be group writable. This fixes the
log file permissions.
I'm not committing this change yet though because it doesn't solve
the mailman.db permissions. My guess is that somewhere in the
SQLAlchemy, PySQLite, SQLite stack, the permissions or umask are
getting forced. I just need to figure out the right way to tell SA
how to set the permissions when using the SQLite engine.
I'd also like to review and eliminate as many other umask setting
stuff in Mailman as possible. Ideally, we'd set the umask in
initialize() and only if we need something other than 002 would we do
another (temporary) umask.
then i was able to create a list by using bin/newlist; it did create a directory in lists/listname@server/ (empty directory)
Yep it still does this because this is where it will store things
like list-specific templates and such. It may eventually go away,
but for now it should be fine to exist and be empty.
I found a sqlite db in data/mailman.db (I understand that to use a
MySQL db I just need to create it and change the SQLAlchemy variable to
reflect this db??)
Should be, although I haven't tried it. Once the SQLite engine works
well enough I definitely plan on trying MySQL at least.
the web server launched ok on port 2580 (I guess I'll have to
ProxyPass to it from apache, which is fine).
Yep, there are comments in Defaults.py about this. We'll get this
into the documentation asap. One question I have is whether we
should make this the /only/ (or only supported) way to run MM2.2?
It broke however when I tried to connect to it (put the list admin
passwd, and got : UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9
in position 207: ordinal not in range(128)maybe because I chose "french" as default language (I then switched to english, created another list, and got to the web ui)
Yikes. The other thing I really want to do is to get our unicode
story cleaned up. Again, ideally, we'd convert all strings to and
from their encodings at the edges of the system and only deal
internally with unicode. That's one reason why I'm using unicode
columns in the SA tables. I will try to reproduce this.
tried to modify the list options thu the web ui and got
SQLError: (OperationalError) attempt to write a readonly database
'UPDATE "List data" SET bounce_processing=? WHERE "Listdata".list_id
= ?' [0, 2]
This will happen if the permissions on data/mailman.db are
incorrect. Try ensuring group write on that file and try again.
I haven't tried yet to convert an older list to the new system.
Cool, again, your testing is much appreciated!
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin)
iQCVAwUBRZ0zwXEjvBPtnXfVAQL/8wP9HkM0YrGZNRMz/VUBDQPSlEi4lYDhkZIU 7PX1/+iBiM2XX8xxpenvoU4mu8lBmJrjIW2mEas36GxiFIdak/MEfu7K6w852WAW 8aMUfl+XrIcyyutIsS25dP54qTbhisvxCjTVgHAbzzDXqe0KP6CmOVGYBk1F++kr 92RNEMP1I9M= =xiyk -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jan 3, 2007, at 9:37 AM, Fil wrote:
I had to tweak permisions on logs/* to be able to launch bin/ mailmanctl start, as they had been created with g-w (the logs/ dir is
correctly g+sxw)
I think I solved the permission problems in r8131. check_perms will
complain more loudly than it should but everything else ought to work
correctly now. Working around SQLite's, er, feature of ignoring
umask was fun.
It broke however when I tried to connect to it (put the list admin
passwd, and got : UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9
in position 207: ordinal not in range(128)maybe because I chose "french" as default language (I then switched to english, created another list, and got to the web ui)
I'll try to take a look at this at some point, but I think there's
still a lot of unicode cleaning up to do. This may be shallow
though; we'll see.
Feel free to keep poking at it, and thanks! :)
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin)
iQCVAwUBRZ31H3EjvBPtnXfVAQKaMAP7BoN4+AZfzIFqe+yejHrbMKZCGYpeMpWv L+CZD0ztrMorjFkQCxsqg+ARiWbSpSDFqz8PWREdZ4Qe2eZ0jvmhT3iljlEqpMGa vx7st04G1p8EqnUrJIR6Ddo4eDdZx3zIRbvDZWPARJlY0YvGft9c+he+osMw7hjm HaPya1HCbqc= =yzxn -----END PGP SIGNATURE-----
participants (5)
-
Barry Warsaw -
Fil -
John W. Baxter -
Tokio Kikuchi -
菊地時夫