I believe I've found out how to reliably reproduce the performance
problemsI've noticed here at VA and at Kanga.Nu, and which Barry and
another (forget name, sorry) have observed as well:
1) Create a moderated list.
2) Subscribe 200 addresses to the list (can be bogus addresses but
the local MTA must accept them)
3) Post at least 30 messages of an average of at least 2K size to
the list.
4) Go to the moderation page, approve every message, and hit
submit.
5) Watch your system load peg and stay there for an obscenely long
time.
--
J C Lawrence Home: claw(a)kanga.nu
---------(*) Linux/IA64 - Work: claw(a)varesearch.com
... Beware of cromagnons wearing chewing gum and palm pilots ...
Some of our users complained about the automatically generated
passwords that are sent out when a list is imported or if an admin
subscribes someone. Especially the ` and ^ characters are major
problem because these may be treated as parts of composite characters
in some enviroments (` followed by a might be displayed as the same
character as à in HTML) and so on. Also, upper case characters
impose an extra mental burden ;)
Anyway, I modified our Mailman which now has a function (method?)
Utils.GetRandomPassword(length)
which generates passwords of the given length with a restricted
alphabet, namely: a-x, 2-9, excluding characters o and l as well
as digits 0 and 1 which may be confused and y, z (german keyboards
swap these, in the past, this cause trouble too ;)
I would like to offer this patch unless there are good reasons why this
should be avoided. The main concern is certainly a higher risk to
crack such passwords (only 30 possibilities instead of 64) but this
could easly be matched by using 5 character passwords:
possibilities strength
---------------------------
64^4 = 16777216 1
30^4 = 810000 0.05
30^5 = 24300000 1.45
As far as I have seen, this patch involves replacing certain calls to
GetRandomSeed in a few places such as:
bin/add_members, Mailman/Cgi/admin.py, Mailman/MailCommandHandler.py
Any comment?
+gg
--
Gerhard.Gonter(a)wu-wien.ac.at Fax: +43/1/31336/702 g.gonter(a)ieee.org
Zentrum fuer Informatikdienste, Wirtschaftsuniversitaet Wien, Austria
Hello all,
> We've made today the i18n just for one file "bin/newlist" and toke
> some minutes. It's too simple.
> You just must apply the attached file "newlist.path.i18n" for
> bin/newlist and generate "mailman.mo" file, like:
> msgfmt -o mailman.mo mailman.po
> After, You just copy to /usr/share/locale/$LANG/mailman.mo
> for LANG=pt_BR:
> cp mailman.mo /usr/share/locale/pt_BR/LC_MESSAGES/mailman.mo
>
> It's working fine.
I'd like to know if there are somebody working in i18n?
thks
fabio(a)conectiva.com.br
A number of people have experienced corruption of config.db files when
the disk gets full. While I was away on vacation, we experienced the
same thing on python.org. The ugly thing is that when the disk gets
full, enough of config.db can be written so that it is a valid marshal
of an empty dictionary. This means that lists with corrupt config.db
files will still load, but will start having very cryptic errors such
as attributes missing on the MailList objects, etc.
The fix in those cases is to free up space on the disk and then copy
config.db.last to config.db for all the corrupted lists. I think this
is suboptimal. I'd rather have config.db /only/ be changed if we can
guarantee (as best as possible) that the marshal was written
completely and successfully.
Attached is a patch to do this. I am not checking it into the CVS
tree at the moment because it's such a critical section of code. I
want to stress it a little more myself, but I would greatly appreciate
comments, or testing by y'all. You might want to use a
non-production, or well backed up installation. I feel pretty
confident about the change, but still, get this wrong and your lists
are toast.
The idea is that the marshal is first written to a temp file called
config.db.tmp.<mypid>. If this succeeds completely, then
1) config.db.last is unlinked
2) a hard link config.db.last -> config.db is created
3) a hard link config.db -> config.db.tmp.<mypid> is created
4) config.db.tmp.<mypid> is unlinked
Comments?
-Barry
Index: MailList.py
===================================================================
RCS file: /projects/cvsroot/mailman/Mailman/MailList.py,v
retrieving revision 1.126
diff -c -r1.126 MailList.py
*** MailList.py 1999/06/15 07:50:54 1.126
--- MailList.py 1999/06/30 15:52:16
***************
*** 743,765 ****
# pretty hosed. That's a good reason to make this a daemon not a
# program.
self.IsListInitialized()
! fname = os.path.join(self._full_path, 'config.db')
! fname_last = fname + ".last"
! file = aside_new(fname, fname_last, reopen=1)
! dict = {}
for key, value in self.__dict__.items():
if key[0] <> '_':
dict[key] = value
try:
! marshal.dump(dict, file)
! file.close()
! except IOError, status:
! # Darn - try to resurrect the old config.db.
! file = aside_new(fname_last, fname, reopen=0)
! self.LogMsg("error",
! "Failed config file write '%s',"
! " old config resurrected." % `status.args`)
! Utils.reraise()
self.CheckHTMLArchiveDir()
def Load(self, check_version = 1):
--- 743,780 ----
# pretty hosed. That's a good reason to make this a daemon not a
# program.
self.IsListInitialized()
! # copy all public attributes to marshalable dictionary
! dict = {}
for key, value in self.__dict__.items():
if key[0] <> '_':
dict[key] = value
+ # we want to write this dict in a marshal, but be especially paranoid
+ # about how we write the config.db, so we'll be as robust as possible
+ # in the face of, e.g. disk full errors. The idea is that we want to
+ # guarantee that config.db is always valid. The old way had the bad
+ # habit of writing an incomplete file, which happened to be a valid
+ # (but bogus) marshal.
+ fname = os.path.join(self._full_path, 'config.db')
+ fname_tmp = fname + '.tmp.' + `os.getpid()`
+ fname_last = fname + ".last"
+ omask = os.umask(007)
try:
! try:
! fp = open(fname_tmp, 'w')
! marshal.dump(dict, fp)
! fp.close()
! except IOError, status:
! os.unlink(fname_tmp)
! self.LogMsg('error',
! 'Failed config.db file write, retaining old state'
! '\n %s' % `status.args`)
! Utils.reraise()
! # now move config.db -> config.db.last
! # then move config.db.tmp.xxx -> config.db
! aside_new(fname, fname_last)
! aside_new(fname_tmp, fname)
! finally:
! os.umask(omask)
self.CheckHTMLArchiveDir()
def Load(self, check_version = 1):
Hi,
This message is in refrence to Jitterbug ID incoming/64.
I was just wondering if anyone had a patch or other fix I could use to
resolve the problem until it is fixed officially in mailman.
Thanks,
Rob
I'm not subscribed to this list, so please Cc: me on any replies.
I hope this is the right place to send bug reports.
Anyway, the bug looks like a think-o in GatewayManager.py:
== 8< cut here 8< ==
Bug in Mailman version 1.0rc2
We're sorry, we hit a bug!
If you would like to help us identify the problem, please email a copy of
this page to the webmaster for this site with a description of what
happened. Thanks!
Traceback:
Traceback (innermost last):
File "/var/lib/mailman/scripts/driver", line 112, in run_main
main()
File "/usr/lib/mailman/Mailman/Cgi/admindb.py", line 112, in main
HandleRequests(doc)
File "/usr/lib/mailman/Mailman/Cgi/admindb.py", line 200, in HandleRequests
list.HandleRequest(request, v, form[comment_key].value)
File "/usr/lib/mailman/Mailman/ListAdmin.py", line 122, in HandleRequest
self.HandlePostRequest(request_data[2:], value, comment)
File "/usr/lib/mailman/Mailman/ListAdmin.py", line 131, in HandlePostRequest
self.Post(msg, 1)
File "/usr/lib/mailman/Mailman/MailList.py", line 1285, in Post
self.SendMailToNewsGroup(msg)
File "/usr/lib/mailman/Mailman/GatewayManager.py", line 143, in SendMailToNewsGroup
msg.SetHeader('Subject', '%s(no subject)' % prefix)
NameError: prefix
------------------------------------------------------------------------
Environment variables:
Variable Value
DOCUMENT_ROOT /var/www/sluglug
UNIQUE_ID N3QIwYByMfAAAC2oPIw
HTTP_ACCEPT_ENCODINGgzip
CONTENT_LENGTH 315
CONTENT_TYPE application/x-www-form-urlencoded
PATH_TRANSLATED /var/www/sluglug/sluglug
REMOTE_ADDR 198.4.83.36
GATEWAY_INTERFACE CGI/1.1
HTTP_COOKIE sluglug:admin="(lp0\012S'198.4.83.36'\012p1\012aI930351160\012aI930361960\012aI-1683555072\012a."
SERVER_PROTOCOL HTTP/1.0
HTTP_ACCEPT_LANGUAGEen
REMOTE_PORT 4012
SERVER_PORT 80
HTTP_CONNECTION Keep-Alive
HTTP_USER_AGENT Mozilla/4.5 [en] (X11; I; Linux 2.2.10 i586)
HTTP_ACCEPT_CHARSET iso-8859-1,*,utf-8
HTTP_ACCEPT image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*
REQUEST_URI /mailman/admindb/sluglug
PATH /bin:/usr/bin:/sbin:/usr/sbin
QUERY_STRING
SCRIPT_FILENAME /usr/lib/mailman/cgi-bin/admindb
PATH_INFO /sluglug
HTTP_HOST sluglug.ucsc.edu
REQUEST_METHOD POST
SERVER_SIGNATURE
SCRIPT_NAME /mailman/admindb
SERVER_ADMIN ben(a)cse.ucsc.edu
SERVER_SOFTWARE Apache/1.3.3 (Unix) Debian/GNU
PYTHONPATH /var/lib/mailman
HTTP_REFERER http://sluglug.ucsc.edu/mailman/admindb/sluglug
SERVER_NAME sluglug.ucsc.edu
Content-type: text/html
Database Updated...
------------------------------------------------------------------------
Administrative requests for 'Sluglug' mailing list
View or edit the list configuration information
There are no pending requests.
------------------------------------------------------------------------
Sluglug list run by che(a)debian.org
[Delivered by Mailman] v 1.0rc2
== 8< cut here 8< ==
Here's a patch that might fix this. I'm not sure, but at a quick
glance this might solve the problem:
--- GatewayManager.py.orig Fri Jun 25 15:56:06 1999
+++ GatewayManager.py Fri Jun 25 15:56:52 1999
@@ -140,7 +140,7 @@
if not re.match('(re:? *)?' + re.escape(subjpref), subj, re.I):
msg.SetHeader('Subject', '%s%s' % (subjpref, subj))
else:
- msg.SetHeader('Subject', '%s(no subject)' % prefix)
+ msg.SetHeader('Subject', '%s(no subject)' % subjpref)
if self.reply_goes_to_list:
del msg['reply-to']
msg.headers.append('Reply-To: %s\n' % self.GetListEmail())
Ben Gertzfield
--
Brought to you by the letters S and D and the number 2.
"Sculch is junk."
Debian GNU/Linux maintainer of Gimp and GTK+ -- http://www.debian.org/
Hello Mailman developers,
Attached to this post please find a patch against the current Mailman CVS
tree which adds support for Kerberos and AFS. Users who do not enable the
Kerberos or AFS functionality will not experience any change in the
functionality of their server vs. the current CVS tree.
To enable support for Kerberos, first install Mailman on a system with a
Kerberized httpd (such as Stronghold, or use mod_auth_kerb). You can then
use the following configuration variables in mm_cfg.py to enable it:
ADMIN_USERS = [] # A list of usernames which should be
# considered site adminstrators when
# authenticated via Kerberos. This is to
# be used in place of the old "site admin
# password" on a Kerberized server.
KERBERIZED_SERVERS = [] # A list denoting which servers should be
# considered Kerberized. Each entry
# should consist of the preferred alias
# for that machine concatenated with a ':'
# and the port, to allow Kerberized and
# non-Kerberized servers to co-exist on a
# single machine.
NONE_KERBERIZED = 0 # Boolean value which disables Kerberos on
# all servers. This overrides
# KERBERIZED_SERVERS.
ALL_KERBERIZED = 0 # Boolean value to enable Kerberos on all
# servers. This overrides NONE_KERBERIZED
# and KERBERIZED_SERVERS.
Of course there are some limitations to the Kerberos functionality. The
Kerberized httpds I know of only support one Kerberos realm at a time. If
you use Kerberos without SSL, you're crazy (your password will go in the
clear). This patch has only been tested with Kerberos V5, but I see no
reason it should not work with V4, because the httpd handles those
differences anyway, I believe (you should upgrade to V5 anyway, though :).
Kerberized servers should interoperate, including sharing lists, cleanly
with non-Kerberized servers. Cookies and passwords (apart from Kerberos
passwords) are now only used by non-Kerberized servers. In theory it
should be possible to substitute other authentication methods in place of
Kerberos, but this has not been tested.
To enable support for AFS, use some or all of the following command-line
arguments to `configure':
--with-mail-preauth=COMMAND
--with-mail-postauth=COMMAND
--with-www-preauth=COMMAND
--with-www-postauth=COMMAND
--with-cron-preauth=COMMAND
--with-cron-postauth=COMMAND
These tell Mailman which commands to execute to get and destroy an AFS
token before and after it performs mail-driven, web-based, and cron tasks.
Different commands are available because your web server may have a
different keytab file than your mail server, and who knows which server
you might use to run the cron jobs. Of course non-AFS commands could be
used instead, but this was added for AFS compatibility.
As indicated by Christopher Lindsey in a previous post, we at NCSA are
hoping to migrate to Mailman soon, and we need these changes to do so. We
hope that this patch will be integrated into the main distribution so we
don't have to maintain a parallel patch.
Thank you for a useful product, and for reading all of that,
--
Paul Hebble <hebble(a)ncsa.uiuc.edu>
NCSA
On Wed, 23 Jun, Rob Francis wrote:
> It seems kind of odd to me that if I know someone's email address on a
> list that I can go to the Info page and enter their email address, and
> then w/o a password see what options they have set. Granted, I can't
> change the options, but why even let someone get to see the options a
> person has selected?
>
> Just wondering if this was a decision made on purpose, or perhaps an
> oversight.
I'd actually say it's somewhere in between.
Originally all the settings pages - admin and admindb, as well as the user
options pages - were like the user options pages, not hidden behind a
password. The others were moved behind the cookie wall, and i expect it
would have been easy enough to hide the user options page, too, except that
then you'd prevent users missing their password from getting at the "send me
my password" option! Kinda defeats the purpose. The proper answer would be
to make the user options page two tier, with the "send me my password"
button in front of the cookie password, and the rest behind. Evidentally
this extra elaboration was high enough effort to exceed the attention
threshold available for the cookie-protection switchover, and nobody's
revisited it since. And in truth, most of the telling information about the
options is revealed on the roster page - the membership on the list, whether
or not they're digest subscribers, and whether or not their delivery is
disabled.
There are many such rough edges - some for simple expediency, and some that
would require more infrastructure to be properly addressed. (An example of
the latter is holding messages for approval directly in the config.db - we
should have at least a basic catalog mechanism in which the mailing lists
can store and retrieve such messages. This would take yet more overhead
effort than the simple accommodation we could make for cookies around the
user options page...)
Ken Manheimer
klm(a)digicool.com
On Thu, 24 Jun 1999, Christoph Lameter wrote:
> Package: mailman
> Version: 1.0rc2-4
> Severity: normal
>
> Got this strange message send to me. Seems to be due to a cron job
someone who is already subscribed tried to subscribe again. mailman should
catch this exception and reply politely, I'm forwarding this upstream.
> Date: Thu, 24 Jun 1999 06:39:05 -0700
> From: pptp-admin(a)opensource.captech.com
> To: pptp-admin(a)opensource.captech.com
> Subject: Unexpected Mailman error
>
> An unexpected Mailman error has occurred in
> MailCommandHandler.ParseMailCommands(). Here is the traceback:
>
> Traceback (innermost last):
> File "/var/lib/mailman/Mailman/MailCommandHandler.py", line 204, in ParseMailCommands
> self.__dispatch[cmd](args, line, msg)
> File "/var/lib/mailman/Mailman/MailCommandHandler.py", line 583, in ProcessConfirmCmd
> self.ProcessConfirmation(cookie)
> File "/var/lib/mailman/Mailman/MailList.py", line 1013, in ProcessConfirmation
> self.ApprovedAddMember(email_addr, password, digest)
> File "/var/lib/mailman/Mailman/MailList.py", line 905, in ApprovedAddMember
> raise e, v
> MMAlreadyAMember
--
Madarasz Gergely gorgo(a)caesar.elte.hu gorgo(a)linux.rulez.org
It's practically impossible to look at a penguin and feel angry.
Egy pingvinre gyakorlatilag lehetetlen haragosan nezni.
HuLUG: http://mlf.linux.rulez.org/
