Mailman 3 July 2000 - Mailman-Developers

Locking issue, Mailman 2.0b4
by John MacKenzie July 18, 2000

July 18, 2000

I'm still Experiencing that locking issue, below is a listing or the lockfiles present beforer I did an rm -f irish-emigrant* so i could access the list admin pages. Anyone come up with a solution yet ? Regards - John <site>.lock irish-emigrant.lock.localweb.local.ie.4535 l.lock l.lock.localweb.local.ie.25619 apology.archiver.lock irish-emigrant.lock.localweb.local.ie.4557 l.lock.localweb.local.ie.13572 l.lock.localweb.local.ie.3952 irish-emigrant.lock irish-emigrant.lock.localweb.local.ie.4562 l.lock.localweb.local.ie.14264 l.lock.localweb.local.ie.3955 -- John MacKenzie | Unix Systems Admin | e: mailto:jmackenzie@local.ie ___________________________________________________________________ Stay in touch your local area through Local Ireland http://chat.local.ie/chat/index.html ___________________________________________________________________ local ireland | dublin | new york | http://www.local.ie t: +353 1 676 8996 f: +353 1 283 9988

2 2

Re: [Mailman-Developers] Cookie problems
by Dan Mick July 18, 2000

July 18, 2000

Dan Mick wrote: > 2) MakeCookie sets c[key]['expires'] to mm_cfg.ADMIN_COOKIE_LIFE, not to > 'expires', which is calculated above; that means the cookie object doesn't > have the 'issued' time added into its 'expires' field. Given that > MakeCookie is apparently the source of the HTTP output, this seems wrong too.. > seems like the Expires time is always going to be just the lifetime, not > the actual expiry date. > > Am I misreading something? Yes...Cookie's _getdate() is used to set the expire time. How unintuitive! "It's just a dictionary that holds attribute values...but this one is only a delta value..." <sigh> Still, point 1 makes me worry.

1 0

chunkify suggestion, with patch.
by Darrell Fuhriman July 18, 2000

July 18, 2000

I've sent a modified version of chunkify() that behaves, I think, I bit more intelligently. I've modified it so that it sorts the recipient list by the domain, then breaks that list into chunks of SMTP_MAX_RCPTS. I've also changed the Default SMTP_MAX_RCPTS to 20 for reasons given below. The advantage of doing it this way is that you get the advantages of smaller recipients passed of too the MTA and consequently better turn around time to the list members. By sorting it, any domains that will be unreachable are grouped together, thus punishing only that delivery process. Ideally, all messages intended for a particular domain would end up in the same delivery process, but my code doesn't do that right now. This should probably entail making a second tuneable variable, say, SMTP_MAX_QUEUE, which is the chunk size, and returning SMTP_MAX_RCPTS to it's original intention of not exceeding an MTA limit. Then if you had many recipients in a single domain, you could group them all together in one chunk exceeding SMTP_MAX_QUEUE, but not more than SMTP_MAX_RCPTS. But, like I said, that doesn't happen right now. Some of you may recognize this as the idea behind bulk_mailer. :) Also, one problem with this code is that the domain_sort fucntion is really rather slow. If anyone has suggestions for speeding it up, that'd be great. :) Darrell Diffs against b4 --- Mailman/Handlers/SMTPDirect.py~ Fri Jun 2 21:59:45 2000 +++ Mailman/Handlers/SMTPDirect.py Mon Jul 17 03:19:46 2000 @@ -106,48 +106,79 @@ def chunkify(recips, chunksize): - # First do a simple sort on top level domain. It probably doesn't buy us - # much to try to sort on MX record -- that's the MTA's job. We're just - # trying to avoid getting a max recips error. Split the chunks along - # these lines (as suggested originally by Chuq Von Rospach and slightly - # elaborated by BAW). - chunkmap = {'com': 1, - 'net': 2, - 'org': 2, - 'edu': 3, - 'us' : 3, - 'ca' : 3, - } - buckets = {} - for r in recips: - tld = None - i = string.rfind(r, '.') - if i >= 0: - tld = r[i+1:] - bin = chunkmap.get(tld, 0) - bucket = buckets.get(bin, []) - bucket.append(r) - buckets[bin] = bucket + # If we turn down the chunksize (i.e. SMTP_MAX_RCPTS), and have + # the addresses sorted by domain, it's much nicer to the MTA and + # to the users. (In the majordomo world, this is what bulk_mailer + # would do.) + # In an ideal world, a single domain wouldn't be split across + # multiple chunks unless a someother threshold had been met. + # I'll save that for sometime when it's not 2:30am. :) + + recips.sort(domain_sort) + # Now start filling the chunks chunks = [] currentchunk = [] - chunklen = 0 - for bin in buckets.values(): - for r in bin: - currentchunk.append(r) - chunklen = chunklen + 1 - if chunklen >= chunksize: - chunks.append(currentchunk) - currentchunk = [] - chunklen = 0 - if currentchunk: + for recip in recips: + if len(currentchunk) >= chunksize: chunks.append(currentchunk) currentchunk = [] - chunklen = 0 + currentchunk.append(recip) + if len(currentchunk) != 0: + chunks.append(currentchunk) return chunks + + +def domain_sort(x, y): + x_longer = 0 + y_longer = 0 + # split the user from the rest (we may need the username later) + x_tmp = string.split(x, '@'); + y_tmp = string.split(y, '@'); + + x_list = string.split(x_tmp[1], '.') + y_list = string.split(y_tmp[1], '.') + x_user = x_tmp[0] + y_user = y_tmp[0] + # now reverse it, to make the code cleaner + x_list.reverse() + y_list.reverse() + + # find out which domain is shorter + # or if they're the same length + if len(x_list) == len(y_list): + for i in range(0, len(x_list)): + ret = cmp(x_list[i], y_list[i]) + if ret != 0: + return ret + # if they're the same length, and we get to this + # point, it's because they're identical domains. + # we'll just compare on the username, 'cause we can + return cmp(x_user, y_user) + elif len(x_list) < len(y_list): + length=len(x_list) + y_longer = 1 + else: + length=len(y_list) + x_longer = 1 + + for i in range(0, length): + ret = cmp(x_list[i], y_list[i]) + if ret != 0: + return ret + # if we get to the point, we've got two domains of the + # form: a.foo.com and foo.com and we decide the longer + # one goes first + if x_longer == 1: + return -1 + if y_longer == 1: + return 1 + # we should never get here + return 0 + def pre_deliver(envsender, msgtext, failures, chunkq): while 1: # Get the next recipient chunk, if there is one --- Mailman/Defaults.py~ Mon Jul 17 03:23:09 2000 +++ Mailman/Defaults.py Mon Jul 17 03:16:47 2000 @@ -56,7 +56,7 @@ PUBLIC_ARCHIVE_URL = '/pipermail' PRIVATE_ARCHIVE_URL = '/mailman/private' -COME_PAGE = 'index.html' +HOME_PAGE = 'index.html' MAILMAN_OWNER = 'mailman-owner@%s' % DEFAULT_HOST_NAME @@ -128,7 +128,7 @@ # Ceiling on the number of recipients that can be specified in a single SMTP # transaction. Set to 0 to submit the entire recipient list in one # transaction. Only used with the SMTPDirect DELIVERY_MODULE. -SMTP_MAX_RCPTS = 500 +SMTP_MAX_RCPTS = 20 # Maximum number of simulatenous subthreads that will be used for SMTP # delivery. After the recipients list is chunked according to SMTP_MAX_RCPTS,

5 8

possible beta4 problem.
by Chuq Von Rospach July 18, 2000

July 18, 2000

I seem to be getting seg faults out of mailman b4: chuqui(a)plaidworks.com 42> chuqui(a)plaidworks.com 42> [Mon Jul 17 13:37:03 2000] [notice] child pid 22880 ex it signal Segmentation fault (11) [Mon Jul 17 13:37:04 2000] [notice] child pid 25902 exit signal Segmentation fau lt (11) As far as I can tell (I'm still snooping, and haven't nailed it down or reproduced it), the dumps are coming from: 09.203.64.2 - - [17/Jul/2000:13:36:34 -0700] "POST /mailman/handle_opts/sharks/ wilki24(a)pacbell.net HTTP/1.0" 200 785 and I've had a few intermittent reports that attempts to change subscription options isn't working right, which would correspond to this... Have no clue why; it works fine for me, and I'm not getting core files to debug iwth (yet), but there seems to be a problem here somewhere... -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) And they sit at the bar and put bread in my jar and say 'Man, what are you doing here?'"

3 4

Bug in 2.0b4
by Chuq Von Rospach July 18, 2000

July 18, 2000

There's a bug in the bounce processing in B4. I just upgraded my productio server (finally!) to mailman, and I'm getting these in the error log: Jul 16 12:38:01 2000 qrunner(20080): Traceback (innermost last): Jul 16 12:38:01 2000 qrunner(20080): File "/home/mailman/cron/qrunner", line 2 09, in ? Jul 16 12:38:01 2000 qrunner(20080): kids = main(lock) Jul 16 12:38:01 2000 qrunner(20080): File "/home/mailman/cron/qrunner", line 1 78, in main Jul 16 12:38:01 2000 qrunner(20080): keepqueued = dispose_message(mlist, ms g, msgdata) Jul 16 12:38:01 2000 qrunner(20080): File "/home/mailman/cron/qrunner", line 6 8, in dispose_message Jul 16 12:38:01 2000 qrunner(20080): if BouncerAPI.ScanMessages(mlist, mime msg): Jul 16 12:38:01 2000 qrunner(20080): File "/home/mailman/Mailman/Bouncers/Boun cerAPI.py", line 54, in ScanMessages Jul 16 12:38:01 2000 qrunner(20080): addrs = func(msg) Jul 16 12:38:01 2000 qrunner(20080): File "/home/mailman/Mailman/Bouncers/DSN. py", line 109, in process Jul 16 12:38:01 2000 qrunner(20080): addrs.append(parseaddr(val)) Jul 16 12:38:01 2000 qrunner(20080): File "/home/mailman/Mailman/Bouncers/DSN. py", line 28, in parseaddr Jul 16 12:38:01 2000 qrunner(20080): atype, addr = string.split(val, ';') Jul 16 12:38:01 2000 qrunner(20080): ValueError : unpack list of wrong size -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) And they sit at the bar and put bread in my jar and say 'Man, what are you doing here?'"

2 2

Re: [Mailman-Developers] chunkify suggestion, with patch.
by J C Lawrence July 18, 2000

July 18, 2000

On Mon, 17 Jul 2000 13:59:26 -0700 (PDT) Darrell Fuhriman <darrell(a)grumblesmurf.net> wrote: > Here's a new patch for SMTPDirect.py. Who decides what get > included in the release, BTW? That would be Barry. BTW: As I'd love to see this patch get taken, please make sure it applies cleanly against the current CVS tree, It helps patches get looked at. -- J C Lawrence Home: claw(a)kanga.nu ---------(*) Other: coder(a)kanga.nu http://www.kanga/nu/~claw/ Keys etc: finger claw(a)kanga.nu --=| A man is as sane as he is dangerous to his environment |=--

1 0

Problem with mailmans locking
by John MacKenzie July 17, 2000

July 17, 2000

Hey guys, Noticed an issue with the lockfiles.... If A lockfile is present and i go to login to the Admin section then it procedes to hang for a long time then bomb out with "document contained no data" However should I delete the lockfile then it logs in instantaeneously... It is However a constantly re-occuring issue. Any ideas ? - John -- John MacKenzie | Unix Systems Admin | e: mailto:jmackenzie@local.ie ___________________________________________________________________ Stay in touch your local area through Local Ireland http://chat.local.ie/chat/index.html ___________________________________________________________________ local ireland | dublin | new york | http://www.local.ie t: +353 1 676 8996 f: +353 1 283 9988

2 4

Subscribe notifications still don't have a 'To:' field
by Marc MERLIN July 17, 2000

July 17, 2000

I reported that a while ago and I was told it was fixed/being fixed. However beta 4 still sends those Emails without inserting a To: field, like mm 1.1. used to do. Marc -- Microsoft is to software what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ (friendly to non IE browsers) Finger marc_f(a)merlins.org for PGP key and other contact information

2 1

2.0beta4 subscribes a pending user without confirmation
by Marc MERLIN July 17, 2000

July 17, 2000

I had an admin page with two pending requests: one blocked message and a user asking to join. I approve the message, submit, the page comes back saying that the message went through, I see the pending sub still waiting. After that, I see in my mailbox that the pending sub was really approved. I reload the web page and it shows that there are no pending requests... BugID 109277 on sourceforge. Marc -- Microsoft is to software what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ (friendly to non IE browsers) Finger marc_f(a)merlins.org for PGP key and other contact information

2 1

Beta 4 problem...
by Chuq Von Rospach July 16, 2000

July 16, 2000

Haven't completely tracked this down yet, but it seems I've got a minor mail loop started across one of the administrative addresses, and the option to limit administrative responses to N days (mine is set to the default 90) isn't catching it. More when I get it. -- Chuq Von Rospach - Plaidworks Consulting (mailto:chuqui@plaidworks.com) Apple Mail List Gnome (mailto:chuq@apple.com) And they sit at the bar and put bread in my jar and say 'Man, what are you doing here?'"

1 0