Mailman 3 November 2000 - Mailman-Developers

Re: [Mailman-Developers] Towards 2.1
by Nigel Metheringham Nov. 23, 2000

Nov. 23, 2000

darrell(a)grumblesmurf.net said: > 2) In pipermail, I'd like to be able to define HTML tags that can be > wrapped around non-important parts of the particular message. > Specifically, this is to enable you to exclude certain parts of the > message from indexing, such as navigation information. (I'm thinking > specifically of the   tags. > That is, of course, a rather obscure feature. :) There is already a patch to do this - its marked as being for htdig, but all it actually does is inject definable indexer on/off statements into the generated HTML. It does assume that your indexer parses are utilises the indexer <META> tags that already appear in pipermail HTML. See https://sourceforge.net/patch/?func=detailpatch&patch_id=102422&group_ id=103 or if that doesn't work, Patch 102422 Nigel. -- [ - Opinions expressed are personal and may not be shared by VData - ] [ Nigel Metheringham Nigel.Metheringham(a)VData.co.uk ] [ Phone: +44 1423 850000 Fax +44 1423 858866 ]

1 0

Re: [Mailman-Developers] Towards 2.1
by Dan Mick Nov. 23, 2000

Nov. 23, 2000

> barry(a)digicool.com (Barry A. Warsaw) writes: > > > >From the TODO list, things I can imagine tackling for 2.1 include: > > I would like to see a couple things added. > > 1) the patch I wrote to sort the list and break it up in chunks > when passed to the mailer -- FWIW, I've been using it since I > wrote it without problems, surely others have, too. Like I said > earlier, at worst, it's an innocuous feature, but in general it > does benefit delivery performance. I've been using it, and I think I love it. I think it ought to be optionally added, too. A pleasant side-effect of the domain sorting is when Hotmail gets screwed up (once or twice a week), all the hotmail mail bounces at the same time. :-}

1 0

v2 Management Documents, GFDL
by Christopher Kolar Nov. 22, 2000

Nov. 22, 2000

I am pleased to announce that v2 of the Mailman list manager's documentation is available at www.aurora.edu/~ckolar/mailman. Aurora University has generously agreed to release these documents under the GNU Free Documentation License (GFDL), details are on the site. --chris

1 0

Re: [Mailman-Developers] Towards 2.1
by Darrell Fuhriman Nov. 22, 2000

Nov. 22, 2000

barry(a)digicool.com (Barry A. Warsaw) writes: > >From the TODO list, things I can imagine tackling for 2.1 include: I would like to see a couple things added. 1) the patch I wrote to sort the list and break it up in chunks when passed to the mailer -- FWIW, I've been using it since I wrote it without problems, surely others have, too. Like I said earlier, at worst, it's an innocuous feature, but in general it does benefit delivery performance. 2) In pipermail, I'd like to be able to define HTML tags that can be wrapped around non-important parts of the particular message. Specifically, this is to enable you to exclude certain parts of the message from indexing, such as navigation information. (I'm thinking specifically of the   tags. That is, of course, a rather obscure feature. :) Darrell

1 0

Errors in mailman that I cannot figure out
by Bradley M. Kuhn Nov. 22, 2000

Nov. 22, 2000

I am writing on behalf of the GNU project. We have an discovered some sort of bug in mailman. The .../logs/errors file is getting: Nov 22 11:24:01 2000 qrunner(29262): Traceback (innermost last): Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 278, in ? Nov 22 11:24:01 2000 qrunner(29262): kids = main(lock) Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 247, in main Nov 22 11:24:01 2000 qrunner(29262): keepqueued = dispose_message(mlist, msg, msgdata) Nov 22 11:24:01 2000 qrunner(29262): File "/home/mailman/cron/qrunner", line 121, in dispose_message Nov 22 11:24:01 2000 qrunner(29262): if BouncerAPI.ScanMessages(mlist, mimemsg): Nov 22 11:24:01 2000 qrunner(29262): File "/com/mailer/mailman/Mailman/Bouncers/BouncerAPI.py", line 59, in ScanMessages Nov 22 11:24:01 2000 qrunner(29262): addrs = func(msg) Nov 22 11:24:01 2000 qrunner(29262): File "/com/mailer/mailman/Mailman/Bouncers/Catchall.py", line 132, in process Nov 22 11:24:01 2000 qrunner(29262): username = string.split(line)[1] Nov 22 11:24:01 2000 qrunner(29262): IndexError : list index out of range This was happening with 2.0beta6, so I upgraded to see if it's a bug fixed by 2.0, but it isn't. I think this may be related to a particular file in the queue being formatted oddly, but the result is the entire queue is held up. I am looking a bit deeper into the code right now, but I'd appreciate help. (Mailing lists at gnu.org are currently down becasue of this bug). I will send more information as I discover it.

1 1

Re: [Mailman-Developers] Re: Future of pipermail?
by J C Lawrence Nov. 22, 2000

Nov. 22, 2000

On Wed, 22 Nov 2000 02:30:30 -0500 Bill Bumgarner <bbum(a)codefab.com> wrote: > Agreed. I would much rather see Mailman have an option to > configure a list such that it is > text-only-bounce-mime-straight-to-/dev/null for those situations > that warrant it (and to remain New Communist Community Compliant). Mailman needs the ability to define basic filters/rules for what is acceptable for being considered as a list posting. Sometimes that's MIME. Sometimes it is the source domain. Sometimes its an LDAP tag relted to a header field, or something else entirely. > Similarly, I would also like to see a per-user option that would > allow a mailing list subscriber to say "try and filter anything to > ASCII and never, ever send me an attachment". While pursueing the LServe game we need to not lose sight of the little guys running a couple lists for their club of 20-some model train enthusiasts. -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

3 2

Re: Future of pipermail?
by Bill Bumgarner Nov. 22, 2000

Nov. 22, 2000

WebDAV works really, really well as an archiver. And it would be particularly cool if the archive already had the MIME pieces exploded out. Even cooler would be if the messages sent to the list were rewritten such that all of the MIME pieces-- typically large, on some lists-- were actually URLs to the archived pieces on teh WebDAV server. If it had an interface that sent an HTTP message to some random webserver/appserver/cgi-bin letting it know a new message was submitted so that server could decide an appropriate time to rebuild an index file, that would allow the presentation of the archive to be offloaded from Mailman. Since WebDAV will be included with every copy of Apache 2.0 by default, there really isn't that much additional "moving pieces" load on the configuration. Mailman already requires diddling the web server configuration-- copy/paste a few more lines isn't that big of a deal. Anyone want to take this on? Oh, wait a minute, that's right... it has already been done! ftp://ftp.codefab.com/pub/unsupported/mailman_20000821_1123.tar.gz It is based on a version of Mailman from sometime around May or so... so, it needs to be updated to the latest version from the repository. However, it DOES work and has been used in production systems [who will remain nameless to protect this community from me venting] for quite some time. The company that hired me to build this actually paid a bunch of $$$ for it to be built-- as such, I actually had the freedom to thiink about doing it right and not just hacking it on as fast as possible. I don't know if I did do it right, but Barry provided quite a bit of guidance and I don't think it is a totally offensive set of changes... b.bum

6 13

Re: [Mailman-Developers] Re: Future of pipermail?
by J C Lawrence Nov. 22, 2000

Nov. 22, 2000

On Tue, 21 Nov 2000 23:09:55 -0800 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > At 10:58 PM -0800 11/21/00, J C Lawrence wrote: >> I've spent considerable time thinking about this and believe it >> is to the lists' advantage in this case to hold firm. > you'll lose. AOL is the first to make this mandatory for their > users, but users want these features. And besides, AOL is the 800 > pound gorilla, and it's eating its lunch on your fence. Do you > really think they're going to change back before your fence > collapses under it? No, but I also realise that AOL actually is't sitting on my fence. They are sitting on other people's fence to be sure but their presence in my neighborhood is small enough to be noise. >> Aside: Less than 5% of my members are from AOL. Less than 1% of >> my semi-regular posters are from AOL. The impact here (for me) >> is quite small. > lucky you. you're not typical then, not given the size of AOL. Precisely. > But -- it's a losing battle, JC. AOL is the first. It won't be the > only. My numbers show me that not only do ~90% of mail users have > support for MIME email (stylized text, html, etc.), but most of > them want to use it. And they don't like having to go through > gyrations to avoid it -- and most have no clue how, and don't want > to. While I agree (with some distaste), those gyrations have shown themselves to have positive value for me. FWIW I also enforce quoting style (new text below quote), quoting formats (must have leading quote characters on quoted lines), correct attributions for all quoted text, etc, which are leading directly against AOL, MSN et al, but which have also helped define the value I offer. It a per-case specific question, not suitable for the general case. > t few people are in that kind of sellers market.... Too true. -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

2 1

Re: [Mailman-Developers] Re: Future of pipermail?
by J C Lawrence Nov. 22, 2000

Nov. 22, 2000

On Tue, 21 Nov 2000 18:46:33 -0800 Chuq Von Rospach <chuqui(a)plaidworks.com> wrote: > At 4:29 PM -0800 11/21/00, J C Lawrence wrote: >> For me WebDAV raises concerns centering around authentication and >> access security. > Authentication is a big bugaboo in general, which Barry and I have > hashed around a bit. More on that someday, maybe. FWLIW I see authentication visavis Mailman as a two level problem: list activities (command confirmations) access control The former can be handled with ad hod dynamically generated tokens much as subscribe confirms are handled now. I've posted some notes on good implementations on this previously (I liked the bit about auto-genning an URL that did the command confirmation). The latter just needs to be abstracted to a small script which accepts two command line parameters: UserID and Password. The user can then replace that script with anything he pleases, thus authenticating agsinst he pleases be it SQL, LDAP, or lunar weather sensors. Ditto BTW holds tru for handling membership lists: just have a tool which when run returns the list of members. Simple command line options then spec returning account details, configs, etc. A little over head for text parsing, but not a whole lot (ObNote XML is a reasonable communications format). Simple, easy to extrapolate, nice efficient piped IO, etc. -- J C Lawrence claw(a)kanga.nu ---------(*) : http://www.kanga.nu/~claw/ --=| A man is as sane as he is dangerous to his environment |=--

3 3

Re: [Mailman-Developers] Re: Future of pipermail?
by Bill Bumgarner Nov. 22, 2000

Nov. 22, 2000

Let me describe a bit about what I envisioned when going down the Mailman+WebDAV implementation path. Maybe it is reasonable, maybe not. I agree with Chuq, Ken and Barry regarding various concerns with Mailman. That the integration of technologies as a part of the required baseline makes the tool *less* flexible and *less* easily accesible; that security is a major concern; and that the future of Mailman should be about refining/abstracting what we have, not glomming new bits onto it. With that said, let me discuss some of the issues I faced and some of the thinking that resulted. This is in no particular order and is not a criticism of anyone's coding contributions! My code is *not*Not*NOT* perfect and it all should be reviewed, as well!! - archival of messages is a lot more than just writing the bodies to a web server and then generating some kind of automatic TOC/index. MIME based email content is not only a reality, but it is often the default. Mailman needs to more gracefully handle mime-types; automatic filters would be one solution [very desirable for things like a pure discussion list] and automatic decoding/filing/handling of attachments is another solution [and very desirable for things like project discussion lists, etc]. - message rewriting is a really powerful concept and should be an abstract and first class part of the Mailman core processing pipeline. When I implemented the decode-and-file-to-WebDAV solution, the first challenge was to build a handler that sat in the delivery pipeline and did some basic decode-and-rewrite-based-on-mailing-list-configuration as the message passed through the pipe. Once decoded/rewritten, then filing/processing/filtering/etc can all be abstracted *away* from the actual decode/rewrite process. - Python's built in MIME type handling code sucks. It is great if you want to *compose* MIME content, it sucks if you want to rip apart existing mime content and have any kind of random access. As such, I turned to mimecntl.py by <some very good developer's name I can't remember>. It was quite useful and enabled me to get the job done, but does not have the most obvious of APIs. Did it improve in 2.0? I don't think so, but have not looked closely. - the current solution for creating an archive is lacking-- that doesn't mean it doesn't work. It works and it works very well, but it is *not* modular, does *not* follow the design patterns of the rest of the tool and makes it *very* difficult to slap in replacement models for archival (has this changed since, say, may? If it has, I'll have to retract this until I look again). Before WebDAV or anything else can be considered, the actual archival process needs to be abstracted away from the hack that is in there now. - gatewaying messages to an HTTP server as they are processed is an extremely powerful addition [and very easy to do]. However, it is useful to *also* add the ability to automatically requeue messages for blastting at the HTTP server if the server returns certain error codes. This is done in the codebase I posted an ftp:// to earlier. This allows for *extremely* flexible integration of Mailman w/other infrastructure. - WebDAV does *not* address security. Nor does it contribute to "security issues". Any web based archive of data is only as secure as the Web server that allows access to the content. Given that the Web is now the *standard* way of presenting things like Mailing list archives, this is a security issue mailman faces *regardless* of the use of WebDAV as the means of storing the archive. WebDAV is merely an extension to the existing HTTP protocol. However, WebDAV does allow for interesting potential security implementations; URLs that automatically time out after X minutes/hours/days are trivial to implement. - for the archival of plain text messages, WebDAV is overkill [as Chuq mentions]. However, as soon as you move to archiving mime attachments, it quickly becomes extremely advantageous to archive various properties with the archived message pieces. Example; if my modified Mailman receives a message from a Macintosh with an attachment, it stores the Creator/Type information [and other extended headers] as a set of properties associated with the Resource on the attachment server. This allows for... - ....restoring decoded attachments and reencoding back to their original state with their original headers is an extremely cool feature. With this [in the modified version of Mailman+WebDAV], it is quite easy to send a request to Mailman that causes it to reconstitute a single attachment in all of its original encoded glory with its original headers based entirely on the decoded information found on the WebDAV server. This message can then be automatically sent to the user and will be treated just like the original attachment in the original message. - if we are to manage the complexity associated with the integration of numerous technologies, it is only going to happen through well refined and highly modular APIs.... b.bum

2 5