[Mailman-developers] Mailman 1.0b2 available

[Ken Manheimer]

Finally, i've got a complete package of mailman 1.0b2 ready for
consumption.  This has some substantial features beyond friday(?)'s 
prerelease, particularly support for private archives and fix of a bug
handling addresses containing quotes.  It also has lots of niggles
polished in preparing the release, so i'd strongly suggest using
getting this version even if you got the prerelease.

Here's the encompassing README.  I (in the grand Python tradition -
didn't mean to do it, but i can see now how this happens) will be away 
the rest of this week, for the most part, but may be able to field
some immediate problems or whatever tomorrow.

Below is the encompassing README - cheers!

Ken Manheimer		  klm@python.org	    703 620-8990 x268
	    (orporation for National Research |nitiatives

	# If you appreciate Python, consider joining the PSA! #
		  # <http://www.python.org/psa/>. #


Distribution README for mailman 1.0b2.

This is mailman v1.0b2.0 - probably the first real, full-function
beta-level release of V1.  In case you don't know, mailman is a
maillist management system with full web-based administration and
membership interfaces.  It is written almost wholly in Python, and
nicely extensible.  See the README in the mailman directory for more
details.

To use it, unpack the tar file and move the mailman directory to
within ~mailman on your system.  See the README in the mailman
directory for more details.

Mailman was devised by John Viega, who developed it pretty far but
lost later versions in a system crash.  The current version is based
on an early release (1.0b1) that john did which i (ken manheimer)
picked up and stashed away, until such time that i could pay attention
to it.  Turns out i had one of the most recent surviving copies, so it
became the basis of the current system.  However, that version was in
some ways still at the prototype stage, so i've spent some time
polishing it up, enough at first to get it running for use with the
maillists here at python.org, and perhaps a bit more to get it looking
better, easier to use, and a bit more solid.  I'd say this release, if
not ready for prime time, is ready for general use and development.

Below is a list of the major new features added since the original
release (1.0b1.1), bugs stomped and still pending, and thoughts about
further develpment necessary and/or desirable.  For anyone interested
in joining in on the development crowd, or just tracking the progress,
there's a maillist for you to join (we're considering using mailman to
run it - kidding!), mailman-developers@python.org.  Visit
http://www.python.org/mailman/listinfo/mailman-developers for more
info.

			     New Features

Web pages much more polished 
 - Better organized, text more finely crafted
 - Easier, more refined layout
 - List info and admin interface overviews, enumerate all public lists
   (via, e.g., http://www.python.org/mailman/listinfo - sans the
   specific list)
 - Admin interface broken into sections, with help elaboration for
   complicated configuration options

Maillist Archives
 - Integrated with a newer, *much* improved, external pipermail - to be
   found at http://starship.skyport.net/crew/amk/maintained/pipermail.html
 - Private archives protected with maillist members passwords, cookie-fied.

Spam prevention
 - New spam prevention measures catch most if not all spam without
   operator intervention or general constraints on who can post to
   list:
     require_explicit_destination option imposes hold of any postings
     that do not have the list name in any of the to or cc header
     destination addresses.  This catches the vast majority of random
     spam.
   Other options (forbidden_posters, bounce_matching_headers) provide
   for filtering of known transgressors.
 - Option obscure_addresses (default on) causes maillist subscriber
   lists on the web to be slightly mangled so they're not directly
   recognizable as email address by web spiders, which might be
   seeking targets for spammers.

Site configuration arrangement organized - in mailman/mailman/modules:
 - When installing, create a mailman/modules/mm_cfg.py (if there's not 
   one already there), using mm_cfg.py.dist as a template.
   mm_default.py contains the distributed defaults, including
   descriptions of the values.  mm_cfg.py does a 'from mm_defaults.py
   import *' to get the distributed defaults.  Include settings in
   mm_cfg.py for any values in mm_defaults.py that need to be
   customized for your site, after the 'from .. import *'.
 See mm_cfg.py.dist for more details.

Logging
 - Major operations (subscription, admin approval, bounce,
   digestification, cgi script failure tracebacks) logged in files
   using a reliable mechanism
 - Wrapper executables log authentication complaints via syslog

Wrappers
 - All cgi-script wrapper executables combined in a single source,
   easier to configure.  (Mail and aliases wrappers separate.)

List structure version migration
 - Provision for automatic update of list structures when moving to a
   new version of the system.  See modules/versions.py.

Code cleaning
 - Many more module docstrings, __version__ settings, more function
   docstrings.
 - Most unqualified exception catches have been replaced with more
   finely targeted catches, to avoid concealing bugs.
 - Lotsa long lines wrapped (pet peeve:).

			  Many Bugs Stomped
			  -----------------

Many bugs stomped.  Some memorable ones:

 - Subscriber addresses containing quotes (eg,
   Ken_O'Manheimer@python.org) were not properly escaped to survive
   transmission through shell to sendmail
 - Specifying a non-existent user for editing user options now
   complains immediately, rather than presenting a form.
 - Ad-hoc /tmp log files (which could bollix things if unwritable) gone
 - User changing from digest to non-digest and vice versa doesn't
   escape sort order.
 - Users are not sorted by address, internally - only on presentation
 - htmlformat.py no longer silently hides bugs in format operations.
   Instead, unexpected objects show as repr() would - easier to debug, 
   without making htmlformat fragile.
 - File creation bracketed by proper umask setting to ensure specified 
   file permissions.
 - Notices are sent for rejections of moderated postings (but not for
   discarded ones).
 - Web pages now all have titles.
 - Duplicate approvals of subscription requests do not lead to
   duplicate memberships.

			     Pending bugs
			     ------------

 - Mime digests a bit feeble - postings containing mime attachments
   not properly handled
 - Bounce processing
   - Turning off bounce processing doesn't?
   - Turning on non-member-post exclusion doesn't
   - More generally mm_utils.AddressesMatch() needs to be really
     thought out.  For instance, klm@python.org vs klm@acm.org should
     be seen as different, but klm@parrot.python.org and
     klm@larch.python.org and klm@python.org should probably be seen
     as the same.  There may be more criteria...
 - Bounces of the confirmation notices sent to people attempting
   web-based subscriptions are, necessarily, directed (via reply-to
   header) to maillist-request addr, and get mishandled there as
   unrecognized requests.  Not sure how to solve this one short of
   recognizing bad subscriber addrs before sending out the notice.

		   Pending Developments and Issues
		   -------------------------------

  Some modest Pending Refinements:

 - Use re module everywhere instead of regexp and regsub.
 - Implement a reasonable administrivia filter for lists (but less
   likely to misfire than majordomo's) (administrivia is, eg,
   unsubscribe request mistakenly sent to list posting addr)
 - Refine any unqualified exception guards to specify target exceptions.
 - Turn standard mailman exceptions into class exceptions.
 - Implement cookies (with moderate lifetimes) for user and admin passwords.
 - Unravel edit-options functionality from cgi/subscribe script - give 
   edit-options a script of its own.  (Not imperative, maybe when the
   bigger project of implementing member profiles is done.)
 - Relax constraint that everything be installed in /home/mailman.
   (I'll probably do this for the next release.  We can probably use
   relative paths for most things, particularly the routine sys.path
   setting in all the python code, and possibly for the wrappers.)
 - Foolproof list deletion (bin/rmlist) a bit by asking for password
   (and have list admin or site password work).
 - Should member deletion be confirmed with email?
 - Submission of edited user options should return to edit-options
   page, like admin options pages do.
 - People mass subscribed on admin form should not require approval,
   even for closed-subscribe lists.

  Some bigger Pending Projects

 - User and Admin Guides - HOWTOs
 - Developers guide - documentation
 - Implement email interface to admin commands.
 - Implement a 'which' mailcmd.  (Probably not hard!  May be good intro
   project for someone wading into mailman development.)
 - Put the admin interface and other things like private subscriber rosters
   *behind* passwords, like the private archives are.  Eg, instead of
   putting the password box on the listinfo form, have it always be just
   a regular link to the roster, but then the roster page would be a
   password page if the roster is private and there's no cookie for it.
 - Generalize and refine admindb so that pending bounces,
   subscriptions, etc are stored in the filesystem, nicely catalogued, 
   and the interfaces for handling them (eg, admindb) shows succinct
   summaries, with links to see the entire things.  (I may work on this.)
 - Develop subscription mechanisms so real confirmation happens, and
   use that as basis for option to change existing subscription
   delivery address.  (Scott cotton is working on this.)
 - Deliver by connect to SMTP 25 instead of sendmail.  (I'm checking
   in modules/smtplib.py module john's played with some.)
 - Robustify aliases handling to not just append entries if they
   already exist, and ideally, edit existing ones if they need to change.
 - Implement a real admin/membership-management section, that includes 
   includes hidden subscribers, provides easy perusal and changing of
   all members (including concealed members) status, including
   disabling or removal from list.
 - Implement some automated validation of installation - check for
   obvious problems with options settings, file access permissions,
   sendmail hookup, etc.
 ? Have an implicit list-managers list for every site, which includes
   all the list managers?
 - Make listinfo page static, regenerated every time admin options are 
   changed.  This will speed up and reduce load for listinfo page
   loading.

  More Fundamental, or just more bigger:), Pending Projects

 - Refine locking to only necessary sections, so there's less room for 
   unnecessary contention.
 - Implement the system as a persistent server.  (John's planning to
   do this.)
 - Implement member profiles, using a class.  For name, organization,
   description, etc, and defaults options settings.  The lists,
   themselves, could still have copies of the specific passwords and
   the compact bitmaps (for the sake of efficiency) for list-specific
   options like digest/nondigest, but the user could set their default
   choices - delivery address, password, delivery mode - in their
   profile.