On Mon, Mar 26, 2012 at 04:57:44PM +0530, Shayan Md wrote:
> I am Shayan, I am doing my masters from IISc Bangalore. I want to take part
> in GSoC from mailman organization. I have fairly good experience in python.
> I worked on whoosh library for my own project. I have experience with
> django also.
> I am planning to work on integrating systers search code into mailman. I
> believe for whole summer this task alone is very small. I'll include couple
> of more tasks in the proposal.
> I went through search code. Code looks pretty standard. As far I understood
> mailman.Searcher.IndexSchema has whoosh schema
> mailman.Seracher.Indexer contains indexing functions which are later used
> in genindex.py
> mailman.Seracher.Mailocate has search function for indexed content
> generated by genindex.py
> mailman.bin.genindex.main() is the main function which indexes the archives
> I am still trying to figure out how mailman client library interacts with
> mailman server. I know how to use client library but I couldn't understand
> internal working of mailman server. I have been reading the code but I
> wasn't able understand much. It would be awesome if you can point me to any
> documentation to understand this. Some api documentation or anything to
> understand code.
Is this integration to be done with mailman2 or mailman3?
In mailman3, the archivers are separated from the mailman core.
> So if you have already developed some stuff, I
> certainly would love to see you put it on the table
> as a candidate for the Mailman 3 default
> archive web UI.
Yes. My CSLA code is on the table for MM2 or 3. It's python, and it's BSD.
More than that, if you folks want it, I'm happy to retool/rewrite it as
needed to make it suitable. (it has lots of great features, but i havn't
touched it in a long time and it could be better) I'll put up a demo of the
mailman list in CSLA tomorrow so folks can check it out.
I also have "some experience" with this space, and by that I mean
mountainous-experience. I coded/managed/architected several versions of
eGroups/Yahoo Groups and Google Groups, including the listprocessor, schema
design, and a few different python mail-to-html formatters. I also wrote my
own personal gmail-like web-mailer in python, which someday I'll get around
Can you share something about dependency philosophy (besides licensing) in
Currently CSLA uses Clearsilver templating (BSD c-module) and sqlite
(public domain) storage.. both of which can be included directly as source
to make installation easy, but they are both c-modules.
I'm quite fond of the Clearsilver orm-to-templates toolchain, which is in
use for lots of big sites. However, I also recognize it's not that popular
in the python commmunity, mostly because we don't promote it, but also
because the template processor c-module (for performance) instead of pure
If it's necessary I could convert CSLA to use django templates, though they
have a nasty design/performance problem with the orm that I think would
prevent me from using it (unless I fixed it).
poor performance, poor mobile compatibility, and lack of benefit for this
kind of application.
> Doing a *better* job of formatting HTML (than pipermail) is easy.
> Doing a *good* job is going to be relatively hard, though.
Isn't that the truth.
Back at Egroups there was a time where it seems like we used to tweak the
formatter almost daily. Deciding when preformatting is necessary vs harmful
is a real pain.
My CSLA python code has a fairly stable msg-to-html formatter that I've
been happy with for a long time. It does very aggressive automatic
repetitive text eliding for thread-display, so you can read many messages
in a thread in a single page without massive quote pileup. View-source is
provided for the unfortunate case where it does something bad.
> AIUI, the current philosophy is that
> (1) the communication from Mailman to the archivers
> will be via LMTP/SMTP, including a Mailman-specific
> header to identify the message's permalink (currently
> the SHA1 hash of the message ID in BASE32
> format, IIRC); and
Using SMTP for an included archiver would require it be a long running
server instead of merely a handler.
Are we talking about third-party-site archivers here?
> (2) the summary, search, and retrieval UI will be a
> separate application from Mailman per se, which will
> communicate with Mailman via the REST API for
> authentication and user configuration purposes.
That makes sense...
CSLA doesn't currently have any concept server-auth. The only stateful
features it has are view-preferences and read-state, neither of which are
important enough to require a password. It uses a password-less system
manually set if they want. I like it, because it doesn't require login to
get some basic browsing prefs and features.
Hooking up an auth system would be necessary for some of the editing ideas
in the document I read, or to allow online posting.
On Mon, Mar 26, 2012 at 6:45 AM, Sidharth Bansal
> Hello Mailman Developers,
> Sorry in advance if this is not the right place for my query.
> I wanted to start contributing to Mailman-Development, and wanted a little
> head start for that. I would love to get any sort of guidance in this
> respect. I am a second year IT student with knowledge of c, c++, and trying
> my hands on web development.
Welcome! This is the right place to ask.
You probably will not find C and C++ very helpful in learning web
development ... especially not Python. You should start with the
Python tutorial (my own preference for my students is the Python 3
but you might want to start with Python 2 at
for web development, since many libraries are only partly ported to
Python 3 at this point).
Then you should install the Bazaar VCS for your platform
do "bzr help launchpad" to learn how to set up your ids for Launchpad,
where Mailman's code repository is hosted, and get the most recent
code for Mailman 3 from
Then come back here, browse the mailman-developers archives, and ask questions!
Hello Mailman enthusiasts!
Use the key, unlock the door
See what your fate might have in store...
Building on the excitement and amazing progress at our sprints at Pycon 2012,
I am very happy to announce the availability of GNU Mailman 3.0 beta 1, code
named "The Twilight Zone".
After nearly four years of design, discussion, and development, we can now see
a clear path to a final release. I thank everyone who has helped us get here,
by participating on the mailman-developers mailing list, the bug tracker, in
private conversations, and code contributions, both to Mailman itself and all
the great projects it builds on. Special thanks go to our recent sprinters,
Andrea Crotti, Florian Fuchs, Toshio Kuratomi, Daniel Mizyrycki, Terri Oda,
Mark Sapiro, and Stephen Turnbull.
While you do want to be careful using 3.0b1 in production, I hope that you
will get a copy of the code and run it through its paces. Several people are
known to be running real mailing lists using the code base. At this point,
the feature set is frozen, as is the database schema. We'll use the schema
migration machinery to do any schema changes from here to the final release.
I'm also ecstatic to announce the first alpha release of Postorius, our new
official name for the Django-based Mailman 3 web user interface. The name was
suggested by core developer Florian Fuchs in honor of a bass hero of both of
ours, Jaco Pastorius. Postorius 1.0 alpha 1 is code named "Space Farm".
Postorius is in large part based on the great work of Anna Senarclens de
Grancy and Benedict Stein who worked on a new Mailman web ui during their
Google Summer of Code projects in 2010 and 2011. This alpha version connects
to Mailman 3.0's REST API to add and edit lists and domains, as well as to
moderate messages. It uses Django's auth app and Mozilla's BrowserID for
authentication (a list of the current features is contained in the NEWS file
of the package). Apart from the current state there are many more ideas left
for the upcoming releases. There is a great team working on the web ui as
well as on a new archiver, so stay tuned, and come join us!
You can download GNU Mailman 3.0b1 from Launchpad or the Python Cheeseshop:
Postorius 1.0a1 is available from Launchpad and Cheeseshop as well:
The GNU Mailman documentation is available online at:
You can submit bug reports to GNU Mailman and Postorius at:
GNU Mailman and Postorius are released under the GNU General Public License
version 3 or later.
(On behalf of the entire GNU Mailman development team)
On Tue, Mar 27, 2012 at 7:20 AM, David Jeske <davidj(a)gmail.com> wrote:
> I'm writing to find out the state of and philosophy surrounding pipermail
> in mailman,
There's been a lot of discussion of this over the years; you should spend
some time in the mailman-developers and mailman-users archives
looking for it, because neither the needs nor Barry's philosophy have
changed much over that time period. It would be really nice if somebody
would summarize it in Python-PEP fashion, but that's above and beyond
the call of duty from your point of view. (Ie, it would be useful, but you
may prefer coding etc, and that's fine too I believe, though I don't speak
for Barry, although I occasionally get lucky channeling him ;-).
> Unfortunately, pipermail doesn't do a good job formatting
> messages for html.. (messages with no line-breaks are the most
> annoying problem I regularly run into)
Doing a *better* job of formatting HTML (than pipermail) is easy.
Doing a *good* job is going to be relatively hard, though.
> If there is some code (and time) I can contribute to mailman/pipermail, I'd
> like to do so.
A better pipermail probably would be a good thing, as transition to
Mailman 3 is likely to take several years, and Mailman 2 is likely to
continue to be used for several years. However, IMO you should
focus on low-hanging fruit, as there are now much better alternatives
to the technology used in pipermail, to the point where rewriting from
scratch (or adapting a 3rd-party product) for Mailman 3 makes sense.
> a) pipermail is fine... if you want to fix a bug or two submit a patch, but
> we don't want to improve it
Pipermail is fine for EOLing Mailman 2. Improving it would be good, but
only if archive formats don't need upgrading and at minimum expense
of effort, IMO.
> b) we're ditching pipermail entirely... in the future sites will have to
> choose an install an external archiver
Pipermail is going the way of the dodo, yes, but there will be something
bundled with Mailman, I'm pretty sure.
> d) we'd love a dynamic-ui replacement for pipermail... as long as it uses
> the same cgi/templating model as mailman ui
A dynamic UI replacement seems to be a very good idea to me.
Sites running mailman at all will be running a dynamic UI for the
admin, so I see no good reason to stick to static for the archives.
AIUI, the current philosophy is that
(1) the communication from Mailman to the archivers will be via
LMTP/SMTP, including a Mailman-specific header to identify the
message's permalink (currently the SHA1 hash of the message ID
in BASE32 format, IIRC); and
(2) the summary, search, and retrieval UI will be a separate
application from Mailman per se, which will communicate
with Mailman via the REST API for authentication and user
Some ideas that have *not* been elaborated as "philosophy", but
which I think are consistent with various desiderata and Barry's
general approach and specific statements are
(3) an archiver Handler for the pipeline will be provided, which
will store posts in maildir format and do nothing else; and
(4) a simple default summary/search/retrieval application will
be bundled with (but have a separate development cycle from)
Even if that's not the current philosophy <wink/> that's the
direction I would propose.
So if you have already developed some stuff, I certainly would
love to see you put it on the table as a candidate for the
Mailman 3 default archive web UI.
I'm writing to find out the state of and philosophy surrounding pipermail
in mailman, to see if there is a productive way to provide some
code/development-time to that part of mailman.
I know there are several decent third-party archivers out there, but many
of the mailing list archives I browse regularly are using pipermail because
it's the mailman default.... one less thing to install, administer, and
upgrade. Unfortunately, pipermail doesn't do a good job formatting
messages for html.. (messages with no line-breaks are the most annoying
problem I regularly run into)
I've written code for this a number of times (eGroups, Yahoo Groups, Google
Groups). I also released an open-source python/clearsilver/sqlite based
archiver with redundant text-eliding, a few different thread views, and
search... ( http://www.clearsilver.net/archive/ ) which is hardly used
both because I don't try to popularize it, and because many sites just
leave the default (pipermail).
If there is some code (and time) I can contribute to mailman/pipermail, I'd
like to do so. I'm writing this message to "take a temperature" and find
out what, if any, contributions would be appreciated the mailman
development team. I could imagine answers like:
a) pipermail is fine... if you want to fix a bug or two submit a patch, but
we don't want to improve it
b) we're ditching pipermail entirely... in the future sites will have to
choose an install an external archiver
c) we'd love pipermail to be improved... but we still want it to be simple,
static-html, and dependency free
d) we'd love a dynamic-ui replacement for pipermail... as long as it uses
the same cgi/templating model as mailman ui
Thoughts? Is there any help I can offer up here?
Hello Mailman Developers,
My name is George Chatzisofroniou, i'm 20 years old and i'm an
undergraduate student in the Department of Informatics at the
University of Piraeus (Greece).
Ι have really good previous experience with Mailman. This is because i
use it for managing mailing lists for almost three years.
I have also developed, with a friend of mine, MailmanStats , a
Python software that outputs statistics for a mailing list based on
Mailman. I think this implements the 'metric' idea in some way. I
would like to know your opinion about MailmanStats.
I'm sending this mail to inform you about my will to be part of
Mailman Development team starting by Google Summer of Code 2012. The
idea that excites me more is the 'Integration of (existing) search
code into Mailman archives'. I think it is better to be developed on
Mailman v3 rather than v2. I realize the significance of a feature
like this. Many times before, i've got through the archives to search
for a specific thread, so an addition like this would be great!
As another student mentioned this idea is kinda small for the whole
summer, so if there is time left i could integrate my MailmanStats 
software into Mailman and/or build CSS styles for the web UI.
Please tell me what you think. I'm also on IRC by the name sophron.
On Mar 25, 2012, at 02:34 PM, Jeff Breidenbach wrote:
>Congratulations! I was able to get Postorius running by following the
>five minute quick start guide. I didn't see archiving settings in the
>user interface, how do I set that up?
As Mark described, mm3 actually has a nicer architecture for archive
integration. You can define multiple archivers system-wide, and there is an
interface you can implement if you want to add a new one.
Archivers are configured in the mailman.cfg file; i.e. they are system-wide.
They cannot currently be configured per-list, although it might be interesting
to someday support enabling or disabling them on a per-list basis. Maybe. ;)
Note that beta2 will have a somewhat improved implementation here. Each
archiver can have a "clobber policy" and "skew interval" which generalizes the
old Pipermail Date: header clobbering rules, so that it can apply to any
The Mail Archive is already supported in mm3, using the RFC 5064 standard and
the X-Message-ID-Hash we discussed a while back. :)
my name is Ana Cutillas and I am a senior Computer Science student from
Spain. I am really interested in working on the Mailman project either with
you directly or with Systers.
I have been reading the list of ideas to implement and I am very interested
in the #6 Creating user profiles (
http://blog.linuxgrrl.com/2012/03/13/mailman-brainstorm/). I have been
wanting to work on a project that involved data mining for a while now and
I think this could be a good opportunity.
In general, I think profiles should have the really straightforward
information: last time they started a conversation, last time they sent an
email to the list, when did they sign up, what time of the day are they
more active, etc. But it should be fairly easy to add cool stuff like, in
case a list allows the use of more than one language, the language the user
uses the most and maybe even percentages of usage, and with some
information retrieval we could get keywords to know what they like to talk
about the most.
I like some of the other ideas too, so I can talk to you about them if you
Hoping to hear from you soon!
I am Shayan, I am doing my masters from IISc Bangalore. I want to take part
in GSoC from mailman organization. I have fairly good experience in python.
I worked on whoosh library for my own project. I have experience with
I am planning to work on integrating systers search code into mailman. I
believe for whole summer this task alone is very small. I'll include couple
of more tasks in the proposal.
I went through search code. Code looks pretty standard. As far I understood
mailman.Searcher.IndexSchema has whoosh schema
mailman.Seracher.Indexer contains indexing functions which are later used
mailman.Seracher.Mailocate has search function for indexed content
generated by genindex.py
mailman.bin.genindex.main() is the main function which indexes the archives
I am still trying to figure out how mailman client library interacts with
mailman server. I know how to use client library but I couldn't understand
internal working of mailman server. I have been reading the code but I
wasn't able understand much. It would be awesome if you can point me to any
documentation to understand this. Some api documentation or anything to