Mailman 3 March 2012 - Mailman-Developers

Re: [Mailman-Developers] any interest in a new built-in web-archive? (i.e. pipermail replacement)
by David Jeske 27 Mar '12

27 Mar '12

I brought up a quick CSLA injest of some mailman-dev posts to show off some basic features. I'm having a little trouble with the swish-e code, so the features that depend on text-indexing arn't working at the moment (search, author search). The current UI uses user-cookies to choose between a 2-pane frames mode and a noframes mode. I set the frames-mode as the default, which is inspired by the old google-groups usenet multi-message tree. Use "prefs" to check out the noframes mode, which is more like pipermail. Take a peek more for general capabilities than UI details (unless you see something you really like). I'm planning to change the UI to be more like the gmail-esq conversation style UIs I've been using since then... (where most/all of a thread's messages are rendered into a single page) I like the outgoing google-groups UI pretty well (not the new AJAXy one). 1) basic "home" for a list with bymonth and top-author metrics, RSS. Extended top-author page. http://dj1.willowmail.com/csla/Mailman-Developers http://dj1.willowmail.com/csla/Mailman-Developers/top_authors?date=2012-03 2) Prefs, including timezone, read-status key, browse-style.. (not sure if the read state is working on my install) http://dj1.willowmail.com/csla/Mailman-Developers/prefs 3) threadlist for the month (frames or noframes in prefs) http://dj1.willowmail.com/csla/Mailman-Developers/browse/month/2012-03/107 4) single-thread-list display (frames or noframes in prefs) http://dj1.willowmail.com/csla/Mailman-Developers/threads?start=1 5) here is a good example of a thread with redundant text eliding http://dj1.willowmail.com/csla/Mailman-Developers/browse_frm/thread/4/221

1 0

Re: [Mailman-Developers] Google Summer of Code: Integration of Search Code
by Toshio Kuratomi 27 Mar '12

27 Mar '12

On Mon, Mar 26, 2012 at 04:57:44PM +0530, Shayan Md wrote: > Hi, > > I am Shayan, I am doing my masters from IISc Bangalore. I want to take part > in GSoC from mailman organization. I have fairly good experience in python. > I worked on whoosh library for my own project. I have experience with > django also. > > I am planning to work on integrating systers search code into mailman. I > believe for whole summer this task alone is very small. I'll include couple > of more tasks in the proposal. > > I went through search code. Code looks pretty standard. As far I understood > > mailman.Searcher.IndexSchema has whoosh schema > mailman.Seracher.Indexer contains indexing functions which are later used > in genindex.py > mailman.Seracher.Mailocate has search function for indexed content > generated by genindex.py > > mailman.bin.genindex.main() is the main function which indexes the archives > > I am still trying to figure out how mailman client library interacts with > mailman server. I know how to use client library but I couldn't understand > internal working of mailman server. I have been reading the code but I > wasn't able understand much. It would be awesome if you can point me to any > documentation to understand this. Some api documentation or anything to > understand code. > Is this integration to be done with mailman2 or mailman3? In mailman3, the archivers are separated from the mailman core. -Toshio

2 1

Re: [Mailman-Developers] any interest in a new built-in web-archive? (i.e. pipermail replacement)
by David Jeske 27 Mar '12

27 Mar '12

> So if you have already developed some stuff, I > certainly would love to see you put it on the table > as a candidate for the Mailman 3 default > archive web UI. Yes. My CSLA code is on the table for MM2 or 3. It's python, and it's BSD. More than that, if you folks want it, I'm happy to retool/rewrite it as needed to make it suitable. (it has lots of great features, but i havn't touched it in a long time and it could be better) I'll put up a demo of the mailman list in CSLA tomorrow so folks can check it out. I also have "some experience" with this space, and by that I mean mountainous-experience. I coded/managed/architected several versions of eGroups/Yahoo Groups and Google Groups, including the listprocessor, schema design, and a few different python mail-to-html formatters. I also wrote my own personal gmail-like web-mailer in python, which someday I'll get around to open-sourcing. Can you share something about dependency philosophy (besides licensing) in Mailman? Currently CSLA uses Clearsilver templating (BSD c-module) and sqlite (public domain) storage.. both of which can be included directly as source to make installation easy, but they are both c-modules. I'm quite fond of the Clearsilver orm-to-templates toolchain, which is in use for lots of big sites. However, I also recognize it's not that popular in the python commmunity, mostly because we don't promote it, but also because the template processor c-module (for performance) instead of pure python. If it's necessary I could convert CSLA to use django templates, though they have a nasty design/performance problem with the orm that I think would prevent me from using it (unless I fixed it). I'm not a fan of javascript client-side rendering because of the generally poor performance, poor mobile compatibility, and lack of benefit for this kind of application. > Doing a *better* job of formatting HTML (than pipermail) is easy. > Doing a *good* job is going to be relatively hard, though. Isn't that the truth. Back at Egroups there was a time where it seems like we used to tweak the formatter almost daily. Deciding when preformatting is necessary vs harmful is a real pain. My CSLA python code has a fairly stable msg-to-html formatter that I've been happy with for a long time. It does very aggressive automatic repetitive text eliding for thread-display, so you can read many messages in a thread in a single page without massive quote pileup. View-source is provided for the unfortunate case where it does something bad. > AIUI, the current philosophy is that > > (1) the communication from Mailman to the archivers > will be via LMTP/SMTP, including a Mailman-specific > header to identify the message's permalink (currently > the SHA1 hash of the message ID in BASE32 > format, IIRC); and Using SMTP for an included archiver would require it be a long running server instead of merely a handler. Are we talking about third-party-site archivers here? > (2) the summary, search, and retrieval UI will be a > separate application from Mailman per se, which will > communicate with Mailman via the REST API for > authentication and user configuration purposes. That makes sense... CSLA doesn't currently have any concept server-auth. The only stateful features it has are view-preferences and read-state, neither of which are important enough to require a password. It uses a password-less system which uses cookies for prefs and a 'read state userid' which a user can manually set if they want. I like it, because it doesn't require login to get some basic browsing prefs and features. Hooking up an auth system would be necessary for some of the editing ideas in the document I read, or to allow online posting.

1 0

Re: [Mailman-Developers] Need guidance to start contributing to Mailman
by Stephen J. Turnbull 27 Mar '12

27 Mar '12

On Mon, Mar 26, 2012 at 6:45 AM, Sidharth Bansal <sidharthbansal.sb(a)gmail.com> wrote: > Hello Mailman Developers, > Sorry in advance if this is not the right place for my query. > I wanted to start contributing to Mailman-Development, and wanted a little > head start for that. I would love to get any sort of guidance in this > respect. I am a second year IT student with knowledge of c, c++, and trying > my hands on web development. Welcome! This is the right place to ask. You probably will not find C and C++ very helpful in learning web development ... especially not Python. You should start with the Python tutorial (my own preference for my students is the Python 3 tutorial at http://docs.python.org/py3k/tutorial/index.html but you might want to start with Python 2 at http://docs.python.org/tutorial/index.html for web development, since many libraries are only partly ported to Python 3 at this point). Then you should install the Bazaar VCS for your platform http://bazaar.canonical.com/ do "bzr help launchpad" to learn how to set up your ids for Launchpad, where Mailman's code repository is hosted, and get the most recent code for Mailman 3 from lp:mailman Then come back here, browse the mailman-developers archives, and ask questions! Steve

1 0

RELEASED: GNU Mailman 3.0 beta 1 and Postorius 1.0 alpha 1
by Barry Warsaw 27 Mar '12

27 Mar '12

Hello Mailman enthusiasts! Use the key, unlock the door See what your fate might have in store... Building on the excitement and amazing progress at our sprints at Pycon 2012, I am very happy to announce the availability of GNU Mailman 3.0 beta 1, code named "The Twilight Zone". After nearly four years of design, discussion, and development, we can now see a clear path to a final release. I thank everyone who has helped us get here, by participating on the mailman-developers mailing list, the bug tracker, in private conversations, and code contributions, both to Mailman itself and all the great projects it builds on. Special thanks go to our recent sprinters, Andrea Crotti, Florian Fuchs, Toshio Kuratomi, Daniel Mizyrycki, Terri Oda, Mark Sapiro, and Stephen Turnbull. While you do want to be careful using 3.0b1 in production, I hope that you will get a copy of the code and run it through its paces. Several people are known to be running real mailing lists using the code base. At this point, the feature set is frozen, as is the database schema. We'll use the schema migration machinery to do any schema changes from here to the final release. I'm also ecstatic to announce the first alpha release of Postorius, our new official name for the Django-based Mailman 3 web user interface. The name was suggested by core developer Florian Fuchs in honor of a bass hero of both of ours, Jaco Pastorius. Postorius 1.0 alpha 1 is code named "Space Farm". Postorius is in large part based on the great work of Anna Senarclens de Grancy and Benedict Stein who worked on a new Mailman web ui during their Google Summer of Code projects in 2010 and 2011. This alpha version connects to Mailman 3.0's REST API to add and edit lists and domains, as well as to moderate messages. It uses Django's auth app and Mozilla's BrowserID for authentication (a list of the current features is contained in the NEWS file of the package). Apart from the current state there are many more ideas left for the upcoming releases. There is a great team working on the web ui as well as on a new archiver, so stay tuned, and come join us! You can download GNU Mailman 3.0b1 from Launchpad or the Python Cheeseshop: https://launchpad.net/mailman http://pypi.python.org/pypi/mailman Postorius 1.0a1 is available from Launchpad and Cheeseshop as well: https://launchpad.net/postorius http://pypi.python.org/pypi/postorius The GNU Mailman documentation is available online at: http://packages.python.org/mailman/ You can submit bug reports to GNU Mailman and Postorius at: https://bugs.launchpad.net/mailman https://bugs.launchpad.net/postorius GNU Mailman and Postorius are released under the GNU General Public License version 3 or later. Enjoy! -Barry (On behalf of the entire GNU Mailman development team)

7 8

Re: [Mailman-Developers] any interest in a new built-in web-archive? (i.e. pipermail replacement)
by Stephen J. Turnbull 27 Mar '12

27 Mar '12

On Tue, Mar 27, 2012 at 7:20 AM, David Jeske <davidj(a)gmail.com> wrote: > I'm writing to find out the state of and philosophy surrounding pipermail > in mailman, There's been a lot of discussion of this over the years; you should spend some time in the mailman-developers and mailman-users archives looking for it, because neither the needs nor Barry's philosophy have changed much over that time period. It would be really nice if somebody would summarize it in Python-PEP fashion, but that's above and beyond the call of duty from your point of view. (Ie, it would be useful, but you may prefer coding etc, and that's fine too I believe, though I don't speak for Barry, although I occasionally get lucky channeling him ;-). > Unfortunately, pipermail doesn't do a good job formatting > messages for html.. (messages with no line-breaks are the most > annoying problem I regularly run into) Doing a *better* job of formatting HTML (than pipermail) is easy. Doing a *good* job is going to be relatively hard, though. > If there is some code (and time) I can contribute to mailman/pipermail, I'd > like to do so. A better pipermail probably would be a good thing, as transition to Mailman 3 is likely to take several years, and Mailman 2 is likely to continue to be used for several years. However, IMO you should focus on low-hanging fruit, as there are now much better alternatives to the technology used in pipermail, to the point where rewriting from scratch (or adapting a 3rd-party product) for Mailman 3 makes sense. > a) pipermail is fine... if you want to fix a bug or two submit a patch, but > we don't want to improve it Pipermail is fine for EOLing Mailman 2. Improving it would be good, but only if archive formats don't need upgrading and at minimum expense of effort, IMO. > b) we're ditching pipermail entirely... in the future sites will have to > choose an install an external archiver Pipermail is going the way of the dodo, yes, but there will be something bundled with Mailman, I'm pretty sure. > d) we'd love a dynamic-ui replacement for pipermail... as long as it uses > the same cgi/templating model as mailman ui A dynamic UI replacement seems to be a very good idea to me. Sites running mailman at all will be running a dynamic UI for the admin, so I see no good reason to stick to static for the archives. AIUI, the current philosophy is that (1) the communication from Mailman to the archivers will be via LMTP/SMTP, including a Mailman-specific header to identify the message's permalink (currently the SHA1 hash of the message ID in BASE32 format, IIRC); and (2) the summary, search, and retrieval UI will be a separate application from Mailman per se, which will communicate with Mailman via the REST API for authentication and user configuration purposes. Some ideas that have *not* been elaborated as "philosophy", but which I think are consistent with various desiderata and Barry's general approach and specific statements are (3) an archiver Handler for the pipeline will be provided, which will store posts in maildir format and do nothing else; and (4) a simple default summary/search/retrieval application will be bundled with (but have a separate development cycle from) Mailman. Even if that's not the current philosophy <wink/> that's the direction I would propose. So if you have already developed some stuff, I certainly would love to see you put it on the table as a candidate for the Mailman 3 default archive web UI.

1 0

any interest in a new built-in web-archive? (i.e. pipermail replacement)
by David Jeske 27 Mar '12

27 Mar '12

I'm writing to find out the state of and philosophy surrounding pipermail in mailman, to see if there is a productive way to provide some code/development-time to that part of mailman. I know there are several decent third-party archivers out there, but many of the mailing list archives I browse regularly are using pipermail because it's the mailman default.... one less thing to install, administer, and upgrade. Unfortunately, pipermail doesn't do a good job formatting messages for html.. (messages with no line-breaks are the most annoying problem I regularly run into) I've written code for this a number of times (eGroups, Yahoo Groups, Google Groups). I also released an open-source python/clearsilver/sqlite based archiver with redundant text-eliding, a few different thread views, and search... ( http://www.clearsilver.net/archive/ ) which is hardly used both because I don't try to popularize it, and because many sites just leave the default (pipermail). If there is some code (and time) I can contribute to mailman/pipermail, I'd like to do so. I'm writing this message to "take a temperature" and find out what, if any, contributions would be appreciated the mailman development team. I could imagine answers like: a) pipermail is fine... if you want to fix a bug or two submit a patch, but we don't want to improve it b) we're ditching pipermail entirely... in the future sites will have to choose an install an external archiver c) we'd love pipermail to be improved... but we still want it to be simple, static-html, and dependency free d) we'd love a dynamic-ui replacement for pipermail... as long as it uses the same cgi/templating model as mailman ui Thoughts? Is there any help I can offer up here?

1 0

[GSoC 2012] Candidate on 'Integration of (existing) search code into Mailman archives'
by George Chatzisofroniou 27 Mar '12

27 Mar '12

Hello Mailman Developers, My name is George Chatzisofroniou, i'm 20 years old and i'm an undergraduate student in the Department of Informatics at the University of Piraeus (Greece). Ι have really good previous experience with Mailman. This is because i use it for managing mailing lists for almost three years. I have also developed, with a friend of mine, MailmanStats [1], a Python software that outputs statistics for a mailing list based on Mailman. I think this implements the 'metric' idea in some way. I would like to know your opinion about MailmanStats. I'm sending this mail to inform you about my will to be part of Mailman Development team starting by Google Summer of Code 2012. The idea that excites me more is the 'Integration of (existing) search code into Mailman archives'. I think it is better to be developed on Mailman v3 rather than v2. I realize the significance of a feature like this. Many times before, i've got through the archives to search for a specific thread, so an addition like this would be great! As another student mentioned this idea is kinda small for the whole summer, so if there is time left i could integrate my MailmanStats [1] software into Mailman and/or build CSS styles for the web UI. Please tell me what you think. I'm also on IRC by the name sophron. Thanks, [1]: http://mailmanstats.latthi.com/ -- George Chatzisofroniou sophron.latthi.com

1 0

Re: [Mailman-Developers] RELEASED: GNU Mailman 3.0 beta 1 and Postorius 1.0 alpha 1
by Barry Warsaw 26 Mar '12

26 Mar '12

On Mar 25, 2012, at 02:34 PM, Jeff Breidenbach wrote: >Congratulations! I was able to get Postorius running by following the >five minute quick start guide. I didn't see archiving settings in the >user interface, how do I set that up? As Mark described, mm3 actually has a nicer architecture for archive integration. You can define multiple archivers system-wide, and there is an interface you can implement if you want to add a new one. Archivers are configured in the mailman.cfg file; i.e. they are system-wide. They cannot currently be configured per-list, although it might be interesting to someday support enabling or disabling them on a per-list basis. Maybe. ;) Note that beta2 will have a somewhat improved implementation here. Each archiver can have a "clobber policy" and "skew interval" which generalizes the old Pipermail Date: header clobbering rules, so that it can apply to any archiver. The Mail Archive is already supported in mm3, using the RFC 5064 standard and the X-Message-ID-Hash we discussed a while back. :) -Barry

1 0

GSoC
by Ana Cutillas 26 Mar '12

26 Mar '12

Hi, my name is Ana Cutillas and I am a senior Computer Science student from Spain. I am really interested in working on the Mailman project either with you directly or with Systers. I have been reading the list of ideas to implement and I am very interested in the #6 Creating user profiles ( http://blog.linuxgrrl.com/2012/03/13/mailman-brainstorm/). I have been wanting to work on a project that involved data mining for a while now and I think this could be a good opportunity. In general, I think profiles should have the really straightforward information: last time they started a conversation, last time they sent an email to the list, when did they sign up, what time of the day are they more active, etc. But it should be fairly easy to add cool stuff like, in case a list allows the use of more than one language, the language the user uses the most and maybe even percentages of usage, and with some information retrieval we could get keywords to know what they like to talk about the most. I like some of the other ideas too, so I can talk to you about them if you want to. Hoping to hear from you soon! - Ana

1 0