Mailman 3 September 2014 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett Nov. 7, 2022

Nov. 7, 2022

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

Adding CSS to mailman archives
by Andrew Daviel Feb. 13, 2015

Feb. 13, 2015

A little while ago (November 2012 probably) I changed a mailing list from Hypermail to Mailman and wanted to keep some of the look-and-feel of the archives, e.g. background colour and fonts. So I patched mailman 2.1.9 in CentOS 5 to include a CSS stylesheet link in each generated HTML page, replacing e.g. <BODY BGCOLOR="#ffffff"> with just <body> and adding e.g. body {background-color: #ffffff ; color: black; } in /pipermail/mailman.css, so that HTML style such as FONT and SIZE is removed from the Python code. (I only have one list on that server and didn't implement per-list style) I also changed a few things e.g. content-type to get zero warnings and errors from the W3C online validator. Now if I want to get DMARC functionality I'll have to jump past the RedHat/CentOS release and build 2.1.18, and re-patch for CSS. I was wondering if there was any interest in incorporating CSS into the official releases. -- Andrew Daviel, TRIUMF, Canada Tel. +1 (604) 222-7376 (Pacific Time)

7 17

Experimental SQLAlchemy port available
by Barry Warsaw Sept. 24, 2014

Sept. 24, 2014

Hi Mailman Hackers! I wanted to let you know about some very cool work that Abhilash Raj has been doing, and to request your help to review and test it. You might know that for a long time, Mailman 3's ORM (Object Relational Mapper) layer was provided by Storm[1]. There were a lot of things to like about Storm, especially because it was such a nice, thin layer over SQL so it was very transparent to use and easy to debug. It had some downsides though, and as the project lost momentum, those downsides became more apparent. These included lack of any Python 3 support, and bugs in the PostgreSQL layer. As I eventually want MM3 to be a Python 3 application, and it looked like Storm would never be ported, I have for a while desperately wanted to port MM3 to SQLAlchemy. At one point early in the MM3 fork, it *was* using SA, but I had some problems with that (details are lost, and unimportant) and switched to Storm. Now, I think it's clear that SA is the best ORM available for Python, and it is Python 3 compatible[2]. Abhilash has done a fantastic port of the MM3 trunk to SA. I have taken his branch, fixed a few test failures, and cleaned up a few things. But really, all the credit goes to Abhilash for his great work. I have not yet merged the SA branch into trunk, for a few reasons. Abhilash is investigating Alembic as the schema migration layer, and I have not had time to test the branch against PostgreSQL. I would also like to invite folks who have deployed MM3 to give the branch some testing, especially to make sure that the new SA-derived schema is compatible with the hand-written schema used by Storm. Adopting SA will mean a break in upgrade guarantees. We won't provide migrations from 3.0b4's schema to 3.0b5, which is why I want them to be effectively compatible. After 3.0b5 is released, we'll likely use Alembic to manage any future schema changes. While I'd rather not make this break, I'm convinced that the SA community can handle all this much better than our crufty pile of schema migration code. If you are able, please test this branch: $ bzr branch lp:~barry/mailman/abhilash That's essentially the 3.0 trunk with Abhilash's branch merged in, and my test fixes and clean ups on top of that. I'd like to merge this to trunk, definitely for 3.0b5, so let's say in the next week or so. Let us know how it goes! Cheers, -Barry [1] https://storm.canonical.com/ [2] We are very likely going to adopt Falcon as a replacement for restish. The latter suffers the same problems as Storm; it's not Python 3 compatible, and it is effectively unmaintained upstream. Falcon is really great, and with the following changes, it will easily support restish-style object-based traversal. Storm and restish are the last Python 2-only dependencies of MM3 core. https://github.com/racker/falcon/pull/307

2 1

Looking for Contribution
by Stephen J. Turnbull Sept. 17, 2014

Sept. 17, 2014

Saurabh Singhal writes: > It would be a pleasure to work with the mailman team and if > possible please let me know from where I should start. Start here: http://wiki.list.org/display/DEV/Home and here for recent lists of projects we considered doable in two months by a single, often new to Mailman, developer: http://wiki.list.org/display/DEV/Google+Summer+of+Code+2014 http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013 Before starting work on any of the GSoC projects, check if work has already been done on them. Ask here, or if you really prefer to do it yourself, you can browse the related branches at https://code.launchpad.net/mailman/ Note that the wiki is changing platforms in the *very* near future. There will be an announcement here, I'm sure, but due to this your wiki credentials may suddenly stop working. The problem is simply that it's not possible to move the password database; any profile you create on the existing wiki will be moved to the new one automatically. You just have to reregister a password. HTH Steve

1 0

Looking for Contribution
by Saurabh Singhal Sept. 16, 2014

Sept. 16, 2014

Hi Sir, For the past few days I've been looking mailman project and am very much willing to participate in the integration of mailman3 and hyperkitty and the development of both.I have an experience in SMTP servers also as trying to develop my own SMTP server using postfix and also a basic knowledge about email verification mechanisms.I have also worked on django before for few of my projects. It would be a pleasure to work with the mailman team and if possible please let me know from where I should start. Regards, Saurabh Singhal.

1 0

Re: [Mailman-Developers] Deb packages for Mailman 3 suite, beta 1 preview
by Barry Warsaw Sept. 16, 2014

Sept. 16, 2014

On Sep 16, 2014, at 03:20 PM, Paul Wise wrote: >From our discussions at DebConf14 it sounds like Barry Warsaw is >working on this but I don't think it will be ready in time. I also >think it is far too early to package mailman3 in a stable distro >release at this time. I wouldn't say "working on this" but maybe "want to work on this". ;) However I agree that it's too early to put MM3 in any stable distribution channel. I think the core would be relatively easy to package up, though I'm much less experienced in correctly packaging Django apps (Postorius and Hyperkitty). There was a web-app session at Debconf14 so I think it might even be better to wait and see what the outcome of that is. For Debian, it could go to experimental which would be a clear sign that it's available to play with, but not deemed stable. For Ubuntu, it would probably be best to stick it in a PPA. My RPM experience is way too rusty to admit having it. Cheers, -Barry

1 0

Deb packages for Mailman 3 suite, beta 1 preview
by Chuck Peters Sept. 16, 2014

Sept. 16, 2014

Has anyone been working on Debian/Ubuntu packages? Debian Jessie/testing will be frozen Nov 5th and the package should be in unstable for at least two weeks (Oct 24th). It's a bit late to make it into the upcoming Ubuntu release, but it would be nice have packages available in a Launchpad PPA (Personal Package Archive). 1. https://release.debian.org/jessie/freeze_policy.html 2. https://wiki.ubuntu.com/UtopicUnicorn/ReleaseSchedule Thanks, Chuck

3 2

PyCharm License for mailman team
by Nicolas Karageuzian Sept. 10, 2014

Sept. 10, 2014

Hi all, I've applied for the pyCharm opensource licence and recieved it with the pleasant surprise that it can be shared away with whole contributor's team. As Jetbrains kindly ask, this key should not be published over unsecure channels (public mailing list or forum), but I can send it on email request. Please feel free to ask, prefer using direct reply as i'll track the thread. Regards -- Nicolas Karageuzian

2 1

JAVA API to administer Mailman
by Alexander Hachmann Sept. 7, 2014

Sept. 7, 2014

Hello, I am happily using Mailman. So first of all a big thanks to all of you developers!! I am currently developing a community platform for a peergroup. There I would like to integrate mailman for list management. So my question is, whether there already is a liabrary written in Java that connects to Mailman? This way I would save development effort. The Version I am using is 2.1.4. Thanks and regards, Alex

3 2

Resetting the database for each test
by Barry Warsaw Sept. 7, 2014

Sept. 7, 2014

I hope Abhilash doesn't mind my following up to the mailing list. This is useful information that should be searchable in the archives. Abhilash is working on a port of the core from Storm to SQLAlchemy as the Python ORM layer. As an aside, this port plus the one I have pending for replacing restish with falcon, would allow us to start the port of Mailman 3 to Python 3. On Sep 07, 2014, at 06:15 AM, Abhilash Raj wrote: >I am facing a problem where I get this error: > > mailman.interfaces.domain.BadDomainSpecificationError: Duplicate email >host: example.com > >while running the tests, now I suspect this probably because the database is >not being reset in between tests. This is exactly right. Here's how the test suite ensures a clean system for each test. Note that this includes resetting the database, but also taking care of things like clearing out the queue directories and other persistent state. The first place to look is in the test layers. These are a Zope-ism adopted by nose2 (the test runner that MM3 uses) that provides for fixture-like resource management. https://nose2.readthedocs.org/en/latest/plugins/layers.html MM3's layers are defined in src/mailman/testing/layers.py. The most common layer is ConfigLayer which ensures that the configuration subsystem is initialized for the tests. ConfigLayer's testTearDown() is run at the end of every test that uses the ConfigLayer or any of its subclass layers. This method is pretty simple; it just calls reset_the_world() which is where all the good stuff happens. See src/mailman/testing/helpers.py. Most of that method should be pretty obvious; the call to config.db._reset() is where the database gets cleared out. _reset() is actually defined in src/mailman/database/factory.,py and placed onto the database object in DatabaseTestingFactory.create(). So how does _reset() work? First, it rolls back the database just to throw away any in progress transactions. It then calls back into the specific database implementations to give them a chance to do some pre-rest actions. It'll do something similar after the next step - it'll make a post-reset callback and then commit any outstanding transactions. The real heart of the method is in the call to ModelMeta._reset()[1]. ModelMeta is the metaclass for all ORM classes, via the Model base class used everywhere. The idea here is that each ORM class registers itself in ModelMeta._class_registry(), as long as the ORM class doesn't have a PRESERVE=False flag, indicating that particular table should *not* be reset after every test (currently only the Version table has this setting). Now you can see what ModelMeta._reset() does. It iterates through every registered ORM class, and removes every row in the associated database. I do it this way instead of dropping every table because it's actually more difficult to re-establish the schema than it is to remove the rows. So the trick for the SQLAlchemy port is to figure out both how to identify which tables to clear out, and how to drop all of those table's rows. Hope that makes sense! Cheers, -Barry [1] I just noticed that ModelMeta._reset() calls _pre_reset() again. That's probably a lurking bug that should be fixed.

1 0