Mailman 3 December 2017 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett Nov. 8, 2022

Nov. 8, 2022

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

3.1.x REST API enhancements around preferred email address
by Scott Koranda March 8, 2018

March 8, 2018

Hello, I am interested in contributing enhancements around preferred address to the REST API for version 3.1.x. To prove out my use case(s) and familiarize myself with the code I have made changes already. Here are some example curl command line invocations and output to illustrate: 1) Create an address for an existing user and mark that address as the user's preferred address: curl -u restadmin:restpass \ -X POST \ -d email=skoranda(a)example.nil \ -d preferred=1 \ http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8/addres… This returns a '201 Created'. 2) Return the preferred address, if set, for a user: curl -u restadmin:restpass \ -X GET \ http://mystestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8 | python -m json.tool { "created_on": "2017-11-11T22:20:45.950949", "display_name": "Scott Koranda", "http_etag": "\"26bc0d9f21eb145248884aedae4c1ffd00b608d2\"", "is_server_owner": false, "password": "$6$rounds=656000$gWGfwpajfZ7rVn5O$Sl559B2TgtpJWXA2i67G5ukjzkV6iTp4NgP.6FJpMFMUTDdDXULAmwdN8YW92w87EdctgqFqAUkUqS6.EOTCz/", "preferred_address": "skoranda(a)example.nil", "self_link": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8", "user_id": "e57ecfd0c0c74a319ac958b18882a6d8" } 3) Update/patch a user to set a preferred address: curl -u restadmin:restpass \ -X PATCH \ -d preferred_address=skoranda(a)example.nil \ http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8 This returns a '204 No Content'. 4) Get all addresses for a user with the preferred address (if set) having the attribute 'preferred' set to 'true': curl -u restadmin:restpass \ -X GET \ http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8/addres… | python -m json.tool { "entries": [ { "email": "skoranda(a)example.nil", "http_etag": "\"492824d150f9f11e32d530d3cad0f76422bddb48\"", "original_email": "skoranda01(a)example.nil", "preferred": true, "registered_on": "2017-11-13T22:35:17.232180", "self_link": "http://mytestbed.com:8001/3.1/addresses/skoranda@example.nil", "user": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8", "verified_on": "2017-11-20T16:48:51.738145" }, { "email": "skoranda01(a)example.nil", "http_etag": "\"f0f28c1c6a925a595269d3fe21ef7fc1515d7670\"", "original_email": "skoranda01(a)example.nil", "registered_on": "2017-11-20T16:46:20.941262", "self_link": "http://mytestbed.com:8001/3.1/addresses/skoranda01@example.nil", "user": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8", "verified_on": "2017-11-20T16:46:20.941595" }, ], "http_etag": "\"89c46bd312c6f1a344e6e6180aed6f11b62e6048\"", "start": 0, "total_size": 2 } Before I submit a pull request I would be grateful for any comments or feedback on the functionality as illustrated above, or any other comments or feedback. I will also begin the "copyright assignment" paperwork as explained at https://wiki.list.org/DEV/Home Thank you for your consideration. Sincerely, Scott Koranda

2 3

Handling additions to REST API in client side
by Abhilash Raj Dec. 29, 2017

Dec. 29, 2017

Hi All, Core's REST API is versioned and any change that break backwards-compatibility cause the version to bump so that clients can take care of that. However, one question that I have been thinking about recently is how to handle additions to REST API that don't necessarily break the backwards compatibility. For example, Core added `max_message_size` attribute to MailingList's REST endpoint, but it hasn't made into any released version yet. Also, Postorius added max_message_size in `Message Acceptance` settings. The problem here is that the entire PUT/PATCH request is going to fail if the currently running version of Core doesn't have `max_message_size` attribute exposed (Unknown Attribute Error). There is no easy way to check for whether the Core has this attribute as API is versioned at 3.1 for both cases. So, how do we actually handle this and maybe future cases like this? -- thanks, Abhilash Raj

5 10

Postorius 1.1.2 security release
by Abhilash Raj Dec. 28, 2017

Dec. 28, 2017

Hi Everyone, I am pleased to announce that Postorius 1.1.2 is released and is up on PyPI[1]. This release fixes a security bug that sets the password of a user in Core to their display name. It is recommended that you upgrade to this version. Postorius (Django) and Mailman Core both have different notion of "user" and "password". When a user account in created in Postorius, it creates a user in Core using the REST API. This bug, causes the password of user created in Core to be set to their display name instead. However, as of now, there are no use cases of the user password in Core and it is present only for historical reasons. So, while this bug is a serious one, it wouldn't result in any real-world exploit. Along with the bug-fix, this release includes a new command that resets *all* user passwords in Core to a random value. Again, there are no use cases of these passwords so resetting *all* of them isn't going to cause any inconvenience to users. This command should be run after the upgrade: $ cd mailman-suite/mailman-suite_project/ $ python manage.py reset_passwords Python 2.7 is the only supported Python version for this release. All versions of Django <=1.11 is supported. For more information about GNU Mailman and Postorius, please see our website: http://list.org The source code is available on Gitlab: https://gitlab.com/mailman/ [1]: https://pypi.org/project/postorius -- thanks, Abhilash Raj

1 0

Re: [Mailman-Developers] ARC PR for Mailman 3
by Peter Münster Dec. 20, 2017

Dec. 20, 2017

On Wed, Dec 13 2017, Gene Shuman wrote: >> foo("01234567890123456789" >> "01234567890123456789") > > Compensating for line lengths this way, just to satisfy a stylistic > requirement, I personally believe would reduce the quality of the > code. Hi, I personally believe that the limitation of the line length would *increase* the quality of the code. (Please, no lines with 500 characters in a text file.) Kind regards, -- Peter

2 1

Re: [Mailman-Developers] Date of Mailman 3 Suite going Python 3 only after Django dropped Python 2
by Simon Hanna Dec. 19, 2017

Dec. 19, 2017

On 12/15/2017 08:03 PM, Tobias Conradi wrote: > https://www.djangoproject.com/weblog/2017/dec/02/django-20-released/ > > 1) Is there a date for when Mailman Suite will switch to Python 3 only, > so user are not required to install two Python versions? AFAIK there are merge requests for all the parts, the only hold-up is testing to make sure they actualy work. You can search for "Python 3" in the gitlab issue trackers for more information. > 2) Is there a date for a switch to Django 2? Since Django 2 only supports Python3 support for it will most probably be added by merging the Python3 branches. I haven't gone over all the details, but I don't think there are any changes that still require porting...

2 1

Re: [Mailman-Developers] no module named django
by Simon Hanna Dec. 18, 2017

Dec. 18, 2017

Replying back to the list :-) > #virtualenv = /srv/django/mailman/env > since you managed to install postorius and get it running, you either > installed it systemwide which would work with it commented out, or you > actually created a virtualenv in which case you should know where the > location of that is. (virtualenv is preferred, you only use the system > packages, if they are installes using your package manager and the > application is too. Otherwise you will run into incompatibilties sooner > or later. > > totally do not know about virtualenv, just follow the guide to install > postorius. > when runserver, it can visit by 192.168.1.96, when "ctrl+c", > 192.168.1.96 not work. virtualenv is a python concept, you should be able to find information online about that. It's about how and where you install python dependencies. > > > #socket = /run/uwsgi/mailman.sock > This is the socket that is created for the webserver to connect to. I'm > actually not sure what happens if you don't include that... I doubt > uwsgi would actually start. > > installed uwsgi and it seems running, I run $ uwsgi, there is no error. > but can't find /run/uwsgi/ directory, usually mailman.sock can be at > which directory? > By convention sockets are placed in /run/ you can put them wherever you want. > I tried with docker, here is its guide with 6 steps: > --- > $ mkdir -p /opt/mailman/core > $ mkdir -p /opt/mailman/web > $ git clone https://github.com/maxking/docker-mailman > $ cd docker-mailman > # Change some configuration variables as mentioned above. > $ docker-compose up -d > ---- > But about configuration in step 5, I can't > find /opt/mailman/core/mailman-extra.cfg > and /opt/mailman/web/settings_local.py, and can't find it in any > directory from /home/lists/docker-mailman/. > so do not know what to with it. is it to create these two files? > Yes, you have to manually create them. The idea is that there are default settings and you would override them in additional files. The default settings are found here https://github.com/maxking/docker-mailman/blob/master/web/mailman-web/setti… https://github.com/maxking/docker-mailman/blob/master/core/assets/mailman.c… There are some settings that you have to overwrite, as mentioned in the readme of https://github.com/maxking/docker-mailman You do so, by specifing them in the respective files that you create in /opt/mailman/ on your host machine. You can search for the settings in the original files in the github repo and then use the same syntax in your new files. > OK, maybe it is realistic for me to wait for debian backport. That will definelty mean less work for you, and you will have it running directly on the machine managed by your package manager. This will probably be easier with upgrades as well.

1 0

no module named django
by 孙志勇 Dec. 17, 2017

Dec. 17, 2017

Hi, when I init uwsgi, I find a problem "importerror no module named django". I install postorius following installation in http://postorius.readthedocs.io/en/latest/setup.html, and visit http://192.168.1.96:8001 it workes well after run: $ sudo python manage.py runserver 0.0.0.0:8001. Do not know what is about development in the page: http://postorius.readthedocs.io/en/latest/development.html, so I pass it. Then go to deployment in http://postorius.readthedocs.io/en/latest/deployment.html. I creat file /home/lists/uwsgi.ini and put in it: --- [uwsgi] chdir = /home/lists/postorius/sample_project #virtualenv = /srv/django/mailman/env #socket = /run/uwsgi/mailman.sock wsgi-file = wsgi.py master = true process = 4 threads = 2 vacuum = true plugin = python2 uid = http gid = http --- then run: $ uwsgi --ini /home/lists/uwsgi.ini it gives the error "importerror, no module named django". I think it is the problem of the below 2 line: #virtualenv = /srv/django/mailman/env #socket = /run/uwsgi/mailman.sock but docs do not mention how to set about this 2 line, so I just comment out the 2 line. I also tried with mailman-suite, also have the same problem. can you help with me? will you think to write a installing instruction with command to follow for people totally don't know django in future? Thank you.

2 1

mailman var directory pip3
by 孙志勇 Dec. 16, 2017

Dec. 16, 2017

Hi, I installed mailman3.1.1 using pip3 on debian9.3. Where is the var directory of mailman?Thank you.

3 2

Date of Mailman 3 Suite going Python 3 only after Django dropped Python 2
by Tobias Conradi Dec. 16, 2017

Dec. 16, 2017

https://www.djangoproject.com/weblog/2017/dec/02/django-20-released/ 1) Is there a date for when Mailman Suite will switch to Python 3 only, so user are not required to install two Python versions? 2) Is there a date for a switch to Django 2? -- Tobias Conradi Rheinsberger Str. 18 10115 Berlin Germany https://tobiasconradi.com

1 0

Gitlab CI jobs stuck.
by Mark Sapiro Dec. 15, 2017

Dec. 15, 2017

CI on Gitlab seems to be stuck. Somehow "docker-runner: using maxking/mailman-ci-runner image." is not available. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 2

ARC PR for Mailman 3
by Stephen J. Turnbull Dec. 13, 2017

Dec. 13, 2017

Gene Shuman writes: > Over the past few months I've developed an patch for Mailman 3, > <https://gitlab.com/mailman/mailman/merge_requests/252> giving it ARC > <https://tools.ietf.org/html/draft-ietf-dmarc-arc-protocol-09> email > authentication capabilities. The work is (99%)completed, and it just needs > somebody to do a final review of the ARC functionality itself so I can > implement any requested changes. So at this point we have two complete ARC patches, yours and Aditya Divekar's from last year's GSoC which I mentored, and I'm pretty sure somebody else is working on it. I think somebody else mentioned a patch, too. > I had been working with Barry Warsaw, I wish he'd mentioned it.... > who had done a preliminary stylistic review, and subsequently, I've > eliminated all raised issues. The branch is rebased as of last > week, and passes all of qa, except one instance, which is something > I'd like to discuss with somebody. (It fails a style test for line > lengths in a test file which contains a bunch of cryptographic > hashes). I don't understand. Python offers several ways to deal with this, such as sha_values = ["0123456789012345678901234567890123456789"] foo(sha_value[0]) or if they're really long, adjacent strings will concatenate: foo("01234567890123456789" "01234567890123456789") I would think either would be fine, depending on your preferred style. Abhilash Raj (who just received the baton from Barry) would have the final say. I guess the main question I have at this point is whether you've participated in any of the interoperability testing that the ARC developers have done. I'll take a look at your MR and see if there are any features that differ across the implementations. Right now I'm buried until the 13th for sure, and more likely until Christmas. I definitely will be able to work on it during the holiday vacation, though. Steve -- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull(a)sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN

2 1

ARC
by Gene Shuman Dec. 1, 2017

Dec. 1, 2017

Hi all, I'm writing again about the ARC PR that I've been working on that I'd like to get reviewed & merged. I've been in touch with Abhilash Raj about this and he's graciously agreed to put some energy into it when he has time. But in general, I've put some thought into it, and have come up with a general plan to streamline this review, to make it as simple as possible for Abhilash or anybody else with the interest or bandwidth to help. First the are 4 major components of the review: - Barry Warsaw did a preliminary review which focused mostly on the code syntax & organization, and I've correspondently updated the code. So first point is verifying this, which should be easy. - Next is verifying that the way in which I've approached this problem is appropriately in line with how mailman is abstractly structured. Also easy. - Next again is verifyomg that the documentation is sufficient from a users(or developers) perspective. I've recently updated the documentation, and believe it to be sufficient. To test this, somebody with possibly marginal familiarity with ARC needs to just try to setup a Mailman server with ARC functionality. This is closely related to the next point. - Finally is verifying that the code behaves cryptographically correctly This last point is clearly what most imperatively needs to be right, and I'll devote a bit of this message towards it. Honestly, I don't think it should be that difficult for somebody to convince themselves Mailman is handling ARC correctly, even knowing little about it. Depending on the level of paranoia here(any of which are justifiable,) there are several routes to going about this. But first an overview of how this PR implements ARC. This may be a bit long winded, so hopefully bear with me. So this PR does ARC signing & validation via a library of mine, authheaders <https://pypi.python.org/pypi/authheaders/0.9.2>, which basically aggregates & passes along email authentication & signing functions to other libraries. It's quite simple & its easy to see that it's not modifying the results its getting from the other libraries. In particular, with respect to ARC, its using a fork of the library dkimpy <https://launchpad.net/dkimpy>. The fork has passed review, and afaik will be merged & released any day now. The fork brings the dkimpy library's ARC implementation(which I wrote) from 99% compliance with the ARC test suite <https://github.com/ValiMail/arc_test_suite>(which I also authored), to 100%. So it's absolutely demonstrably clear that the library that my ARC Mailman PR is using is cryptographically correct. The test suite itself has also been validated from several other directions, so at this point is authoritative. In particular, I've worked with the engineer at Google responsible for their ARC implementation, Brandon Long, on this, and I have also worked in depth with the major authors of the ARC rfc to clarify the protocol, illuminate edge cases, and bring the document to completion. But in particular, wrt Google, I've personally verified that their ARC implementation is 100% compliant with the spec & the test suite. And their ARC implementation is live and is validating & signing 100% of their email traffic atm. So here is my simple proposal for validating the cryptographic validity of my PR. Set up a Mailman server with ARC configured properly. Add some google email addresses to the outbound list, send a bunch of emails, and inspect the headers to see that ARC appropriately verifies. Then take the same Mailman server and turn of ARC and see that the messages fail. As a further step you can set up a nested sequence of Mailman servers, with some or all or none of them ARC enabled, and then check the results. I personally think it is within reason to trust Google as an authority here, as I've personally followed their development, and know that their implementation is 100% compliant. As an additional level of security here, we can combine this with direct validation fo dkimpy. I can directly show somebody how to test dkimpy against the ARC test suite themselves. I don't think this is necessary, as I've done it hundreds of times, but in 30m I can help somebody check this for themselves personally Another thing to note. There are tests added to the Mailman test suite that directly test the crypto. They aren't comprehensive, but they are there & they pass. So to be summarize, I'm perfectly confident that that the ARC implementation in Mailman is rock solid, as I've personally written the intermediary layer, the actual ARC implementation in dkimpy, the ARC test suite, and have been a non-trivial contributor to the specification itself. And with any of the above steps necessary for this validation, I'm available to help. What ever I can do to make this as painless as possible, I'm available for. Now to the next level of paranoia, which is the bottom level, and would prove beyond a shadow of a doubt that Mailman's ARC implementation is absolutely cryptographically compliant. First, a bit about the ARC test suite. It's a bunch of yaml files that specifies various messages, edge cases, etc, and what the output should be. It's designed abstractly so that you can test it against any system which is ARC capable, by just creating a system of translation layers between the test suite & the software being tested. It's been tested against dkimpy in such a manner. It's also been tested against googles ARC implementation, OpenARC, and others. It's quite general. It would be possible to directly run the test suite against Mailman. This would be a bit awkward, as its a mail server, as opposed to a library, but it could be done. And this literally would be bullet proof evidence that the system is behaving cryptographically correctly. However the translation layer may be a pain to write. But I haven't put too much thought into that. I personally think that the argument that the library, dkimpy, which the ARC PR is currently using passes test-suite validation is strong enough. But I'm an expert wrt ARC, and I understand that argument via claims of expertise aren't really all that valid. So the bottom line is this. Either I, or somebody else can wire up some ARC enabled Mailman servers, point them at Google, and check that things check out, using Google's ARC implementation as a reference. And possibly to demonstrate running the ARC test suite against dkimpy. The other option is making a harness to directly run the ARC test suite against mailman. This is doable, and would be a hassle, but is not out of the question. I could likely personally do this, and would happily do so. Thank you to anybody who has read this far. Please let me know your thoughts. And don't hesitate with any questions about ARC or DMARC, or other email authentication protocols. I'm a near encyclopedic resource about these things at this point, and am happy to share my knowledge. Looking forward to working with guys! Regards, =Gene

1 0