Mailman 3 February 2018 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett 07 Nov '22

07 Nov '22

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

3.1.x REST API enhancements around preferred email address
by Scott Koranda 08 Mar '18

08 Mar '18

Hello, I am interested in contributing enhancements around preferred address to the REST API for version 3.1.x. To prove out my use case(s) and familiarize myself with the code I have made changes already. Here are some example curl command line invocations and output to illustrate: 1) Create an address for an existing user and mark that address as the user's preferred address: curl -u restadmin:restpass \ -X POST \ -d email=skoranda(a)example.nil \ -d preferred=1 \ http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8/addres… This returns a '201 Created'. 2) Return the preferred address, if set, for a user: curl -u restadmin:restpass \ -X GET \ http://mystestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8 | python -m json.tool { "created_on": "2017-11-11T22:20:45.950949", "display_name": "Scott Koranda", "http_etag": "\"26bc0d9f21eb145248884aedae4c1ffd00b608d2\"", "is_server_owner": false, "password": "$6$rounds=656000$gWGfwpajfZ7rVn5O$Sl559B2TgtpJWXA2i67G5ukjzkV6iTp4NgP.6FJpMFMUTDdDXULAmwdN8YW92w87EdctgqFqAUkUqS6.EOTCz/", "preferred_address": "skoranda(a)example.nil", "self_link": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8", "user_id": "e57ecfd0c0c74a319ac958b18882a6d8" } 3) Update/patch a user to set a preferred address: curl -u restadmin:restpass \ -X PATCH \ -d preferred_address=skoranda(a)example.nil \ http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8 This returns a '204 No Content'. 4) Get all addresses for a user with the preferred address (if set) having the attribute 'preferred' set to 'true': curl -u restadmin:restpass \ -X GET \ http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8/addres… | python -m json.tool { "entries": [ { "email": "skoranda(a)example.nil", "http_etag": "\"492824d150f9f11e32d530d3cad0f76422bddb48\"", "original_email": "skoranda01(a)example.nil", "preferred": true, "registered_on": "2017-11-13T22:35:17.232180", "self_link": "http://mytestbed.com:8001/3.1/addresses/skoranda@example.nil", "user": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8", "verified_on": "2017-11-20T16:48:51.738145" }, { "email": "skoranda01(a)example.nil", "http_etag": "\"f0f28c1c6a925a595269d3fe21ef7fc1515d7670\"", "original_email": "skoranda01(a)example.nil", "registered_on": "2017-11-20T16:46:20.941262", "self_link": "http://mytestbed.com:8001/3.1/addresses/skoranda01@example.nil", "user": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8", "verified_on": "2017-11-20T16:46:20.941595" }, ], "http_etag": "\"89c46bd312c6f1a344e6e6180aed6f11b62e6048\"", "start": 0, "total_size": 2 } Before I submit a pull request I would be grateful for any comments or feedback on the functionality as illustrated above, or any other comments or feedback. I will also begin the "copyright assignment" paperwork as explained at https://wiki.list.org/DEV/Home Thank you for your consideration. Sincerely, Scott Koranda

2 3

Subject munging with non-ascii
by Bob Puff 27 Feb '18

27 Feb '18

Greetings, I just got a report today from one of my Mailman 2 lists, where I have a subject prefix set (like [Listname] ). It seems that if the subject line has non-ascii characters in it, Mailman is not messing with it, and adding in the requested prefix. Could this be an easy fix? Bob

2 1

Re: [Mailman-Developers] Postorius, Hyperkitty and python support
by Mark Sapiro 26 Feb '18

26 Feb '18

On 02/25/2018 08:14 PM, Abhilash Raj wrote: > On Sun, Feb 25, 2018, at 6:03 PM, Mark Sapiro wrote: >> >> ERROR: test_middleware_request >> (postorius.tests.mailman_api_tests.test_middleware.TestMiddleware) >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site- >> packages/mock/mock.py", >> line 1305, in patched >> return func(*args, **keywargs) >> File >> "/var/MM/3/master/postorius/src/postorius/tests/mailman_api_tests/ >> test_middleware.py", >> line 47, in test_middleware_request >> response.context['error']) >> File >> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site- >> packages/django/test/utils.py", >> line 73, in __getitem__ >> raise KeyError(key) >> KeyError: 'error' >> >> >> (that from py35-djangolatest, but py35-django111 and py35-django20 have >> the same failure and there are no py27 tests defined. > > > Are you sure you are trying from the git-head of postorius? I'll do some more checking tomorrow. I seem to be having various anomalous issues such as python_dateutil-1.5 (not compatible with Python 3) getting installed in my Python 3 virtualenv. The issue I'm chasing now is editing domains doesn't work. I can add domains and delete domains, but if I try to edit I get the edit page which doesn't have a box for the mail_host which is the name of the domain I'm editing and it says "Please enter a domain name" which is the error for a missing (required) mail_host, and if I submit the form, all I get is Please check the errors below. What I can't figure out is why the mail_host box is not on the form. Anyway, I have to put this aside until tomorrow, but I'll check further then. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 2

Postorius, Hyperkitty and python support
by Mark Sapiro 25 Feb '18

25 Feb '18

What is the state of the current Postorius, Hyperkitty and django_mailman3 branches with respect to Python version support. I am trying to do some things with Postorius at the moment and with Python 2 I get import errors, specifically on from urllib.error import HTTPError which seems to work only in Python 3. Are these supposed to be bi-lingual or Python 3 only, or are they just unstable and not working at the moment? My immediate concern is to try to bring <https://gitlab.com/mailman/postorius/merge_requests/186> up to date. If I run tox on an unmodified gitlab head, It only runs py35 tests (it would run py36, but I don't have that installed) and they all fail like this: ERROR: test_middleware_request (postorius.tests.mailman_api_tests.test_middleware.TestMiddleware) ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-packages/mock/mock.py", line 1305, in patched return func(*args, **keywargs) File "/var/MM/3/master/postorius/src/postorius/tests/mailman_api_tests/test_middleware.py", line 47, in test_middleware_request response.context['error']) File "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-packages/django/test/utils.py", line 73, in __getitem__ raise KeyError(key) KeyError: 'error' (that from py35-djangolatest, but py35-django111 and py35-django20 have the same failure and there are no py27 tests defined. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 2

Re: [Mailman-Developers] Postorius, Hyperkitty and python support
by Barry Warsaw 25 Feb '18

25 Feb '18

On Feb 25, 2018, at 18:30, Abhilash Raj <maxking(a)asynchronous.in> wrote: >> Are these supposed to be bi-lingual or Python 3 only, or are they just >> unstable and not working at the moment? > > They are (or soon going to be) all Python 3 only. How awesome is that! Once this lands, we should definitely advertise this on the social medias. -Barry

1 0

Gitlab CI failure
by Mark Sapiro 23 Feb '18

23 Feb '18

I enabled shared runners and CI now runs, but I get a failure only on pgsql. This appears consistent, i.e. the same tests failed in both py35-nocov-pg and py36-nocov-pg on two separate runs, apparently because of messages like + /builds/msapiro/mailman/.tox/py36-nocov-pg/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>. + """) I don't see this failure when I run py35-nocov-pg locally even though both my local tox and CI install psycopg2==2.7.4 I am at a loss as to how to proceed. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 1

Re: [Mailman-Developers] Gitlab CI stuck.
by Mark Sapiro 23 Feb '18

23 Feb '18

On 02/23/2018 09:17 AM, Abhilash Raj wrote: > > Can you please Shared-Runners for your repo? > > The CI machine has been down for a few days and we don't know why. I (and Barry) have been using the Shared Runners. I've now enabled shared runners and CI is now running. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

Gitlab CI stuck.
by Mark Sapiro 23 Feb '18

23 Feb '18

I'm seeing This job is stuck, because you don't have any active runners that can run this job. Go to Runners page I go to the runners page and I see Runners activated for this project 678886ce #9640 docker-runner: using maxking/mailman-ci-runner image. Has Mysql and Postgresql images linked. The status circle left of 678886ce is grey and if I hover over it I see "Runner is offline, last contact was 10 days ago" -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 1

mailman 2.x user/group name
by Okan Demirmen 08 Feb '18

08 Feb '18

Hi all, I noticed a conversation on the recent thread releasing 2.1.26, but wanted to start a new thread that'll hopefully stay on-topic. Because the mailman configure requires the user and group to exist prior to running, as well as hard coding the actual values once run, it becomes very hard for porters/packagers when trying to support multiple mail servers and their respective user/group configurations (sendmail, postfix, exim, openstmpd, etc); and additionally the various web servers. Either we have to create a package for every combination or come up with another solution. What I did for the OpenBSD ports tree was to rewrite check_caller() in src/common.c (with src/common.h addition). It allows one to set the username, groupname, cgi-gid and mail-gid all via configure without these having to be setup beforehand. From the log message: - Rewrite src/common.c:check_caller() for the cgi/mail wrapper to now look at the defined group membership instead, _mailmanq (a new group). This allows the administrator to switch mail servers and web servers without requiring a FLAVOR for each combination; but rather, by simply adding the cgi/mail user to the _mailmanq group. This is a diversion from upstream, but will be proposed. At least sthen@ and dlg@ agree to go in this general direction. Here as well: https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/mail/mailman/patches/patch-… https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/mail/mailman/patches/patch-… In the OpenBSD ports tree, configure is now run with: --with-username=_mailman \ --with-groupname=_mailman \ --with-cgi-gid=_mailmanq \ --with-mail-gid=_mailmanq and the admin can change/migrate mail servers as well as web servers by just mucking with group memberships; no need to rebuild - allows mailman to be packaged basically. I realize now I never sent this upstream, so better late than never :) Inline patch to src/common.[ch] to follow, hopefully in the same style as the original. Considerations, thoughts? Thanks, Okan $OpenBSD: patch-src_common_h,v 1.1 2016/04/22 16:42:14 okan Exp $ --- src/common.h.orig Sun Mar 20 13:48:18 2016 +++ src/common.h Sun Mar 20 13:53:00 2016 @@ -27,6 +27,7 @@ #include <errno.h> #include <sys/types.h> #include <grp.h> +#include <pwd.h> #include <unistd.h> /* GETGROUPS_T gets set in the makefile by configure */ @@ -52,6 +53,7 @@ extern const char* logident; #define MAIL_ILLEGAL_COMMAND 6 #define ADDALIAS_USAGE_ERROR 7 #define GROUP_NAME_NOT_FOUND 8 +#define USER_NAME_NOT_FOUND 9 /* $OpenBSD: patch-src_common_c,v 1.1 2016/04/22 16:42:14 okan Exp $ --- src/common.c.orig Sun Feb 28 15:47:44 2016 +++ src/common.c Sun Mar 20 16:22:35 2016 @@ -119,45 +119,39 @@ fatal(const char* ident, int exitcode, char* format, . void check_caller(const char* ident, const char* parentgroup) { - GID_T mygid = getgid(); - struct group *mygroup = getgrgid(mygid); - char* option; - char* server; + struct passwd *pw; + struct group *gr; + char **g; + int ok = 0; char* wrapper; - if (running_as_cgi) { - option = "--with-cgi-gid"; - server = "web"; - wrapper = "CGI"; - } - else { - option = "--with-mail-gid"; - server = "mail"; - wrapper = "mail"; - } + pw = getpwuid(getuid()); + if (pw == NULL) + fatal(ident, USER_NAME_NOT_FOUND, + "Failure to find username"); - if (!mygroup) - fatal(ident, GROUP_NAME_NOT_FOUND, - "Failure to find group name for GID %d. Mailman\n" - "expected the %s wrapper to be executed as group\n" - "\"%s\", but the system's %s server executed the\n" - "wrapper as GID %d for which the name could not be\n" - "found. Try adding GID %d to your system as \"%s\",\n" - "or tweak your %s server to run the wrapper as group\n" - "\"%s\".", - mygid, wrapper, parentgroup, server, mygid, mygid, - parentgroup, server, parentgroup); + gr = getgrnam(parentgroup); + if (gr == NULL) + fatal(ident, GROUP_NAME_NOT_FOUND, + "Failure to find \"%s\" group", parentgroup); - if (strcmp(parentgroup, mygroup->gr_name)) - fatal(ident, GROUP_MISMATCH, - "Group mismatch error. Mailman expected the %s\n" - "wrapper script to be executed as group \"%s\", but\n" - "the system's %s server executed the %s script as\n" - "group \"%s\". Try tweaking the %s server to run the\n" - "script as group \"%s\", or re-run configure, \n" - "providing the command line option `%s=%s'.", - wrapper, parentgroup, server, wrapper, mygroup->gr_name, - server, parentgroup, option, mygroup->gr_name); + for (g = gr->gr_mem; *g; g++) { + if (strcmp(pw->pw_name, *g) == 0) { + ok = 1; + break; + } + } + + if (running_as_cgi) + wrapper = "CGI"; + else + wrapper = "mail"; + + if (ok == 0) + fatal(ident, GROUP_MISMATCH, + "Group mismatch error. Mailman expected the %s\n" + "wrapper script to be executed by a member of\n" + "\"%s\" group.", wrapper, parentgroup); }

2 2