Mailman-Developers February 2018

mailman-developers@python.org

7 participants
10 discussions

3.1.x REST API enhancements around preferred email address
by Scott Koranda 08 Mar '18

08 Mar '18

Hello, I am interested in contributing enhancements around preferred address to the REST API for version 3.1.x. To prove out my use case(s) and familiarize myself with the code I have made changes already. Here are some example curl command line invocations and output to illustrate: 1) Create an address for an existing user and mark that address as the user's preferred address: curl -u restadmin:restpass \ -X POST \ -d email=skoranda(a)example.nil \ -d preferred=1 \ http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8/addres… This returns a '201 Created'. 2) Return the preferred address, if set, for a user: curl -u restadmin:restpass \ -X GET \ http://mystestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8 | python -m json.tool { "created_on": "2017-11-11T22:20:45.950949", "display_name": "Scott Koranda", "http_etag": "\"26bc0d9f21eb145248884aedae4c1ffd00b608d2\"", "is_server_owner": false, "password": "$6$rounds=656000$gWGfwpajfZ7rVn5O$Sl559B2TgtpJWXA2i67G5ukjzkV6iTp4NgP.6FJpMFMUTDdDXULAmwdN8YW92w87EdctgqFqAUkUqS6.EOTCz/", "preferred_address": "skoranda(a)example.nil", "self_link": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8", "user_id": "e57ecfd0c0c74a319ac958b18882a6d8" } 3) Update/patch a user to set a preferred address: curl -u restadmin:restpass \ -X PATCH \ -d preferred_address=skoranda(a)example.nil \ http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8 This returns a '204 No Content'. 4) Get all addresses for a user with the preferred address (if set) having the attribute 'preferred' set to 'true': curl -u restadmin:restpass \ -X GET \ http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8/addres… | python -m json.tool { "entries": [ { "email": "skoranda(a)example.nil", "http_etag": "\"492824d150f9f11e32d530d3cad0f76422bddb48\"", "original_email": "skoranda01(a)example.nil", "preferred": true, "registered_on": "2017-11-13T22:35:17.232180", "self_link": "http://mytestbed.com:8001/3.1/addresses/skoranda@example.nil", "user": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8", "verified_on": "2017-11-20T16:48:51.738145" }, { "email": "skoranda01(a)example.nil", "http_etag": "\"f0f28c1c6a925a595269d3fe21ef7fc1515d7670\"", "original_email": "skoranda01(a)example.nil", "registered_on": "2017-11-20T16:46:20.941262", "self_link": "http://mytestbed.com:8001/3.1/addresses/skoranda01@example.nil", "user": "http://mytestbed.com:8001/3.1/users/e57ecfd0c0c74a319ac958b18882a6d8", "verified_on": "2017-11-20T16:46:20.941595" }, ], "http_etag": "\"89c46bd312c6f1a344e6e6180aed6f11b62e6048\"", "start": 0, "total_size": 2 } Before I submit a pull request I would be grateful for any comments or feedback on the functionality as illustrated above, or any other comments or feedback. I will also begin the "copyright assignment" paperwork as explained at https://wiki.list.org/DEV/Home Thank you for your consideration. Sincerely, Scott Koranda

2 3

Subject munging with non-ascii
by Bob Puff 28 Feb '18

28 Feb '18

Greetings, I just got a report today from one of my Mailman 2 lists, where I have a subject prefix set (like [Listname] ). It seems that if the subject line has non-ascii characters in it, Mailman is not messing with it, and adding in the requested prefix. Could this be an easy fix? Bob

2 1

Re: [Mailman-Developers] Postorius, Hyperkitty and python support
by Mark Sapiro 27 Feb '18

27 Feb '18

On 02/25/2018 08:14 PM, Abhilash Raj wrote: > On Sun, Feb 25, 2018, at 6:03 PM, Mark Sapiro wrote: >> >> ERROR: test_middleware_request >> (postorius.tests.mailman_api_tests.test_middleware.TestMiddleware) >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site- >> packages/mock/mock.py", >> line 1305, in patched >> return func(*args, **keywargs) >> File >> "/var/MM/3/master/postorius/src/postorius/tests/mailman_api_tests/ >> test_middleware.py", >> line 47, in test_middleware_request >> response.context['error']) >> File >> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site- >> packages/django/test/utils.py", >> line 73, in __getitem__ >> raise KeyError(key) >> KeyError: 'error' >> >> >> (that from py35-djangolatest, but py35-django111 and py35-django20 have >> the same failure and there are no py27 tests defined. > > > Are you sure you are trying from the git-head of postorius? I'll do some more checking tomorrow. I seem to be having various anomalous issues such as python_dateutil-1.5 (not compatible with Python 3) getting installed in my Python 3 virtualenv. The issue I'm chasing now is editing domains doesn't work. I can add domains and delete domains, but if I try to edit I get the edit page which doesn't have a box for the mail_host which is the name of the domain I'm editing and it says "Please enter a domain name" which is the error for a missing (required) mail_host, and if I submit the form, all I get is Please check the errors below. What I can't figure out is why the mail_host box is not on the form. Anyway, I have to put this aside until tomorrow, but I'll check further then. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 2

Postorius, Hyperkitty and python support
by Mark Sapiro 26 Feb '18

26 Feb '18

What is the state of the current Postorius, Hyperkitty and django_mailman3 branches with respect to Python version support. I am trying to do some things with Postorius at the moment and with Python 2 I get import errors, specifically on from urllib.error import HTTPError which seems to work only in Python 3. Are these supposed to be bi-lingual or Python 3 only, or are they just unstable and not working at the moment? My immediate concern is to try to bring <https://gitlab.com/mailman/postorius/merge_requests/186> up to date. If I run tox on an unmodified gitlab head, It only runs py35 tests (it would run py36, but I don't have that installed) and they all fail like this: ERROR: test_middleware_request (postorius.tests.mailman_api_tests.test_middleware.TestMiddleware) ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-packages/mock/mock.py", line 1305, in patched return func(*args, **keywargs) File "/var/MM/3/master/postorius/src/postorius/tests/mailman_api_tests/test_middleware.py", line 47, in test_middleware_request response.context['error']) File "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-packages/django/test/utils.py", line 73, in __getitem__ raise KeyError(key) KeyError: 'error' (that from py35-djangolatest, but py35-django111 and py35-django20 have the same failure and there are no py27 tests defined. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 2

Re: [Mailman-Developers] Postorius, Hyperkitty and python support
by Barry Warsaw 26 Feb '18

26 Feb '18

On Feb 25, 2018, at 18:30, Abhilash Raj <maxking(a)asynchronous.in> wrote: >> Are these supposed to be bi-lingual or Python 3 only, or are they just >> unstable and not working at the moment? > > They are (or soon going to be) all Python 3 only. How awesome is that! Once this lands, we should definitely advertise this on the social medias. -Barry

1 0

Gitlab CI failure
by Mark Sapiro 23 Feb '18

23 Feb '18

I enabled shared runners and CI now runs, but I get a failure only on pgsql. This appears consistent, i.e. the same tests failed in both py35-nocov-pg and py36-nocov-pg on two separate runs, apparently because of messages like + /builds/msapiro/mailman/.tox/py36-nocov-pg/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>. + """) I don't see this failure when I run py35-nocov-pg locally even though both my local tox and CI install psycopg2==2.7.4 I am at a loss as to how to proceed. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 1

Re: [Mailman-Developers] Gitlab CI stuck.
by Mark Sapiro 23 Feb '18

23 Feb '18

On 02/23/2018 09:17 AM, Abhilash Raj wrote: > > Can you please Shared-Runners for your repo? > > The CI machine has been down for a few days and we don't know why. I (and Barry) have been using the Shared Runners. I've now enabled shared runners and CI is now running. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

Gitlab CI stuck.
by Mark Sapiro 23 Feb '18

23 Feb '18

I'm seeing This job is stuck, because you don't have any active runners that can run this job. Go to Runners page I go to the runners page and I see Runners activated for this project 678886ce #9640 docker-runner: using maxking/mailman-ci-runner image. Has Mysql and Postgresql images linked. The status circle left of 678886ce is grey and if I hover over it I see "Runner is offline, last contact was 10 days ago" -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 1

mailman 2.x user/group name
by Okan Demirmen 08 Feb '18

08 Feb '18

Hi all, I noticed a conversation on the recent thread releasing 2.1.26, but wanted to start a new thread that'll hopefully stay on-topic. Because the mailman configure requires the user and group to exist prior to running, as well as hard coding the actual values once run, it becomes very hard for porters/packagers when trying to support multiple mail servers and their respective user/group configurations (sendmail, postfix, exim, openstmpd, etc); and additionally the various web servers. Either we have to create a package for every combination or come up with another solution. What I did for the OpenBSD ports tree was to rewrite check_caller() in src/common.c (with src/common.h addition). It allows one to set the username, groupname, cgi-gid and mail-gid all via configure without these having to be setup beforehand. From the log message: - Rewrite src/common.c:check_caller() for the cgi/mail wrapper to now look at the defined group membership instead, _mailmanq (a new group). This allows the administrator to switch mail servers and web servers without requiring a FLAVOR for each combination; but rather, by simply adding the cgi/mail user to the _mailmanq group. This is a diversion from upstream, but will be proposed. At least sthen@ and dlg@ agree to go in this general direction. Here as well: https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/mail/mailman/patches/patch-… https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/mail/mailman/patches/patch-… In the OpenBSD ports tree, configure is now run with: --with-username=_mailman \ --with-groupname=_mailman \ --with-cgi-gid=_mailmanq \ --with-mail-gid=_mailmanq and the admin can change/migrate mail servers as well as web servers by just mucking with group memberships; no need to rebuild - allows mailman to be packaged basically. I realize now I never sent this upstream, so better late than never :) Inline patch to src/common.[ch] to follow, hopefully in the same style as the original. Considerations, thoughts? Thanks, Okan $OpenBSD: patch-src_common_h,v 1.1 2016/04/22 16:42:14 okan Exp $ --- src/common.h.orig Sun Mar 20 13:48:18 2016 +++ src/common.h Sun Mar 20 13:53:00 2016 @@ -27,6 +27,7 @@ #include <errno.h> #include <sys/types.h> #include <grp.h> +#include <pwd.h> #include <unistd.h> /* GETGROUPS_T gets set in the makefile by configure */ @@ -52,6 +53,7 @@ extern const char* logident; #define MAIL_ILLEGAL_COMMAND 6 #define ADDALIAS_USAGE_ERROR 7 #define GROUP_NAME_NOT_FOUND 8 +#define USER_NAME_NOT_FOUND 9 /* $OpenBSD: patch-src_common_c,v 1.1 2016/04/22 16:42:14 okan Exp $ --- src/common.c.orig Sun Feb 28 15:47:44 2016 +++ src/common.c Sun Mar 20 16:22:35 2016 @@ -119,45 +119,39 @@ fatal(const char* ident, int exitcode, char* format, . void check_caller(const char* ident, const char* parentgroup) { - GID_T mygid = getgid(); - struct group *mygroup = getgrgid(mygid); - char* option; - char* server; + struct passwd *pw; + struct group *gr; + char **g; + int ok = 0; char* wrapper; - if (running_as_cgi) { - option = "--with-cgi-gid"; - server = "web"; - wrapper = "CGI"; - } - else { - option = "--with-mail-gid"; - server = "mail"; - wrapper = "mail"; - } + pw = getpwuid(getuid()); + if (pw == NULL) + fatal(ident, USER_NAME_NOT_FOUND, + "Failure to find username"); - if (!mygroup) - fatal(ident, GROUP_NAME_NOT_FOUND, - "Failure to find group name for GID %d. Mailman\n" - "expected the %s wrapper to be executed as group\n" - "\"%s\", but the system's %s server executed the\n" - "wrapper as GID %d for which the name could not be\n" - "found. Try adding GID %d to your system as \"%s\",\n" - "or tweak your %s server to run the wrapper as group\n" - "\"%s\".", - mygid, wrapper, parentgroup, server, mygid, mygid, - parentgroup, server, parentgroup); + gr = getgrnam(parentgroup); + if (gr == NULL) + fatal(ident, GROUP_NAME_NOT_FOUND, + "Failure to find \"%s\" group", parentgroup); - if (strcmp(parentgroup, mygroup->gr_name)) - fatal(ident, GROUP_MISMATCH, - "Group mismatch error. Mailman expected the %s\n" - "wrapper script to be executed as group \"%s\", but\n" - "the system's %s server executed the %s script as\n" - "group \"%s\". Try tweaking the %s server to run the\n" - "script as group \"%s\", or re-run configure, \n" - "providing the command line option `%s=%s'.", - wrapper, parentgroup, server, wrapper, mygroup->gr_name, - server, parentgroup, option, mygroup->gr_name); + for (g = gr->gr_mem; *g; g++) { + if (strcmp(pw->pw_name, *g) == 0) { + ok = 1; + break; + } + } + + if (running_as_cgi) + wrapper = "CGI"; + else + wrapper = "mail"; + + if (ok == 0) + fatal(ident, GROUP_MISMATCH, + "Group mismatch error. Mailman expected the %s\n" + "wrapper script to be executed by a member of\n" + "\"%s\" group.", wrapper, parentgroup); }

2 2

Mailman 2.1.26 Security release Feb 4, 2018
by Mark Sapiro 08 Feb '18

08 Feb '18

An XSS vulnerability in the Mailman 2.1 web UI has been reported and assigned CVE-2018-5950 which is not yet public. I plan to release Mailman 2.1.26 along with a patch for older releases to fix this issue on Feb 4, 2018. At that time, full details of the vulnerability will be public. This is advance notice of the upcoming release and patch for those that need a week or two to prepare. The patch will be small and only affect one module. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 6

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

Mailman-Developers February 2018