Subject field not included in wrapper of DMARC emails
Dear Mailman folk,
I am subscribed to an OpenWRT mailing list, which (according to the X-Mailman-Verson header in emails I receive from the list) uses Mailman 2.1.29.
Most of the mail on that list comes through just fine.
DMARC email comes through with the Subject: field blank, though.
Is that expected? https://wiki.list.org/DEV/DMARC does not appear to mention the Subject: header being left out of the wrapper, when Mailman wraps a message to satisfy DMARC. This means that in my MUA, those messages are displayed without a Subject.
An example of an affected email is shown below, including the full headers.
Is there a Mailman 2 setting that I could recommend to the OpenWRT list managers, that would make Mailman copy the Subject: header from the original message to the wrapper, so that my MUA would show the subject as intended by the sender?
Thank you in advance for your help,
Sam
----- Forwarded message from "Etan Kissling (IC) via openwrt-devel" -----
Return-Path: <openwrt-devel-bounces+sampablokuper=posteo.net@lists.openwrt.org> Delivered-To: unknown Received: from posteo.de (185.67.36.145:993) by zian2 with IMAP4-SSL; 08 Feb 2021 21:32:21 -0000 Delivered-To: sampablokuper@posteo.net Received: from proxy02.posteo.name ([127.0.0.1]) by dovecot12 (Dovecot) with LMTP id 5oSRHYDhIGBLfgAAJesNpQ for <sampablokuper@posteo.net>; Mon, 08 Feb 2021 10:02:45 +0100 Received: from proxy02.posteo.de ([127.0.0.1]) by proxy02.posteo.name (Dovecot) with LMTP id GTflGLf5IGDQWQEAGFAyLg ; Mon, 08 Feb 2021 10:02:45 +0100 Received: from mailin04.posteo.de (unknown [10.0.0.64]) by proxy02.posteo.de (Postfix) with ESMTPS id 4DZ0Qq63k7z12Ck for <sampablokuper@posteo.net>; Mon, 8 Feb 2021 10:02:27 +0100 (CET) Received: from mx04.posteo.de (mailin04.posteo.de [127.0.0.1]) by mailin04.posteo.de (Postfix) with ESMTPS id 9627620F6B for <sampablokuper@posteo.net>; Mon, 8 Feb 2021 10:02:27 +0100 (CET) X-Virus-Scanned: amavisd-new at posteo.de X-Spam-Flag: NO X-Spam-Score: -1.533 X-Spam-Level: X-Spam-Status: No, score=-1.533 tagged_above=-1000 required=7 tests=[MAILING_LIST_MULTI=-1, MISSING_SUBJECT=1.767, POSTEO_BTC_B=0.01, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, T_RCVD_IN_ABUSIX_WHITE=0.01] autolearn=disabled Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by mx04.posteo.de (Postfix) with ESMTPS id 4DZ0Qk1lB3z10Zm for <sampablokuper@posteo.net>; Mon, 8 Feb 2021 10:02:21 +0100 (CET) Authentication-Results: posteo.de; dmarc=none (p=none dis=none) header.from=lists.openwrt.org Authentication-Results: posteo.de; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b=xkN+h566; dkim-atps=neutral DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type:List-Help: Reply-To:List-Archive:List-Unsubscribe:List-Subscribe:Cc:From:List-Post: List-Id:Message-ID:MIME-Version:To:References:Date:In-reply-to:Subject: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=BodNZRxYJ/ulISXrw/8ihEr0xuP4150RxcU4mqQbzK4=; b=xkN+h566WmBldbwfdHkBfI157 PDL/CI0WdfrRg5YTGiuFXnzvBoBLF5IU/Bhnrmhetwjs9z85NME1nKUXF49jqPTUc3D2/zqtd5vMk eVXY3/uAHIzbMWZL/bQQkRCADY44YEKBGflD0HlIttHy9gtqaZLBalRL435qoqxfhab8GntX0GiVO GSBqAmBp5VObGm+1oj//HwddZz4QuFWyEObG4MPsqtw7MFGK8FVw1Dumq36cUxICpD7teIcUhDP1y jLQ5xuMsSXJfeGL+NPp4Ki5pWywa/L4W9GAlOijWiFXVtk++j66AoyHHsjDwYyklgieQzOW/37ha/ G0ztzuEKw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l92Nh-0000HM-50; Mon, 08 Feb 2021 08:58:53 +0000 In-reply-to: <YBz6Pjl2SFe6Ky0L@thunderclap> Date: Mon, 08 Feb 2021 09:58:38 +0100 References: <YBz6Pjl2SFe6Ky0L@thunderclap> To: Baptiste Jonglez <baptiste@bitsofnetworks.org> MIME-Version: 1.0 Message-ID: <mailman.1473.1612774732.941.openwrt-devel@lists.openwrt.org> List-Id: OpenWrt Development List <openwrt-devel.lists.openwrt.org> List-Post: <mailto:openwrt-devel@lists.openwrt.org> From: "Etan Kissling (IC) via openwrt-devel" <openwrt-devel@lists.openwrt.org> Precedence: list Cc: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 X-BeenThere: openwrt-devel@lists.openwrt.org List-Subscribe: <https://lists.openwrt.org/mailman/listinfo/openwrt-devel>, <mailto:openwrt-devel-request@lists.openwrt.org?subject=subscribe> List-Unsubscribe: <https://lists.openwrt.org/mailman/options/openwrt-devel>, <mailto:openwrt-devel-request@lists.openwrt.org?subject=unsubscribe> List-Archive: <http://lists.openwrt.org/pipermail/openwrt-devel/> Reply-To: "Etan Kissling (IC)" <etan_kissling@apple.com> List-Help: <mailto:openwrt-devel-request@lists.openwrt.org?subject=help> Content-Type: multipart/mixed; boundary="===============4994977490922860622==" Sender: openwrt-devel <openwrt-devel-bounces@lists.openwrt.org> Errors-To: openwrt-devel-bounces+sampablokuper=posteo.net@lists.openwrt.org X-getmail-retrieved-from-mailbox: INBOX Content-Length: 7001
The sender domain has a DMARC Reject/Quarantine policy which disallows sending mailing list messages using the original "From" header.
To mitigate this problem, the original message has been wrapped automatically by the mailing list software.
Received: from ma1-aaemail-dr-lapp02.apple.com ([17.171.2.68]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l92Nd-0000EE-Uz for openwrt-devel@lists.openwrt.org; Mon, 08 Feb 2021 08:58:51 +0000 Received: from pps.filterd (ma1-aaemail-dr-lapp02.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp02.apple.com (8.16.0.42/8.16.0.42) with SMTP id 1188r62T023472; Mon, 8 Feb 2021 00:58:42 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=content-type : mime-version : subject : from : in-reply-to : date : cc : content-transfer-encoding : message-id : references : to; s=20180706; bh=+2HgbfEc8d2AnW/LqWiaafnkQdP0EWNyIoczFiPBM/Q=; b=i8Hp/kqjR8nz8o0yxx5I898YX8rveNwKUkxOQoINhu4EFpDNIpuiNp31MRQ2U/8UnP5N PvyWiHBqu3F8ccfF/gWXFT2SxV5hiucAXwfYxMDJqehDFXA0mFmu7JCjeA5QtGKH485K KiTzkGqyTQ2oGBg7YJAkH3Em6d80BdwSJIY6kJtmrlBmau+2C7tRiPDJVTI522TzblY2 KLcSq+AVpYhnoM2m2fIYzQZYT8pXAz9h7Y8P8OpsO/xlpr0Rt3gmfoIXLbkLOXVgZKXo TEyQj5ElOrieQUanyx0B3g9EpiuS6R7uUsABjyz+9GeSY41T7GwnXOlMKoma0n3FwqHS nQ== Received: from crk-mailsvcp-mta-lapp02.euro.apple.com (crk-mailsvcp-mta-lapp02.euro.apple.com [17.66.55.14]) by ma1-aaemail-dr-lapp02.apple.com with ESMTP id 36hrnvuw60-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Mon, 08 Feb 2021 00:58:42 -0800 Received: from crk-mailsvcp-mmp-lapp02.euro.apple.com (crk-mailsvcp-mmp-lapp02.euro.apple.com [17.72.136.16]) by crk-mailsvcp-mta-lapp02.euro.apple.com (Oracle Communications Messaging Server 8.1.0.7.20201203 64bit (built Dec 3 2020)) with ESMTPS id <0QO700RCRCXS0B00@crk-mailsvcp-mta-lapp02.euro.apple.com>; Mon, 08 Feb 2021 08:58:40 +0000 (GMT) Received: from process_milters-daemon.crk-mailsvcp-mmp-lapp02.euro.apple.com by crk-mailsvcp-mmp-lapp02.euro.apple.com (Oracle Communications Messaging Server 8.1.0.7.20201203 64bit (built Dec 3 2020)) id <0QO700K00CFZA900@crk-mailsvcp-mmp-lapp02.euro.apple.com>; Mon, 08 Feb 2021 08:58:40 +0000 (GMT) X-Va-A: X-Va-T-CD: 9afdb31050d7d5317f70deb897281ad1 X-Va-E-CD: 67e6792f740908ddf1e16c3353593782 X-Va-R-CD: b67e6762d78a271b674cde5fdf0795bd X-Va-CD: 0 X-Va-ID: 3c6f3588-7a2f-4a6c-b999-58547bff9a59 X-V-A: X-V-T-CD: 9afdb31050d7d5317f70deb897281ad1 X-V-E-CD: 67e6792f740908ddf1e16c3353593782 X-V-R-CD: b67e6762d78a271b674cde5fdf0795bd X-V-CD: 0 X-V-ID: a85131a9-49e2-48c0-b1b4-e57b988944b5 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.737 definitions=2021-02-08_02:2021-02-08,2021-02-08 signatures=0 Received: from [17.235.216.39] by crk-mailsvcp-mmp-lapp02.euro.apple.com (Oracle Communications Messaging Server 8.1.0.7.20201203 64bit (built Dec 3 2020)) with ESMTPSA id <0QO700FJLCXQ2B00@crk-mailsvcp-mmp-lapp02.euro.apple.com>; Mon, 08 Feb 2021 08:58:40 +0000 (GMT) Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: Upcoming 19.07.7 release From: "Etan Kissling (IC)" <etan_kissling@apple.com> In-reply-to: <YBz6Pjl2SFe6Ky0L@thunderclap> Date: Mon, 08 Feb 2021 09:58:38 +0100 Cc: openwrt-devel@lists.openwrt.org Content-transfer-encoding: quoted-printable Message-id: <1C978121-80FE-4E89-B587-60893371EF8A@apple.com> References: <YBz6Pjl2SFe6Ky0L@thunderclap> To: Baptiste Jonglez <baptiste@bitsofnetworks.org> X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.737 definitions=2021-02-08_02:2021-02-08,2021-02-08 signatures=0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210208_035850_214778_58700E6F X-CRM114-Status: GOOD ( 16.59 ) X-Spam-Score: -5.8 (-----) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-5.8 points) pts rule name description ---- ---------------------- -------------------------------------------------- -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at https://www.dnswl.org/, high trust [17.171.2.68 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [17.171.2.68 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.6 DKIMWL_WL_HIGH DKIMwl.org - High trust sender
I have posted a few backports to 19.07 from master a few weeks back, with these subjects:
- [PATCH 19.07] mbedtls: add config option to compile with hkdf
- [PATCH 19.07] hostapd: add multicast_to_unicast and per_sta_vif
- [PATCH 19.07] hostapd: enable CTRL_IFACE_MIB for hostapd-full
- [PATCH 19.07] nf-conntrack: allow querying conntrack info in nfqueue
- [PATCH 19.07] libnetfilter-queue: update to 1.0.5
My patches don't end up in Patchwork for some reason.
If those are acceptable, be sure to take the latest submission for the patches that were submitted multiple times.
Thanks
Etan
On 5 Feb 2021, at 08:56, Baptiste Jonglez <baptiste@bitsofnetworks.org> wrote:
Hi,
We are planning a new 19.07 release in about a week (probably next week-end).
If you are aware of changes that need to be integrated, now is the time to do it or mention it here!
I plan to test & integrate a workaround for this ramips stability issue: https://bugs.openwrt.org/index.php?do=details&task_id=2628
Baptiste
PS: please don't ask about 21.XX
openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
----- End forwarded message -----
On 2/13/21 10:44 PM, Sam Kuper wrote:
Most of the mail on that list comes through just fine.
DMARC email comes through with the Subject: field blank, though.
Is that expected?
Thank you for reporting this. It is a bug. It turns out that the logic for determining the headers in the wrapper does not include Subject: in the case where the list has no subject_prefix. This is now reported at <https://bugs.launchpad.net/mailman/+bug/1915655> Although I know you don't control the list, here's the patch I will commit after testing ```
=== modified file 'Mailman/Handlers/WrapMessage.py' --- Mailman/Handlers/WrapMessage.py 2018-06-17 23:47:34 +0000 +++ Mailman/Handlers/WrapMessage.py 2021-02-14 18:20:24 +0000 @@ -63,13 +63,17 @@ # make a copy of the msg, then delete almost everything and set/copy # what we want. omsg = copy.deepcopy(msg) + # If CookHeaders didn't change the Subject: we need to keep it too. + if 'subject' not in [key.lower() for key in + msgdata.get('add_header', {}).keys()]: + KEEPERS.append('subject') for key in msg.keys(): if key.lower() not in KEEPERS: del msg[key] msg['MIME-Version'] = '1.0' msg['Message-ID'] = Utils.unique_message_id(mlist) # Add the headers from CookHeaders. - for k, v in msgdata['add_header'].items(): + for k, v in msgdata.get('add_header', {}).items(): msg[k] = v # Are we including dmarc_wrapped_message_text? I.e., do we have text and # are we wrapping because of dmarc_moderation_action?
Perhaps you can convince the admins of the Mailman site to apply the
patch or add a subject_prefix to the list, either of which will avoid
the issue.
--
Mark Sapiro <mark@msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
On 2/14/21 10:34 AM, Mark Sapiro wrote:
Thank you for reporting this. It is a bug. It turns out that the logic for determining the headers in the wrapper does not include Subject: in the case where the list has no subject_prefix.
This is now reported at <https://bugs.launchpad.net/mailman/+bug/1915655>
Although I know you don't control the list, here's the patch I will commit after testing
Just for the record, that patch had issues. Here's the patch that will be committed: ```
=== modified file 'Mailman/Handlers/WrapMessage.py' --- Mailman/Handlers/WrapMessage.py 2018-06-17 23:47:34 +0000 +++ Mailman/Handlers/WrapMessage.py 2021-02-14 19:51:07 +0000 @@ -63,13 +63,19 @@ # make a copy of the msg, then delete almost everything and set/copy # what we want. omsg = copy.deepcopy(msg) + # If CookHeaders didn't change the Subject: we need to keep it too. + # Get a fresh list. + keepers = list(KEEPERS) + if 'subject' not in [key.lower() for key in + msgdata.get('add_header', {}).keys()]: + keepers.append('subject') for key in msg.keys(): - if key.lower() not in KEEPERS: + if key.lower() not in keepers: del msg[key] msg['MIME-Version'] = '1.0' msg['Message-ID'] = Utils.unique_message_id(mlist) # Add the headers from CookHeaders. - for k, v in msgdata['add_header'].items(): + for k, v in msgdata.get('add_header', {}).items(): msg[k] = v # Are we including dmarc_wrapped_message_text? I.e., do we have text and # are we wrapping because of dmarc_moderation_action?
``` -- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sun, Feb 14, 2021 at 10:34:08AM -0800, Mark Sapiro wrote:
Thank you for reporting this. It is a bug. It turns out that the logic for determining the headers in the wrapper does not include Subject: in the case where the list has no subject_prefix.
This is now reported at https://bugs.launchpad.net/mailman/+bug/1915655
Thanks for investigating and filing a report for the bug!
Sending much Free Software love to Gnu, this V-day. Thank you for maintaining Mailman :)
Perhaps you can convince the admins of the Mailman site to apply the patch or add a subject_prefix to the list, either of which will avoid the issue.
I will alert the admins to this thread.
On Sun, Feb 14, 2021 at 12:37:54PM -0800, Mark Sapiro wrote:
Just for the record, that patch had issues. Here's the patch that will be committed [...]
Thank you for double-checking the patch!
Sam
-- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing?
() ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
participants (2)
-
Mark Sapiro -
Sam Kuper