Mailman 3 March 2020 - Mailman-coders

[ mailman-Patches-657951 ] Generate RSS summary in archives
by SourceForge.net 19 Aug '22

19 Aug '22

Patches item #657951, was opened at 2002-12-23 14:17 Message generated for change (Comment added) made by bwarsaw You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=657951&group_i… Category: Pipermail Group: Mailman 2.2 / 3.0 Status: Open Resolution: None Priority: 7 Submitted By: A.M. Kuchling (akuchling) Assigned to: Nobody/Anonymous (nobody) Summary: Generate RSS summary in archives Initial Comment: Here's a first-draft patch. Things that need fixing: * The generated RSS feed needs to be validated. (It passed the W3C's RDF validator, but RSS validators still need to be checked.) * The date should be given in YYYY-MM-DD format, which requires parsing the .fromdate attribute. * How do I get the URL for an archived message? The generated RSS currently just uses the filename, which is wrong. How do I get at the PUBLIC_ARCHIVE_URL setting? * Getting the most recent N postings is inefficient; the code loops through all of the archived messages and takes the last N of them. We could add .last() and .prev() methods to the Database class, but that's more ambitious for 2.1beta than I like. (Would be nice to get this into 2.1final...) * The list index page should have a LINK element pointing to the RSS file. Please make any comments you have, and I'll rework the patch accordingly. ---------------------------------------------------------------------- >Comment By: Barry A. Warsaw (bwarsaw) Date: 2003-04-18 18:43 Message: Logged In: YES user_id=12800 Andrew, to get the url for the archived message use mlist.GetBaseArchiveURL(), which knows about private vs. public archives, the host name and the list name. From there you should be able to tack on just the part of the path under "archives/private/listname". See Mailman/Handlers/Scrubber.py for an example. Only other minor comment: NUM_ARTICLES can probably go in Defaults.py.in ---------------------------------------------------------------------- Comment By: Justin Mason (jmason) Date: 2003-03-26 16:49 Message: Logged In: YES user_id=935 big thumbs up from me too. Much better solution than http://taint.org/mmrss/ ;) ---------------------------------------------------------------------- Comment By: Uche Ogbuji (uche) Date: 2003-03-17 20:09 Message: Logged In: YES user_id=38966 I'd like to add my vote to this item. This is a fantastic idea, Andrew. Thanks. --Uche ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2002-12-23 15:42 Message: Logged In: YES user_id=11375 Updated patch: * Dates are now rendered as ISO-8601 (date only, not the time of the message) * By hard-wiring 2002-December, I got the RSS to validate using Mark Pilgrim's validator. ---------------------------------------------------------------------- Comment By: captain larry (captainlarry) Date: 2002-12-23 14:36 Message: Logged In: YES user_id=147905 Just voting for support here. This is *great* thanks for the patch and I hope the maintainers include it as soon as it's appropriate :) Adam. ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2002-12-23 14:27 Message: Logged In: YES user_id=12800 Deferring until post-2.1 ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2002-12-23 14:21 Message: Logged In: YES user_id=11375 Argh; SF choked on the file upload. Attaching the patch again... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=300103&aid=657951&group_i…

2 1

[Bug 1462755] [NEW] qrunner crashes on invalid unicode sequence
by Thijs Kinkhorst 28 Feb '22

28 Feb '22

Public bug reported: When a message contains an invalud unicode sequence in its header, qrunner flat out crashes on that: May 17 15:32:20 2015 (981) Uncaught runner exception: 'utf8' codec can't decode byte 0xe9 in position 18: invalid continuation byte May 17 15:32:20 2015 (981) Traceback (most recent call last): File "/var/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop self._onefile(msg, msgdata) File "/var/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose more = self._dopipeline(mlist, msg, msgdata, pipeline) File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline sys.modules[modname].process(mlist, msg, msgdata) File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 239, in process i18ndesc = uheader(mlist, mlist.description, 'List-Id', maxlinelen=998) File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 65, in uheader return Header(s, charset, maxlinelen, header_name, continuation_ws) File "/usr/lib/python2.7/email/header.py", line 183, in __init__ self.append(s, charset, errors) File "/usr/lib/python2.7/email/header.py", line 267, in append ustr = unicode(s, incodec, errors) UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 18: invalid continuation byte May 17 15:32:20 2015 (981) SHUNTING: 1431869540.389822+156779307d54473d0eb732994bb67eee95733285 A solution for this specific case is to have Mailman/Handlers/CookHeaders.py pass the erorrs='replace' parameter. I would say that this is actually a bug in python-email, since I think it doesn't make sense to set errors to "strict" rather than something like "replace" when the intention is to parse stuff so free-formed, under-specd and user-controlled as email. Nonetheless, Mailman already sets errors='replace' in some places so it might aswell add it here. ** Affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1462755 Title: qrunner crashes on invalid unicode sequence To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1462755/+subscriptions

6 12

[Bug 965532] [NEW] Need a script to upgrade from MM2 to MM3
by Barry Warsaw 13 Dec '21

13 Dec '21

Public bug reported: We need a script, documentation, or other procedure to help people migrate from Mailman 2 to Mailman 3. ** Affects: mailman Importance: Undecided Status: New ** Tags: mailman3 -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/965532 Title: Need a script to upgrade from MM2 to MM3 To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/965532/+subscriptions

8 21

[Bug 1609516] [NEW] [needs-packaging] GNU Mailman v3
by Kẏra 13 May '21

13 May '21

Public bug reported: Mailman 3 is essentially five projects: Mailman Core Postorius - The Web UI for Mailman Mailman Client - The REST API Client HyperKitty - The Archiver for Mailman Mailman Bundler - Installer for Mailman Suite including all above projects URL: https://www.gnu.org/software/mailman/ ** Affects: mailman Importance: Undecided Status: New ** Affects: ubuntu Importance: Undecided Status: New ** Affects: debian Importance: Unknown Status: Unknown ** Tags: needs-packaging ** Bug watch added: Debian Bug tracker #799292 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799292 ** Also affects: debian via http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799292 Importance: Unknown Status: Unknown ** Also affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1609516 Title: [needs-packaging] GNU Mailman v3 To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1609516/+subscriptions

3 7

[Bug 544477] [NEW] setting localhost in postfix_lmtp breaks delivery
by Patrick Ben Koetter 26 Sep '20

26 Sep '20

Public bug reported: Setting localhost in postfix_lmtp works against Postfix defaults and breaks delivery. Reason: The Postfix lmtp client only uses dns queries by default to search for hostnames. If the DNS server does not provide an answer for "localhost" delivery from the lmtp client to mailman fails. Solution: Provide "127.0.0.1" instead of "localhost". This does not require DNS and is even faster because it saves DNS lookups. Example: hey2(a)mailman.state-of-mind.de lmtp:[localhost.localdomain]:8024 ** Affects: mailman Importance: Undecided Status: New ** Tags: 3.0 mailman -- setting localhost in postfix_lmtp breaks delivery https://bugs.launchpad.net/bugs/544477 You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman.

4 7

[Bug 1731604] [NEW] VERP fails if the recipient address local part is quoted.
by Mark Sapiro 13 Apr '20

13 Apr '20

Public bug reported: We have seen (very rarely) cases where a recipient address has a quoted local part such as "jr."@example.org. This results in a VERPed sender address like list-bounces+"jr."=example.org(a)example.com which is syntactically invalid. The VERPed sender should be "list- bounces+jr.=example.org"@example.com in this case. I.e., the entire VERPed sender local part should be quoted, not just the recipient local part. This also requires BounceRunner to recognize this and restore the original local part. ** Affects: mailman Importance: Medium Assignee: Mark Sapiro (msapiro) Status: In Progress -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1731604 Title: VERP fails if the recipient address local part is quoted. To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1731604/+subscriptions

1 6

[Bug 1859011] [NEW] bounce mail processed multiple times -> oom crash of BounceRunner
by Michael Menge 13 Apr '20

13 Apr '20

Public bug reported: System: RHEL 7.7 Mailman: mailman-2.1.15-26.el7_4.1.x86_64 Description: We have 2 Mailinglist with > 7000 Members which trigger a spike in processed bounces (more than 3000 in an sinle run), followed by an Out of Memory situation in the BounceRunner and an > 20 GB bounce-event-XXXX.pck file. We tried to mitigate the problem by increasing the system memory, running the BoucneRunner every minute and limiting the number of mails delivered at ounce by postfix. But it happened again: Dec 30 19:53:29 2019 (13392) <BounceRunner at 140395473885088> processing 4134 queued bounces ... Dec 30 19:53:59 mx09 kernel: [13392] 41 13392 2755797 1874474 5337 825695 0 python ... Dec 30 19:53:59 mx09 kernel: Out of memory: Kill process 13392 (python) score 896 or sacrifice child Dec 30 19:53:59 mx09 kernel: Killed process 13392 (python), UID 41, total-vm:11023188kB, anon-rss:7497896kB, file-rss:0kB, shmem-rss:0kB We analyzed the bounce-event file, extracting data with "stings". This time we extracted the postfix mail queue ids from the received headers with our listserver. We found the following: cat /tmp/bounce-20191230.txt | sed 's/;//' | sort | uniq -c | sort -n 1 01A7DE9314 1 10F6AE9319 1 18456E930E 1 27D0BAC960 1 3B51CE9316 1 57C2DAC992 1 5D3B2E9310 1 5EF11E9311 1 63054E9312 1 69377E9313 1 ED636E930F 2 29884E9315 2 49ECEAC98D 2 99A16A9DA7 2 (Postfix) 3 59EE1AC995 3 CEB61AC996 192 C12E9D3B48 3929 F2BEEE9318 4122 CC58AAC993 4134 6EFD1A9DA7 As the bounces the last 2 qids are from the original mail send to the list (6EFD1A9DA7), and one of mails send by mailman to 500 members of that list (CC58AAC993). (F2BEEE9318 and C12E9D3B48) are both bounces from members that where deliverd only once to /usr/lib/mailman/mail/mailman bounces NAME-OF-HUGE-LIST So from me it seems that somehow some few bounce where "multiplied" so that ~20 real bounce produced 4134 virtual bounce. I see the potential of a deny of service attack, as it could be used to fill up the disk where the bounce-event files get dumped But I don't know if this would warrant marking "This bug is a security vulnerability". ** Affects: mailman Importance: Undecided Status: New -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1859011 Title: bounce mail processed multiple times -> oom crash of BounceRunner To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1859011/+subscriptions

3 14

[Bug 1838866] [NEW] Mailman 2.1: hardcoded site-packages dir prefix doesn't work on some 64 bit filesystem layouts
by Hanno Böck 07 Mar '20

07 Mar '20

Public bug reported: Mailman's paths.py.in contains code to add the python site-packages dir to the path. This uses a hardcoded "lib" prefix: # Include Python's site-packages directory. sitedir = os.path.join(sys.prefix, 'lib', 'python'+sys.version[:3], 'site-packages') sys.path.append(sitedir) On 64 bit systems this is usually lib64 and some distros started removing the compatibility symlink lib->lib64 and split up lib and lib64 (e.g. latest Gentoo profiles). So relying on "lib" as a prefix is not compatible any more. Python has a function getsitepackages in the site module and I believe it's best to rely on python knowing best where its directories are. It returns an array, so this would look like this: sitedirs = site.getsitepackages() for sitedir in sitedirs: sys.path.append(sitedir) Patch attached, please apply. ** Affects: mailman Importance: Undecided Status: New ** Attachment added: "fix sitedir path on 64 bit filesystem layouts" https://bugs.launchpad.net/bugs/1838866/+attachment/5280705/+files/mailman-… -- You received this bug notification because you are a member of Mailman Coders, which is subscribed to GNU Mailman. https://bugs.launchpad.net/bugs/1838866 Title: Mailman 2.1: hardcoded site-packages dir prefix doesn't work on some 64 bit filesystem layouts To manage notifications about this bug go to: https://bugs.launchpad.net/mailman/+bug/1838866/+subscriptions

6 10

Mailman-coders March 2020