Mailman 3 - Mailman-Developers

FYI: Another attempt at describing From: rewriting
by Alessandro Vesely Oct. 6, 2021

Oct. 6, 2021

Hi, Douglas Foster posted this draft to the IETF. For discussion, see: https://mailarchive.ietf.org/arch/msg/dmarc/Fp1nnmBc242kxvxM7seVwoQ6ccU Best Ale ---------- Forwarded message --------- From: <internet-drafts(a)ietf.org> Date: Sun, Oct 3, 2021 at 8:14 PM Subject: New Version Notification for draft-fosterd-dmarc-compliant-mailing-lists-00.txt To: Douglas Foster <dougfoster.emailstandards(a)gmail.com> A new version of I-D, draft-fosterd-dmarc-compliant-mailing-lists-00.… [View More]

1 0

Ann: Postorius 1.3.6b1 is now out
by Abhilash Raj Sept. 29, 2021

Sept. 29, 2021

Hey Everyone, I am pleased to announce that Postorius 1.3.6b1 is now out. This is the first pre-release for 1.3.6, which is slated to release next week soon after Mailman Core 3.3.5 release. This release requires Mailman Core 3.3.5 release, so if you are trying out this release, please also upgrade Mailman Core to 3.3.5rc1 (which was just released today). This release includes several new features and bug fixes. A full list of changes is available here[1]. With this release, we have … [View More]

1 1

Ann: Mailman Core 3.3.5b1 is out
by Abhilash Raj Sept. 29, 2021

Sept. 29, 2021

Hello Everyone, I am pleased to announce that Mailman Core 3.3.5b1 is now out. It is a pre-release for 3.3.5, which is slated to come out in 3 weeks from now. I am planning for a 2 week beta period, after which I'll release the first RC and then stable a week after. Right now I am not planning to release a second beta version, but if there are several changes in Core in the next week or so, then I might. This release includes a lot of bug fixes and some new features. It also includes … [View More]

1 2

Docker-mailman 0.4.0 release
by Abhilash Raj Sept. 23, 2021

Sept. 23, 2021

Hey Everyone, I tagged and released 0.4.0 version of container images for Mailman. This release includes several fixes that have been accumulating in the repo for how Mailman is setup in the containers. This release does not include any mailman component version bump from 0.3.12 release. More specifically, the major change is that we no longer require static IP configuration in the container images (like previously) and can handle dynamic addresses that are allocated to the containers by … [View More]

1 0

Django primer for Mailman admins
by Abhilash Raj Sept. 20, 2021

Sept. 20, 2021

Hey Everyone, This weekend I wrote some documentation[1] that I thought would be useful for Mailman admins when dealing with Django, since it is a frequent topic on -users list. The intent was to add only required information for admins to work with Mailman installs and provide enough pointers to right places, wherein more information can be found instead of duplicating them in Mailman's docs since it it often goes stale. Since the most common theme of questions are configuration (… [View More]

1 0

Freenode or Libera for IRC?
by Damian Johnson Sept. 16, 2021

Sept. 16, 2021

Hi all. I'm a prospective new contributor [1] presently running through the onboard process. Wikipedia transitioned from Freenode to Libera [2] which is run by Freenode's former operators [3]. I haven't paid further attention to their breakup drama, but Andrew Lee's post about a "Joseon Empire" on Freenode's frontpage [4] doesn't fill me with confidence. Just food for thought but has Mailman considered moving to Libera [5]? Cheers! -Damian [1] https://blog.atagar.com/ [2] https://meta.… [View More]

3 3

Mailman container images 0.3.12 is now out
by Abhilash Raj Sept. 5, 2021

Sept. 5, 2021

Hey Everyone, I have just tagged release 0.3.12 on Github for container images for Mailman 3[1]. This release includes the fix for CVE-2021-40347 that was announced earlier today. For the folks using 0.3 or 0.3.11 release tags, it is highly recommended that you upgrade to this release. This release also bumps the version of Mailman Core to 3.3.4, Mailmanclient to 3.3.3 and Django-mailman3 to 1.3.7. Note that is since the main and v0.3.12 branches are different in many ways, the default … [View More]documentation[2] and the docker-compose.yaml files in the main branch aren't accurate if you are using the stable release. Please refer to the README[3] at v0.3.12 tag in the Github repo for more accurate docker-compose.yaml and documentation. The project has grown large enough that we need to start versioning the documentation, if someone has experience with versioning docs using Github pages and mkdocs, then I very much need some help here! For those of you who are using the rolling release, it is recommended that you **don't** upgrade to this stable release. The fixes have been pulled into the rolling tags too, so just make sure that you upgrade to the latest published version of rolling release, which as of this writing should be based off on fda837f8d15540e190992c30f7971f50fca54dac commit[4]. This might not be the latest by the time you upgrade if I add a new commit, so look for versions published after 4:00 PM PST 9/5/2021. I am also working on cutting a new release, 0.4.0, which is backwards incompatible with the setup required to talk to web server and MTA (hence the minor version bump!). That should bring the rolling releases and stable releases closer to each other and add improvements around not needing static IPs in the docker network anymore, plus several bug fixes. If someone wants to test the upgrade to 0.4.0 from 0.3 release and is willing to try out the instructions at [5], it would give me some confidence in cutting out the release sooner. The only thing stopping the release of 0.4.0 images is that I haven't verified if the upgrade from 0.3 is documented enough or not. For all the registries listed in README[6], I am still trying to push to Quay (maybe I need to just skip pushing to Quay :-). So, just use the other two to pull the images, Github (ghcr.io) has more generous pull download policy for un-authenticated users though. [1]: https://github.com/maxking/docker-mailman/releases/tag/v0.3.12 [2]: https://asynchronous.in/docker-mailman/ [3]: https://github.com/maxking/docker-mailman/tree/v0.3.12 [4]: https://github.com/maxking/docker-mailman/commit/fda837f8d15540e190992c30f7… [5]: https://asynchronous.in/docker-mailman/news/#upgrading-to-040-release [6]: https://github.com/maxking/docker-mailman#container-registries -- thanks, Abhilash Raj (maxking) [View Less]

1 0

New vulnerability in Hyperkitty master branch
by Abhilash Raj Sept. 5, 2021

Sept. 5, 2021

Hey Everyone, A new vulnerability was reported against Hyperkitty’s git master branch branch which can expose the archives of a private Mailing List through the new Feeds API that was added to Hyperkitty recently to someone who isn't a member or logged-in. Thanks to Ngo Wei Lin for reporting this vulnerability. This bug does not affect any stable released version of Hyperkitty and only affects installations from source (1.3.5b1 version). To differentiate from the vulnerable version, I … [View More]

1 0

CVE-2021-40347: New security release for Postorius
by Abhilash Raj Sept. 5, 2021

Sept. 5, 2021

Everyone, A security vulnerability was reported against Postorius recently which allows any logged-in user to unsubscribe any other member on any other list on same Mailman installation using a specially crafted POST request due to a missing ownership check. This has been assigned CVE-2021-40347. This affects all past versions of Postorius including 1.0.0. Thanks to Kunal Mehta for the security report and a quick patch to fix the vulnerability. I am also attaching a minimal patch that … [View More]

1 0

using kernel events to watch queue directories
by Phil Budne Aug. 6, 2021

Aug. 6, 2021

Some time ago I had done a (crude) patch to mailman v2 to use FreeBSD "kqueue" to allow runners to wake up immediately when a file was queued, enabling most runners to sleep indefinitely, keeping overhead on my small Virtual Private Server to a minimum. I offered the patch to the FreeBSD mailman "port" maintainer, but they only expressed interest in a mailman3 patch. Well, I've recently/finally switched over to mailman v3, and I just did what I hope is a cleaner version of the change. I didn'… [View More]t see a PyPI library with cross platform support that didn't want to usurp the main loop, and since I already had the kqueue code, I wrote the simplest possible "DirWatcher" class family. The change aims to "first, do no harm": it will always fall back to using sleep. It will also use the Linux "inotify" facility using the PyPI inotify_simple package (if installed), which I debugged "in the lab". NOTE!! I'm almost certainly seeing signal handling problems when I try to shut down mailman (under FreeBSD), so the changes should not be considered ready for production in any way! I'm not a zope interfaces user, nor a student of mailman project conventions, so again, I'm not asserting the code, as is, is ready for integration. I AM interested in knowing if there's any interest in this, before I bother setting myself up on gitlab, and bringing the change forward from 3.3.1. Something I consider an open issue is that right now I've changed the default sleep_time in schema.cfg from 1 minute to 15 minutes, which is FAR less than ideal if the code falls back to (blind) sleep between directory scans. I considered looking to see if the current instance's class had overridden _do_periodic from the base Runner class, and if not, sleeping indefinitely (or at least a "long time"), but that seemed like a hack, and for testing, it was easier to not write any ugly code, and just wack schema.cfg. Since the changes are small, I'm including them in-line below. ==== core/dirwatch.py """ Simple O/S independent Class to sleep until a single directory changes (written for Mailman) Phil Budne <phil(a)regressive.org> August 5, 2021 """ import errno # kqueue import os # kqueue import select # kqueue import time # kqueue, base from mailman.interfaces.dirwatch import IDirWatcher from public import public from zope.interface import implementer _alternatives = [] # list of classes to try DEBUG = False # TEMP!!! @public @implementer(IDirWatcher) class BaseDirWatcher: """See `IDirWatch`.""" USE = True def __init__(self, directory): """Create a dirwatch object. :param directory: The directory to watch. """ self.directory = directory # TEMP!!! if DEBUG: self.log = open("/tmp/dirwatch.log", "a") def watch(self, float_sec): """See `IDirWatcher`.""" self.debug("BaseDirWatcher sleeping %s", float_sec) # TEMP time.sleep(float_sec) def debug(self, format, *args): # TEMP!!! if DEBUG: self.log.write(("%s [%d] " + format + "\n") % ((time.time(), os.getpid(),) + args)) self.log.flush() @public @implementer(IDirWatcher) class KQueueDirWatcher(BaseDirWatcher): def __init__(self, directory): super().__init__(directory) self._kq = self._kq_dirfd = None self._kq = select.kqueue() self._kq_dirfd = os.open(self.directory, os.O_RDONLY) ev = select.kevent(self._kq_dirfd, filter=select.KQ_FILTER_VNODE, flags =select.KQ_EV_ADD|select.KQ_EV_CLEAR, fflags=select.KQ_NOTE_WRITE) # install ev, return no events, don't sleep self._kq.control([ev], 0, 0) def watch(self, float_sec): self.debug("kq.watch %s %s", self.USE, self._kq) # TEMP if self.USE and self._kq: try: self.debug('calling kq.control wait=%s', float_sec) # TEMP # no changes to events, return at most 10 events # (should only ever return 1?) events = self._kq.control([], 10, float_sec) self.debug('kq.control returned %d event(s)', len(events)) # TEMP # ignore returned events return except KeyboardInterrupt: raise except OSError as exc: # suppress EINTR, like time.sleep if exc.errno == errno.EINTR: return self.debug('kq.control exception: %s', exc) # TEMP super().sleep(float_sec) _alternatives.append(KQueueDirWatcher) # Linux inotify not an included battery. # Ubuntu (and debian?) have pyinotify (old/unmaintained?) which wants # to take over the main loop, and do callbacks. If one felt # strongly about it, one could create a pyinotify.Notifier # object with a timeout value set, create an event handler that # sets a bool in the DirWatcher object, and have watch call # {check,read,process}_events (or something like that). But # inotify_simple is so much .... simpler. Since determining # what's available happens at run time, you could have BOTH! try: import inotify_simple @public @implementer(IDirWatcher) class INotifySimpleDirWatcher(BaseDirWatcher): def __init__(self, directory): super().__init__(directory) self.inotify = inotify_simple.INotify() # just look for renames self.inotify.add_watch(directory, inotify_simple.masks.MOVE) self.debug('inotify_simple init succeeded') # TEMP def watch(self, float_seconds): self.debug("inotify.watch %s %s", self.USE, self.inotify) # TEMP if self.USE and self.inotify: try: events = self.inotify.read(int(float_seconds*1000 + 0.5)) self.debug("inotify_simple read returned %d", len(events)) # TEMP return except KeyboardInterrupt: raise except OSError as exc: print(exc) # suppress EINTR, like time.sleep if exc.errno == errno.EINTR: return except Exception as exc: pass super().watch(float_seconds) _alternatives.append(INotifySimpleDirWatcher) except: pass def factory(directory): for dw_class in _alternatives: if dw_class.USE: try: return dw_class(directory) except: self.debug("%s init failed", dw_class) # TEMP dw_class.USE = False return BaseDirWatcher(directory) if __name__ == '__main__': watcher = factory('.') watcher.debug("got %s", watcher) # TEMP # enough time to type "touch foo" (and "mv foo bar" on Linux) watcher.watch(20.0) ==== interfaces/dirwatch.py """Interface for directory watchers.""" from public import public from zope.interface import Interface @public class IDirWatcher(Interface): """The directory watcher.""" def watch(self, float_sec): """Watch for new files in directory, for at most float_sec seconds. No value returned. """ ==== diffs --- config/schema.cfg-mm3.3.1 2021-07-07 17:24:59.000000000 -0400 +++ config/schema.cfg 2021-08-05 20:53:06.552912000 -0400 @@ -256,7 +256,9 @@ # The sleep interval for the runner. It wakes up once every interval to # process the files in its slice of the queue directory. Some runners may # ignore this. -sleep_time: 1s +# PLB: Using dirwatch, don't need to poll. unless runner has +# _do_periodic work. +sleep_time: 15m [database] --- core/runner.py-mm3.3.1 2020-04-10 01:53:06.000000000 -0400 +++ core/runner.py 2021-08-05 17:36:41.769775000 -0400 @@ -293,7 +293,10 @@ """See `IRunner`.""" if filecnt or self.sleep_float <= 0: return - time.sleep(self.sleep_float) + if self.switchboard: + self.switchboard.snooze(self.sleep_float) + else: + time.sleep(self.sleep_float) def _short_circuit(self): """See `IRunner`.""" --- interfaces/switchboard.py-mm3.3.1 2020-04-10 01:53:06.000000000 -0400 +++ interfaces/switchboard.py 2021-08-05 17:47:46.712414000 -0400 @@ -84,3 +84,6 @@ time, so moving them is enough to ensure that a normal dequeing operation will handle them. """ + + def snooze(self, sleep_float): + """Wait at most sleep_float seconds for a change to the queue.""" [View Less]

1 0