Mailman 3 March 2015 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett 07 Nov '22

07 Nov '22

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

Mailman 2.1.20 release.
by Mark Sapiro 31 Mar '15

31 Mar '15

I am pleased to announce the release Mailman 2.1.20. Python 2.4 is the minimum supported, but Python 2.7 is recommended. This release has one new feature, a few bug fixes, an update to the Polish message catalog and a fix for a recently discovered security vulnerability. See the attached README for more details. Mailman is free software for managing email mailing lists and e-newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites. For more information, please see: http://www.list.org http://www.gnu.org/software/mailman http://mailman.sourceforge.net/ Mailman 2.1.20 can be downloaded from https://launchpad.net/mailman/2.1/ http://ftp.gnu.org/gnu/mailman/ https://sourceforge.net/projects/mailman/ -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

Security patch and Mailman 2.1.20 to be released on 31 March
by Mark Sapiro 31 Mar '15

31 Mar '15

A security vulnerability in Mailman has been found and fixed. It has been assigned CVE-2015-2775. The details of this vulnerability and fix will be announced next Tuesday, 31 March 2015, at which time both a patch for this specific vulnerability and Mailman 2.1.20 will be released. In addition to this security fix, Mailman 2.1.20 includes a new feature allowing a list owner to change a list member's address through the admin Membership Management... Section, and a couple of minor bug fixes. The new feature is a fix for <https://launchpad.net/bugs/266809>. The bugs fixed are: <https://launchpad.net/bugs/1426825>, <https://launchpad.net/bugs/1426829> and <https://launchpad.net/bugs/1427389>. The security vulnerability, the details of which are currently private, is <https://launchpad.net/bugs/1437145>. The security vulnerability only affects those installations which use Exim, Postfix's postfix_to_mailman.py or similar programmatic (not aliases) MTA delivery to Mailman, and have untrusted local users on the Mailman server. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 1

Fine grained subscription control, scaling, noise, relevancy, subscription rules, dynamic lists
by Andrew Stuart 29 Mar '15

29 Mar '15

Refer to the bottom of this email for some previous quotes from @barry on this topic. I’ve also had off list discussions with Barry in which he has mentioned this same topic so it seems to have some previous thought gone into it. I’m wanting to raise the topic of “fine grained subscription control” (for want of a better term) for discussion. Please note these thoughts refer to discussion style lists rather than notification style lists. Please prefix every sentence here with IMO - I’m not saying I’m right about anything, just putting forward food for thought. Also, I’m no Mailman expert so if my assumptions are plain wrong please let me know. One of the core problems of mailing lists (at least as implemented in Mailman) is that there’s not much middle ground between being very involved (i.e receive all posts to list), or being almost-not-involved (i.e. receiving daily digests or no digest). List noise and relevancy is the main problem and it’s a big impediment to lists being more widely used. Very engaged users might be ok with constantly deleting messages that they are not interested in (i.e. irrelevant noise). Less engaged users will be annoyed with the noise and unsubscribe, or will switch to digest notification which I suggest to you is close to unsubscription anyway because that user no longer sees emails that they might have been interested in had they known there was a message on that subject. Digest users I suggest are at best observers rather than participators. The more active the list, the greater the noise problem, the more users will drop out of the list. As a list gains more subscribers, eventually even the most engaged user will have had enough of the volume of messages and will drop back to digest or write some mailbox rule that moves the emails to a folder, effectively dis-engaging them from the list conversation flow. Thus Mailman discussion-style lists currently don’t scale. Even for low volume lists, noise is a major inhibitor to usage in certain contexts. Consider for example the CIO of a company or the manager of a division of 30 developers. There are various mailing lists being used by the various projects that they are responsible for. The leader is not participating however because the noise from those lists would be overwhelming. They would however like to be partipcating in list discussions either that they initiated, are explicitly copied into, or relate to topics (keywords) that they are interested in. In reality, I’m not convinced Mailman would often be used in contexts like this currently because the relevancy/noise/digest/unsubscribe problem is a showstopper. The systers have recognised this problem and their solution is Dynamic sublists as described here: http://wiki.list.org/DEV/Dynamic%20Sublists "List subscribers can decide whether to be a part of new conversations or not. If the users decide to subscribe to new conversations, then they will receive all the messages of a new conversation unless they explicitly unsubscribe from it. If they otherwise decide to unsubscribe from new conversations,then they will receive only the first message of every new conversation unless they explicitly subscribe to it.” I don’t think dynamic sublists are an effective solution (IMO, remember?). Getting the root message in all conversations is still too noisy and its a clumsy mechanic to have to opt in to further discussion and my thinking is that opt-in-to-discuss will impose a barrier to engagement that will reduce user interaction - put another way - “opt in to every discussion? too hard”. Possibly a solution worth considering is fine-grained subscription control, in which there is a set of SEND and DONTSEND rules at a List default level and at a user level. SEND: Discussions where list member email address is in to/from/cc: Discussions where to/from/cc contains one or more of: sally(a)example.org, davina(a)example.org, *.example.com Discussions where subject contains keyword: hyperkitty Discussions where body contains keyword: hyperkitty DONTSEND: Discussions where list member email address is in to/from/cc: Discussions where to/from/cc contains one or more of: sally(a)example.org, davina(a)example.org Discussions where subject contains keyword: Java Discussions where body contains keyword: ruby The technical hurdle to making this work is that Mailman needs access to historical messages to make it work (i.e. integrating some level of aqrchive like functionality into the Mailman database). I suggest this this may not be as hard as it sounds and hey, we’ve got a database there anyway so why not use it? One possibility is that this could be pushed into archiving but I don’t think that actually is practical - such concepts really need to be built deep into the core. I’m not advocating archiving-in-core here because I think archiving should like outside core. I do think however that there is value in the core having access to archive data to implement fine grained subscription control and I think it should do so on its own terms (and in its own database) rather than asking archivers to provide the needed functionality. Fine grained subscription control whilst probably relatively involved to implement, is sure to have a huge payoff in giving users more relevance and less noise, perhaps increasing uptake of Mailman in situations where the noise and relevancy problem currently make Mailman impractical. Thoughts anyone? as Quoting @barry …. http://mailman.9.n7.nabble.com/Improving-the-archives-tt31296.html#a31340 “”"I like this for several reasons. I've long wanted a bridge between the traditional mailing list and a forum because to me they’re related along a spectrum of emotional investment. What I mean is this. For the subjects and projects I care deeply about, I join the mailing list. I want to be intimately involved in the day-to-day collaboration that being subscribed gives me. I care enough about that that I'm willing to put up with the pain that comes along with mailing lists, such as the overhead for subscribing, deleting topics I don't care about, the occasional spam, the overhead of going on vacation or leaving the list, etc. But there are even more topics or projects that I have only a fleeting interest in. Say I find a bug in some X program, or wake up and decide to learn how to use setuptools, or find that some recent update broke my Linux server. In all those cases, I might want to start a thread of discussion or ask a question, and be very involved in that thread for a week or two. Then, my interest wanes, or I get my question answered, or other projects pique my interest. Mailing lists are pretty bad at managing those kinds of fleeting involvement, but forums are quite nice. There's usually fairly low overhead (and probably even less if OpenID and such were in widespread adoption) for joining, and when I lose interest the forum doesn't fill up my inbox. OTOH, forums seem good for short 'instant' messages, but not so good (IMO) for free ranging, detailed discussions. So there's a spectrum. “””

2 3

Re: [Mailman-Developers] GSoC 15 - Interested in contributing to Hyperkitty
by David Udelson 27 Mar '15

27 Mar '15

Stephen commented on my proposal that instead of storing all downloads on the server, we should only cache recent ones and build the others on request, because many archives are only downloaded once when they are ported to a different archiver. Since this is in contradiction to the suggestion Aurelien made, I'm going to leave my proposal as-is for now. Hopefully the link to this thread in my proposal will be a source of up-to-date information on project constraints during the application review process. On Thu, Mar 26, 2015 at 11:15 AM, David Udelson <dru5(a)cornell.edu> wrote: > > I'm already using the "jobs" infrastructure provided by the > > django-extensions package: > > http://django-extensions.readthedocs.org/en/latest/jobs_scheduling.html > > Cool. I didn't know about this extension, but it looks like it does what > we need. So the background process would be its own file in the jobs > directory, and we could leave it to the admin to setup the crontab? > > > I have another test server with more current info if you want, but I > > break it regularly. It's lists-dev.cloud.fedoraproject.org > > Thanks for linking this. I got my own local dev server working yesterday, > but this one is much more populated. > > > We do put the attachment in the mbox, as a MIME component like in > > every email. > > I see how this works now. Are the attachments always Base64 encoded? > > >> Another possible "nice-to-have" feature I thought of yesterday is a > >> download link that scripts can use to get archives (e.g. > >> "/download?year=x&month=y"). On the other hand, maybe this is just a > >> security risk that has no actual use case, but I'd still like to have a > >> second opinion on this. > > > > Well, there still is the authentication issue. > > I guess getting the scripts to authenticate would be a little complicated, > but otherwise does this seem like something worth including? If my proposal > gets accepted, I'm ok with leaving this as an open question until it > becomes clear whether or not I'm going to have extra time at the end of the > summer. > > Thanks, > David > > > > > On Thu, Mar 26, 2015 at 7:33 AM, Aurelien Bompard <aurelien(a)bompard.org> > wrote: > >> > In my proposal I suggested using any of several asynchronous job queue >> > libraries, such as Celery or Huey. These all use redis as a back-end. >> > Because I have no experience with asynchronous job queues, I'm not sure >> if >> > this is too much baggage for our purposes. Maybe we just don't want the >> > extra dependencies. >> >> Yeah, we don't want to add another database or an AMQP server just for >> that. We must keep it simple for admins to deploy. >> >> > Regarding cron jobs, there's also django-background-task which is a >> simple >> > django addon that might do what we need. Again, if we don't want/need >> the >> > extra dependency, rolling our own cron job should be fairly >> > straight-forward. >> >> I'm already using the "jobs" infrastructure provided by the >> django-extensions package: >> http://django-extensions.readthedocs.org/en/latest/jobs_scheduling.html >> I did consider django-background-task but django-extensions seemed >> like a better fit, because django-background-task seems written for >> delayed tasks, not periodic tasks (well, a task could call itself >> again when done, but it seems like a hack). I'm not opposed to >> switching to django-background-task if we use the "delayed job" >> feature or if we need the extra flexibility of choosing exactly how >> many seconds apart we want our tasks to run. >> >> >> If we choose to pre-build the mbox files, we can't simply have them >> > served through the webserver, because some lists are private >> > >> > Then there is also an authentication step? >> >> Yeah, we must use HyperKitty's authentication and check if the user is >> allowed to see the archive. So the files can't be served by the >> webserver like static files. >> >> > I noticed on the test server >> > that I can't actually look at any of the mailing lists because they're >> all >> > private. >> >> If you're looking at lists.stg.fedoraproject.org, it's currently very >> outdated (still running the Python2-compatible branch of Mailman 3). I >> have another test server with more current info if you want, but I >> break it regularly. It's lists-dev.cloud.fedoraproject.org >> >> > When we create the mbox file, do we simply note that an attachment >> existed >> > (e.g. "Attachment: myattachment.txt") or do we actually put the >> attachment >> > in the mbox? AFAIK mbox is a plaintext format, so if the latter is the >> case >> > then I'm not exactly sure how this would work... >> >> We do put the attachment in the mbox, as a MIME component like in >> every email. If you choose "view source" when looking at an email with >> attachments, you'll see how it's done. >> >> > Are there going to be any issues handling unicode foreign characters or >> > with file locks? Right now it looks like we should only have one process >> > handling the mbox, but is it possible that more than one could be >> spawned >> > somehow? >> >> No, mbox files are not designed for concurrent writes, so it's better >> to have a single process write to them. >> >> > Another possible "nice-to-have" feature I thought of yesterday is a >> > download link that scripts can use to get archives (e.g. >> > "/download?year=x&month=y"). On the other hand, maybe this is just a >> > security risk that has no actual use case, but I'd still like to have a >> > second opinion on this. >> >> Well, there still is the authentication issue. >> >> Aurélien >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers(a)python.org >> https://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> https://mail.python.org/mailman/options/mailman-developers/dru5%40cornell.e… >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > >

1 0

Accessibility of interfaces
by Andrew Stuart 27 Mar '15

27 Mar '15

Dave You seem to have an interest in Mailman being accessible. Care to share some requirements/requests/tips/pointers for the various people who are writing user interface code that talks to Mailman. I’d like to take this sort of thing into account early in the process if possible so now is a good time to talk about what you want/need. as

1 0

Re: [Mailman-Developers] Javascript Client for Mailman
by David Andrews 27 Mar '15

27 Mar '15

At 01:51 AM 3/25/2015, Ana Badescu wrote: >Hello, > >While writing my proposal I came across 2 important issues related to the >Javascript Client for the Mailman project that have yet to be raised: > >2. I'd also like to make part of the project, a node.js application that >uses the Mailman Javascript client and offers all the functionality >Postorious does. Is that a good idea? This doesn't make the scope of the >project too large or unattainable. >Please remember that anything you develop should be accessible to >disabled persons. This is possible with JAVA generally, but means >some planning, using the right libraries etc. Dave David Andrews and long white cane Harry. E-Mail: dandrews(a)visi.com or david.andrews(a)nfbnet.org

1 0

GSoC 15 - Proposal "A Dashboard for Admins/Owners/Moderators"
by Shreyas Lakhe 26 Mar '15

26 Mar '15

Hi guys, I have updated my proposal for "A Dashboard for Admins/Owners/Moderators" project. I have tried to address the issues raised. Reviews on it are welcome. Regards, Shreyas Lakhe

1 0

Small change to REST API for held subscription requests
by Barry Warsaw 26 Mar '15

26 Mar '15

I want to make some minor backward incompatible changes to the JSON representation for pending mailing lists requests, e.g. subscription holds. You'll see these for example in urls like: >>> dump_json('http://localhost:9001/3.0/lists/ant@example.com/requests') entry 0: delivery_mode: regular display_name: Anne Person email: anne(a)example.com http_etag: "..." language: en request_id: ... type: subscription when: 2005-08-01T07:49:23 The first change is that `type: subscription` requests will change their `address` key to `email` for consistency with other parts of the API. Second, there will be no `password` key. Let me know if this is a problem, and I can copy `email` to `address` so that both would appear, and add a dummy `password` key. Cheers, -Barry

6 10

Re: [Mailman-Developers] GSoC 15 - Interested in contributing to Hyperkitty
by David Udelson 26 Mar '15

26 Mar '15

> I'm already using the "jobs" infrastructure provided by the > django-extensions package: > http://django-extensions.readthedocs.org/en/latest/jobs_scheduling.html Cool. I didn't know about this extension, but it looks like it does what we need. So the background process would be its own file in the jobs directory, and we could leave it to the admin to setup the crontab? > I have another test server with more current info if you want, but I > break it regularly. It's lists-dev.cloud.fedoraproject.org Thanks for linking this. I got my own local dev server working yesterday, but this one is much more populated. > We do put the attachment in the mbox, as a MIME component like in > every email. I see how this works now. Are the attachments always Base64 encoded? >> Another possible "nice-to-have" feature I thought of yesterday is a >> download link that scripts can use to get archives (e.g. >> "/download?year=x&month=y"). On the other hand, maybe this is just a >> security risk that has no actual use case, but I'd still like to have a >> second opinion on this. > > Well, there still is the authentication issue. I guess getting the scripts to authenticate would be a little complicated, but otherwise does this seem like something worth including? If my proposal gets accepted, I'm ok with leaving this as an open question until it becomes clear whether or not I'm going to have extra time at the end of the summer. Thanks, David On Thu, Mar 26, 2015 at 7:33 AM, Aurelien Bompard <aurelien(a)bompard.org> wrote: > > In my proposal I suggested using any of several asynchronous job queue > > libraries, such as Celery or Huey. These all use redis as a back-end. > > Because I have no experience with asynchronous job queues, I'm not sure > if > > this is too much baggage for our purposes. Maybe we just don't want the > > extra dependencies. > > Yeah, we don't want to add another database or an AMQP server just for > that. We must keep it simple for admins to deploy. > > > Regarding cron jobs, there's also django-background-task which is a > simple > > django addon that might do what we need. Again, if we don't want/need the > > extra dependency, rolling our own cron job should be fairly > > straight-forward. > > I'm already using the "jobs" infrastructure provided by the > django-extensions package: > http://django-extensions.readthedocs.org/en/latest/jobs_scheduling.html > I did consider django-background-task but django-extensions seemed > like a better fit, because django-background-task seems written for > delayed tasks, not periodic tasks (well, a task could call itself > again when done, but it seems like a hack). I'm not opposed to > switching to django-background-task if we use the "delayed job" > feature or if we need the extra flexibility of choosing exactly how > many seconds apart we want our tasks to run. > > >> If we choose to pre-build the mbox files, we can't simply have them > > served through the webserver, because some lists are private > > > > Then there is also an authentication step? > > Yeah, we must use HyperKitty's authentication and check if the user is > allowed to see the archive. So the files can't be served by the > webserver like static files. > > > I noticed on the test server > > that I can't actually look at any of the mailing lists because they're > all > > private. > > If you're looking at lists.stg.fedoraproject.org, it's currently very > outdated (still running the Python2-compatible branch of Mailman 3). I > have another test server with more current info if you want, but I > break it regularly. It's lists-dev.cloud.fedoraproject.org > > > When we create the mbox file, do we simply note that an attachment > existed > > (e.g. "Attachment: myattachment.txt") or do we actually put the > attachment > > in the mbox? AFAIK mbox is a plaintext format, so if the latter is the > case > > then I'm not exactly sure how this would work... > > We do put the attachment in the mbox, as a MIME component like in > every email. If you choose "view source" when looking at an email with > attachments, you'll see how it's done. > > > Are there going to be any issues handling unicode foreign characters or > > with file locks? Right now it looks like we should only have one process > > handling the mbox, but is it possible that more than one could be spawned > > somehow? > > No, mbox files are not designed for concurrent writes, so it's better > to have a single process write to them. > > > Another possible "nice-to-have" feature I thought of yesterday is a > > download link that scripts can use to get archives (e.g. > > "/download?year=x&month=y"). On the other hand, maybe this is just a > > security risk that has no actual use case, but I'd still like to have a > > second opinion on this. > > Well, there still is the authentication issue. > > Aurélien > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > https://mail.python.org/mailman/options/mailman-developers/dru5%40cornell.e… > > Security Policy: http://wiki.list.org/x/QIA9 >

1 0