Mailman 3 June 2014 - Mailman-Developers

before next release: disable backscatter in default installation
by Jo Rhett 07 Nov '22

07 Nov '22

Hi. There's a fairly simple problem here that needs to be addressed. And it's mostly a documentation/install problem. I'm hoping we can get this resolved before the next release. PROBLEM: Mailman comes out of the box ready to backscatter spam people. Yes, it's easy enough to fix. But because it comes stock this way, and is documented to install this way, most people install it to do this. Those of us who work in abuse departments are tired of hearing "well that's how Mailman works". We also object to having to teach people how to fix their mailman installations because it's not documented in the current manual. This is *exactly* like Sendmail 14 years ago. We didn't accept it then, and Sendmail fixed the problem. RESOLUTION: Mailman default installation should not backscatter in a default configuration. 1. Don't create backscatter aliases for subscribe/unsubscribe/etc by default. Nearly everyone uses web based signup. 2. Discard or hold messages from non-subscribers by default. I would think that it would be perfectly reasonable to have documentation on how to enable the 1980s-style -request / -subscribe etc aliases. However this documentation should have a note that this is against the AUP of nearly every network provider, and enabling it will likely cause them to get listed in various blacklists as a backscatter source. FYI: I know that this goes against the instincts of many old-time mailing list advocates here. But after dealing with a 10k/hour backscatter DoS my tolerance for this problem is understandably limited. Yes, it was a sweet day back in the 1980s. I was running a mailing list server and several UUCP gateways at the time, so I remember them well. But those days are past, and we need to deal with the reality of today. -- Jo Rhett Net Consonance : consonant endings by net philanthropy, open source and other randomness

21 168

Re: [Mailman-Developers] Disabling SELinux
by Stephen J. Turnbull 01 Jul '14

01 Jul '14

Aurelien Bompard writes: > For your information, I have written a small SELinux module for Mailman3, > it's included in my Fedora RPM and works fine. In the future (when MM3 is > out), I'll ask for integration in the core policy. "Core policy" means at SELinux? Why not put it in a contrib directory in MM3, too? Why not now?

2 1

[CLI Project] Portioning Commands as Singular or Plural
by Stephen J. Turnbull 30 Jun '14

30 Jun '14

Rajeev S writes: > The commands surely do become more intuitive and make more sense, > but this has a negative,the number of commands almost doubles. The number of instances of "class Command" or whatever it's called doesn't need to double, though. You could simply have the parser recognize the relevant plurals as equivalent to the corresponding singular forms. If you want to make life hard, you could do that but in addition in the parse constrain the command to number agreement (ie, singular forms of the command must take exactly one argument, while the plural form must take zero, two, or more arguments). You could really cheat by simply matching the shortest unambiguous prefix, so that show usersallofthemifyouplease shows all the users, and (probably) just "show u" does too. I think this is perhaps the most efficient solution, although it's not obvious that unsophisticated users would like it as much as us greybeards do. For bonus points, if the prefix is ambiguous, show the matching possibilities. A trie might be a good data structure for this, and you could build it on the fly from the list of command names (where ambiguity would of course be a fatal error). Finally, if you *do* allow abbreviations, you should restrict that usage to interactive use; scripts should not be allowed to abbreviate (they'll break if a new command makes the abbreviation ambiguous).

1 0

[CLI Project] Portioning Commands as Singular or Plural
by Rajeev S 30 Jun '14

30 Jun '14

Hi, A modification has been suggested for the Command line tools in an IRC discussion with Abhilash. It was suggested that, to make commands more intuitive, the commands be partitioned into singular and plural as per the scale of the action performed. For example, In the current scenario >>>./mmclient show user //Lists all users in the system >>> ./mmclient show user foo(a)bar.com //Describes the user foo(a)bar.com Usage after the suggested change >>>./mmclient show users //Lists all users in the system >>>./mmclient show user foo(a)bar.com //Describes user foo(a)bar.com The commands surely do become more intuitive and make more sense, but this has a negative,the number of commands almost doubles. Further, Use cases like delete, subscribe and unsubscribe support actions on multiple users simultaneously. Currently, the following commands are valid >>>./mmclient delete user a(a)b.com b(a)c.com d(a)e.com >>>./mmclient delete user a(a)b.com If the suggested change is applied, the following commands must work >>>./mmclient delete users a(a)b.com b(a)c.com d(a)e.com >>>./mmclient delete user a(a)b.com both of which perform the same action, using the same procedure. Another option might be to use *only plurals*, which seems OK for most of the use cases.

1 0

[CLI Project] Support of filtering of objects in Command Shell
by Rajeev S 29 Jun '14

29 Jun '14

Hi, Now the mid term evaluations are done, I would be building the command line shell, as planned. I would like to present how I would like to tackle the requirement at hand. As mentioned in my proposal and the most logical approach, all the common functionalities between the shell and the already built command line tools would use the same classes, the ones built for the command tools. As this is a command shell, I wish to give it a SQL like feel. However, The SQL commands SELECT, INSERT INTO do not look nice in Mailman context, as we are dealing with objects and not tables. I would be using the object related terms like SHOW, CREATE etc for the respective actions. However, I would like to discuss about the support for the WHERE clause by the shell. The WHERE clause is not mentioned in my proposal, but I feel that this feature ought to be in the project. If the commands require to support a WHERE clause, as SQL does, For example, >>> show users where display_name = 'foo' and subscribed_list_ids contains = 'list(a)domain.org <mailto:%27list@domain.org>' The steps involved in processing a command with a WHERE clause would be 1.Parse the command 2.Filter the results from the database 3.Perform the action using the command line tool methods if possible else write a new procedure. For the Step 2, there are two approaches in hand: 1.Use the Storm library to query the database directly, as mailman core does. 2.Use the REST API By using the first approach, a good performance improvement is achieved, by leaving the data filtering to database engine. But this limits the usability of of the CLI to the system in which the mailman database exists, as the remote connections to DB servers are usually disabled. By using the second approach, the performance is bad, as it requires a normal python `for` loop that compares the the results one by one. However, this approach might be easier in case that the mailman installation be managed remotely, if its OK to enable the REST API to accept connections from remote machines.(I believe this is currently not supported/disabled by default.) By building the WHERE clause, its costlier to reuse the command line tools methods in many use cases, like delete, subscribe etc. Further, as I have already mentioned to Abhilash, it demands more time. I am not sure if that will fit into GSoC time frame. I am confident that I can make significant progress, but not sure if it can be 'completed'. Also, I am fine on completing it even after GSoC. So, the final question is, Should the CLI support filtering using the WHERE clause? Or should I proceed as per my proposal, building a basic Mailman shell and leave the rest to be completed in future?

3 2

GSoC updates till midterm
by varun sharma 24 Jun '14

24 Jun '14

Hi everyone, This will be my second update email to mm-dev after gsoc started, i'll be sending weekly summaries in the future. The ci-tool(please suggest some nice name) is almost finished, i need few more days to get it working on multiple projects under mailman suite. The following parts of ci-tool are working: ./ci-tool --pull project_name ./ci-tool --pull all ./ci-tool --test project_name ./ci-tool --test all ./ci-tool --try project_name The projects under the mailman suite are listed in a config file(currently settings.py) which contains the a dict projects which has further dictionaries to store attributes like path, vcs, src location. For parsing the commands form the command line, i have used argparse module and while implementation of commands, i have created two different files: commands.py and utils.py. In commands, i've declared the classes for various commands like pull, test and try and for making a tcp connection via twisted, two more classes for Client and Factory. In utils.py, i've tried to declare the error classes which i'll be able to differentiate easily and a logging class. The most interesting command in this whole tool is the 'try' command, which will take the diff of the project and send it to the server to test it on the server and run all the tests on different installations (in this case i am using buildbot's slaves). I am able to send the diff to server and test it on a single server and now working on how to send the patches for multiple projects of suite, in case the developer is working on multiple projects(like postorius and hyperkitty) at the same time. That's all for the status report, i have to say that whatever i've done won't be possible without the help of my awesome mentors: florian, stylistica, steve and maxking. The source of my project is available at http://bazaar.launchpad.net/~varun/mailman/ci_tool/files. I try to remain active on #mailman and #mailman-gsoc for most of the daytime under nick 'varun'. Thanks Varun

1 0

[CLI Project] Preference management commands added
by Rajeev S 22 Jun '14

22 Jun '14

Hi, As promised, I have completed the preference related commands and pushed it as revision 60 [1]. The preferences form a new scope, along with user, domains and lists. There are 2 commands associated with the `preference` scope. 1.Show Preference ./mmclient show preference [scope] [scope_args] [key] That is, to view a preference of a list member ./mmclient show preference member --email a(a)b.com --list list(a)domain.org receive_own_postings 2. Update Preference Similar to the above command ./mmclient show preference [scope] [scope_args] [key] [value] ./mmclient show preference member --email a(a)b.com --list list(a)domain.org receive_own_postings true The supported preference scopes are global [2] user --email address --email member --email --list Also, the command scope resolution code in cmdparser.py, which was repeated once per scope, has been replaced with a better and faster approach. Footnotes: [1] http://bazaar.launchpad.net/~rajeevs1992/mailman.client/mailmancli/revision… [2] Upon updating/editing the global preferences, a `HTTP Error 405 : Method not allowed is raised`. I tried as sudo but still no good. Is that a mailman bug or a design decision?

1 0

Disabling SELinux
by Stephen J. Turnbull 19 Jun '14

19 Jun '14

I thought I sent this but my MUA disagrees.... A user (one among many) writes: > Mark, > Good straw to grasp at. I thought SElinux had already been disabled but it > hadn't. It seems to be working now. Do we have better advice than just "disable SELinux" to offer? I ask because "solving" the DMARC "problem" seems to involve trust of Mediators (the technical RFC term for middlemen like Mailman) by Author Domains (RFC-ese for domains with DMARC policies about verifying From:). Asking upstream to trust sites which have deliberately disabled security features seems like poor strategy at best. Note: reply-to set to Mailman-Developers. Please follow up there. If you want to follow but not subscribe to MM-D, perhap the archives are an option: http://www.mail-archive.com/mailman-developers%40python.org/. Steve

3 2

[CLI Project]Commands for adding Moderators and owners
by Rajeev S 19 Jun '14

19 Jun '14

Hi, I have pushed r59 (In fact, 2 days before) which adds commands to perform the following actions - Add moderators ./mmclient add-moderator list list(a)domain.org --user a(a)b.com b(a)c.com d(a)e.com - Add owners ./mmclient add-owner list list(a)domain.org --user a(a)b.com b(a)c.com d(a)e.com - Remove Moderator ./mmclient remove-moderator list list(a)domain.org --user a(a)b.com b(a)c.com d(a)e.com - Remove Owner ./mmclient remove-owner list list(a)domain.org --user a(a)b.com b(a)c.com d(a)e.com All the above commands support an extra --quiet flag to suppress the action feedback message. http://bazaar.launchpad.net/~rajeevs1992/mailman.client/mailmancli/revision…

1 0

Two more DMARC mitigations
by John Levine 18 Jun '14

18 Jun '14

* Forwarding signature The IETF DMARC list is discussing a mutant weak DKIM signature from a sending system (e.g. Yahoo and AOL) that would survive forwarding, but contains a list of forwarding target domains. It's only considered valid if it's with a signature from the forwarding domain, i.e., the list. This is nice for list operators, since it requires nothing beyond not stripping the signature header, and signing on the way out. * Submit and sign When a user at a p=reject signs up for a list, you demand an OAUTH API token if the the provider supports it, otherwise their host system password. You can check it on the spot and skip the challenge email to confirm opt-in. When the user sends mail to the list, the list makes whatever changes it's going to make (subject tag, footers, whatever) and then uses the API or SUBMIT to resend it through the host system. When it arrives back at the list, it has the host system's DKIM signature and the list can resend it to the subscribers with the valid signature. It also has to checks DMARC on incoming messages, and if the policy has changed, holds onto the message and send a notice saying go back to the web site and give us your credentials to stay on the list. This is less nice, it's a lot of software development. Collecting credentials gives the mail operators heartburn, but as I told them at a meeting today, if you want all mail to be authenticated, that's what you asked for. There is definite interest at large mail providers at least for the first one. Dunno if they're interested in the second, but since it uses features they already have, it matters less. R's, John

7 23