Mailman 3 March 2016 - Mailman-Developers

Re: [Mailman-Developers] Interested in the GSoC idea 'GitLab/development tools integration'
by Simon Hanna March 11, 2016

March 11, 2016

On 03/11/2016 10:54 AM, Wasim Thabraze wrote: > Hi Abhilash, > > So the tool can be anything like web based thing, a module or a plugin?! Am > I thinking wrong? > > Before I can propose a way, I need your help in knowing more about the > project. > > 1. Can anyone use it or only the admin? I guess a good choice would be moderators or owners. > 2. The input to the tool is a thread from a mailing list. But how does the > tool actually take a thread from a mailing list? Hyperkitty? I guess Hyperkitty is a good place to put it in. The core functionality will (should) probably be reusable anyway. > 3. Are there any such tools in the software world that are more or less > doing similar work? (This query would help me in getting a clear cut idea > of the project.) You will have to use gitlab/hub apis. There should be python libraries to do that. There are some projects that create github issues out of email. Not sure if they are open source though... > > > Can you please give more information about the query #2?! Or can point me > to any links that would help me in getting a brief idea. In the end it really depends on what you want to do...

1 0

Re: [Mailman-Developers] Interested in the GSoC idea 'GitLab/development tools integration'
by Abhilash Raj March 11, 2016

March 11, 2016

Hi Wasim, On 03/10/2016 03:09 AM, Wasim Thabraze wrote: > Hello everyone, > > I am Wasim Thabraze, a Computer Science Undergraduate. I have thoroughly > gone through the GSoC ideas page and have narrowed down my choices to the > project 'GitLab/development tools integration'. > > I have experience with GitLab, GitHub and also used their API's to build > stuff. > > I have gone through the idea and got a rough idea about how the final > outcome of the tool should be. > > I feel glad if someone can help me in getting started with this project. If you can outline how and what would the integration exactly mean, I (or anyone else) would be able to comment and help you more. You can layout rough ideas here too, doesn't have be anything concrete right now. Apart from this channel, You can also chat with the mentors on IRC, we are on and off IRC throughout the day on #mailman. Also, please note that you need to solve atleast one issue as a part of the application process for GSoC. -- thanks, Abhilash Raj

2 1

Re: [Mailman-Developers] List Passwords via Mailman REST api
by Barry Warsaw March 10, 2016

March 10, 2016

On Mar 10, 2016, at 12:54 PM, Dominic Dambrogia wrote: >If you can point me towards how to set passwords via the REST api or a hint >of why I'm failing to moderate emails I would truthfully appreciate it. Hi Dominic, Currently the moderator_password can't be set via REST. You would have to do it from the mailman shell. It's probably an easy addition, and I wouldn't be opposed, but there's the open question as to whether the moderator password is a good idea in the long term. But, since that's the way it currently works, it probably makes sense to allow setting it via REST. Cheers, -Barry

1 0

List Passwords via Mailman REST api
by Dominic Dambrogia March 10, 2016

March 10, 2016

Hi, my name is Dominic. I'm on the final steps of finishing my Mailman mailing list server. The last problem I'm having is moderating emails. Everything for mailman on my application is written in a PHP wrapper class built around your REST api. Currently, we have no passwords set for lists, so when we reply to our moderation requests for nonmembers posting to mailing lists with "Approved: " or "Approved:", our moderation actions are failing. I've been living in your pythonhosted documentation as of recently and have yet to find anything about setting passwords for lists via the REST api. If you can point me towards how to set passwords via the REST api or a hint of why I'm failing to moderate emails I would truthfully appreciate it. Thank you,

1 0

Basic workflow of the ARC implementation
by Stephen J. Turnbull March 10, 2016

March 10, 2016

Aditya Divekar writes: > We need to generate a private and a public key for the signing purposes. > For testing purposes, and while working on the code, I can probably > generate the keys locally using the openssl tool. In production, these keys are a *site* resource. I see no reason why we need to generate keys automatically in this case, and in fact many sites will use their DKIM keys AIUI. > As a rough sketch of implementation, > > 1. ARC Seal > > The tags - > > NOTE - Now, here we needed to check the given message for any > pre-existing signatures in most of the fields. For this I think a > separate module can be created which can extract the previous ARC > headers if they exist from the message. The code for this can be > again used from the dkimpy package. This should be trivial to do with the Python email package, too. I don't really see that a separate module would be useful, since we'll want to extract a fixed set of headers (ARC- and DKIM-specified). Of course it should be factored into a separate function (or perhaps a generic "extract_fields" function and a couple of derivatives with the DKIM list (just DKIM-Signature?) and the ARC list (ARC-Seal, ARC-Authentication-Results, and ARC-Message-Signature). GSoC meta: You don't need to go as far as deciding the factoring of this function in your proposal, but on the other hand it wouldn't hurt. It's low importance (see below for high-priority tasks). Another factoring issue: should you "import dkimpy" and call dkimpy.foo, or should you "from dkimpy import foo, bar, baz"? (Doesn't need to be settled for a while, and you can even try both at a small cost in time and effort.) > i: The value for this tag can be determined by performing a check > on the original signature and seeing if there were previous ARC > headers. If yes, we increment the value of the previous "i" by 1, > and if no, make it 1. Note that the I-D (Internet-Draft) provides a specific algorithm for canonicalizing i in the presence of missing i or duplicate i. > a: The value for this is fixed. > > t,s,d: These can be obtained by using pre-existing dkimpy package. > > b: Now we can compute the header hash using the dkimpy package > again by using the headers given here > <https://tools.ietf.org/html/draft-andersen-arc-02#section-5.1.1.3>. > Here, we call the dkimpy package and get the signature for the above > headers and then affix it to the "b="header. > > cv: Use the same check as "i", if there is already an ARC i.e. i>1, then we > make it as "V", else "N". OK. > The ARC Seal gets computed here. > > NOTE-For giving the "s" and "d" (selector and domain tag values), > we will need to produce records for these where the key can be > stored so that it is available for query by the verifiers (I still > have to look up this mechanism). OK. GSoC meta: you can get away without actually specifying the mechanism in your proposal, but you should be able to say where you will look it up when the time comes. (ARC Seal will not necessarily be your first milestone, and you can book up on standards "just in time".) Knowing where to look is medium priority IMO. (IMO means "other mentors may feel differently, pay more attention to comments they make than the ones I make"!) > 2. ARC Message Signature > > The tags - > > i: The value for this tag is determined similar to the "i" tag for > the ARC Seal. Isn't this inaccurate? This "i" must *match* the Seal "i", no? > a,t,s,d: These can again be obtained from the dkimpy package. > > bh: The body hash. This can be obtained from the package. Here, we set the > canonicalization to 'relaxed' and get the body hash. "The package" == dkimpy, right? > h: The "h=" header list is signed with the implicit list (as given > in the draft) and any explicit list that we want in addition. > Now, for mailing lists, the recommended headers are - > List-Id:List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:Reply-To: > and any other fields added by the list like the Precedence, XTopics or > else. > We will also sign the DKIM signature of the previous mail here if > available(suggested by the draft). > For this signing, we can use a modified version where the > FROZEN_HEADERS (headers that are signed by default) will specify > the implicit headers according to ARC specs (Another option could > be to get all the implicit+explicit headers signed by the package, > extract the h header, and modify it to include only the explicit > headers.) > b: The signature, calculated from the package. The h tag is used as > described above. > > > 3. ARC Authentication Results. > The "i" tag simply takes on the value same as the above "i" tags. > Now from our previous conversation, as you suggested, the > authenticity of the previous MTA who sent us the mail is not sure > to be trusted. So in the case where we don't trust the previous > MTA, we will have to perform our own DMARC, SPF, DKIM testing of > the recieved mail. If previous ARC chain exists i.e. cv=V, then we > perform the ARC test too. I'm not sure you understood me. We *always* verify the preceding MTA's claims, even if we trust them, because of spoofing and man-in-the-middle attacks on the Internet itself. (More precisely, verification is required by the I-D, and the rationale is spoofing and MITM.) The MTA we "may or may not trust" is our *own* MTA. > Now for performing the tests - > In one of the earlier mails, we discussed the use of the "authres" > package for generating the authentication results header. The > package conforms to the RFC7001 format, and now the format used is > RFC7601. But according to the changes that I verified, we can use > the package without any changes. (The changes were mostly related > to extra specifications that are optional. > Can be skipped for our purposes) > So the "authres" package can be called here for generating the AAR. Great! > If we need to perform the ARC test, then the module for that will > have to be implemented manually. Though most of the code from the > package for DKIM verification can be used. Yes. > This is also the point where we detect if the mail is spam or not. No. ARC modifies the message, and therefore is a Handler, which does not make decisions about disposition. We provide our authentication results to later Rules, probably by adding a field (or several) to msg_data. (This is a Mailman design policy, for detailed rationale ask Barry. The basic point is that passing ARC or even DMARC certainly does not mean the message is not spam, and vice versa, even if failure is a strong indication.) > If the arc test fails, then there is something fishy here. DKIM, > DMARC, SPF may fail, but the failure of this test means the mail is > not authentic. At this point the mail should probably be discarded > (or any other measures that need to be taken). No. The mail cannot be proved authentic by ARC, but that may be due to changes to the message at intervening hops. There may be more sophisticated tests (eg, a PGP signature on a MIME body) that can prove it authentic. > Now coming to the testing part. There can be a number of tests like > verifying the generated ARC signature, changing the body of the > message, failing when the implicitly signed AMS headers are changed > and other such tests. This is a little vague, but testing is hard. You'll learn it as you go along. > Is this the workflow you were expecting to see, or should I write a > more explanatory draft ? What you have written so far is basically OK as far as the *work*flow part goes. However, you also need a *schedule* with *milestones*. The *schedule* should present the Google deadlines, any times that you will be out of communication (eg, because of travel), and any periods of more than two days that you expect to be unable to work. Google expect you to treat the internship as a fulltime job, so "working time" is like 40 hours a week. But you should not expect to be able to make up two days or more that you would normally be working by working times that you normally don't. A few people can do that, most can't. Therefore, schedule as if you can't until you've proven you can. Also, many people can successfully work 4 days x 10 hours, but more than 10 hours/day and you're pushing human limits. So you should be thinking in terms of a 5 day work week, or maybe 4 if you have really good reason or previous experience. Finally, your schedule should include your milestones. A *milestone* is more than just a subgoal for your project. It needs to be objectively verifiable. "ARC Seal code completed" is not a milestone. "ARC Seal code completed, unit tests pass, merge request posted to GitLab" is a milestone. A better one would say "unit tests *pass with 100% branch coverage*." Note that all of these are things you check yourself and don't depend on others. You can include things like "review passed" and "code merged", but these depend on others so you should leave lots of room for delay. "Code completed" is not a milestone because it doesn't specify the quality of the code. I count about 9 well-defined tasks in your message that could be made into good milestones. You should try to come up with your own list, but if you're really not happy with your list, send it to me and ask for help and I'll give you my ideas. In an ideal world with experienced GSoC students, we would like to see approximately one milestone per week in your schedule. More than that and it becomes a theoretical exercise because you always end up finding out that the right order to do the subtasks is different from the one you scheduled. Schedules are only useful if there's a reasonable expectation you'll keep to them. On the other hand you should have more than two, because there should be one milestone describing what you'll be reviewed on for the midterm, and another for the final. The point of having a schedule is so you'll know how good your plan was and whether you'll be able to keep your promises. Keep in mind that if you get in schedule trouble, you have four options: (1) work harder, (2) get help, (3) renegotiate your commitments for midterm and final, and (4) get lucky. Don't depend on #4 -- that's how students fail. The amount of help you can get and renegotiation you can do are limited, of course, but when #1 looks likely to fail, #2 and #3 may be available. > (Also wherever I have mentioned the use of dkimpy, a lot of custom > implementation is needed to suit our requirements. ) You should say that when you first mention dkimpy. In general this is expected, though, so not terribly important.

2 1

Re: [Mailman-Developers] Help regarding an error during installation
by Simon Hanna March 10, 2016

March 10, 2016

On 03/10/2016 04:18 PM, Daman Singh wrote: > Hello everyone! > > I am trying to install mailman for development using mailman-bundler. I > followed the instructions given here > http://mailman-bundler.readthedocs.org/en/latest/ , but i am facing a > problem. > When i run buildout command i get an error saying,"no module named 'zlib'" > but on running ipython, when i import zlib it works all right. > I am using a virtualenvironment. Basically module named zlib is already > installed but "buildout" command is still giving me error. > Anyone having its fix kindly provide me its fix I'm unaware of any dependency on zlib. If you want to install mailman for development please refer to this wiki page http://wiki.list.org/DEV/SetupDevEnvironment

1 0

Help regarding an error during installation
by Daman Singh March 10, 2016

March 10, 2016

Hello everyone! I am trying to install mailman for development using mailman-bundler. I followed the instructions given here http://mailman-bundler.readthedocs.org/en/latest/ , but i am facing a problem. When i run buildout command i get an error saying,"no module named 'zlib'" but on running ipython, when i import zlib it works all right. I am using a virtualenvironment. Basically module named zlib is already installed but "buildout" command is still giving me error. Anyone having its fix kindly provide me its fix Best Daman

1 0

Interested in the GSoC idea 'GitLab/development tools integration'
by Wasim Thabraze March 10, 2016

March 10, 2016

Hello everyone, I am Wasim Thabraze, a Computer Science Undergraduate. I have thoroughly gone through the GSoC ideas page and have narrowed down my choices to the project 'GitLab/development tools integration'. I have experience with GitLab, GitHub and also used their API's to build stuff. I have gone through the idea and got a rough idea about how the final outcome of the tool should be. I feel glad if someone can help me in getting started with this project. I hope I can code with Mailman this summer, and I really appreciate your comments. Regards, Wasim https://github.com/waseem18 http://www.thabraze.me

1 0

Re: [Mailman-Developers] Discussion On Project Idea "Preset List Settings Templates" .
by Stephen J. Turnbull March 9, 2016

March 9, 2016

Harshit Bansal writes: > I think the "Permissions Systems" would have nothing to do with the > core. It would be related to Postorius. We will have to create a style > model separately in Postorius which would store the style name and the > user who created it. Then only the user who has created the style > would be granted the permission to edit it. Then there is no *permissions* system! For example, one project last year created a Javascript client -- that would completely bypass the "permissions" system as you describe it. You could imagine that style changes are a "friendly users" feature, and so the "style owner" system would be a *safety* feature of the Postorius UI rather than an *authorization* feature of styles. But in an enterprise context (eg, a virtual hosting service), I'm sure that users will think of it as an authorization system. While at present it seems unlikely that there would be multiple interfaces on one hosting service, you never know what users will do[1]. Also, it would not be obvious to somebody who installed the node.js Mailman client that they are likely bypassing "security" as documented in the typical Mailman manuals and tutorials that you would find on the web. > From UI point of view, there is one more feature that comes to my mind: > When the users are presented with an interface to customize the style, > we could provide them with two views, one 'simple' and another > 'advanced'. The 'simple' one would show only the *frequently* (or most > commonly) changed attributes while the 'advanced' one will show all > the attributes. This will help them in customizing the styles. > I would like to hear some of your comments on it as well. Doing this well is a hard problem. It seems to me that it is likely that the "frequently changed attributes" will differ depending on the base style. Also, there's a "self-fulfilling prophecy" aspect: the attributes that don't make it to the "simple" view will be less discoverable, and of course they will be less frequently tweaked. How about a "pluggable" page? Ie, where site or domain admins can set a list of items, which can be either a string (explanatory comment) or a widget (a UI object permitting customization of one attribute or perhaps a container for customizing several related attributes). I suspect you'll find that the current pages are already factored so that it won't be too hard to accomplish this. Of course we would provide some initial lists that we think are good. I think the time has come to think about adding feedback on our UI to the UI. Of course we won't do it behind the admin's back, nor default it on, and we would allow admins to configure the information to be sent. But I think our user base would be pretty enthusiastic about contributing things like customized UIs and list styles. Footnotes: [1] The Harvard Law: Under the most rigorously controlled conditions of pressure, temperature, volume, humidity, and other variables, the organism will do as it damn well pleases. http://scienceblogs.com/clock/2007/05/15/and-now-the-scientists-will-do/ I think I've seen this attached to a person's name, and it would have been in the early '70s, so predating the Usenet post.

1 0

Guides for contributing to mailman
by Simon Hanna March 9, 2016

March 9, 2016

Hi, I took the liberty of updating a couple of pages on the wiki. Noticeable additions are the following two pages: http://wiki.list.org/DEV/HowToContributeGit which is intended to be a howto regarding git. It should cover the basics by giving example executions of commands. Real documentation and tutorials can be found easily on the web. This is also the place for code of conduct regarding gitlab commits/issues/merge requests. http://wiki.list.org/DEV/SetupDevEnvironment This are brief setup instructions for the mailman packages. It is not supposed to be complete or explain anything in details. readthedocs.org is a much better place for such information. I kindly request you to read through it and help improve it. I placed links on the top of the dev homepage http://wiki.list.org/DEV/Home It would be helpful if all "newcomers" are directed to these two pages when they ask questions about setting up. Feel free to add to the pages when you think more information is necessary of if people turn out to need more help with how to do stuff in git/gitlab Simon

2 1