Mailman 3 February 2015 - Mailman-Developers

Removal of REST API method /lists/{list_identifier}/config/{sub_resource}
by Andrew Stuart 18 Feb '15

18 Feb '15

####################################################################### exhibit 1: The REST API provides GET /lists/{list_identifier}/config PUT /lists/{list_identifier}/config PATCH /lists/{list_identifier}/config This allows reading of all list config variables (see bottom of this message for which ones). It also allows for setting of any individual config variable. ####################################################################### exhibit 2: The REST API also provides GET /lists/{list_identifier}/config/{config_variable} PATCH /lists/{list_identifier}/config/{config_variable} Which allows any individual config variable to be read and modified. ####################################################################### Effectively exhibit2 completely duplicates exhibit1 for no substantial gain. exhibit2 significantly increases the complexity of the overall system due to its fine grained duplication of everything in exhibit1. Right now I’m facing write a bunch of code to handle the fine grained config variable setting, all the while thinking “but this is all duplicate functionality”. I’m wondering if anyone would miss exhibit 2 if it was deleted from the REST API, saving on testing and complexity but losing nothing. Have I missed something? MAYBE its possible that different config items might have different user access requirements, justifying fine grained control, however I can’t see how right now. Thoughts? ... dict( ... acceptable_aliases=['one(a)example.com', 'two(a)example.com'], ... admin_immed_notify=False, ... admin_notify_mchanges=True, ... administrivia=False, ... advertised=False, ... anonymous_list=True, ... archive_policy='never', ... autorespond_owner='respond_and_discard', ... autorespond_postings='respond_and_continue', ... autorespond_requests='respond_and_discard', ... autoresponse_grace_period='45d', ... autoresponse_owner_text='the owner', ... autoresponse_postings_text='the mailing list', ... autoresponse_request_text='the robot', ... display_name='Fnords', ... description='This is my mailing list', ... include_rfc2369_headers=False, ... allow_list_posts=False, ... digest_size_threshold=10.5, ... posting_pipeline='virgin', ... filter_content=True, ... first_strip_reply_to=True, ... convert_html_to_plaintext=True, ... collapse_alternatives=False, ... reply_goes_to_list='point_to_list', ... reply_to_address='bee(a)example.com', ... send_welcome_message=False, ... subject_prefix='[ant]', ... welcome_message_uri='mailman:///welcome.txt', ... default_member_action='hold', ... default_nonmember_action='discard', ... )

1 1

REST API PUT versus PATCH - is there in any point in PUT and PATCH?
by Andrew Stuart 18 Feb '15

18 Feb '15

There are several cases in the REST API where there is a PUT method that requires all fields to be present, and also a PATCH method which is the same except that not all fields are required. Is there any actual benefit in those PUT methods? I can’t see what they offer over the PATCH methods. Maybe its worth thinking about removing all PUT methods that don’t offer any substantial gain over the PATCH methods, reducing need for testing, documentation and complexity. Here are the PUT and PATCH methods in the REST API: AddressesVARUserPUT ListsVARArchiversPATCH ListsVARArchiversPUT ListsVARConfigPATCH ListsVARConfigPUT ListsVARConfigVARPATCH MembersVARPATCH MembersVARPreferencesPATCH MembersVARPreferencesPUT UsersVARPATCH UsersVARPUT UsersVARPreferencesPATCH UsersVARPreferencesPUT as

2 2

Re: [Mailman-Developers] [GSoC14] Full Anonymization Project Idea
by Rashi Karanpuria 18 Feb '15

18 Feb '15

Hi I am a prospective Gsoc'15 applicant. I found the Anonymous mailing list idea for project interesting. While researching about the same I found this thread. I have a queries regarding this : Can't we keep our encryption application away from the site admins by creating an interface between sender and the switchboard ,what I'm trying to imply here is, can't we process the encryption before the mail goes to switchboard or server for further processing. Let say,the email sent by the sender gets into the interface first where it strips out the message header and encrypts the email address, so when the email goes to the switchboard it is already modified, does not contain original sender's address but has the details about where it is to be sent. Now when the receiver sends the reply then the email gets to the list moderator and after that it passes through the same interface which now decrypts it to the original address and forwards it after encrypting the sender's address. Stephen J. Turnbull writes: > In general, what is missing from these "anonymization" proposals are > use cases, user stories which display the reasons for anonymity and the > definition of anonymity (for example, should repeated posts from a > given subscriber have the same "From" field or not?) For example, the > following organizations want a "fully anonymized" list: > > 1. An Alcoholics Anonymous meeting. > 2. A therapy group for battered wives led by a professional therapist > (whose identity is known to all, and who knows all realspace > identities -- but maybe can't match them to list identities). > 3. A corporate whistleblower/suggestion box. > 4. A terrorist cell. (I'm not suggesting we should *care* about > serving these people well, and maybe we should try *not* to serve > them -- it's an intellectual exercise.) > 5. A tax evaders users' group. (ditto) > > How do their needs differ? How are they similar? How well does your > proposal serve their needs? I'm not too serious about that specific > list, individual students may or may not have experience and knowledge > of those use cases. But I would strongly prefer to work with a > student who thinks about these issues *explicitly* and *concretely* in > terms of use cases. In particular, it may seem obvious that we can't > protect the subscriber database against the site admin/root, but then > we have to give up on use case 3 above. Or do we? > > These are hard, *hard*, HARD questions. Even Bruce Schneier (if you > don't know who he is, find out!) might not get the answers at first > try. I don't ask you to get the hard ones at all! But sometimes the > answer is obvious from just asking the question (use case 3 vs root > access), so you'd best ask some of those easy ones. :-) > > About this specific proposal: > > Kshitij Gupta writes: > > > As I understand, we can do this in the following ways: > > 1. For each subscriber on the mailing list generate a random encryption > and > > decryption key, which will be store in the database. > > If the keys never leave the database, why not use symmetric > encryption? Why do different subscribers need different keys? > > If they do leave the database, how are they distributed? How is the > distribution protected from the standard attacks (eg, man in the > middle)? > > > 2. Everytime user sends the mail we can encrypt the email to a hash > which > > will then be used as the pseudo id for the user. To do this we can > either > > use salt (as in time) to ensure a new email id is generated everytime > > I don't understand why you would ever want this, let alone why there > is a use case common enough to be worth implementing in Mailman. > > That doesn't mean there isn't any, but please explain. > > > or without salt which equivalently fixes a single id for the user. > > 3. From the email we can cleanup headers, converting the users timezone > to > > a standard UTC timezone. > > You also probably need to handle Message-ID specially. > > > 4. We can also hash the users original email id and append it as a > > signature to sign the mail, ensuring the authenticity of mail in a > > conversation. > > That's not how digital signatures are done, and only those with access > to a descryption key can check authenticity. > > > 5. For replies, the person replying can respond to the message, the > email > > address will then be decrypted by matching against the list of all > > decryption keys and matching the digest of the mail id for additional > > security and forwarding it to the intended user. > > I'm not sure I understand the "additional security part." In any > case, if you have a "digest", why not use that as a unique key into > the user database, so that the actual decryption becomes an > authentication, and only needs to be done once? > > > The above steps (in my understanding): > > 1. Will allow users to anonymously post to mailing lists. > > Except that the site admin knows where to find each user. The site > admin had better be the only entity with such access. > > > 2. Ensure nobody can pretend to be someone else in a thread via the > > personal salt. > > But how about spoofing subscriptions? Do we care about that? What if > a user happens to know the address of another user, and spoofs that? > > > 3. Allow users to communicate in threads and reply to each other. > > 4. Use a constant space for users in the database, at the cost of > matching > > against multiple decryption keys and then checking against the hashed > email. > > Is constant space really an issue? > > > Look forward to some feedback and hope to contribute to the mailman > > community. > > > > [1]- https://code.launchpad.net/apparmor-profile-tools , however the > code > > was recently merged into the branch upstream at: > > https://code.launchpad.net/apparmor where development continues. > > > > Regards, > > Kshitij Gupta > > _______________________________________________ > > Mailman-Developers mailing list > > Mailman-Developers(a)python.org > > https://mail.python.org/mailman/listinfo/mailman-developers > > Mailman FAQ: http://wiki.list.org/x/AgA3 > > Searchable Archives: > > http://www.mail-archive.com/mailman-developers%40python.org/ > > Unsubscribe: > > https://mail.python.org/mailman/options/mailman-developers/stephen%40xemacs… > > > > Security Policy: http://wiki.list.org/x/QIA9 > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org > > Security Policy: http://wiki.list.org/x/QIA9 >

2 3

Building plug-ins for mailman.
by Aanand Shekhar Roy 18 Feb '15

18 Feb '15

Thanks everyone for your invaluable suggestions,I was also thinking about having features as plug-ins like 1.As discussed with @Rashi007-Timed delay or timed delivery of messages - Suppose I wish to post a message to list but instead of composing and sending I prefer to send it delayed like I wish to post a message at 4am but since i would be sleeping at that time so what I can do,I will post my message at 10pm at night and a delaying or timed delivery plugin can deliver it at 4am. 2.Limited post by specific members - Being the list owner, I wish to restrict the number of posts one can do on the mailing list.Like a specific set of members can post 1 post per day,another set of members can do 10 posts per day. 3.Gmail has a plug-in system that allows listing of all your email subscriptions and unsubscribe with one-click from every list you dont read anymore...that will save much time unsubscribing to a lot of email lists and newsletters that you never read.Can we have one for mailman? Based on the discussions held previously on this topic,few ideas that have been found suitable are: 1.Timed vacations, allowing a user to postpone or discard email for a certain number of days or weeks. 2.Asking for confirmation if a member's post include confidential information.

2 1

Building plug-ins for mailman.
by Aanand Shekhar Roy 18 Feb '15

18 Feb '15

Hi all, I am currently finding good plugin ideas that might make there way into mailman 3 system.I had discussed the same with @terri on irc.Some ideas i have already found good are: 1. DNS BlockList 2. Confirmation from user before posting phone nos in emails 3. SpamAsassin as it is I wish to ask you all about other problems that mailman users face that may be solved by a plugin application or any other ideas you wish to see in the form of plugins .Any suggestions would be of great help. Thanks in advance

5 7

Re: [Mailman-Developers] authentication, authorization and permissions FYI
by Andrew Stuart 18 Feb '15

18 Feb '15

Sorry I meant SAML looks similar to JWT but XML based. On 18 Feb 2015, at 10:45 am, Dick Visser <visser(a)terena.org> wrote: sorry meant to sent to list, just did that now On Wed, Feb 18, 2015 at 12:42 AM, Andrew Stuart <andrew.stuart(a)supercoders.com.au> wrote: > Hi Dick, > > If you re-post as a reply to the mailing list I can send a reply there. > > thanks > > as > > > On 18 Feb 2015, at 10:40 am, Dick Visser <visser(a)terena.org> wrote: > > Hi Andrew > > Is this about the MM3 web interface? > If so, I wonder if it could work with SAML authentication. > This can be done with PySAML2 (https://github.com/rohe/pysaml2), or in > an Apache module (mod_mellon). > The latter takes care of the authentication, and any web app protected > by it an rely on (for instance) REMOTE_USER always being available. > > Thanks > > Dick > > > On Sun, Feb 15, 2015 at 9:05 AM, Andrew Stuart > <andrew.stuart(a)supercoders.com.au> wrote: >> I’ve just finished (mostly) implementing most of the authentication, authorization and permissions systems. Thought you might be interested in a little on how it works. >> >> To discuss this further I need to explain a few things - apologies to those who i am teaching to suck eggs. >> >> The auth server needs a few key subsystems to get the job done - authorization, authentication and permissions. >> >> AUTHENTICATION - this is the process of the end user identifying themselves in the first place (in this case, by logging in with a username and password), then for every request beyond that, providing evidence that they are who they say they are - in this case, that’s done by giving them a token when they log in successfully, then requiring that the token be returned along with each subsequent request. >> >> AUTHORIZATION - this is the process of determining if the authorized user is allowed - according to the rules of our application - to access the resource that they have requested. For example, user Jenny sends a request to access the members of a list named supermaillinglist(a)marketingcompany.com. Before we can execute that request we need to determine if she is *authorized* to do so. >> >> PERMISSIONS - for the authorization process to do its job, it needs information about WHO is allowed WHAT LEVEL OF ACCESS to WHICH RESOURCE. This is the *permissions system*. It’s VERY easy to understand. Nothing more to it than a database table that looks like this >> user_id = Column(String, nullable=False) >> role = Column(String, nullable=False) >> resource_id = Column(String, nullable=False) >> resource_type = Column(Enum('list', 'domain', 'archive','server','domain', name='resource_type'), nullable=False) >> >> The permissions system is a small extension of the existing Mailman permissions system. >> user_id = the Mailman user id >> role = one of ['domainowner', 'serverowner', 'userowner', 'listowner', 'listmember', 'listmoderator'] >> resource_type one of ['list', 'domain', 'archive','server',’domain'] >> resource_id = Mailman list_id or Mailman domain or ‘server’ or ‘user' >> >> So with four pieces of information you can define access for any user to any Mailman resource. The permissions system is nothing more than a plain SQL table with four columns, that’s all. The important thing is that it unifies all Mailman resources into those four pieces of permission information. >> >> Tying it all together - the auth proxy: >> >> 1: receives an inbound HTTP API request >> 2: determines (from the token that came with the request) the authenticated identity of the requesting user >> 3: determines (from the request URL/field params) which Mailman resource is being requested (i.e. a list or a domain or something else) >> 4: determines (from the request URL/field params) the type of activity that the requesting user wishs to carry out against that resource (create new, delete, rename, for example) >> 5: asks the authorization process “is this user allowed to carry out this type of activity against this resource?” >> 6: the authorization process has its own set of business rules for working out if a particular user is allowed to carry out the requested activity against the requested resource. It uses the permissions table to work out if user is allowed to access resource in the requested manner. >> 7: finally, if the authorisation process approves, then the request is carried out. >> >> Phew! >> >> Any questions welcome. I’m hoping to have working code to show before Pycon. I want to get it sufficiently complete such that it works when I first release it. >> >> >> >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers(a)python.org >> https://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/visser%40terena.… >> >> Security Policy: http://wiki.list.org/x/QIA9 > > > > -- > Dick Visser > Sr. System & Networking Engineer > GÉANT Association, Amsterdam Office (formerly TERENA) > Singel 468D, 1017 AW Amsterdam, the Netherlands > Tel: +31 (0) 20 530 4488 > > GÉANT Association > Networking. Services. People. > > Learn more at: http://www.géant.org > -- Dick Visser Sr. System & Networking Engineer GÉANT Association, Amsterdam Office (formerly TERENA) Singel 468D, 1017 AW Amsterdam, the Netherlands Tel: +31 (0) 20 530 4488 GÉANT Association Networking. Services. People. Learn more at: http://www.géant.org

1 0

Re: [Mailman-Developers] authentication, authorization and permissions FYI
by Andrew Stuart 18 Feb '15

18 Feb '15

Hi Dick, >>>Is this about the MM3 web interface? No I’m working on a REST API proxy which uses JSON web token for authentication - sounds like SAML is somewhat similar but JSON based. Postorius/Hyperkitty provides an end user web interface to MM3. I’m not sure what mechanism they use for user authentication - maybe Mozilla Persona, which I think uses JSON web token anyway. as On 18 Feb 2015, at 10:45 am, Dick Visser <visser(a)terena.org> wrote: Hi Andrew Is this about the MM3 web interface? If so, I wonder if it could work with SAML authentication. This can be done with PySAML2 (https://github.com/rohe/pysaml2), or in an Apache module (mod_mellon). The latter takes care of the authentication, and any web app protected by it can rely on (for instance) REMOTE_USER always being available. Thanks Dick On Sun, Feb 15, 2015 at 9:05 AM, Andrew Stuart <andrew.stuart(a)supercoders.com.au> wrote: > I’ve just finished (mostly) implementing most of the authentication, authorization and permissions systems. Thought you might be interested in a little on how it works. > > To discuss this further I need to explain a few things - apologies to those who i am teaching to suck eggs. > > The auth server needs a few key subsystems to get the job done - authorization, authentication and permissions. > > AUTHENTICATION - this is the process of the end user identifying themselves in the first place (in this case, by logging in with a username and password), then for every request beyond that, providing evidence that they are who they say they are - in this case, that’s done by giving them a token when they log in successfully, then requiring that the token be returned along with each subsequent request. > > AUTHORIZATION - this is the process of determining if the authorized user is allowed - according to the rules of our application - to access the resource that they have requested. For example, user Jenny sends a request to access the members of a list named supermaillinglist(a)marketingcompany.com. Before we can execute that request we need to determine if she is *authorized* to do so. > > PERMISSIONS - for the authorization process to do its job, it needs information about WHO is allowed WHAT LEVEL OF ACCESS to WHICH RESOURCE. This is the *permissions system*. It’s VERY easy to understand. Nothing more to it than a database table that looks like this > user_id = Column(String, nullable=False) > role = Column(String, nullable=False) > resource_id = Column(String, nullable=False) > resource_type = Column(Enum('list', 'domain', 'archive','server','domain', name='resource_type'), nullable=False) > > The permissions system is a small extension of the existing Mailman permissions system. > user_id = the Mailman user id > role = one of ['domainowner', 'serverowner', 'userowner', 'listowner', 'listmember', 'listmoderator'] > resource_type one of ['list', 'domain', 'archive','server',’domain'] > resource_id = Mailman list_id or Mailman domain or ‘server’ or ‘user' > > So with four pieces of information you can define access for any user to any Mailman resource. The permissions system is nothing more than a plain SQL table with four columns, that’s all. The important thing is that it unifies all Mailman resources into those four pieces of permission information. > > Tying it all together - the auth proxy: > > 1: receives an inbound HTTP API request > 2: determines (from the token that came with the request) the authenticated identity of the requesting user > 3: determines (from the request URL/field params) which Mailman resource is being requested (i.e. a list or a domain or something else) > 4: determines (from the request URL/field params) the type of activity that the requesting user wishs to carry out against that resource (create new, delete, rename, for example) > 5: asks the authorization process “is this user allowed to carry out this type of activity against this resource?” > 6: the authorization process has its own set of business rules for working out if a particular user is allowed to carry out the requested activity against the requested resource. It uses the permissions table to work out if user is allowed to access resource in the requested manner. > 7: finally, if the authorisation process approves, then the request is carried out. > > Phew! > > Any questions welcome. I’m hoping to have working code to show before Pycon. I want to get it sufficiently complete such that it works when I first release it. > > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/visser%40terena.… > > Security Policy: http://wiki.list.org/x/QIA9 -- Dick Visser Sr. System & Networking Engineer GÉANT Association, Amsterdam Office (formerly TERENA) Singel 468D, 1017 AW Amsterdam, the Netherlands Tel: +31 (0) 20 530 4488 GÉANT Association Networking. Services. People. Learn more at: http://www.géant.org

1 0

authentication, authorization and permissions FYI
by Andrew Stuart 18 Feb '15

18 Feb '15

I’ve just finished (mostly) implementing most of the authentication, authorization and permissions systems. Thought you might be interested in a little on how it works. To discuss this further I need to explain a few things - apologies to those who i am teaching to suck eggs. The auth server needs a few key subsystems to get the job done - authorization, authentication and permissions. AUTHENTICATION - this is the process of the end user identifying themselves in the first place (in this case, by logging in with a username and password), then for every request beyond that, providing evidence that they are who they say they are - in this case, that’s done by giving them a token when they log in successfully, then requiring that the token be returned along with each subsequent request. AUTHORIZATION - this is the process of determining if the authorized user is allowed - according to the rules of our application - to access the resource that they have requested. For example, user Jenny sends a request to access the members of a list named supermaillinglist(a)marketingcompany.com. Before we can execute that request we need to determine if she is *authorized* to do so. PERMISSIONS - for the authorization process to do its job, it needs information about WHO is allowed WHAT LEVEL OF ACCESS to WHICH RESOURCE. This is the *permissions system*. It’s VERY easy to understand. Nothing more to it than a database table that looks like this user_id = Column(String, nullable=False) role = Column(String, nullable=False) resource_id = Column(String, nullable=False) resource_type = Column(Enum('list', 'domain', 'archive','server','domain', name='resource_type'), nullable=False) The permissions system is a small extension of the existing Mailman permissions system. user_id = the Mailman user id role = one of ['domainowner', 'serverowner', 'userowner', 'listowner', 'listmember', 'listmoderator'] resource_type one of ['list', 'domain', 'archive','server',’domain'] resource_id = Mailman list_id or Mailman domain or ‘server’ or ‘user' So with four pieces of information you can define access for any user to any Mailman resource. The permissions system is nothing more than a plain SQL table with four columns, that’s all. The important thing is that it unifies all Mailman resources into those four pieces of permission information. Tying it all together - the auth proxy: 1: receives an inbound HTTP API request 2: determines (from the token that came with the request) the authenticated identity of the requesting user 3: determines (from the request URL/field params) which Mailman resource is being requested (i.e. a list or a domain or something else) 4: determines (from the request URL/field params) the type of activity that the requesting user wishs to carry out against that resource (create new, delete, rename, for example) 5: asks the authorization process “is this user allowed to carry out this type of activity against this resource?” 6: the authorization process has its own set of business rules for working out if a particular user is allowed to carry out the requested activity against the requested resource. It uses the permissions table to work out if user is allowed to access resource in the requested manner. 7: finally, if the authorisation process approves, then the request is carried out. Phew! Any questions welcome. I’m hoping to have working code to show before Pycon. I want to get it sufficiently complete such that it works when I first release it.

2 1

Soliciting feedback on idea for rounding out the permissions model.
by Andrew Stuart 17 Feb '15

17 Feb '15

I had an idea about rounding out the Mailman permissions model, interested in hearing thoughts on it. Obviously there has been considerable discussion on this topic before. Mailman already carries much of the information needed for determining user permissions to Mailman resources. Only two things are missing: 1: the ability to define a user as being a “serverowner” 2: the ability to define a user as being a “domainowner” (You’ll need to look at this email in plain text to see the table properly). The Mailman permissions model currently looks like this: resource_type roles resource_id user_identifier where to find permission --------------------------------------------------------------------------------------------------------- user userowner n/a UUID (defined in user record) list listowner list_id subscriber (defined in list member record) list listmember list_id subscriber (defined in list member record) list listmoderator list_id subscriber (defined in list member record) list listnonmember list_id subscriber (defined in list member record) I am suggesting adding two further permissions to the existing permissions model, which would look like this: resource_type roles resource_id user_id where to find permission --------------------------------------------------------------------------------------------------------- server serverowner n/a UUID (not currently defined in Mailman) domain domainowner mail_host UUID (not currently defined in Mailman) To implement, it would need to be possible to define as user as being a ‘serverowner’, and also to be able to define a user as being a ‘domainowner’ for any given domain. If it were possible to do so within the Mailman core then there would be a completely usable permissions model entirely within Mailman, and no need to store any additional permissions data outside Mailman. The permissions model would allow definition of user access to any Mailman resource including domains and servers. The interpretation of the permissions would still be up to the application that consumes the REST API, as is currently the case. Thoughts? as

3 6

A couple of archiving questions
by Andrew Stuart 16 Feb '15

16 Feb '15

Can anyone point me in the direction of a good explanation of how archiving works? I’ve found bits and pieces but the picture isn’t totally clear - it appears that archiving requires some sort of archive handler function to be written and installed as part of Mailman? Is there any reason why it should not be possible to be able to pull messages scheduled for archiving out through the REST API as file attachments? thanks

5 9

Mailman-Developers February 2015