Mailman-Developers June 2013

mailman-developers@python.org

12 participants
12 discussions

A list of discussion topics: GSoC OpenPGP Integration

by Abhilash Raj

This is a list of topics that probably needs to be discussed in detail again. I tried to mention in breif about the discussions in past personally with a someone or on mm-dev list. Please ignore the topics which you feel has already reached a inference. It is a long mail though. * How to ensure the keys belong the email it says it does? One method proposed for this was to send a confirmation email to the email address, but what if the email is intercepted in between and the attacker confirms the sign up of the person he is trying to impersonate? Or this is a problem that can be solved with SSL and is not of our concern? * Inline pgp should be supported or not? Although its not included in one the best practices but some people still use it for the purpose of signing only a part of the email. * How to handle partial multipart/signed messages? In previous discussion it was pointed out that we should strip off the parts till we actually arrive at a point where the signature verifies all the text and then send that part to the subscribers. Other options were to discard these kind of email or let the owner decide what he wants. * Should mailman keep the signature of sender before signing or strip if off? In previous discussions it was mentioned that user's signature can be kept as it is and the list re-signs the message before sending the subscribers as subscribers may want to confirm if the email was originally sent by the user in the From address. But there might be a case of anonymous list where identity of the sender should not be revealed and is only verified that any one the members sent the email. So can we introduce this as an option? About introducing options I agree to a point mentioned in the mailman-pgp-smime patch for mailman where it says that although there are lots of ways we can allow the list owner to configure a list, we should only provide minimal options to avoid unexpected scenarios where the list's security is compromised due to lots of options. * Should we make the sending of signed copies of email default? We can introduce an option to send signed email by default for those who want may verify the email, but is there a point in sending singed email when the mails received are not signed? Or this could be a good point? * Email interface to manage keys? Mailman has a set of email commands to process the requests for subscription, change password and many other things over email without a web interface, so to allow managing keys over email we could allow receiving signed emails with new keys and command to add them, the confirmation mail for the new key will then be sent to the same address but would expect a signed reply from the new key. * How are we actually using the web-of-trust model of OpenPGP? Should there be any restrictions on the key to implement the web-of-trust model which advocates for the authenticity of the key provided by the user? Something like it must be signed by 2 or 3 people or anything like that? About the implementation I decided to take up with the processing of email part first and then setting up the PKI for the users. The flow would be something like this: * the message is queued in incoming queue * the incoming runner wakes up, finds the message and calls a few functions to verify the signature of the message(assuming the function already has public key of the user from somewhere) * If the message signature is found to be valid the message is then passed on to onther runners as usual (without stripping of the signature as per my assumption till now, need discussion on this) else it is dropped or bounced depending on the state of verification( like if the signature is older we can inform the sender as the delay may have been due to smtp deliver and simply drop the message if the signature is verified to be wrong). * Now a valid signature would be a signature signed within 4 days(default and configurable by list owner) from when mailman receives the email.(Is there someway to also check if the signature was previously sent on the list? so that we can block that also?) * After that the message passes on to various other queue and runners as usual and when finally it arrives at outgoing runner it again calls a few functions to sign the message and send it to MTA. I think the function to prase message, check signature, resign message could be there in utilities as a gpg.py module. Which is where I think I should first start from. I was looking for a python wrapper for gnupg and found two options: [python-gnupg][2] and [gnupg-interface][3]. GnuPG-Interface was what was used in pgp-smime patch for mailman 2.1.5. I don't have much idea about waht should be used, will post on mailman-developers. About the outgoing messages i was thinking if we can create a signing queue and sign runner which simply signs each message with list's private-key and then the message moves on to outgoing queue where it can be delivered without any furthur changes. Any ideas about this? -- Abhilash Raj

7 years, 7 months

Re: [Mailman-Developers] Mailman search function (was?? Re: Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python))

by Mark Sapiro

On 06/29/2013 07:33 PM, Chris Cargile wrote: > FWIW, I'd also like very much if anyone could provide a few lines recap of > the activity on the ModernArchiving topic under "initiatives and proposals" > as I feel my thoughts on searching/archiving tie into the > objectives/requirements listed there: > http://wiki.list.org/display/DEV/ModernArchiving See <https://fedorahosted.org/hyperkitty/> -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

7 years, 8 months

Re: [Mailman-Developers] Mailman search function (was?? Re: Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python))

by Mark Sapiro

On 06/29/2013 07:12 PM, Chris Cargile wrote: > I'm not sure where things stand regarding search for mailman- > however, I found Ian > Hicks<http://lists.whatwg.org/pipermail/commit-watchers-whatwg.org/>' > approach (of applying the HT-Dig capabilities to enhance a MM v2.0 instance > interesting, but perhaps overkill and/or not quite right) but a good > case-in-point for ideas. It appears the archive at the above URL employs the indexing and htdig patches for Mailman 2,1 originally developed by Richard Barrett <http://www.openinfo.co.uk/mm/index.html> and now maintained by me <http://www.msapiro.net/mm/>. > I resolved my need for making 'search' on my lists practically achievable > but in terms of overall integration into the fabric of mailman code, it is > insufficient. What more do you want? > If you want to see my search implementation, check out a prototype from me > on my AWS instance<http://ec2-54-224-162-128.compute-1.amazonaws.com/pipermail/mailman/> > . > > Please share your thoughts here or directly to me ;) Your prototype looks fine, but what's behind /cgi-bin/search.py ? Don't look for anything to be released for MM 2.1. The patches for htdig are available and other methods are referenced in the FAQ. If anything were to be released, it would be the htdig patches because that's what I use, but it's problematic because of the htdig dependency. For Mailman 3 the intent is that archivers will be "plugable", and the defaults will be <http://www.mail-archive.com> and <https://fedorahosted.org/hyperkitty/> -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

7 years, 8 months

Re: [Mailman-Developers] Mailman search function (was?? Re: Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python))

by Chris Cargile

FWIW, I'd also like very much if anyone could provide a few lines recap of the activity on the ModernArchiving topic under "initiatives and proposals" as I feel my thoughts on searching/archiving tie into the objectives/requirements listed there: http://wiki.list.org/display/DEV/ModernArchiving More concisely, where I have search capability, my goal of integrating more 'robustly' is I think one that will rely upon a channel of communication being established with the archiving mechanisms, if I understand the mm 2.0 codebase well enough. It may have been a while since anyone here has spent much time in the version mm 2.0 code, but would love to start discussing/working with anyone who's taking that on in v3.0..anyone? On Sat, Jun 29, 2013 at 10:12 PM, Chris Cargile <follybeachris(a)gmail.com>wrote: > I have observed the mailman development activities for a while but have not > chimed in recently. > > My initial aim for contributing was to offer input as to the search > functionality feature request. > > In as much as there is a lot to gain from the continual flow of ideas that > I feel are spawned by my watching and considering the list posts, I find at > times it is a little arduous of a task to get the fine details I seek on > specific matters such as the search features' progress (this is intended to > be as much as compliment as anything, I suppose) ...or that I am grateful > for a chance to suggest my technique to tackle this 'burden' of organizing. > > Whereas, certain resources may lead one to dismissively settle on allowing > google site-crawlers to work, this technique doesn't work (I'm glad) for > many, and I'm not sure where things stand regarding search for mailman- > however, I found Ian > Hicks<http://lists.whatwg.org/pipermail/commit-watchers-whatwg.org/>' > approach (of applying the HT-Dig capabilities to enhance a MM v2.0 instance > interesting, but perhaps overkill and/or not quite right) but a good > case-in-point for ideas. > > I resolved my need for making 'search' on my lists practically achievable > but in terms of overall integration into the fabric of mailman code, it is > insufficient. > > If you want to see my search implementation, check out a prototype from me > on my AWS instance< > http://ec2-54-224-162-128.compute-1.amazonaws.com/pipermail/mailman/> > . > > Please share your thoughts here or directly to me ;) > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/follybeachris%40g… > > Security Policy: http://wiki.list.org/x/QIA9 >

7 years, 8 months

Mailman search function (was?? Re: Like to participate in GSOC 2013 (GNU Mailman) - Like some ideas proposed in the list (Django, Python))

by Chris Cargile

I have observed the mailman development activities for a while but have not chimed in recently. My initial aim for contributing was to offer input as to the search functionality feature request. In as much as there is a lot to gain from the continual flow of ideas that I feel are spawned by my watching and considering the list posts, I find at times it is a little arduous of a task to get the fine details I seek on specific matters such as the search features' progress (this is intended to be as much as compliment as anything, I suppose) ...or that I am grateful for a chance to suggest my technique to tackle this 'burden' of organizing. Whereas, certain resources may lead one to dismissively settle on allowing google site-crawlers to work, this technique doesn't work (I'm glad) for many, and I'm not sure where things stand regarding search for mailman- however, I found Ian Hicks<http://lists.whatwg.org/pipermail/commit-watchers-whatwg.org/>' approach (of applying the HT-Dig capabilities to enhance a MM v2.0 instance interesting, but perhaps overkill and/or not quite right) but a good case-in-point for ideas. I resolved my need for making 'search' on my lists practically achievable but in terms of overall integration into the fabric of mailman code, it is insufficient. If you want to see my search implementation, check out a prototype from me on my AWS instance<http://ec2-54-224-162-128.compute-1.amazonaws.com/pipermail/mailman/> . Please share your thoughts here or directly to me ;)

7 years, 8 months

Mailman structure

by Nghia Vu Truong

Hi everyone, I am currently doing a project of expanding Mailman features. However, I am a newbie, so I want to understand the structure of Mailman coding files. Could any one know about this please help me, or please let me know which documents related to this problem! Thanks so much Nghia Truong

7 years, 8 months

Auth/auth for Django

by Barry Warsaw

I just saw this come across python-announce. Would this at all be helpful for Postorius or Hyperkitty? http://peterhudec.github.io/authomatic/index.html http://peterhudec.github.io/authomatic/examples/django-simple.html -Barry

7 years, 8 months

Re: [Mailman-Developers] GSOC Project Discussion

by Barry Warsaw

On May 13, 2013, at 10:52 PM, varun sharma wrote: >1. Suspend emails in vacation and post from multiple email ids: >IMember interface present in interfaces/member.py is already providing the >delivery_status enum in _client module via rest api but the user.save() is >currently not functional, so the member preferences are not getting saved >right now. Similarly, IUser interface in interfaces/user.py is providing >the user.addresses attribute from IAddress interface to _client module but >due to the problem of user object not getting saved, this functionality is >also not working. For these two features, i don't think there is any >requirement from mailman core interfaces. I think to do vacations right, you will need some additional support in the core. One question to answer first is: what's the scope of a vacation stop? Does it apply to an address, user, member, or all of the above? For example: when I go on vacation, do I want all deliveries to stop to all mailing lists I'm subscribed to on a particular site, or just to some subscriptions? My thinking is that two use cases would be good to support: - Stop all deliveries from this site for a user. This would mean setting vacation stops for all of the user's subscriptions to all of their mailing lists on a domain (it probably should not be cross-domain). - Stop all deliveries for a specific subscription or list of subscriptions. For example, when I go on vacation, I might want to only receive mailman-developers traffic but nothing else from python.org. These two use cases imply to me that a vacation stop is tied to a subscription (i.e. an IMember) and a global stop is just a trivial case of a set of specific stops. A few other core-touching changes: - The expiration date for a vacation stop should be set when the stop is created. A user should not have to do anything to start getting their email again. When the expiration date passes, the traffic starts up again. - You should be able to set a vacation stop to start sometime in the future. E.g. I know I am going on vacation from next Tuesday to Friday, so let me set that up now while I'm thinking about it. Don't make me wait until Tuesday morning when I'm freaking out about catching my flight. :) - You should be able to query the system to see what vacation stops you have coming up, or are in progress. Maybe even view the history of your vacation stops (but that implies more record keeping, so maybe not). - You should be able to adjust the start and end dates of the vacation stop, if they haven't already passed. You could use this to extend a vacation, or change the start date to Monday if you have to leave early. A special case of this would also be to cancel a vacation stop right now. - Remember that vacations shouldn't be used as a way to more permanently disable delivery to a mailing list. All of this implies to me a new table/record which records future and non-expired vacation stops. The table would have a start and end datetime and a link to an IMember. Then likely the member_recipients.py handler would take these stops into account when calculating who to deliver this message to. Question: Should you be able to add a vacation stop to moderator or owner emails? >2. Merge identities : >Not exactly sure about additional core requirements, but i think it can be >achieved by expanding IAddress interface. Mostly this will be able sending out and coordinating the various confirmations messages that will be necessary. E.g. you're logged in as anne(a)example.com and you make a claim to merge aperson(a)example.com. You need to prove that you own the latter address too in some way. Once you've confirmed ownership of both addresses, you need to IUser.link() them both to the same IUser. This may involve unlinking and deleting an obsolete IUser record from an IAddress. Cheers, -Barry

7 years, 8 months

Wiki Migration Update

by Paul Boddie

Hello, It's been a couple of months or so since my last update, and I finally got round to doing some more work on the content migration from Confluence to MoinMoin. As always, the results can be found here: http://mmwiki.boddie.org.uk/ At the moment, I'm still using the archived content from April 2013, and I took the opportunity this time round to fix various parsing and translation issues, particularly with the Confluence wiki markup (not the XHTML variant that the most recent revisions of pages use), also reworking some fairly fundamental mechanisms involved in parsing the wiki markup. Some previously mentioned fixes have been made: http://mmwiki.boddie.org.uk/COM/Organizations_that_use_Mailman (Anchors are now generated for the headings.) Some support for macros now exists, such as for the "anchor", "color" and "toc" macros. Meanwhile, some things wait for more time and energy to be spent on them: Author information is still not preserved in the page import process, but this will be tested in future. The way comments are presented on pages still needs improving. It might also be nice to have a list of attachments on pages that have them, and I will take a look to see how Confluence tends to present such things. And some things won't get fixed until the hosting is changed, such as for pages ending in a question mark. As I have mentioned previously, the source code for the converter can be found here: http://hgweb.boddie.org.uk/ConfluenceConverter/ Please take a look at your favourite pages and let me know where improvements can be made to the conversion process. Paul

7 years, 8 months

Re: [Mailman-Developers] Systers VM for Mailman Development

by Richard Wackerbarth

Hi, You have provided me with EXACTLY the information that I requested. To operate in this mode, I recommend the following: 1) Read enough about Vagrant to be able to "freeze" an image of the VM and use that as a base for your actual work. That way, you can work through the initial setup of your own customized installation and freeze the result, That way you won't need to repeat those steps every time you want to make a new VM. Then, by working in an expendable copy of the VM, when you "mess up" (and you will :) ), you can just toss it away (vagrant destroy) and start over with a clean copy. 2) Whether you are working in a VM, or directly on your host machine, I strongly recommend the use of virtualenv and virtualenvwrapper. (I like the added functions of the wrapper. But the important concept is the use of the virtual environments) Virtual environments are like a "poor man's" virtual machine but they don't encounter the overhead of a virtual machine. They allow you to have independent sets of python and python libraries installed. Thus one program can use a different version of a library without affecting other programs. In my setup, I use a separate virtual environment to represent each logical machine that runs a part of the mailman system. Thus my mail handler (the "core" part of mailman) is in a separate virtual environment from the web server (postorius). 3) When it comes to development, the various python modules can be divided into three groups. a) The code that you are developing b) Other code that is under development but for which you want to use a pre-release version. (or any "source" distribution) c) Additional library functions upon which those parts depend. For the vagrant virtual machine organization that you will be using: Place the (c) routines in /vagrant and keep them under your SCM (version control). You can have the (b) routines in the home directory on the virtual machine. The (a) routines will be loaded into the site-packages for the individual virtual environments as a part of the installation process. If you follow this layout, it will be easy to replace any of the various components whenever you desire. Since we are new to working with Vagrant, a setup for the mailman project such as I describe is still a "work-in-progress". Expect some revisions during the summer. However, they should not have any negative impact on your work because all of your code will be stored under the /vagrant mount point and can be moved readily from one virtual machine configuration to the next. Now, for your code, I recommend that you start by having a separate folder for each of the virtual environments. In my current setup, I have two virtual environments "VE-mailman" and "VE-webserver". In your source, you can have: /vagrant/webserver/my_django_app1/ /vagrant/webserver/my_django_app2/ /vagrant/webserver/my_website/ /vagrant/mailman/some_addition_to_mm-core/ Of course, depending on just what your project requires, you may not need all of these parts. Eventually, I would expect to see a setup script that is able to use this layout to automate the building of a suggested installation script. I hope that this helps. Feel free to ask questions and make suggestions to improve this setup. Richard On Jun 15, 2013, at 10:44 PM, Julia Proft <jproft(a)conncoll.edu> wrote: > Hi Richard, <snip> > this is my first time using virtual machines in development, and I'm figuring it out as I go! Right now, my game plan is to edit on my Windows host, and I'll keep those files in a shared folder that is accessible through the VM... then I can execute the files on the VM. <snip> > Best, > Julia > On Sat, Jun 15, 2013 at 3:35 PM, Richard Wackerbarth <rkw(a)dataplex.net> wrote: > Hi, Julia. > > I'm Richard "Wacky" Wackerbarth, one of the Mailman developers and a Mentor for GSoC. > > I notice that, in a discussion on the systems-dev mailing list, you indicated that your host machine is running "Windows 7". Since that is one of the least "Unix friendly" environments, I was wondering just how you utilize the Virtual Machine images in your development cycle. > > It appears that the machines are set up to be "an appliance" (which is great if all you want to do is deploy it). However, using ssh to communicate in a terminal window isn't very friendly for developers. > > So I'm wondering just where you store your code that is under development, how do you edit it, and on what host you execute it? <snip>

7 years, 8 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

Mailman-Developers June 2013