Mailman 3 April 2013 - Mailman-Developers

Re: [Mailman-Developers] Architecture for extra profile info
by Richard Wackerbarth April 18, 2013

April 18, 2013

What I am suggesting is that we use the JSON format and URLs that are generated by this combination. If you later decide to implement it in a different framework, you can still use the same external representation. On Apr 18, 2013, at 4:33 PM, Florian Fuchs <flo.fuchs(a)gmail.com> wrote: > 2013/4/18 Richard Wackerbarth <rkw(a)dataplex.net>: >> Yes, yes. Please re-invent the wheel once again. >> >> And while you are at it, you might just remove the dependancies on zope and storm, etc. >> >> I think that you are missing the point that, at this time, this is intended to provide the capabilities that MM-core chooses not to implement. >> Those website components (HK and Postorius) are being driven by Django and you only make it more difficult to implement them when choose a different schema to model/present the data. > > I don't want to re-invent the wheel and I don't advocate dismissing > Django. Django is probably perfect for the prototype. It might also be > perfect for the final thing. But we don't know what database we're > gonna use in the end (which determines if the Django ORM still fits), > or if we'll use many HTML templates etc. If we end up not using many > of the things that make Django great, we might wanna think about if we > want to use it. And might still come to the conclusion that we do. All > I'm saying is: The environment for such an app could be a little > different to Postorius/HK, so let's spend some thought on it. Is this > such a bad idea? > > Florian

1 0

Re: [Mailman-Developers] Architecture for extra profile info
by Richard Wackerbarth April 18, 2013

April 18, 2013

Yes, yes. Please re-invent the wheel once again. And while you are at it, you might just remove the dependancies on zope and storm, etc. I think that you are missing the point that, at this time, this is intended to provide the capabilities that MM-core chooses not to implement. Those website components (HK and Postorius) are being driven by Django and you only make it more difficult to implement them when choose a different schema to model/present the data. On Apr 18, 2013, at 2:29 PM, Florian Fuchs <flo.fuchs(a)gmail.com> wrote: > 2013/4/18 Richard Wackerbarth <richard(a)nfsnet.org>: >> On Apr 18, 2013, at 1:19 AM, Florian Fuchs <flo.fuchs(a)gmail.com> wrote: >>> 3) It doesn't need Django. >>> Since it will not deliver any HTML (except an oAuth login form -- see >>> 5.) and it doesn't need to be integrated into any existing web site, >>> we can choose a more light-weight framework. >> >> Here I take exception. Dismissing Django is a restriction that unnecessarily affects the ease of implementation and, in the common case, complicates the integration of the functionality. >> >> Although it could be implemented without Django, it could also be implemented as a Django "app". >> An instance of a django server can then serve the functionality. As an alternative, where appropriate, this "app" would directly "drop in" to an instance of Postorius or an enterprise website. >> >> One of the advantages of Django is that it can be used as a rapid prototyping mechanism. Simplified interfaces to the data are "free" and more elaborate ones can be added in an incremental fashion. >> Also, rather than writing custom modules for things such as authentication and REST interfaces, there is the large community of third-party extensions which readily integrate to provide that functionality. > > It's not that I don't want to use Django. I just wanted to point out > that we won't need much of it for a pure JSON API. OTOH adding a new > dependency by using another framework is probably not a good idea > either. So if we want to keep the number of dependencies low, the > alternative would probably be to use no framework at all and use > restish for the, well, REST stuff (or whatever library the core will > be using in the future). > > Florian > >> I would advocate that this "User" module make it appear as if stores the entire "record" for the user. >> In the implementation, it could actually store parts of the user information in multiple databases (one of which could be the MM-core). > > +1

2 1

Re: [Mailman-Developers] Architecture for extra profile info
by Richard Wackerbarth April 18, 2013

April 18, 2013

On Apr 18, 2013, at 2:39 PM, Florian Fuchs <flo.fuchs(a)gmail.com> wrote: > 2013/4/18 Terri Oda <terri(a)zone12.com>: >> We're probably going to be running around with a bit of a hack job for the >> user database in the near future (either done by a student who needs it in a >> hurry or done by one of the core devs to support a student who needs it in a >> hurry) so while I don't like to design on the assumption it's going to go >> wrong, I think in this case planning for a redesign might be prudent because >> it's pretty clear we don't actually know all our requirements. > > I think in this case it makes sense to spend a more time on the API as > seen from the outside (urls, methods, json responses) than the actual > implementation. If the API is good, it's totally acceptable if the > underlying implementation will be redesigned later. I agree and I would advocate that that API model be based on the interface that you can readily get from a django database model and the django-rest-framework. There is a big advantage to being able to readily edit the model and have the corresponding changes to the interface automatically generated. Since HK and Postorius already use Django, you are not adding any significant dependancies. And you can leverage a existing code base for the implementation details.

1 0

GSoC 2013 - GNU Mailman - Introduction and Project Discussion
by Stephen J. Turnbull April 18, 2013

April 18, 2013

Sreyanth writes: > 3. Anti-spam / anti-abuse in Mailman. A couple of people have mentioned anti-spam, and it's a frequently requested feature. Nevertheless, I don't think we should spend Google money and mentor time on it. 1. Mailman is the wrong place to do filtering. It's equally effective, normally covers more messages, and is somewhat more efficient in resource usage to do it at the MTA. 2. Any new algorithms *should* be made available at the MTA level where they can be best put to use by more people. This implies something that either plugs into existing filters (such as spamassassin) or MTAs (ie, milters) rather than a Handler. 3. Adapting existing filters is generally pretty trivial: you write a 10-line custom Handler that pipes it to an external process. This isn't big enough for a GSoC project. 4. To the extent that new algorithms are involved, I have doubts that Mailman mentors have the kind of expertise needed to really help with such a project (I could be wrong, but I certainly don't know much about that kind of text processing, and I don't know that anybody else in Mailman has expertise in it). On the other hand, I don't know which project in GSoC would be a better place for it. It's possible to argue that Mailman is a reasonable place for it, but IMHO we probably shouldn't. Regarding anti-abuse, we would like to do something about problems like backscatter. However, I have to wonder how much *code* (vs *specification* and *design*) is needed for those problems. If the project is really spec-heavy, it's probably not really what Google has in mind (based on comments on the mentors' list, not on any official Google pronouncements, though).

5 9

Re: [Mailman-Developers] Setting up a VM.
by Terri Oda April 18, 2013

April 18, 2013

Hi Peter, Sorry about not getting back to you earlier; I'm the org admin for the PSF for the first time this year, and in case that wasn't enough, we've grown considerably and I've got a huge amount of email. Thanks for pinging me off-list to let me know I missed this one once you figured out i was the one who needed to see it. The short answer is just "+1" (Which, if you're not used to this particular terminology, means "that's a good idea; go ahead and do it") So yeah, go ahead and get a VM set up; you don't need to ask. You'll probably want to follow the mailman install instructions here: http://pythonhosted.org/mailman/ Chris, who was working on such a thing before, seems to have gotten stuck somewhere. You might want to ping him to see if the two of you can work together to sort out any problems (I'm cc'ing him so he sees this mail) or if he can offer any advice. You might also want to do a quick search through the archives to see what we told him when he asked about setting things up. You probably also want to talk to other incoming students who've already set up their dev environment since they'll have any issues they encountered fresh in their minds! If you're looking for some help via IRC, most of our mentors and a decent number of students are on IRC when they have some time. I think wacky is our most stalwart IRC'er, so you may want to ping him if you're having trouble getting someone's attention. Many of us are are on IRC while busy doing other things, so if you want someone's attention the best way is to say their irc nick name in channel: most IRC clients are set up to help bring anything directed at us to our attention. When you get to the point of a final working VM, either post a link here & to the wiki, or if you need hosting, ask on the mailing list. I know I have access to some machines that can do it, but I'm sure I'm not the only one. FYI: I've got a friend flying in for a visit tonight, so I'm pretty much going to be offline as much as possible for a couple of days, but hopefully that should get you pointers to other people who be more available to help this weekend. Terri On 13-04-11 3:23 AM, Peter Markou wrote: > Hello again community. > > Two days ago I made a sort introduction of my self and stated that I'm > interested in implementing the "Web Posting Interface" project(through this > e-mail - > http://www.mail-archive.com/mailman-developers%40python.org/msg13451.html). > > In order to getting started with Mailman development I went through the > equivalent quote(here - > http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013). > Since I'm able to consume the services described here : > https://okeanos.grnet.gr/services/cyclades/ , I would like to setup a VM > and also make it available for future developers. > > If you're interested in this, poke me to arrange a meeting in IRC and > discuss any further details. > > Best regards, > Peter Markou. > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/terri%40zone12.com > > Security Policy: http://wiki.list.org/x/QIA9 >

2 1

Re: [Mailman-Developers] anti-spam filter
by Stephen J. Turnbull April 18, 2013

April 18, 2013

Pratik Sarkar writes: > Coming back to the project,do we need to write patches for the > SpamAssassin or SpamBayes(which are integrated in Mailman) No. You write a Handler which delegates to those external packages. A Handler is a sort of plug-in. > or do we need to start over and make a whole new filter and start > reinventing the wheels? Don't do that unless you think you can write a filter that does a significantly better job on some dimension than the existing ones. > And somewhere I read that the filtering of messages posted by > non-subscribed members are done manually. That's incorrect. Mailman's mail processing is architected as a pipeline of Handlers. Each Handler can act as a filter, or edit the message, or perform a side effect (delivering the mail to subscribers or saving it in an archive), then passes the message on to the next Handler (unless it raises an exception). Filters have a selection of options: throw away the message silently (DISCARD), return to sender (REJECT), quarantine (HOLD), or pass on for further processing. (The words in all uppercase are actually exception names.) One of the filters checks whether the mail is from a subscriber, but it is not otherwise special, and in principle both subscribers and non-subscribers may be subject to various filters.

1 0

Re: [Mailman-Developers] Architecture for extra profile info
by Stephen J. Turnbull April 18, 2013

April 18, 2013

Florian Fuchs writes: > But maybe we can take a moment to think about the usefulness of such a > feature and the possibilities this might open up, rather than > dismissing the use of a certain technology right off the bat. I'm not dismissing the use; I'm saying an authentication provider is out of scope for Mailman, for several reasons. An auth client is something we clearly need. The mail auth we use (plain text passwords) is clearly weak, and we need to improve that. Web auth is probably OK for most sites if we use HTTP Basic Auth over HTTPS. OAuth (or "social auth" or Persona) is clearly a major convenience for users, and probably significantly increases security all by itself (no need for Post-Its with the Mailman password you almost never use on your monitor bezel). Both HTTPS+BasicAuth and OAuth have the advantage that Mailman can delegate the handling of transmission of secrets over the Internet to professions. Given that HTTPS+BasicAuth is probably good enough for most of our users, and most of the rest will want to use Google/Yahoo/whatever rather than YAP, where's the need for a provider? And I *am* thinking about the possibilities implementing a provider opens up. Namely, the possibility that we'll distribute buggy code. :-P > I remember several discussions during PyCon(s) and on IRC where > scenarios of different mailman instances talking to each other came > up. Of course that doesn't mean implementing a generic oAuth provider > is the only answer to this. If there are better and easier solutions, > fine. The OAuth2 protocol spec (RFC 6749) makes obscure reference to "scope" and tokens acceptable to multiple resource servers. But I don't think sharing authorizations is something that a provider deals with.

1 0

Re: [Mailman-Developers] anti-spam filter
by Stephen J. Turnbull April 18, 2013

April 18, 2013

Pratik Sarkar writes: > I am working on the proposal.And how many slots are there for the filter > project? There are no slots for the filter project as such. The whole Mailman project has slots, and they are somewhat fluid, since we operate under the umbrella of the Python Software Foundation. For further details, like actual numbers, Terri Oda (PSF org admin) is authoritative. Please be patient on this, as Terri is *very* busy right now. Also, you may not get hard numbers until the selection is actually made, since the PSF has to balance many projects.

3 2

Re: [Mailman-Developers] GSOC 2013 : Authenticated REST-API in Postorius/Django
by Rahul Gaur April 18, 2013

April 18, 2013

Hi , Sorry for late reply, there's lot of college work on hand. > Thanks for creating the two examples! I took a quick look at them and > it looks like both frameworks are pretty easy to use if a resource is > based on a typical Django model. The Mailman resources OTOH are not > retrieved from a database but from another API -- Mailman's core REST > API that is only served locally (postorius uses the mailman.client > library to access that data). Which of the two frameworks would you > consider more flexible if you want to get your data from anything else > but a Django DB model (I *think* Tastypie has different classes for > model resources and everythingelse-resources)? > I looked into Tastypie and yes the model resources is just more tightly wrapped class which focus on django ORM and then there's this resources class which could be used for non orm models. I also went through a similar example on Tastypie, for using nosql db for resources. Here is a link to the example I found on Tastypie docs [0] . They are linking Riak for data resources, and Riak client used here communicates with core Risk for data sources with the APIs. So, I believe similar thing could be achieved with our project. > > *$ curl http://127.0.0.1:8080/formpost/ -H 'Accept:text/html' > > > > The Django-rest-frameworks API's returns web browsable representation of > > the data by default :) > > Sounds nice! :-) > > > So , the confusion I am facing here is that the Postorius is already mature > > and stable app and while playing with the two frameworks for API's what I > > figured out is that if I opt for django-rest-framework , I would need to > > write new Views ? > > But if I go for TastyPie , I won't have to touch the existing Views and > > write authenticated API's. > > > > Hence to implement authenticated API's , which one would be better ? > > Hard to say - judging by how they're used in the examples, both look > good. But there are some more things you might want to consider before > choosing one: > > Authentication methods > Which auth methods are provided out of the box by the frameworks? And > which one(s) would you like to support (OAuth(2), BasicAuth over SSL, > ...)? I guess choosing and implementing authentication could become > one of the major sub-tasks of that project. Talking about authentication, django-restframework has a wrapper for OAuth2 out of the box, but I will have to check how to use non form data sources. I know there must be a solution for that, but then again in django-rest framework REST APIs are handled from the views.py. I don't really want to fiddle with existing views.py in Postorius. Maybe we could discuss more about this , accounting what would be better in the long run. Personally I would be sticking to Tastypie for authentication as well. If we plan to support Oauth2 , it could also be achieved with Tastypie. Couple of days back I saw some similar examples on github. I would try to provide some examples on the topics to support my proposal. Hopefully I would be able to do that by this weekend . > > Support for Django 1.5 and Python3: Isn't the support for Python 3 just experimental in django 1.5 .? And neither of the two third party frameworks support python 3 fully in their current release. Django rest framework has some amount of experimental support but not sure about Tastypie. > At some point in the not so fare future mailman and postorius will be > ported to Python3. Django 1.5 already supports Python3, so it would be > a bonus point if the rest framework supports it, too (or if their > developers are making efforts to do so...). I know the support for python 3 is essential to the project, but as of now sticking I would prefer to sticking to Python 2.7.x for now. I would be having a month in hand before official GSOC CODING period starts. During this time I would be getting stared with the project so that there would be some substantial code ready before official coding time starts. I guess its roughly two months from now , so if there's a release for these frameworks with python 3 support . Porting it wouldn't be that much of an issue. > Another idea (not sure if it's a good one): Maybe also take a closer > look at Django's class-based views. They are capable of routing > requests to the same URI to different dispatch methods, depending on > the HTTP method. So that might be also be an interesting option. > Yes, this is similar to what class based views in django-restframework offers. To be honest, I don't really have much idea about django class based views.I am looking into this and I would let you know. Also I have couple of other thinks I would like to discus. Maybe we could discuss more about this over the irc. Regards! Rahul

1 0

Build a mailman3 development virtual machine with Vagrant and Chef
by Xu Wang April 18, 2013

April 18, 2013

Hi there, Although mailman3 installation is well documented, but putting everything together is still not a trivial job. To ease the task, I made a mailman3 chef cookbook and vagrant file: https://github.com/xuwang/mailman3-vbox If you already had virtualbox/vagrant installed on your mac or pc, build a working mailman3 vm should only take a few minutes. I'm sure there will be things I missed or got it wrong, but it just a starter. Hope this helps, Xu Wang

1 0