Mailman-Developers May 2013

mailman-developers@python.org

18 participants
34 discussions

GSoC Project Discussion - Web Posting Interface.

by Peter Markou

Hello to all the members of the community, I've applied a proposal on Melange for the Web Posting Interface. The major subject of this project, as described in the wiki, is to allow logged in users to post message from the web interface and integrate it into the appropriate pages(e.g the hyperkitty archives, the postorius list settings pages). Now regarding Richard's email, "GSoC Applicants". After having a comprehensive examination in Hyperkitty I found that the functionality for posting messages through web exists. After having a conversation with abompard, that was confirmed. I suppose that if I complement this functionality with some unit tests wouldn't mind(since unit tests are missing). So what's left for me is to: -integrate/implement this functionality in the postorius list settings pages and -implement the features that I proposed, in order to make the mailman web interface usable as a web forum. These features are: - User's filter tools, - User Profiles(support for name, photo, bio, past posts & files, statistics about posts) - Top X Threads Of All-Time, - Monthly List Health, - Mentioned in Thread references, - Keyword-Based Thread Browse(HK has support for tags on threads). I would like to ask if someone is going to add any existing information to User's profile, so I could leverage this functionality. From mentors-developers I would like their opinion/feedback about these features and if they fit to Mailman's web interface. Best regards, Peter Markou.

7 years, 11 months

GSOC Project Discussion

by varun sharma

Hi Guys, In my project "Better user settings management", i need to extend the Client class of mailman.client and i will be adding a lot of instance methods and variables to the class. Right now, i am having a look at the mailman core and mailman interfaces to figure out if the method/implementation is already present in the core and if not then i will need some help from barry and other developers to include it in the core. As i am now working on writing the software requirement specification, i hope to get most of the requirements from core by this weekend.I had a chat with wacky on #mailman yesterday about the same and i will post my requirements from the core by monday. Thanks Varun

7 years, 11 months

Re: [Mailman-Developers] GSoC Applicants

by Stephen J. Turnbull

Shanu Salunke writes: > What I need help with as a starting point (at least) is how to "link" the > functionality with the (django) front-end. One solution (thank you abomard) > is using the API that (postorius also uses) MM3 and HK have support > for. This isn't your job, IMO. Your proposal is about integrating various functionality in a single UI. I agree that Django is a good implementation technology, because the Postorius and HyperKitty applications use it. How can integration be achieved? Step 1. Arms-length, proof-of-concept technology: use HTML frames to organize Postorius and HyperKitty connections along with any "native" "applets" you need (web-posting, mail-posting, NNTP posting and/or reading) in a single display. Yes, frames suck for various reasons, but they're an easy way to impose organization without *any* cooperation from Postorius and HyperKitty. This step should go to the point of allowing you to conveniently *organize* multiple lists. You probably won't be able to conveniently *manage* individual lists because of the "arms-length" nature of the technology. Don't even try; move on to step 2. Step 2. All these apps are Django apps. Django templates support inclusion and inheritance. It should be possible to get better integration with template inclusion as a substitute for HTML frames. Step 3. Design an "API" for your templates, and propose changes to Postorius and HyperKitty templates to better integrate them into your app. Step 4. Design an API for your views that can cooperate with low-level REST APIs that Postorius and HyperKitty might export, to allow the various apps to reside on different servers. Your Django models can probably be very simple, just a list nickname, some identification (RFC 2919 List-Id), and a bunch of URLs (mail posting, HyperKitty archives, Postorius subscriber and admin interfaces, other functionality if any) in a single model. That's not a promise, of course, but I think you can start there.

7 years, 11 months

GSoc - Requirement from Mentor to complete the project

by Stephen J. Turnbull

Abhilash Raj writes: > 1) I need to discuss about the design of the idea that I want to > implement. I know the rough bits but need help and comments on > what I proposed. This would require views from the whole > mailman-community. This is place for that, not private consultation with the mentor. Just start spelling out your ideas, and post them here for comment as they're developed to the point you can get useful comment. And don't worry about whether you are doing it right: just do it. Sooner is better. One reason for this approach is that as Google sees GSoC, one purpose is encouraging the student to develop his or her relationship with the community.[1] The best case is somebody who enters with several months of presence on the developer lists (but that is in no way required, it's more an "it would be nice if" example). We actually would like have the proposals posted here, but unfortunately that tends to interfere with evaluation for several reasons. (One important one is that often several students propose similar projects, and if the proposals are public there's a very natural tendency for them to converge, making selection nearly impossible.) Now that we have the proposals in full, we'd like you to start engaging the community by posting your ideas, designs, and questions here. It's probably best *not* to post big chunks of your proposal. Maybe the "short description" as an introduction, but after that you should pick a particular task and get the community to help you start on it. > 2) Apart from the corner cases mentioned in the proposal there might be > many other not caught in my eye. I would need them to be pointed out so > that a solution of it can discussed and then implemented. > > 3) There is a lot of cryptography in this project, so I would need a more > experienced view to find and remove the security loopholes. These are more specific to your topic, but they are also best done on this list, especially the issues related to security. The advantage of the mentor system is that you have somebody who has to look at your stuff. Still, the best way is to get comments from a broad selection of developers and users. > 4) Lastly since I am still a noob in programming I would need regular > review on the code for the first 2-3 weeks so that I cam improve > and write bug free production code. Well, the first thing to learn is that nobody writes bug-free code. Anticipate that your code is going to be buggy, catch as many as you can systematically (I recommend Watts Humphrey's book "Introduction to the Personal Software Process", I'm sure other mentors have their favorites). Then get reviews from others (including but not limited to your mentor(s)). Once again, what we are looking for (Richard posted as representative of the mentor team) is discussion of your plans: the pieces of the design puzzle, of course, but also your motivation for proposing the project, and your plans for advancing it. I emphasize what Richard has already mentioned: this is not a test where you have to do everything yourself to get points. You will be evaluated on the functionality you present to us at the midterm and final evaluations, not on whether you wrote all the code yourself. Some of the code you can borrow from existing modules, and some you can beg from others. Your mentors are going to be working on their own projects, and schedules can be adjusted to provide pieces of the puzzle you need sooner rather than later. Other experienced developers may be willing to help out. In some cases you can get parts of what you need from other students in GSoC (but this requires delicate negotiation and careful management by the mentors involved; we'll cross that bridge if we come to it). The rest of the code you have to write, and there's a minimum (a substantial size, at that) we'll be expecting. But the more code you can get from elsewhere, the better the product you'll produce this summer! Footnotes: [1] The others, of course, are financial support for a student getting practical experience in software development, concentrating on coding, and development of worthwhile software for the mentoring org.

7 years, 11 months

GSOC - Seeking insight on Authenticated RESTful APIs

by Stephen J. Turnbull

Before I get started on the content, note that I'm going to be rather critical of most posts of all students for a while, mostly negative. **This is not a reflection of the quality of the candidate or of the proposals involved.** Instead, it reflects the fact that we need to learn to communicate efficiently with each other. Some of that can come from mentor adjustment, but (for reasons I'll go into some other time if asked) mostly adjustment is going to need to be on the student side. It follows that there are two kinds of criticism here. One is of the content of the project. We'll try to be constructive there, but sometimes we can only ask questions which often sounds pretty negative (especially to students with no good answers yet!) I'm sorry about that, but it's necessary. The other is of the presentation, especially of content that is inappropriate and should not be included. That necessarily is going to be mostly negative. Rahul Gaur writes: > Hi all , > > I have applied for the project mentioned on the ideas page *Authenticated > REST-API in Postorius/Django - *[0] > So as per the Project idea, I have been doing some research on my end > figuring out what would be the best way to achieve this. > Since Postorius is based on Django , I have been trying Django specific > solutions to serve the functionality currently offered by the Postorius > web interface in the form of RESTful API's. The above is a good introduction. > As of now , I have been working on two different Django based frameworks to > build RESTful APIs. > > 1. Django-REST-Framework > > 2. Django-Tastypie > > Initially , I was familiar with tastypie and I was strongly considering to > use Tastypie only for this project. However after some advice from Richard > , I am trying to be more flexible and I am trying to evaluate both the > frameworks separately. With all due respect to Richard (if that's what he advised), "being flexible" here is not a good reason to look at multiple frameworks. I would recommend going with the framework you know. > I am yet to figure out how to use [django-tasty-pie] with Non-ORM > data resources precisely how to integrate it into postorius so I > can serve the APIs . This, on the other hand, is a reason to consider changing, if it's really unclear how to use Tasty Pie to do the job. But why are you specifying "non-ORM"? Django already provides the ORM, and Postorius is (eventually) going to provide access to all "user-servicable parts" (where in this context "user" includes site admin, list admin, moderator, and subscriber for sure, and possibly other roles as they are defined). For Postorius's UI face, that is going to pretty much have to go through the ORM. If you need access to something that's not in the ORM, I would suggest fixing the models. > I would be glad , if I could get some more help on this. You need to say what you're trying to access for help on this. Your description of working with the django-rest-framework below is much easier to provide help for. If this is the best you can do for Tasty Pie I'd say you're pretty much blocked on this route, and you should just switch to django-rest-framework, unless you find a clear reason to prefer Tasty Pie. In any case, I think first you should think about the APIs you want to serve, rather than writing code that serves no clear purpose. 1. Who does the API serve (site admin, list admin, moderator, subscriber, ...)? 2. What do they want to accomplish? 3. What needs to be done in the Mailman core to have that effect? 4. What facility does Postorius provide for dealing with Mailman? (In this particular case, why isn't it accessible via the Django ORM and models used by Postorius?) 5. How would you express this in a REST API? On to django-rest-framework: > I don't know if this is the right approach to the project , but let me try > to explain : > Since Django-rest-framework has following concept : > It provides serializers to convert complex data into python native data > type and which could be easily rendered into json. Wacky likes the HAL+JSON approach, I think. http://stateless.co/hal_specification.html http://tools.ietf.org/html/draft-kelly-json-hal-05 I think that's a good place to start for defining structures. It seems very flexible, and there may already be Python modules for implementing it. > mail_host = serializers.CharField(max_length=200) > url_host = serializers.CharField(max_length=200) > contact_add = serializers.CharField(max_length=200) > description = serializers.CharField(max_length=200) "Which of these things is not like the others?" Ie, why "contact_add" and not "contact_list" or "contacts"? > def __init__(self, field ,field2 , field3, field4): > self.mail_host = field > self.url_host = field2 > self.contact_add = field3 > self.description = field4 Give the parameters descriptive names: # n.b. Fixed the commas. def __init__(self, mail_host, url_host, contact_add, description): self.mail_host = mail_host self.url_host = url_host self.contact_add = contact_add self.description = description Maybe it looks a little wasteful (and violates DRY), but this is the best way I've found. > And my Views is something like this : > def all(self, request, *args, **kwargs): > dict = [] This is a list, not a dict. > dictb = [] > dictc = [] > dictd = [] Why a list per attribute? This isn't an very extensible or maintainable API, because you'll have to change all methods like this one every time you add or remove or rename an attribute. > Please let me know If I am working in the right direction Well, I assume you actually ran the program, so you know it works. But I don't see what it's *for*, so I can't say if you're going in the right direction or not. > Now about the authentication part , what would be the best possible > solution for providing authentication ? At this point, no authentication at all. Only allow connections from localhost, and let them do anything the interface permits. Delegate authentication to the client (which has to run on the same host, and presumably will be a web service so the site admin can use HTTPS + HTTP Basic Auth in the webserver). Authorization is another matter. Wacky and I discussed ACLs a little bit, and I *think* (but haven't asked him directly) he would support an "Authorizable" mixin class (or decorator, maybe) that would be applied to every "piece of information" that your API serves. BTW: It should be obviously that I'm not real clear on what "Authorizable" actually does, or what "pieces of information" are. That's your problem, in some sense, although we can discuss it here. > Also there has been a discussion on "Architecture for extra profile > info" , wouldn't this feature would be helpful in providing > specific set of permission for what part of data can be accessed by > authenticated API users ? Well, no; the point of that thread was to try to define the API for extra profile info. The thread got sidetracked into authentication, but there's a serious cart before horse problem here. Some information is so sensitive that it shouldn't be available on the net at all. Does Mailman store anything like that? Are you sure? (Hint: the extra profile information might actually come from an enterprise database.) If you can't answer questions like that, designing authn or authz systems is not possible. I think you should ignore the auth* problem, and concentrate on what information your API should provide, what packages it should come in, and the RESTful API to accomplish that. Once you have some direction on that, where to add the auth* APIs, and what they're going to look like should be much more clear. > Would it be more along the lines that any authenticated restful api user > would be able to access all the resources offered by public facing RESTful > API ? Surely not. For example, subscribers aren't allowed to create lists in Mailman 2, so I would assume any services provided by Mailman 3 should implement the same restriction. > Another thing I have been considering , what would be the end > deliverables of this project , what I mean to say is will the > Public facing RESTful API be offering functionality currently > possible with postorius via RESTful APIs or it should also include > more features ? Well, Postorius is "currently" pretty incomplete, so we'd better do more than "currently possible". > if this is the actual requirement of the project rather than > relaying the core api via postorius into a new api. Since the "consumers" of this API are really Terri and Florian, you'd better talk to them. But since they're interested in "extra" profile information, I think you need to be prepared for extensive additional requirements beyond a mere relay of the core. This is the important question at this stage. I think you're asking too many questions, in fact. You need more focus. I suggest some questions about getting your feet wet with the frameworks are OK, plus discussion of what information your REST API needs to be prepared to provide are what you need now.

7 years, 11 months

Re: [Mailman-Developers] GSoC Applicants

by Shanu Salunke

Hey all, As I understand it, this thread is meant for looking at your project from an implementation point of view (correct me if I'm wrong). Well, my proposed project is called "A Modern Interface for Member's Area" and is basically aims at providing the existing (and some new) MM-HK functionality to daily users (not admins) in a forum like interface. They can view conversations, reply, subscribe, unsubscribe, create public profiles and more like your run-of-the-mill online communities. The main aim is to try and make MM easier to use from a new-user point of view. (And for personal reasons; I have signed up for three MM-using communities and I cannot keep them straight) What I need help with as a starting point (at least) is how to "link" the functionality with the (django) front-end. One solution (thank you abomard) is using the API that (postorius also uses) MM3 and HK have support for. But since I have no experience with REST and I think some other projects are working on the apis... I would like to hear your views. - Shanu Salunke

7 years, 11 months

Re: [Mailman-Developers] GSOC - Seeking insight on Authenticated RESTful APIs

by Rahul Gaur

Links : [0] - http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013 [1] - https://github.com/aregee/rest-mailman [2] - https://github.com/aregee/mailman ------------------------------------------------------------------------------------------------------- *Rahul Gaur* *irc : iamaregee2* *web: *http://www.rahulgaur.info* * *blogs : *aregee.wordpress.com , <http://sanencynicalwriter.wordpress.com/> http://sanencynicalwriter.wordpress.com/ *fb:* http://facebook.com/iamaregee *github: *https://github.com/aregee

7 years, 11 months

GSOC - Seeking insight on Authenticated RESTful APIs

by Rahul Gaur

Hi all , I have applied for the project mentioned on the ideas page *Authenticated REST-API in Postorius/Django - *[0] So as per the Project idea, I have been doing some research on my end figuring out what would be the best way to achieve this. Since Postorius is based on Django , I have been trying Django specific solutions to serve the functionality currently offered by the Postorius web interface in the form of RESTful API's. As of now , I have been working on two different Django based frameworks to build RESTful APIs. 1. Django-REST-Framework 2. Django-Tastypie Initially , I was familiar with tastypie and I was strongly considering to use Tastypie only for this project. However after some advice from Richard , I am trying to be more flexible and I am trying to evaluate both the frameworks separately. At the moment I am going through Tastypie Documentation and resources , as I am yet to figure out how to use it with Non-ORM data resources precisely how to integrate it into postorius so I can serve the APIs . I would be glad , if I could get some more help on this. Secondly with the django-rest-framework , I tried certain approach [2] which firstly helped me in integrating the data served by the Mailman Core REST server and relay it via HTTP using the django-rest-framework. I don't know if this is the right approach to the project , but let me try to explain : Since Django-rest-framework has following concept : It provides serializers to convert complex data into python native data type and which could be easily rendered into json. Then these data sets could be structured and served with the class based views. So lets say if I want to GET localhost:8001/ postorius/settings/domains/index I tried took the following approach : # Serializer class MailmanSerializer(serializers.Serializer): mail_host = serializers.CharField(max_length=200) url_host = serializers.CharField(max_length=200) contact_add = serializers.CharField(max_length=200) description = serializers.CharField(max_length=200) # model class MailmanObject(object): def __init__(self, field ,field2 , field3, field4): self.mail_host = field self.url_host = field2 self.contact_add = field3 self.description = field4 And my Views is something like this : #Views class ApiView(generics.ListCreateAPIView): serializer_class = MailmanSerializer def all(self, request, *args, **kwargs): dict = [] dictb = [] dictc = [] dictd = [] for i in client.domains: dictc.append(i.mail_host) dict.append(i.base_url) dictb.append(i.contact_address) dictd.append(i.description) listObj = MailmanObject(field=dictc,field2= dict,field3=dictb, field4=dictd) serializer = self.serializer_class(listObj) return Response(serializer.data) and when I make curl -X GET http://127.0.0.1:8000/postorius/api/settings/domains/index/<http://127.0.0.1:8000/postorius/api/settings/domains/index/> *HTTP 200 OK* *Vary:* Accept *Content-Type:* text/html *Allow:* HEAD, GET, POST, OPTIONS {"mail_host":["aregee.com","rahulgaur.info"],"url_host":["aregee.net","rahulgaur.net"],"contact_add":["postmaster(a)aregee.com","postmaster(a)rahulgaur.info"],"description":["mail list","mail_list"]} Please let me know If I am working in the right direction and also do point out what are the things I have been overlooking in my approach. I can also use some help in evaluating which of the two frameworks would be suitable for the project. I would be glad if I can more insight on what are the technicals skills I need to focus and some reading recommendations for gaining better insight to complete this project successfully. I am also reading Leonard Richardson's and Sam Ruby's O'Reilly book on RESTful Web Services as recommended by Barry. Now about the authentication part , what would be the best possible solution for providing authentication ? There have been lot of debate on this topic already , I don't want to add more complexities but I would like to know what would be the most workable way of providing authentication for the starters.Maybe we can consider a different approach over the summer. Also there has been a discussion on "Architecture for extra profile info" , wouldn't this feature would be helpful in providing specific set of permission for what part of data can be accessed by authenticated API users ? Would it be more along the lines that any authenticated restful api user would be able to access all the resources offered by public facing RESTful API ? Another thing I have been considering , what would be the end deliverables of this project , what I mean to say is will the Public facing RESTful API be offering functionality currently possible with postorius via RESTful APIs or it should also include more features ? *Stephen Turnbull* May 2, 2013, 7:23 p.m.<http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/argee/26…> > > This proposal is very complete. It will take some time to evaluate the > proposal as a whole. But I am still concerned that the proposal seems to > propose a new API to be offered by Postorius. Postorius is already > designed as a client of the core REST API, it seems to me that it would be > more useful to improve the core API rather than relay via Postorius into a > new API. > Steve mentioned the following on melange , maybe we can discuss more on this. As I have already mentioned , I based my proposal upon the idea of introducing the authenticated public facing RESTful API's in Postorius so most of my approach has been Django centric. I have just done with my college practicals and end semester exams would commence from 17th of may up till 31st of may and I would have lot of free time after that. I can still dedicate a week and look into this topic , if this is the actual requirement of the project rather than relaying the core api via postorius into a new api. Thanks & Regards, Rahul ------------------------------------------------------------------------------------------------------- *Rahul Gaur* *irc : iamaregee2* *web: *http://www.rahulgaur.info* * *blogs : *aregee.wordpress.com , <http://sanencynicalwriter.wordpress.com/> http://sanencynicalwriter.wordpress.com/ *fb:* http://facebook.com/iamaregee *github: *https://github.com/aregee

7 years, 11 months

Re: [Mailman-Developers] GSoc - Requirement from Mentor to complete the project

by Richard Wackerbarth

Abhilash, Thanks for responding promptly and publicly. You, and the others should expect the discussions you mention without including them in response to my question. The discussion with your mentor is "given" -- That's one of the main responsibilities of a mentor. You may need to initiate community discussion about parts of your design to assure that your design meets the community expectations. But let me take your project as an example of the kind of input that I am seeking. You propose to use public key crypto in the processing stream of mail messages. In order to do so, you will need to associate the appropriate key with the corresponding user. You will also need to have some configuration parameters that describe the list policy and handling of the encryption. Now, I suggest that you need, but might reasonably expect someone else to provide, the following: 1) A framework for storing the keys and handling the association of the key with a particular user. This same framework would be used to associate other methods of identification and authentication with the users. You would be expected to implement the parts that are specific to public key. 2) A mechanism to display and edit parameters that describe the list policy as it refers to encryption. Again, as above, you would specify the parameters and provide the handlers that implement the policies, but you can expect someone else to handle the GUI presentation that turn them on and off, or otherwise configure them. I won't claim that the items that I suggest above are sufficient, or perhaps even necessary, but they can be used as examples for the kind of input that I am encouraging all of the applicants to discuss. There are other applicants who have proposed to develop, for example, GUI components. They should "step up" and declare that their proposed work will provide <some aspect that you need>. Thus, we can observe that the projects complement each other. I hope to see this discussion continue. As for "mentor review" of your design. Let me point out that you do not have to be accepted by GSoC in order to contribute to MM development. I know that I, and I believe the other mentors, as well as other members of the community, would be quite willing to discuss "design" with anyone who wishes to help improve the overall system. That discussion can start at any time. We don't need to wait for the GSoC positions to be announced. Again, thanks for your participation. Richard On May 5, 2013, at 7:23 AM, Abhilash Raj <raj.abhilash1(a)gmail.com> wrote: > In response to Richard's mail these are the rough bits of what I will be needing from my mentor if I am selected: > > 1) I need to discuss > Abhilash Raj

7 years, 11 months

GSoc - Requirement from Mentor to complete the project

by Abhilash Raj

Hello all, In response to Richard's mail these are the rough bits of what I will be needing from my mentor if I am selected: 1) I need to discuss about the design of the idea that I want to implement. I know the rough bits but need help and comments on what I proposed. This would require views from the whole mailman-community. 2) Apart from the corner cases mentioned in the proposal there might be many other not caught in my eye. I would need them to be pointed out so that a solution of it can discussed and then implemented. 3) There is a lot of cryptography in this project, so I would need a more experienced view to find and remove the security loopholes. 4) Lastly since I am still a noob in programming I would need regular review on the code for the first 2-3 weeks so that I cam improve and write bug free production code. -- Abhilash Raj

7 years, 11 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

Mailman-Developers May 2013