Hello to all the members of the community,
I've applied a proposal on Melange for the Web Posting Interface.
The major subject of this project, as described in the wiki, is to
allow logged in users to post message from the web interface and
integrate it into the appropriate pages(e.g the hyperkitty archives,
the postorius list settings pages).
Now regarding Richard's email, "GSoC Applicants". After having a
comprehensive examination in Hyperkitty I found that the functionality
for posting messages through web exists. After having a conversation
with abompard, that was confirmed. I suppose that if I complement this
functionality with some unit tests wouldn't mind(since unit tests are
So what's left for me is to:
-integrate/implement this functionality in the postorius list settings
-implement the features that I proposed, in order to make the mailman web
interface usable as a web forum.
These features are:
- User's filter tools,
- User Profiles(support for name, photo, bio, past posts & files,
statistics about posts)
- Top X Threads Of All-Time,
- Monthly List Health,
- Mentioned in Thread references,
- Keyword-Based Thread Browse(HK has support for tags on threads).
I would like to ask if someone is going to add any existing information to
profile, so I could leverage this functionality. From mentors-developers I
like their opinion/feedback about these features and if they fit to
In my project "Better user settings management", i need to extend the
Client class of mailman.client and i will be adding a lot of instance
methods and variables to the class. Right now, i am having a look at the
mailman core and mailman interfaces to figure out if the
method/implementation is already present in the core and if not then i will
need some help from barry and other developers to include it in the core.
As i am now working on writing the software requirement specification, i
hope to get most of the requirements from core by this weekend.I had a chat
with wacky on #mailman yesterday about the same and i will post my
requirements from the core by monday.
Shanu Salunke writes:
> What I need help with as a starting point (at least) is how to "link" the
> functionality with the (django) front-end. One solution (thank you abomard)
> is using the API that (postorius also uses) MM3 and HK have support
This isn't your job, IMO.
Your proposal is about integrating various functionality in a single
UI. I agree that Django is a good implementation technology, because
the Postorius and HyperKitty applications use it.
How can integration be achieved?
Step 1. Arms-length, proof-of-concept technology: use HTML frames to
organize Postorius and HyperKitty connections along with any
"native" "applets" you need (web-posting, mail-posting, NNTP
posting and/or reading) in a single display. Yes, frames suck for
various reasons, but they're an easy way to impose organization
without *any* cooperation from Postorius and HyperKitty.
This step should go to the point of allowing you to conveniently
*organize* multiple lists. You probably won't be able to
conveniently *manage* individual lists because of the
"arms-length" nature of the technology. Don't even try; move on
to step 2.
Step 2. All these apps are Django apps. Django templates support
inclusion and inheritance. It should be possible to get better
integration with template inclusion as a substitute for HTML
Step 3. Design an "API" for your templates, and propose changes to
Postorius and HyperKitty templates to better integrate them into
Step 4. Design an API for your views that can cooperate with low-level
REST APIs that Postorius and HyperKitty might export, to allow the
various apps to reside on different servers.
Your Django models can probably be very simple, just a list nickname,
some identification (RFC 2919 List-Id), and a bunch of URLs (mail
posting, HyperKitty archives, Postorius subscriber and admin
interfaces, other functionality if any) in a single model. That's not
a promise, of course, but I think you can start there.
Abhilash Raj writes:
> 1) I need to discuss about the design of the idea that I want to
> implement. I know the rough bits but need help and comments on
> what I proposed. This would require views from the whole
This is place for that, not private consultation with the mentor.
Just start spelling out your ideas, and post them here for comment as
they're developed to the point you can get useful comment. And don't
worry about whether you are doing it right: just do it. Sooner is better.
One reason for this approach is that as Google sees GSoC, one purpose
is encouraging the student to develop his or her relationship with the
community. The best case is somebody who enters with several months
of presence on the developer lists (but that is in no way required,
it's more an "it would be nice if" example).
We actually would like have the proposals posted here, but
unfortunately that tends to interfere with evaluation for several
reasons. (One important one is that often several students propose
similar projects, and if the proposals are public there's a very
natural tendency for them to converge, making selection nearly
impossible.) Now that we have the proposals in full, we'd like you to
start engaging the community by posting your ideas, designs, and
It's probably best *not* to post big chunks of your proposal. Maybe
the "short description" as an introduction, but after that you should
pick a particular task and get the community to help you start on it.
> 2) Apart from the corner cases mentioned in the proposal there might be
> many other not caught in my eye. I would need them to be pointed out so
> that a solution of it can discussed and then implemented.
> 3) There is a lot of cryptography in this project, so I would need a more
> experienced view to find and remove the security loopholes.
These are more specific to your topic, but they are also best done on
this list, especially the issues related to security. The advantage
of the mentor system is that you have somebody who has to look at your
stuff. Still, the best way is to get comments from a broad selection
of developers and users.
> 4) Lastly since I am still a noob in programming I would need regular
> review on the code for the first 2-3 weeks so that I cam improve
> and write bug free production code.
Well, the first thing to learn is that nobody writes bug-free code.
Anticipate that your code is going to be buggy, catch as many as you
can systematically (I recommend Watts Humphrey's book "Introduction to
the Personal Software Process", I'm sure other mentors have their
favorites). Then get reviews from others (including but not limited
to your mentor(s)).
Once again, what we are looking for (Richard posted as representative
of the mentor team) is discussion of your plans: the pieces of the
design puzzle, of course, but also your motivation for proposing the
project, and your plans for advancing it.
I emphasize what Richard has already mentioned: this is not a test
where you have to do everything yourself to get points.
You will be evaluated on the functionality you present to us at the
midterm and final evaluations, not on whether you wrote all the code
yourself. Some of the code you can borrow from existing modules, and
some you can beg from others. Your mentors are going to be working on
their own projects, and schedules can be adjusted to provide pieces of
the puzzle you need sooner rather than later. Other experienced
developers may be willing to help out. In some cases you can get
parts of what you need from other students in GSoC (but this requires
delicate negotiation and careful management by the mentors involved;
we'll cross that bridge if we come to it).
The rest of the code you have to write, and there's a minimum (a
substantial size, at that) we'll be expecting. But the more code you
can get from elsewhere, the better the product you'll produce this
 The others, of course, are financial support for a student
getting practical experience in software development, concentrating on
coding, and development of worthwhile software for the mentoring org.
Before I get started on the content, note that I'm going to be rather
critical of most posts of all students for a while, mostly negative.
**This is not a reflection of the quality of the candidate or of the
proposals involved.** Instead, it reflects the fact that we need to
learn to communicate efficiently with each other. Some of that can
come from mentor adjustment, but (for reasons I'll go into some other
time if asked) mostly adjustment is going to need to be on the student
It follows that there are two kinds of criticism here. One is of the
content of the project. We'll try to be constructive there, but
sometimes we can only ask questions which often sounds pretty negative
(especially to students with no good answers yet!) I'm sorry about
that, but it's necessary. The other is of the presentation,
especially of content that is inappropriate and should not be
included. That necessarily is going to be mostly negative.
Rahul Gaur writes:
> Hi all ,
> I have applied for the project mentioned on the ideas page *Authenticated
> REST-API in Postorius/Django - *
> So as per the Project idea, I have been doing some research on my end
> figuring out what would be the best way to achieve this.
> Since Postorius is based on Django , I have been trying Django specific
> solutions to serve the functionality currently offered by the Postorius
> web interface in the form of RESTful API's.
The above is a good introduction.
> As of now , I have been working on two different Django based frameworks to
> build RESTful APIs.
> 1. Django-REST-Framework
> 2. Django-Tastypie
> Initially , I was familiar with tastypie and I was strongly considering to
> use Tastypie only for this project. However after some advice from Richard
> , I am trying to be more flexible and I am trying to evaluate both the
> frameworks separately.
With all due respect to Richard (if that's what he advised), "being
flexible" here is not a good reason to look at multiple frameworks. I
would recommend going with the framework you know.
> I am yet to figure out how to use [django-tasty-pie] with Non-ORM
> data resources precisely how to integrate it into postorius so I
> can serve the APIs .
This, on the other hand, is a reason to consider changing, if it's
really unclear how to use Tasty Pie to do the job.
But why are you specifying "non-ORM"? Django already provides the
ORM, and Postorius is (eventually) going to provide access to all
"user-servicable parts" (where in this context "user" includes site
admin, list admin, moderator, and subscriber for sure, and possibly
other roles as they are defined). For Postorius's UI face, that is
going to pretty much have to go through the ORM. If you need access
to something that's not in the ORM, I would suggest fixing the models.
> I would be glad , if I could get some more help on this.
You need to say what you're trying to access for help on this. Your
description of working with the django-rest-framework below is much
easier to provide help for. If this is the best you can do for Tasty
Pie I'd say you're pretty much blocked on this route, and you should
just switch to django-rest-framework, unless you find a clear reason
to prefer Tasty Pie.
In any case, I think first you should think about the APIs you want to
serve, rather than writing code that serves no clear purpose.
1. Who does the API serve (site admin, list admin, moderator,
2. What do they want to accomplish?
3. What needs to be done in the Mailman core to have that effect?
4. What facility does Postorius provide for dealing with Mailman?
(In this particular case, why isn't it accessible via the Django
ORM and models used by Postorius?)
5. How would you express this in a REST API?
On to django-rest-framework:
> I don't know if this is the right approach to the project , but let me try
> to explain :
> Since Django-rest-framework has following concept :
> It provides serializers to convert complex data into python native data
> type and which could be easily rendered into json.
Wacky likes the HAL+JSON approach, I think.
I think that's a good place to start for defining structures. It
seems very flexible, and there may already be Python modules for
> mail_host = serializers.CharField(max_length=200)
> url_host = serializers.CharField(max_length=200)
> contact_add = serializers.CharField(max_length=200)
> description = serializers.CharField(max_length=200)
"Which of these things is not like the others?" Ie, why "contact_add"
and not "contact_list" or "contacts"?
> def __init__(self, field ,field2 , field3, field4):
> self.mail_host = field
> self.url_host = field2
> self.contact_add = field3
> self.description = field4
Give the parameters descriptive names:
# n.b. Fixed the commas.
def __init__(self, mail_host, url_host, contact_add, description):
self.mail_host = mail_host
self.url_host = url_host
self.contact_add = contact_add
self.description = description
Maybe it looks a little wasteful (and violates DRY), but this is the
best way I've found.
> And my Views is something like this :
> def all(self, request, *args, **kwargs):
> dict = 
This is a list, not a dict.
> dictb = 
> dictc = 
> dictd = 
Why a list per attribute? This isn't an very extensible or
maintainable API, because you'll have to change all methods like this
one every time you add or remove or rename an attribute.
> Please let me know If I am working in the right direction
Well, I assume you actually ran the program, so you know it works.
But I don't see what it's *for*, so I can't say if you're going in the
right direction or not.
> Now about the authentication part , what would be the best possible
> solution for providing authentication ?
At this point, no authentication at all. Only allow connections from
localhost, and let them do anything the interface permits. Delegate
authentication to the client (which has to run on the same host, and
presumably will be a web service so the site admin can use HTTPS +
HTTP Basic Auth in the webserver).
Authorization is another matter. Wacky and I discussed ACLs a little
bit, and I *think* (but haven't asked him directly) he would support
an "Authorizable" mixin class (or decorator, maybe) that would be
applied to every "piece of information" that your API serves.
BTW: It should be obviously that I'm not real clear on what
"Authorizable" actually does, or what "pieces of information" are.
That's your problem, in some sense, although we can discuss it here.
> Also there has been a discussion on "Architecture for extra profile
> info" , wouldn't this feature would be helpful in providing
> specific set of permission for what part of data can be accessed by
> authenticated API users ?
Well, no; the point of that thread was to try to define the API for
extra profile info. The thread got sidetracked into authentication,
but there's a serious cart before horse problem here. Some
information is so sensitive that it shouldn't be available on the net
at all. Does Mailman store anything like that? Are you sure? (Hint:
the extra profile information might actually come from an enterprise
database.) If you can't answer questions like that, designing authn
or authz systems is not possible.
I think you should ignore the auth* problem, and concentrate on what
information your API should provide, what packages it should come in,
and the RESTful API to accomplish that. Once you have some direction
on that, where to add the auth* APIs, and what they're going to look
like should be much more clear.
> Would it be more along the lines that any authenticated restful api user
> would be able to access all the resources offered by public facing RESTful
> API ?
Surely not. For example, subscribers aren't allowed to create lists
in Mailman 2, so I would assume any services provided by Mailman 3
should implement the same restriction.
> Another thing I have been considering , what would be the end
> deliverables of this project , what I mean to say is will the
> Public facing RESTful API be offering functionality currently
> possible with postorius via RESTful APIs or it should also include
> more features ?
Well, Postorius is "currently" pretty incomplete, so we'd better do more
than "currently possible".
> if this is the actual requirement of the project rather than
> relaying the core api via postorius into a new api.
Since the "consumers" of this API are really Terri and Florian, you'd
better talk to them. But since they're interested in "extra" profile
information, I think you need to be prepared for extensive additional
requirements beyond a mere relay of the core.
This is the important question at this stage. I think you're asking
too many questions, in fact. You need more focus. I suggest some
questions about getting your feet wet with the frameworks are OK, plus
discussion of what information your REST API needs to be prepared to
provide are what you need now.
As I understand it, this thread is meant for looking at your project from
an implementation point of view (correct me if I'm wrong).
Well, my proposed project is called "A Modern Interface for Member's Area"
and is basically aims at providing the existing (and some new) MM-HK
functionality to daily users (not admins) in a forum like interface. They
can view conversations, reply, subscribe, unsubscribe, create public
profiles and more like your run-of-the-mill online communities. The main
aim is to try and make MM easier to use from a new-user point of view. (And
for personal reasons; I have signed up for three MM-using communities and I
cannot keep them straight)
What I need help with as a starting point (at least) is how to "link" the
functionality with the (django) front-end. One solution (thank you abomard)
is using the API that (postorius also uses) MM3 and HK have support for.
But since I have no experience with REST and I think some other projects
are working on the apis... I would like to hear your views.
- Shanu Salunke
Hi all ,
I have applied for the project mentioned on the ideas page *Authenticated
REST-API in Postorius/Django - *
So as per the Project idea, I have been doing some research on my end
figuring out what would be the best way to achieve this.
Since Postorius is based on Django , I have been trying Django specific
solutions to serve the functionality currently offered by the Postorius
web interface in the form of RESTful API's.
As of now , I have been working on two different Django based frameworks to
build RESTful APIs.
Initially , I was familiar with tastypie and I was strongly considering to
use Tastypie only for this project. However after some advice from Richard
, I am trying to be more flexible and I am trying to evaluate both the
At the moment I am going through Tastypie Documentation and resources , as
I am yet to figure out how to use it with Non-ORM data resources precisely
how to integrate it into postorius so I can serve the APIs .
I would be glad , if I could get some more help on this.
Secondly with the django-rest-framework , I tried certain approach 
which firstly helped me in integrating the data served by the Mailman Core
REST server and relay it via HTTP using the django-rest-framework.
I don't know if this is the right approach to the project , but let me try
to explain :
Since Django-rest-framework has following concept :
It provides serializers to convert complex data into python native data
type and which could be easily rendered into json.
Then these data sets could be structured and served with the class based
So lets say if I want to GET localhost:8001/
I tried took the following approach :
mail_host = serializers.CharField(max_length=200)
url_host = serializers.CharField(max_length=200)
contact_add = serializers.CharField(max_length=200)
description = serializers.CharField(max_length=200)
def __init__(self, field ,field2 , field3, field4):
self.mail_host = field
self.url_host = field2
self.contact_add = field3
self.description = field4
And my Views is something like this :
serializer_class = MailmanSerializer
def all(self, request, *args, **kwargs):
dict = 
dictb = 
dictc = 
dictd = 
for i in client.domains:
listObj = MailmanObject(field=dictc,field2= dict,field3=dictb,
serializer = self.serializer_class(listObj)
and when I make
curl -X GET http://127.0.0.1:8000/postorius/api/settings/domains/index/<http://127.0.0.1:8000/postorius/api/settings/domains/index/>
*HTTP 200 OK* *Vary:* Accept *Content-Type:* text/html *Allow:* HEAD, GET,
Please let me know If I am working in the right direction and also do point
out what are the things I have been overlooking in my approach.
I can also use some help in evaluating which of the two frameworks would be
suitable for the project.
I would be glad if I can more insight on what are the technicals skills I
need to focus and some reading recommendations for gaining better insight
to complete this project successfully.
I am also reading Leonard Richardson's and Sam Ruby's O'Reilly book on
RESTful Web Services as recommended by Barry.
Now about the authentication part , what would be the best possible
solution for providing authentication ?
There have been lot of debate on this topic already , I don't want to add
more complexities but I would like to know what would be the most workable
way of providing authentication for the starters.Maybe we can consider a
different approach over the summer.
Also there has been a discussion on "Architecture for extra profile info" ,
wouldn't this feature would be helpful in providing specific set of
permission for what part of data can be accessed by authenticated API users
Would it be more along the lines that any authenticated restful api user
would be able to access all the resources offered by public facing RESTful
Another thing I have been considering , what would be the end deliverables
of this project , what I mean to say is will the Public facing RESTful API
be offering functionality currently possible with postorius via RESTful
APIs or it should also include more features ?
*Stephen Turnbull* May 2, 2013, 7:23
> This proposal is very complete. It will take some time to evaluate the
> proposal as a whole. But I am still concerned that the proposal seems to
> propose a new API to be offered by Postorius. Postorius is already
> designed as a client of the core REST API, it seems to me that it would be
> more useful to improve the core API rather than relay via Postorius into a
> new API.
Steve mentioned the following on melange , maybe we can discuss more on
As I have already mentioned , I based my proposal upon the idea of
introducing the authenticated public facing RESTful API's in Postorius so
most of my approach has been Django centric.
I have just done with my college practicals and end semester exams would
commence from 17th of may up till 31st of may and I would have lot of free
time after that.
I can still dedicate a week and look into this topic , if this is the
actual requirement of the project rather than relaying the core api via
postorius into a new api.
Thanks & Regards,
*irc : iamaregee2*
*blogs : *aregee.wordpress.com , <http://sanencynicalwriter.wordpress.com/>
Thanks for responding promptly and publicly.
You, and the others should expect the discussions you mention without including them in response to my question.
The discussion with your mentor is "given" -- That's one of the main responsibilities of a mentor.
You may need to initiate community discussion about parts of your design to assure that your design meets the community expectations.
But let me take your project as an example of the kind of input that I am seeking.
You propose to use public key crypto in the processing stream of mail messages.
In order to do so, you will need to associate the appropriate key with the corresponding user.
You will also need to have some configuration parameters that describe the list policy and handling of the encryption.
Now, I suggest that you need, but might reasonably expect someone else to provide, the following:
1) A framework for storing the keys and handling the association of the key with a particular user.
This same framework would be used to associate other methods of identification and authentication with the users.
You would be expected to implement the parts that are specific to public key.
2) A mechanism to display and edit parameters that describe the list policy as it refers to encryption.
Again, as above, you would specify the parameters and provide the handlers that implement the policies, but you can expect someone else to handle the GUI presentation that turn them on and off, or otherwise configure them.
I won't claim that the items that I suggest above are sufficient, or perhaps even necessary, but they can be used as examples for the kind of input that I am encouraging all of the applicants to discuss.
There are other applicants who have proposed to develop, for example, GUI components. They should "step up" and declare that their proposed work will provide <some aspect that you need>. Thus, we can observe that the projects complement each other.
I hope to see this discussion continue.
As for "mentor review" of your design. Let me point out that you do not have to be accepted by GSoC in order to contribute to MM development. I know that I, and I believe the other mentors, as well as other members of the community, would be quite willing to discuss "design" with anyone who wishes to help improve the overall system.
That discussion can start at any time. We don't need to wait for the GSoC positions to be announced.
Again, thanks for your participation.
On May 5, 2013, at 7:23 AM, Abhilash Raj <raj.abhilash1(a)gmail.com> wrote:
> In response to Richard's mail these are the rough bits of what I will be needing from my mentor if I am selected:
> 1) I need to discuss
> Abhilash Raj
In response to Richard's mail these are the rough bits of what I will be
needing from my mentor if I am selected:
1) I need to discuss about the design of the idea that I want to implement.
I know the rough bits but need help and comments on what I proposed. This
would require views from the whole mailman-community.
2) Apart from the corner cases mentioned in the proposal there might be
many other not caught in my eye. I would need them to be pointed out so
that a solution of it can discussed and then implemented.
3) There is a lot of cryptography in this project, so I would need a more
experienced view to find and remove the security loopholes.
4) Lastly since I am still a noob in programming I would need regular
review on the code for the first 2-3 weeks so that I cam improve and write
bug free production code.