[Mailman-Developers] GSOC - Seeking insight on Authenticated RESTful APIs

Before I get started on the content, note that I'm going to be rather critical of most posts of all students for a while, mostly negative. **This is not a reflection of the quality of the candidate or of the proposals involved.** Instead, it reflects the fact that we need to learn to communicate efficiently with each other. Some of that can come from mentor adjustment, but (for reasons I'll go into some other time if asked) mostly adjustment is going to need to be on the student side.

It follows that there are two kinds of criticism here. One is of the content of the project. We'll try to be constructive there, but sometimes we can only ask questions which often sounds pretty negative (especially to students with no good answers yet!) I'm sorry about that, but it's necessary. The other is of the presentation, especially of content that is inappropriate and should not be included. That necessarily is going to be mostly negative.

Rahul Gaur writes:

Hi all ,

I have applied for the project mentioned on the ideas page *Authenticated REST-API in Postorius/Django - *[0] So as per the Project idea, I have been doing some research on my end figuring out what would be the best way to achieve this. Since Postorius is based on Django , I have been trying Django specific solutions to serve the functionality currently offered by the Postorius web interface in the form of RESTful API's.

The above is a good introduction.

As of now , I have been working on two different Django based frameworks to build RESTful APIs.

1. Django-REST-Framework

2. Django-Tastypie

Initially , I was familiar with tastypie and I was strongly considering to use Tastypie only for this project. However after some advice from Richard , I am trying to be more flexible and I am trying to evaluate both the frameworks separately.

With all due respect to Richard (if that's what he advised), "being flexible" here is not a good reason to look at multiple frameworks. I would recommend going with the framework you know.

I am yet to figure out how to use [django-tasty-pie] with Non-ORM data resources precisely how to integrate it into postorius so I can serve the APIs .

This, on the other hand, is a reason to consider changing, if it's really unclear how to use Tasty Pie to do the job.

But why are you specifying "non-ORM"? Django already provides the ORM, and Postorius is (eventually) going to provide access to all "user-servicable parts" (where in this context "user" includes site admin, list admin, moderator, and subscriber for sure, and possibly other roles as they are defined). For Postorius's UI face, that is going to pretty much have to go through the ORM. If you need access to something that's not in the ORM, I would suggest fixing the models.

I would be glad , if I could get some more help on this.

You need to say what you're trying to access for help on this. Your description of working with the django-rest-framework below is much easier to provide help for. If this is the best you can do for Tasty Pie I'd say you're pretty much blocked on this route, and you should just switch to django-rest-framework, unless you find a clear reason to prefer Tasty Pie.

In any case, I think first you should think about the APIs you want to serve, rather than writing code that serves no clear purpose.

1. Who does the API serve (site admin, list admin, moderator, subscriber, ...)?

2. What do they want to accomplish?

3. What needs to be done in the Mailman core to have that effect?

4. What facility does Postorius provide for dealing with Mailman? (In this particular case, why isn't it accessible via the Django ORM and models used by Postorius?)

5. How would you express this in a REST API?

On to django-rest-framework:

I don't know if this is the right approach to the project , but let me try to explain : Since Django-rest-framework has following concept : It provides serializers to convert complex data into python native data type and which could be easily rendered into json.

Wacky likes the HAL+JSON approach, I think. http://stateless.co/hal_specification.html http://tools.ietf.org/html/draft-kelly-json-hal-05

I think that's a good place to start for defining structures. It seems very flexible, and there may already be Python modules for implementing it.

mail_host = serializers.CharField(max_length=200)
url_host = serializers.CharField(max_length=200)
contact_add = serializers.CharField(max_length=200)
description = serializers.CharField(max_length=200)

"Which of these things is not like the others?" Ie, why "contact_add" and not "contact_list" or "contacts"?

def __init__(self, field ,field2 , field3, field4):
    self.mail_host = field
    self.url_host = field2
    self.contact_add = field3
    self.description = field4

Give the parameters descriptive names:

# n.b. Fixed the commas.
def __init__(self, mail_host, url_host, contact_add, description):
    self.mail_host = mail_host
    self.url_host = url_host
    self.contact_add = contact_add
    self.description = description

Maybe it looks a little wasteful (and violates DRY), but this is the best way I've found.

And my Views is something like this :

def all(self, request, *args, **kwargs):
    dict = []

This is a list, not a dict.

    dictb  = []
    dictc = []
    dictd = []

Why a list per attribute? This isn't an very extensible or maintainable API, because you'll have to change all methods like this one every time you add or remove or rename an attribute.

Please let me know If I am working in the right direction

Well, I assume you actually ran the program, so you know it works. But I don't see what it's *for*, so I can't say if you're going in the right direction or not.

Now about the authentication part , what would be the best possible solution for providing authentication ?

At this point, no authentication at all. Only allow connections from localhost, and let them do anything the interface permits. Delegate authentication to the client (which has to run on the same host, and presumably will be a web service so the site admin can use HTTPS + HTTP Basic Auth in the webserver).

Authorization is another matter. Wacky and I discussed ACLs a little bit, and I *think* (but haven't asked him directly) he would support an "Authorizable" mixin class (or decorator, maybe) that would be applied to every "piece of information" that your API serves.

BTW: It should be obviously that I'm not real clear on what "Authorizable" actually does, or what "pieces of information" are. That's your problem, in some sense, although we can discuss it here.

Also there has been a discussion on "Architecture for extra profile info" , wouldn't this feature would be helpful in providing specific set of permission for what part of data can be accessed by authenticated API users ?

Well, no; the point of that thread was to try to define the API for extra profile info. The thread got sidetracked into authentication, but there's a serious cart before horse problem here. Some information is so sensitive that it shouldn't be available on the net at all. Does Mailman store anything like that? Are you sure? (Hint: the extra profile information might actually come from an enterprise database.) If you can't answer questions like that, designing authn or authz systems is not possible.

I think you should ignore the auth* problem, and concentrate on what information your API should provide, what packages it should come in, and the RESTful API to accomplish that. Once you have some direction on that, where to add the auth* APIs, and what they're going to look like should be much more clear.

Would it be more along the lines that any authenticated restful api user would be able to access all the resources offered by public facing RESTful API ?

Surely not. For example, subscribers aren't allowed to create lists in Mailman 2, so I would assume any services provided by Mailman 3 should implement the same restriction.

Another thing I have been considering , what would be the end deliverables of this project , what I mean to say is will the Public facing RESTful API be offering functionality currently possible with postorius via RESTful APIs or it should also include more features ?

Well, Postorius is "currently" pretty incomplete, so we'd better do more than "currently possible".

if this is the actual requirement of the project rather than relaying the core api via postorius into a new api.

Since the "consumers" of this API are really Terri and Florian, you'd better talk to them. But since they're interested in "extra" profile information, I think you need to be prepared for extensive additional requirements beyond a mere relay of the core.

This is the important question at this stage. I think you're asking too many questions, in fact. You need more focus. I suggest some questions about getting your feet wet with the frameworks are OK, plus discussion of what information your REST API needs to be prepared to provide are what you need now.