[Twisted-Python] How to design REST API with Twisted?
![](https://secure.gravatar.com/avatar/2dcefdfea4fae17911a38da01edaa77d.jpg?s=120&d=mm&r=g)
Hi, I'm confused about how to design REST APIs with Twisted, especially when my app have to interact with MySQL. Is there any basic design patterns for this kind of situation? Thanks!
![](https://secure.gravatar.com/avatar/63756693d9b79ff633f86ff5672652e9.jpg?s=120&d=mm&r=g)
Hey! On 10/25/15 13:04, Wang Yan wrote:
I do Spyne, and I think it's _fa-bu-lous_ for building any web API on top of twisted. It's a bit different from what you to with stock Twisted though. More specifically, there's no NOT_DONE_YET. The client hangs as long as you keep returning `Deferred`s. Once you return a non-deferred (and hopefully something that's compatible with your designated return type) that object is serialized using the `out_protocol` you pass to your `Application` and the resulting byte stream is written to the outgoing stream of your transport of choice. In case of a HTTP-based api, it's passed to the `transport.write()` function of a `twisted.web.Resource` instance, along with the outgoing headers that you set. As for the MySQL part, I use SQLAlchemy exclusively from inside a function called by deferToThread. Spyne integrates with SQLAlchemy as well, so you can return directly what SQLA returns, be it objects or rows. Here's a simple example: https://github.com/arskom/spyne/blob/master/examples/twisted/resource.py There's a boilerplate generator here: http://spyne.io/ I hope you find it useful. Any questions, people@spyne.io is also at your service. Best, Burak
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Oct 25, 2015, at 5:13 AM, Burak Arslan <burak.arslan@arskom.com.tr> wrote:
It's a bit different from what you to with stock Twisted though. More specifically, there's no NOT_DONE_YET.
I should just mention that NOT_DONE_YET actually _predates_ Deferreds, and one glorious day, we will get rid of it :). See for example https://tm.tl/288 <https://tm.tl/288>. -glyph
![](https://secure.gravatar.com/avatar/2dcefdfea4fae17911a38da01edaa77d.jpg?s=120&d=mm&r=g)
I'm planning to design some REST APIs with Twisted in Python. For example, I want to use the HTTP method "GET" to fetch a single user's information: GET http://myhost:8000/api/v1.0/users/[user_id] I know I should inherit the twisted.web.resource.Resource and implement "getChild" by myself. The question is, should I implement a class for each segment of the URI? If so, I have to implement class API, class V1, class Users and Class User. In other words, if there're 10 segments in the URI, do I have to implement 10 classes to represent those resources? At 2015-10-25 20:13:58, "Burak Arslan" <burak.arslan@arskom.com.tr> wrote: Hey! On 10/25/15 13:04, Wang Yan wrote: Hi, I'm confused about how to design REST APIs with Twisted, especially when my app have to interact with MySQL. Is there any basic design patterns for this kind of situation? I do Spyne, and I think it's fa-bu-lous for building any web API on top of twisted. It's a bit different from what you to with stock Twisted though. More specifically, there's no NOT_DONE_YET. The client hangs as long as you keep returning `Deferred`s. Once you return a non-deferred (and hopefully something that's compatible with your designated return type) that object is serialized using the `out_protocol` you pass to your `Application` and the resulting byte stream is written to the outgoing stream of your transport of choice. In case of a HTTP-based api, it's passed to the `transport.write()` function of a `twisted.web.Resource` instance, along with the outgoing headers that you set. As for the MySQL part, I use SQLAlchemy exclusively from inside a function called by deferToThread. Spyne integrates with SQLAlchemy as well, so you can return directly what SQLA returns, be it objects or rows. Here's a simple example: https://github.com/arskom/spyne/blob/master/examples/twisted/resource.py There's a boilerplate generator here: http://spyne.io/ I hope you find it useful. Any questions, people@spyne.io is also at your service. Best, Burak
![](https://secure.gravatar.com/avatar/b6ca8106c6e4a099f39eaddfe2b21100.jpg?s=120&d=mm&r=g)
On 11/3/15 10:15 PM, Wang Yan wrote:
Twisted is too low-level for such use in my opinion. We use Klein (https://github.com/twisted/klein) which offers nice abstraction above Twisted. I know this does not answer your question. Just wanted to make sure that you are aware of possible options. -Shakkhar [...]
![](https://secure.gravatar.com/avatar/2dcefdfea4fae17911a38da01edaa77d.jpg?s=120&d=mm&r=g)
Thank you to remind me of Klein. I'm very new to Twisted and Klein. What I've known is that Klein is a little like Flask, with which I can implement a UserAPI as follows: from flask import Flask from flask.ext.restful import Api, Resource api = Flask(__name__) api = Api(app) class UserAPI(Resource): def get(self, user_id): pass def post(self, user_id): pass api.add_resource(UserAPI, '/users/<int:id>', endpoint='user') Is there any similar usage in Klein? 在 2015-11-04 12:09:29,"Mashiat Sarker Shakkhar" <mashiat.sarker@gmail.com> 写道: On 11/3/15 10:15 PM, Wang Yan wrote: I'm planning to design some REST APIs with Twisted in Python. For example, I want to use the HTTP method "GET" to fetch a single user's information: GET http://myhost:8000/api/v1.0/users/[user_id] I know I should inherit the twisted.web.resource.Resource and implement "getChild" by myself. The question is, should I implement a class for each segment of the URI? If so, I have to implement class API, class V1, class Users and Class User. In other words, if there're 10 segments in the URI, do I have to implement 10 classes to represent those resources? Twisted is too low-level for such use in my opinion. We use Klein (https://github.com/twisted/klein) which offers nice abstraction above Twisted. I know this does not answer your question. Just wanted to make sure that you are aware of possible options. -Shakkhar [...]
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
Yes; in fact this extension for Flask is similar to how twisted.web works internally. I think you're looking for something like this: from klein import run, route from twisted.web.resource import Resource class User(Resource, object): def __init__(self, user_id): super(User, self).__init__() self.user_id = user_id def render_GET(self, request): pass def render_POST(self, request): pass @route("/users/<int:user_id>") def user(request, user_id): return User(user_id) run("localhost", 8080)
![](https://secure.gravatar.com/avatar/2dcefdfea4fae17911a38da01edaa77d.jpg?s=120&d=mm&r=g)
Absolutely that's what I'm looking for! Thanks a lot! However, I'm wondering if Twisted and Klein support filtering results. For example, the User table in MySQL contains id, name and age fields. How can I get all users whose name are "Alan" and age > 16? It seems I need a filter, but I'm not sure how to implement such a filter or resource with Twisted and Klein. 在 2015-11-06 06:02:53,"Glyph Lefkowitz" <glyph@twistedmatrix.com> 写道: On Nov 5, 2015, at 1:28 AM, Wang Yan <snailcoder@163.com> wrote: Thank you to remind me of Klein. I'm very new to Twisted and Klein. What I've known is that Klein is a little like Flask, with which I can implement a UserAPI as follows: from flask import Flask from flask.ext.restful import Api, Resource api = Flask(__name__) api = Api(app) class UserAPI(Resource): def get(self, user_id): pass def post(self, user_id): pass api.add_resource(UserAPI, '/users/<int:id>', endpoint='user') Is there any similar usage in Klein? Yes; in fact this extension for Flask is similar to how twisted.web works internally. I think you're looking for something like this: from klein import run, route from twisted.web.resource import Resource class User(Resource, object): def __init__(self, user_id): super(User, self).__init__() self.user_id = user_id def render_GET(self, request): pass def render_POST(self, request): pass @route("/users/<int:user_id>") def user(request, user_id): return User(user_id) run("localhost", 8080)
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Nov 12, 2015, at 7:33 AM, Wang Yan <snailcoder@163.com> wrote:
Absolutely that's what I'm looking for! Thanks a lot!
Glad I could help!
However, I'm wondering if Twisted and Klein support filtering results. For example, the User table in MySQL contains id, name and age fields. How can I get all users whose name are "Alan" and age > 16? It seems I need a filter, but I'm not sure how to implement such a filter or resource with Twisted and Klein.
Klein is pretty purely a web-facing framework; it just helps you built the HTTP stuff that answers queries. It doesn't help you talk to a database at all; you have to figure out your own way to do that. This is historically a weak area for Twisted, sadly; you might find https://pypi.python.org/pypi/twextpy/ <https://pypi.python.org/pypi/twextpy/> useful as this brings the state of the art forward somewhat. https://twistedmatrix.com/trac/ticket/7917 <https://twistedmatrix.com/trac/ticket/7917> is a ticket to bring that stuff into Twisted itself; if you are interested in Twisted/Database interaction, perhaps you could help split up the code there into smaller branches that can be merged. Thanks! -glyph
![](https://secure.gravatar.com/avatar/63756693d9b79ff633f86ff5672652e9.jpg?s=120&d=mm&r=g)
Hey! On 10/25/15 13:04, Wang Yan wrote:
I do Spyne, and I think it's _fa-bu-lous_ for building any web API on top of twisted. It's a bit different from what you to with stock Twisted though. More specifically, there's no NOT_DONE_YET. The client hangs as long as you keep returning `Deferred`s. Once you return a non-deferred (and hopefully something that's compatible with your designated return type) that object is serialized using the `out_protocol` you pass to your `Application` and the resulting byte stream is written to the outgoing stream of your transport of choice. In case of a HTTP-based api, it's passed to the `transport.write()` function of a `twisted.web.Resource` instance, along with the outgoing headers that you set. As for the MySQL part, I use SQLAlchemy exclusively from inside a function called by deferToThread. Spyne integrates with SQLAlchemy as well, so you can return directly what SQLA returns, be it objects or rows. Here's a simple example: https://github.com/arskom/spyne/blob/master/examples/twisted/resource.py There's a boilerplate generator here: http://spyne.io/ I hope you find it useful. Any questions, people@spyne.io is also at your service. Best, Burak
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Oct 25, 2015, at 5:13 AM, Burak Arslan <burak.arslan@arskom.com.tr> wrote:
It's a bit different from what you to with stock Twisted though. More specifically, there's no NOT_DONE_YET.
I should just mention that NOT_DONE_YET actually _predates_ Deferreds, and one glorious day, we will get rid of it :). See for example https://tm.tl/288 <https://tm.tl/288>. -glyph
![](https://secure.gravatar.com/avatar/2dcefdfea4fae17911a38da01edaa77d.jpg?s=120&d=mm&r=g)
I'm planning to design some REST APIs with Twisted in Python. For example, I want to use the HTTP method "GET" to fetch a single user's information: GET http://myhost:8000/api/v1.0/users/[user_id] I know I should inherit the twisted.web.resource.Resource and implement "getChild" by myself. The question is, should I implement a class for each segment of the URI? If so, I have to implement class API, class V1, class Users and Class User. In other words, if there're 10 segments in the URI, do I have to implement 10 classes to represent those resources? At 2015-10-25 20:13:58, "Burak Arslan" <burak.arslan@arskom.com.tr> wrote: Hey! On 10/25/15 13:04, Wang Yan wrote: Hi, I'm confused about how to design REST APIs with Twisted, especially when my app have to interact with MySQL. Is there any basic design patterns for this kind of situation? I do Spyne, and I think it's fa-bu-lous for building any web API on top of twisted. It's a bit different from what you to with stock Twisted though. More specifically, there's no NOT_DONE_YET. The client hangs as long as you keep returning `Deferred`s. Once you return a non-deferred (and hopefully something that's compatible with your designated return type) that object is serialized using the `out_protocol` you pass to your `Application` and the resulting byte stream is written to the outgoing stream of your transport of choice. In case of a HTTP-based api, it's passed to the `transport.write()` function of a `twisted.web.Resource` instance, along with the outgoing headers that you set. As for the MySQL part, I use SQLAlchemy exclusively from inside a function called by deferToThread. Spyne integrates with SQLAlchemy as well, so you can return directly what SQLA returns, be it objects or rows. Here's a simple example: https://github.com/arskom/spyne/blob/master/examples/twisted/resource.py There's a boilerplate generator here: http://spyne.io/ I hope you find it useful. Any questions, people@spyne.io is also at your service. Best, Burak
![](https://secure.gravatar.com/avatar/b6ca8106c6e4a099f39eaddfe2b21100.jpg?s=120&d=mm&r=g)
On 11/3/15 10:15 PM, Wang Yan wrote:
Twisted is too low-level for such use in my opinion. We use Klein (https://github.com/twisted/klein) which offers nice abstraction above Twisted. I know this does not answer your question. Just wanted to make sure that you are aware of possible options. -Shakkhar [...]
![](https://secure.gravatar.com/avatar/2dcefdfea4fae17911a38da01edaa77d.jpg?s=120&d=mm&r=g)
Thank you to remind me of Klein. I'm very new to Twisted and Klein. What I've known is that Klein is a little like Flask, with which I can implement a UserAPI as follows: from flask import Flask from flask.ext.restful import Api, Resource api = Flask(__name__) api = Api(app) class UserAPI(Resource): def get(self, user_id): pass def post(self, user_id): pass api.add_resource(UserAPI, '/users/<int:id>', endpoint='user') Is there any similar usage in Klein? 在 2015-11-04 12:09:29,"Mashiat Sarker Shakkhar" <mashiat.sarker@gmail.com> 写道: On 11/3/15 10:15 PM, Wang Yan wrote: I'm planning to design some REST APIs with Twisted in Python. For example, I want to use the HTTP method "GET" to fetch a single user's information: GET http://myhost:8000/api/v1.0/users/[user_id] I know I should inherit the twisted.web.resource.Resource and implement "getChild" by myself. The question is, should I implement a class for each segment of the URI? If so, I have to implement class API, class V1, class Users and Class User. In other words, if there're 10 segments in the URI, do I have to implement 10 classes to represent those resources? Twisted is too low-level for such use in my opinion. We use Klein (https://github.com/twisted/klein) which offers nice abstraction above Twisted. I know this does not answer your question. Just wanted to make sure that you are aware of possible options. -Shakkhar [...]
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
Yes; in fact this extension for Flask is similar to how twisted.web works internally. I think you're looking for something like this: from klein import run, route from twisted.web.resource import Resource class User(Resource, object): def __init__(self, user_id): super(User, self).__init__() self.user_id = user_id def render_GET(self, request): pass def render_POST(self, request): pass @route("/users/<int:user_id>") def user(request, user_id): return User(user_id) run("localhost", 8080)
![](https://secure.gravatar.com/avatar/2dcefdfea4fae17911a38da01edaa77d.jpg?s=120&d=mm&r=g)
Absolutely that's what I'm looking for! Thanks a lot! However, I'm wondering if Twisted and Klein support filtering results. For example, the User table in MySQL contains id, name and age fields. How can I get all users whose name are "Alan" and age > 16? It seems I need a filter, but I'm not sure how to implement such a filter or resource with Twisted and Klein. 在 2015-11-06 06:02:53,"Glyph Lefkowitz" <glyph@twistedmatrix.com> 写道: On Nov 5, 2015, at 1:28 AM, Wang Yan <snailcoder@163.com> wrote: Thank you to remind me of Klein. I'm very new to Twisted and Klein. What I've known is that Klein is a little like Flask, with which I can implement a UserAPI as follows: from flask import Flask from flask.ext.restful import Api, Resource api = Flask(__name__) api = Api(app) class UserAPI(Resource): def get(self, user_id): pass def post(self, user_id): pass api.add_resource(UserAPI, '/users/<int:id>', endpoint='user') Is there any similar usage in Klein? Yes; in fact this extension for Flask is similar to how twisted.web works internally. I think you're looking for something like this: from klein import run, route from twisted.web.resource import Resource class User(Resource, object): def __init__(self, user_id): super(User, self).__init__() self.user_id = user_id def render_GET(self, request): pass def render_POST(self, request): pass @route("/users/<int:user_id>") def user(request, user_id): return User(user_id) run("localhost", 8080)
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Nov 12, 2015, at 7:33 AM, Wang Yan <snailcoder@163.com> wrote:
Absolutely that's what I'm looking for! Thanks a lot!
Glad I could help!
However, I'm wondering if Twisted and Klein support filtering results. For example, the User table in MySQL contains id, name and age fields. How can I get all users whose name are "Alan" and age > 16? It seems I need a filter, but I'm not sure how to implement such a filter or resource with Twisted and Klein.
Klein is pretty purely a web-facing framework; it just helps you built the HTTP stuff that answers queries. It doesn't help you talk to a database at all; you have to figure out your own way to do that. This is historically a weak area for Twisted, sadly; you might find https://pypi.python.org/pypi/twextpy/ <https://pypi.python.org/pypi/twextpy/> useful as this brings the state of the art forward somewhat. https://twistedmatrix.com/trac/ticket/7917 <https://twistedmatrix.com/trac/ticket/7917> is a ticket to bring that stuff into Twisted itself; if you are interested in Twisted/Database interaction, perhaps you could help split up the code there into smaller branches that can be merged. Thanks! -glyph
participants (4)
-
Burak Arslan
-
Glyph Lefkowitz
-
Mashiat Sarker Shakkhar
-
Wang Yan