[Twisted-Python] Twisted for http/2 (RESTful) webservices - client/server
![](https://secure.gravatar.com/avatar/685c7291c36b2895bfa08cc1334e05c7.jpg?s=120&d=mm&r=g)
From my reading so far, I've come to realize that they don't use the edge-HTTP/2.0 approach i.e. delegate HTTP/2.0 termination to nginx as proxy
Hi, Quick intro - I am new to Python, and come from Java background (with some exposure to NIO). I am part of a small team setup for testing of some 5G telecom network software that have micro-service decomposition and the components communicate with each other using REST interface over HTTP/2.0 (over TLS). then talk HTTP/1.1 internally, but each component is a HTTP/2.0 endpoint (client and server as they switch roles). My understanding is that Twisted framework supports HTTP/2.0 via hyper-h2.
If someone could shed some clarity on the above, and if possible point to some relevant tutorials and samples that may help me get started would be highly appreciated. cheers, Jay
![](https://secure.gravatar.com/avatar/a93db92c60e9d5435d204407264457f2.jpg?s=120&d=mm&r=g)
Jayanth Acharya <jayachar88@gmail.com> writes:
Flask is a threaded/synchronous framework. Klein provides something similar based on Twisted on the server-side, and treq provides something like requests at the client side. Depending on what you're doing, you might also be interested in Crossbar.io and Autobahn which provide RPC and PubSub services. There is a "REST Bridge" here to bridge from plain HTTP (including HTTP/2) services to the WAMP protocol (which does RPC and PubSub). -- meejah
![](https://secure.gravatar.com/avatar/685c7291c36b2895bfa08cc1334e05c7.jpg?s=120&d=mm&r=g)
Thanks Meejah. Appreciate your clearing up that Flask is not based on twisted, not sure I got that impression earlier. Checked Klein, and it seems quite promising (and seem quite similar to Flask, from developer standpoint). From the documentation it was not quite evident to me, as to how to use HTTP2 under Klein. I did find a reference to HTTP2 in this pull-request, but again not clear to me: https://github.com/twisted/klein/pull/113 Checked treq, but is it really advantageous over request ? Also found no reference to HTTP2 in treq documents. Alao thanks for the references to Crossbar.io and Autobahn. Will check them out shortly. cheers, Jay On Fri, May 24, 2019 at 7:52 PM meejah <meejah@meejah.ca> wrote:
![](https://secure.gravatar.com/avatar/a93db92c60e9d5435d204407264457f2.jpg?s=120&d=mm&r=g)
Jayanth Acharya <jayachar88@gmail.com> writes:
Klein uses Twisted Web underneath. So, it's "merely" a matter of installing the right things essentially.
Checked treq, but is it really advantageous over request ? Also found no reference to HTTP2 in treq documents.
Yes, because "requests" is a synchronous library that won't use Twisted at all. As above, treq uses twisted web so it will use http2 if the underlying twisted has been set up to use http2 (by installing the right things). p.s. if you need real-time help/discussion, #twisted on freenode is a friendly and active channel. Cheers, meejah
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On May 24, 2019, at 1:37 PM, Amber Brown <hawkowl@atleastfornow.net> wrote:
To note -- Twisted does not currently have HTTP/2 client support, only server support. Treq will not talk H2 at the moment, but Klein+Twisted will happily serve it up.
This is simply a matter of nobody having implemented it yet though - if anyone would like to hook up twisted.web.client.Agent, and thereby treq, to HTTP/2, that would be great! -g
![](https://secure.gravatar.com/avatar/685c7291c36b2895bfa08cc1334e05c7.jpg?s=120&d=mm&r=g)
Thanks, understood. My Python knowledge is somewhat rudimentary and Twisted knowledge even more so, but if the current project at hand can afford some lax, I will see if I could help in any way. Even though Twisted is an open-source, volunteer driven/implemented project, is there a roadmap for Twisted ? On Sun, May 26, 2019 at 2:01 AM Glyph <glyph@twistedmatrix.com> wrote:
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On May 26, 2019, at 12:19 AM, Jayanth Acharya <jayachar88@gmail.com> wrote:
Thanks, understood. My Python knowledge is somewhat rudimentary and Twisted knowledge even more so, but if the current project at hand can afford some lax, I will see if I could help in any way. Even though Twisted is an open-source, volunteer driven/implemented project, is there a roadmap for Twisted ?
Not really. We do have >2000 open bugs, and we would like to have zero, so that is a roadmap of a kind :). But my attempts at roadmap-setting emails to this list have largely gone unanswered, so I'm not sure we have consensus around big long-term vision. However, in general we want better integration with the stdlib asyncio, more debugging niceties, better practical logging tools, an overhauled HTTP server interface, and to eliminate the remainder of our py2-only legacy (as you can see here: http://blog.habnab.it/twisted-depgraph/ <http://blog.habnab.it/twisted-depgraph/> we're *almost* there!). -g
![](https://secure.gravatar.com/avatar/eee0b758e45f8555f03352f74f723409.jpg?s=120&d=mm&r=g)
Hello, On dv., maig 24 2019, Jayanth Acharya wrote:
Just let me add something that may have been "obvious to everyone" on that PR and not to you at that point, but it will be "obvious" to you after the next 5-20 minutes: Notice how that PR introduces the "endpoints" concept, that is a *very* useful concept in Twisted, which basically allows you to add support for pretty much any transport without any effort. So, if you want to run an HTTP2 server, you need two things: 1. Read this bit of the klein and twisted documentation: https://klein.readthedocs.io/en/latest/examples/alternativerunning.html#exam... https://twistedmatrix.com/documents/current/core/howto/endpoints.html 2. Then you have to install the optional twisted[http2] dependency *and* because HTTP2 requires TLS, you have to enable that by using the endpoints API (e.g. in klein). That's it for the server. Twisted's documentation *could* be a bit better on this, if you want to take a go at it, I'm reasonably sure everyone would be happy. The only mention I found for http2 on twisted docs is: https://twistedmatrix.com/documents/current/installation/howto/optional.html... There is also this article which AFAIK still applies, except the optional dependency is twisted[http2] instead of twisted[h2]: https://pawelmhm.github.io/python/twisted/http2/python3/2016/07/30/twisted-h... Notice that an easy one would be to edit the documentation I linked to before, and add an example that runs an HTTP2 server, future people like you will be able to find a small example with the search bar :-). -- Evilham
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On May 27, 2019, at 2:16 AM, Evilham <contact@evilham.com> wrote:
Twisted's documentation *could* be a bit better on this, if you want to take a go at it, I'm reasonably sure everyone would be happy.
Certainly I am happy any time anyone has a go at improving Twisted's docs - and this is an area I'd wager lots of people are interested in! -g
![](https://secure.gravatar.com/avatar/a93db92c60e9d5435d204407264457f2.jpg?s=120&d=mm&r=g)
Jayanth Acharya <jayachar88@gmail.com> writes:
Flask is a threaded/synchronous framework. Klein provides something similar based on Twisted on the server-side, and treq provides something like requests at the client side. Depending on what you're doing, you might also be interested in Crossbar.io and Autobahn which provide RPC and PubSub services. There is a "REST Bridge" here to bridge from plain HTTP (including HTTP/2) services to the WAMP protocol (which does RPC and PubSub). -- meejah
![](https://secure.gravatar.com/avatar/685c7291c36b2895bfa08cc1334e05c7.jpg?s=120&d=mm&r=g)
Thanks Meejah. Appreciate your clearing up that Flask is not based on twisted, not sure I got that impression earlier. Checked Klein, and it seems quite promising (and seem quite similar to Flask, from developer standpoint). From the documentation it was not quite evident to me, as to how to use HTTP2 under Klein. I did find a reference to HTTP2 in this pull-request, but again not clear to me: https://github.com/twisted/klein/pull/113 Checked treq, but is it really advantageous over request ? Also found no reference to HTTP2 in treq documents. Alao thanks for the references to Crossbar.io and Autobahn. Will check them out shortly. cheers, Jay On Fri, May 24, 2019 at 7:52 PM meejah <meejah@meejah.ca> wrote:
![](https://secure.gravatar.com/avatar/a93db92c60e9d5435d204407264457f2.jpg?s=120&d=mm&r=g)
Jayanth Acharya <jayachar88@gmail.com> writes:
Klein uses Twisted Web underneath. So, it's "merely" a matter of installing the right things essentially.
Checked treq, but is it really advantageous over request ? Also found no reference to HTTP2 in treq documents.
Yes, because "requests" is a synchronous library that won't use Twisted at all. As above, treq uses twisted web so it will use http2 if the underlying twisted has been set up to use http2 (by installing the right things). p.s. if you need real-time help/discussion, #twisted on freenode is a friendly and active channel. Cheers, meejah
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On May 24, 2019, at 1:37 PM, Amber Brown <hawkowl@atleastfornow.net> wrote:
To note -- Twisted does not currently have HTTP/2 client support, only server support. Treq will not talk H2 at the moment, but Klein+Twisted will happily serve it up.
This is simply a matter of nobody having implemented it yet though - if anyone would like to hook up twisted.web.client.Agent, and thereby treq, to HTTP/2, that would be great! -g
![](https://secure.gravatar.com/avatar/685c7291c36b2895bfa08cc1334e05c7.jpg?s=120&d=mm&r=g)
Thanks, understood. My Python knowledge is somewhat rudimentary and Twisted knowledge even more so, but if the current project at hand can afford some lax, I will see if I could help in any way. Even though Twisted is an open-source, volunteer driven/implemented project, is there a roadmap for Twisted ? On Sun, May 26, 2019 at 2:01 AM Glyph <glyph@twistedmatrix.com> wrote:
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On May 26, 2019, at 12:19 AM, Jayanth Acharya <jayachar88@gmail.com> wrote:
Thanks, understood. My Python knowledge is somewhat rudimentary and Twisted knowledge even more so, but if the current project at hand can afford some lax, I will see if I could help in any way. Even though Twisted is an open-source, volunteer driven/implemented project, is there a roadmap for Twisted ?
Not really. We do have >2000 open bugs, and we would like to have zero, so that is a roadmap of a kind :). But my attempts at roadmap-setting emails to this list have largely gone unanswered, so I'm not sure we have consensus around big long-term vision. However, in general we want better integration with the stdlib asyncio, more debugging niceties, better practical logging tools, an overhauled HTTP server interface, and to eliminate the remainder of our py2-only legacy (as you can see here: http://blog.habnab.it/twisted-depgraph/ <http://blog.habnab.it/twisted-depgraph/> we're *almost* there!). -g
![](https://secure.gravatar.com/avatar/eee0b758e45f8555f03352f74f723409.jpg?s=120&d=mm&r=g)
Hello, On dv., maig 24 2019, Jayanth Acharya wrote:
Just let me add something that may have been "obvious to everyone" on that PR and not to you at that point, but it will be "obvious" to you after the next 5-20 minutes: Notice how that PR introduces the "endpoints" concept, that is a *very* useful concept in Twisted, which basically allows you to add support for pretty much any transport without any effort. So, if you want to run an HTTP2 server, you need two things: 1. Read this bit of the klein and twisted documentation: https://klein.readthedocs.io/en/latest/examples/alternativerunning.html#exam... https://twistedmatrix.com/documents/current/core/howto/endpoints.html 2. Then you have to install the optional twisted[http2] dependency *and* because HTTP2 requires TLS, you have to enable that by using the endpoints API (e.g. in klein). That's it for the server. Twisted's documentation *could* be a bit better on this, if you want to take a go at it, I'm reasonably sure everyone would be happy. The only mention I found for http2 on twisted docs is: https://twistedmatrix.com/documents/current/installation/howto/optional.html... There is also this article which AFAIK still applies, except the optional dependency is twisted[http2] instead of twisted[h2]: https://pawelmhm.github.io/python/twisted/http2/python3/2016/07/30/twisted-h... Notice that an easy one would be to edit the documentation I linked to before, and add an example that runs an HTTP2 server, future people like you will be able to find a small example with the search bar :-). -- Evilham
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On May 27, 2019, at 2:16 AM, Evilham <contact@evilham.com> wrote:
Twisted's documentation *could* be a bit better on this, if you want to take a go at it, I'm reasonably sure everyone would be happy.
Certainly I am happy any time anyone has a go at improving Twisted's docs - and this is an area I'd wager lots of people are interested in! -g
participants (5)
-
Amber Brown
-
Evilham
-
Glyph
-
Jayanth Acharya
-
meejah