native websockets support available for review
Hello friends, The native websockets branch that I've been puttering with is now available for review, so I'm asking for someone to review it: https://github.com/twisted/twisted/pull/12397/files I'm excited to get this landed and released. WebSockets are probably the most popular and common form of bidirectional asynchronous communication happening on the internet today, and it's been a bit of a weakness that we don't have a simple showcase for that in Twisted. IMAP and SSH are great to have, but they're not the first thing people go looking for these days when they're looking to do something async. We have of course had some access to the WebSockets world via Autobahn, but first-party integration will have some significant advantages: We can put a WebSockets example onto our minimal-examples showcase on twisted.org, because the example is similarly very small, and now included with no imports from other libraries. This is the major thing; a dozen or so lines of code illustrating how to push data from Twisted into a browser that a user can immediately interact with is an important missing piece of the pitch of Twisted both as a library and as a web server. There will be a much simpler onboarding experience to this functionality. Autobahn is still there for anyone who wants WAMP or XBR, but users won't be presented with those confusing concepts when just trying to do simple server-side push. Since the gross private API internals that we rely upon are our own, we don't have the version-skew problems that Autobahn does <https://github.com/crossbario/autobahn-python/blob/7bc85b34e200640ab98a41cfd...>, nor do we need overhead like anyio. Since this is a much more recent implementation it also comes with modern niceties like coroutines rather than inlineCallbacks, comprehensive type annotations, and 100% code coverage. This should also be a pretty easy layup of a review: There is full code coverage and I believe full public API documentation coverage. There's new narrative documentation <https://twisted--12397.org.readthedocs.build/en/12397/web/howto/web-in-60/we...> explaining how to use it. The protocol implementation itself is delegated to the existing, minimal, sans-io wsproto library <https://github.com/python-hyper/wsproto> so there's not a lot of protocol implementation stuff to review. The tests are all the new, good kind (i.e.: no native sockets, no real reactors), using IOPump (and improving that infrastructure a bit as I passed through it), with (hopefully!) no new flakiness or weird slowdowns. The downside is that it is brushing up against 1500 changed lines, with most of that being additions, so it is on the longer side. Thanks in advance for volunteering, -g
Aloha now this is great news! I'm the one who dragged nevow over to Python3 with a lot of help from others and I'm still using nevow/athena for multiuser card game serving. The port of nevow was never reviewed but the code is rock stable, our servers are self hosted on FreeBSD with server uptime measured in years. I always wanted to replace the athena long polling with websockets but didn't find the energy nor the time in the last few years. The nevow framework is of course much bigger than athena but a lot of the concepts in nevow are now firmly rooted in Twisted. This would allow peeling out the athena/LivePage part and make it into a txAthena extension. Is there any interest from people here to work on a lightweight option for creating webapps with Twisted instead of dealing with the JavaScript server side craze? Mahalo, Werner On 4/13/25 15:44, glyph@twistedmatrix.com wrote:
Hello friends,
The native websockets branch that I've been puttering with is now available for review, so I'm asking for someone to review it:
https://github.com/twisted/twisted/pull/12397/files
I'm excited to get this landed and released. WebSockets are probably the most popular and common form of bidirectional asynchronous communication happening on the internet today, and it's been a bit of a weakness that we don't have a simple showcase for that in Twisted. IMAP and SSH are great to have, but they're not the _first_ thing people go looking for these days when they're looking to do something async.
We have of course had some access to the WebSockets world via Autobahn, but first-party integration will have some significant advantages:
* _We can put a WebSockets example onto our minimal-examples showcase on twisted.org, because the example is similarly very small, and now included with no imports from other libraries._ This is the major thing; a dozen or so lines of code illustrating how to push data from Twisted into a browser that a user can _immediately interact with_ is an important missing piece of the pitch of Twisted both as a library and as a web server. * There will be a much simpler onboarding experience to this functionality. Autobahn is still there for anyone who wants WAMP or XBR, but users won't be presented with those confusing concepts when just trying to do simple server-side push. * Since the gross private API internals that we rely upon are our own, we don't have the version-skew problems that Autobahn does <https://github.com/crossbario/autobahn-python/blob/7bc85b34e200640ab98a41cfd...>, nor do we need overhead like anyio. * Since this is a much more recent implementation it also comes with modern niceties like coroutines rather than inlineCallbacks, comprehensive type annotations, and 100% code coverage.
This should also be a pretty easy layup of a review:
* There is full code coverage and I believe full public API documentation coverage. * There's new narrative documentation <https://twisted--12397.org.readthedocs.build/en/12397/web/howto/web-in-60/we...> explaining how to use it. * The protocol implementation itself is delegated to the existing, minimal, sans-io wsproto library <https://github.com/python-hyper/wsproto> so there's not a lot of protocol implementation stuff to review. * The tests are all the new, good kind (i.e.: no native sockets, no real reactors), using IOPump (and improving that infrastructure a bit as I passed through it), with (hopefully!) no new flakiness or weird slowdowns.
The downside is that it _is_ brushing up against 1500 changed lines, with most of that being additions, so it is on the longer side.
Thanks in advance for volunteering,
-g
_______________________________________________ Twisted mailing list --twisted@python.org To unsubscribe send an email totwisted-leave@python.org https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived athttps://mail.python.org/archives/list/twisted@python.org/message/JHHAZXS5ESQ... Code of Conduct:https://twisted.org/conduct
Glad to hear you're happy to see it! If you're maintaining Nevow, I am happy to just give you direct push access to the repo & PyPI. Code review and quality standards are great, but if it just results in an unused and moribund package, that's not great. (I'd love to harmonize it with twisted.web.template of course so we have less duplication, but even that is a maitnenance project which needs momentum.) Anyway, I still do need a code review to get this landed :) Thanks, -g
On Apr 16, 2025, at 2:05 PM, Werner Thie <werner@thieprojects.ch> wrote:
Aloha
now this is great news! I'm the one who dragged nevow over to Python3 with a lot of help from others and I'm still using nevow/athena for multiuser card game serving. The port of nevow was never reviewed but the code is rock stable, our servers are self hosted on FreeBSD with server uptime measured in years.
I always wanted to replace the athena long polling with websockets but didn't find the energy nor the time in the last few years. The nevow framework is of course much bigger than athena but a lot of the concepts in nevow are now firmly rooted in Twisted. This would allow peeling out the athena/LivePage part and make it into a txAthena extension.
Is there any interest from people here to work on a lightweight option for creating webapps with Twisted instead of dealing with the JavaScript server side craze?
Mahalo, Werner
On 4/13/25 15:44, glyph@twistedmatrix.com <mailto:glyph@twistedmatrix.com> wrote:
Hello friends,
The native websockets branch that I've been puttering with is now available for review, so I'm asking for someone to review it:
https://github.com/twisted/twisted/pull/12397/files
I'm excited to get this landed and released. WebSockets are probably the most popular and common form of bidirectional asynchronous communication happening on the internet today, and it's been a bit of a weakness that we don't have a simple showcase for that in Twisted. IMAP and SSH are great to have, but they're not the first thing people go looking for these days when they're looking to do something async.
We have of course had some access to the WebSockets world via Autobahn, but first-party integration will have some significant advantages:
We can put a WebSockets example onto our minimal-examples showcase on twisted.org, because the example is similarly very small, and now included with no imports from other libraries. This is the major thing; a dozen or so lines of code illustrating how to push data from Twisted into a browser that a user can immediately interact with is an important missing piece of the pitch of Twisted both as a library and as a web server. There will be a much simpler onboarding experience to this functionality. Autobahn is still there for anyone who wants WAMP or XBR, but users won't be presented with those confusing concepts when just trying to do simple server-side push. Since the gross private API internals that we rely upon are our own, we don't have the version-skew problems that Autobahn does <https://github.com/crossbario/autobahn-python/blob/7bc85b34e200640ab98a41cfd...>, nor do we need overhead like anyio. Since this is a much more recent implementation it also comes with modern niceties like coroutines rather than inlineCallbacks, comprehensive type annotations, and 100% code coverage.
This should also be a pretty easy layup of a review:
There is full code coverage and I believe full public API documentation coverage. There's new narrative documentation <https://twisted--12397.org.readthedocs.build/en/12397/web/howto/web-in-60/we...> explaining how to use it. The protocol implementation itself is delegated to the existing, minimal, sans-io wsproto library <https://github.com/python-hyper/wsproto> so there's not a lot of protocol implementation stuff to review. The tests are all the new, good kind (i.e.: no native sockets, no real reactors), using IOPump (and improving that infrastructure a bit as I passed through it), with (hopefully!) no new flakiness or weird slowdowns.
The downside is that it is brushing up against 1500 changed lines, with most of that being additions, so it is on the longer side.
Thanks in advance for volunteering,
-g
_______________________________________________ Twisted mailing list -- twisted@python.org <mailto:twisted@python.org> To unsubscribe send an email to twisted-leave@python.org <mailto:twisted-leave@python.org> https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived at https://mail.python.org/archives/list/twisted@python.org/message/JHHAZXS5ESQ... Code of Conduct: https://twisted.org/conduct
Twisted mailing list -- twisted@python.org To unsubscribe send an email to twisted-leave@python.org https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived at https://mail.python.org/archives/list/twisted@python.org/message/TGSDFK2EQ3R... Code of Conduct: https://twisted.org/conduct
Aloha Glyph I never actually reviewed code of the Twisted project but would like to help you land the code. I tried a few times to review code but then gave up, one of the reasons being that I felt intimidated working on code and concepts which often took me too long in my opinion too grasp. Anyway, what's the best way to start the process? Mahalo, Werner On 4/16/25 12:23, glyph@twistedmatrix.com wrote:
Glad to hear you're happy to see it!
If you're maintaining Nevow, I am happy to just give you direct push access to the repo & PyPI. Code review and quality standards are great, but if it just results in an unused and moribund package, that's not great. (I'd love to harmonize it with twisted.web.template of course so we have less duplication, but even that is a maitnenance project which needs momentum.)
Anyway, I still do need a code review to get this landed :)
Thanks,
-g
On Apr 16, 2025, at 2:05 PM, Werner Thie <werner@thieprojects.ch> wrote:
Aloha
now this is great news! I'm the one who dragged nevow over to Python3 with a lot of help from others and I'm still using nevow/athena for multiuser card game serving. The port of nevow was never reviewed but the code is rock stable, our servers are self hosted on FreeBSD with server uptime measured in years.
I always wanted to replace the athena long polling with websockets but didn't find the energy nor the time in the last few years. The nevow framework is of course much bigger than athena but a lot of the concepts in nevow are now firmly rooted in Twisted. This would allow peeling out the athena/LivePage part and make it into a txAthena extension.
Is there any interest from people here to work on a lightweight option for creating webapps with Twisted instead of dealing with the JavaScript server side craze?
Mahalo, Werner
On 4/13/25 15:44, glyph@twistedmatrix.com wrote:
Hello friends,
The native websockets branch that I've been puttering with is now available for review, so I'm asking for someone to review it:
https://github.com/twisted/twisted/pull/12397/files
I'm excited to get this landed and released. WebSockets are probably the most popular and common form of bidirectional asynchronous communication happening on the internet today, and it's been a bit of a weakness that we don't have a simple showcase for that in Twisted. IMAP and SSH are great to have, but they're not the _first_ thing people go looking for these days when they're looking to do something async.
We have of course had some access to the WebSockets world via Autobahn, but first-party integration will have some significant advantages:
* _We can put a WebSockets example onto our minimal-examples showcase on twisted.org, because the example is similarly very small, and now included with no imports from other libraries._ This is the major thing; a dozen or so lines of code illustrating how to push data from Twisted into a browser that a user can _immediately interact with_ is an important missing piece of the pitch of Twisted both as a library and as a web server. * There will be a much simpler onboarding experience to this functionality. Autobahn is still there for anyone who wants WAMP or XBR, but users won't be presented with those confusing concepts when just trying to do simple server-side push. * Since the gross private API internals that we rely upon are our own, we don't have the version-skew problems that Autobahn does <https://github.com/crossbario/autobahn-python/blob/7bc85b34e200640ab98a41cfd...>, nor do we need overhead like anyio. * Since this is a much more recent implementation it also comes with modern niceties like coroutines rather than inlineCallbacks, comprehensive type annotations, and 100% code coverage.
This should also be a pretty easy layup of a review:
* There is full code coverage and I believe full public API documentation coverage. * There's new narrative documentation <https://twisted--12397.org.readthedocs.build/en/12397/web/howto/web-in-60/we...> explaining how to use it. * The protocol implementation itself is delegated to the existing, minimal, sans-io wsproto library <https://github.com/python-hyper/wsproto> so there's not a lot of protocol implementation stuff to review. * The tests are all the new, good kind (i.e.: no native sockets, no real reactors), using IOPump (and improving that infrastructure a bit as I passed through it), with (hopefully!) no new flakiness or weird slowdowns.
The downside is that it _is_ brushing up against 1500 changed lines, with most of that being additions, so it is on the longer side.
Thanks in advance for volunteering,
-g
_______________________________________________ Twisted mailing list --twisted@python.org To unsubscribe send an email totwisted-leave@python.org https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived athttps://mail.python.org/archives/list/twisted@python.org/message/JHHAZXS5ESQ... Code of Conduct:https://twisted.org/conduct
Twisted mailing list -- twisted@python.org To unsubscribe send an email to twisted-leave@python.org https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived at https://mail.python.org/archives/list/twisted@python.org/message/TGSDFK2EQ3R... Code of Conduct: https://twisted.org/conduct
_______________________________________________ Twisted mailing list --twisted@python.org To unsubscribe send an email totwisted-leave@python.org https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived athttps://mail.python.org/archives/list/twisted@python.org/message/NVLIRJOU3XJ... Code of Conduct:https://twisted.org/conduct
Hi Werner On Fri, 18 Apr 2025 at 21:46, Werner Thie <werner@thieprojects.ch> wrote:
Aloha Glyph
I never actually reviewed code of the Twisted project but would like to help you land the code. I tried a few times to review code but then gave up, one of the reasons being that I felt intimidated working on code and concepts which often took me too long in my opinion too grasp.
Anyway, what's the best way to start the process?
You can help by checking the documentation and giving the Web Socket code a try. For example, check that the example work and the code makes sense If the documentation is hard to read, leave a feedback as this is a valid review comment. The documentation and the examples should be easy to follow for anyone with basic knowledge of network programming and WebSockets. You can read the current version of the documentation here: https://twisted--12397.org.readthedocs.build/en/12397/web/howto/web-in-60/we... You can also check that the API documentation makes sense You can few the websocket module docs here https://twisted--12397.org.readthedocs.build/en/12397/api/twisted.web.websoc... You can send any feedback as a comment at https://github.com/twisted/twisted/pull/12397 Cheers
Mahalo, Werner
On 4/16/25 12:23, glyph@twistedmatrix.com wrote:
Glad to hear you're happy to see it!
If you're maintaining Nevow, I am happy to just give you direct push access to the repo & PyPI. Code review and quality standards are great, but if it just results in an unused and moribund package, that's not great. (I'd love to harmonize it with twisted.web.template of course so we have less duplication, but even that is a maitnenance project which needs momentum.)
Anyway, I still do need a code review to get this landed :)
Thanks,
-g
On Apr 16, 2025, at 2:05 PM, Werner Thie <werner@thieprojects.ch> <werner@thieprojects.ch> wrote:
Aloha
now this is great news! I'm the one who dragged nevow over to Python3 with a lot of help from others and I'm still using nevow/athena for multiuser card game serving. The port of nevow was never reviewed but the code is rock stable, our servers are self hosted on FreeBSD with server uptime measured in years.
I always wanted to replace the athena long polling with websockets but didn't find the energy nor the time in the last few years. The nevow framework is of course much bigger than athena but a lot of the concepts in nevow are now firmly rooted in Twisted. This would allow peeling out the athena/LivePage part and make it into a txAthena extension.
Is there any interest from people here to work on a lightweight option for creating webapps with Twisted instead of dealing with the JavaScript server side craze?
Mahalo, Werner
On 4/13/25 15:44, glyph@twistedmatrix.com wrote:
Hello friends,
The native websockets branch that I've been puttering with is now available for review, so I'm asking for someone to review it:
https://github.com/twisted/twisted/pull/12397/files
I'm excited to get this landed and released. WebSockets are probably the most popular and common form of bidirectional asynchronous communication happening on the internet today, and it's been a bit of a weakness that we don't have a simple showcase for that in Twisted. IMAP and SSH are great to have, but they're not the *first* thing people go looking for these days when they're looking to do something async.
We have of course had some access to the WebSockets world via Autobahn, but first-party integration will have some significant advantages:
- *We can put a WebSockets example onto our minimal-examples showcase on twisted.org <http://twisted.org>, because the example is similarly very small, and now included with no imports from other libraries.* This is the major thing; a dozen or so lines of code illustrating how to push data from Twisted into a browser that a user can *immediately interact with* is an important missing piece of the pitch of Twisted both as a library and as a web server. - There will be a much simpler onboarding experience to this functionality. Autobahn is still there for anyone who wants WAMP or XBR, but users won't be presented with those confusing concepts when just trying to do simple server-side push. - Since the gross private API internals that we rely upon are our own, we don't have the version-skew problems that Autobahn does <https://github.com/crossbario/autobahn-python/blob/7bc85b34e200640ab98a41cfd...>, nor do we need overhead like anyio. - Since this is a much more recent implementation it also comes with modern niceties like coroutines rather than inlineCallbacks, comprehensive type annotations, and 100% code coverage.
This should also be a pretty easy layup of a review:
- There is full code coverage and I believe full public API documentation coverage. - There's new narrative documentation <https://twisted--12397.org.readthedocs.build/en/12397/web/howto/web-in-60/we...> explaining how to use it. - The protocol implementation itself is delegated to the existing, minimal, sans-io wsproto library <https://github.com/python-hyper/wsproto> so there's not a lot of protocol implementation stuff to review. - The tests are all the new, good kind (i.e.: no native sockets, no real reactors), using IOPump (and improving that infrastructure a bit as I passed through it), with (hopefully!) no new flakiness or weird slowdowns.
The downside is that it *is* brushing up against 1500 changed lines, with most of that being additions, so it is on the longer side.
Thanks in advance for volunteering,
-g
_______________________________________________ Twisted mailing list -- twisted@python.org To unsubscribe send an email to twisted-leave@python.orghttps://mail.python.org/mailman3/lists/twisted.python.org/ Message archived at https://mail.python.org/archives/list/twisted@python.org/message/JHHAZXS5ESQ... Code of Conduct: https://twisted.org/conduct
_______________________________________________ Twisted mailing list -- twisted@python.org To unsubscribe send an email to twisted-leave@python.org https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived at https://mail.python.org/archives/list/twisted@python.org/message/TGSDFK2EQ3R... Code of Conduct: https://twisted.org/conduct
_______________________________________________ Twisted mailing list -- twisted@python.org To unsubscribe send an email to twisted-leave@python.orghttps://mail.python.org/mailman3/lists/twisted.python.org/ Message archived at https://mail.python.org/archives/list/twisted@python.org/message/NVLIRJOU3XJ... Code of Conduct: https://twisted.org/conduct
_______________________________________________ Twisted mailing list -- twisted@python.org To unsubscribe send an email to twisted-leave@python.org https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived at https://mail.python.org/archives/list/twisted@python.org/message/6Q7TZA4KBHG... Code of Conduct: https://twisted.org/conduct
-- Adi Roiban
Hello Twisted, I'd like to help landing this code too, but exactly as Werner said, reviewing Twisted code can feel a bit intimidating. As an example, I tried this branch and the code samples. They work fine, and are easy to follow along. I also read the PR and Adi's review, and that's when things started to get a bit complicated. I got lost in the subtilities of testing, and interfaces. Just to be clear, this is not a complaint at all :). I fully understand that the Twisted codebase is complex, and that it takes a while to be familiar with. I'm just trying to convey my feeling of "I'd like to help, but I'm struggling to bring something to the PR, apart from some very minor remarks". Twisted is by far my favorite Python project, and I find incredible that it still has momentum almost 25 years after its first release. Thank you Glyph, Adi, and others for your dedication during all these years. -- Greg
Hi Greg Thanks for your message. If you are happy with the documentation and the examples, feel free to leave a message on any PR. This is valid feedback for the devs working on that PR Reviewing the documentation and the example is as important as the code review... maybe even more important. If you have time, you can do some manual testing. For example run some "negative tests", try to trigger some errors or call the API with invalid values. Check that the error message makes sense and it's easy to understand what needs to be done to fix the errors. This is an example where it's very useful to have a review from someone that is not "attached" to the source code. For example, the developer of the PR might observe an error, but might not notice that the error message is not intuitive. There is not much use to have advanced features in Twisted, as long as the API is hard to use or hard to understand and nobody is using that code. Any review is important and you don't have to review the full spectrum of that code. And if we get help with reviewers validating the documentation, it means that other devs can focus their review on the low-level code. Cheers On Tue, 29 Apr 2025 at 17:32, Grégoire JUGE via Twisted <twisted@python.org> wrote:
Hello Twisted,
I'd like to help landing this code too, but exactly as Werner said, reviewing Twisted code can feel a bit intimidating. As an example, I tried this branch and the code samples. They work fine, and are easy to follow along.
I also read the PR and Adi's review, and that's when things started to get a bit complicated. I got lost in the subtilities of testing, and interfaces.
Just to be clear, this is not a complaint at all :). I fully understand that the Twisted codebase is complex, and that it takes a while to be familiar with. I'm just trying to convey my feeling of "I'd like to help, but I'm struggling to bring something to the PR, apart from some very minor remarks".
Twisted is by far my favorite Python project, and I find incredible that it still has momentum almost 25 years after its first release.
Thank you Glyph, Adi, and others for your dedication during all these years.
-- Greg _______________________________________________ Twisted mailing list -- twisted@python.org To unsubscribe send an email to twisted-leave@python.org https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived at https://mail.python.org/archives/list/twisted@python.org/message/5QPHCNBPUGH... Code of Conduct: https://twisted.org/conduct
-- Adi Roiban
This is fantastic progress for Twisted—first-party WebSocket support is huge! Being able to spin up a real-time async example without extra dependencies lowers the bar for experimentation massively. Can already imagine this pairing perfectly with lightweight frontend demos—like real-time location tracking or traffic push alerts using MapQuest https://mapsdirectionsdriving.com/ APIs. Looking forward to seeing this land and be part of the core showcase!
On May 6, 2025, at 12:07 AM, driving directions via Twisted <twisted@python.org> wrote:
This is fantastic progress for Twisted—first-party WebSocket support is huge! Being able to spin up a real-time async example without extra dependencies lowers the bar for experimentation massively. Can already imagine this pairing perfectly with lightweight frontend demos—like real-time location tracking or traffic push alerts using MapQuest https://mapsdirectionsdriving.com/ APIs. Looking forward to seeing this land and be part of the core showcase! _______________________________________________
Thanks for saying so! I think I've responded to all the review feedback (there was a lot of it, and not all of it was in review comments, so I might have passed over a few things) but it still needs an approval from someone: https://github.com/twisted/twisted/pull/12397 -g
participants (5)
-
Adi Roiban -
driving directions -
glyph＠twistedmatrix.com -
Grégoire JUGE -
Werner Thie