[Twisted-Python] gRPC support in Twisted Python
![](https://secure.gravatar.com/avatar/1044e0807067dc2998e3f8739f7e3bb4.jpg?s=120&d=mm&r=g)
Hello, Is there a plan (or an implementation) to support gRPC within Twisted Python? My understanding is that gRPC is built using Futures and creates its own threads for all its event handling. There is also a gRPC Python package (grpcio 1.0.0) that is available for python 2.7. In order to use gRPC with Twisted Python in 2.7 is the only way to have gRPC run in its own thread? Thanks Khen
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Sep 28, 2016, at 6:13 AM, Nursimulu, Khen <knursimu@ciena.com> wrote:
Is there a plan (or an implementation) to support gRPC within Twisted Python? My understanding is that gRPC is built using Futures and creates its own threads for all its event handling. There is also a gRPC Python package (grpcio 1.0.0) that is available for python 2.7. In order to use gRPC with Twisted Python in 2.7 is the only way to have gRPC run in its own thread?
There's no plan that I'm aware of. You could definitely run gRPC in a thread currently, although it would be nice if grpc worked natively with Twisted. Probably contributing this upstream into the gRPC project would be the best way to start, and if they're not receptive, starting a separate 'txgRPC' project. -glyph
![](https://secure.gravatar.com/avatar/1044e0807067dc2998e3f8739f7e3bb4.jpg?s=120&d=mm&r=g)
Thanks Glyth for the prompt response. From: <twisted-python-bounces@twistedmatrix.com> on behalf of Glyph Lefkowitz <glyph@twistedmatrix.com> Reply-To: "twisted-python@twistedmatrix.com" <twisted-python@twistedmatrix.com> Date: Wednesday, September 28, 2016 at 2:15 PM To: "twisted-python@twistedmatrix.com" <twisted-python@twistedmatrix.com> Subject: Re: [Twisted-Python] gRPC support in Twisted Python On Sep 28, 2016, at 6:13 AM, Nursimulu, Khen <knursimu@ciena.com<mailto:knursimu@ciena.com>> wrote: Is there a plan (or an implementation) to support gRPC within Twisted Python? My understanding is that gRPC is built using Futures and creates its own threads for all its event handling. There is also a gRPC Python package (grpcio 1.0.0) that is available for python 2.7. In order to use gRPC with Twisted Python in 2.7 is the only way to have gRPC run in its own thread? There's no plan that I'm aware of. You could definitely run gRPC in a thread currently, although it would be nice if grpc worked natively with Twisted. Probably contributing this upstream into the gRPC project would be the best way to start, and if they're not receptive, starting a separate 'txgRPC' project. -glyph
![](https://secure.gravatar.com/avatar/84965d0d894b45875d8045bc5762ba57.jpg?s=120&d=mm&r=g)
[+Nathaniel] https://groups.google.com/forum/m/#!topic/grpc-io/RpkyqqQy8TU/discussion Hi. I'd like to link you to the above discussion and cc it's original poster. On Wed, Sep 28, 2016, 22:51 Werner Thie <werner@thieprojects.ch> wrote:
![](https://secure.gravatar.com/avatar/01c6deeb9d1a1053553d353caa464581.jpg?s=120&d=mm&r=g)
If you're planning on working on this, it'd probably be worth taking a look at the Twisted bindings for Thrift (a competing RPC system from FB, based on the Google PB paper). The generator[2] isn't the best place to start (imperative C++ that outputs python), but the code that it generates served us very well for years (generally, communicating twisted->cassandra, or twisted->scala services). [1] https://thrift.apache.org/ [2] https://github.com/apache/thrift/blob/master/compiler/cpp/src/thrift/generat... On 29 September 2016 at 08:34, Stephen Thorne <stephen@thorne.id.au> wrote:
![](https://secure.gravatar.com/avatar/1212e163b57fdc15fe3b1c10f455d0ab.jpg?s=120&d=mm&r=g)
On Thu, Sep 29, 2016 at 5:41 PM, James Broadhead <jamesbroadhead@gmail.com> wrote:
If you're planning on working on this,
We don't currently have plans to implement it ourselves; we're trying to invoke the magic of open source. So far the feature requests we've gotten have been all over the map (gevent <http://www.gevent.org/>/asyncio <https://docs.python.org/3/whatsnew/3.4.html#asyncio>/async-await <https://docs.python.org/3/whatsnew/3.5.html#pep-492-coroutines-with-async-an...>) that what's most important right now is just fostering the conversation among everyone who is interested about what the desired behavior of gRPC Python will be. it'd probably be worth taking a look at the Twisted bindings for Thrift (a
Please contribute this to the thread <https://groups.google.com/d/topic/grpc-io/RpkyqqQy8TU/discussion>/design document <https://github.com/grpc/grpc-experiments/blob/master/python/async/README.md> /conversation. -Nathaniel
![](https://secure.gravatar.com/avatar/20f06bf01a3d002acdf02cd8feacb237.jpg?s=120&d=mm&r=g)
On Thu, Oct 6, 2016 at 11:49 AM, Nathaniel Manista <nathaniel@google.com> wrote:
![](https://secure.gravatar.com/avatar/1212e163b57fdc15fe3b1c10f455d0ab.jpg?s=120&d=mm&r=g)
On Thu, Oct 6, 2016 at 12:11 PM, Manish Tomar <manish.tomar@gmail.com> wrote:
It will be nice if it can be implemented without IO [1] as Cory rightfully points out everywhere.
I'm a big fan of sans I/O; I think it will be interesting to see how much it can be applied to the C-language library that implements most of the behavior of gRPC Python. -N
![](https://secure.gravatar.com/avatar/214c694acb154321379cbc58dc91528c.jpg?s=120&d=mm&r=g)
On this note, if anyone wanted to write a greenfield gRPC implementation for Twisted directly, rather than using the C++ library that Nathaniel has been working on, hyper-h2 and Twisted’s built-in HTTP/2 logic is likely to be a very useful leaping off point. My high-level understanding of gRPC is that at a wire-protocol level the other things required are some protocol buffers and some code generation. That’s definitely not *nothing*, but neither is it the same as implementing the whole thing in one go. Cory
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Sep 28, 2016, at 6:13 AM, Nursimulu, Khen <knursimu@ciena.com> wrote:
Is there a plan (or an implementation) to support gRPC within Twisted Python? My understanding is that gRPC is built using Futures and creates its own threads for all its event handling. There is also a gRPC Python package (grpcio 1.0.0) that is available for python 2.7. In order to use gRPC with Twisted Python in 2.7 is the only way to have gRPC run in its own thread?
There's no plan that I'm aware of. You could definitely run gRPC in a thread currently, although it would be nice if grpc worked natively with Twisted. Probably contributing this upstream into the gRPC project would be the best way to start, and if they're not receptive, starting a separate 'txgRPC' project. -glyph
![](https://secure.gravatar.com/avatar/1044e0807067dc2998e3f8739f7e3bb4.jpg?s=120&d=mm&r=g)
Thanks Glyth for the prompt response. From: <twisted-python-bounces@twistedmatrix.com> on behalf of Glyph Lefkowitz <glyph@twistedmatrix.com> Reply-To: "twisted-python@twistedmatrix.com" <twisted-python@twistedmatrix.com> Date: Wednesday, September 28, 2016 at 2:15 PM To: "twisted-python@twistedmatrix.com" <twisted-python@twistedmatrix.com> Subject: Re: [Twisted-Python] gRPC support in Twisted Python On Sep 28, 2016, at 6:13 AM, Nursimulu, Khen <knursimu@ciena.com<mailto:knursimu@ciena.com>> wrote: Is there a plan (or an implementation) to support gRPC within Twisted Python? My understanding is that gRPC is built using Futures and creates its own threads for all its event handling. There is also a gRPC Python package (grpcio 1.0.0) that is available for python 2.7. In order to use gRPC with Twisted Python in 2.7 is the only way to have gRPC run in its own thread? There's no plan that I'm aware of. You could definitely run gRPC in a thread currently, although it would be nice if grpc worked natively with Twisted. Probably contributing this upstream into the gRPC project would be the best way to start, and if they're not receptive, starting a separate 'txgRPC' project. -glyph
![](https://secure.gravatar.com/avatar/84965d0d894b45875d8045bc5762ba57.jpg?s=120&d=mm&r=g)
[+Nathaniel] https://groups.google.com/forum/m/#!topic/grpc-io/RpkyqqQy8TU/discussion Hi. I'd like to link you to the above discussion and cc it's original poster. On Wed, Sep 28, 2016, 22:51 Werner Thie <werner@thieprojects.ch> wrote:
![](https://secure.gravatar.com/avatar/01c6deeb9d1a1053553d353caa464581.jpg?s=120&d=mm&r=g)
If you're planning on working on this, it'd probably be worth taking a look at the Twisted bindings for Thrift (a competing RPC system from FB, based on the Google PB paper). The generator[2] isn't the best place to start (imperative C++ that outputs python), but the code that it generates served us very well for years (generally, communicating twisted->cassandra, or twisted->scala services). [1] https://thrift.apache.org/ [2] https://github.com/apache/thrift/blob/master/compiler/cpp/src/thrift/generat... On 29 September 2016 at 08:34, Stephen Thorne <stephen@thorne.id.au> wrote:
![](https://secure.gravatar.com/avatar/1212e163b57fdc15fe3b1c10f455d0ab.jpg?s=120&d=mm&r=g)
On Thu, Sep 29, 2016 at 5:41 PM, James Broadhead <jamesbroadhead@gmail.com> wrote:
If you're planning on working on this,
We don't currently have plans to implement it ourselves; we're trying to invoke the magic of open source. So far the feature requests we've gotten have been all over the map (gevent <http://www.gevent.org/>/asyncio <https://docs.python.org/3/whatsnew/3.4.html#asyncio>/async-await <https://docs.python.org/3/whatsnew/3.5.html#pep-492-coroutines-with-async-an...>) that what's most important right now is just fostering the conversation among everyone who is interested about what the desired behavior of gRPC Python will be. it'd probably be worth taking a look at the Twisted bindings for Thrift (a
Please contribute this to the thread <https://groups.google.com/d/topic/grpc-io/RpkyqqQy8TU/discussion>/design document <https://github.com/grpc/grpc-experiments/blob/master/python/async/README.md> /conversation. -Nathaniel
![](https://secure.gravatar.com/avatar/20f06bf01a3d002acdf02cd8feacb237.jpg?s=120&d=mm&r=g)
On Thu, Oct 6, 2016 at 11:49 AM, Nathaniel Manista <nathaniel@google.com> wrote:
![](https://secure.gravatar.com/avatar/1212e163b57fdc15fe3b1c10f455d0ab.jpg?s=120&d=mm&r=g)
On Thu, Oct 6, 2016 at 12:11 PM, Manish Tomar <manish.tomar@gmail.com> wrote:
It will be nice if it can be implemented without IO [1] as Cory rightfully points out everywhere.
I'm a big fan of sans I/O; I think it will be interesting to see how much it can be applied to the C-language library that implements most of the behavior of gRPC Python. -N
![](https://secure.gravatar.com/avatar/214c694acb154321379cbc58dc91528c.jpg?s=120&d=mm&r=g)
On this note, if anyone wanted to write a greenfield gRPC implementation for Twisted directly, rather than using the C++ library that Nathaniel has been working on, hyper-h2 and Twisted’s built-in HTTP/2 logic is likely to be a very useful leaping off point. My high-level understanding of gRPC is that at a wire-protocol level the other things required are some protocol buffers and some code generation. That’s definitely not *nothing*, but neither is it the same as implementing the whole thing in one go. Cory
participants (8)
-
Cory Benfield
-
Glyph Lefkowitz
-
James Broadhead
-
Manish Tomar
-
Nathaniel Manista
-
Nursimulu, Khen
-
Stephen Thorne
-
Werner Thie