[Twisted-Python] understanding IConsumer
Hi Team, I am sorry if this is coming off as a newbie question. Its probably because I don't understand it enough. I would like to: 1. connectTCP to a host, port 2. use the transport to write 3. Use consumer to read from the transport what the tcp server has written to I wasn't able to find information on consumer and how it gets triggered when data is available especially in the case I have Thanks, -Roshan
I'm not the best person to respond here but I think you should not want or need to implement your own consumer... Twisted transports and protocols are implemented using producers and consumers. On Thu, Sep 3, 2015 at 3:08 AM, Roshan Cherian <cherian.rosh@gmail.com> wrote:
On Sep 2, 2015, at 6:08 PM, Roshan Cherian <cherian.rosh@gmail.com> wrote:
Hi Team,
Hi Roshan! Sorry it took a while to get back to you. Busy week.
I am sorry if this is coming off as a newbie question. Its probably because I don't understand it enough.
Newbie questions are fine, no need to apologize :).
I would like to:
1. connectTCP to a host, port
You should not use connectTCP directly. Use Endpoints: <https://twistedmatrix.com/documents/15.4.0/core/howto/endpoints.html <https://twistedmatrix.com/documents/15.4.0/core/howto/endpoints.html>>. (But this doesn't affect the rest of your question.)
2. use the transport to write
Should be easy enough; `self.transport.write´.
3. Use consumer to read from the transport what the tcp server has written to
You don't need to use the producer / consumer API just to read from the transport. The transport does this automatically, and calls your protocol's dataReceived method every time data is received from the transport.
I wasn't able to find information on consumer and how it gets triggered when data is available especially in the case I have
I'm not sure what you mean about "how it gets triggered"; the producer and consumer APIs are documented here: <https://twistedmatrix.com/documents/15.4.0/core/howto/producers.html <https://twistedmatrix.com/documents/15.4.0/core/howto/producers.html>> but it would help if you could explain what exactly you're trying to achieve with them. -glyph
Hi Glyph and David, Thanks for your answers and thanks for taking time to answer. It was indeed my wrong understanding. I was trying to understand what HTTPClientChannelRequest does in terms of producer and consumer (Its a lot deep considering me being relatively new at this). Looking further at the code I do understand that we have a producer which generates the necessary data which can be written out using the transport and we have a consumer which works of the data from the dataReceived (I hope my understanding is correct and I don't know how it is buffered and I don't know how the flow tells the consumer that there is data for use). I understand there is https://twistedmatrix.com/documents/15.4.0/core/howto/producers.html, however I was hoping if there is an example of a simple consumer which could be written using the data from dataReceived (that is assuming whatever I understood above is correct). I didn't find enough information on how to create a consumer. Thanks, -Roshan On Mon, Sep 7, 2015 at 11:37 PM, Glyph <glyph@twistedmatrix.com> wrote:
On Sep 8, 2015, at 11:24 PM, Roshan Cherian <cherian.rosh@gmail.com <mailto:cherian.rosh@gmail.com>> wrote:
I understand there is https://twistedmatrix.com/documents/15.4.0/core/howto/producers.html <https://twistedmatrix.com/documents/15.4.0/core/howto/producers.html>, however I was hoping if there is an example of a simple consumer which could be written using the data from dataReceived (that is assuming whatever I understood above is correct). I didn't find enough information on how to create a consumer.
What do you mean by "create a consumer"? You already have an IConsumer created for you: your transport. Do you want to implement your own? For what reason? -glyph
Thanks Glyph for setting my understanding right on this. By create a consumer I meant having to write one purely to understand the concept and have one running example so that when looking at consumers like ProducerStream it will make my understanding more clear. Thanks, -Roshan On Wed, Sep 9, 2015 at 11:40 AM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
The best way to come to a good understanding of IConsumer and IProducer is to write something that transfers large amounts of data over a slow, or heavily asymmetric link, and make sure its memory usage stays consistent; just implementing the methods on IConsumer won't be that illuminating.
I'm not the best person to respond here but I think you should not want or need to implement your own consumer... Twisted transports and protocols are implemented using producers and consumers. On Thu, Sep 3, 2015 at 3:08 AM, Roshan Cherian <cherian.rosh@gmail.com> wrote:
On Sep 2, 2015, at 6:08 PM, Roshan Cherian <cherian.rosh@gmail.com> wrote:
Hi Team,
Hi Roshan! Sorry it took a while to get back to you. Busy week.
I am sorry if this is coming off as a newbie question. Its probably because I don't understand it enough.
Newbie questions are fine, no need to apologize :).
I would like to:
1. connectTCP to a host, port
You should not use connectTCP directly. Use Endpoints: <https://twistedmatrix.com/documents/15.4.0/core/howto/endpoints.html <https://twistedmatrix.com/documents/15.4.0/core/howto/endpoints.html>>. (But this doesn't affect the rest of your question.)
2. use the transport to write
Should be easy enough; `self.transport.write´.
3. Use consumer to read from the transport what the tcp server has written to
You don't need to use the producer / consumer API just to read from the transport. The transport does this automatically, and calls your protocol's dataReceived method every time data is received from the transport.
I wasn't able to find information on consumer and how it gets triggered when data is available especially in the case I have
I'm not sure what you mean about "how it gets triggered"; the producer and consumer APIs are documented here: <https://twistedmatrix.com/documents/15.4.0/core/howto/producers.html <https://twistedmatrix.com/documents/15.4.0/core/howto/producers.html>> but it would help if you could explain what exactly you're trying to achieve with them. -glyph
Hi Glyph and David, Thanks for your answers and thanks for taking time to answer. It was indeed my wrong understanding. I was trying to understand what HTTPClientChannelRequest does in terms of producer and consumer (Its a lot deep considering me being relatively new at this). Looking further at the code I do understand that we have a producer which generates the necessary data which can be written out using the transport and we have a consumer which works of the data from the dataReceived (I hope my understanding is correct and I don't know how it is buffered and I don't know how the flow tells the consumer that there is data for use). I understand there is https://twistedmatrix.com/documents/15.4.0/core/howto/producers.html, however I was hoping if there is an example of a simple consumer which could be written using the data from dataReceived (that is assuming whatever I understood above is correct). I didn't find enough information on how to create a consumer. Thanks, -Roshan On Mon, Sep 7, 2015 at 11:37 PM, Glyph <glyph@twistedmatrix.com> wrote:
On Sep 8, 2015, at 11:24 PM, Roshan Cherian <cherian.rosh@gmail.com <mailto:cherian.rosh@gmail.com>> wrote:
I understand there is https://twistedmatrix.com/documents/15.4.0/core/howto/producers.html <https://twistedmatrix.com/documents/15.4.0/core/howto/producers.html>, however I was hoping if there is an example of a simple consumer which could be written using the data from dataReceived (that is assuming whatever I understood above is correct). I didn't find enough information on how to create a consumer.
What do you mean by "create a consumer"? You already have an IConsumer created for you: your transport. Do you want to implement your own? For what reason? -glyph
Thanks Glyph for setting my understanding right on this. By create a consumer I meant having to write one purely to understand the concept and have one running example so that when looking at consumers like ProducerStream it will make my understanding more clear. Thanks, -Roshan On Wed, Sep 9, 2015 at 11:40 AM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
The best way to come to a good understanding of IConsumer and IProducer is to write something that transfers large amounts of data over a slow, or heavily asymmetric link, and make sure its memory usage stays consistent; just implementing the methods on IConsumer won't be that illuminating.
participants (4)
-
David Stainton -
Glyph -
Glyph Lefkowitz
-
Roshan Cherian