[Twisted-Python] Chunked decoding missing from HTTPClient?
I see that HTTPChannel knows about chunking and indeed works well. However shouldn't HTTPClient also support chunking? Or Am I missing something to get chunking working for the response from a server? Barry
On Tue, Aug 15, 2017 at 8:43 AM, Barry Scott <barry.scott@forcepoint.com> wrote:
I see that HTTPChannel knows about chunking and indeed works well.
However shouldn't HTTPClient also support chunking? Or Am I missing something to get chunking working for the response from a server?
Take a look at http://twistedmatrix.com/documents/current/web/howto/client.html which documents the newer HTTP client API in Twisted. You may also want to have a look at https://treq.readthedocs.io/en/latest/ which provides another layer on top of that which may be even easier to use for your purposes. HTTPClient is old, implements HTTP/1.0, and will probably be deprecated and removed before too much longer. Jean-Paul
On Tuesday, 15 August 2017 14:12:21 BST Jean-Paul Calderone wrote:
On Tue, Aug 15, 2017 at 8:43 AM, Barry Scott <barry.scott@forcepoint.com>
wrote:
I see that HTTPChannel knows about chunking and indeed works well.
However shouldn't HTTPClient also support chunking? Or Am I missing something to get chunking working for the response from a server?
Take a look at http://twistedmatrix.com/documents/current/web/howto/client.html which documents the newer HTTP client API in Twisted.
I'm interested in using a subclass of HTTPClientParser with Agent. It does not seem that I can do this simply and have Agent use my class. I see that its hardwired in HTTP11ClientProtocol.request(). Is this possible?
You may also want to have a look at https://treq.readthedocs.io/en/latest/ which provides another layer on top of that which may be even easier to use for your purposes.
Looks nice, but too high level for our use.
HTTPClient is old, implements HTTP/1.0, and will probably be deprecated and removed before too much longer.
Got it.
Jean-Paul
Barry
On Mon, Aug 21, 2017 at 5:52 AM, Barry Scott <barry.scott@forcepoint.com> wrote:
On Tuesday, 15 August 2017 14:12:21 BST Jean-Paul Calderone wrote:
On Tue, Aug 15, 2017 at 8:43 AM, Barry Scott <barry.scott@forcepoint.com
wrote:
I see that HTTPChannel knows about chunking and indeed works well.
However shouldn't HTTPClient also support chunking? Or Am I missing something to get chunking working for the response from a server?
Take a look at http://twistedmatrix.com/documents/current/web/howto/client.html which documents the newer HTTP client API in Twisted.
I'm interested in using a subclass of HTTPClientParser with Agent. It does not seem that I can do this simply and have Agent use my class.
I see that its hardwired in HTTP11ClientProtocol.request().
Yep, seems to be.
Is this possible?
If you can convince folks there's a good reason to want to do this, you can probably get a patch accepted that allows it. Why do you want to do this? Jean-Paul
You may also want to have a look at https://treq.readthedocs.io/en/latest/ which provides another layer on top of that which may be even easier to use for your purposes.
Looks nice, but too high level for our use.
HTTPClient is old, implements HTTP/1.0, and will probably be deprecated
and
removed before too much longer.
Got it.
Jean-Paul
Barry
On Monday, 21 August 2017 11:27:53 BST Jean-Paul Calderone wrote:
On Mon, Aug 21, 2017 at 5:52 AM, Barry Scott <barry.scott@forcepoint.com>
wrote:
On Tuesday, 15 August 2017 14:12:21 BST Jean-Paul Calderone wrote:
On Tue, Aug 15, 2017 at 8:43 AM, Barry Scott <barry.scott@forcepoint.com
wrote:
I see that HTTPChannel knows about chunking and indeed works well.
However shouldn't HTTPClient also support chunking? Or Am I missing something to get chunking working for the response
from a
server?
Take a look at http://twistedmatrix.com/documents/current/web/howto/client.html which documents the newer HTTP client API in Twisted.
I'm interested in using a subclass of HTTPClientParser with Agent. It does not seem that I can do this simply and have Agent use my class.
I see that its hardwired in HTTP11ClientProtocol.request().
Yep, seems to be.
Is this possible?
If you can convince folks there's a good reason to want to do this, you can probably get a patch accepted that allows it.
Why do you want to do this?
There are places in twisted where we hardened it against malformed headers etc. Thoses changes might be something you want to take as a patch or we might need to keep subclassing pieces of the twisted stack. I've got a long way with moving to twisted 17.5 from 2.0, the last piece, I hope, is figuring out how to allow us to add code to the processing without maintaining out of tree patches to twisted itself. Barry
Jean-Paul
You may also want to have a look at https://treq.readthedocs.io/en/latest/ which provides another layer on top of that which may be even easier to use for your purposes.
Looks nice, but too high level for our use.
HTTPClient is old, implements HTTP/1.0, and will probably be deprecated
and
removed before too much longer.
Got it.
Jean-Paul
Barry
participants (2)
-
Barry Scott -
Jean-Paul Calderone