"It is shaped the same. The reason you're seeing the error is due to the issue I pointed out above." Just to be clear you mean that proxies can reshape the call in flight? On Mon, Jan 30, 2017 at 6:01 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
On Jan 30, 2017, at 19:44, Kevin Mcintyre <kebin70@gmail.com> wrote:
On Mon, Jan 30, 2017 at 4:59 PM, Jean-Paul Calderone < exarkun@twistedmatrix.com> wrote:
On Mon, Jan 30, 2017 at 7:44 PM, Glyph Lefkowitz <glyph@twistedmatrix.com
wrote:
Gotcha. I guess what I meant was that you shouldn't care about this at the application level, but you're talking about an operational concern, not an application-level concern.
Perhaps this should be a tunable on Agent somehow. Can you file a ticket?
Well... It *is* sort of tuneable, as you implied earlier.
If you pass an IBodyProducer with a non-None length, Agent will send a Content-Length header - not use chunked Transfer-Encoding. FileBodyProducer doesn't know how to determine the length of a StringIO, so you get chunked with this example.
Not quite true: FileBodyProducer *does* know how to compute the length of a StringIO, via seek():
from twisted.web.client import FileBodyProducer from io import BytesIO from cStringIO import StringIO FileBodyProducer(BytesIO(b"some bytes")).length 10L FileBodyProducer(StringIO(b"some bytes")).length 10
If you write the JSON to a regular file and FileBodyProducer(open(the
file)) you'll get a Content-Length request. You could also write a new (trivial) IBodyProducer that does know how to compute the length of a StringIO.
The documentation doesn't exactly spell this out - but the only reason `length` is part of the interface is to be able to generate the Content-Length header.
What would the ticket be? Expanded documentation to make this behavior into an explicit guarantee of the interface? A new toggle somewhere to force Agent.request into one mode or the other (regardless of the performance consequences)?
Rather than "regardless" of the consequences, perhaps just a maximum body size. If we made it an explicit guarantee in Agent, perhaps the interface change would not be in '.request' (which, as a formal interface used both inside and outside of Twisted, is fairly fixed), but rather a new `NonChunkedBodyProducer` concrete class that would implement IBodyProducer in terms of another IBodyProducer which either does or doesn't have a `length`. (Or a function that does same, always returning its argument if `length` is already set...)
And that stuff you said about proxies before was true ... So even if you
can control this in Agent, you're still not *guaranteed* the server will see what you send.
Thanks - that's exactly what I was looking for.
*But note* when using FileBodyProducer(StringIO(json.dumps(~blah))) -- I still see a content-length in the request header as it's received by twisted. Am I correct to assume the request is agnostic...meaning it's shaped the same for twisted as it is for apache.
It is shaped the same. The reason you're seeing the error is due to the issue I pointed out above.
Headers({'host': ['localhost:8080'], 'connection': ['close'], 'content-length': ['671'], 'user-agent': ['PassengerTest']})
-glyph
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python