[issue1627] Problem with httplib and Content-Length: -1

[Álvaro Iradier]

New submission from Álvaro Iradier:

When opening an IP Webcam URL with urllib2, the response is a continuous
secuence of Jpeg files from the server, preceded by the following headers:

Server: DM-Web
Content-type:
multipart/x-mixed-replace;boundary=0plm(Pico-Web:Server-Push:Boundary-String)1qaz
Content-length: -1

As you can see, the Content-Type is multipart/x-mixed-replace, and the
Content-Length reported by the server is '-1', which is strange (I guess
it would be better not to report Content-Length)

The problem is trying to read anything from the file-like object
returned by urlopen will block forever. Problem seems to be here, in
httplib.py, class HTTPResponse, method 'begin':

...
        # will the connection close at the end of the response?
        self.will_close = self._check_close()

        # do we have a Content-Length?
        # NOTE: RFC 2616, S4.4, #3 says we ignore this if tr_enc is
"chunked"
        length = self.msg.getheader('content-length')
        if length and not self.chunked:
            try:
                self.length = int(length)
            except ValueError:
                self.length = None
        else:
            self.length = None
...

The length attribute is being set to '-1' which leads to blocking when
reading from the endless stream of data. (See the read method in class
_fileobject, socket.py).

I don't know if this is the right fix, but I would suggest changing:

        length = self.msg.getheader('content-length')
        if length and not self.chunked:

to:

        length = self.msg.getheader('content-length')
        if length >= 0 and not self.chunked:

----------
components: Library (Lib)
messages: 58624
nosy: airadier
severity: major
status: open
title: Problem with httplib and Content-Length: -1
type: behavior
versions: Python 2.5

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue1627>
__________________________________