urllib2 httplib.BadStatusLine exception while opening a page on an Oracle HTTP Server

[O Peng]

I'm running into a similar problem with the BadStatusLine.
The source code for httplib.py in the problem is as follows:

class HTTPResponse:
    ...
    def _read_status(self):
        line = self.fp.readline()
        ...
        if not line:
            # Presumably, the server closed the connection before
            # sending a valid response.
            raise BadStatusLine(line)

However, I found that right before the 'raise BadStatusLine(line)'
when I ran the following:

restOfResponse = self.fp.read()
print restOfResponse

restOfResponse is NOT empty.  In fact, when I run self.fp.read() at
the beginning of the begin() function, it is not empty at all.
This leads me to believe there is a bug with the self.fp.readline()
(socket._fileobject.readline()) function.  For me it only fails
sometimes.

This behavior is only observed on Windows, Python 2.5.  Running it on
Mac OS X, Python 2.5 yielded no problems.

On Jan 19, 3:48 pm, ak <akte... at gmail.com> wrote:
> On Jan 19, 10:00 pm, ak <akte... at gmail.com> wrote:
>
>
>
> > Hi everyone,
>
> > I have a problem with urllib2 on this particular url, hosted on an
> > Oracle HTTP Server
>
> >http://www.orange.sk/eshop/sk/portal/catalog.html?type=post&subtype=p...
>
> > which gets 302 redirected tohttps://www.orange.sk/eshop/sk/catalog/post/phones.html,
> > after setting a cookie through the Set-Cookie header field in the 302
> > reply. This works fin with firefox.
>
> > However, with urllib2 and the following code snippet, it doesn't work
>
> > --------
> > import cookiejar
> > import urllib2
>
> > cookiejar = cookielib.LWPCookieJar()
> > opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookiejar))
> > url = 'http://www.orange.sk/eshop/sk/portal/catalog.html?
> > type=post&subtype=phone&null'
> > req = urllib2.Request(url, None)
> > s=opener.open(req)
> > --------
>
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in <module>
> >   File "/usr/lib/python2.5/urllib2.py", line 387, in open
> >     response = meth(req, response)
> >   File "/usr/lib/python2.5/urllib2.py", line 498, in http_response
> >     'http', request, response, code, msg, hdrs)
> >   File "/usr/lib/python2.5/urllib2.py", line 419, in error
> >     result = self._call_chain(*args)
> >   File "/usr/lib/python2.5/urllib2.py", line 360, in _call_chain
> >     result = func(*args)
> >   File "/usr/lib/python2.5/urllib2.py", line 582, in http_error_302
> >     return self.parent.open(new)
> >   File "/usr/lib/python2.5/urllib2.py", line 381, in open
> >     response = self._open(req, data)
> >   File "/usr/lib/python2.5/urllib2.py", line 399, in _open
> >     '_open', req)
> >   File "/usr/lib/python2.5/urllib2.py", line 360, in _call_chain
> >     result = func(*args)
> >   File "/usr/lib/python2.5/urllib2.py", line 1115, in https_open
> >     return self.do_open(httplib.HTTPSConnection, req)
> >   File "/usr/lib/python2.5/urllib2.py", line 1080, in do_open
> >     r = h.getresponse()
> >   File "/usr/lib/python2.5/httplib.py", line 928, in getresponse
> >     response.begin()
> >   File "/usr/lib/python2.5/httplib.py", line 385, in begin
> >     version, status, reason = self._read_status()
> >   File "/usr/lib/python2.5/httplib.py", line 349, in _read_status
> >     raise BadStatusLine(line)
> > httplib.BadStatusLine
>
> > Trying the redirected url directly doesn't work either (trying with
> > Firefox will give an HTML error page, as the cookie is not set yet,
> > but trying with urllib2 gives the same exception as previously,
> > whereas it should return the HTML error page)
> > This works correctly on other urls on this website (http(s)://www.orange.sk).
>
> > Am I doing anything wrong or is this a bug in urllib2 ?
>
> > -- ak
>
> Actually, I was wrong on the last point, this does *not* work onhttps://www.orange.sk(but does onhttp://www.orange.sk). IMHO, this
> means either urllib2 or the server misimplemented HTTPS.
>
> Here's some output with debuglevel=1 :
>
> >>> opener.open(urllib2.Request('http://www.orange.sk/', None, headers))
>
> reply: 'HTTP/1.1 200 OK\r\n'
> header: Date: Mon, 19 Jan 2009 21:44:03 GMT
> header: Server: Oracle-Application-Server-10g/10.1.3.1.0 Oracle-HTTP-
> Server
> header: Set-Cookie:
> JSESSIONID=0a19055a30d630c427bda71d4e26a37ca604b9f590dc.e3eNaNiRah4Pe3aSch8 Sc3yOc40;
> path=/web
> header: Expires: Mon, 19 Jan 2009 21:44:13 GMT
> header: Surrogate-Control: max-age="10"
> header: Content-Type: text/html; charset=ISO-8859-2
> header: X-Cache: MISS fromwww.orange.sk
> header: Connection: close
> header: Transfer-Encoding: chunked
> <addinfourl at 137417292 whose fp = <socket._fileobject object at
> 0x831348c>>
>
> >>> opener.open(urllib2.Request('https://www.orange.sk/', None, headers))
>
> reply: ''
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib/python2.5/urllib2.py", line 381, in open
>     response = self._open(req, data)
>   File "/usr/lib/python2.5/urllib2.py", line 399, in _open
>     '_open', req)
>   File "/usr/lib/python2.5/urllib2.py", line 360, in _call_chain
>     result = func(*args)
>   File "/usr/lib/python2.5/urllib2.py", line 1115, in https_open
>     return self.do_open(httplib.HTTPSConnection, req)
>   File "/usr/lib/python2.5/urllib2.py", line 1080, in do_open
>     r = h.getresponse()
>   File "/usr/lib/python2.5/httplib.py", line 928, in getresponse
>     response.begin()
>   File "/usr/lib/python2.5/httplib.py", line 385, in begin
>     version, status, reason = self._read_status()
>   File "/usr/lib/python2.5/httplib.py", line 349, in _read_status
>     raise BadStatusLine(line)
> httplib.BadStatusLine
>
> As you can see the reply from the server seems empty (which results in
> the BadStatusLine exception)
>
> Any help greatly appreciated.
>
> -- ak