[New-bugs-announce] [issue22231] httplib: unicode url will cause an ascii codec error when combined with a utf-8 string header

[Bob Chen]

New submission from Bob Chen:

Try to run these two script below, and you will understand what I'm talking about.

If you specified an url and it happened to be an unicode string(which is quite common in python because python processes string as unicode and you could possibly get it from somewhere else), and your header contains a utf-8 string converted from a foreign language, like u'呵呵', then the codec error occurred.

File "/usr/lib/python2.7/httplib.py", line 808, in _send_output
    msg = "\r\n".join(self._buffer) 


# -*- encoding: utf-8 -*-
# should fail
import httplib, urllib
params = urllib.urlencode({'@number': 12524, '@type': 'issue', '@action': 'show'})
headers = {"Content-type": "application/x-www-form-urlencoded",
        "Accept": "text/plain", 'notes': u'呵呵'.encode('utf-8')}
conn = httplib.HTTPConnection(u"bugs.python.org")
conn.request("POST", u"http://bugs.python.org/any_url", params, headers)
response = conn.getresponse()
print response.status, response.reason



# -*- encoding: utf-8 -*-
# should be ok
import httplib, urllib
params = urllib.urlencode({'@number': 12524, '@type': 'issue', '@action': 'show'})
headers = {"Content-type": "application/x-www-form-urlencoded",
        "Accept": "text/plain", 'notes': u'呵呵'.encode('utf-8')}
conn = httplib.HTTPConnection(u"bugs.python.org")
conn.request("POST", "http://bugs.python.org/any_url", params, headers)
response = conn.getresponse()
print response.status, response.reason

----------
components: Library (Lib)
messages: 225553
nosy: Bob.Chen
priority: normal
severity: normal
status: open
title: httplib: unicode url will cause an ascii codec error when combined with a utf-8 string header
type: crash
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22231>
_______________________________________