Hi, Slowly but surely learning more about lxml.html by using it to do some scrapping. I encountered a unicode problem trying to submit the following form. <form name="Lien1" method="POST" action="http://recherche2.assemblee-nationale.fr/resultats_tribun.jsp" id="Lien1"> <input type="hidden" name="id_auteur" value="Aboud Élie"> <input type="hidden" name="nom_auteur" value="Élie Aboud"> <input type="hidden" name="legislature" value="13"> <input type="hidden" name="typedoc" value="Questions"> </form> Which can be found under the Questions link of http://www.assemblee-nationale.fr/13/tribun/fiches_id/267457.asp#P3 ===== UnicodeEncodeError Traceback (most recent call last) /Users/eugene/Documents/Dev/parlorama/code/<ipython console> in <module>() /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/lxml/html/__init__.pyc in submit_form(form, extra_values, open_http) 819 if open_http is None: 820 open_http = open_http_urllib --> 821 return open_http(form.method, form.action, values) 822 823 def open_http_urllib(method, url, values): /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/lxml/html/__init__.pyc in open_http_urllib(method, url, values) 836 data = None 837 else: --> 838 data = urlencode(values) 839 return urlopen(url, data) 840 /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib.pyc in urlencode(query, doseq) 1267 for k, v in query: 1268 k = quote_plus(str(k)) -> 1269 v = quote_plus(str(v)) 1270 l.append(k + '=' + v) 1271 else: UnicodeEncodeError: 'ascii' codec can't encode character u'\xc9' in position 6: ordinal not in range(128) ===== I tried to address the problem by encoding the values in the form fields as suggested here : http://mail.python.org/pipermail/tutor/2007-May/054340.html but in a python shell doing
form.fields['id_auteur'] u'Aboud \xc9lie' form.fields['id_auteur'] = form.fields['id_auteur'].encode('utf-8') [...] ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes
Would welcome advice or guidance ... if I want to make urlopen "happy" I am "displeasing" ElementTree :( Thanks for your help, -- EuGeNe -- I lend my books on COlivri http://www.colivri.org/user/eugene, do you?