POST data with 401 authentication using urllib(2)

[Pieter Edelman]

Hi all,

I'm trying to submit some data using a POST request to a HTTP server with
BASIC authentication with python, but I can't get it to work. Since it's
driving me completely nuts, so here's my cry for help.

The server is an elog logbook server (http://midas.psi.ch/elog/). It is
protected with a password and an empty username. I can login both using
urllib and urllib2 (suppose the password is "foobar", the logbook is
running on port 8181, and I need the directory "Artikelen")

urllib:
class AuthenticatedURL(urllib.FancyURLopener) :
  def prompt_user_passwd(this, host, realm):
    return("", "foobar")

url = AuthenticatedURL()
data = ""
f = url.open("http://localhost:8181/Artikelen")

urllib2:
auth_handler = urllib2.HTTPBasicAuthHandler()
auth_handler.add_password('Artikelen', 'localhost:8181', '', 'foobar')
opener = urllib2.build_opener(auth_handler) urllib2.install_opener(opener)
f = urllib2.urlopen('http://localhost:8181/Artikelen')

So far so good. But I want to submit some data via CGI. I can do this by
making a GET request out of the URL:
f = url.open("http://localhost:8181/Artikelen/?cmd=Submit&reply_to=80")
This works as advertised (also with urllib2).

However, since I also need to upload files, I want to do this via a POST
request (as it is donenormally via the web interface). However, if I try
this using either urllib or urllib2, it throws an exception:

data = {"cmd" : "Submit", "reply_to" : "80"}

urllib:
f = url.open("http://localhost:8181/Artikelen/", urllib.urlencode(data))
Traceback (most recent call last):
  File "login.py", line 12, in ?
    f = url.open("http://pde.dyndns.org:8181", data)
  File "/usr/lib/python2.3/urllib.py", line 183, in open
    return getattr(self, name)(url, data)
  File "/usr/lib/python2.3/urllib.py", line 308, in open_http
    return self.http_error(url, fp, errcode, errmsg, headers, data)
  File "/usr/lib/python2.3/urllib.py", line 323, in http_error
    return self.http_error_default(url, fp, errcode, errmsg, headers)
  File "/usr/lib/python2.3/urllib.py", line 551, in http_error_default
    return addinfourl(fp, headers, "http:" + url)
  File "/usr/lib/python2.3/urllib.py", line 837, in __init__
    addbase.__init__(self, fp)
  File "/usr/lib/python2.3/urllib.py", line 787, in __init__
    self.read = self.fp.read
AttributeError: 'NoneType' object has no attribute 'read'

urllib2:
f = urllib2.urlopen('http://localhost:8181/Artikelen/',
  urllib.urlencode(data))
Traceback (most recent call last):
  File "login2.py", line 11, in ?
    f = urllib2.urlopen('http://localhost:8181/Artikelen/', data)
  File "/usr/lib/python2.3/urllib2.py", line 129, in urlopen
    return _opener.open(url, data)
  File "/usr/lib/python2.3/urllib2.py", line 326, in open
    '_open', req)
  File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.3/urllib2.py", line 901, in http_open
    return self.do_open(httplib.HTTP, req)
  File "/usr/lib/python2.3/urllib2.py", line 895, in do_open
    return self.parent.error('http', req, fp, code, msg, hdrs)
  File "/usr/lib/python2.3/urllib2.py", line 346, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.3/urllib2.py", line 659, in http_error_401
    host, req, headers)
  File "/usr/lib/python2.3/urllib2.py", line 638, in http_error_auth_reqed
    return self.retry_http_basic_auth(host, req, realm)
  File "/usr/lib/python2.3/urllib2.py", line 648, in retry_http_basic_auth
    return self.parent.open(req)
  File "/usr/lib/python2.3/urllib2.py", line 326, in open
    '_open', req)
  File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.3/urllib2.py", line 901, in http_open
    return self.do_open(httplib.HTTP, req)
  File "/usr/lib/python2.3/urllib2.py", line 895, in do_open
    return self.parent.error('http', req, fp, code, msg, hdrs)
  File "/usr/lib/python2.3/urllib2.py", line 352, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.3/urllib2.py", line 412, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error -1:

In fact, I get these exact same errors if "data" is empty. I'm
wondering if this is supported in python. I can't find anything about it
in TFM. If it's not supported, does anybody have an idea how to do it? If
it is supported, can anybody tell what I'm dowing wrong?

Any help is greatly appreciated.

Pieter Edelman