Suggestion: "Completing" HTTP support in Python. (was Re: SSL and POST with httplib or urllib

[Paul Robinson]

Robb Shecter wrote:
> 
> Hi,
> 
> I was really suprised to see that HTTP client support in Python isn't
> really quite complete.  What I mean is, I was hoping to find an
> interface at a higher level, like this:
> 
> Wishful example #1
> --------------
> h = HTTP('my.host.name', '/loginscript', 8080)
> h.add_parameter('username', 'Robb')
> (code, msg, headers) = h.post()
> 
> Wishful example #2
> --------------
> h = HTTP('my.host.name', 8080)
> h.add_header('X-local-hack', 'true')
> (code, msg, headers) = h.get()

Surely this level is higher than HTTP.

The way in which forms are posted is described in the W3C's HTML
specification [ http://www.w3.org/TR/1998/REC-html40-19980424 ] - this
isn't part of the HTTP standard. The "protocol" for submitting forms,
setting cookies, sending files (RFC 1867) etc all use HTTP but are not
"part of it".

I'm not so sure why your second example is wishful, as (unless I'm
missing something) that's pretty much the way it works at the moment -
isn't it?

If you are looking for a more high level approach then why not try
something like this:
http://cobweb.quantisci.co.uk/pub/english.cgi/d8014813/cl_forms.py

It's rough and ready...no comments (although it's all obvious) and
probably works at least some of the time.

Basic use is create a form object, add controls to it (with values)
submit it, read response.

For example (login method of a user object - the rest of the object is
only of interest for my own application):

def login(self):
    form = cl_forms.form('', self.authURI)
    form.addElement(cl_forms.text('uname', self.userID))
    form.addElement(cl_forms.text('password', self.passwd))
    from time import time
    form.addElement(cl_forms.hidden('thetime', `time()`))
    #<INPUT type="hidden" name="thetime" value="939835494.803">
    #print form
    errcode, errmsg, headers, data = form.submit(self.server, self)
    if errcode == 200:
	print 'authUser login :', errcode, errmsg
	pass
    elif errcode == 401:
	raise TypeError, 'Unknown authentication method.\nWWW-Authenticate:
'+headers.get('WWW-Authenticate')
    else:
	print 'authUser login :', errcode, errmsg, headers.headers, data

> ...without this kind of interface, I bet there's a lot of duplication
> going on.  And, cookie support would of course also be great.  If
> anything, as an option that could be enabled, where cookie information
> would be managed by an HTTP object from invocation to invocation.

It would seem you're looking for a "browser" rather than a pure HTTP
implementation - which Python already provides. I'm not sure (having
never looked at it) but elements of Grail may be of use.

> So - is anybody else interested in seeing a client interface that
> allows the whole HTTP spec to be used, without getting down into the
> mucky details?  I can see that I'll be working on this for sure.  I'd
> be much more motivated if I knew that my work wasn't going to just sit
> around.  I think this kind of interface should be in the core library.

I think this would be great - although I believe re-phrasing that as "So
- is anybody else interested in seeing a client interface that allows
GETing, POSTing of form data and cookie support to be done, without
getting down into the mucky details of the HTTP representation of
these?" would be more accurate.

I for one, would be interested in this kind of functionality for the
library - my own attitude has been to simply "knock one up" as and when
I've needed it.

	Paul.

-----------------------------------
Business Collaborator Team
Enviros Software Solutions
http://www.businesscollaborator.com
-----------------------------------