Python as a HTTP Client

[Fredrik Lundh]

David Rasmussen wrote:

> I do know about www.python.org. I do an extensive amount of googling in
> general and searching at python.org before I ask questions such as this.
> I did stumble upon urllib, urllib2 and httplib in the documentation, but
> let me assure you, as a newbie, that finding this documentation doesn't
> make one go "ah, this is what I was looking for".

> Specifically, I can't see from reference documentation whether something
> even smarter or more highlevel exists.

hmm.  so if that was your question, why did you write:

    I am writing a program that has to do some lightweight HTTP
    communication with a webserver on the internet. I haven't checked, but
    I'm sure I could do something lowlevel like opening a socket myself and
    then send/receive everything myself on this (how do I do that?), but
    I'd bet that Python have some module which is more high level.
    Something that would just let me connect using an URL, send a few GETs,
    and receive the answer as a string/file etc.

?  ("I haven't checked ... but I'd bet" doesn't really sound like "I've checked
the docs and found a couple of modules that seem to do this, but I wonder
if there is something better out there")

...especially if you had already seen the tutorial's

    Internet Protocols

    There are a number of modules for accessing the internet and processing
    internet protocols. Two of the simplest are urllib2 for retrieving data
    from urls /.../

    (followed by a brief example that shows how to read from an URL)

or the reference guide's

     urllib -- Open arbitrary resources by URL

    This module provides a high-level interface for fetching data across the
    World Wide Web. In particular, the urlopen() function is similar to the built-
    in function open(), but accepts Universal Resource Locators (URLs) instead
    of filenames.

or

    urllib2 -- extensible library for opening URLs

    The urllib2 module defines functions and classes which help in opening URLs
    (mostly HTTP) in a complex world -- basic and digest authentication, re-
    directions, cookies and more.

which all seem to match "something that would just let me connect
using an URL" pretty well.

</F>