urllib2.UserAgent [was: Re: [Web-SIG] So what's missing?]

[John J Lee]

On Wed, 29 Oct 2003, John J Lee wrote:
[...]
> Problems to be solved:
>
>  - awkward to dynamically change behaviour of user-agent -- you have
>    to build a OpenerDirector every time you want to change things
>  - unhelpful separation by default of HTTP and HTTPS
>  - unhelpful separation by default of various server authentication
>    schemes
>  - no ability to do partial fetches
>  - no ability to do HEAD and PUT
>
> ...any more?
>
> The last two need changes in the rest of urllib2, of course.
[...]

A few other things this class should handle (eventually) in a friendly
fashion.  Some of them require work to httplib / urllib / urllib2.

 - timeouts
 - connection caching
 - robots.txt observance (using existing std. lib. module)
 - caching
 - convenient debugging (showing redirections, response bodies, etc.)
 - cookies
 - HTML HEAD section http-equiv handling
 - Refresh handling
 - seekability of responses (required for doing http-equiv)
 - control of From and User-Agent headers; maybe just leave this as-
    is: ie. the addheaders attribute


John