urllib2.UserAgent [was: Re: [Web-SIG] So what's missing?]
John J Lee
jjl at pobox.com
Wed Oct 29 16:51:24 EST 2003
On Wed, 29 Oct 2003, John J Lee wrote:
[...]
> Problems to be solved:
>
> - awkward to dynamically change behaviour of user-agent -- you have
> to build a OpenerDirector every time you want to change things
> - unhelpful separation by default of HTTP and HTTPS
> - unhelpful separation by default of various server authentication
> schemes
> - no ability to do partial fetches
> - no ability to do HEAD and PUT
>
> ...any more?
>
> The last two need changes in the rest of urllib2, of course.
[...]
A few other things this class should handle (eventually) in a friendly
fashion. Some of them require work to httplib / urllib / urllib2.
- timeouts
- connection caching
- robots.txt observance (using existing std. lib. module)
- caching
- convenient debugging (showing redirections, response bodies, etc.)
- cookies
- HTML HEAD section http-equiv handling
- Refresh handling
- seekability of responses (required for doing http-equiv)
- control of From and User-Agent headers; maybe just leave this as-
is: ie. the addheaders attribute
John
More information about the Web-SIG
mailing list