urllib, urllib2, httplib -- Begging for consolidation?

[A. Keyton Weissinger]

Is anyone specifically heading up maintenance/updates/etc to these modules
now? Whose "baby" are these modules?

What do YOU think?

Keyton

> -----Original Message-----
> From: python-list-admin at python.org
> [mailto:python-list-admin at python.org]On Behalf Of brueckd at tbye.com
> Sent: Tuesday, May 07, 2002 11:27 AM
> To: python-list at python.org
> Subject: Re: urllib, urllib2, httplib -- Begging for consolidation?
>
>
> On Tue, 7 May 2002, A. Keyton Weissinger wrote:
>
> > Am I the only one that thinks these need to be pulled together
> some? I saw a
> > PEP (268?) where there are some rumblings about adding some
> things to it as
> > well. Maybe a combo project?
>
> Yes, part of the problem is that it's not obvious when you should use
> which (e.g. urllib vs. urllib2).
>
> BUT, if there were to occur some sort of consolidation (meaning,
> introducing incompatibilities or a whole new module), then we should use
> that as an opportunity to restructure/redesign that whole set of modules
> because, IMO, they've evolved past their original design. If we can come
> up with a good organization, the actual implementation could be
> handled by
> various members of the community.
>
> The original premise of urllib, that it helps your app open any type of
> URL in roughly the same way, is pretty neat but now both urllib and
> urllib2 have lots of stuff tacked on that is pretty HTTP-specific. Also,
> I usually need to support only one protocol and I know in advance which
> that is (usually HTTP, sometimes FTP), but the httplib docs imply that
> httplib is more of an internal module.
>
> So... if we were to change something, I'd like us to build a rich HTTP
> library that supports the super easy use case (gimme the data at this URL,
> optionally posting this data right here first) as well as more complicated
> cases (add in these request headers before sending the request to the
> origin). It would be in this module (or one closely tied to it) that we'd
> capture knowledge about the HTTP protocol, such as parsing and building
> HTTP 1.0 and 1.1 compliant request and response headers, handling cookies,
> basic and digest authentication, '\n' vs. '\r\n' line endings, easy-to-use
> HTTPS, etc. Supporting routines (like quote, urlencode, urlparse) can
> either be imported and exposed through the HTTP module, or kept in a
> module with better definied boundaries.
>
> We could take the same approach with other protocols, and include modules
> for FTP, plain files, etc. With all those in place we could still
> have the
> "open any type of URL" routine built on top, but it should work only for
> the simplest of use cases; if you need something more complex then you'd
> go use the corresponding protocol library yourself.
>
> I'm not suggesting that we scrap the current protocol modules (they've be
> very, very useful); it's just that over time they've grown up and are due
> for some redesign/refactoring (the kind that will not be backwards
> compatible).
>
> -Dave
>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>