[Web-SIG] [stdlib-sig] Choosing one of two options for url* in the stdlib reorg
brett at python.org
Sat Mar 1 05:06:27 CET 2008
On Fri, Feb 29, 2008 at 3:47 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> Brett Cannon wrote:
> > On Fri, Feb 29, 2008 at 11:52 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> >> On 2008-02-29 20:20, Brett Cannon wrote:
> >> >> So, I'd be +1 on the second approach, provided that those
> >> >> two classes make the transition into url.request as
> >> >> well. Otherwise, I'm +1 on the first approach and -1
> >> >> on the second.
> >> >>
> >> >
> >> > Just to make sure I got this straight, as long as the two classes
> >> > without the urllib._urlopener support from urlopen() are moved forward
> >> > you are happy with this?
> >> Yes.
> >> > What about making urllib an external library people can download and
> >> > install using distutils?
> >> Well, removing urllib from the std lib doesn't mean that the module
> >> is gone, but that's true for most modules in the std lib, right ?
> >> The key argument for doing the std lib reorg - as I understand it -
> >> was to be able to have a 2to3.py take care of changing the imports
> >> in a script to make it work on 3.x.
> > Yes, as well as to clean out the cruft in the stdlib.
> Depends on what you call cruft :-)
> Just calling a module urllib2 doesn't make it better or less
> crufty. urllib and urllib2 use a different approach to more or
> less the same thing.
> The complexity of using both is roughly the same. Both have
> their ups and downs. Both come with a urlopen() that does the
> trick most of the time. Neither implements the full stack of what
> you'd need for a web crawler.
> It's apples and oranges more than anything else. Some like
> apples, others oranges.
But see, I don't want to manage both an apple tree and orange tree. At
that point people will want cranberries, pineapples. And god forbid we
get into supporting various nuts! =)
Seriously, I just don't want to support two different approaches to
the same problem.
> >> If you're now suggesting to move modules out of the way with no
> >> option to automatically port them to 3.x, then you're going far
> >> beyond that original intent.
> > No, the modules are already ported to 3.0. This has been done all
> > along in order for the test suite in Python to continue to work.
> > Making them external just means that if you want to use it in Python
> > 3.0 you need to download the source and run the included setup.py to
> > install the module. Then 2to3 doesn't have to change anything as the
> > original urllib imports can just import the copy downloaded and
> > installed into site-packages. The only thing people would have to
> > contend with is a Py3KWarning stating that the module has been removed
> > from the stdlib but available for download from PyPI.
> Again, that applies to most modules in the stdlib.
> It's not really an argument for dropping the more used module in
> favor of a different module without any real benefit.
Benefit to old users, no. Benefit to the developers, definitely.
Benefit to new users, yes as there will be less to deal with.
> >> My main argument for keeping urllib logic in place is current
> >> use of that module.
> > Right, which is why I asked if people thought the current usage was
> > small enough in terms of using things other than urlopen() that it
> > would be okay to pull it out. And giving people the option to download
> > and install a 3.0-compliant version would fill in the gap for anyone
> > else who still wants the code.
> >> If you look at Google code search (which only
> >> scans OSS software and not even all of it), you get:
> >> import urllib -urllib.py -test_urllib -urllib2
> >> 28,800 matches
> >> + urllib.URLopener -FancyURLopener
> >> 300 matches
> >> + urllib.FancyURLopener
> >> 700 matches
> >> compared to:
> >> import urllib2 -urllib2.py -test_urllib2
> >> 10,700 matches
> >> + urllib2.Opener
> >> 300 matches
> >> If you compare those results to searches for other modules
> >> in the std lib, you'll find that those figures are high up
> >> on the scale.
> > Right, but how many of those urlopen() calls are just urlopen("some
> > url"), which has a practical drop-in replacement from urllib2.
> That's the wrong question. urlopen() is provided by both
> and also the most used API in those modules.
> You have to ask yourself whether
> it's ok to ask the maintainers of those ~1000 code modules
> using urllib for subclassing from the two main classes
> URLopener and FancyURLopener to download an external dependency
> from PyPI or ship the module with their code.
Well, I obviously think it is. Question is what do other people think.
Can other people weigh in on this? Separate urllib or just toss in the
two classes and continue to support the two separate approaches?
More information about the Web-SIG