[Web-SIG] [stdlib-sig] Choosing one of two options for url* in the stdlib reorg

[M.-A. Lemburg]

Brett Cannon wrote:
> On Fri, Feb 29, 2008 at 11:52 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 2008-02-29 20:20, Brett Cannon wrote:
>>  >>  So, I'd be +1 on the second approach, provided that those
>>  >>  two classes make the transition into url.request as
>>  >>  well. Otherwise, I'm +1 on the first approach and -1
>>  >>  on the second.
>>  >>
>>  >
>>  > Just to make sure I got this straight, as long as the two classes
>>  > without the urllib._urlopener support from urlopen() are moved forward
>>  > you are happy with this?
>>
>>  Yes.
>>
>>
>>  > What about making urllib an external library people can download and
>>  > install using distutils?
>>
>>  Well, removing urllib from the std lib doesn't mean that the module
>>  is gone, but that's true for most modules in the std lib, right ?
>>
>>  The key argument for doing the std lib reorg - as I understand it -
>>  was to be able to have a 2to3.py take care of changing the imports
>>  in a script to make it work on 3.x.
>>
> 
> Yes, as well as to clean out the cruft in the stdlib.

Depends on what you call cruft :-)

Just calling a module urllib2 doesn't make it better or less
crufty. urllib and urllib2 use a different approach to more or
less the same thing.

The complexity of using both is roughly the same. Both have
their ups and downs. Both come with a urlopen() that does the
trick most of the time. Neither implements the full stack of what
you'd need for a web crawler.

It's apples and oranges more than anything else. Some like
apples, others oranges.

>>  If you're now suggesting to move modules out of the way with no
>>  option to automatically port them to 3.x, then you're going far
>>  beyond that original intent.
>>
> 
> No, the modules are already ported to 3.0. This has been done all
> along in order for the test suite in Python to continue to work.
> Making them external just means that if you want to use it in Python
> 3.0 you need to download the source and run the included setup.py to
> install the module. Then 2to3 doesn't have to change anything as the
> original urllib imports can just import the copy downloaded and
> installed into site-packages. The only thing people would have to
> contend with is a Py3KWarning stating that the module has been removed
> from the stdlib but available for download from PyPI.

Again, that applies to most modules in the stdlib.

It's not really an argument for dropping the more used module in
favor of a different module without any real benefit.

>>  My main argument for keeping urllib logic in place is current
>>  use of that module.
> 
> Right, which is why I asked if people thought the current usage was
> small enough in terms of using things other than urlopen() that it
> would be okay to pull it out. And giving people the option to download
> and install a 3.0-compliant version would fill in the gap for anyone
> else who still wants the code.
> 
>> If you look at Google code search (which only
>>  scans OSS software and not even all of it), you get:
>>
>>  import urllib -urllib.py -test_urllib -urllib2
>>         28,800 matches
>>
>>  + urllib.URLopener -FancyURLopener
>>         300 matches
>>
>>  + urllib.FancyURLopener
>>         700 matches
>>
>>  compared to:
>>
>>  import urllib2 -urllib2.py -test_urllib2
>>         10,700 matches
>>
>>  + urllib2.Opener
>>         300 matches
>>
>>  If you compare those results to searches for other modules
>>  in the std lib, you'll find that those figures are high up
>>  on the scale.
> 
> Right, but how many of those urlopen() calls are just urlopen("some
> url"), which has a practical drop-in replacement from urllib2.

That's the wrong question. urlopen() is provided by both
and also the most used API in those modules.

You have to ask yourself whether
it's ok to ask the maintainers of those ~1000 code modules
using urllib for subclassing from the two main classes
URLopener and FancyURLopener to download an external dependency
from PyPI or ship the module with their code.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 29 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611