[Python-checkins] [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)

M.-A. Lemburg mal at egenix.com
Tue Apr 18 19:15:19 CEST 2006


Phillip J. Eby wrote:
> At 10:55 AM 4/18/2006 +0200, M.-A. Lemburg wrote:
>> Phillip.eby wrote:
>> > Author: phillip.eby
>> > Date: Tue Apr 18 02:59:55 2006
>> > New Revision: 45510
>> >
>> > Modified:
>> >    python/trunk/Lib/pkgutil.py
>> >    python/trunk/Lib/pydoc.py
>> > Log:
>> > Second phase of refactoring for runpy, pkgutil, pydoc, and setuptools
>> > to share common PEP 302 support code, as described here:
>> >
>> > http://mail.python.org/pipermail/python-dev/2006-April/063724.html
>>
>> Shouldn't this new module be named "pkglib" to be in line with
>> the naming scheme used for all the other utility modules, e.g. httplib,
>> imaplib, poplib, etc. ?
> 
> It's not a new module; it was previously a module with only one function
> in it, introduced in Python 2.3.  If it were a new module, I'd have
> inquired about a name for it first.

Didn't realize that. Too bad the time machine didn't work on
this one :-/

>> > pydoc now supports PEP 302 importers, by way of utility functions in
>> > pkgutil, such as 'walk_packages()'.  It will properly document
>> > modules that are in zip files, and is backward compatible to Python
>> > 2.3 (setuptools installs for Python <2.5 will bundle it so pydoc
>> > doesn't break when used with eggs.)
>>
>> Are you saying that the installation of setuptools in Python 2.3
>> and 2.4 will then overwrite the standard pydoc included with
>> those versions ?
> 
> Yes.  As far as I can tell, there were no API changes to pydoc during
> this time, so this is effectively a "hot fix".

Why should a 3rd party extension be hot-fixing the standard
Python distribution ?

> This hot-fixing doesn't apply to setuptools system packages built with
> --root or --single-version-externally-managed, however, so OS vendors
> who build packages that wrap setuptools will have to make an explicit
> decision whether to also apply any fixes.  If they do not, an end-user
> can of course install setuptools in their local PYTHONPATH and the
> hotfix will take effect.

What does setuptools have to do with pydoc ? Why should a user
installing setuptools assume that some arbitrary stdlib modules
get (effectively) replaced by installing setuptools ?

If you want to provide a hot fix for Python 2.3 and 2.4, you
should make this a separate install, so that users are aware
that their Python distribution is about to get modified in ways
that have nothing to do with setuptools.

>> I bothered by the fact that installing setuptools actually changes
>> the standard Python installation by either overriding stdlib modules
>> or monkey-patching them at setuptools import time.
> 
> Please feel free to propose alternative solutions that will still ensure
> that setuptools "just works" for end-users.  Both this and the pydoc
> hotfix are "practicality beats purity" issues.

Not really. I'd consider them design flaws.

distutils is built to be extended without having to monkey-patch
it, e.g. you can easily override commands with your own variants
by supplying them via the cmdclass and distclass keyword arguments
to setup().

By monkey patching distutils during import of setuptools,
you effectively *change* distutils at run-time and not only
for the application space that you implement in setuptools,
but for all the rest of the application.

If an application wants to use setuptools for e.g. plugin
management, then standard distutils features will get
replaced by setuptools implementations which are not compatible
to the standard distutils commands, effectively making it
impossible to access the original versions.

Monkey patching is only a last resort in case nothing
else works. In this case, it's not needed, since distutils
provides the interfaces needed to extend its command classes
via the setup() call.

See e.g. mxSetup.py in the eGenix extensions for an example
of how effective the distutils design can be put to use
without having to introduce lots of unwanted side-effects.

>> Add setuptools to the stdlib ? I'm still missing the PEP for this
>> along with the needed discussion touching among other things,
>> the change of the distutils standard "python setup.py install"
>> to install an egg instead of a site package.
> 
> Setuptools in the stdlib simply means that people wanting to use it can
> import it.  It does not affect programs that do not import it.  It also
> means that "python -m easy_install" is available without having to first
> download ez_setup.py.

Doesn't really seem like much of an argument for the
addition... the above is true for any 3rd party module.

> As for discussion, Guido originally brought up the question here a few
> months ago, and it's been listed in PEP 356 for a while.  I've also
> posted things related to the inclusion both here and in distutils-sig.

I know, but the discussions haven't really helped much in
getting the setuptools design compatible with standard
distutils.

Unless that's being put in place, I'm -1 on the addition,
due to the invasive nature of setuptools and its various
side-effects on systems management.

Note that it only takes one single module in an application
doing "import setuptools" for whatever reason, to have it
modify distutils at run-time.

The PEP is needed to document these effects, discuss their
merits/drawbacks and find solutions to the problems prior to
creating a status quo that's hard to modify after the
fact due to the large install base of the Python distribution.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 18 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


More information about the Python-checkins mailing list