[Python-checkins] [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)

[M.-A. Lemburg]

Phillip J. Eby wrote:
> At 07:15 PM 4/18/2006 +0200, M.-A. Lemburg wrote:
>> Why should a 3rd party extension be hot-fixing the standard
>> Python distribution ?
> 
> Because setuptools installs things in zip files, and older versions of 
> pydoc don't work for packages zip files.

That doesn't answer my question.

>> If you want to provide a hot fix for Python 2.3 and 2.4, you
>> should make this a separate install, so that users are aware
>> that their Python distribution is about to get modified in ways
>> that have nothing to do with setuptools.
> 
> Their Python distribution is not "modified" -- new modules are merely 
> placed on sys.path ahead of the stdlib.  (Which, I might add, is a 
> perfectly normal process in Python -- nothing stops users from installing 
> their own version of pydoc or any other module via PYTHONPATH.)
> 
> Note also that uninstalling setuptools by removing the .egg file or 
> directory will effectively remove the hot fix, since the modules live in 
> the .egg, not in the stdlib.

Whether you place a module with the same name in front of
the stdlib path in PYTHONPATH (e.g. copy it into site-packages)
or replace the file in the Python installation is really the
same thing to the user.

Third-party extension *should not do this* !

It's OK to have private modified copies of a module inside
a package or used inside an application, but
"python setup.py install" should never (effectively)
replace a Python stdlib module with some modified copy
without explicit user interaction.

>> If an application wants to use setuptools for e.g. plugin
>> management, then standard distutils features will get
>> replaced by setuptools implementations which are not compatible
>> to the standard distutils commands, effectively making it
>> impossible to access the original versions.
> 
> Please do a little research before you spread FUD.  The 'pkg_resources' 
> module is used for runtime plugin management, and it does not monkeypatch 
> anything.

I'm talking about the setuptools package which does apply
monkey patching and is needed to manage the download and
installation of plugin eggs, AFAIK.

>> Monkey patching is only a last resort in case nothing
>> else works. In this case, it's not needed, since distutils
>> provides the interfaces needed to extend its command classes
>> via the setup() call.
> 
> The monkeypatching is there so that the easy_install command can build eggs 
> for packages that use the distutils.  It's also there so that other 
> distutils extensions that monkeypatch distutils (and there are a few of 
> them out there) will have a better chance of working with setuptools.
> 
> I originally took a minimally-invasive approach to setuptools-distutils 
> interaction, but it was simply not possible to provide a high-degree of 
> backward and "sideward" compatibility without it.  In fact, I seem to 
> recall finding some behaviors in some versions of distutils that can't be 
> modified without monkeypatching, although the details are escaping me at 
> this moment.

That's a very vague comment.

The distutils mechanism for providing your own command classes
lets you take complete control over distutils if needed.

What's good about it, is that this approach doesn't modify anything
inside distutils at run-time, but does these modifications
on a per-setup()-call basis.

As for setuptools, you import the package and suddenly distutils
isn't what's documented on python.org anymore.

>>> As for discussion, Guido originally brought up the question here a few
>>> months ago, and it's been listed in PEP 356 for a while.  I've also
>>> posted things related to the inclusion both here and in distutils-sig.
>>
>> I know, but the discussions haven't really helped much in
>> getting the setuptools design compatible with standard
>> distutils.
> 
> That's because the job was already done.  :) 

Not much of an argument, if you ask me.

Some of the design decisions you made in setuptools are simply wrong
IMHO and these need to be discussed in a PEP process.

> The setuptools design bends 
> over backwards to be compatible with Python 2.3 and 2.4 versions of 
> distutils, not to mention py2exe, Pyrex, and other distutils extensions, 
> along with the quirky uses of distutils that exist in dozens of distributed 
> Python packages.
> 
> However, I think you and I may perhaps have different definitions of 
> "compatibility".  Mine is that things "just work" and users don't have to 
> do anything special.  For that definition, setuptools is extremely 
> compatible with the standard distutils.  In many situations it's more 
> compatible than the distutils themselves, in that more things "just work".  ;)

You've implemented your own view of "just works". This is fine,
but please remember that Python is a collaborative work, so
design decisions have to be worked out in collaboration as
well.

There aren't all that many things that are wrong in setuptools,
but some of them are essential:

* setuptools should not monkey patch distutils on import

* the standard "python setup.py install" should continue to
  work as documented in the Python documentation; any new install
  command should use a different name, e.g. "install_egg"

* placing too many ZIP files on PYTHONPATH is a bad idea
  since it slows down import searches for all Python
  applications, not just ones relying on eggs; a solution
  to this would be to have a PYTHONEGGPATH which is then
  only scanned by egg-aware modules/applications

* the user should have freedom of choice in whether to
  have her Python installation rely on eggs or not (and
  not only --by-using-some-complicated-options)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 18 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::