At 09:02 PM 4/18/2006 +0200, M.-A. Lemburg wrote:
Phillip J. Eby wrote:
At 07:15 PM 4/18/2006 +0200, M.-A. Lemburg wrote:
Why should a 3rd party extension be hot-fixing the standard Python distribution ?
Because setuptools installs things in zip files, and older versions of pydoc don't work for packages zip files.
That doesn't answer my question.
That is the answer to the question you asked: "why hot-fix?" Because setuptools uses zip files, and older pydocs crash when trying to display the documentation for a package (not module; modules were fine) that is in a zip file.
Third-party extension *should not do this* !
If you install setuptools, you presumably would like for things to work, and the hot fix eliminates a bug that interferes with them working.
I'm not sure, however, what you believe any of that has to do with python-checkins or python-dev. The version of setuptools that will do this is not yet released, and the hotfix aspect will be documented. If it causes actual problems for actual setuptools users, it will be actually fixed or actually removed. :) (Meanwhile, the separately-distributed setuptools package is not a part of Python, any more than the Optik is, despite Python including the 'optparse' package spun off from Optik.)
I'm talking about the setuptools package which does apply monkey patching and is needed to manage the download and installation of plugin eggs, AFAIK.
In which case the monkeypatching is needed, to handle situations like building eggs for third-party non-setuptools libraries listed as dependencies.
You can't have it both ways; either you need setuptools or you don't. If you do, you will almost certainly need the monkeypatching, and you'll definitely need it to build eggs for non-setuptools-based packages.
What's good about it, is that this approach doesn't modify anything inside distutils at run-time, but does these modifications on a per-setup()-call basis.
What's bad about it, is that it provides no options for virtualizing the execution of other packages' setup scripts. The 'run_setup' mechanism was horribly broken in the versions of the distutils that I wrestled with.
(I also don't see anybody stepping up to provide alternative implementations or patches to third-party setup scripts or distutils extensions to deal with this issue.)
There aren't all that many things that are wrong in setuptools, but some of them are essential:
- setuptools should not monkey patch distutils on import
Please propose an alternative, or better still, produce a patch. Be sure that it still allows distutils extensions like py2exe to work. The only real alternative to monkeypatching that I'm aware of for that issue is to actually merge setuptools with the distutils, but my guess is that you'll like that even less. :)
the standard "python setup.py install" should continue to work as documented in the Python documentation; any new install command should use a different name, e.g. "install_egg"
placing too many ZIP files on PYTHONPATH is a bad idea since it slows down import searches for all Python applications, not just ones relying on eggs; a solution to this would be to have a PYTHONEGGPATH which is then only scanned by egg-aware modules/applications
the user should have freedom of choice in whether to have her Python installation rely on eggs or not (and not only --by-using-some-complicated-options)
These questions have been hashed to death on the distutils-sig already, but I will try to summarize my responses here as briefly as practical while still covering the full spectrum of issues. I would begin by saying that the tradeoffs I've made favor inexperienced users at the expense of experts (who I assume are capable of learning to use --root or --single-version-externally managed in order to have finer-grained control).
Your proposals, however, generally favor experts at the expense of the average user, and treat it as axiomatic that the benefits provided by setuptools are not worth having, no matter how small the cost. By contrast, users who *want* to use setuptools for either distribution or installation of packages, would rather it "just work", no matter how much complexity is required behind-the-scenes. They don't care about my cost to implement, they just care that they type "easy_install foo" to get a package from PyPI or "setup.py sdist bdist_egg upload" to put it there.
Therefore, it makes no sense to apply your design approach to setuptools, since by your criteria it wouldn't exist in the first place! After all, expert users can munge distutils extensions to their hearts' content (and you certainly have done so already).
Next, your own measurements posted to the distutils-sig debunked the PYTHONPATH performance question, IMO: if you installed *every Python package listed on PyPI at that time* as a zip file (assuming that the average zipfile directory size was the same as in your tests, around 1100 total packages on PyPI, and linear degradation), Python startup time would have increased by little more than half a second. To me, that says the performance is pretty darn good, since the average user isn't going to install anywhere near that many packages.
Note also that installing eggs with -m (or --multi-version) doesn't put them on sys.path to begin with unless they're needed at runtime, so if you care about performance tweaking, you can just use -m for everything, or put it in your distutils.cfg or ~/.pydistutils.cfg file so it's automatic. Since scripts that depend on eggs can automatically find and add the eggs to sys.path at runtime, they don't need the eggs to be on sys.path.
The reason they are put on sys.path by default is to make the default case of installation "just work". Specifically, it's so that "setup.py install" works the same way as it does with the distutils (in terms of what the users can *do* with a package after installation, not in terms of how the files are laid out).
As far as I can tell, your remaining arguments can be reduced to this:
Users who care about how stuff works inside (i.e. experts) will have to use command-line options to get the old behaviors or to tweak the new ones, when working with packages that *explicitly chose* to use setuptools.
And that is a perfectly valid concern -- for an audience that's quite capable of learning to do something new, however. Also note that nothing stops people from creating packages that warp distutils into installing things in weird places, so arguing that "setup.py install" should always do the same thing is moot anyway; you would not get your wish even if setuptools had never existed.
May I suggest that perhaps a better solution to the actual practical problem here is to make the distutils "install" command accept the options that setuptools supports? Then, people who want it can just use --single-version-externally-managed with any package under Python 2.5 and be guaranteed to get the old-fashioned behavior (that can't be cleanly upgraded or uninstalled without a package manager).
As far as I know, however, most tools needing the old installation layout use "install --root" anyway (including bdist_wininst, bdist_msi, bdist_rpm, etc.), and setuptools automatically switches to compatibility mode in that case. So setuptools "just works" with these tools already.