Re: [Distutils] Finding dependencies and Limiting download
At 10:13 AM 3/16/2006 -0600, Ian Bicking wrote:
It occurs to me that maybe it would be easiest to create an RPM that includes Paste, PasteDeploy, and PasteScript all together? That would simplify all of this, it seems.
That would certainly fix the conflict problem in the short term. In the long run, I'd like a better solution, though. Separate components for namespace packages are too good a thing to give up on without a fight. I'm tempted to see if there isn't a way to create an import hook in pkg_resources that would allow namespace packages to work without having an __init__.py. That would eliminate the file conflicts, albeit at the expense of requiring pkg_resources to be imported before any namespace packages. That seems problematic, though, since it would only be needed for --svem pacakges. The fundamental problem is that you end up having to have dummy packages to hold the __init__.py, or else you have to have something that gets imported first. It seems a bit fragile. One hack would be to add a .pth file for each --svem install of a namespace package, something like "PasteDeploy-nspkg.pth", that contains some code to activate the namespace package, like: import sys,new; \ m = sys.modules.setdefault('paste',new.module('paste')); \ p = os.path.join(sys._getframe(1).f_locals['sitedir'], 'paste'); \ mp = m.__path__ = getattr(m,'__path__',[]); \ (p not in mp) and mp.append(p) Yeah, that would do the trick, if --svem installs left out the __init__.py for declared namespace packages. Of course, this slows down Python startup time when you have a lot of --svem namespace packages installed, and doesn't work with "python -S", but what else can we do? If a given system packager wants to handle namespace packages differently, we could probably accomodate that via modifiers to the --svem mode. Really, the hardest part of implementing this is going to be the bits of setuptools that need to munge the build_py and install_lib commands to leave out __init__ files for namespace packages. I'll probably also need to document how to deal with certain legacy situations where there is an official package that "owns" the namespace and needs to install an __init__.py that actually contains code.
Phillip J. Eby wrote:
At 10:13 AM 3/16/2006 -0600, Ian Bicking wrote:
It occurs to me that maybe it would be easiest to create an RPM that includes Paste, PasteDeploy, and PasteScript all together? That would simplify all of this, it seems.
That would certainly fix the conflict problem in the short term.
In the long run, I'd like a better solution, though. Separate components for namespace packages are too good a thing to give up on without a fight. I'm tempted to see if there isn't a way to create an import hook in pkg_resources that would allow namespace packages to work without having an __init__.py. That would eliminate the file conflicts, albeit at the expense of requiring pkg_resources to be imported before any namespace packages.
Another solution is at the package level -- RPMs can overwrite each other's files, if they are spec'd out to do that somehow. This is not very clean -- I don't know which if any package is allowed to ultimately delete that file. With some post-install hooks you could just create the file, and delete when there's nothing left in the package. This would have to be handled for every packaging system. But there's only a couple anyone uses. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
At 02:37 PM 3/16/2006 -0600, Ian Bicking wrote:
Another solution is at the package level -- RPMs can overwrite each other's files, if they are spec'd out to do that somehow. This is not very clean -- I don't know which if any package is allowed to ultimately delete that file. With some post-install hooks you could just create the file, and delete when there's nothing left in the package. This would have to be handled for every packaging system. But there's only a couple anyone uses.
Er, define "couple" and "anyone". :) I've heard from folks using Gentoo, RPM, .deb, and win32.exe just to start with. And let's not forget the people doing darwinports and other BSD systems. And that's all only talking about packaging systems that have existing users wrapping setuptools and setuptools-based packages, not general Python-using systems! So it's a bit more than a couple. And I suspect that most, if not all, would just as soon that "setup.py install --root" should "just work" without having to change their build processes, and my proposed solution would ensure that. Indeed, I believe that this fix for namespace packages is probably the last remaining impediment to setuptools' ability to achieve Total World Domination, except for the introduction of .egg-info files in Python 2.5. As far as I can tell, these are the only things left before system packages and user packages can all "just get along".
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday 16 March 2006 22:37, Ian Bicking wrote:
Another solution is at the package level -- RPMs can overwrite each other's files, if they are spec'd out to do that somehow.
Yes, you could do that in the %post section where post installation script can be put. It's executed as root so it can do rm -rf / if needed.
This is not very clean -- I don't know which if any package is allowed to ultimately delete that file.
All packages are installed as root. So any package could do that. There where the trust comes into picture. I trust fedoraproject.org as my OS supplier and they digitally sign every package, so it's unlikely that any package would behave badly in my system.
With some post-install hooks you could just create the file, and delete when there's nothing left in the package. This would have to be handled for every packaging system. But there's only a couple anyone uses.
First, doing so would never be accepted in most Linux distrubtions and it would never pass their QA examination. It would break the package checksums. If I would like to check that 'what is wrong in that software' and would run $ rpm -V postfix S.5....T. c /etc/postfix/main.cf And I can see that that file's size, md5sum and time differes, but it doesn't matter as it is a condifuration file. If that would have listed other files, I would have removed and reinstalled it to fix it. It would also break file listing done with rpm -q -l <package>. Screwing the RPM database from a kitchen door will screw the package metadata and ruin the whole purpose of using RPM. So, deleting files from the package bookeeping would flag the file missing and create an error. I think that there is something wrong in the installation if it overwrites each other's files during the process. BR, Tuju - -- Ajatteleva ihminen tarvitsee unta. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.1 (GNU/Linux) iD8DBQFEGoPUQnPP29fw0xMRAohpAJwPdBcPkc1cohf+W0oXEcKKqhRHSACcDO3w KwJgoVrYabii6SctDZuF2XU= =XrO4 -----END PGP SIGNATURE-----
Juha Tuomala wrote:
On Thursday 16 March 2006 22:37, Ian Bicking wrote:
Another solution is at the package level -- RPMs can overwrite each other's files, if they are spec'd out to do that somehow.
Yes, you could do that in the %post section where post installation script can be put. It's executed as root so it can do rm -rf / if needed.
This is not very clean -- I don't know which if any package is allowed to ultimately delete that file.
All packages are installed as root. So any package could do that. There where the trust comes into picture. I trust fedoraproject.org as my OS supplier and they digitally sign every package, so it's unlikely that any package would behave badly in my system.
Well, the question is which should. E.g., if you install PasteScript, then Paste, then remove PasteScript, PasteScript should leave paste/__init__.py since Paste also needs it. In practice, I tend to notice files like this are just left around by packagers. I don't think that would be appropriate at all in this situation, though (it would mean there would still be an importable "paste" even though there'd be nothing in it).
I think that there is something wrong in the installation if it overwrites each other's files during the process.
This is why Eggs have their namespace package support, but it relies on the packages being installed in separate hierarchies even though they share the same package space. If Phillip can fix that, then great, we should all be happy ;) Yet another option would be to make Paste a requirement for PasteDeploy and PasteScript, and rely on Paste installing paste/__init__.py Namespace packages *are* rather annoying, I'll admit. If I knew when I first wrote them how I wanted paste.* factored into three packages, I probably wouldn't have used a namespace package. The only real reason I see for it is legacy package names, or when you really really like hierarchy (and I don't). The issue with installing as an egg is that either you have to have postinstall scripts handle easy-install.pth (or rpm-install.pth or something), or you have to handle the fact that no egg will be installed by default. But otherwise you can just install an egg instead of using --single-version... -- and I don't know if there's really a reason not to do that at this time. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
At 10:21 AM 3/17/2006 -0600, Ian Bicking wrote:
If Phillip can fix that, then great, we should all be happy ;)
FYI, I've got a patch in my source checkout now and it appears to work great for the simple cases. I still have a few more complex scenarios to test, like bdist_wininst roundtripping with easy_install, and some documentation/release notes to update. I hope to have the fix in Subversion in a few hours.
participants (3)
-
Ian Bicking -
Juha Tuomala -
Phillip J. Eby