Mailman 3 March 2006 - Distutils-SIG

Using namespace packages? READ THIS
by Phillip J. Eby March 18, 2006

March 18, 2006

Conversely, if you're not using namespace packages with setuptools, you don't need to read this. Remain calm, everything is fine. :) If you do use them, please read on. Most of what's in here will probably not affect most people anyway, even if they do use namespace packages. But just take a few moments to check and make sure these issues don't apply to you. I just checked in a fix for the issue with namespace packages in system package installation scenarios. This fix unfortunately puts some significant limitations on what you can have in a namespace package __init__.py. Basically, *all* you should have in *every* namespace package __init__.py is: __import__('pkg_resources').declare_namespace(__name__) because the __init__.py files are now *not installed* when using --root or --single-version-externally-managed. So anything else you put in these files, is just not going to work in that case. Now, if you're doing what the ll-libs were doing, where you have one project containing the __init__.py, and the others not containing an __init__.py, and the main package doesn't declare a namespace but the others do, then that scenario will still work. If you don't know what this means, don't worry about it. :) Here's the really goofy part, that you need to deal with if you are currently distributing any win32.exe installers for a project that uses namespace packages: If you build a win32.exe with the latest setuptools in SVN, and your users use easy_install to install it, they *must* upgrade to the latest setuptools as well, or they will get a bunch of stuff about easy_install not knowing how to process various files, and the install will fail. (This is because setuptools now includes additional .pth files to munge namespace package stuff, and the old .exe-to-.egg conversion doesn't know how to handle it.) So anyway, if you care about this scenario (you distribute bdist_wininst win32.exe files but *not* win32.egg files), you should make sure you and your users are on the same setuptools version. But if you provide win32.egg files or do *not* provide win32.exe files, everything's cool.

3 3

paster.py failure on WinXP
by Ger Flanagan March 17, 2006

March 17, 2006

Hello all I posted the following message to the 'pylons-discuss' newsgroup, but maybe someone here can help, as I think it's related to setuptools? (Myghty installed the package 'setuptools-0.6a5-py2.4.egg') Thanks Gerard ----- Message ------ Hello all If this is off-topic, I apologise in advance. I'm interested in Pylons but I thought I'd check out Myghty first - I installed the latest version (Myghty-1.0.1 on Windows XP) and I can run the example shoppingcart application with no problems. However when I try to create a simple test app of my own using the 'paster.py' script, I get this error: >C:\DEVELOPMENT\myghty\test>paster create --template=myghty_simple simple C:\Program: can't open file 'Files\Python\Python24\python.exe': [Errno 2] No such file or directory As you can see, the location of my 'python.exe' is C:\Program Files\Python\Python24 but it looks like the space in the path may be causing a problem. (This has never been a problem before). I see that Pylons has a 'paster' command - presumably this is the same script that comes with the Myghty installation - has anyone come across this? Any advice would be welcome. Thanks. Gerard ------ End Message ------- ___________________________________________________________ Win a BlackBerry device from O2 with Yahoo!. Enter now. http://www.yahoo.co.uk/blackberry

2 1

Missing packages
by Juha Tuomala March 17, 2006

March 17, 2006

% paster Traceback (most recent call last): File "/usr/bin/paster", line 5, in ? from pkg_resources import load_entry_point File "/usr/lib/python2.3/site-packages/pkg_resources.py", line 2356, in ? working_set.require(__requires__) File "/usr/lib/python2.3/site-packages/pkg_resources.py", line 585, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/lib/python2.3/site-packages/pkg_resources.py", line 483, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: Cheetah % It looks like paster cannot find cheetach, even it's installed. I've compiled and installed cheetach RPM package. What is the thing what setuptools looks and doesn't find? That egg info? % rpm -qil python-cheetah Name : python-cheetah Relocations: (not relocatable) Version : 1.0 Vendor: (none) Release : 1 Build Date: Thu 16 Mar 2006 01:52:16 PM EET Install Date: Thu 16 Mar 2006 01:52:33 PM EET Build Host: XXXX.XXXXX.fi Group : Development/Libraries Source RPM: python-cheetah-1.0-1.src.rpm Size : 1993082 License: MIT Signature : (none) Packager : Juha Tuomala <XXXX(a)XXXXXX.fi> URL : http://sourceforge.net/projects/cheetahtemplate Summary : Template engine and code-generator Description : Cheetah is a Python-powered template engine and code-generator. It is similar to the Jakarta project's Velocity. /usr/bin/cheetah /usr/bin/cheetah-compile /usr/lib64/python2.3/site-packages/Cheetah /usr/lib64/python2.3/site-packages/Cheetah/CacheRegion.py /usr/lib64/python2.3/site-packages/Cheetah/CacheRegion.pyc /usr/lib64/python2.3/site-packages/Cheetah/CheetahWrapper.py /usr/lib64/python2.3/site-packages/Cheetah/CheetahWrapper.pyc /usr/lib64/python2.3/site-packages/Cheetah/Compiler.py /usr/lib64/python2.3/site-packages/Cheetah/Compiler.pyc /usr/lib64/python2.3/site-packages/Cheetah/DummyTransaction.py /usr/lib64/python2.3/site-packages/Cheetah/DummyTransaction.pyc /usr/lib64/python2.3/site-packages/Cheetah/ErrorCatchers.py /usr/lib64/python2.3/site-packages/Cheetah/ErrorCatchers.pyc /usr/lib64/python2.3/site-packages/Cheetah/FileUtils.py /usr/lib64/python2.3/site-packages/Cheetah/FileUtils.pyc /usr/lib64/python2.3/site-packages/Cheetah/Filters.py /usr/lib64/python2.3/site-packages/Cheetah/Filters.pyc /usr/lib64/python2.3/site-packages/Cheetah/ImportHooks.py /usr/lib64/python2.3/site-packages/Cheetah/ImportHooks.pyc /usr/lib64/python2.3/site-packages/Cheetah/ImportManager.py /usr/lib64/python2.3/site-packages/Cheetah/ImportManager.pyc /usr/lib64/python2.3/site-packages/Cheetah/NameMapper.py /usr/lib64/python2.3/site-packages/Cheetah/NameMapper.pyc /usr/lib64/python2.3/site-packages/Cheetah/Parser.py /usr/lib64/python2.3/site-packages/Cheetah/Parser.pyc /usr/lib64/python2.3/site-packages/Cheetah/Servlet.py /usr/lib64/python2.3/site-packages/Cheetah/Servlet.pyc /usr/lib64/python2.3/site-packages/Cheetah/SettingsManager.py /usr/lib64/python2.3/site-packages/Cheetah/SettingsManager.pyc /usr/lib64/python2.3/site-packages/Cheetah/SourceReader.py /usr/lib64/python2.3/site-packages/Cheetah/SourceReader.pyc /usr/lib64/python2.3/site-packages/Cheetah/Template.py /usr/lib64/python2.3/site-packages/Cheetah/Template.pyc /usr/lib64/python2.3/site-packages/Cheetah/TemplateCmdLineIface.py /usr/lib64/python2.3/site-packages/Cheetah/TemplateCmdLineIface.pyc /usr/lib64/python2.3/site-packages/Cheetah/Templates /usr/lib64/python2.3/site-packages/Cheetah/Templates/SkeletonPage.py /usr/lib64/python2.3/site-packages/Cheetah/Templates/SkeletonPage.pyc /usr/lib64/python2.3/site-packages/Cheetah/Templates/SkeletonPage.tmpl /usr/lib64/python2.3/site-packages/Cheetah/Templates/_SkeletonPage.py /usr/lib64/python2.3/site-packages/Cheetah/Templates/_SkeletonPage.pyc /usr/lib64/python2.3/site-packages/Cheetah/Templates/__init__.py /usr/lib64/python2.3/site-packages/Cheetah/Templates/__init__.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tests /usr/lib64/python2.3/site-packages/Cheetah/Tests/CheetahWrapper.py /usr/lib64/python2.3/site-packages/Cheetah/Tests/CheetahWrapper.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tests/FileRefresh.py /usr/lib64/python2.3/site-packages/Cheetah/Tests/FileRefresh.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tests/NameMapper.py /usr/lib64/python2.3/site-packages/Cheetah/Tests/NameMapper.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tests/SyntaxAndOutput.py /usr/lib64/python2.3/site-packages/Cheetah/Tests/SyntaxAndOutput.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tests/Template.py /usr/lib64/python2.3/site-packages/Cheetah/Tests/Template.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tests/Test.py /usr/lib64/python2.3/site-packages/Cheetah/Tests/Test.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tests/__init__.py /usr/lib64/python2.3/site-packages/Cheetah/Tests/__init__.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tests/unittest_local_copy.py /usr/lib64/python2.3/site-packages/Cheetah/Tests/unittest_local_copy.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tools /usr/lib64/python2.3/site-packages/Cheetah/Tools/CGITemplate.py /usr/lib64/python2.3/site-packages/Cheetah/Tools/CGITemplate.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tools/MondoReport.py /usr/lib64/python2.3/site-packages/Cheetah/Tools/MondoReport.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tools/MondoReportDoc.txt /usr/lib64/python2.3/site-packages/Cheetah/Tools/RecursiveNull.py /usr/lib64/python2.3/site-packages/Cheetah/Tools/RecursiveNull.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tools/SiteHierarchy.py /usr/lib64/python2.3/site-packages/Cheetah/Tools/SiteHierarchy.pyc /usr/lib64/python2.3/site-packages/Cheetah/Tools/__init__.py /usr/lib64/python2.3/site-packages/Cheetah/Tools/__init__.pyc /usr/lib64/python2.3/site-packages/Cheetah/Utils /usr/lib64/python2.3/site-packages/Cheetah/Utils/Indenter.py /usr/lib64/python2.3/site-packages/Cheetah/Utils/Indenter.pyc /usr/lib64/python2.3/site-packages/Cheetah/Utils/Misc.py /usr/lib64/python2.3/site-packages/Cheetah/Utils/Misc.pyc /usr/lib64/python2.3/site-packages/Cheetah/Utils/VerifyType.py /usr/lib64/python2.3/site-packages/Cheetah/Utils/VerifyType.pyc /usr/lib64/python2.3/site-packages/Cheetah/Utils/WebInputMixin.py /usr/lib64/python2.3/site-packages/Cheetah/Utils/WebInputMixin.pyc /usr/lib64/python2.3/site-packages/Cheetah/Utils/__init__.py /usr/lib64/python2.3/site-packages/Cheetah/Utils/__init__.pyc /usr/lib64/python2.3/site-packages/Cheetah/Utils/optik /usr/lib64/python2.3/site-packages/Cheetah/Utils/optik/__init__.py /usr/lib64/python2.3/site-packages/Cheetah/Utils/optik/__init__.pyc /usr/lib64/python2.3/site-packages/Cheetah/Utils/optik/errors.py /usr/lib64/python2.3/site-packages/Cheetah/Utils/optik/errors.pyc /usr/lib64/python2.3/site-packages/Cheetah/Utils/optik/option.py /usr/lib64/python2.3/site-packages/Cheetah/Utils/optik/option.pyc /usr/lib64/python2.3/site-packages/Cheetah/Utils/optik/option_parser.py /usr/lib64/python2.3/site-packages/Cheetah/Utils/optik/option_parser.pyc /usr/lib64/python2.3/site-packages/Cheetah/Version.py /usr/lib64/python2.3/site-packages/Cheetah/Version.pyc /usr/lib64/python2.3/site-packages/Cheetah/__init__.py /usr/lib64/python2.3/site-packages/Cheetah/__init__.pyc /usr/lib64/python2.3/site-packages/Cheetah/_namemapper.so /usr/share/doc/python-cheetah-1.0 /usr/share/doc/python-cheetah-1.0/CHANGES /usr/share/doc/python-cheetah-1.0/LICENSE /usr/share/doc/python-cheetah-1.0/README /usr/share/doc/python-cheetah-1.0/TODO % -- Ajatteleva ihminen tarvitsee unta.

4 6

Re: [Distutils] Finding dependencies and Limiting download
by Phillip J. Eby March 17, 2006

March 17, 2006

At 10:13 AM 3/16/2006 -0600, Ian Bicking wrote: >It occurs to me that maybe it would be easiest to create an RPM that >includes Paste, PasteDeploy, and PasteScript all together? That would >simplify all of this, it seems. That would certainly fix the conflict problem in the short term. In the long run, I'd like a better solution, though. Separate components for namespace packages are too good a thing to give up on without a fight. I'm tempted to see if there isn't a way to create an import hook in pkg_resources that would allow namespace packages to work without having an __init__.py. That would eliminate the file conflicts, albeit at the expense of requiring pkg_resources to be imported before any namespace packages. That seems problematic, though, since it would only be needed for --svem pacakges. The fundamental problem is that you end up having to have dummy packages to hold the __init__.py, or else you have to have something that gets imported first. It seems a bit fragile. One hack would be to add a .pth file for each --svem install of a namespace package, something like "PasteDeploy-nspkg.pth", that contains some code to activate the namespace package, like: import sys,new; \ m = sys.modules.setdefault('paste',new.module('paste')); \ p = os.path.join(sys._getframe(1).f_locals['sitedir'], 'paste'); \ mp = m.__path__ = getattr(m,'__path__',[]); \ (p not in mp) and mp.append(p) Yeah, that would do the trick, if --svem installs left out the __init__.py for declared namespace packages. Of course, this slows down Python startup time when you have a lot of --svem namespace packages installed, and doesn't work with "python -S", but what else can we do? If a given system packager wants to handle namespace packages differently, we could probably accomodate that via modifiers to the --svem mode. Really, the hardest part of implementing this is going to be the bits of setuptools that need to munge the build_py and install_lib commands to leave out __init__ files for namespace packages. I'll probably also need to document how to deal with certain legacy situations where there is an official package that "owns" the namespace and needs to install an __init__.py that actually contains code.

3 5

Another packaging strategy...
by Ian Bicking March 17, 2006

March 17, 2006

There's another strategy for making packages that I thought I'd bring up -- using PYTHONPATH and creating a large bundled package. That means, if you have an application, distribute all the dependencies in the application's package. Stuff them in /usr/share/PKG/lib or something. This doesn't work for installation of libraries, but then you never *need* to install a library as a package. It's the application you are ultimately interested in deploying and distributing. And I think the non-system-package installation techniques are generally much more appropriate for the actual development process. I don't think the bundled package is necessarily *easy* to build right now. But the result seems easy to manage. I don't really know much about the build process for packages, so what is ultimately involved to do this I'm not sure. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

1 0

Re: [Distutils] Finding dependencies and Limiting download
by Phillip J. Eby March 16, 2006

March 16, 2006

At 12:24 PM 3/16/2006 +0200, Juha Tuomala wrote: >On Wednesday 15 March 2006 21:33, Phillip J. Eby wrote: > > Well, you can work around this, but it's a pain because it makes it > > impossible for you to use setup.py develop. What you do is delete the > > __init__.py from all the packages, except for a special extra project that > > includes only the __init__.py. Then you make all the packages depend on > > that one. (And each package with a missing __init__ needs to have the > > package name explicitly listed in setup(), since find_packages() won't see > > it's there otherwise.) > >Where those __init__.py files should be deleted? From SVN snapshot >taken from Script? Or from installed target chroot during RPM packaging? Either the source tree you're building from, or the installed chroot. > > At that point, setuptools will add automatically-created __init__.py files > > into any eggs it makes, but it will not include them in > > --single-version-externally-managed installs. > >Currently I'm doing > > %{__python} setup.py install --root $RPM_BUILD_ROOT >- --single-version-externally-managed > >And it installs below: > > /usr/lib/python2.3/site-packages/paste/script > >which is logcical but also below: > > /usr/lib/python2.3/site-packages/PasteScript-0.5.1dev-py2.3.egg-info/ > >so that must be the egg-info you talked about. Yes. Again, please note that "setup.py bdist_rpm" can do a lot of this for you without you needing to do it manually. > > The downside, however, is that it makes "setup.py develop" useless for the > > original source packages. > >I don't even know what that 'develop' does, recall reading the docs that it >helps with syncing the svn snapshot or something. Yeah, I wouldn't worry about it, given what you're trying to do. I was just pointing out why Ian can't just remove the __init__.py files from the original source code; it would interfere with him working on the package normally. >So, all what I would like to do is do > > make install > >and say > > paster serve <plaa> <plaa> > >and if that would work, it would be great. Somehow I'm stuck with this >setuptools/egg stuff what looks that it's making things complicated. > >Doing it manually is not an option as everything is installed with RPM >what allows strict management. Actually, it's the way in which you're building the RPMs that's making things complicated here. If you just did this: export PYTHONPATH=/some/project/dir easy_install -d /some/project/dir Paste PasteDeploy PasteScript You'd get everything neatly installed in /some/project/dir with nothing installed anywhere else, and no need for an RPM. (It would also install a copy of Cheetah with the missing egg-info.) But if you really *must* have an RPM, you can just make an RPM of the resulting directory, as long as you don't mind it always being installed to /some/project/dir. (Because the created files will contain some absolute paths.) Anyway, you could do something like "/usr/local/paste" as the target directory, and just put in whatever you need to either symlink the executables from there to a bin directory. If you don't want to have to set PYTHONPATH at runtime, you can use the "administrator install" setup instructions for easy_install to make /usr/local/paste an automatic "site" directory when Python starts. All this having been said, I think there *is* a reasonable way to work around the __init__.py issue for packaging tools like RPM, but what I've just given you should work now, so that you don't have to wait for any new features to come out in setuptools.

1 0

setup.py test doesn't honor --find-links
by Tres Seaver March 16, 2006

March 16, 2006

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Can we extend the test command (deriving from easy_install, I guess, instead of Command), to take the same options it does? I tried hacking at it a bit, but without much success so far: I've gotten it to accept the '--find-links' option, but not actually to *use* it: $ cd ~/projects/eggz/src/zope.tales $ ../../bin/python setup.py test \ --find-links="http://download.zope.org/distribution" running test running egg_info writing requirements to src/zope.tales.egg-info/requires.txt writing src/zope.tales.egg-info/PKG-INFO writing top-level names to src/zope.tales.egg-info/top_level.txt writing namespace_packages to src/zope.tales.egg-info/namespace_packages.txt warning: manifest_maker: standard file not found: should have one of README, README.txt writing manifest file 'src/zope.tales.egg-info/SOURCES.txt' running build_ext Checking .pth file support in . /home/tseaver/projects/PyCon2006/eggification/sandbox/src/zope.tales/../../bin/python - -E -c pass Searching for zope.testing Reading http://www.python.org/pypi/zope.testing/ Couldn't find index page for 'zope.testing' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://www.python.org/pypi/ No local packages or download links found for zope.testing error: Could not find suitable distribution for Requirement.parse('zope.testing') I'm attaching the patch against the current SVN head, Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver(a)palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFEFt/z+gerLs4ltQ4RAvdFAKCRSHP44tnDHIKdnHP7dEtZxvyqBgCdFWBE dULlC535P+qeO8m5MZLogig= =lM5x -----END PGP SIGNATURE-----

2 3

Finding dependencies and Limiting download
by Juha Tuomala March 15, 2006

March 15, 2006

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi I'm new with setuptools and probably asking wrong questions. :-) I'm making RPM package from python-paste for CentOS Linux installation and came across some problems. I do the installation with: setup.py install --root $RPM_BUILD_ROOT and it seems to work pretty well, but it doesn't find my already installed cheetah package below site-pacakges and thus starts searching it from the net. Does the dependencies have to be installed as an egg so that setuptools could find them? (I doubt that) Second thing is that how do I limit the download using 'install' ? With 'easy_install' I could use --allow-hosts=None or -H, but is it so that it's not available for 'install'? Any help would be appreciated. BR, Tuju - -- Ajatteleva ihminen tarvitsee unta. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.1 (GNU/Linux) iD8DBQFEGBzfQnPP29fw0xMRAinEAJ9acsH9V/upVnx+Z2vTfVwaxHC//gCfdGsp mn39L3REbs9DKXFOIYzILbI= =nn4X -----END PGP SIGNATURE-----

3 6

(no subject)
by FETCHMAIL-DAEMON＠ns3565.ovh.net March 15, 2006

March 15, 2006

This message was too large (SMTP error 552).

1 0

Re: [Distutils] easy_install ./mymodule.py
by Phillip J. Eby March 13, 2006

March 13, 2006

At 09:55 AM 3/13/2006 +0200, Ville Vainio wrote: >I'm thinking of the extreme "scaling down" scenario here, shipping >only a single .py file (a "script"). When you have a setup.py script >you need to bundle it with the module. > >When in the past you had a directory full of .py scripts that you >added to PATH, now you could just go there and do easy_install *.py. I'm not following why that would be any better. Why not just add a setup.py to that directory?

2 1