Mailman 3 January 2009 - Distutils-SIG

using 'python setup.py test' with an index?
by Jorge Vargas 12 Jan '09

12 Jan '09

Hi, I got the following situation. I have a big package which depends on several others projects just for unit tests, the package is a maskup of several others. Therefore I'm adding those dependencies to tests_require, and everything is working great. Now I have stumble upon a package that has no setup.py (they install with configure/make) but someone "forked" it and releases a tarball/egg version on a private index. Therefore my question, how can I add this test_require dep, and tell it to fetch that one only from that index? In case you were wondering what the project it, it's a (not released yet) set of "helpers" to mount wsgi apps "with one liners" into a bigger TurboGears2 application. The current (and temporal) hg repository is here http://toscawidgets.org/hg/tgext.wsgiapps/, we currently have WSGIControllers for trac, mercurial, TileCache and Zine, and the project is less than 48hrs old!

1 0

Re: [Distutils] get_metadata in Distutils
by Phillip J. Eby 12 Jan '09

12 Jan '09

At 06:48 PM 1/10/2009 +0100, Tarek Ziadé wrote: >Hello, > >I am going to work on a change in Distutils to include a get_metadata >API, http://bugs.python.org/issue4908 > >I don't see yet if this can be hard to do, and I will probably >discover it while doing it, > >Since this is basically what has been done in setuptools, I thaught >that you might wanted to help around for this change ? I'd suggest looking at the pkg_resources code, particularly find_distributions and its related functions. Note, for example, that egg layouts can be nested (even when zipped!), and that the metadata can have different filenames, depending on the installation format. Version parsing also has certain peculiarities, which also means that people doing simple string comparisons on the version field is probably not going to suffice.

2 1

[distutils] make the storage of the password optional in .pypirc
by Tarek Ziadé 11 Jan '09

11 Jan '09

Hello I would like to make the storage of the password optional in .pypirc. If unstored, the register command would prompt for it. More details here: http://bugs.python.org/issue4394 Any thaught or objection on this ? Regards Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/

8 18

Re: [Distutils] Unable to import python script after installing using easy_install
by Phillip J. Eby 11 Jan '09

11 Jan '09

At 03:03 PM 1/9/2009 -0600, ray terrill wrote: >I'm using python2.4 to try to package and deliver a custom python >script. I'm unable to import the package. > >My setup.py looks like the following: >from setuptools import setup, find_packages >setup( > name = "randomscript", > version = "1.0", > packages = find_packages(), >) > >I'm building the package using the following: >python setup.py bdist_egg >running bdist_egg >running egg_info >writing randomscript.egg-info/PKG-INFO >writing top-level names to randomscript.egg-info/top_level.txt >writing dependency_links to randomscript.egg-info/dependency_links.txt >reading manifest file 'randomscript.egg-info/SOURCES.txt' >writing manifest file 'randomscript.egg-info/SOURCES.txt' >installing library code to build/bdist.linux-i686/egg >running install_lib >warning: install_lib: 'build/lib' does not exist -- no Python >modules to install The above is an indication that your source code isn't being found. My guess is that since you're saying you want to install a "script" but can't import it, what you actually have is a standalone *module*, not a script or a package. find_packages() cannot find such modules, you must list them individually, e.g.: py_modules = ['randomscript'], rather than listing them in the 'packages' parameter. By the way, if you plan to distribute your module via PyPI, it is recommended that you use 'sdist' to build a source distribution in addition to (or instead of) an .egg file. .egg files are a very specialized distribution format intended for things like application plugins and site-local distribution (e.g., administrators making binary packages available across a campus or company network, etc.) That is, they're more aimed at end-user distribution use cases, than at programmer or "open source" distribution use cases. Or as I sometimes say, they're more of a "deployment" format than a "distribution" format.

2 3

Way to ask distutils what version of a package is installed?
by ray terrill 10 Jan '09

10 Jan '09

Is there a way to ask distutils what version of a package is currently installed? I provided the version number when creating my setup.py, but when this is deployed over 800+ clients, I'd like to have a way to interrogate the system to determine which version is installed. Thanks, -Ray

3 4

Unable to import python script after installing using easy_install
by ray terrill 10 Jan '09

10 Jan '09

I'm using python2.4 to try to package and deliver a custom python script. I'm unable to import the package. My setup.py looks like the following: from setuptools import setup, find_packages setup( name = "randomscript", version = "1.0", packages = find_packages(), ) I'm building the package using the following: python setup.py bdist_egg running bdist_egg running egg_info writing randomscript.egg-info/PKG-INFO writing top-level names to randomscript.egg-info/top_level.txt writing dependency_links to randomscript.egg-info/dependency_links.txt reading manifest file 'randomscript.egg-info/SOURCES.txt' writing manifest file 'randomscript.egg-info/SOURCES.txt' installing library code to build/bdist.linux-i686/egg running install_lib warning: install_lib: 'build/lib' does not exist -- no Python modules to install creating build/bdist.linux-i686/egg creating build/bdist.linux-i686/egg/EGG-INFO copying randomscript.egg-info/PKG-INFO -> build/bdist.linux-i686/egg/EGG-INFO copying randomscript.egg-info/SOURCES.txt -> build/bdist.linux-i686/egg/EGG-INFO copying randomscript.egg-info/dependency_links.txt -> build/bdist.linux-i686/egg/EGG-INFO copying randomscript.egg-info/top_level.txt -> build/bdist.linux-i686/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating 'dist/randomscript-1.0-py2.4.egg' and adding 'build/bdist.linux-i686/egg' to it removing 'build/bdist.linux-i686/egg' (and everything under it) I'm installing the package using the following: easy_install randomscript-1.0-py2.4.egg Processing randomscript-1.0-py2.4.egg Copying randomscript-1.0-py2.4.egg to /usr/lib/python2.4/site-packages Adding randomscript 1.0 to easy-install.pth file Installed /usr/lib/python2.4/site-packages/randomscript-1.0-py2.4.egg Processing dependencies for randomscript==1.0 Finished processing dependencies for randomscript==1.0 If I attempt to use the script, I get an import error, but I do see the script listed in my sys.path: ImportError: No module named randomscript ['/home/oracle/python/dist', '/usr/lib/python2.4/site-packages/setuptools-0.6c9-py2.4.egg', '/usr/lib/python2.4/site-packages/randomscript-1.0-py2.4.egg', '/usr/lib/python24.zip', '/usr/lib/python2.4', '/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib-tk', '/usr/lib/python2.4/lib-dynload', '/usr/lib/python2.4/site-packages'] Also, if I cd to /usr/lib/python2.4/site-packages/, I see that my egg is there, but is not a directory (should it be)? -rw-r--r-- 1 root root 819 2009-01-09 20:06 randomscript-1.0-py2.4.egg Any help would be appreciated. -Ray

2 1

pre-release versioning problems with sdist, bdist_rpm, bdist_debian
by Gerry Reno 10 Jan '09

10 Jan '09

When I create a set of distro packages for a specific distro, I need to distribute both a tarball and a distro-specific package such as an RPM or a deb archive. Some people like installing things from tarballs, others like using the distro package manager. So I let them have it both ways. We can generate the tarball using 'sdist' and generate the distro-package with 'bdist_rpm', etc. This all works great as long as we're talking final releases here: 4.9.0, 5.0.0. But, whenever you start to introduce pre-release candidate versioning, eg: 5.0.0_rc2, then things unravel. What I have been attempting to do, and is apparently is like trying to find the holy grail, is to be able to generate both tarballs and distro-packages for both pre-release candidates and final releases, that exhibit the following behavior: tarballs: foo-5.0.0_rc1.tar.gz (version=5.0.0_rc1); (extracts a foo-5.0.0_rc1 directory; 'sdist' requires version set like this so tarball name is unique and extract dir is unique) distro-package (pre-release): foo-5.0.0-0_rc1.noarch.rpm (version=5.0.0 and release=0_rc1) distro-package (final release): foo-5.0.0-1.noarch.rpm (version=5.0.0 and release=1); 'bdist_rpm' requires version and release set like this so final release will update the pre-release candidate package The problem is that there appears to be no way to get a coordinated behavior between 'sdist' and 'bdist_rpm' as far as version and release strings are concerned that will satisfy both tarballs and bdist packages. 'sdist' knows nothing about 'release' string so you are forced to put the pre-release candidate information into the 'version' string' but as soon as you do that then you break 'bdist' packaging. And to complicate matters, the world seems to have settled for making pre-release candidate version strings like '5.0.0_rc1' for many applications. For example: We have an application, foo-5.0.0, and we want to put out some pre-release candidates for testing, so we set the version to "5.0.0_rc1" in setup.py. We create the source distribution with: $ python setup.py sdist which creates a source archive, foo-5.0.0_rc1.tar.gz. We extract this archive and 'cd' into the foo-5.0.0_rc1 directory and create a built distribution with: $ python setup.py bdist_rpm which creates source and binary RPMS in the form: foo-5.0.0_rc1-1.noarch.rpm. So we think everything is fine. Everyone installs and tests using the pre-release candidate and subsequent candidates but when you eventually get to the final release, foo-5.0.0, and build your final release RPMS, foo-5.0.0-1.noarch.rpm, you find that it will not update your last pre-release candidate RPM, foo-5.0.0_rcX-1.noarch.rpm because it is not "rpm newer". I'm hoping there is some distutils guru that can tell me how to solve this enigma or show me how to extend distutils to accomplish what is needed. Regards, Gerry PS: In case anyone notices, yes, I posted a similar question yesterday on the python list without much response, but the question really belongs on this list.

2 7

environment variables in buildout config files?
by Barry Warsaw 09 Jan '09

09 Jan '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just converted Mailman 3 to use zc.buildout. Thanks to Gary Poster for providing some good examples and the impetus for my force vacation hacking. :) I especially like being able to specify defaults in my ~/.buildout/ default.cfg file. One thing that bugs me though is that I can't use environment variables like $HOME there. This is a minor pain because I can't share my .buildout directory between Linux and OS X machines (say, via version control). It appears as though I have to be explicit in the path, with /Users/barry on OS X and /home/barry on Linux. Similarly, in my buildout.cfg file, if I include a develop field pointing to some source directories (other than .), I can't use $HOME/ projects/foo/bar here. Any thoughts on supporting environment variables in .cfg files? Or do they work and I'm just experiencing operator error? ;) A couple of other minor questions. In my ~/.buildout/default.cfg file, I have [buildout] eggs-directory=/Users/barry/.buildout/eggs download-cache=/Users/barry/.buildout/download-cache This is great because I can share that stuff between lots of development branches. However buildout will create the eggs directory if needed, but complain that download-cache doesn't exist. It seems like buildout could/should create download-cache on demand. I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout? I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that? Thanks, Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSWJnUHEjvBPtnXfVAQInXwP/VGi1JoXW2qMV7u1JhvVSIF2fm3WiCz01 z7M/OenoLzdpicidNY5+gfus8Y8b5qKosA/8XAhA5BM9WOmebjMuaWsR0EZiQEY5 LXwhHlEskoYJlhYRbnnCQyEfTZOIBB57kMGsgKNnRDETz4f7XWEqKvIhdBERWBhx gWtQAufcE4c= =81NH -----END PGP SIGNATURE-----

3 8

zc.recipe.testrunner hook?
by Barry Warsaw 06 Jan '09

06 Jan '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm not sure this is the best place to ask about zc.recipe.testrunner, but I will anyway :). Besides converting to zc.buildout, over my recent forced vacation, I also converted Mailman 3 to zc.recipe.testrunner. This is another bit of awesome from Zope I'm glad to be using. One small problem though: I need to hook into the option parser to add a few more options. For example, I'm adding a -e/--stderr option which sets up logging propagation for some of my subprocess tests. The way I've done it is to add this to my buildout.cfg file: [test] recipe = zc.recipe.testrunner ... # Hack in extra arguments to zope.testrunner. initialization = from mailman.testing.layers import ConfigLayer; ConfigLayer.hack_options_parser() The hack_options_parser() method looks like this: def hack_options_parser(cls): """Hack our way into the zc.testing framework. Add our custom command line option parsing into zc.testing's. We do the imports here so that if zc.testing isn't invoked, this stuff never gets in the way. This is pretty fragile, depend on changes in the zc.testing package. There should be a better way! """ from zope.testing.testrunner.options import parser parser.add_option('-e', '--stderr', action='callback', callback=cls.handle_stderr, help=_('Propagate log errors to stderr.')) So as the comment asks passive-aggressively: is there a better way? Thanks, Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSWJotnEjvBPtnXfVAQJvbwP+NkKyX9bYWLrZQ4L4b3wWbCfL9l72EyQi RJh0EPQn1Efz6Oe7h+oTojbGxmpviHyJl+OnrO3OrwPxR5miXK3dWFmtdh7r/TUm jRz3m4k58UwVe2/uWnlHGQ4c/4ooBP0kYZecwTms2cUtzZFkVF/IUoSPvJl/80R2 5q1ZBq9P5PE= =pUiS -----END PGP SIGNATURE-----

1 0

Re: [Distutils] Easy installer on Windows Python 2.6
by Phillip J. Eby 04 Jan '09

04 Jan '09

At 10:16 AM 1/4/2009 -0600, Paul Vincent Craven wrote: >There is no link here for a Windows installer for Python 2.6: >http://pypi.python.org/pypi/setuptools >Only 2.3, 2.4, and 2.5. > >How does a person install 'easy_install' for Python 2.6 on windows? You might see if the 2.5 installer works. If not, you can use the source and run a bdist_wininst to make a 2.6 installer.

1 0

Easy installer on Windows Python 2.6
by Paul Vincent Craven 04 Jan '09

04 Jan '09

There is no link here for a Windows installer for Python 2.6: http://pypi.python.org/pypi/setuptools Only 2.3, 2.4, and 2.5. How does a person install 'easy_install' for Python 2.6 on windows? -- Paul Vincent Craven http://www.cravenfamily.com

1 0

Setuptools for Python 3 patch.
by Lennart Regebro 04 Jan '09

04 Jan '09

I have in issue 56 ( http://bugs.python.org/setuptools/issue56 ) added a patch for setuptools, that enable it to be ported to Python 3. If somebody would like to review the patches, I would be much obliged. There are many choices I've done that could be done differently, stylistically. The porting itself is done with /opt/python30/bin/2to3 -wn . /opt/python30/bin/2to3 -dw /tmp/setuptools.LOCAL/setuptools/tests/api_tests.txt but you currently need to patch 2to3 (with patches which I sent to this list yesterday by mistake, sorry about that). Also, I have not done patching to remove all 2to3 warnings. There are many "foo = map(fun, bar)" calls in the code, and these get converted by 2to3 to the ugly "foo = list(map(fun, bar))". Changing them all to "foo = [fun(x) for in in bar]" would be the Python3-approved way of doing it, but I only fixed the things that was needed to make the port work. I'm going to make a blogpost about the porting experience. I'd also like to provide a source dist tgz for testing by others in that blog post, of that's OK. If it's not OK, then tell me. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

1 0

Some suggested patches
by Lennart Regebro 04 Jan '09

04 Jan '09

When trying to port setuptools, I found some things I think are bugs, missing library reorganisation fixes more specifically. Firstly, there are a lot of things missing from the urllib fixer, most significantly all the split* methods. Secondly, htmlentitydefs has moved, and this doesn't seem to be handled. Am I correct that these are bugs, or should something else be done? A suggested patch is included, although I haven't made any tests for it, I wanted to check here first. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64

1 1