Mailman 3 November 2010 - Distutils-SIG

setuptools.archive_util.unpack_archive dies in case of bad tarball
by Łukasz Nowak 18 Nov '10

18 Nov '10

Hello, Please see attached tarkill.tar.gz and buildout profile. This tarball containts dead link: ls -l tarkill lrwxrwxrwx 1 luke users 4 2010-11-18 19:56 kill -> kill After running attached buildout profile and having python shell setuptools will enter infinite loop: >>> import sys >>> sys.version_info sys.version_info(major=2, minor=7, micro=0, releaselevel='final', serial=0) >>> import tempfile >>> import setuptools.archive_util >>> tempdir = tempfile.mkdtemp() >>> setuptools.archive_util.unpack_archive('tarkill.tar.gz', tempdir) ^CTraceback (most recent call last): File "<console>", line 1, in <module> File "/srv/home/luke/python2.7/setup/eggs/setuptools-0.6c12dev_r85381-py2.7.egg/setuptools/archive_util.py", line 67, in unpack_archive driver(filename, extract_dir, progress_filter) File "/srv/home/luke/python2.7/setup/eggs/setuptools-0.6c12dev_r85381-py2.7.egg/setuptools/archive_util.py", line 190, in unpack_tarfile member = tarobj._getmember(linkpath) File "/srv/home/luke/python2.7/parts/python2.7/lib/python2.7/tarfile.py", line 2357, in _getmember members = self.getmembers() File "/srv/home/luke/python2.7/parts/python2.7/lib/python2.7/tarfile.py", line 1801, in getmembers self._check() KeyboardInterrupt Attached workaround which breaks the loop, but I think that such deadlink shall be created, just like tar does it. Regards, Luke -- Łukasz Nowak IT Specialist email(a)lnowak.com http://lnowak.com/ Skype: Shufla jid: shufla(a)jabster.pl gg: 1157726 ``Use the Source, Luke...''

1 0

M2Crypto install failing
by Leonardo Rochael Almeida 18 Nov '10

18 Nov '10

Hi, With this simple buildout: [buildout] parts = m2c [m2c] recipe = zc.recipe.egg eggs = M2Crypto I'm getting this failure: Installing 'M2Crypto'. We have no distributions for M2Crypto that satisfies 'M2Crypto'. Getting distribution for 'M2Crypto'. Running easy_install: /usr/bin/python "-c" "from setuptools.command.easy_install import main; main()" "-mUNxd" "/home/leo/opt/m2cryptotest/eggs/tmp5wm_oK" "-q" "/tmp/tmphof_2Jget_dist/M2Crypto-0.20.2.win32.zip" path=/home/leo/opt/m2cryptotest/eggs/setuptools-0.6c12dev_r85381-py2.6.egg error: Couldn't find a setup script in /tmp/tmphof_2Jget_dist/M2Crypto-0.20.2.win32.zip An error occured when trying to install M2Crypto 0.20.2.win32. Look above this message for any errors that were output by easy_install. While: Installing m2c. Getting distribution for 'M2Crypto'. Error: Couldn't install: M2Crypto 0.20.2.win32 AFAICT, This is due to the fact that M2Crypto win32 egg available here: http://chandlerproject.org/pub/Projects/MeTooCrypto/M2Crypto-0.20.2.win32.z… And linked from the project "Home Page" as registered on PyPI takes precedence over the package stored here: http://pypi.python.org/packages/source/M/M2Crypto/M2Crypto-0.20.2.tar.gz And linked from the PyPI page itself. If I'm not mistaken, this is also because "win32" in M2Crypto-0.20.2.win32.zip looks like part of the version, and not the architecture. Is this analysis correct? Is there a quick way to blacklist this package in buildout or setuptools? For the record, I can work-around this by downloading the right M2Crypto package, then doing: bin/buildout setup src/M2Crypto-0.20.2 bdist_egg and moving and unpacking the generated egg in the egg cache directory, but this is really ugly. Cheers, Leo

1 0

Retrieval of python PKG-INFO data for installed packages
by Alan Franzoni 17 Nov '10

17 Nov '10

Hello, is it possible to retrieve the same info which is stored in PKG-INFO for an installed package? I've seen this project: http://pypi.python.org/pypi/pkginfo doesn't anything built in setuptools/distribute/pkg_resources achieve a similar result? e.g., while in a lib, I'd like to do something like: import mymodule get_pkg_info(mymodule).project_name ? -- Alan Franzoni -- contact me at public(a)[mysurname].eu

2 1

Where should I put my tests in my package?
by W. Matthew Wilson 17 Nov '10

17 Nov '10

Right now, I've got a project that started off looking sort of like this: pitz/ pitz/ cmdline/ webapp/ entity/ tests/ test_cmdline.py test_entity.py test_webapp.py In my setup.py file, I have this line packages=find_packages(exclude=[''tests']), so that the tests don't get installed when the code is installed. This works OK for now, but I'm adding more folders under the original folders, like this: pitz/ pitz/ cmdline/ webapp/ handlers/ __init__.py statichandler.py entity/ tests/ test_cmdline.py test_entity.py test_webapp.py test_handlers.py test_statichandler.py It is getting more difficult to figure out the links between test files and code files. I have a few questions: 1. I would like to put lots of tests/ folders next to the code they test, rather than at the top level of the project. How do I make sure the tests folders don't get installed? 2. Should I worry about not installing tests? Should I install them? 3. Is there any pattern I should follow for organizing my tests within my project? Thanks for the help. Matt -- W. Matthew Wilson matt(a)tplus1.com http://tplus1.com

4 3

setup.py script using python interpreter from previous run
by Geoff Bache 10 Nov '10

10 Nov '10

Hi all, I have the following setup.py script: #!/usr/bin/env python from distutils.core import setup scripts=["hello.py"] setup(scripts=scripts) I have two different python installations (using virtualenv) where I wish to install this program. So I do ~/tmp/test_setup/python1/bin/python setup.py install which creates a file at /users/geoff/tmp/test_setup/python1/bin/hello.py, that looks like this: #!/users/geoff/tmp/test_setup/python1/bin/python print "Hello" So far so good. But then I also install it somewhere else: ~/tmp/test_setup/python2/bin/python setup.py install which creates a file at /users/geoff/tmp/test_setup/python2/bin/hello.py which refers to "python1", i..e it has the same contents as the first one. Which is clearly not what I want. Is this a bug? Or have I missed something in my setup.py? It works of course if I remove the generated "build" directory by hand, but it's not so nice to have to remember to do that. (I'm using Python2.6 on Linux) Regards, Geoff Bache

4 6

[RFC] eggbuild
by Manlio Perillo 06 Nov '10

06 Nov '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi. I'm pleased to announce the eggbuild package. This is a proof of concept for some ideas I have in mind for Python packages distribution. The package name, the implemented setuptools command names and the API are unstable, and may change in the future. The project is currently available at: http://hg.mperillo.ath.cx/eggbuild. The README and TODO file contain some informations, and I will try to not replicate them here. Basically, the idea is to build a setuptools distribution format that, like PyInstaller, is *self contained* (containing all required packages and all external shared libraries). However, unlike PyInstaller, there are some important differences, listed in the README file. I have tested the eggbuild command with a patched psycopg2 package, on a Linux Debian Squeeze system. Just for information, the resulting egg is 1.4 Mb big, when a normal egg is 156 Kb big. The resulting EGG-INFO\native_libs file is available here: http://paste.pocoo.org/show/285677/ The patch for psycopg2 is available here: http://paste.pocoo.org/show/285662/ The code *seems* to work. When importing the psycopg2 module, the *correct* shared libraries are loaded. At least the correct libpq.so.5 library is used. The other libraries required by libpq.so.5 are searched in the system default location, and I need to check if this can be solved without having to patch each library (adding LD_RPATH). A simple solution is to set LD_LIBRARY_PATH environment variable (but there are some issues to be solved). Now, what I want to implement is a setuptools command that will build a Debian package, with the following features: * pre-install script will create a virtual environment, using virtualenv. The virtualenv should be created in /opt/<pkg-name>. * the package and required packages are stored in the deb data file, in egg binary format, and will be automatically copied to the virtual python installation. The main feature to implement is, however, a full MSI installer. However I don't know how MSI is supposed to work, and I will probably need some help (at least some good tutorial about MSI technology, what can be and what can not be done). I will appreciate comments and suggestions about the eggbuild design and implementation. Thanks Manlio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkzRxNwACgkQscQJ24LbaUQimACfRhi6kJUbPR8RkOz16Zcjni8T AA8AmQGqdakXHSuZTMc1oKbOaGPJKq2p =Pwqc -----END PGP SIGNATURE-----

2 7

An observation on how system packagers and developers can be friends
by Jim Fulton 05 Nov '10

05 Nov '10

Periodically, in various venues, we discuss the challenges of deploying applications with or in spite of system packaging of Python and system packaging philosophies. (Note that I'm mainly talking g about deploying applications, as opposed to individual Python packages.) In my experience, the conversation usually unfolds with developers (like me:) snarkely saying how annoyed we are and systems packagers (and administrators) explaining all of the rules they have to follow and that we should follow too. In practice, I think the result is usually that developers make an end-run around system packaging system, either building in production, or by building system packages that bundle dependencies to get needed control. The result is that at least some of the system packagers goals are subverted. In my experience, both sides have legitimate goals, but the problem is rarely, if ever, approached from the point of view of recognizing both sets of goals coming up with a (probably new) solution that addresses both sets of goals. (Note that "doing things the way I've always done them" is *not* a valid goal. :) I believe that solutions that address both sets of goals are possible and even practical, but it has to start with a consideration of the underlying goals. Jim -- Jim Fulton

9 11

Re: [Distutils] package data not installed from python .egg file
by P.J. Eby 05 Nov '10

05 Nov '10

At 09:32 AM 11/3/2010 +0100, Roban Hultman Kramer wrote: >Can anyone suggest what I need to do to get the package_data files to >be installed when easy_install unzips the egg file? The issue is the '..' in your data file names; currently setuptools.archive_util refuses to extract files with '..' in their path. I'm going to go ahead and fix this, but even after I do, please note that it will continue to be a problem for versions of easy_install that are out there "in the wild".

2 1

Security issue with Distutils register is still actual
by anatoly techtonik 03 Nov '10

03 Nov '10

Hello, Does anybody care that PyPI password are stored in a well-known location in cleartext and developers are forced to store them when they submit packages for review? http://bugs.python.org/issue9995 -- anatoly t.

3 7

package data not installed from python .egg file
by Roban Hultman Kramer 03 Nov '10

03 Nov '10

I'm trying to include some data files in a python package using the setuptools package_data option. I'm then accessing the files with pkg_resources. This works perfectly when the python .egg file is installed as-is (i.e. still zipped). But when the egg file is unzipped during installation, the data files are not installed. In other words, if I run: python setup.py bdist_egg cd dist sudo easy_install -z EnrichPy-0.1.001-py2.6.egg then the egg file is installed (with the data safely zipped inside) and everything works. On the other hand, if I run sudo easy_install -Z EnrichPy-0.1.001-py2.6.egg then the data files are not installed. I have a directory called EnrichPy-0.1.001-py2.6.egg/enrichpy/ under dist-packages, but it contains only my source files, not my data files. Can anyone suggest what I need to do to get the package_data files to be installed when easy_install unzips the egg file? Notes: The package is available at http://github.com/roban/EnrichPy I can test it by running: import enrichpy.yields enrichpy.yields.Data_vdHG().data If that exits without errors, then pkg_resources is finding the installed data. I asked this question on stackoverflow without response: http://stackoverflow.com/questions/4043315/package-data-not-installed-from-… -- Roban Hultman Kramer Zwicky Fellow Institute for Astronomy ETH Zürich (Swiss Federal Institute of Technology)

1 0