Mailman 3 April 2007 - Distutils-SIG

I need to create a customized Fedora live cd with Python applications on it
by Stanley A. Klein 24 Apr '07

24 Apr '07

I need to create a customized Fedora live CD with Python applications on it. The live CD requires rpm's of the applications. (The issues are similar to a customized Ubuntu live CD but Ubuntu requires deb's.) It seems that the distutils/setup.py functions that create rpms are broken. I get errors most of the time that I try to use them. The Python community seems to be focused on eggs, that are useful if individually installed but don't match the distribution formats of the Linux distros. Those distribution formats are what the distros use for their live CD's. Is there a way to convert eggs to rpm's? Can the distutils be fixed to reliably create rpm's? Is there some other way to install eggs on distro customized live CD's? Stan Klein

2 1

programmatically adding entry points
by Antoine Pitrou 24 Apr '07

24 Apr '07

hello, First, I don't know if this is the right mailing-list for asking questions about pkg_resources. If it isn't, please be kind to point me to the right place. I've read the pkg_resources API (*) and I don't see any way of programmatically registering a (possibly temporary) entry-point. The only documented way to add entry points is to write setup scripts. But if I want to write unit tests for entry point-related behaviour, I don't want to run setup scripts in my unit tests. Is there a simple way to do this? (*) http://peak.telecommunity.com/DevCenter/PkgResources Thanks Antoine.

2 2

Re: [Distutils] setuptools http auth
by Phillip J. Eby 23 Apr '07

23 Apr '07

At 01:07 PM 4/23/2007 -0700, Rob Cakebread wrote: >On 4/23/07, Phillip J. Eby <pje(a)telecommunity.com> wrote: >> >>Argh. setuptools.package_index isn't HTML-unquoting URLs. That needs to >>be fixed. :( >> > >I'm not sure if this is the best way to go about it, I couldn't find >anything in the stdlib >to unencode it, but this works: > >--- setuptools/package_index.py (revision 54927) >+++ setuptools/package_index.py (working copy) >@@ -385,6 +385,7 @@ > of `tmpdir`, and the local filename is returned. Various errors > may be > raised if a problem occurs during downloading. > """ >+ spec = spec.replace("@", "@") > if not isinstance(spec,Requirement): > scheme = URL_SCHEME(spec) > if scheme: There are actually a bunch of places where these URLs get used, so the decoding actually needs to happen all over. So I'm implementing a more complete solution.

1 0

Re: [Distutils] setuptools http auth
by Phillip J. Eby 23 Apr '07

23 Apr '07

At 12:04 AM 4/23/2007 -0700, Rob Cakebread wrote: >Hi, > >I have http://anonymous@tools.assembla.com/svn/yolk/trunk/#egg=yolk-dev >in my PKG-INFO. > >When I try to use 'easy_install yolk==dev' I get: > >Best match: yolk dev >Downloading >http://anonymous@tools.assembla.com/svn/yolk/trunk/#egg=yolk-dev >error: Download error for http://anonymous&: (-2, 'Name or service not known') > >I can see that when I use 'setup.py register' the @ is changed to @ >Is there any way around this? Argh. setuptools.package_index isn't HTML-unquoting URLs. That needs to be fixed. :(

2 1

setuptools http auth
by Rob Cakebread 23 Apr '07

23 Apr '07

Hi, I have http://anonymous@tools.assembla.com/svn/yolk/trunk/#egg=yolk-dev in my PKG-INFO. When I try to use 'easy_install yolk==dev' I get: Best match: yolk dev Downloading http://anonymous@tools.assembla.com/svn/yolk/trunk/#egg=yolk-dev error: Download error for http://anonymous&: (-2, 'Name or service not known') I can see that when I use 'setup.py register' the @ is changed to @ Is there any way around this? Thanks, Rob

1 0

Derivative files
by Ian Bicking 23 Apr '07

23 Apr '07

I made a distutils command to take the Javascript files in an application and put them together and compress them. I'd like the result to kind of always be there; it would be distributed with the egg for instance. But I don't really want it in source control. So I think I need to make this part of the build somehow. What should I look for to do that? -- Ian Bicking | ianb(a)colorstudy.com | http://blog.ianbicking.org | Write code, do good | http://topp.openplans.org/careers

2 3

BUG: Stub loaders not created for non distutils .pyd files when using setuptools.
by Jamie Kirkpatrick 20 Apr '07

20 Apr '07

All I have come across what I believe is a bug in setuputils bdist_egg command. According to this page http://peak.telecommunity.com/ DevCenter/EggFormats#zip-file-issues it is not possible for python's zip loader to load shared libraries directly from the zip files themselves therefore a special stub loader py file is generated for each C extension included in a package that works around this issue. The problem I have found becomes apparent when pyd extension libraries are compiled using a process that lives outside of distutils itself (ie not using the Extension class). The code in bdist_egg.py only generates stub loaders for extensions that are built by distutils - any pyd files included in another manner are not processed. This is actually inconsistent with a method elsewhere in the distribution called exe_to_egg or somesuch where all resources are scanned and any .pyd files get a stub generated. I've created a simple patch (attached) to fix the issue. It checks all the resources found in the all_outputs list and generates stubs for any pyd files found - this covers off cases where you have pyd files included that are not generated by distutils itself. I understand that the preferred method would be to use distutils to do all extension builds but in my case this is not possible or desirable. I'm dynamically generating a set of extensions as the result of a toolchain that includes SWIG and needs to be refused across multiple languages so we use Ribosome to build the actual extension files in a portable manner instead of distutils. I then copy them into the final package as datafiles. I hope that either my patch (or a modified version) can be incorporated or someone can suggest another workaround that would fix the issue allowing externally generated pyd files to have wrappers auto-generated by bdist_egg. Many thanks in advance JKP

2 3

sandbox violation
by Flávio Codeço Coelho 18 Apr '07

18 Apr '07

Hi, I have a setup.py that has recently stopped working with a sandbox-violation error message. During installation I install files under /usr/share Is installation of file outside the python path forbidden now? Is ther any workaraound? thanks, Flávio

4 4

easy_install fails with tarfile.ExtractError: could not change mode
by Michael Dunstan 14 Apr '07

14 Apr '07

Hi, I've found that for easy_install fails to install zc.recipe.testrunner for python 2.3 on OS X. % easy_install-2.3 zc.recipe.testrunner Searching for zc.recipe.testrunner Reading http://cheeseshop.python.org/pypi/zc.recipe.testrunner/ Reading http://svn.zope.org/zc.buildout Reading http://cheeseshop.python.org/pypi/zc.recipe.testrunner/1.0.0b6 Best match: zc.recipe.testrunner 1.0.0b6 Downloading http://cheeseshop.python.org/packages/source/z/zc.recipe.testrunner/zc.reci… Processing zc.recipe.testrunner-1.0.0b6.tar.gz Traceback (most recent call last): File "/opt/local/bin/easy_install-2.3", line 7, in ? sys.exit( File "/opt/local/lib/python2.3/site-packages/setuptools-0.6c5-py2.3.egg/setuptools/command/easy_install.py", line 1670, in main with_ei_usage(lambda: File "/opt/local/lib/python2.3/site-packages/setuptools-0.6c5-py2.3.egg/setuptools/command/easy_install.py", line 1659, in with_ei_usage return f() File "/opt/local/lib/python2.3/site-packages/setuptools-0.6c5-py2.3.egg/setuptools/command/easy_install.py", line 1674, in <lambda> distclass=DistributionWithoutHelpCommands, **kw File "/opt/local/lib/python2.3/distutils/core.py", line 149, in setup dist.run_commands() File "/opt/local/lib/python2.3/distutils/dist.py", line 907, in run_commands self.run_command(cmd) File "/opt/local/lib/python2.3/distutils/dist.py", line 927, in run_command cmd_obj.run() File "/opt/local/lib/python2.3/site-packages/setuptools-0.6c5-py2.3.egg/setuptools/command/easy_install.py", line 211, in run self.easy_install(spec, not self.no_deps) File "/opt/local/lib/python2.3/site-packages/setuptools-0.6c5-py2.3.egg/setuptools/command/easy_install.py", line 446, in easy_install return self.install_item(spec, dist.location, tmpdir, deps) File "/opt/local/lib/python2.3/site-packages/setuptools-0.6c5-py2.3.egg/setuptools/command/easy_install.py", line 471, in install_item dists = self.install_eggs(spec, download, tmpdir) File "/opt/local/lib/python2.3/site-packages/setuptools-0.6c5-py2.3.egg/setuptools/command/easy_install.py", line 626, in install_eggs unpack_archive(dist_filename, tmpdir, self.unpack_progress) File "/opt/local/lib/python2.3/site-packages/setuptools-0.6c5-py2.3.egg/setuptools/archive_util.py", line 67, in unpack_archive driver(filename, extract_dir, progress_filter) File "/opt/local/lib/python2.3/site-packages/setuptools-0.6c5-py2.3.egg/setuptools/archive_util.py", line 192, in unpack_tarfile tarobj._extract_member(member,dst) # XXX Ugh File "/opt/local/lib/python2.3/tarfile.py", line 1414, in _extract_member self.chmod(tarinfo, targetpath) File "/opt/local/lib/python2.3/tarfile.py", line 1532, in chmod raise ExtractError, "could not change mode" tarfile.ExtractError: could not change mode zsh: 1148 exit 1 easy_install-2.3 zc.recipe.testrunner The root of the problem seems to a file mode with the set-group-ID-on-execution bit set in the source distribution of zc.recipe.testrunner. Here is a demonstration of that: >>> import os >>> import tempfile >>> tmp_dir = tempfile.mkdtemp(prefix="easy_install-") >>> example_dir = os.path.join(tmp_dir, 'example') >>> os.mkdir(example_dir) >>> os.chmod(example_dir, 02755) Traceback (most recent call last): File "<stdin>", line 1, in ? OSError: [Errno 1] Operation not permitted: '/tmp/easy_install-UlgpCH/example' I see the same failing behaviour for handling of tempfile and chmod for Python 2.4 on OS X. However there is an explicit egg for python 2.4: http://cheeseshop.python.org/pypi/zc.recipe.testrunner/1.0.0b6#download which does not have set-group-ID-on-execution bit set for any of the archive members. So the egg easy installs just fine in that case. And I don't see any such problems using FreeBSD. Michael

2 2

[ANN] pypicache 0.1
by Robin Bryce 14 Apr '07

14 Apr '07

Hi All, I've just uploaded pypicache 0.1 to the PyPI index. I think it might be interesting to readers of this list. pypicache aims to be a relatively simple way to produce &maintain a personal package index - possibly just your PYTHON_EGG_CACHE directory - and have it published to easy_install over a LAN or intranet. Please let me know what you make of pypicache. I'm especially interested in whether or not I've correctly implemented the Project/Page mappings and down load urls. And in general please point out any glaringly bad ideas ! I'm currently hosting a small example index at http://svn.wiretooth.com/pypi/. There is a small 'first use' guide based on the steps I took to this index up and running: http://svn.wiretooth.com/svn/open/pypicache/trunk/example-firstuse.html Some more links general info: http://svn.wiretooth.com/svn/open/pypicache/tags/release-0.1/pypicache.html tool help: http://svn.wiretooth.com/svn/open/pypicache/tags/release-0.1/indexpackages-… The tool produces exactly one html page per (safe) project/version combination and generates a single mod-rewrite text map to provide the /Project/, Project/VER uri name space. Everything it generates, apache configuration and project pages, is contained in the index directory beside the distributions. Robin

1 0