Mailman 3 April 2009 - Distutils-SIG

Re: [Distutils] How to make easy_install handle platlibs?
by P.J. Eby April 12, 2009

April 12, 2009

At 12:36 AM 4/12/2009 -0700, Buck Golemon wrote: >On Fri, Apr 10, 2009 at 9:15 PM, P.J. Eby ><<mailto:pje@telecommunity.com>pje(a)telecommunity.com> wrote: >At 08:53 PM 4/10/2009 -0700, Buck wrote: >I see the kernel version and architecture, but this is insufficient; >RedHat 4 and RedHat 5 both use a 2.6 kernel, but the difference in >provided libraries are sufficient to make many (most?) "impure" >libraries stop working ( numpy, python-ldap, and hashlib for >examples). > >easy_install apparently knows when packages are platform-dependant, >and the necessary directory is sys.exec_prefix (or maybe better: >distutils.sysconfig.get_python_lib(plat_specific=1) ), so this seems >like an easy change. Would a patch be accepted? > > >It would be a pretty major patch, since the distutils logic that's >used for determining the installation directory is applied long >before easy_install even knows what it's installing (and therefore, >whether it's platform dependent). > >It would probably be a lot easier to improve the platform string >generation and comparison logic, as has been done for OS X. > >If the distutils logic is used, then why does numpy install to the >platlib folder using distutils, but into the nonplatlib folder using >easy_install? Please read what I wrote after the word "distutils", i.e.: """the distutils logic that's used for determining the installation directory is applied long before easy_install even knows what it's installing""" That is, the installation directory is determined *before* any packages are examined, so there is no way to know whether prefix or exec-prefix should be used. So prefix is used to determine the default installation location. Currently, there's only a way to specify one installation location. >Your tone seems to suggest that such an effort might be accepted upstream. I'm suggesting that fixing the platform strings would be more beneficial (i.e. benefit other use cases besides this one), and would probably be a LOT easier to implement as a patch, because the changes would be limited to the internals of a few well-defined, isolated functions. Of course, if you happen to find some sane way to get easy_install to do what you want, I'll certainly take a look at it. I'm just hard pressed to imagine how you'd go about doing it without major refactoring that would be very difficult to verify was done correctly.

1 0

Buildout recipe for managing .deb packages?
by Kent Tenney April 12, 2009

April 12, 2009

Howdy, Is there a recommended way to manage Debian system packages with a buildout? Thanks, Kent

8 13

Adding alias into Distutils ?
by Tarek Ziadé April 12, 2009

April 12, 2009

Hello setuptools 'alias' command is very handy <http://peak.telecommunity.com/DevCenter/setuptools#alias-define-shortcuts-f…> What about integrating it into Distutils ? Regards Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/

4 3

Re: [Distutils] How to make easy_install handle platlibs?
by P.J. Eby April 11, 2009

April 11, 2009

At 04:09 PM 4/10/2009 -0700, Buck wrote: >I'm trying to separate platform-specific and platform-independent >files on my Python installation. ./configure --prefix --exec-prefix >does this nicely for Python proper, but modules seem to be a whole >different story. > > >From my testing, distutils handles this OK, putting "pure" python >modules into the --prefix directory and modules with binary .so's into >the --execprefix. I guess it defaults the install-platbase to be the >same as the --exec-prefix used in configure? > >When running easy_install on the same packages (elementtree and >numpy), they all go into the platform-independant area. > >Is this expected? Eggs that are platform-dependent include the platform information in their filename; there isn't any support for putting them in different directories as well.

2 1

How to make easy_install handle platlibs?
by Buck April 10, 2009

April 10, 2009

I'm trying to separate platform-specific and platform-independent files on my Python installation. ./configure --prefix --exec-prefix does this nicely for Python proper, but modules seem to be a whole different story. >From my testing, distutils handles this OK, putting "pure" python modules into the --prefix directory and modules with binary .so's into the --execprefix. I guess it defaults the install-platbase to be the same as the --exec-prefix used in configure? When running easy_install on the same packages (elementtree and numpy), they all go into the platform-independant area. Is this expected? I tried settings some things in the distutils.cfg, but nothing I did seemed to help. --Buck

1 0

Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
by P.J. Eby April 10, 2009

April 10, 2009

At 05:18 PM 4/10/2009 +0100, Paul Moore wrote: >Hmm, and this is *precisely* one of the things I dislike. I don't like >anything being installed except on my request. I know this won't affect your use case, but for the benefit of everyone else listening, I just want to mention that there is a --no-deps option to easy_install. It should probably print what the dependencies are, but at least it exists.

1 0

Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
by Paul Moore April 10, 2009

April 10, 2009

2009/4/8 Tarek Ziadé <ziade.tarek(a)gmail.com>: >> Nobody seems to be attempting to collect requirements here. >> > > I do, in the wiki. I am trying to synchronize the work done at Pycon, and > in the future. I am trying to synthethize the needs there. > > http://wiki.python.org/moin/DistutilsVersionFight > http://wiki.python.org/moin/Distutils/ManifestPluginSystem > http://wiki.python.org/moin/Distutils/Metadata > http://wiki.python.org/moin/Distutils/StaticMetadata > http://wiki.python.org/moin/Distutils/StandardizeEggInfo > > What will happen in any case, is that I am going to move Distutils > forward and make decisions for Python 2.7, based on those pages that will > become PEPs at some point. I'm aware of this, and sorry if my comments seemed to ignore this (but see below) > Some people will complain here as usual, but nevertheless. :-) > So, if you and anyone wants to help, read those wiki pages > and join, by commenting them or completing them. Hmm, I'm not sure where I'd put my requirements in that lot. So I'll try putting them here (and risk the flames :-)) Please feel free to incorporate them in the Wiki in any form you feel is appropriate. 1. (Meta-requirement) I want to be able to download a Windows installer[1] for *every* package I need. 1a. This means that the barrier for packagers building Windows installers should be as low as possible. 1b. It also means that other formats (e.g. eggs) should offer no benefit over Windows installers I'm willing to concede that not all developers can be expected to build distributions for Windows (but if they are building *anything* for Windows, it should be "obvious" that a Windows installer is the only sane choice[2]). Given that, the following requirements are what I'd want for stuff I'm expected to build myself: 2. For pure Python extensions, I can take a "standard" source tarball[3] and simply run the "standard" distutils command to build a Windows installer. 2a. I can do this on any machine, even if it has no network connection (obviously I'd get the source tarball on disk or equivalent in that case). 3. For C extensions with no dependency on external libraries, I could do the same assuming I had the "correct" C compiler present (mingw or the right version of MSVC). 4. For C extensions with external dependencies, it gets more complex - but essentially it should be "easy" for the developer to write setup.py in such a way that I can download the external dependencies in some well-defined form, configure a few file locations, then build a binary without further interaction. 4a. This is certainly possible - it's how Python itself builds 4b. This is the only case, really, where pure distutils currently fails to satisfy *my* needs. But as I'd much rather have binary builds in this case, and as I currently simply don't use packages in this category that don't have binary distributions, I don't have any specific requirements here. [1] In current terms, "Windows installer" means either bdist_wininst or bdist_msi - something that integrates with Windows "Add/Remove programs". [2] That's the problem (for me) that setuptools/easy_install has introduced. There are now packages which build Windows binaries in egg format only. Clearly there is some advantage to be had currently in distributing eggs, which is *not* present in Windows installers. That advantage is not relevant to me, so I lose out because there's no Windows installer, even though the developer clearly has the means to produce one. [3] I don't distribute packages, so I am not 100% sure what a "standard" is, but I guess sdist is what I mean here, in current terms. I hope this helps. It's intended to. I tried very hard to avoid it being an anti-setuptools comment, but I have to say that before setuptools/easy_install came along, EVERY SINGLE ONE of my requirements was satisfied by core distutils (even 4 - as the fact that building binaries in this context was difficult enough that all projects in this category supplied bdist_wininst installers as a matter of course). As a user, it feels to me as if all the benefits of setuptools (and by association, of this discussion) are for the developer, at the expense of the end user (well, end users in my situation, at least). I'd like to be proved wrong over this, of course :-) Paul.

16 40

getting installation location from distutils without changing the filesystem?
by Matthias Klose April 10, 2009

April 10, 2009

The current python.m4 macro in automake (aclocal) is a bit broken, because '${prefix}' is passed literally to get_platform_lib(). currently trying to replace this with something different. Besides fixing the quoting, the implmentation seems to be questionable (currently distutils.sysconfig.get_platform_lib() is used in this test), because distutils.sysconfig.get_platform_lib() points to the platform library, not the installation the module should be installed to, which is not the same in all cases. Now trying not to use get_platform_lib(), but trying to run an setup install command with --dry-run, and make this work with any distutils included in 2.0 and up. - $ python -c "from distutils.core import setup; setup()" -n install running install running build running install_egg_info Creating /usr/local/lib/python2.6/site-packages/ Writing /usr/local/lib/python2.6/site-packages/UNKNOWN-0.0.0.egg-info and parsing the output only works for python versions providing egg-info files. - not relying on .egg-info's: $ python -c "from distutils.core import setup; setup(py_modules=['conftest'])" -n install running install running build running build_py creating build creating build/lib.linux-i686-2.6 copying conftest.py -> build/lib.linux-i686-2.6 running install_lib warning: install_lib: 'build/lib.linux-i686-2.6' does not exist -- no Python modules to install running install_egg_info Creating /usr/local/lib/python2.6/dist-packages/ Writing /usr/local/lib/python2.6/dist-packages/UNKNOWN-0.0.0.egg-info doesn't build and install the module. Building it before creates the `build' directory. All the tests could be done in a temporary directory, just using the build -b option doesn't work, because install doesn't know anything about -b. running install_lib copying build/lib.linux-i686-2.6/conftest.py -> /usr/local/lib/python2.6/dist-packages error: file '/usr/local/lib/python2.6/dist-packages/conftest.py' does not exist finishes with an error although run with -n, needs again parsing of the output. $ python -c "from distutils.core import setup; \ print setup().get_command_obj('install', 0).install_purelib" -n -q install /usr/local/lib/python2.6/dist-packages looks short, but I'm unsure about the access to the install_purelib attribute of the install command. Seems to work for python versions 2.3 and newer. Would be glad if somebody still has 2.[012] and could check. I would like to use the last option, but not yet sure about it should be allowed, or if you have to fall back to the parsing of the second option. Matthias

1 0

Re: [Distutils] setuptools still supported on Python 2.3?
by P.J. Eby April 9, 2009

April 9, 2009

At 07:43 AM 4/9/2009 +0200, Lennart Regebro wrote: >I noticed when installing setuptools on Python 2.3 that easy_install >fails because it tries to import subprocess, which is a new module in >2.4. I personally do not need Python 2.3 support any longer, but the >docs say it should be supported, which seems to be only partly true. > >The rest of setuptools seems to work on Python 2.3 still, although I >haven't tried it in practice. But the tests run. There is no code in setuptools itself that uses subprocess. Are you sure it's not a plugin, or the package you're installing?

2 1

buildout complains of unused options that are used...
by Chris Withers April 9, 2009

April 9, 2009

Hi All, I have the following buildout: [buildout] ... zeo_address = 127.0.0.1:2000 ... ... [zeo.conf] recipe = collective.recipe.template input = etc/zeo.conf.in output = etc/zeo.conf ...and buildout complains that the zeo_address line isn't used, except that it is, but collective.recipe.template. What's causing this whine and what should be doing what to mark that option as used? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

2 3