Mailman 3 January 2009 - Distutils-SIG

Solaris and distutils: Need to pass LIBDIR explicitly with -L when building extensions?
by Dave Peterson 02 Feb '09

02 Feb '09

I am trying to build a number of projects that use Python extensions on Solaris 10 and I've discovered that nothing with extensions will link unless I explicitly pass in a '-L/path/to/python/lib/dir' because libpython2.5.so is not otherwise found when the '-lpython2.5' argument is specified during linking. This doesn't seem right as I don't have to specify this explicitly on Linux or Windows. Admittedly I'm using a custom built Python, so perhaps I missed some configure argument? Here's my Python build process: ./configure --prefix=/home/dpeterson/py/build --enable-shared make make test make install It is built with the gcc provided by Sun in a default install of Solaris 10, which means the gcc compiler and the sun linker. "gcc -v" gives: Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.10/3.4.3/specs Configured with: /builds/sfw10-gate/usr/src/cmd/gcc/gcc-3.4.3/configure --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++ --enable-shared Thread model: posix gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) If that all seems correct, then it appears the issue is the finalize_options() source in lib/distutils/commands/build_ext.py. There are a number of "if" blocks that explicitly append the value of distutils.sysconfig.get_config_vars('LIBDIR') to the list of library_dirs used to link built extensions with. However, there doesn't seem to be one of these for Solaris / sunos. Is this just an oversite? I've verified that adding a simple block with a 'sunos' value gate solves the problem and I no longer need to be explicit about providing '-L'. Am I missing something or should this patch be submitted as a bugfix on the issue tracker? -- Dave

7 11

Uninstall command, the return
by Tarek Ziadé 01 Feb '09

01 Feb '09

Hello Many people are asking for an uninstall command. While there are possible side-effects when removing all installed files, I think it worths it... I would like to introduce an uninstall command in distutils, using Marc-André Lemburg's mxSetup tool (see http://www.egenix.com/products/python/mxBase/) and turning it into a uninstall command. It's quite straighforward since it uses the install command in dry-run mode to get the files to remove. It requires of course to keep the source. Next, (in a second step) I was wondering if a uninstall registery could not be a good thing to have, to store a record of the installed files so there's no need to keep the source for uninstallation. This would required a new command, (and a detailed specification of course) There's an open ticket here about that (#4673) Any thoughts ? Regards Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/

7 11

(no subject)
by Zooko O'Whielacronx 31 Jan '09

31 Jan '09

Floris Bruynooghe <floris.bruynooghe(a)gmail.com> writes: > > In my ideal world python package developers would only ever have to > worry about uploading an sdist. ... > If I am correct the only reason people started to go away from this is > because Windows users don't tend to have compilers installed. I recently made this transition. For Tahoe-1.2.0 I tried to make sdists the canonical, preferred format (not, of course, omitting other formats such as .deb's). It turns out that using sdists as the primary, preferred format, failed in a lot of cases. A huge number of cases is that there is no C compiler or no Python.h -- not only on Windows but increasingly on Linux. My sons's OLPCs, for example, come with Python as the standard, integrated programming language, but not with a C compiler -- they can't afford the storage space! Many Python packages -- including three or four of the ones used by Tahoe -- have optional C/C++ extension modules which are only for CPU optimization or which are only for specific platforms or uses. This ticket is to change distutils so that these modules can be marked as optional and building the package can finish even when the compilation of the extension fails: http://bugs.python.org/issue4706 Another large number of cases was that some Python packages require special environmental customization, such as pyOpenSSL, which requires that OpenSSL binaries have been installed in just the right way before pyOpenSSL's build will work. (You would be surprised at just how much work it is to get OpenSSL installed in just the right way so that pyOpenSSL can build -- the pyOpenSSL maintainers still haven't accomplished it on Windows.) Then there is the problem that compiling certain packages takes way too long, such as my own pycryptopp package, which takes minutes to build (compiling a lot of C++ files from Wei Dai's Crypto++ library). Finally, there is the problem that a large number of users refused to *try* the sdist install because they (more or less correctly) assumed that it would require a lot of manual administration of build tools on their part and would fail on their particular platform. So, for the imminent Tahoe-1.3.0 release, I'm trying to make bdist_eggs be the default, preferred format for any packages which include C/C++ extensions, and sdist for pure-Python packages (not, of course, excluding other formats such as .deb's). Here is the decentralized, secure directory where I'm accumulating these eggs: http://testgrid.allmydata.org:3567/uri/URI:DIR2-RO:snrfwfxatrci35zdgjnzxxx2… Now if pypi is unreachable (which seems to happen a lot recently), the Tahoe build process will get the packages from that directory. Also, that URL includes the secure hash of a public key, and only authorized uploaders have the private key, and every entry in the directory is signed by that private key, so we can be more confident that these packages are the ones uploaded by those authorized people. Regards, Zooko --- Tahoe, the Least-Authority Filesystem -- http://allmydata.org store your data: $10/month -- http://allmydata.com/?tracking=zsig

1 0

[ANN] Reorganized Distutils Wiki, Cookbook, Proposal Index, FAQ and more
by Akira Kitada 31 Jan '09

31 Jan '09

Hi, I reorganized Distutils page on wiki.python.org. Here are some of the contents. * http://wiki.python.org/moin/Distutils * http://wiki.python.org/moin/Distutils/Cookbook * http://wiki.python.org/moin/Distutils/Proposals * http://wiki.python.org/moin/Distutils/FAQ

1 0

Re: [Distutils] Uninstall command, the return
by P.J. Eby 30 Jan '09

30 Jan '09

At 12:17 PM 1/30/2009 +0100, Tarek Ziadé wrote: >this code is still incomplete, I am waiting for PJE feedback to add >the support for zip files that contains >several .egg folders or zip files. But the current version is able to >find a package and its metadata. What feedback are you waiting for? I thought I already pointed you to the file format details page, and the code in pkg_resources. FWIW, I think the uninstall manifest should be done as a file within an .egg-info directory, as it's the shortest route to setuptools/distutils compatibility in this area. Distutils' install_egg_info would just make a directory with PKG-INFO and the manifest, and easy_install would record the manifest. At that point, easy_install would also be able (in principle) to stop: 1. using .pth files, 2. moving eggs to the front of PYTHONPATH, and 3. patching site.py ...which would probably make a lot of people happier. (It could stop doing these things because it could just extract active versions directly to their target directories, and move inactive versions to .egg files or directories.) I'm not saying the change will be *easy*, and uninstall will still have to be careful about namespace packages (you'd need to scan all the manifests in the target directory to see who else is using those files), but the raw material would be there to get the job done. The manifest should contain size/date/checksum information, and should use /-separated paths, and make them relative paths based on the sys.path directory, for any file that is a child of the sys.path directory. (i.e., if installed to site-packages, then any file in that package that is under site-packages should be listed relative to site-packages.) In this way, moving an entire directory (e.g., moving one of your PYTHONPATH directories) will not break the manifest. The /-separation part is also important: I've seen environments where the same libraries are in use on a network with Linux, Mac, *and* Windows machines - /-separation is the only way such an install manifest would be readable across all 3.

2 1

Zip-safe way of detecting plugins?
by Arve Knudsen 29 Jan '09

29 Jan '09

What's the safe way of detecting plugins in a package that is potentially zipped? Thanks, Arve

1 0

[ANN] stdeb 0.2.2 released
by Andrew Straw 29 Jan '09

29 Jan '09

stdeb ("setuptools debian") produces Debian source packages from Python packages via a new distutils command, sdist_dsc. Automatic defaults are provided for the Debian package, but many aspects of the resulting package can be customized via a configuration file. This new release, 0.2.2, adds optional automatic dependency finding from Zooko O'Whielacronx. Additionally, I have moved development to github. The homepage: http://github.com/astraw/stdeb Download: http://pypi.python.org/pypi/stdeb/0.2.2 Thanks and enjoy, Andrew

1 0

announcing setuptools_trial -- a setuptools plugin to run unit tests with trial
by zooko 28 Jan '09

28 Jan '09

Hey folks, if you use setuptools to build your project and you want to use Twisted trial to run the unit tests, just install setuptools_trial and run "python ./setup.py trial": http://pypi.python.org/pypi/setuptools_trial Thanks to Chris Galvan and the folks from the Elisa project. Regards, Zooko

1 0

including an empty directory
by Domen Kožar 28 Jan '09

28 Jan '09

I'm having issues to include an empty directory to setuptools package. package_data={'data':['templates/*tmpl']} this includes all the files, but there is data/templates/Test directory that is empty and there is no way of globing it.

5 10

Anyone interested in updating/reoriganizing distutils pages on wiki.python.org?
by Akira Kitada 25 Jan '09

25 Jan '09

Hi, I'm planning to reorganize distutils pages on wiki.python.org. Before I begin, I would like to know if there are anyone interested in working on this. My goals are: - Collect distutils information into one place - One top page and sub pages - Update contents of distutils page. - Have a link to distutils top page on the wiki. - etc, etc. If there is anyone interested in this, please reply this mail.

2 2