Mailman 3 March 2013 - Distutils-SIG

distlib 0.1.0 released on PyPI
by Vinay Sajip March 3, 2013

March 3, 2013

The first release of distlib - 0.1.0 - is now available on PyPI. Distlib is a low-level library of packaging functionality which is intended to be used as the basis for third-party packaging tools. This release contains the following components: 1. distlib.database - this implements a database of installed distributions, as defined by PEP 376, and distribution dependency graph logic. Support is also provided for non-installed distributions (i.e. distributions registered with metadata on an index like PyPI), including the ability to scan for dependencies and the building of dependency graphs. 2. distlib.index - this implements a means of performing operations on an index, such as registering a project, uploading a distribution or uploading documentation. Support is included for verifying SSL connections (with domain matching) and signing/verifying packages using GnuPG. 3. distlib.metadat - this implements distribution metadata as defined by PEP 345, PEP 314 and PEP 241. Support for more recent initiatives (e.g. PEP 426 - Metadata 2.0) awaits their finalisation. 4. distlib.markers - this implements environment markers as defined by PEP 345. 5. distlib.manifest - this implements lists of files used in packaging source distributions. 6. distlib.locators - allows the finding of distributions, whether on PyPI (using XML-RPC or via the "simple" scraping interface), local directories or some other source. A locator using extended JSON metadata is provided which allows dependency resolution without the need to download any distribution. 7. distlib.resources - this allows access to data files stored in Python packages, both in the file system and in .zip files. 8. distlib.scripts - facilitates installing of scripts with adjustment of shebang lines and support for native Windows executable launchers. 9. distlib.version - implements version specifiers as defined by PEP 386, but also supports working with "legacy" versions (setuptools/distribute) and semantic versions. Support for the latest version numbering scheme (PEP 426) is not far off. 10. distlib.wheel - this provides support for building and installing from the Wheel format for binary distributions (see PEP 427). 11. distlib.util - this contains miscellaneous functions and classes which are useful in packaging, but which do not fit neatly into one of the other packages in distlib. The package implements enhanced globbing functionality such as the ability to use ** in patterns to specify recursing into subdirectories. Documentation is available at [1], which will be regularly updated as development progresses, and [2], which will be updated when a release is made on PyPI. You should be able to add Disqus comments to the documentation at [1] to indicate improvements you'd like to see, or clarifications you'd like to add. Continuous integratiin test results are available at [3]. Issues should be raised using the BitBucket tracker at [4]. You can clone the repository at [5] and submit pull requests, if you'd like to contribute. I welcome your feedback. Regards, Vinay Sajip [1] http://distlib.readthedocs.org/ [2] http://pythonhosted.org/distlib/ [3] https://travis-ci.org/vsajip/distlib/ [4] https://bitbucket.org/vinay.sajip/distlib/issues/new [5] https://bitbucket.org/vinay.sajip/distlib/

2 1

Re: [Distutils] buildout and build-time dependencies
by Ben Acland March 2, 2013

March 2, 2013

> On Mar 1, 2013 5:54 AM, "Reinout van Rees" <reinout(a)vanrees.org> wrote: >> >> On 28-02-13 20:31, Ben Acland wrote: >>> >>> tl;dr: how to handle build time dependencies between python modules >>> using buildout, without looking stupid or including .tar.gz files in my >>> repo. >> numpy and scipy are a hell to install. > Would there be something we could do to help these important projects be > easier to install? I had a thought last night, after digging through zc.buildout for a while. In buildout's easy_install.py's _call_easy_install, around line 299 in v2.0.1, you call distribute's easy_install using this: exit_code = subprocess.call( list(args), env=dict(os.environ, PYTHONPATH=path)) I *think* it would work to pass in a list of other egg-producing parts, then look for those eggs at this point and add them to the path. Alternatively, a list of paths would get the job done, but you'd have to lock down the egg version before hard coding the path. Ben

1 0

wheel setup.cfg extensions
by Daniel Holth March 2, 2013

March 2, 2013

Where should these go / be defined? The wheel project understands some setup.cfg extensions that are important for Metadata 2.0 and wheel. The [metadata] section allows you to override the setup(install_requires=[...]) with values containing environment markers. [metadata] also lets you specify a LICENSE.txt to be copied into the .dist-info directory (not mentioned in METADATA) just so the resulting wheel is less likely to violate the common "include the license with all copies of this software" restriction. The [wheel] section lets you instruct bdist_wheel to tag your wheel as a universal wheel. [metadata] provides-extra = tool signatures faster-signatures requires-dist = distribute >= 0.6.30 argparse; python_version == '2.6' keyring; extra == 'signatures' dirspec; sys.platform != 'win32' and extra == 'signatures' ed25519ll; extra == 'faster-signatures' license-file = LICENSE.txt [wheel] universal=1 # use py2.py3 tag for pure-python dist

1 0

buildout and build-time dependencies
by Ben Acland March 1, 2013

March 1, 2013

Hey folks, <butt_covering> I've googled around, hit IRC, and tried a bunch of stuff to fix this. Even so, maybe I missed something obvious. Sorry if this question is redundant. </butt_covering> I'm using buildout to install a series of python modules, each of which is dependent on the others *at build time.* For example, numpy, then scipy. If I try to build scipy without numpy around, the scipy build fails complaining about not being able to find "numpy.distutils...". I can include the full text of the error here, but get the feeling that you either know what I'm talking about or not by this point. There doesn't seem to be a way to do this cleanly using buildout. Instead, I have had to put .tar.gz versions of each of the libraries in my repo along with bootstrap.py, and am installing them using a step like this: [numpy] recipe = collective.recipe.cmd cmds = mkdir tmp_numpy cd tmp_numpy tar -xzvf ${buildout:packages}/numpy-1.7.0.tar.gz ${buildout:directory}/bin/buildout setup numpy-1.7.0 install cd ../ rm -rf tmp_numpy on_install = true on_update = true ... obviously, this won't fly if I don't have sudo access, so I'm having to run buildout inside of a virtualenv. Blah blah blah, I can go into the gory details if you want, but the bottom line things that I dislike about this are: 1) I have to include the packages in the repo. 2) I'm going to have to teach my labmates how to use virtualenv AND buildout. 3) After I install them this way, I have to install them again as follows in order to get a linked interpreter: [pyscripts] recipe = zc.recipe.egg:scripts dependent-scripts = true eggs = numpy scipy networkx pydicom traits wsgiref nibabel nipype SO. Please, tell me that I'm stupid and that I've missed something obvious. tl;dr: how to handle build time dependencies between python modules using buildout, without looking stupid or including .tar.gz files in my repo. Thanks for your time and good work, Ben

3 2