The first release of distlib - 0.1.0 - is now available on PyPI. Distlib is a
low-level library of packaging functionality which is intended to be used as
the basis for third-party packaging tools.
This release contains the following components:
1. distlib.database - this implements a database of installed distributions,
as defined by PEP 376, and distribution dependency graph logic. Support is
also provided for non-installed distributions (i.e. distributions
registered with …
[View More]metadata on an index like PyPI), including the ability to
scan for dependencies and the building of dependency graphs.
2. distlib.index - this implements a means of performing operations on an
index, such as registering a project, uploading a distribution or uploading
documentation. Support is included for verifying SSL connections (with
domain matching) and signing/verifying packages using GnuPG.
3. distlib.metadat - this implements distribution metadata as defined by PEP
345, PEP 314 and PEP 241. Support for more recent initiatives (e.g. PEP 426
- Metadata 2.0) awaits their finalisation.
4. distlib.markers - this implements environment markers as defined by PEP
345.
5. distlib.manifest - this implements lists of files used in packaging source
distributions.
6. distlib.locators - allows the finding of distributions, whether on PyPI
(using XML-RPC or via the "simple" scraping interface), local directories
or some other source. A locator using extended JSON metadata is provided
which allows dependency resolution without the need to download any
distribution.
7. distlib.resources - this allows access to data files stored in Python
packages, both in the file system and in .zip files.
8. distlib.scripts - facilitates installing of scripts with adjustment of
shebang lines and support for native Windows executable launchers.
9. distlib.version - implements version specifiers as defined by PEP 386, but
also supports working with "legacy" versions (setuptools/distribute) and
semantic versions. Support for the latest version numbering scheme (PEP
426) is not far off.
10. distlib.wheel - this provides support for building and installing from the
Wheel format for binary distributions (see PEP 427).
11. distlib.util - this contains miscellaneous functions and classes which are
useful in packaging, but which do not fit neatly into one of the other
packages in distlib. The package implements enhanced globbing functionality
such as the ability to use ** in patterns to specify recursing into
subdirectories.
Documentation is available at [1], which will be regularly updated as
development progresses, and [2], which will be updated when a release is made
on PyPI. You should be able to add Disqus comments to the documentation at [1]
to indicate improvements you'd like to see, or clarifications you'd like to
add.
Continuous integratiin test results are available at [3].
Issues should be raised using the BitBucket tracker at [4]. You can clone the
repository at [5] and submit pull requests, if you'd like to contribute.
I welcome your feedback.
Regards,
Vinay Sajip
[1] http://distlib.readthedocs.org/
[2] http://pythonhosted.org/distlib/
[3] https://travis-ci.org/vsajip/distlib/
[4] https://bitbucket.org/vinay.sajip/distlib/issues/new
[5] https://bitbucket.org/vinay.sajip/distlib/
[View Less]
> On Mar 1, 2013 5:54 AM, "Reinout van Rees" <reinout(a)vanrees.org> wrote:
>>
>> On 28-02-13 20:31, Ben Acland wrote:
>>>
>>> tl;dr: how to handle build time dependencies between python modules
>>> using buildout, without looking stupid or including .tar.gz files in my
>>> repo.
>> numpy and scipy are a hell to install.
> Would there be something we could do to help these important projects be
> easier to install?
I had a …
[View More]thought last night, after digging through zc.buildout for a while.
In buildout's easy_install.py's _call_easy_install, around line 299 in v2.0.1, you call distribute's easy_install using this:
exit_code = subprocess.call(
list(args),
env=dict(os.environ, PYTHONPATH=path))
I *think* it would work to pass in a list of other egg-producing parts, then look for those eggs at this point and add them to the path. Alternatively, a list of paths would get the job done, but you'd have to lock down the egg version before hard coding the path.
Ben
[View Less]
Where should these go / be defined?
The wheel project understands some setup.cfg extensions that are
important for Metadata 2.0 and wheel. The [metadata] section allows
you to override the setup(install_requires=[...]) with values
containing environment markers.
[metadata] also lets you specify a LICENSE.txt to be copied into the
.dist-info directory (not mentioned in METADATA) just so the resulting
wheel is less likely to violate the common "include the license with
all copies of this …
[View More]software" restriction.
The [wheel] section lets you instruct bdist_wheel to tag your wheel as
a universal wheel.
[metadata]
provides-extra =
tool
signatures
faster-signatures
requires-dist =
distribute >= 0.6.30
argparse; python_version == '2.6'
keyring; extra == 'signatures'
dirspec; sys.platform != 'win32' and extra == 'signatures'
ed25519ll; extra == 'faster-signatures'
license-file = LICENSE.txt
[wheel]
universal=1 # use py2.py3 tag for pure-python dist
[View Less]
Hey folks,
<butt_covering>
I've googled around, hit IRC, and tried a bunch of stuff to fix this. Even so, maybe I missed something obvious. Sorry if this question is redundant.
</butt_covering>
I'm using buildout to install a series of python modules, each of which is dependent on the others *at build time.* For example, numpy, then scipy. If I try to build scipy without numpy around, the scipy build fails complaining about not being able to find "numpy.distutils...". I can …
[View More]include the full text of the error here, but get the feeling that you either know what I'm talking about or not by this point.
There doesn't seem to be a way to do this cleanly using buildout. Instead, I have had to put .tar.gz versions of each of the libraries in my repo along with bootstrap.py, and am installing them using a step like this:
[numpy]
recipe = collective.recipe.cmd
cmds =
mkdir tmp_numpy
cd tmp_numpy
tar -xzvf ${buildout:packages}/numpy-1.7.0.tar.gz
${buildout:directory}/bin/buildout setup numpy-1.7.0 install
cd ../
rm -rf tmp_numpy
on_install = true
on_update = true
... obviously, this won't fly if I don't have sudo access, so I'm having to run buildout inside of a virtualenv. Blah blah blah, I can go into the gory details if you want, but the bottom line things that I dislike about this are:
1) I have to include the packages in the repo.
2) I'm going to have to teach my labmates how to use virtualenv AND buildout.
3) After I install them this way, I have to install them again as follows in order to get a linked interpreter:
[pyscripts]
recipe = zc.recipe.egg:scripts
dependent-scripts = true
eggs = numpy
scipy
networkx
pydicom
traits
wsgiref
nibabel
nipype
SO. Please, tell me that I'm stupid and that I've missed something obvious.
tl;dr: how to handle build time dependencies between python modules using buildout, without looking stupid or including .tar.gz files in my repo.
Thanks for your time and good work,
Ben
[View Less]