Mailman 3 February 2013 - Distutils-SIG

Telling distutils about requirements
by Erik Bernoth 11 Feb '13

11 Feb '13

Hi everybody, I think I pretty much read all of the http://docs.python.org/2/distutils/and started to create a pypi repository for my project ( http://pypi.python.org/pypi/monk_tf). Now there are some things that are not so clear from the documentation, with the most important being requirement handling. I have the same requirements written down in two ways: a) a requirements.txt file, which can be called with pip install -r requirements.txt. Yet I don't see any user downloading a requirements.txt file from somewhere, then installing it and only then afterwards getting started with actually installing the package they want to install. Who would do that? b) requires attribute in the setup function call in setup.py. For some reason pip completely seems to ignore it. I tested the following way (come along with the code from https://github.com/DFE/MONK, if you like): $ cd MONK $ python setup.py sdist $ cd dist $ tar xfvz monk_tf-v0.1.1.tar.gz $ cd monk_tf-v0.1.1 $ python setup.py install running install running build running build_py running install_lib running install_egg_info Writing /usr/local/lib/python2.7/dist-packages/monk_tf-v0.1.1.egg-info $python >> import monk_tf (Exception, because a required package can't be found) So this also didn't seem to install any of the required packages. I'd really like to know, what I am doing wrong here. Anybody ideas or suggestions? Is there another way to tell distutils about the packages that should be installed before my package is installed? Cheers Erik

3 9

Problems installing otrace - unexpanded "@libdir@" string
by Skip Montanaro 11 Feb '13

11 Feb '13

At work we use encap to create packages to distribute to our desktops and server. Creating a package involves installing it into an isolated directory, then creating an encap package from the directory's contents. Packages which expect to use distutils are a bit problematic, but with some magic sauce on the command line, these have become tractable. I'm having trouble installing otrace (http://pypi.python.org/pypi/otrace). It appears to go well enough: % python setup.py install --single-version-externally-managed --root / --install-lib=/var/tmp/python27_otrace-0.30.9/lib/python2.7/site-packages --prefix=/var/tmp/python27_otrace-0.30.9 running install running build running build_py running install_lib creating /var/tmp/python27_otrace-0.30.9 creating /var/tmp/python27_otrace-0.30.9/lib creating /var/tmp/python27_otrace-0.30.9/lib/python2.7 creating /var/tmp/python27_otrace-0.30.9/lib/python2.7/site-packages copying build/lib/otrace.py -> /var/tmp/python27_otrace-0.30.9/lib/python2.7/site-packages byte-compiling /var/tmp/python27_otrace-0.30.9/lib/python2.7/site-packages/otrace.py to otrace.pyc running install_egg_info running egg_info writing otrace.egg-info/PKG-INFO writing top-level names to otrace.egg-info/top_level.txt writing dependency_links to otrace.egg-info/dependency_links.txt writing entry points to otrace.egg-info/entry_points.txt reading manifest file 'otrace.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'otrace.egg-info/SOURCES.txt' Copying otrace.egg-info to /var/tmp/python27_otrace-0.30.9/lib/python2.7/site-packages/otrace-0.30.9-py2.7.egg-info running install_scripts Installing otrace script to /var/tmp/python27_otrace-0.30.9/bin When I look at the generated otrace script I see something that doesn't look right though: % cat /var/tmp/python27_otrace-0.30.9/bin/otrace #!/opt/local/bin/python # EASY-INSTALL-ENTRY-SCRIPT: 'otrace==0.30.9','console_scripts','otrace' __requires__ = 'otrace==0.30.9' import sys sys.path.append('@libdir@') from pkg_resources import load_entry_point if __name__ == '__main__': sys.exit( load_entry_point('otrace==0.30.9', 'console_scripts', 'otrace')() ) What is "@libdir@", and why isn't it being expanded to something useful? This script appears to be generated by Distutils. It's not part of otrace proper. Looking back at other installs (virtualenv, pylint, pip, and a few others), I see that otrace is not the first afflicted package. Unfortunately, all search attempts using Google have failed, as no matter what I've tried, it elides the @-signs from my queries.Needless to say, there are gazillions of instances of "libdir" related to distutils. (Bing seems to behave better in this regard, but still yielded nothing useful.) Any suggestions appreciated. Thx... Skip

4 4

buildout 2.0.0b2 released
by Jim Fulton 08 Feb '13

08 Feb '13

Changes: http://pypi.python.org/pypi/zc.buildout/2.0.0b2#b2-2013-02-02 This release adds features from buildout-versions. Much thanks to Reinout van Rees for doing this work and doing it so quickly. If problems aren't discovered first, I plan to make the final release next weekend. To install the new release, download: http://downloads.buildout.org/2/bootstrap.py if you haven't already. (It hasn't changed since 2.0.0a6.) Then bootstrap projects with: python bootstrap.py -t Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm

2 1

Re: [Distutils] PEP 426, round 733 ;)
by a.cavallo＠cavallinux.eu 06 Feb '13

06 Feb '13

Mine wasn't an objection: it was a plain refusal. On Wed 06/02/13 18:00, "Daniel Holth" dholth(a)gmail.com wrote: > On Wed, Feb 6, 2013 at 11:37 AM, wrote: > Feel free to adopt whatever you think is the "best" practice: I dont > understand > whats wrong with 1.1.99 instead the "magic" 1.2b2. > > I followed these "lengthy discussions" .. if an agreement was found > and was > technically sound why do you think people still arguing about that? > And were > talking years not hours to come up with those peps. > > Its non-adopted, non-final predecessor turns 8 in April. > Unfortunately these kinds of things can be argued endlessly. > > I like a joke from time to time: python -c print "1.2.dev1" "1.2.1" -> False > Even easier in my unicors populated universe. > > Ill simply ignore anything about those peps, for what it matters > > In that case after these last objections are dropped lets accept this > PEP. Hooray! You can start generating Metadata 1.3 today with > bdist_wheel, and distribute already understands the Provides-Extra: > feature used to represent setuptools extras. > Description-in-body-section is also trouble-free with no changes to > pkg_resources. > > As a comparison, rubygems says > (http://guides.rubygems.org/patterns/#prerelease-gems [1]) > > Many gem developers have versions of their gem ready to go out for > testing or “edge” releases before a big gem release. RubyGems > supports the concept of “prerelease” versions, which could be > betas, alphas, or anything else that isn’t worthy of a real gem > release yet. > > Taking advantage of this is easy. All you need is one or more > letters in the gem version. For example, here’s what a prerelease > gemspec’s version field might look like: > > Gem::Specification.new do |s| s.name [2] = "hola" s.version = > "1.0.0.pre" > > Other prerelease version numbers might include 2.0.0.rc1, or > 1.5.0.beta.3. It just has to have a letter in it, and you’re set. > These gems can then be installed with the --pre flag, like so: > > Daniel Holth > > > > Links: > ------ > [1] http://guides.rubygems.org/patterns/#prerelease-gems > [2] http://s.name > >

3 3

Re: [Distutils] PEP 426, round 733 ;)
by a.cavallo＠cavallinux.eu 06 Feb '13

06 Feb '13

Feel free to adopt whatever you think is the "best" practice: I don't understand what's wrong with 1.1.99 instead the "magic" 1.2b2. I followed these "lengthy discussions" .. if an agreement was found and was technically sound why do you think people still arguing about that? And we're talking years not hours to come up with those peps. I like a joke from time to time: python -c 'print "1.2.dev1" < "1.2.1"' -> False Even easier in my unicors populated universe. I'll simply ignore anything about those peps, for what it matters. On Wed 06/02/13 17:01, "Erik Bray" erik.m.bray(a)gmail.com wrote: > On Tue, Feb 5, 2013 at 10:53 AM, Donald Stufft <donal > d.stufft(a)gmail.com> wrote: > On Tuesday, February 5, 2013 at 10:49 AM, a.cav > allo(a)cavallinux.eu wrote: > > > Or until when somebody will explain if X.Y.devN > comes before of after > X.Y.N... > > > > Easy, before. > > Which is explained in the PEP pretty clearly I thought. But if you're > still not sure there were lengthy discussions on this very mailing > list that led to that decision. For what it's worth, there were > people who argued the opposite too, but eventually it seems like a > consensus was reached. > > >

2 1

Re: [Distutils] PEP 426, round 733 ;)
by a.cavallo＠cavallinux.eu 06 Feb '13

06 Feb '13

I respectfully disagree, for what it matters I will ignore the whole stuff. These PEPs have been around for ages. And I supsect they will be around until the end of times. Or until when somebody will explain if X.Y.devN comes before of after X.Y.N... On Tue 05/02/13 15:44, "Nick Coghlan" ncoghlan(a)gmail.com wrote: > On Wed, Feb 6, 2013 at 12:15 AM, <a.cav > allo(a)cavallinux.eu> wrote: > > > Not quite correct: 1.1.x and 1.2.x are two separate > branches in Django. > > > So along the 1.1.x branch the timestamp identifies > what comes first and after. > Think of a plane where the coordinates are the > timestamp and the branch: to > compare you need fix one of the two (the branch in > this case). > > > BTW the fact that in your mind 1.2.x comes later > than 1.1.x is because you're > already "sorting" them adding a semantic value built > into the version *LABEL*. > For a counter example how would compare then 1.x and > 2.x: is 2.x continuing along > 1.x or is it a completely unrelated source? And that > *cannot* be captured. > > > My point in suggesting a timestamp is because it > freeze the repository state in a > unique way: scms have already that concept built > into it (even distributed scm > have that!). > > The version scheme is not going to change. The point of PEP 386 was, > to a very large extent, to define a scheme that *existing PyPI > projects* either already comply with, or will require only minor > cosmetic changes to comply with (such as inserting an extra period to > turn X.YdevN into X.Y.devN). > > In other words, the intent was not to invent a new versioning scheme, > but merely to formalise a subset of one that already existed in the > wild. One of the main goals of PEP 426 is to *lower* barriers to > adoption for the new metadata standard, not to create new ones - > throwing away the work that went into creating the PEP 386 versioning > scheme would be a foolish waste of time and energy. Even the problem > with the sorting of dev releases was enough to hinder adoption of v1.2 > of the metadata - there's no way we're going to try to enforce a more > radical shift in the community approaches versioning. We already know > the most likely outcome of such an effort: people will simply stick > with v1.1 of the metadata scheme and continuing to use the existing > packaging tools, as migrating to the new ones would require a whole > lot pointless busywork to redesign their build processes and > workflows. > > Regards, > Nick. > > -- > Nick Coghlan | ncoghlan@g > mail.com | Brisbane, Australia > >

3 2

Re: [Distutils] PEP 426, round 733 ;)
by a.cavallo＠cavallinux.eu 06 Feb '13

06 Feb '13

I followed in the past the discussion that lead to the pep. I've lost any interest in it because I had a clear perception there was more interest in "splitting the hair" that solving any real problem, *and that was years ago*. That alone should trigger some doubt about the validity of the points made. My whole point would be: if walks like a duck sounds like a duck *don't call it a plane*. *If a version has to be enforced* it'd better sommething simple as a major.minor[.micro] or an integer or a timestamp for that matter. Something that could be "simply" sorted without requiring magic handling. Ideally it would be something that connects to the source revision number (as in subversion) or the integral id or even a timestamp (that's what an exported version must be). BTW Rpm has a "version" and (as fallback) an Epoch field overriding the version to "repair" such a problem: not used often these days, but it is there. On Tue 05/02/13 09:08, "Ronald Oussoren" ronaldoussoren(a)mac.com wrote: > > On 4 Feb, 2013, at 20:59, Antonio Cavallo <a.cav > allo(a)cavallinux.eu> wrote: > > > Because the version number is just more > complicated? The details have been ... > > > Nope, the whole point is it shouldn't. If that has > to be enforced why adding "marketing alert" to it? Why choosing something > complex over something simple? > > > In the correct world (mine where unicorns live > freely) I should be able to retrieve the code that goes with an installed > package just using that version and rebuild it (in a repeatable > way)! > > > Or you're talking about a "label" instead a version? > In which case you shouldn't really compare them! > I'm talking about version numbers in the real world and those just aren't > that simple. The scheme described in the PEP is fairly complex, but as Nick > mentioned not all features will be used at the same time. The scheme also > gives a clear and sane meaning to almost all version numbers actually used > on PyPI, please check the archives to see the discussions that led to this > PEP. > That doesn't mean that the documentation can't nudge users towards a simple > major.minor[.micro] scheme, possibly referencing semantic versioning. > Ronald > >

6 6

mebs, the meta-build system
by Daniel Holth 06 Feb '13

06 Feb '13

or, removing distutils without replacing it. ** not an actual project ** Proposing mebs, the meta-build system. What if pip did not depend on setuptools or distutils and the stdlib did not include distutils or any other build system? Instead, the installer can only install binary packages, and build systems do not install but only build binary packages. A very simple meta-build system "mebs" is used to recognize sdists and build binary packages. Build systems provide plugins having three methods, .recognize() .metadata() .build() An installer downloads an sdist. For each installed build plugin, .recognize(dir) is called. The first plugin to return True is used. The plugin's .metadata(dir) is called, returning Metadata 1.3+ format metadata in a simple email.parser.Parser() multi-dict interface. Used to fetch any build/install requirements. .build(dir) is called to create the binary package. The installer installs the binary package. The first time you download an sdist, the build system/installer has to download a pre-built binary package of distutils (because it has been removed from the standard library). Alternatively a minimal setup.cfg build_system=x is consulted to download something other than distutils, the newly installed build plugin recognizes the sdist, and the package is built and installed. You wouldn't have to actually remove distutils, but every other build system could be made to compete with it on equal footing. Binary packages are used as the lingua franca because, unlike build systems, binary packages are simple enough that there can be multiple compatible implementations.

4 5

Re: [Distutils] PEP 426, round 733 ;)
by a.cavallo＠cavallinux.eu 05 Feb '13

05 Feb '13

I agree *completely* with Philippe here. If a version standard will be enforced what's the point of making it more complicated that a sequential number or something along x.y.z? In the end that's what the version number is. On Mon 04/02/13 16:31, "Philippe Ombredanne" pombredanne(a)nexb.com wrote: > On Mon, Feb 4, 2013 at 2:10 PM, Nick Coghlan <ncoghlan@g > mail.com> wrote: > As usual, PEP inline below and on the web > at > http://www.python.org/dev/peps/pep-0426/ >> Version scheme > > ============== > > Version numbers must comply with the following > scheme:: > N.N[.N]+[{a|b|c|rc}N][.postN][.devN] > > > > Version numbers which do not comply with this scheme > are an error. Projects > which wish to use non-compliant version numbers must > restrict themselves > IMHO a version (or eventually its dot-separated segments with > precedence from left to right) should increase when sorted > lexicographically so it is never ambiguous for a human reading a list .... > Are we trying to make a version -- which is an engineering must -- > into something that has also some semantics about the level of > completion of a project or some "marketing" alert on the level of > maturity of a software release? Could we stay instead in the realm of > engineering? > > I think that trying to inject things like alpha, beta, post, dev, > release candidates and the likes in this is trying to bake in too many > things that are eventually just the practices of some projects and > should not be the frozen practice baked in a PEP. Instead, this > should be left to project authors to define their own scheme as long > as it sorts lexicographically (eventually by segments, with precedence > from left to right). > > -- > Philippe Ombredanne > > +1 650 799 0949 | pombredanne@n > exB.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com _______________________________________________ > Distutils-SIG maillist - Distutils-SIG(a)python.org > http://mail.python.org/mailman/listinfo/distutils-sig > >

4 4

PEP 426, round 733 ;)
by Nick Coghlan 04 Feb '13

04 Feb '13

(I think we're getting pretty close now...) As usual, PEP inline below and on the web at http://www.python.org/dev/peps/pep-0426/ Diff from the previous version is here (the abstract is fixed in the next commit): http://hg.python.org/peps/rev/7f36cb23fb6d >From the commit message: - include an example of Python-Requires + environment markers - allow a bare Extension field (and give an example use case) - be explicit that projects that refuse to use compliant version numbering must continue to use metadata v1.1 - add guidelines for sorting across metadata versions - add guidelines for dependencies across metadata versions - point out that the new sort order is more consistent with that of pkg_resources - mention the implicit test and doc features in the rationale for supporting optional features in general - consistently use sentence case for headings, instead of a mix of sentence case and title case - other typo fixes Regards, Nick. ================================== PEP: 426 Title: Metadata for Python Software Packages 1.3 Version: $Revision$ Last-Modified: $Date$ Author: Daniel Holth <dholth(a)fastmail.fm>, Donald Stufft <donald(a)stufft.io>, Nick Coghlan <ncoghlan(a)gmail.com> Discussions-To: Distutils SIG Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 30 Aug 2012 Abstract ======== This PEP describes a mechanism for adding metadata to Python distributions. It includes specifics of the field names, and their semantics and usage. This document specifies version 1.3 of the metadata format. Version 1.0 is specified in PEP 241. Version 1.1 is specified in PEP 314. Version 1.2 is specified in PEP 345. Version 1.3 of the metadata format adds fields designed to make third-party packaging of Python Software easier and defines a formal extension mechanism. It also support for optional features of distributions and allows the description to be placed into a payload section. Finally, this version addresses several issues with the previous iteration of the standard version numbering scheme. Metadata files ============== The syntax defined in this PEP is for use with Python distribution metadata files. The file format is a simple UTF-8 encoded Key: value format with case-insensitive keys and no maximum line length, optionally followed by a blank line and a payload containing a description of the distribution. This format is parseable by the ``email`` module with an appropriate ``email.policy.Policy()``. When ``metadata`` is a Unicode string, ```email.parser.Parser().parsestr(metadata)`` is a serviceable parser. There are two standard locations for these metadata files: * the ``PKG-INFO`` file included in the base directory of Python source distribution archives (as created by the distutils ``sdist`` command) * the ``.dist-info/METADATA`` files in a Python installation database, as described in PEP 376. Other tools involved in Python distribution may also use this format. Encoding ======== Metadata 1.3 files are UTF-8 with the restriction that keys must be ASCII. Parser implementations should be aware that older versions of the Metadata specification do not specify an encoding. Metadata header fields ======================= This section specifies the names and semantics of each of the supported fields in the metadata header. In a single Metadata 1.3 file, fields marked with "(optional)" may occur 0 or 1 times. Fields marked with "(multiple use)" may be specified 0, 1 or more times. Only "Metadata-Version", "Name", "Version", and "Summary" must appear exactly once. The fields may appear in any order within the header section of the file. Metadata-Version ---------------- Version of the file format; "1.3" is the only legal value. Example:: Metadata-Version: 1.3 Name ---- The name of the distribution. Example:: Name: BeagleVote Version ------- A string containing the distribution's version number. See `Version scheme`_ below. Example:: Version: 1.0a2 Summary ------- A one-line summary of what the distribution does. Example:: Summary: A module for collecting votes from beagles. Description (optional, deprecated) ---------------------------------- Starting with Metadata 1.3, the recommended place for the description is in the payload section of the document, after the last header. The description does not need to be reformatted when it is included in the payload. See `Describing the Distribution`_ for more information on the expected contents of this field. Since a line separator immediately followed by another line separator indicates the end of the headers section, any line separators in a ``Description`` header field must be suffixed by whitespace to indicate continuation. It is an error to provide both a ``Description`` header and a metadata payload. Keywords (optional) ------------------- A list of additional whitespace separated keywords to be used to assist searching for the distribution in a larger catalog. Example:: Keywords: dog puppy voting election Home-page (optional) -------------------- A string containing the URL for the distribution's home page. Example:: Home-page: http://www.example.com/~cschultz/bvote/ Download-URL (optional) ----------------------- A string containing the URL from which this version of the distribution can be downloaded. (This means that the URL can't be something like ".../BeagleVote-latest.tgz", but instead must be ".../BeagleVote-0.45.tgz".) Project-URL (multiple use) -------------------------- A string containing a label and a browsable URL for the project, separated by the last occurrence of comma and space ", ". The label consists of any permitted header text, including commas. Example:: Bug, Issue Tracker, http://bitbucket.org/tarek/distribute/issues/ Author (optional) ----------------- A string containing the author's name at a minimum; additional contact information may be provided. Example:: Author: C. Schultz, Universal Features Syndicate, Los Angeles, CA <cschultz(a)peanuts.example.com> Author-email (optional) ----------------------- A string containing the author's e-mail address. It contains a name and e-mail address in the RFC 5322 recommended ``Address Specification`` format. Example:: Author-email: "C. Schultz" <cschultz(a)example.com> Maintainer (optional) --------------------- A string containing the maintainer's name at a minimum; additional contact information may be provided. Note that this field is intended for use when a project is being maintained by someone other than the original author: it should be omitted if it is identical to ``Author``. Example:: Maintainer: C. Schultz, Universal Features Syndicate, Los Angeles, CA <cschultz(a)peanuts.example.com> Maintainer-email (optional) --------------------------- A string containing the maintainer's e-mail address. It has the same format as ``Author-email``. Note that this field is intended for use when a project is being maintained by someone other than the original author: it should be omitted if it is identical to ``Author-email``. Example:: Maintainer-email: "C. Schultz" <cschultz(a)example.com> License (optional) ------------------ Text indicating the license covering the distribution where the license is not a selection from the "License" Trove classifiers. See "Classifier" below. This field may also be used to specify a particular version of a license which is named via the ``Classifier`` field, or to indicate a variation or exception to such a license. Examples:: License: This software may only be obtained by sending the author a postcard, and then the user promises not to redistribute it. License: GPL version 3, excluding DRM provisions The full text of the license would normally be included in a separate file. Classifier (multiple use) ------------------------- Each entry is a string giving a single classification value for the distribution. Classifiers are described in PEP 301 [2]. Examples:: Classifier: Development Status :: 4 - Beta Classifier: Environment :: Console (Text Based) Provides-Dist (multiple use) ---------------------------- Each entry contains a string naming a requirement that is satisfied by installing this distribution. These strings must be of the form ``Name`` or ``Name (Version)``, following the formats of the corresponding field definitions. For ease of metadata consumption, distributions are required to explicitly include a ``Provides-Dist`` entry for their own name and version. This also allows developers of a project to discourage users explicitly depending on the project (by deliberately omitting this entry). A distribution may provide additional names, e.g. to indicate that multiple projects have been merged into and replaced by a single distribution or to indicate that this project is a substitute for another. For instance, distribute (a fork of setuptools) can include a ``Provides-Dist: setuptools`` entry to prevent the conflicting package from being downloaded and installed when distribute is already installed. A distribution that has been merged with another might ``Provides-Dist`` the obsolete name(s) to satisfy any projects that require the obsolete distribution's name. A distribution may also provide a "virtual" project name, which does not correspond to any separately-distributed project: such a name might be used to indicate an abstract capability which could be supplied by one of multiple projects. E.g., multiple projects might supply RDBMS bindings for use by a given ORM: each project might declare that it provides ``ExampleORM-somedb-bindings``, allowing other projects to depend only on having at least one of them installed. A version declaration may be supplied and must follow the rules described in `Version scheme`_. The distribution's version number will be implied if none is specified. Examples:: Provides-Dist: ThisProject Provides-Dist: AnotherProject (3.4) Provides-Dist: virtual_package Provides-Extra (multiple use) ----------------------------- A string containing the name of an optional feature. Must be printable ASCII, not containing whitespace, comma (,), or square brackets []. May be used to make a dependency conditional on whether the optional feature has been requested. See `Optional Features`_ for details on the use of this field. Example:: Name: beaglevote Provides-Extra: pdf Requires-Dist: reportlab; extra == 'pdf' Requires-Dist: nose; extra == 'test' Requires-Dist: sphinx; extra == 'doc' Obsoleted-By (optional) ----------------------- Indicates that this project is no longer being developed. The named project provides a substitute or replacement. A version declaration may be supplied and must follow the rules described in `Version specifiers`_. Possible uses for this field include handling project name changes and project mergers. Examples:: Name: BadName Obsoleted-By: AcceptableName Name: SeparateProject Obsoleted-By: MergedProject (>=4.0.0) Requires-Dist (multiple use) ---------------------------- Each entry contains a string naming some other distutils project required by this distribution. The format of a requirement string is identical to that of a distribution name (e.g., as found in the ``Name:`` field) optionally followed by a version declaration within parentheses. The distribution names should correspond to names as found on the `Python Package Index`_; often the same as, but distinct from, the module names as accessed with ``import x``. Version declarations must follow the rules described in `Version specifiers`_ Distributions may also depend on optional features of other distributions. See `Optional Features`_ for details. Examples:: Requires-Dist: pkginfo Requires-Dist: PasteDeploy Requires-Dist: zope.interface (>3.5.0) Dependencies mentioned in ``Requires-Dist`` may be installed exclusively at run time and are not guaranteed to be available when creating or installing a package. If a dependency is needed during distribution creation or installation *and* at run time, it should be listed under both ``Requires-Dist`` and ``Setup-Requires-Dist``. Setup-Requires-Dist (multiple use) ---------------------------------- Like ``Requires-Dist``, but names dependencies needed in order to build, package or install the distribution. Commonly used to bring in extra compiler support or a package needed to generate a manifest from version control. Distributions may also depend on optional features of other distributions. See `Optional Features`_ for details. Examples:: Setup-Requires-Dist: custom_setup_command Dependencies mentioned in ``Setup-Requires-Dist`` may be installed exclusively for setup and are not guaranteed to be available at run time. If a dependency is needed during distribution creation or installation *and* at run time, it should be listed under both ``Requires-Dist`` and ``Setup-Requires-Dist``. Requires-Python (multiple use) ------------------------------ This field specifies the Python version(s) that the distribution is guaranteed to be compatible with. Version declarations must be in the format specified in `Version specifiers`_. Examples:: Requires-Python: 3.2 Requires-Python: >3.1 Requires-Python: >=2.3.4 Requires-Python: >=2.5,<2.7 If specified multiple times, the Python version must satisfy all such constraints to be considered compatible. This is most useful in combination with appropriate `Environment markers`_. For example, if a feature was initially introduced to Python as a Unix-specific addition, and then Windows support was added in the subsequent release, this could be indicated with the following pair of entries:: Requires-Python: >= 3.1 Requires-Python: >= 3.2; sys.platform == 'win32' Requires-External (multiple use) -------------------------------- Each entry contains a string describing some dependency in the system that the distribution is to be used. This field is intended to serve as a hint to downstream project maintainers, and has no semantics which are meaningful to the ``distutils`` distribution. The format of a requirement string is a name of an external dependency, optionally followed by a version declaration within parentheses. Because they refer to non-Python software releases, version numbers for this field are **not** required to conform to the format described in `Version scheme`_: they should correspond to the version scheme used by the external dependency. Notice that there is no particular rule on the strings to be used. Examples:: Requires-External: C Requires-External: libpng (>=1.5) Platform (multiple use) ----------------------- A Platform specification describing an operating system supported by the distribution which is not listed in the "Operating System" Trove classifiers. See `Classifier`__ above. __ `Classifier (multiple use)`_ Examples:: Platform: ObscureUnix Platform: RareDOS Supported-Platform (multiple use) --------------------------------- Binary distributions containing a metadata file will use the Supported-Platform field in their metadata to specify the OS and CPU for which the binary distribution was compiled. The semantics of the Supported-Platform field are not specified in this PEP. Example:: Supported-Platform: RedHat 7.2 Supported-Platform: i386-win32-2791 Extension (multiple use) ------------------------ An ASCII string, not containing whitespace or the ``/`` character, that indicates the presence of extended metadata. The additional fields defined by the extension are then prefixed with the name of the extension and the ``/`` character. For example:: Extension: Chili Chili/Type: Poblano Chili/Heat: Mild To avoid name conflicts, it is recommended that distribution names be used to identify metadata extensions. This practice will also make it easier to find authoritative documentation for metadata extensions. As the order of the metadata headers is not constrained, the ``Extension: Chili`` field may appear before or after the corresponding extension fields ``Chili/Type:`` etc. Values in extension fields must still respect the general formatting requirements for metadata headers. A bare ``Extension: Name`` entry with no corresponding extension fields is permitted. It may, for example, indicate the expected presence of an additional metadata file rather than the presence of extension fields. An extension field with no corresponding ``Extension: Name`` entry is an error. Describing the distribution =========================== The distribution metadata should include a longer description of the distribution that may run to several paragraphs. Software that deals with metadata should not assume any maximum size for the description. The recommended location for the description is in the metadata payload, separated from the header fields by at least one completely blank line (that is, two successive line separators with no other characters between them, not even whitespace). Alternatively, the description may be provided in the `Description`__ metadata header field. Providing both a ``Description`` field and a payload is an error. __ `Description (optional, deprecated)`_ The distribution description can be written using reStructuredText markup [1]_. For programs that work with the metadata, supporting markup is optional; programs may also display the contents of the field as plain text without any special formatting. This means that authors should be conservative in the markup they use. Version scheme ============== Version numbers must comply with the following scheme:: N.N[.N]+[{a|b|c|rc}N][.postN][.devN] Version numbers which do not comply with this scheme are an error. Projects which wish to use non-compliant version numbers must restrict themselves to metadata v1.1 (PEP 314) or earlier, as those versions do not mandate a particular versioning scheme. Suffixes and ordering --------------------- The following suffixes are the only ones allowed at the given level of the version hierarchy and they are ordered as listed. Within a numeric release (``1.0``, ``2.7.3``):: .devN, aN, bN, cN, rcN, <no suffix>, .postN Within an alpha (``1.0a1``), beta (``1.0b1``), or release candidate (``1.0c1``, ``1.0rc1``):: .devN, <no suffix>, .postN Within a post release (``1.0.post1``):: devN, <no suffix> Note that ``devN`` and ``postN`` must always be preceded by a dot, even when used immediately following a numeric version (e.g. ``1.0.dev456``, ``1.0.post1``). Within a given suffix, ordering is by the value of the numeric component. Note that `rc` will always sort after `c` (regardless of the numeric component) although they are semantically equivalent. It is suggested that within a particular project you do not mix `c` and `rc`, especially within the same numeric version. Example version order --------------------- :: 1.0.dev456 1.0a1 1.0a2.dev456 1.0a12.dev456 1.0a12 1.0b1.dev456 1.0b2 1.0b2.post345 1.0c1.dev456 1.0c1 1.0 1.0.post456.dev34 1.0.post456 1.1.dev1 Ordering across different metadata versions ------------------------------------------- After making a release with a given metadata version, it is assumed that projects will not revert to an older metadata version. Accordingly, and to allow projects with non-compliant version schemes to more easily migrate to the version scheme defined in this PEP, releases should be sorted by their declared metadata version *before* being sorted by the distribution version. Software that processes distribution metadata should account for the fact that metadata v1.0 (PEP 241) and metadata v1.1 (PEP 314) do not specify a standard version numbering or sorting scheme. This PEP does not mandate any particular approach to handling such versions, but acknowledges that the de facto standard for sorting such versions is the scheme used by the ``pkg_resources`` component of ``setuptools``. For metadata v1.2 (PEP 345), the recommended sort order is defined in PEP 386 (It is expected that projects where the defined PEP 386 sort order is incorrect will skip straight from metadata v1.1 to metadata v1.3). Version specifiers ================== A version specifier consists of a series of version clauses, separated by commas. Each version clause consists of an optional comparison operator followed by a version number. For example:: 0.9, >= 1.0, != 1.3.4, < 2.0 Each version number must be in the standard format described in `Version scheme`_. Comparison operators must be one of ``<``, ``>``, ``<=``, ``>=``, ``==`` and ``!=``. When no comparison operator is provided, it is equivalent to using the following pair of version clauses:: >= V, < V+1 where ``V+1`` is the "next version" after ``V``, as determined by incrementing the last numeric component in ``V`` (for example, if ``V == 1.0a3``, then ``V+1 == 1.0a4``, while if ``V == 1.0``, then ``V+1 == 1.1``). The comma (",") is equivalent to a logical **and** operator. Whitespace between a conditional operator and the following version number is optional, as is the whitespace around the commas. Pre-releases of any kind (indicated by the presence of ``dev``, ``a``, ``b``, ``c`` or ``rc`` in the version number) are implicitly excluded from all version specifiers, *unless* a pre-release version is explicitly mentioned in one of the clauses. For example, this specifier implicitly excludes all pre-releases of later versions:: >= 1.0 While these specifiers would include them:: >= 1.0a1 >= 1.0c1 >= 1.0, != 1.0b2 >= 1.0, < 2.0.dev123 Dependency resolution tools should use the above rules by default, but may also allow users to request the following alternative behaviours: * accept already installed pre-releases for all version specifiers * retrieve and install available pre-releases for all version specifiers Post releases and purely numeric releases receive no special treatment - they are always included unless explicitly excluded. Given the above rules, projects which include the ``.0`` suffix for the first release in a series, such as ``2.5.0``, can easily refer specifically to that version with the clause ``2.5.0``, while the clause ``2.5`` refers to that entire series. Projects which omit the ".0" suffix for the first release of a series, by using a version string like ``2.5`` rather than ``2.5.0``, will need to use an explicit clause like ``>= 2.5, < 2.5.1`` to refer specifically to that initial release. Some Examples: - ``Requires-Dist: zope.interface (3.1)``: any version that starts with 3.1, excluding pre-releases. - ``Requires-Dist: zope.interface (==3.1)``: equivalent to ``Requires-Dist: zope.interface (3.1)``. - ``Requires-Dist: zope.interface (3.1.0)``: any version that starts with 3.1.0, excluding pre-releases. Since that particular project doesn't use more than 3 digits, it also means "only the 3.1.0 release". - ``Requires-Python: 3``: Any Python 3 version, excluding pre-releases. - ``Requires-Python: >=2.6,<3``: Any version of Python 2.6 or 2.7, including post releases (if they were used for Python). It excludes pre releases of Python 3. - ``Requires-Python: 2.6.2``: Equivalent to ">=2.6.2,<2.6.3". So this includes only Python 2.6.2. Of course, if Python was numbered with 4 digits, it would include all versions of the 2.6.2 series, excluding pre-releases. - ``Requires-Python: 2.5``: Equivalent to ">=2.5,<2.6". - ``Requires-Dist: zope.interface (3.1,!=3.1.3)``: any version that starts with 3.1, excluding pre-releases of 3.1 *and* excluding any version that starts with "3.1.3". For this particular project, this means: "any version of the 3.1 series but not 3.1.3". This is equivalent to: ">=3.1,!=3.1.3,<3.2". - ``Requires-Python: >=3.3a1``: Any version of Python 3.3+, including pre-releases like 3.4a1. Depending on distributions that use non-compliant version schemes ----------------------------------------------------------------- A distribution using this version of the metadata standard may need to depend on another distribution using an earlier version of the metadata standard and a non-compliant versioning scheme. The normal ``Requires-Dist`` and ``Setup-Requires-Dist`` fields can be used for such dependencies, so long as the dependency itself can be expressed using a compliant version specifier. For more exotic dependencies, a metadata extension would be needed in order to express the dependencies accurately while still obeying the restrictions on standard version specifiers. The ``Requires-External`` field may also be used, but would not be as amenable to automatic processing. Environment markers =================== An **environment marker** is a marker that can be added at the end of a field after a semi-colon (";"), to add a condition about the execution environment. Here are some example of fields using such markers:: Requires-Dist: pywin32 (>1.0); sys.platform == 'win32' Requires-Dist: foo (1,!=1.3); platform.machine == 'i386' Requires-Dist: bar; python_version == '2.4' or python_version == '2.5' Requires-External: libxslt; 'linux' in sys.platform The micro-language behind this is a simple subset of Python: it compares only strings, with the ``==`` and ``in`` operators (and their opposites), and with the ability to combine expressions. Parentheses are supported for grouping. The pseudo-grammar is :: EXPR [in|==|!=|not in] EXPR [or|and] ... where ``EXPR`` belongs to any of these: - python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1]) - python_full_version = sys.version.split()[0] - os.name = os.name - sys.platform = sys.platform - platform.version = platform.version() - platform.machine = platform.machine() - platform.python_implementation = platform.python_implementation() - a free string, like ``'2.4'``, or ``'win32'`` - extra = (name of requested feature) or None Notice that ``in`` is restricted to strings, meaning that it is not possible to use other sequences like tuples or lists on the right side. The fields that benefit from this marker are: - ``Requires-Python`` - ``Requires-External`` - ``Requires-Dist`` - ``Setup-Requires-Dist`` - ``Provides-Dist`` - ``Classifier`` Optional features ================= Distributions may use the ``Provides-Extra`` field to declare additional features that they provide. Environment markers may then be used to indicate that particular dependencies (as specified in ``Requires-Dist`` or ``Setup-Requires-Dist``) are needed only when a particular optional feature has been requested. Other distributions then require an optional feature by placing it inside square brackets after the distribution name when declaring the dependency. Multiple features can be requisted by separating them with a comma within the brackets. The full set of dependency requirements is then the union of the sets created by first evaluating the `Requires-Dist` (or `Setup-Requires-Dist`) fields with `extra` set to `None` and then to the name of each requested feature. Example:: Requires-Dist: beaglevote[pdf] -> requires beaglevote, reportlab at run time Setup-Requires-Dist: beaglevote[test, doc] -> requires beaglevote, sphinx, nose at setup time It is legal to specify `Provides-Extra` without referencing it in any `Requires-Dist`. It is an error to request a feature name that has not been declared with `Provides-Extra`. The following feature names are implicitly defined for all distributions: - `test`: dependencies that are needed in order to run automated tests - `doc`: dependencies that are needed in order to generate documentation Listing these implicit features explicitly in a ``Provides-Extra`` field is permitted, but not required. Updating the metadata specification =================================== The metadata specification may be updated with clarifications without requiring a new PEP or a change to the metadata version. Adding new features (other than through the extension mechanism), or changing the meaning of existing fields, requires a new metadata version defined in a new PEP. Summary of differences from \PEP 345 ==================================== * Metadata-Version is now 1.3 * Most fields are now optional * Explicit permission for in-place clarifications without releasing a new version of the specification * General reformatting of the PEP to make it easier to read * Values are now expected to be UTF-8 * Changed the version scheme (eliminating the dependency on PEP 386) * Changed interpretation of version specifiers * Explicit handling of ordering and dependencies across metadata versions * Support for packaging, build and installation dependencies * the new ``Setup-Requires-Dist`` field * Optional feature mechanism * the new ``Provides-Extra`` field * ``extra`` expression defined for environment markers. * optional feature support in ``Requires-Dist`` and ``Setup-Requires-Dist`` * Metadata extension mechanism * the new ``Extension`` field and extension specific fields * Updated obsolescence mechanism * the new ``Obsoleted-By`` field * the ``Obsoletes-Dist`` field has been removed * Simpler description format * the ``Description`` field is now deprecated * A payload (containing the description) may appear after the headers. * Other changed fields: - ``Requires-Python`` (explicitly flagged as multiple use) - ``Project-URL`` (commas permitted in labels) * Clarified fields: - ``Provides-Dist`` - ``Keywords`` The rationale for major changes is given in the following sections. Standard encoding and other format clarifications ------------------------------------------------- Several aspects of the file format, including the expected file encoding, were underspecified in previous versions of the metadata standard. To simplify the process of developing interoperable tools, these details are now explicitly specified. Changing the version scheme --------------------------- The key change in the version scheme in this PEP relative to that in PEP 386 is to sort top level developmental releases like ``X.Y.devN`` ahead of alpha releases like ``X.Ya1``. This is a far more logical sort order, as projects already using both development releases and alphas/betas/release candidates do not want their developmental releases sorted in between their release candidates and their full releases. There is no rationale for using ``dev`` releases in that position rather than merely creating additional release candidates. The updated sort order also means the sorting of ``dev`` versions is now consistent between the metadata standard and the pre-existing behaviour of ``pkg_resources`` (and hence the behaviour of current installation tools). Making this change should make it easier for affected existing projects to migrate to the latest version of the metadata standard. Furthermore, as the version scheme in use is dependent on the metadata version, it was deemed simpler to merge the scheme definition directly into this PEP rather than continuing to maintain it as a separate PEP. This will also allow all of the distutils-specific elements of PEP 386 to finally be formally rejected. Changing the interpretation of version specifiers ------------------------------------------------- The previous interpretation of version specifiers made it very easy to accidentally download a pre-release version of a dependency. This in turn made it difficult for developers to publish pre-release versions of software to the Python Package Index, as such an action would lead to users inadvertently downloaded pre-release software. The previous interpretation also excluded post-releases from some version specifiers for no adequately justified reason. The updated interpretation is intended to make it difficult to accidentally accept a pre-release version as satisfying a dependency, while allowing pre-release versions to be explicitly requested when needed. Packaging, build and installation dependencies ---------------------------------------------- The new ``Setup-Requires-Dist`` field allows a distribution to indicate when a dependency is needed to package, build or install the distribution, rather than being needed to run the software after installation. This should allow distribution tools to effectively support a wider range of distribution requirements. Support for optional features of distributions ---------------------------------------------- The new ``Provides-Extra`` field allows distributions to declare optional features, and to use environment markers to reduce their dependencies when those features are not requested. Environment markers may also be used to require a later version of Python when particular features are requested. The ``Requires-Dist`` and ``Setup-Requires-Dist`` fields then allow distributions to require optional features of other distributions. The ``test`` and ``doc`` features are implicitly defined for all distributions, as one key motivation for this feature is to encourage distributions to explicitly declare the dependencies needed to run their automatic tests, or build their documentation, without demanding those dependencies be present in order to merely install or use the software. Support for metadata extensions ------------------------------- The new ``Extension`` field effectively allows sections of the metadata namespace to be delegated to other distributions, while preserving a standard overal format metadata format for easy of processing by distribution tools that do not support a particular extension. It also works well in combination with the new ``Setup-Requires-Dist`` field to allow a distribution to depend on tools which *do* know how to handle the chosen extension, and the new optional features mechanism, allowing support for particular extensions to be provided as optional features. Updated obsolescence mechanism ------------------------------ The marker to indicate when a project is obsolete and should be replaced has been moved to the obsolete project (the new ``Obsoleted-By`` field), replacing the previous marker on the replacement project (the removed ``Obsoletes-Dist`` field). This should allow distribution tools to more easily warn users of obsolete projects and their suggested replacements. The ``Obsoletes-Dist`` header is removed rather than deprecated as it is not widely supported, and so removing it does not present any significant barrier to tools and projects adopting the new metadata format. Simpler description format -------------------------- Distribution descriptions are often quite long, sometimes including a short guide to using the module. Moving them into the file payload allows them to be formatted neatly as reStructuredText without needing to carefully avoid the introduction of a blank line that would terminate the header section. The ``Description`` header is deprecated rather than removed to support easier conversion of existing tools and projects to the new metadata format. References ========== This document specifies version 1.3 of the metadata format. Version 1.0 is specified in PEP 241. Version 1.1 is specified in PEP 314. Version 1.2 is specified in PEP 345. The initial attempt at a standardised version scheme, along with the justifications for needing such a standard can be found in PEP 386. .. [1] reStructuredText markup: http://docutils.sourceforge.net/ .. _`Python Package Index`: http://pypi.python.org/pypi/ .. [2] PEP 301: http://www.python.org/dev/peps/pep-0301/ Appendix ======== Parsing and generating the Metadata 1.3 serialization format using Python 3.3:: # Metadata 1.3 demo from email.generator import Generator from email import header from email.parser import Parser from email.policy import Compat32 from email.utils import _has_surrogates class MetadataPolicy(Compat32): max_line_length = 0 continuation_whitespace = '\t' def _sanitize_header(self, name, value): if not isinstance(value, str): return value if _has_surrogates(value): raise NotImplementedError() else: return value def _fold(self, name, value, sanitize): body = ((self.linesep+self.continuation_whitespace) .join(value.splitlines())) return ''.join((name, ': ', body, self.linesep)) if __name__ == "__main__": import sys import textwrap pkg_info = """\ Metadata-Version: 1.3 Name: package Version: 0.1.0 Summary: A package. Description: Description =========== A description of the package. """ m = Parser(policy=MetadataPolicy()).parsestr(pkg_info) m['License'] = 'GPL' description = m['Description'] description_lines = description.splitlines() m.set_payload(description_lines[0] + '\n' + textwrap.dedent('\n'.join(description_lines[1:])) + '\n') del m['Description'] # Correct if sys.stdout.encoding == 'UTF-8': Generator(sys.stdout, maxheaderlen=0).flatten(m) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

4 3