On Friday, June 22, 2012 at 1:05 AM, Nick Coghlan wrote:
- I reject setup.cfg, as I believe ini-style configuration files are not appropriate for a metadata format that needs to include file listings and code fragments
- I reject bento.info (http://bento.info), as I think if we accept yet-another-custom-configuration-file-format into the standard library instead of just using YAML, we're even crazier than is already apparent
- I shall use "dist.yaml" as my proposed name for my "I wish I could define packages like this" format (and yes, that means adding yaml support to the standard library is part of the wish)
- many of the details below will be flawed, but I want to give a clear idea for how a concept like this might work in practice
- we need to define a clear set of build phases, and then design the dist metadata format accordingly. For example: - source - uses a "source" section in dist.yaml - "source/install" maps source files directly to desired install locations - essentially what the setup.cfg Resources section tries to do - used for pure Python code, documentation, etc - See below for example - "source/files" defines a list of extra files to be included - "source/exclude" defines the list of files to be excluded - "source/run" defines a Python fragment to be executed - serves a similar purpose to the "files" section in setup.cfg - creates a temporary directory (and sets it as the working directory) - dist.yaml is copied to the temporary directory - all files to be installed are copied to the temporary directory - all extra files are copied to the temporary directory - the Python fragment in "source/run" is executed (which can thus easily add more files) - if sdist archive creation is requested, entire contents of temporary directory are included - build - uses a "build" section in dist.yaml - "build/install" maps built files to desired install locations - like source/install, but for build artifacts - compiled C extensions, .pyc and .pyo files, etc would all go here - "build/run" defines a Python fragment to be executed - "build/files" defines the list of files to be included - "build/exclude" defines the list of files to be excluded - "build/requires" defines extra dependencies not needed at runtime - starting environment is a source directory that is either: - preexisting (e.g. to allow building in-place in the source tree) - created by running source first - created by unpacking an sdist archive - the Python fragment in "build/run" is executed to trigger the build - if the build succeeds (i.e. doesn't throw an exception) - create a temporary directory - copy dist.yaml - copy all specified files - this is the easiest way to exclude build artifacts from the distribution, while still keeping them around to enable incremental builds - if bdist_simple archive creation is requested, entire contents of temporary directory are included - other bdist formats (such as bdist_rpm) will have their own rules for getting from the bdist_simple format to the platform specific format - install - uses an "install" section in dist.yaml - "install/pre" defines a Python fragment to be executed before copying files - "install/post" defines a Python fragment to be executed after copying files - starting environment is a bdist_simple directory that is either: - preexisting (e.g. to allow creation by system packaging tools) - created by running build first - created by unpacking a bdist_simple archive - end result is a fully installed and usable piece of software - test - uses a "test" section in dist.yaml - "test/run" defines a Python fragment to be executed to start the tests - "test/requires" defines extra dependencies needed to run the test suite
I dislike some of the (implementation) details, but in general I think this is a good direction to go in. Less trying to force tools to work together by hijacking setup.py or something and more "this is a package, it contains the data you need to install, and how to install it, you installation tool can use this data however it pleases to make sure it is installed." I feel like this is (one of?) the missing piece of the puzzle to define a set of standards that _any_ package creation, or installation tool can implement and gain interoperability. I don't want to argue over implementation details as I think that is premature right now, so this concept has a big +1 from me. RPM, deb, etc has a long history and a lot of shared knowledge so looking at them and adapting it to work cross platform is likely to be huge win.