[Distutils] Alternate static metadata PEP submission...

David Lyon david.lyon at preisshare.net
Tue Oct 13 08:18:11 CEST 2009

On Mon, 12 Oct 2009 00:17:10 +0200, Tarek Ziadé <ziade.tarek at gmail.com>
> Hey
> this is the PEP for setup.cfg, as requested :
> http://www.python.org/dev/peps/pep-0390
> Please comment,
> Tarek

Can I have assistance submitting an alternate PEP?

Generally speaking you never answer any email from me but in
this case I ask that you assist in respect to having a fair and 
due process. 

I have subscribed to peps at python.org but havent received an
email back. I have posted it to peps at python.org.

Here is my PEP...

PEP: 0
Title: An improved python application/package setup system using setup.info
Version: 1.0
Last-Modified: 13-10-2009
Author: David Lyon
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 13-Oct-2009
Discussions-To: david.lyon at preisshare.net


This document describes an improved way for application
and package developers to specify information for installation
to different systems compared to traditional method that uses 


This PEP proposes a change to the way that applications
and libraries are installed in python, by using a new
file called setup.info rather than setup.py. To hold 
installation information.

For example:

name = artistflair
version = 1.2
description = artistflair is a package written in
    spare time to show colour variations.

packages = pyopengl
    objectrelationalmapper >= 1.6

[dependencies python<=2.4]
packages = lxml >= 2.5

[dependencies windows]
packages = win32com

[dependencies linux]
packages = pyodbc

scripts = artisticflairgui.py

files = artisticflair.conf

files = artisticdb.db
directory = db

directory = docs
url = index.html

script = checksystem.py

[postinstall linux]
script = builddocumentation.py

script = uninstallcleanup.py


Traditionally, python has used a setup.py to
perform package installation.

Metadata has been hardcoded into setup.py
which is then passed to distutils and the
installation is performed.

Time has revealed limitations with the setup.py
system and this pep proposes a more flexible and
powerful system for installation using setup
information stored as parameters within a 
setup.info file.

The advantages of this new approach are:

 - a clean way to handle variations due to different
   operating systems.

 - a clean way to specify package dependencies
   (including conditional dependencies based
    on operating system and python interpretor

 - a way to handle pre and post installation

 - a way to handle package/application uninstallation

A boilerplate setup.py will be provided to accompany 
setup.info file on platforms where distutils isn't
patched. This would provide backward compatability
with previous python versions.

The setup.info file will be based on the the standard 
python ConfigParser file format format.

Adantages for using setup.info

 1) It's easier to extend past the simplest use case

 2) It's more developer friendly.

 3) Provides an easier way to specify package dependencies

 4) Provides a portable setup system across platforms

 5) Easier to integrate within the development build
    process where handling version numbers can often
    be a challenge.

Section Name Construction

The setup.info file is built as a standard configparser
format file.

Section names are built from a combination of a specifier
then an option operating system name and finally an
optional python version specifier.

The following shows a very powerful way to build conditional
installation for different operating systems and platforms
that starts from very simple and can be extended with effort
into the very complex.

Most developers will choose the simplest option.

Sections are built in the following format

 specifier [operating_system] [pythonversion]

A simple real world example might be:

packages = objectmapper >= 2.5

[dependencies windows]
packages = win32com

[dependencies linux]
packages = pyodbc

What this means is that for all platforms, package objectmapper
>= version 2.5 is required. However on windows, there's an 
additional depency that win32com is required. On Linux or Mac,
win32com won't be required.

Then, on linux the package pyodbc is required, but not on
the mac or windows.

Therefore the above list gives us the following dependency

platform       windows linux   mac
objectmapper     yes    yes    yes
win32com         yes    no     no
pyodbc           no     yes    no

Thus we end up with a way of conditionally being able to
specify sections that reflect the differences in underlying

The specifier isn't limited to just the dependency section
and can be applied to provide conditional installation
on any of the following sections:

 dependencies preinstall postinstall scripts
 application documentation uninstall datafiles.

Operating system is one of the following values
but also may contain a version number which is
used to denote the operating system:
 mac ('9.5','10.0')

This would provide ways to handle challenges that
arrise for particular operating system versions.

[postinstall windows=2000]

preinstall linux-redhat<=6]

PythonVersion is one of the following

 python ('2.3','2.4','2.5','2.6','2.7','3.0,'3.1')

So an example might be:

[dependencies python<=2.4]
packages = lxml

Package Metadata

Traditionally, package metadata has been stored within
the setup.py program file as hardcoded variables.

#!/usr/bin/env python

from distutils.core import setup

      description='Demonstration of color usage',
      author='David Lyon',
      author_email='dlyon at noemail.pls',

This PEP proposes moving the metadata from setup.py
into setup.info to the section known as setup.

name = Artisticflair
version = 1.0
description = Demonstration of color usage
author = David Lyon
author_email = dlyon at noemail.pls
url = http://www.python.org

Readers may question why this information is here
in addition to being in the PKG-INFO file.

The reason is that it is intended that the PKG-INFO 
file could be created with the metadata from this section
by reading the [setup] section and writing the
values sequentially.

In a development environment, reading a version number
and updating it when the file is in configparser format
is much easier than manually editing the setup.py file.

Therefore, many tasks relating to the production of
documentation, help files and the like will benefit
greatly from the introduction of a setup.info file.

The existance of a correspending PKG-INFO file will
be complement the setup.info file. It's expected
that the PKG-INFO file will be created from the
metadata information held inside the setup.info file.

This will preserve backwards compatability. 

Dependency Handling [dependencies]

To handle package/application dependencies, it is proposed
that there be a section within setup.info called 
'dependencies' with a multiline list 'packages'. ie:

packages = numpy >= 1.1
      biolife > 1.5

Each package consists of a single line entry, with an
optional operator ("==",">=",">") and a version string.

There are many real world examples where dependencies
changed based on the destination platform. So the
section name can be changed so that additional
dependencies can be added for specific platforms.

[depencies linux]
packages = pyodbc

[dependencies windows python<=2.4]
packages = lxml >= 2.5

Pre Installation Processing [preinstall]

It's often neccessary when doing an install to
run a script to probe the system to determine if
an installation is at all possible.

Here's the simplest use case, specifying a script
that runs to check the system:

script = checksystem.py

In the following case, different scripts are run
depending on the operating system:

script = checksystem.py

[preinstall linux]
script = checkmylinux.sh

[preinstall mac-os/x]
script = checkmymac.py

[preinstall windows]
script = checkmypc.bat

When calling the script, a command line parameter
is supplied. That parameter is destination directory
to which it's calculated that installation will
go to.

For example:

   checkmysystem.py /usr/local/python2.5/lib/site-packages/artistflair-1.2

or on windows:

   checkmysystem.py "c:\python2.5\lib\site-packages\artistflair-1.2"

If scripts return a 0 return code, installation
will proceeed. If a non-zero return code is
encountered, installation won't proceed and the
user will be notified.

Post Install Processing [postinstall]

After install, that is, after the program/package directories are
created and all the files are copied, it's often neccessary to go
off and do initialisation of some sort.

Typically, it's likely to be something such as:

 - generate/compile documentation (man pages or sphinx files)

 - migrate a users old database

 - remove old versions of an application

 - register files with the operating system

To accomplish this, postinstall processing support is required.

script = builddocumentation.py

[postinstall linux]
script = buildmanpages.py

[postinstall mac]
script = makeitlookniceforme.py

Application Scripts [Application]

Traditionally, setup.py hasn't provided ready support for the
installation of applications.

This PEP proposes that the task of registering python applications
becomes much simpler with the addition of an application section
specifically oriented to assist developers get their applications
registered on the user desktop.

In simple terms, what this section is about is registering the
python scripts with the window manager. On Linux - that might
be KDE or Gnome, on Windows - the program manager, and on the
Mac - the desktop.

scripts = artisticflairgui.py

The above example would cause the application to be installed
in an "artistflair" folder (derived from [setup]/name) created
on the window manager.

Configuration files [Configuration]

Most real world applications carry with them a configuration
file of some sort. It is certainly not mandatory that this
section exists but if it does it will provide a way to
carry configuration files to their appropriate end

Where configuration files are actually stored is different
between operating platforms. On Linux and Mac, files go
to a user directory, or possibly /etc. On windows they
go to \Documents and Settings\user\application data\myapp.

files = artisticflair.conf

By specifying the configuration file, the system will
ensure that the configuration file goes to the correct

Data Files [Datafiles]

In any application, data files may be an install option. 

This section specifies what the datafiles are and the
name of the subdirectory that they will be placed in.

files = artisticdb.dbf
directory = db


Most projects carry documentation of some sort.

More importantly, a developer wants to make it very
easy for the user to be able to access that documentation.

By specifying the documentation files in the documentation
section, shortcuts to the documentation can then be created
in the window manager to enable the user to easily access

Traditionally, documentation has been built offline and
kept in with the package. With the Internet, there's been
a move to online documentation. The mechanism described
here handles both very simply.

directory = docs
url = index.html
readme = readme.txt

In the above snippet, all documentation is locally stored 
and has been placed in a directory called 'docs'. At install
time, that entire directory and subdirectores will be
copied to the local system.

Within the artisticflair/docs subdirectory, the main
documentation file is index.html. A readme file is also
provided. Both these will have a shortcut created for
them within the local window manager.

For online documentation, simply point the documentation
url to an online link.

url = www.python.org/docs

The last example shows how to do documentation for different
platforms. For example windows users may be disoriented
when given screen shots to mac software, and vice-versa.

There is a difference in logic with operating system and
python version handling in the section names for 'documentation'.

The difference is that documentation is more specific and
there is usually only one copy of documentation.

For example, if we are on the mac, we only want the mac
documentation shown. We don't want the linux documentation
to be available (unlesss we decide to do that).

Therefore, the documentation section mechanism allows
us to clearly seperate different documentation for
different platforms. 

url = index.html

[Documentation mac]
url = www.nourl.at/artisticflairmac.html

[Documentation linux]
url = www.nourl.at/artisticflairlin.html

[Documentation linux-kde]
url = www.nourl.at/artisticflairlinkde.html

[Documentation windows]
url = www.nourl.at/artisticflairwin.html

[Documentation windows-vista]
url = www.nourl.at/artisticflairwinvista.html

The final accomodation is the language prefix. For
example to specify multilingual documentation:

url = index.html
url-es = index_es.html
url-fr = index_fr.html
url-de = index_de.html


This section is to provide a script to do a final
cleanup before uninstall takes place.

script = dropmytables.py

It's expected that setup.py will perform the actual

Processing Sequence

This proposal suggests that setup.py be enhanced to allow
reading of the metadata from setup.info as package/application
install time.

A boilerplate setup.py will be made available to read data
from the setup.info file, before passing control to distutils
to perform the package installation.

Therefore, the python tradition of 'python setup.py install' can
be preserved.


All the people on the distutils mailing list who asked for


This document has been placed in the public domain.

More information about the Distutils-SIG mailing list