[Python-Dev] First draft of "sysconfig"

[Tarek Ziadé]

Hello,

A while ago I've proposed to refactor the APIs that provides access to
the installation paths and configuration variables at runtime into a
single module called "sysconfig", and make it easier for all
implementations to work with them.

I've started a branch and worked on it, and I'd like to ask here for
some feedback. And in particular from Jython and IronPython people
because they would probably need to work in that file for their
implementation and/or propose things to add. My understanding is that
we have people like Phillip (Jenvey), Michael F., Frank W. in this
list so they can comment directly and I don't need to cross-post this
mail elsewhere.

== Installation schemes ==

First, the module contains the installation schemes for each platform
CPython uses.
An install scheme is a mapping where the key is the "code" name for a
directory, and
the value the path of that directory, with some $variable that can be expanded.

Install schemes are stored in a private mapping, where the keys are
usually the value of os.name,
and the value, the mapping I've mentionned earlier.

So, for example, the paths for win32 are:

_INSTALL_SCHEMES = {
  ...
  'nt': {
        'stdlib': '$base/Lib',
        'platstdlib': '$base/Lib',
        'purelib': '$base/Lib/site-packages',
        'platlib': '$base/Lib/site-packages',
        'include': '$base/include',
        'platinclude': '$base/include',
        'scripts': '$base/Scripts',
        'data'   : '$base',
        },
  ...
}

where each key corresponds to a directory that contains some Python files:

- stdlib : root of the standard library
- platstdlib: root of platform-specific elements of the standard library
- purelib: the site-packages directory for pure python modules
- platlib: the site-packages directory for platform-specific modules
- include: the include dir
- platinclude: the include dir for platform-specific files
- scripts: the directory where scripts are added
- data: the directory where data file are added

All these directory are read and used by:
- distutils when a package is installed, so the install command can
dispatch files in the right place
- site.py, when Python is initialized

IOW, any part of the stdlib can use these paths to locate and work
with Python files.

The public APIs are:

* get_path_names() : returns a list of the path names ("stdlib",
"platstdlib", etc.)

* get_paths(scheme, vars)  : Returns a mapping containing an install scheme.
   - "scheme" is the name of the scheme, if not provided will get the
default scheme of the current platform
   - vars is an optonal mapping that can provide values for the
various $variables. Notice that they all have
      default values, for example $base == sys.prefix.

    for example: get_paths('nt')

* get_path(name, scheme, vars): Returns one path corresponding to the scheme.

  for example : get_paths('stdlib', 'nt')

Those API are generic, but maybe we could add specific APIs like:

* get_stdlib_path('nt')

These API are basically a refactoring of what already exist in
distutils/command/install.py

== Configuration variables ==

distutils.sysconfig currently provides some APIs to read values from
files like Makefile and pyconfig.h.

These API have been placed in the global sysconfig module:

* get_config_vars(): return a dictionary of all configuration
variables relevant for the current platform.

* get_config_var(name): Return the value of a single variable

* get_platform():  Return a string that identifies the current
platform. (this one is used by site.py for example)

* get_python_version() : return the short python version
(sys.version[:3]) -- this one could probably go away but is useful
because that's the marker used by Python in some paths.

== code, status, next steps ==

The code of the module can be viewed here, it's a revamp of distutils.sysconfig:

http://svn.python.org/view/*checkout*/python/branches/tarek_sysconfig/Lib/sysconfig.py?content-type=text%2Fplain

I've refactored distutils/ and site.py so they work with this new
module, and added deprecation warnings in distutils.sysconfig.

All tests pass in the branch, but note that the code is still using
the .h and Makefile files. This will probably be removed later in
favor of a static _sysconfig.py file generated when Python is built,
containing the variables sysconfig reads. I'll do this second step
after I get some feedback on the proposal.

Regards
Tarek

-- 
Tarek Ziadé | http://ziade.org