[Distutils] Making pip and PyPI work with conda packages

[Daniel Holth]

The conda package specification is published at
http://conda.pydata.org/docs/spec.html

The file format is nice and simple. "A conda package is a bzipped tar
archive (.tar.bz2) which contains metadata under the info/ directory,
and a collection of files which are installed directly into an install
prefix. The format is identical across platforms and operating
systems. It is important to note that during the install process, all
files are basically just extracted into the install prefix, with the
exception of the ones in info/."

(Compare to the Debian package format's embedded metadata and content archives.)

It has concise metadata in info/index.json

{
  "arch": "x86_64",
  "build": "py27_138_g4f40f08",
  "build_number": 138,
  "depends": [
    "jinja2",
    "jsonpointer",
    "jsonschema",
    "mistune",
    "pandoc",
    "pygments",
    "python 2.7*",
    "pyzmq",
    "terminado",
    "tornado"
  ],
  "license": "MIT License",
  "name": "ipython-we",
  "platform": "linux",
  "subdir": "linux-64",
  "version": "3.1.0"
}

The package includes its build recipe in info/recipe

This particular package has setuptools metadata in
lib/python2.7/site-packages/ipython-3.1.0-py2.7.egg-info

On the index, conda packages are organized by placing packages for a
platform+architecture in their own (sub)directory, not by putting all
that information in the filename. According to the docs it doesn't
interpret the platform metadata.

When conda installs a package, it gets unpacked into a common
directory and then linked into each environment, so that it can be
installed to lots of environments without taking up much extra disk
space. Packages can have link and unlink scripts to provide custom
behavior (perhaps fixing up some paths for files that can't just be
linked) when this happens.

It occurs to me that the setuptools packaging in general is more like
a shared library format .so or .dll, aka libraries searched for along
a path, than an OS level package manager.