[Distutils] Manifests

[Paul Prescod]

I love distutils. It is a key piece of Python infrastructure. But I'm 
not happy with one of its design decisions.

Consider the case of Joe Blow. He creates the simplest possible 
distribution structure:

from distutils.core import setup

setup(
     name = "pyximport",
     version = "1.0",
     url = "http://www.example.com",
     author = "Joe Blow",
     author_email = "joe at example.com",
     py_modules = ["doit"])

The distutils docs[1] say:

"If you don't supply an explicit list of files (or instructions on how 
to generate one), the sdist command puts a minimal default set into the 
source distribution."

The default set described is just what Joe needs. He is happy. He tries 
to create the distribution:

bash-2.05a$ python setup.py sdist
running site customization
running sdist
warning: sdist: manifest template 'MANIFEST.in' does not exist (using 
default file list)
warning: sdist: standard file not found: should have one of README, 
README.txt
writing manifest file 'MANIFEST'
...


Oops. Forgot the README. So Joe adds the README. He also remembers that 
there is a nother module: "dothat". So he updates py_modules in 
setup.py. He regenerates and gets no errors. He uploads to his website. 
The problem is: his users WILL NOT get README or dothat.py. Distutils 
will not update the MANIFEST file. Joe probably didn't even notice that 
a MANIFEST exists and certainly won't care about it until it bites him 
in the ass. Even if he read the documentation closely, he would get a 
wrong impression:

"if either MANIFEST.in or the setup script ( setup.py )   are more 
recent than MANIFEST , recreate MANIFEST by reading MANIFEST.in"

That's not quite true because of the special case described in 
Distutil's source code:

     # The contents of the manifest file almost certainly depend on the
     # setup script as well as the manifest template -- so if the setup
     # script is newer than the manifest, we'll regenerate the manifest
     # from the template.  (Well, not quite: if we already have a
     # manifest, but there's no template -- which will happen if the
     # developer elects to generate a manifest some other way -- then we
     # can't regenerate the manifest, so we don't.)

I propose that if distutils is going to generate a MANIFEST then it 
should generate it to its temporary cruft directory, not to the source 
directory. If there is some reason that a user would want to hand-edit 
the output of this process and use it from then on as INPUT then they 
can copy it OUT of their distribution to the top-level directory where 
distutils will treat it from then on as INPUT. Their explicit copying 
action will indicate that they want to take control of its maintenance 
henceforth.

I really don't know why anyone would want to hand-maintain (or use a 
program to generate) a MANIFEST file when they could just as easily 
hand-maintain or code-generate the much more flexible manifest template 
file.

  ===

(mostly unrelated thought:) I am not clear why distutils separates the 
concept of manifest template from setup.py. Couldn't you express your 
complete set of dependencies using globs etc. in the Python code? The 
whole manifest template seems like just one more file syntax to learn 
and one more file to maintain.

  Paul Prescod