[Distutils] MANIFEST destiny :)

[Phillip J. Eby]

At 10:29 AM 11/16/2005 -0600, Ian Bicking wrote:
>M.-A. Lemburg wrote:
>>>* Always, always, always build MANIFEST, and always include both the 
>>>MANIFEST file and MANIFEST.in (if present) in the source distribution.
>>
>>-1 on always building MANIFEST.
>>This would miss the point of managing MANIFEST files
>>independently of your package files, e.g. using
>>Makefiles or other tools dealing with file dependencies,
>>checkouts, etc.
>
>How about always build it, automatically, if some particular option is not 
>passed to setup()?  This would only apply to packages using setuptools, 
>not distutils, and there's other things that existing distutils packages 
>might have to tweak to move to setuptools.

Especially in cases where they have complex custom builds.  People with 
stuff as sophisticated as SciPy, mx* stuff, Twisted, Zope, are not people I 
expect to move to setuptools any time soon.  (I've heard rumors about Zope 
using it for Products, but reasonably speaking I think it's going to take 
some time before the Zope core could be moved to it, probably by adapting 
the zpkg tools to generate setuptools-based setup scripts.)

My thinking here is more about the long term, and mostly setuptools is 
"disruptive technology" anyway, because it's shifting the balance of 
distribution power towards smaller and simpler tools.  One of the reasons 
complex packages are dominant today is because they reduce the number of 
things you have to install.  Setuptools makes the number of things you have 
to install a lot less relevant, so in the "new world", smaller packages 
will tend to dominate.  In order to effect that shift, it's got to become a 
lot easier for a complete distutils novice to turn out a package.


>   Maybe it would be as simple as a keyword argument that refers to a 
> routine to build the MANIFEST file; and if you build MANIFEST by hand, 
> then you just make the function a no-op.

Now *that's* an interesting idea.  I'll have to give that one some 
thought.  Thanks!


>Also, disable all command-line options that control when/if/how MANIFEST 
>is generated; these seem peculiar to me.  MANIFEST generation seems like 
>package metadata, not something particular to your build, and command-line 
>options seem like a bad way to control that.  It's a little fuzzy, though, 
>since command-line options and setup.cfg are equivalent, and setup.cfg 
>options often feel like package data.  So... I don't know what the middle 
>ground is there.  Either way, these aren't things that seem like they 
>should be regularly tweaked.  I don't know, I don't have any packages with 
>complex (or even simple) build processes, so I only know that the current 
>system is mysterious to me, and almost but doesn't always work without my 
>intervention.

Marc's explanation of these options is actually pretty reasonable as a 
projected use case.  If there are any *actual* field uses, I'll consider 
trying to support them.  Of course, if those use cases are ones where 
there's no reasonable likelihood of the project moving to setuptools, it 
doesn't make sense to support them in setuptools.  It would be more a 
matter of, if some o the relevant setuptools stuff makes it back into the 
distutils, it would need that additional degree of support.


>I think Phillip also mentioned using MANIFEST for package_data?  At least 
>insofar as the packages and the MANIFEST intersect, this seems like a good 
>idea.

Yeah, it would make your find_package_data() less necessary.  The reason I 
haven't put it in yet, is because I've been pondering the issue of how to 
integrate these notions of "the distribution contents".  While I can 
envision scenarios where you'd want certain data files to get built on the 
target user's machine, there's still the option of manually or 
programmatically specifying those files in package_data.  (That is, I 
intend to make include_package_data be supplemental to the existing 
package_data option, not a replacement for it, except in the sense that for 
simple cases you won't need package_data any more.)