[Distutils] Pre-pre-PEP: Requirements for the Python BUILDS Specification

[Phillip J. Eby]

At 08:58 PM 10/7/2008 +0200, Josselin Mouette wrote:
>Le mardi 07 octobre 2008 à 14:40 -0400, Phillip J. Eby a écrit :
> > At 09:57 AM 10/7/2008 +0200, Josselin Mouette wrote:
> > >Symlinks are a real pain to handle. We can use them transparently
> > >for .pyc files, but if we want to relocate data files to some other
> > >directories, currently it has to be done by hand, and this is why most
> > >maintainers don̢۪t do it.
>.
> >
> > Which is why the idea for the BUILDS spec to include a way for
> > automated tools to do it, so that you won't have to do it manually.
>
>If you write a tool to do that, why not make it simply move files
>properly and let the code locate them, instead of adding yet another
>layer on top of the existing stuff? The tool will not be more
>complicated this way.

I'm not sure I follow you.  What I'm proposing is that:

1. A BUILDS "manifest" for a project should list the files included 
in the project by what "kind" of file they are, as well as by 
inter-file relationships (e.g., "these 3 files are part of Python 
package 'foo.bar' within this project, and they are static data used 
at runtime but not read or written").

2. BUILDS-compatible install tools will be free to install those 
files wherever they like (or create .debs, .rpms, etc. that do), and 
use symlinks so that the installed project can find them in the 
places that the developer wanted to find them.

Is this what you're asking for?  Because it's what I've been trying 
to say we should do.


> > Or are you saying that that isn't an improvement over the current 
> situation?
>
>It is. I didn’t understand you wanted to automate the symlinks creation.
>It indeed means less burden on the maintainers, but it would be a shame
>to keep the same mess in binary packages, while having a tool to do that
>would allow to make things cleaner.

I'm not sure I understand you here, either.  I'm saying that a 
BUILDS-compatible install tools for Debian and RPM systems should 
create system packages with the stuff wherever the OS people want it 
to go, *and* symlinks so the Python code can find it.  And that this 
should be an automated process, to the extent possible/practical.

For example, if Debian needs metadata that's not in the core BUILDS 
metadata, it should be possible within BUILDS to include it, so that 
it can be contributed upstream.  Likewise, it should be possible for 
a BUILDS install tool to use a third-party supplement to get that 
data, so that when you're initially porting a project, you could 
write a file with the additional data while doing the initial port, 
then submit that file upstream.


> > Yes, it's true that I'm saying that developers should not be
> > *required* to add the extra data to their packages, but that they
> > should be *able* to, and if it is trivial to add the extra data, most
> > should accept patches or respond to requests to do so.
> >
> > Right now, it's not even *possible* for them to do so, however.
>
>It is possible - if they use autoconf instead.

Sure, but who's going to convince them to do *that*?  ;-)

Less flippantly, our goal here is to have something that has a better 
cost/benefit ratio for adoption by a Python developer who has not 
seen any pressing need for autoconf in their life to date.  And if 
somebody wants to make a BUILDS tool to generate autoconf sources 
from a project manifest, then great!


>BTW, I would consider it a good approach to try bringing BUILDS on par
>with the autotools capabilities. These tools have serious drawbacks, but
>they were written with the ability to work for distributors in mind.

Sure.  I like stealing designs and requirements definition from 
mature projects, even if you might argue that I chose poor role 
models for setuptools.  (Java and OSGi.)

However, since autotools compatibility isn't a goal in itself, I 
think I'll leave it to those people who have interest/experience to 
propose what specific autotools features they'd like to see in 
BUILDS, and the use cases they have for them.  If I were to just skim 
the docs trying to figure that out, I might think that something cool 
in it is actually useful, or vice versa.  ;-)  So, it's better here I 
think to hear from experienced users such as yourself.