[Distutils] Random bits

[Michael Hudson]

On Mon, 29 Nov 1999, Greg Ward wrote:

> On 28 November 1999, Michael Hudson said:
> > Yes! I spent a frustrating few minutes trying to work out why files I was
> > trying to exclude from the distribution were turning up in the tarball...
> 
> Oops, sorry.

That's OK :-)

> > Also, excluding files from the dist in general is a pain. 
> > 
> > I have bytecodehacks in CVS; I don't want to distribute the CVS folders. I
> > found no way to do this except building the MANIFEST file using a shell
> > script (which kind of defeats the point of the manifest, I'd have
> > thought).
> > 
> > Also excluding a single file doesn't work.
> [...more ranting about what doesn't work in the Distutils MANIFEST file...]
> 
> Yeah, I've been bitten by a couple of these things myself; others, I
> haven't seen.  As I said in my last post, this was a cobbled-together
> hack and not-at-all well thought-out.  Guess it's time to fix that.

It would be good if the distutils could be used to distribute stuff...

[snip two phase plan]

> Anyways, I'm open to ideas on how best to handle the
> MANIFEST.in/MANIFEST thing (and syntax for the former).  Tell me what
> features you want, whether you even like the idea of going from a
> short, simple list of exclude/include patterns to an explicit list of
> every file, how you would like to use such a system, etc.

Right, here's an idea. I'm not sure it's the best way to go, or even a
reasonble one, but I'm just going to braindump for a while.

At the base, I think you want to be able to supply a Python function that
gets passed a path to a file and returns 1 if it should be included or 0
if not.

But better would be a class (more Pythonic) with an "accept" method, e.g.

class PermissiveAcceptor(Acceptor): # I'll explain Acceptor later
    def accept(self,file):
        return 1

then you need a way of combining Acceptors so that you can combine:

class BackupRejector(Acceptor):
    def accept(self,file):
        return file[-1] <> '~'

class PycRejector(Acceptor):
    def accept(self,file):
        if len(file) > 4:
           return file[-4:] <> ".pyc"

Now my idea of how to do this is to define methods in the Acceptor class

class Acceptor:
    def __and__(self,rhs):
        return AndingAcceptor(self,rhs)
    def __or__(self,rhs):
        return OringAcceptor(self,rhs)
    def __neg__(self):
        return NotAcceptor(self)

class AndingAcceptor(Acceptor):
    def __init__(self,one,two):
        self.one = one
        self.two = two
    def accept(self,file):
        return self.one.accept(file) and self.two.accept(file)

class OringAcceptor(Acceptor):
    def __init__(self,one,two):
        self.one = one
        self.two = two
    def accept(self,file):
        return self.one.accept(file) or self.two.accept(file)

then:

myacceptor = PycRejector() & BackupRejector()

[ Random aside #1: For stateless acceptors like BackupRejector it might be
  better to do

class _BackupRejector(Acceptor):
    def accept(self,file):
        return file[-1] <> '~'

BackupRejector = _BackupRejector() ]

[ Random aside #2: guess who's being doing some functional programming of
  late? ]

Then setup would take another argument like "file_chooser" that would be
called with every file in the heirachy.

[ Random aside #3: hmm... maybe should have accept_file, accept_dir ]

This would not be a particularly efficient solution, but that shouldn't
really be a problem, should it?

Then one has to provide a set of classes built in to distutils so that in
the common cases, one doesn't have to write lots of code.

Obvious candiate is MANIFESTAcceptor...

A somewhat heavier weight idea of mine is too have files in each directory
that are like MANIFESTs, but just for that directory. A bit like
.cvsignore ... so you would put in a file called .distutils-manifest lines
like:

*.py ! hacked_up.py
extra_support_file

I would personally find that preferable to one top level file. What would
be even nicer in some ways would be the ability to have rules that
affected deeper level directories sort-of Acquisition style (if that makes
sense to anyone - I know what I mean, btu I can't seem to express it
properly and I've rabbited on for long enough now...)

> (My main concern is that we *not* just have a simple MANIFEST file
> that the developer has to create and maintain; that's how Perl's
> MakeMaker expects you to work, and it's a PITA.  It provides a module
> to help you generate the MANIFEST the first time, but after that
> you're pretty much on your own.)

This is a fate to be avoided.

Thank you for your patience. Comments appreciated (I can think of a few
problems already...)

Michael