[Distutils] How to customize...

[Andy Robinson]

Hello everybody,

I'm hoping for some guidance on the best way to extend
distutils and on whether it is suitable for problems
Reportlab currently face.  It seems very good at 
packing up bits of a python package hierarchy and recreating
the same hierarchy elsewhere, but I can't figure out
how to do some things.  (Well, I can find ways, but figure I
might be subverting the design or missing obvious things).
The manual chapter on extending distutils is, ahem, short.

I'm hoping these will be easy for you all!

(1) Cross-wiring the output
---------------------------
I want to create a little distribution for a
corporate customer which direspects my package hierarchy.  
Reportlab has a 'reportlab' package for open source code, 
and an 'rlextra' package for other stuff.  We also make a 
CVS tree for each customer. My CVS tree for the customer 
has this:

   hugeco/littleapp/littleapp.py
   hugeco/littleapp/setup.py   <---presumably, see below

I want to make a single directory which includes a bunch of
files from elsewhere in our tree.  I want to add in these
into what the customer gets:
   rlextra/utils/rc4.py
   rlextra/utils/gloop.py
   rlextra/utils/cgisupport.py

We have often done yucky stuff like checking in a copy
of the rlextra files under the customer directory, and
are fed up doing it. We can write code so we only have one
version in the right place, but the customer inports
the local onem and avoid duplications in CVS.

Ideally, running sdist would assemble the 4 files above and
create a combined output distribution which just dumped all
4 modules.  Thus they would get a tar.gz containing
   littleapp-0.5/littleapp.py
   littleapp-0.5/rc4.py
   littleapp-0.5/gloop.py
   littleapp-0.5/cgisupport.py

The manual descrivbes doing this for data files but
not for source.  Can you pull source from arbitrary
directories?

I KNOW that ideally I'd give them the rlextra tree or its
subset packagized on their path, but we get situations
where a self-contained app is really more desrirable;
we can't inflict changes in the rlextra tree on other
apps at the same time.

How do I do this?

(2) Where to put setup scripts
-----------------------------
This is a minor "best practice" detail.  The standard 
document presumes your setup.py is above your source
tree.  All my CVS projects live under C:\code.
If I want to have setup scripts for project1, project2,
project3, obviously I cannot have one setup.py above
all 3.  Do people normally maintain the setup script
within the directory they want to pack up, and do
a translation on the directories? or
have multiple setup_project1.py, setup_project2.py
etc above?

(3) Shipping closed source software:
-----------------------------------
Sometimes we only want to ship .pyc files.  We're also
looking at things like pyz archives, maybe with some
additions so they can only be opened with a license
key for paid-only software, but also just for the
convenience.  Is there anyone who has thought about 
extending distutils this way?  how should I go about
it?  writing a new command (zdist??)

(4) Verification on site
------------------------
With our own customer apps we include a 'signed source manifest'
with checksums of each file in the manifest, and a 'signed
target manifest' which does the same for pyc files and
other compiled/optimized objects.  Then at run time
we can have verification functions.  This helps because
sometimes the customer fails to accurately promote some
of a few hundred files onto a web cluster; we want to
get back something saying that either the distro is
perfect, or various files disagree with the manifest.

In fact, I envision a meta-api where you could ask
any app to verfy itself, run a test suite or so on.
setup.py is the obvious hook.

Is this a useful extension to distutils? Is there a 
good place to put such ideas in?  Or would it be
easier to roll our own?

(5) General pace of activity
----------------------------
I am new to this sig.  Is much happening?  Is distutils
basically seen as a solved problem or is a lot
underway?

Best Regards,

Andy Robinson
CEO/Chief Architect, Reportlab inc.