[Distutils] generating pyc and pyo files

[Andrew Dalke]

Greg Ward says:
> Compiling in the build area before installation means that you can
> install them with arbitrary privileges, owner, and group. That gets
> tricker using compileall.py.
> 
> For example, let's say that "root" is doing an installation and the
> target should be owned by bin:bin. Can't do that with compileall.py,
> AFAIK.

Sure, you cannot do that with compileall, but you can in the 
Makefile.  After all, this is pretty much identical to setting
the right permissions for compiled emacs files.

I'm not sure I see the problem.  The possibility I see is a
Makefile like (though there is no way to tell compileall to
compile a given list of files, it's just an example):

MODULE = SpamSkit
PYTHON_FILES = __init__.py spam.py eggs.py vikings.py

PYTHON_CLEAN = $(PYTHON_FILES:.py.pyc) $(PYTHON_FILES:.py.pyo)
PYTHON_INSTALL = $(PYTHON_FILES) $(PYTHON_CLEAN)
PYTHON_SITE = /usr/local/lib/python1.5/site-packages

INSTALL = /usr/bin/install -c
INSTALL_DATA = ${INSTALL} -m 644

all:
  $(compileall) -f $(PYTHON_FILES)

install:
  $(INSTALL_DATA) $(PYTHON_INSTALL) $(PYTHON_SITE)/$(MODULE)

clean:
  rm -f $(PYTHON_CLEAN)


so the .pyc and .pyo files would be installed with the same
permissions as the .py files.  And if you wanted to be more
specific, you could tweak install (or the install-hook for
automake) or INSTALL_DATA as needed.


Ummm, after rereading your message I realized I don't understand
it as well as I thought I did.  It looks like you're suggesting
to do the byte compiles locally before the install (which I'm now
leaning towards) because you can modify the permissions better
that way.

If so, I disagree with that.  With a standard unix user account I
cannot modify all the permissions bits (like setuid) or the
owner/group.  I need to be root for that, and normally that only
happens before the "make install".

The problem there is that some (many?) places don't NFS export
with root write privs to their clients.  My home directory is
not writeable by root except on the file server.  So if I am root
and root tries to modify the permissions before the copy, it will
fail, while if it copies first then modifies the permissions, it
will work.

So the options I see are:
1) copy .py files to the install directory
   run compileall on that directory (or the equivalent)
   change permissions on the files in the install directory as needed

2) run compileall in the build directory
   copy the .py{,c,o} files to the install directory
   change permissions on the files in the install directory as needed

and #2 is my lean-to, as it were.

It seems that having a "compileall" which takes a list of files
rather than directories will be useful.  Otherwise for option (1)
installing a single file (not module) to
   /usr/local/lib/python1.5/site-packages (or site-python)
and then doing compileall on that directory may cause all modules to
be (re)compiled.

For option (2) you will run into problems with test/development .py
files in the developer's build directory which aren't valid python
files and hence will cause compileall to file.  This isn't a direct
problem for distutils since we can assume that all .py files will
be installed, but it is important to bear in mind.

Oh!  Plus, what's the policy for installing python files in some
place like /usr/local/bin?  Must all such scripts be compiled during
the distutils install process?  If not, then compileall on
/usr/local/bin could cause problems.

How about added an option, like "-f", to compileall to support
a list of files which should be compiled?

						Andrew
						dalke@bioreason.com