[Distutils] installing to a non-standard directory

[K. Richard Pixley]

On 20101119 17:51, P.J. Eby wrote:
> At 03:18 PM 11/19/2010 -0800, K. Richard Pixley wrote:
>> I'm trying to install to a non-standard directory.
>
> Why?  It makes a difference as to what the best way to do it is.
>
> For example, if you're planning to install a Python application or 
> create a restricted development environment, you might be better off 
> using a virtualenv.  Or, alternatively, you may want to configure your 
> ~/.pydistutils.cfg to set the default installation paths rather than 
> having to specify them on the command line for every package you install.
I'm aware of those and they are not relevant in this context.
>> distutils.errors.DistutilsError: can't create or remove files in 
>> install directory
>>
>> The following error occurred while trying to add or remove files in the
>> installation directory:
>>
>>     [Errno 2] No such file or directory: 
>> '/home/rich/projects/rcmp/junk/lib/python2.6/site-packages/test-easy-install-12743.pth'
>>
>> The installation directory you specified (via --install-dir, 
>> --prefix, or
>> the distutils default setting) was:
>>
>>     /home/rich/projects/rcmp/junk/lib/python2.6/site-packages/
>>
>> This directory does not currently exist.  Please create it and try 
>> again, or
>> choose a different installation directory (using the -d or --install-dir
>> option).
>
> As the above says, the path you're trying to install to doesn't exist.
Yes.  Although using --root= doesn't result in this problem.  So some 
portion of the install process knows how to create the appropriate 
directory tree and yet that is explicitly being skipped in this context 
for some mysterious reason.
>   (The next thing you'll find after you fix that, is that it isn't on 
> your PYTHONPATH.)
No, that won't be a problem in this context.
> What are you trying to install, and why do you want to install it in 
> that specific directory?
I really don't think it matters, but if you insist, there are several 
contexts I'm looking at.

Context #1: an alternate packaging mechanism.  I want to use that 
packaging mechanism because it's the most appropriate way to package and 
distribute my module for this application.  The packaging mechanism 
involves calling "install", (typically "make install prefix=/tmp/jail", 
but there's no reason it couldn't be "python setup.py install"), in a 
sort of jail.  Anything installed during that encapsulation is recorded 
and those pieces are the ones which are picked up and packaged by the 
automatic packaging system.

Context #2: cross development.  I'm running /on/ one system producing a 
root file system which will be used later to boot a new system.  This is 
another application of "make install PREFIX=/tmp/newroot"

Context #3: this one is complicated, but since you think it's relevant...

Bitbake is a tool written in python for managing Openembedded, 
(http://openembedded.org).  In rough terms you can think of bitbake as a 
sort of make replacement which builds a collection of components into 
binary packages and then installs those packages into a captive root 
file system.  The usual use of this system is cross development where 
the target system is woefully underpowered, (Eg, a 600Mhz arm system 
with a few Megabytes of memory), so we'd prefer to use our battery of 
x86 servers, (~64 servers, each of which are ~24 cores, 24G memory, 4 - 
8 way striped disks, etc).  The difference in build time here is 
literally the difference between about 20 minutes, (on the server farm), 
vs several months on a single, native arm based system.

Bitbake runs as a python application using the host's python 
interpreter.  But for code management reasons, bitbake itself and all of 
the bitbake input for building the system are stored as source.  This 
allows bitbake to follow the code as code is branched and the like and 
allows for multiple build directories on the same server without needing 
to use a virtual machine, (which has a huge performance overhead vs raw 
metal), or needing to reload the server, (which would also make the 
system unwieldy).

In this particular application, I want to write the application in 
python so that I can manage nose based tests outside of bitbake and so 
that I can produce a shell level cover into my tool.  However, the 
primary use of the tool will be from within bitbake.  The code of my 
tool will need to follow the source code of the system, not bitbake, so 
putting the tool into bitbake directly isn't appropriate nor is it 
appropriate to install the tool globally on the build machine(s).  
Similarly, a python virtual environment isn't relevant or appropriate 
here either as the top level control of the system is already bitbake so 
our interpreter is running before we have a chance to virtualize it.  
And there's no good reason to change that architecture.

What I want to do is to manage and build my tool in python, including 
nose tests outside of bitbake, as a component within openembedded.  This 
means that bitbake will build the component and install the component 
into a staging area which is unique to this particular build directory.  
It can then import the tool from the local staging area and use it.

What I want to do is the moral equivalent of "make install prefix=foo" 
or "make install DESTDIR=foo".  It's a common task with numerous 
applications.  This shouldn't be difficult and I really can't believe 
that I'm the first person using python to want to do this.  It's a 10 
character command line option.  There should be something comparably 
simple in distutils/setuptools/whatever, no?

--rich
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101120/2f33eebf/attachment.html>