[Distutils] installing to a non-standard directory
K. Richard Pixley
rich at noir.com
Sun Nov 21 00:39:31 CET 2010
On 20101119 17:51, P.J. Eby wrote:
> At 03:18 PM 11/19/2010 -0800, K. Richard Pixley wrote:
>> I'm trying to install to a non-standard directory.
> Why? It makes a difference as to what the best way to do it is.
> For example, if you're planning to install a Python application or
> create a restricted development environment, you might be better off
> using a virtualenv. Or, alternatively, you may want to configure your
> ~/.pydistutils.cfg to set the default installation paths rather than
> having to specify them on the command line for every package you install.
I'm aware of those and they are not relevant in this context.
>> distutils.errors.DistutilsError: can't create or remove files in
>> install directory
>> The following error occurred while trying to add or remove files in the
>> installation directory:
>> [Errno 2] No such file or directory:
>> The installation directory you specified (via --install-dir,
>> --prefix, or
>> the distutils default setting) was:
>> This directory does not currently exist. Please create it and try
>> again, or
>> choose a different installation directory (using the -d or --install-dir
> As the above says, the path you're trying to install to doesn't exist.
Yes. Although using --root= doesn't result in this problem. So some
portion of the install process knows how to create the appropriate
directory tree and yet that is explicitly being skipped in this context
for some mysterious reason.
> (The next thing you'll find after you fix that, is that it isn't on
> your PYTHONPATH.)
No, that won't be a problem in this context.
> What are you trying to install, and why do you want to install it in
> that specific directory?
I really don't think it matters, but if you insist, there are several
contexts I'm looking at.
Context #1: an alternate packaging mechanism. I want to use that
packaging mechanism because it's the most appropriate way to package and
distribute my module for this application. The packaging mechanism
involves calling "install", (typically "make install prefix=/tmp/jail",
but there's no reason it couldn't be "python setup.py install"), in a
sort of jail. Anything installed during that encapsulation is recorded
and those pieces are the ones which are picked up and packaged by the
automatic packaging system.
Context #2: cross development. I'm running /on/ one system producing a
root file system which will be used later to boot a new system. This is
another application of "make install PREFIX=/tmp/newroot"
Context #3: this one is complicated, but since you think it's relevant...
Bitbake is a tool written in python for managing Openembedded,
(http://openembedded.org). In rough terms you can think of bitbake as a
sort of make replacement which builds a collection of components into
binary packages and then installs those packages into a captive root
file system. The usual use of this system is cross development where
the target system is woefully underpowered, (Eg, a 600Mhz arm system
with a few Megabytes of memory), so we'd prefer to use our battery of
x86 servers, (~64 servers, each of which are ~24 cores, 24G memory, 4 -
8 way striped disks, etc). The difference in build time here is
literally the difference between about 20 minutes, (on the server farm),
vs several months on a single, native arm based system.
Bitbake runs as a python application using the host's python
interpreter. But for code management reasons, bitbake itself and all of
the bitbake input for building the system are stored as source. This
allows bitbake to follow the code as code is branched and the like and
allows for multiple build directories on the same server without needing
to use a virtual machine, (which has a huge performance overhead vs raw
metal), or needing to reload the server, (which would also make the
In this particular application, I want to write the application in
python so that I can manage nose based tests outside of bitbake and so
that I can produce a shell level cover into my tool. However, the
primary use of the tool will be from within bitbake. The code of my
tool will need to follow the source code of the system, not bitbake, so
putting the tool into bitbake directly isn't appropriate nor is it
appropriate to install the tool globally on the build machine(s).
Similarly, a python virtual environment isn't relevant or appropriate
here either as the top level control of the system is already bitbake so
our interpreter is running before we have a chance to virtualize it.
And there's no good reason to change that architecture.
What I want to do is to manage and build my tool in python, including
nose tests outside of bitbake, as a component within openembedded. This
means that bitbake will build the component and install the component
into a staging area which is unique to this particular build directory.
It can then import the tool from the local staging area and use it.
What I want to do is the moral equivalent of "make install prefix=foo"
or "make install DESTDIR=foo". It's a common task with numerous
applications. This shouldn't be difficult and I really can't believe
that I'm the first person using python to want to do this. It's a 10
character command line option. There should be something comparably
simple in distutils/setuptools/whatever, no?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Distutils-SIG