[Distutils] Third-party packages inside a project
Giovanni Bajo
rasky at develer.com
Wed Nov 14 12:45:03 CET 2007
Hello,
for our Windows-only applications, we use a directory setup (working
copy) like this:
/lib
/a
/b
/c
/src
boot.py
pkg1/
pkg2/
pkg3/
/lib/{a,b,c} are third-party standard distutils packages downloaded from
Internet. We stuff them within the working copy because we don't want
any external dependency besides Python itself (basically, if you install
Python and do a checkout, you must have everything needed to run the
application).
/src contains our own application source code. boot.py is basically the
entry point, and the code is contained within several sub-packages.
Third-party packages are installed into /src. Basically, after
compilation, you end up with something like:
/src
boot.py
a/ [installed]
b/ [installed]
c/ [installed]
pkg1/
pkg2/
pkg3/
If you run boot.py from /src, this works fine (and has worked fine for
years).
In the past few months, we realized that this layout is probably going
to break in the future (moving to newer python versions and making the
application cross-platform) because:
1) It doesnt't work for third-party packages which use setuptools.
Setuptools always creates and installs a .pth file (to the best of my
understanding), and .pth files are not process inside /src, because it's
not part of PYTHONPATH. I can't find a viable fix for this:
* Creating a sitecustomize.py inside /src (which does something
like site.addsitedir(os.getcwd()) does not work anymore since Python
2.5. It looks like it never worked under *NIX, and was "fixed" under
Windows lately. sitecustomize.py is not imported from the current
directory, probably for security reason (code injection).
* Pointing PYTHONPATH to /src is a PITA, since developers always
move among projects and different branches of the same project (so
different working copy), and asking them to remember to change
PYTHONPATH everytime is simply a recipe for disaster.
* Using a "Custom Installation Locations" as suggested by
EasyInstall docs is not a solution, because those are "tricks" to
install python packages into an user-specific directory (instead of the
system directory). I need a solution for a project-specific directory.
2) In Python 2.4, we were able to directly run a script in a
subpackages (eg: a test). Basically, from /src, you could run "python
pkg1/test_foo.py", and that would work correctly (= absolute imports
like "import pkg2" were working as expected). In Python 2.5, this does
not work anymore.
I was wondering if someone could suggest a fix, especially for #1: I
can't believe there's no way to ask distutils/setuptools to install a
project-specific package (I am aware of the chapter "Custom Installation
Locations" in EasyInstall docs, but that's more about installing
packages at user-level instead of system-level; it doesn't cover the
project-level which I need).
Sorry for the long mail (I reckon it's better to err on the side of
verbosity to make sure everything is clear).
--
Giovanni Bajo
More information about the Distutils-SIG
mailing list