Hello,
We have a quite large project of some quite stable extension modules and lots of frequently changing python modules. The whole thing works under Solaris and Linux. Both operating systems are accessing the same file tree using NFS.
My problem now is that setup on a mixed module installes everything into the system dependent branch of the install directories. It would be nice if there is a simple customizing of the setup procedure to make setup install the extension modules into the system depending branch and the python modules into the system independent branch. Of course I can issue two "setup" commands in the setup.py, but that produces ugly results when it comes to generating binary installation packages.
Regards Berthold
On 18 Mar 2003 16:15:18 +0100, Berthold Höllmann wrote:
My problem now is that setup on a mixed module installes everything into the system dependent branch of the install directories.
Errmmmm, setup.py install --help
Options for 'install' command: ... --install-base base installation directory (instead of --prefix or -- home) --install-platbase base installation directory for platform-specific files (instead of --exec-prefix or --home) --install-purelib installation directory for pure Python module distributions --install-platlib installation directory for non-pure module distributions
Ciao, Jürgen
"Juergen Hermann" jh@web.de writes:
On 18 Mar 2003 16:15:18 +0100, Berthold Höllmann wrote:
My problem now is that setup on a mixed module installes everything into the system dependent branch of the install directories.
Errmmmm, setup.py install --help
Options for 'install' command: ... --install-base base installation directory (instead of --prefix or -- home) --install-platbase base installation directory for platform-specific files (instead of --exec-prefix or --home) --install-purelib installation directory for pure Python module distributions --install-platlib installation directory for non-pure module distributions
Errmmm, what I am looking for is a way to have, installing mixed modules, the extension modules installed in the --install-platlib tree and the pure python modules installed in the --install-purelib path. Now when you install mixed moudules everything gets installed into the --install-platlib tree, even the pure python modules.
Regards, Berthold
Berthold Höllmann wrote:
"Juergen Hermann" jh@web.de writes:
On 18 Mar 2003 16:15:18 +0100, Berthold Höllmann wrote:
My problem now is that setup on a mixed module installes everything into the system dependent branch of the install directories.
Errmmmm, setup.py install --help
Options for 'install' command: ... --install-base base installation directory (instead of --prefix or -- home) --install-platbase base installation directory for platform-specific files (instead of --exec-prefix or --home) --install-purelib installation directory for pure Python module distributions --install-platlib installation directory for non-pure module distributions
Errmmm, what I am looking for is a way to have, installing mixed modules, the extension modules installed in the --install-platlib tree and the pure python modules installed in the --install-purelib path. Now when you install mixed moudules everything gets installed into the --install-platlib tree, even the pure python modules.
If you are using packages, that's the only way it works (you can't have two parallel package structures with the same base name in Python). If you are distributing the files as top-level files, then this looks like a bug.
"M.-A. Lemburg" mal@lemburg.com writes:
Berthold Höllmann wrote:
"Juergen Hermann" jh@web.de writes:
On 18 Mar 2003 16:15:18 +0100, Berthold Höllmann wrote:
My problem now is that setup on a mixed module installes everything into the system dependent branch of the install directories.
Errmmmm, setup.py install --help
Options for 'install' command: ... --install-base base installation directory (instead of --prefix or -- home) --install-platbase base installation directory for platform-specific files (instead of --exec-prefix or --home) --install-purelib installation directory for pure Python module distributions --install-platlib installation directory for non-pure module distributions
Errmmm, what I am looking for is a way to have, installing mixed modules, the extension modules installed in the --install-platlib tree and the pure python modules installed in the --install-purelib path. Now when you install mixed moudules everything gets installed into the --install-platlib tree, even the pure python modules.
If you are using packages, that's the only way it works (you can't have two parallel package structures with the same base name in Python). If you are distributing the files as top-level files, then this looks like a bug.
So it is, I guess a bug. I build a small example:
cat spam.c
#include <Python.h>
static PyObject * spam_system(self, args) PyObject *self; PyObject *args; { char *command; int sts;
if (!PyArg_ParseTuple(args, "s", &command)) return NULL; sts = system(command); return Py_BuildValue("i", sts); } static PyObject *SpamError;
static PyMethodDef SpamMethods[] = { {"system", spam_system, METH_VARARGS, "Execute a shell command."}, {NULL, NULL, 0, NULL} /* Sentinel */ };
void initspam(void) { PyObject *m, *d;
m = Py_InitModule("spam", SpamMethods); d = PyModule_GetDict(m); SpamError = PyErr_NewException("spam.error", NULL, NULL); PyDict_SetItemString(d, "error", SpamError); }
cat pork.py
import spam as _spam
def spam(): return _spam.system("ls -l")
cat setup.py
from distutils.core import setup,Extension
setup (name = "pork.spam", version = "0.0.0", description = "just a test", author = 'Berthold Höllmann', author_email = "hoel@GL-Group.com", ext_modules = [Extension('spam', ['spam.c'])], package_dir = {'pork': '.'}, py_modules = ['pork.pork'], )
touch __init__.py python setup.py build
...
python setup.py -n install
Adding parser accelerators ... Done. running install running build running build_py not copying ./__init__.py (output up-to-date) not copying ./pork.py (output up-to-date) running build_ext skipping 'spam' extension (up-to-date) running install_lib copying build/lib.linux-i686-2.2/pork/pork.py -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork copying build/lib.linux-i686-2.2/pork/__init__.py -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork copying build/lib.linux-i686-2.2/spam.so -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages error: file '/usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork/pork.py' does not exist [16957 refs]
python -c "import distutils.sysconfig as d;print d.get_python_lib();print d.get_python_lib(True)"
Adding parser accelerators ... Done. /usr/local/fitools/sandbox/lib/python2.2/site-packages /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages [5792 refs]
In my understanding pork.py sould have gone to /usr/local/fitools/sandbox/lib/python2.2/site-packages/pork instead of /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork (same of course for __init__.py).
Regards Berthold
Berthold Höllmann wrote:
"M.-A. Lemburg" mal@lemburg.com writes:
If you are using packages, that's the only way it works (you can't have two parallel package structures with the same base name in Python). If you are distributing the files as top-level files, then this looks like a bug.
So it is, I guess a bug. I build a small example:
I don't think so...
setup (name = "pork.spam", version = "0.0.0", description = "just a test", author = 'Berthold Höllmann', author_email = "hoel@GL-Group.com", ext_modules = [Extension('spam', ['spam.c'])], package_dir = {'pork': '.'},
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
py_modules = ['pork.pork'],)
"M.-A. Lemburg" mal@lemburg.com writes:
Berthold Höllmann wrote:
"M.-A. Lemburg" mal@lemburg.com writes:
If you are using packages, that's the only way it works (you can't have two parallel package structures with the same base name in Python). If you are distributing the files as top-level files, then this looks like a bug.
So it is, I guess a bug. I build a small example:
I don't think so...
setup (name = "pork.spam", version = "0.0.0", description = "just a test", author = 'Berthold Höllmann', author_email = "hoel@GL-Group.com", ext_modules = [Extension('spam', ['spam.c'])], package_dir = {'pork': '.'},
^^^^^^^^^^^^^^^^^^^^^^^^^^^^py_modules = ['pork.pork'],)
sorry, of course, but, I moved pork.py and __init__.py to the new generated pork subdirectory, removed the "package_dir" line and
python setup.py -n install
Adding parser accelerators ... Done. running install running build running build_py copying pork/__init__.py -> build/lib.linux-i686-2.2/pork not copying pork/pork.py (output up-to-date) running build_ext skipping 'spam' extension (up-to-date) running install_lib copying build/lib.linux-i686-2.2/pork/pork.py -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork copying build/lib.linux-i686-2.2/spam.so -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages error: file '/usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork/pork.py' does not exist [16946 refs]
Do you accept this as an error?
Regards Berthold
Berthold Höllmann wrote:
"M.-A. Lemburg" mal@lemburg.com writes:
Berthold Höllmann wrote:
"M.-A. Lemburg" mal@lemburg.com writes:
If you are using packages, that's the only way it works (you can't have two parallel package structures with the same base name in Python). If you are distributing the files as top-level files, then this looks like a bug.
So it is, I guess a bug. I build a small example:
I don't think so...
setup (name = "pork.spam", version = "0.0.0", description = "just a test", author = 'Berthold Höllmann', author_email = "hoel@GL-Group.com", ext_modules = [Extension('spam', ['spam.c'])], package_dir = {'pork': '.'},
^^^^^^^^^^^^^^^^^^^^^^^^^^^^py_modules = ['pork.pork'],)
sorry, of course, but, I moved pork.py and __init__.py to the new generated pork subdirectory, removed the "package_dir" line and
You should also rename the py_modules entry to 'pork'.
python setup.py -n install
Uhm, where did you tell distutils to install in two different directories for platform dependent and independent files ?
Adding parser accelerators ... Done. running install running build running build_py copying pork/__init__.py -> build/lib.linux-i686-2.2/pork not copying pork/pork.py (output up-to-date) running build_ext skipping 'spam' extension (up-to-date) running install_lib copying build/lib.linux-i686-2.2/pork/pork.py -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork copying build/lib.linux-i686-2.2/spam.so -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages error: file '/usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork/pork.py' does not exist [16946 refs]
Do you accept this as an error?
No, because 'pork.pork' is not allowed in py_modules (it's a hack that sometimes works, but not intended as feature AFAIK).
"M.-A. Lemburg" mal@lemburg.com writes:
Berthold Höllmann wrote:
"M.-A. Lemburg" mal@lemburg.com writes:
Berthold Höllmann wrote:
"M.-A. Lemburg" mal@lemburg.com writes:
...
No, because 'pork.pork' is not allowed in py_modules (it's a hack that sometimes works, but not intended as feature AFAIK).
But even this changes do not cure the problem:
cat setup.py ; python setup.py -n install
from distutils.core import setup,Extension
setup (name = "pork.spam", version = "0.0.0", description = "just a test", author = 'Berthold Höllmann', author_email = "hoel@GL-Group.com", ext_modules = [Extension('spam', ['spam.c'])], packages = ['pork'] )
Adding parser accelerators ... Done. running install running build running build_py not copying pork/pork.py (output up-to-date) not copying pork/__init__.py (output up-to-date) running build_ext skipping 'spam' extension (up-to-date) running install_lib copying build/lib.linux-i686-2.2/spam.so -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages copying build/lib.linux-i686-2.2/pork/pork.py -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork copying build/lib.linux-i686-2.2/pork/__init__.py -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork
the pure python package still gets installed alongside the only loosly extension module.
Regards Berthold
Berthold Höllmann wrote:
No, because 'pork.pork' is not allowed in py_modules (it's a hack that sometimes works, but not intended as feature AFAIK).
But even this changes do not cure the problem:
cat setup.py ; python setup.py -n install
As I wrote before: you have to explicitly *tell* distutils to install in two different directories !
""" packages/egenix-mx-base> python setup.py install --help Global options: --verbose (-v) run verbosely (default) --quiet (-q) run quietly (turns verbosity off) --dry-run (-n) don't actually do anything --help (-h) show detailed help message
Options for 'mx_install' command: --prefix installation prefix --exec-prefix (Unix only) prefix for platform-specific files --home (Unix only) home directory to install under
vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
--install-base base installation directory (instead of --prefix or -- home) --install-platbase base installation directory for platform-specific files (instead of --exec-prefix or --home)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
--root install everything relative to this alternate root directory --install-purelib installation directory for pure Python module distributions --install-platlib installation directory for non-pure module distributions --install-lib installation directory for all module distributions (overrides --install-purelib and --install-platlib) --install-headers installation directory for C/C++ headers --install-scripts installation directory for Python scripts --install-data installation directory for data files --compile (-c) compile .py to .pyc [default] --no-compile don't compile .py files --optimize (-O) also compile with optimization: -O1 for "python -O", -O2 for "python -OO", and -O0 to disable [default: -O0] --force (-f) force installation (overwrite any existing files) --skip-build skip rebuilding everything (for testing/debugging) --record filename in which to record list of installed files
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help """
from distutils.core import setup,Extension
setup (name = "pork.spam", version = "0.0.0", description = "just a test", author = 'Berthold Höllmann', author_email = "hoel@GL-Group.com", ext_modules = [Extension('spam', ['spam.c'])], packages = ['pork'] )
Adding parser accelerators ... Done. running install running build running build_py not copying pork/pork.py (output up-to-date) not copying pork/__init__.py (output up-to-date) running build_ext skipping 'spam' extension (up-to-date) running install_lib copying build/lib.linux-i686-2.2/spam.so -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages copying build/lib.linux-i686-2.2/pork/pork.py -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork copying build/lib.linux-i686-2.2/pork/__init__.py -> /usr/local/fitools/sandbox/linux/lib/python2.2/site-packages/pork
the pure python package still gets installed alongside the only loosly extension module.
Regards Berthold
"M.-A. Lemburg" mal@lemburg.com writes:
Berthold Höllmann wrote:
No, because 'pork.pork' is not allowed in py_modules (it's a hack that sometimes works, but not intended as feature AFAIK).
But even this changes do not cure the problem:
cat setup.py ; python setup.py -n install
As I wrote before: you have to explicitly *tell* distutils to install in two different directories !
vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
--install-base base installation directory (instead of --prefix or -- home) --install-platbase base installation directory for platform-specific files (instead of --exec-prefix or --home)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sorry, but I still understand why. My Python/distutil installation has the default --install-base as --prefix which is already different from --install-platbase, or --exec-prefix. If I install a pure Python module or package it gets installed into "--prefix" which is "/usr/local/fitools/sandbox/lib/python2.2/site-packages" and packages holding extension modules get installed into "--exec-prefix" which is "/usr/local/fitools/sandbox/linux/lib/python2.2/site-packages". Why doesn't it work when I use both in one "setup" command? Default values for "--prefix" and "--exec-prefix" are taken from the initial "./configure" in the python source tree, arent they?
Regards Berthold
-
Berthold Höllmann -
Berthold Höllmann -
Juergen Hermann -
M.-A. Lemburg