bdist_rpm and bdist on x86-64
setup.py bdist and bdist_rpm appear to fail on my x86-64 machine running Fedora Core 2. ... running build_scripts creating build/scripts-2.3 copying and adjusting scripts/veusz -> build/scripts-2.3 copying and adjusting scripts/veusz_listen -> build/scripts-2.3 changing mode of build/scripts-2.3/veusz from 664 to 775 changing mode of build/scripts-2.3/veusz_listen from 664 to 775 installing to build/bdist.linux-x86_64/dumb running install error: invalid Python installation: unable to open /usr/lib/python2.3/config/Makefile (No such file or directory) It appears that distutils is looking in /usr/lib/python2.3/config rather than /usr/lib64/python2.3/config I haven't been able to easily locate where it's going wrong. Any ideas?? I'm using the distutils from Python-2.4.1 on Python-2.3.3 (64 bit). Thanks Jeremy -- Jeremy Sanders <jeremy@jeremysanders.net> http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053
Jeremy Sanders wrote:
setup.py bdist and bdist_rpm appear to fail on my x86-64 machine running Fedora Core 2.
... running build_scripts creating build/scripts-2.3 copying and adjusting scripts/veusz -> build/scripts-2.3 copying and adjusting scripts/veusz_listen -> build/scripts-2.3 changing mode of build/scripts-2.3/veusz from 664 to 775 changing mode of build/scripts-2.3/veusz_listen from 664 to 775 installing to build/bdist.linux-x86_64/dumb running install error: invalid Python installation: unable to open /usr/lib/python2.3/config/Makefile (No such file or directory)
It appears that distutils is looking in /usr/lib/python2.3/config rather than /usr/lib64/python2.3/config
I haven't been able to easily locate where it's going wrong. Any ideas??
Check the Makefile you Python version has installed in lib/pythonX.X/config/Makefile distutils reads the specs from that file.
I'm using the distutils from Python-2.4.1 on Python-2.3.3 (64 bit).
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 15 2005)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
On Fri, 15 Apr 2005, M.-A. Lemburg wrote:
Check the Makefile you Python version has installed in lib/pythonX.X/config/Makefile
There is no /usr/lib/python2.3/config/Makefile. It is installed in /usr/lib64/python2.3/config/Makefile. That file contains # Expanded directories BINDIR= $(exec_prefix)/bin LIBDIR= $(exec_prefix)/lib64 MANDIR= /usr/share/man INCLUDEDIR= /usr/include CONFINCLUDEDIR= $(exec_prefix)/include SCRIPTDIR= $(prefix)/lib64 # Detailed destination directories BINLIBDEST= $(LIBDIR)/python$(VERSION) LIBDEST= $(SCRIPTDIR)/python$(VERSION) INCLUDEPY= $(INCLUDEDIR)/python$(VERSION) CONFINCLUDEPY= $(CONFINCLUDEDIR)/python$(VERSION) LIBP= $(LIBDIR)/python$(VERSION) So it looks like distutils is looking in the wrong place. If you look in sysconfig.py: def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): """Return the directory containing the Python library (standard or site additions). If 'plat_specific' is true, return the directory containing platform-specific modules, i.e. any module from a non-pure-Python module distribution; otherwise, return the platform-shared library directory. If 'standard_lib' is true, return the directory containing standard Python library modules; otherwise, return the directory for site-specific modules. If 'prefix' is supplied, use it instead of sys.prefix or sys.exec_prefix -- i.e., ignore 'plat_specific'. """ if prefix is None: prefix = plat_specific and EXEC_PREFIX or PREFIX if os.name == "posix": libpython = os.path.join(prefix, "lib", "python" + get_python_version()) if standard_lib: return libpython else: return os.path.join(libpython, "site-packages") elif os.name == "nt": if standard_lib: return os.path.join(prefix, "Lib") else: if get_python_version() < "2.2": return prefix else: return os.path.join(PREFIX, "Lib", "site-packages") elif os.name == "mac": if plat_specific: if standard_lib: return os.path.join(prefix, "Lib", "lib-dynload") else: return os.path.join(prefix, "Lib", "site-packages") else: if standard_lib: return os.path.join(prefix, "Lib") else: return os.path.join(prefix, "Lib", "site-packages") elif os.name == "os2": if standard_lib: return os.path.join(PREFIX, "Lib") else: return os.path.join(PREFIX, "Lib", "site-packages") else: raise DistutilsPlatformError( "I don't know where Python installs its library " "on platform '%s'" % os.name) Under the posix section, distutils assumes that Python is installed in /usr/lib/python-X.X, where it's really in /usr/lib64/python-X.X. This is clearly a distutils bug. Distutils should be looking under lib64 for 64-bit x86 systems. All linux x86-64 distributions use lib64 instead of lib for 64 bit libraries. It looks like this code needs to be cleverer. Jeremy -- Jeremy Sanders <jeremy@jeremysanders.net> http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053
Jeremy Sanders wrote:
On Fri, 15 Apr 2005, M.-A. Lemburg wrote:
Check the Makefile you Python version has installed in lib/pythonX.X/config/Makefile
There is no /usr/lib/python2.3/config/Makefile. It is installed in /usr/lib64/python2.3/config/Makefile.
That file contains
# Expanded directories BINDIR= $(exec_prefix)/bin LIBDIR= $(exec_prefix)/lib64 MANDIR= /usr/share/man INCLUDEDIR= /usr/include CONFINCLUDEDIR= $(exec_prefix)/include SCRIPTDIR= $(prefix)/lib64
# Detailed destination directories BINLIBDEST= $(LIBDIR)/python$(VERSION) LIBDEST= $(SCRIPTDIR)/python$(VERSION) INCLUDEPY= $(INCLUDEDIR)/python$(VERSION) CONFINCLUDEPY= $(CONFINCLUDEDIR)/python$(VERSION) LIBP= $(LIBDIR)/python$(VERSION)
So it looks like distutils is looking in the wrong place.
If you look in sysconfig.py:
def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): """Return the directory containing the Python library (standard or site additions).
If 'plat_specific' is true, return the directory containing platform-specific modules, i.e. any module from a non-pure-Python module distribution; otherwise, return the platform-shared library directory. If 'standard_lib' is true, return the directory containing standard Python library modules; otherwise, return the directory for site-specific modules.
If 'prefix' is supplied, use it instead of sys.prefix or sys.exec_prefix -- i.e., ignore 'plat_specific'. """ if prefix is None: prefix = plat_specific and EXEC_PREFIX or PREFIX
if os.name == "posix": libpython = os.path.join(prefix, "lib", "python" + get_python_version())
This is the problem line ^^^^^^^
if standard_lib: return libpython else: return os.path.join(libpython, "site-packages")
elif os.name == "nt": if standard_lib: return os.path.join(prefix, "Lib") else: if get_python_version() < "2.2": return prefix else: return os.path.join(PREFIX, "Lib", "site-packages")
elif os.name == "mac": if plat_specific: if standard_lib: return os.path.join(prefix, "Lib", "lib-dynload") else: return os.path.join(prefix, "Lib", "site-packages") else: if standard_lib: return os.path.join(prefix, "Lib") else: return os.path.join(prefix, "Lib", "site-packages")
elif os.name == "os2": if standard_lib: return os.path.join(PREFIX, "Lib") else: return os.path.join(PREFIX, "Lib", "site-packages")
else: raise DistutilsPlatformError( "I don't know where Python installs its library " "on platform '%s'" % os.name)
Under the posix section, distutils assumes that Python is installed in /usr/lib/python-X.X, where it's really in /usr/lib64/python-X.X. This is clearly a distutils bug. Distutils should be looking under lib64 for 64-bit x86 systems.
All linux x86-64 distributions use lib64 instead of lib for 64 bit libraries. It looks like this code needs to be cleverer.
Patches are welcome :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 15 2005)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
On Fri, 15 Apr 2005, M.-A. Lemburg wrote:
All linux x86-64 distributions use lib64 instead of lib for 64 bit libraries. It looks like this code needs to be cleverer.
Patches are welcome :-)
Attached is a patch which allows distutils to find the lib directory on x86-64. It uses sys.path to search for the Makefile. It also changes command/install.py to change platlib to lib64 if lib64 was returned by get_python_lib. It might be a bit messy for distutils of course, but it shouldn't affect non-x86-64 platforms. Opinions? Jeremy -- Jeremy Sanders <jeremy@jeremysanders.net> http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053 --- sysconfig.py~ 2005-03-03 11:08:02.000000000 +0000 +++ sysconfig.py 2005-04-16 12:20:03.499916704 +0100 @@ -99,8 +99,17 @@ prefix = plat_specific and EXEC_PREFIX or PREFIX if os.name == "posix": - libpython = os.path.join(prefix, - "lib", "python" + get_python_version()) + # search for Makefile in path to locate lib directory, in case + # python isn't in /prefix/lib (e.g. /prefix/lib64) + for i in sys.path: + if os.access( os.path.join(i, "config", "Makefile"), os.F_OK ): + libpython = i + break + else: + # Makefile not found in path + libpython = os.path.join(prefix, + "lib", "python" + get_python_version()) + if standard_lib: return libpython else: --- command/install.py~ 2005-01-20 19:14:17.000000000 +0000 +++ command/install.py 2005-04-16 13:12:07.264032376 +0100 @@ -10,6 +10,7 @@ import sys, os, string from types import * +from distutils.sysconfig import get_python_lib from distutils.core import Command from distutils.debug import DEBUG from distutils.sysconfig import get_config_vars @@ -36,17 +37,25 @@ 'data' : '$base', } + +# Normally unix distributions use lib to store python platlib +# However, under x86-64, platlib uses lib64 +unix_platlib = 'lib' +if get_python_lib().find('lib64') != -1: + unix_platlib = 'lib64' + INSTALL_SCHEMES = { 'unix_prefix': { 'purelib': '$base/lib/python$py_version_short/site-packages', - 'platlib': '$platbase/lib/python$py_version_short/site-packages', + 'platlib': + '$platbase/%s/python$py_version_short/site-packages' % unix_platlib, 'headers': '$base/include/python$py_version_short/$dist_name', 'scripts': '$base/bin', 'data' : '$base', }, 'unix_home': { 'purelib': '$base/lib/python', - 'platlib': '$base/lib/python', + 'platlib': '$base/%s/python' % unix_platlib, 'headers': '$base/include/python/$dist_name', 'scripts': '$base/bin', 'data' : '$base',
Jeremy Sanders wrote:
On Fri, 15 Apr 2005, M.-A. Lemburg wrote:
All linux x86-64 distributions use lib64 instead of lib for 64 bit libraries. It looks like this code needs to be cleverer.
Patches are welcome :-)
Attached is a patch which allows distutils to find the lib directory on x86-64. It uses sys.path to search for the Makefile.
It also changes command/install.py to change platlib to lib64 if lib64 was returned by get_python_lib.
It might be a bit messy for distutils of course, but it shouldn't affect non-x86-64 platforms.
Opinions?
Jeremy
------------------------------------------------------------------------
--- sysconfig.py~ 2005-03-03 11:08:02.000000000 +0000 +++ sysconfig.py 2005-04-16 12:20:03.499916704 +0100 @@ -99,8 +99,17 @@ prefix = plat_specific and EXEC_PREFIX or PREFIX
if os.name == "posix": - libpython = os.path.join(prefix, - "lib", "python" + get_python_version()) + # search for Makefile in path to locate lib directory, in case + # python isn't in /prefix/lib (e.g. /prefix/lib64) + for i in sys.path: + if os.access( os.path.join(i, "config", "Makefile"), os.F_OK ):
Why not os.exists() ?
+ libpython = i + break + else: + # Makefile not found in path + libpython = os.path.join(prefix, + "lib", "python" + get_python_version()) +
Wouldn't checking for the lib file in either lib64 or lib be more reliable ?
if standard_lib: return libpython else: --- command/install.py~ 2005-01-20 19:14:17.000000000 +0000 +++ command/install.py 2005-04-16 13:12:07.264032376 +0100 @@ -10,6 +10,7 @@
import sys, os, string from types import * +from distutils.sysconfig import get_python_lib from distutils.core import Command from distutils.debug import DEBUG from distutils.sysconfig import get_config_vars @@ -36,17 +37,25 @@ 'data' : '$base', }
+ +# Normally unix distributions use lib to store python platlib +# However, under x86-64, platlib uses lib64 +unix_platlib = 'lib' +if get_python_lib().find('lib64') != -1: + unix_platlib = 'lib64' +
Hmm, this is not entirely correct, e.g. Suse 9.2 puts the site-packages directory and all the other .py files into /usr/lib64 as well - not only the platforma dependent files. Not sure what other AMD64 distros do... but I have a feeling the /lib/ should *always* be replaced by unix_platlib.
INSTALL_SCHEMES = { 'unix_prefix': { 'purelib': '$base/lib/python$py_version_short/site-packages', - 'platlib': '$platbase/lib/python$py_version_short/site-packages', + 'platlib': + '$platbase/%s/python$py_version_short/site-packages' % unix_platlib, 'headers': '$base/include/python$py_version_short/$dist_name', 'scripts': '$base/bin', 'data' : '$base', }, 'unix_home': { 'purelib': '$base/lib/python', - 'platlib': '$base/lib/python', + 'platlib': '$base/%s/python' % unix_platlib, 'headers': '$base/include/python/$dist_name', 'scripts': '$base/bin', 'data' : '$base',
------------------------------------------------------------------------
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 18 2005)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
On Mon, 18 Apr 2005, M.-A. Lemburg wrote:
Why not os.exists() ?
Probably better :-)
Wouldn't checking for the lib file in either lib64 or lib be more reliable ?
That might be better. I don't know whether it's possible that more than one python is installed (one in /lib and one in /lib64).
Hmm, this is not entirely correct, e.g. Suse 9.2 puts the site-packages directory and all the other .py files into /usr/lib64 as well - not only the platforma dependent files.
Not sure what other AMD64 distros do... but I have a feeling the /lib/ should *always* be replaced by unix_platlib.
It looks like RedHat/Fedora patch their package to only put the platform specific files in /lib64 (that's how I made my patch). Perhaps this isn't a good idea to do then :-( I wonder whether it would be possible for distribution to set these values somewhere. Couldn't python have a sys function to return its Makefile? (anyway, I've managed to get my setup.py to work with the built-in distutils...) Jeremy -- Jeremy Sanders <jeremy@jeremysanders.net> http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053
Jeremy Sanders wrote:
On Mon, 18 Apr 2005, M.-A. Lemburg wrote:
Why not os.exists() ?
s/os.exists/os.path.exists
Probably better :-)
Wouldn't checking for the lib file in either lib64 or lib be more reliable ?
That might be better. I don't know whether it's possible that more than one python is installed (one in /lib and one in /lib64).
That would certainly be possible.
Hmm, this is not entirely correct, e.g. Suse 9.2 puts the site-packages directory and all the other .py files into /usr/lib64 as well - not only the platforma dependent files.
Not sure what other AMD64 distros do... but I have a feeling the /lib/ should *always* be replaced by unix_platlib.
It looks like RedHat/Fedora patch their package to only put the platform specific files in /lib64 (that's how I made my patch).
Perhaps this isn't a good idea to do then :-( I wonder whether it would be possible for distribution to set these values somewhere. Couldn't python have a sys function to return its Makefile?
It's rather easy to find the right Makefile: just look for os.py (the landmark used in Python to initialize sys.path on startup), join it's path with "config" and you're there. More details can be found in Modules/getpath.c of the Python source distribution. You can then have distutils parse that file to extract the other details. Note that this doesn't work if you're importing from a ZIP file.
(anyway, I've managed to get my setup.py to work with the built-in distutils...)
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 19 2005)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
On Tue, Apr 19, 2005 at 03:00:49PM +0200, M.-A. Lemburg wrote:
Jeremy Sanders wrote:
On Mon, 18 Apr 2005, M.-A. Lemburg wrote:
Why not os.exists() ?
s/os.exists/os.path.exists
Probably better :-)
Wouldn't checking for the lib file in either lib64 or lib be more reliable ?
That might be better. I don't know whether it's possible that more than one python is installed (one in /lib and one in /lib64).
That would certainly be possible.
Hmm, this is not entirely correct, e.g. Suse 9.2 puts the site-packages directory and all the other .py files into /usr/lib64 as well - not only the platforma dependent files.
Not sure what other AMD64 distros do... but I have a feeling the /lib/ should *always* be replaced by unix_platlib.
It looks like RedHat/Fedora patch their package to only put the platform specific files in /lib64 (that's how I made my patch).
Perhaps this isn't a good idea to do then :-( I wonder whether it would be possible for distribution to set these values somewhere. Couldn't python have a sys function to return its Makefile?
Fedora does it this way because of .noarch.rpm packages. Pure python libraries should run just fine both on x86 and x86_64, and since it's /usr/lib on x86, x86_64 has to know about /usr/lib too (which is sort of confusing). I believe you can get away without patching anything, if you do: from distutils import sysconfig print sysconfig.get_python_lib() to which you either pass plat_specific = 0 or 1. This will properly parse the right Makefile (which is probably what you ended up doing). Misa
On Tue, 19 Apr 2005, Mihai Ibanescu wrote:
I believe you can get away without patching anything, if you do:
from distutils import sysconfig print sysconfig.get_python_lib()
to which you either pass plat_specific = 0 or 1. This will properly parse the right Makefile (which is probably what you ended up doing).
The problem is that this doesn't find the Makefile if you use the unpatched distutils on Fedora on x86_64. Fedora patches distutils to look in lib64. The unpatched distutils looks in lib, and never finds it. It would be nice if Python knew where its Makefile was. Jeremy -- Jeremy Sanders <jeremy@jeremysanders.net> http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053
On Tue, Apr 19, 2005 at 02:55:03PM +0100, Jeremy Sanders wrote:
On Tue, 19 Apr 2005, Mihai Ibanescu wrote:
I believe you can get away without patching anything, if you do:
from distutils import sysconfig print sysconfig.get_python_lib()
to which you either pass plat_specific = 0 or 1. This will properly parse the right Makefile (which is probably what you ended up doing).
The problem is that this doesn't find the Makefile if you use the unpatched distutils on Fedora on x86_64. Fedora patches distutils to look in lib64. The unpatched distutils looks in lib, and never finds it.
Sorry if I missed it from previous e-mails. Why don't you use Fedora's patched distutils then? :-)
It would be nice if Python knew where its Makefile was.
Python does, if it's in the standard location. /usr/lib64 was not necessarily something that worked well for Python, at least not from the very beginning, and that's because there is a prefix and an exec_prefix, and then an unpatched python will just slap a lib after that. If multilib were developed to use a totally different prefix than /usr (like /usr64), it would definitely have been easier. That, or I am missing the obvious fixes :-) Misa
Mihai Ibanescu wrote:
On Tue, Apr 19, 2005 at 02:55:03PM +0100, Jeremy Sanders wrote:
On Tue, 19 Apr 2005, Mihai Ibanescu wrote:
I believe you can get away without patching anything, if you do:
from distutils import sysconfig
print sysconfig.get_python_lib()
to which you either pass plat_specific = 0 or 1. This will properly parse the right Makefile (which is probably what you ended up doing).
The problem is that this doesn't find the Makefile if you use the unpatched distutils on Fedora on x86_64. Fedora patches distutils to look in lib64. The unpatched distutils looks in lib, and never finds it.
Sorry if I missed it from previous e-mails. Why don't you use Fedora's patched distutils then? :-)
I wonder why Linux distros always have to make things more complicated by patching code in the standard lib or using non-standard compile time options...
It would be nice if Python knew where its Makefile was.
Jeremy, didn't you get my last posting ? It is rather easy to find the Makefile since the config/ dir is stored in the same place as the rest of the Python std lib (provided the distros didn't munge that fact as well, like e.g. some which remove distutils from the std lib <grumble>).
Python does, if it's in the standard location. /usr/lib64 was not necessarily something that worked well for Python, at least not from the very beginning, and that's because there is a prefix and an exec_prefix, and then an unpatched python will just slap a lib after that. If multilib were developed to use a totally different prefix than /usr (like /usr64), it would definitely have been easier.
That, or I am missing the obvious fixes :-)
I think that Linux distros should approach the Python developers first before fiddling around with Python internals. We would come up with a solution that works for everybody, not just distro XYZ users. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 20 2005)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
On Wed, 20 Apr 2005, M.-A. Lemburg wrote:
Jeremy, didn't you get my last posting ? It is rather easy to find the Makefile since the config/ dir is stored in the same place as the rest of the Python std lib (provided the distros didn't munge that fact as well, like e.g. some which remove distutils from the std lib <grumble>).
Yes, we could just do something like os.path.join(os.path.dirname(os.__file__), 'config', 'Makefile') to find the Makefile location (as long as os doesn't become built-in, like sys, or ends up in a zip file). There's still the problem of working out where x86-64 non specfic things should get installed in lib64 or lib. I don't see an easy answer there. It is a real shame that RedHat/Fedora didn't get this sorted before they started distributing python on x86-64. I wonder what SuSE are doing with lib/lib64? Jeremy -- Jeremy Sanders <jeremy@jeremysanders.net> http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053
Jeremy Sanders wrote:
On Wed, 20 Apr 2005, M.-A. Lemburg wrote:
Jeremy, didn't you get my last posting ? It is rather easy to find the Makefile since the config/ dir is stored in the same place as the rest of the Python std lib (provided the distros didn't munge that fact as well, like e.g. some which remove distutils from the std lib <grumble>).
Yes, we could just do something like
os.path.join(os.path.dirname(os.__file__), 'config', 'Makefile')
to find the Makefile location (as long as os doesn't become built-in, like sys, or ends up in a zip file).
There's still the problem of working out where x86-64 non specfic things should get installed in lib64 or lib. I don't see an easy answer there.
It is a real shame that RedHat/Fedora didn't get this sorted before they started distributing python on x86-64. I wonder what SuSE are doing with lib/lib64?
SuSE 9.2 puts the 64-bit version into /usr/lib64 (the complete lib, including the site-packages) and the 32-bit version into /usr/lib. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 20 2005)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
Mihai Ibanescu wrote:
It looks like RedHat/Fedora patch their package to only put the platform specific files in /lib64 (that's how I made my patch).
Perhaps this isn't a good idea to do then :-( I wonder whether it would be possible for distribution to set these values somewhere. Couldn't python have a sys function to return its Makefile?
Fedora does it this way because of .noarch.rpm packages. Pure python libraries should run just fine both on x86 and x86_64, and since it's /usr/lib on x86, x86_64 has to know about /usr/lib too (which is sort of confusing).
Not only that, I doubt that this will work well unless you now have two site-packages dirs (one in /usr/lib for Python only extensions and one in /usr/lib64 for combined Python and binary extensions).
I believe you can get away without patching anything, if you do:
from distutils import sysconfig print sysconfig.get_python_lib()
to which you either pass plat_specific = 0 or 1. This will properly parse the right Makefile (which is probably what you ended up doing).
Uhm, this is the function that Jeremy was patching - the problem being that the "lib" part of the path is hard-coded into this function. It doesn't parse the Makefile, BTW, and plat_specific only adds site-packages to the returned directory - it doesn't have anything to do with platform specific code. The Makefile is the only place where Python itself can look up how it was configured. It's easy to find: distutils.sysconfig.get_makefile_filename(); distutils.sysconfig.parse_makefile(filename) then does the rest. Or in one go: distutils.sysconfig.get_config_vars(). The interesting parts of the Makefile are these variables: # Detailed destination directories BINLIBDEST= $(LIBDIR)/python$(VERSION) LIBDEST= $(SCRIPTDIR)/python$(VERSION) INCLUDEPY= $(INCLUDEDIR)/python$(VERSION) CONFINCLUDEPY= $(CONFINCLUDEDIR)/python$(VERSION) LIBP= $(LIBDIR)/python$(VERSION) I guess we should change the install.py to use these values instead of building its own little world of pathnames. What do you think ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 20 2005)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
On Wed, Apr 20, 2005 at 11:00:14AM +0200, M.-A. Lemburg wrote:
Mihai Ibanescu wrote:
It looks like RedHat/Fedora patch their package to only put the platform specific files in /lib64 (that's how I made my patch).
Perhaps this isn't a good idea to do then :-( I wonder whether it would be possible for distribution to set these values somewhere. Couldn't python have a sys function to return its Makefile?
Fedora does it this way because of .noarch.rpm packages. Pure python libraries should run just fine both on x86 and x86_64, and since it's /usr/lib on x86, x86_64 has to know about /usr/lib too (which is sort of confusing).
Not only that, I doubt that this will work well unless you now have two site-packages dirs (one in /usr/lib for Python only extensions and one in /usr/lib64 for combined Python and binary extensions).
I believe you can get away without patching anything, if you do:
from distutils import sysconfig print sysconfig.get_python_lib()
to which you either pass plat_specific = 0 or 1. This will properly parse the right Makefile (which is probably what you ended up doing).
Uhm, this is the function that Jeremy was patching - the problem being that the "lib" part of the path is hard-coded into this function.
It doesn't parse the Makefile, BTW, and plat_specific only adds site-packages to the returned directory - it doesn't have anything to do with platform specific code.
Indeed it doesn't parse the Makefile. I believe I was remembering something else about Makefiles. Sorry. This is what we have in Fedora: if os.name == "posix": if plat_specific or standard_lib: lib = "lib64" else: lib = "lib" libpython = os.path.join(prefix, lib, "python" + get_python_version()) if standard_lib: return libpython else: return os.path.join(libpython, "site-packages") This is what stock python 2.4.1 does: libpython = os.path.join(prefix, "lib", "python" + get_python_version()) if standard_lib: return libpython else: return os.path.join(libpython, "site-packages")
The Makefile is the only place where Python itself can look up how it was configured. It's easy to find: distutils.sysconfig.get_makefile_filename(); distutils.sysconfig.parse_makefile(filename) then does the rest. Or in one go: distutils.sysconfig.get_config_vars().
The interesting parts of the Makefile are these variables:
# Detailed destination directories BINLIBDEST= $(LIBDIR)/python$(VERSION) LIBDEST= $(SCRIPTDIR)/python$(VERSION) INCLUDEPY= $(INCLUDEDIR)/python$(VERSION) CONFINCLUDEPY= $(CONFINCLUDEDIR)/python$(VERSION) LIBP= $(LIBDIR)/python$(VERSION)
I guess we should change the install.py to use these values instead of building its own little world of pathnames.
What do you think ?
This makes a lot of sense. Anything that can prevent the ugly patches for multilib. Misa
Jeremy Sanders wrote:
... Under the posix section, distutils assumes that Python is installed in /usr/lib/python-X.X, where it's really in /usr/lib64/python-X.X. This is clearly a distutils bug. Distutils should be looking under lib64 for 64-bit x86 systems.
All linux x86-64 distributions use lib64 instead of lib for 64 bit libraries.
Just as note: distutils should read this from the Makefile rather than trying to guess - whether or not lib64 is used depends on the distribution, the way the particular Python version was installed, etc. (e.g. a user installed Python version will still install to /usr/lib per default - even on 64-bit platforms). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 15 2005)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
On Fri, 15 Apr 2005, M.-A. Lemburg wrote:
Just as note: distutils should read this from the Makefile rather than trying to guess - whether or not lib64 is used depends on the distribution, the way the particular Python version was installed, etc. (e.g. a user installed Python version will still install to /usr/lib per default - even on 64-bit platforms).
Is this a packaging bug in fedora? Jeremy -- Jeremy Sanders <jeremy@jeremysanders.net> http://www.jeremysanders.net/ Cambridge, UK Public Key Server PGP Key ID: E1AAE053
Jeremy Sanders wrote:
On Fri, 15 Apr 2005, M.-A. Lemburg wrote:
Just as note: distutils should read this from the Makefile rather than trying to guess - whether or not lib64 is used depends on the distribution, the way the particular Python version was installed, etc. (e.g. a user installed Python version will still install to /usr/lib per default - even on 64-bit platforms).
Should have been "/usr/local/lib/python2.4".
Is this a packaging bug in fedora?
No, it's just the way most Linux distro's tweak the Python setup to support 64-bit along side with 32-bit ones (which then usually live in /usr/lib et al). You often have problems with this choice if you want to build things from source on x86-64. In most cases, you need to set LDFLAGS to -L/usr/lib64 or edit Makefiles. to adjust the compiler settings. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 15 2005)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
participants (3)
-
Jeremy Sanders -
M.-A. Lemburg -
Mihai Ibanescu