Debian/Ubuntu patch help (was: ANN: NumPy 1.6.2 release candidate 1)
On Sat, May 12, 2012 at 9:17 PM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
On Sat, May 12, 2012 at 6:22 PM, Sandro Tosi <matrixhasu@gmail.com> wrote:
Hello,
On Sat, May 5, 2012 at 8:15 PM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
Hi,
I'm pleased to announce the availability of the first release candidate of NumPy 1.6.2. This is a maintenance release. Due to the delay of the NumPy 1.7.0, this release contains far more fixes than a regular NumPy bugfix release. It also includes a number of documentation and build improvements.
Sources and binary installers can be found at https://sourceforge.net/projects/numpy/files/NumPy/1.6.2rc1/
Please test this release and report any issues on the numpy-discussion mailing list. ... BLD: add support for the new X11 directory structure on Ubuntu & co.
We've just discovered that this fix is not enough. Actually the new directories are due to the "multi-arch" feature of Debian systems, that allows to install libraries from other (foreign) architectures than the one the machine is (the classic example, i386 libraries on a amd64 host).
the fix included to look up in additional directories is currently only for X11, while for example Debian has fftw3 that's multi-arch-ified and thus will fail to be detected.
Could this fix be extended to include all other things that are checked? for reference the bug in Debian is [1]; there was also a patch[2] in previous versions, that was using gcc to get the multi-arch paths - you might use as a reference, or to implement something debian-systems-specific.
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=640940 [2] http://anonscm.debian.org/viewvc/python-modules/packages/numpy/trunk/debian/patches/50_search-multiarch-paths.patch?view=markup&pathrev=21168
It would be awesome is such support would end up in 1.6.2 .
Hardcoding some more paths to check in distutils/system_info.py should be OK, also for 1.6.2 (will require a new RC).
The --print-multiarch thing looks very questionable. As far as I can tell, it's a Debian specific gcc patch, only available in gcc 4.6 and up. Ubuntu before 11.10 release also doesn't have it. Therefore I don't think use of --print-multiarch is appropriate for numpy for now, and certainly not a change I'd like to make to distutils right before a release.
If anyone with access to a Debian/Ubuntu system could come up with a patch which adds the right paths to system_info.py, that would be great.
Hi, if there's anyone wants to have a look at the above issue this week, that would be great. If there's a patch by this weekend I can create a second RC, so we can still have the final release before the end of this month (needed for Debian freeze). Otherwise a second RC won't be needed. Ralf
On Tue, May 15, 2012 at 11:52 AM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Sat, May 12, 2012 at 9:17 PM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Sat, May 12, 2012 at 6:22 PM, Sandro Tosi <matrixhasu@gmail.com> wrote:
Hello,
On Sat, May 5, 2012 at 8:15 PM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
Hi,
I'm pleased to announce the availability of the first release candidate of NumPy 1.6.2. This is a maintenance release. Due to the delay of the NumPy 1.7.0, this release contains far more fixes than a regular NumPy bugfix release. It also includes a number of documentation and build improvements.
Sources and binary installers can be found at https://sourceforge.net/projects/numpy/files/NumPy/1.6.2rc1/
Please test this release and report any issues on the numpy-discussion mailing list. ... BLD: add support for the new X11 directory structure on Ubuntu & co.
We've just discovered that this fix is not enough. Actually the new directories are due to the "multi-arch" feature of Debian systems, that allows to install libraries from other (foreign) architectures than the one the machine is (the classic example, i386 libraries on a amd64 host).
the fix included to look up in additional directories is currently only for X11, while for example Debian has fftw3 that's multi-arch-ified and thus will fail to be detected.
Could this fix be extended to include all other things that are checked? for reference the bug in Debian is [1]; there was also a patch[2] in previous versions, that was using gcc to get the multi-arch paths - you might use as a reference, or to implement something debian-systems-specific.
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=640940 [2] http://anonscm.debian.org/viewvc/python-modules/packages/numpy/trunk/debian/patches/50_search-multiarch-paths.patch?view=markup&pathrev=21168
It would be awesome is such support would end up in 1.6.2 .
Hardcoding some more paths to check in distutils/system_info.py should be OK, also for 1.6.2 (will require a new RC).
The --print-multiarch thing looks very questionable. As far as I can tell, it's a Debian specific gcc patch, only available in gcc 4.6 and up. Ubuntu before 11.10 release also doesn't have it. Therefore I don't think use of --print-multiarch is appropriate for numpy for now, and certainly not a change I'd like to make to distutils right before a release.
If anyone with access to a Debian/Ubuntu system could come up with a patch which adds the right paths to system_info.py, that would be great.
Hi, if there's anyone wants to have a look at the above issue this week, that would be great.
If there's a patch by this weekend I can create a second RC, so we can still have the final release before the end of this month (needed for Debian freeze). Otherwise a second RC won't be needed.
For NumPy 1.7.0, the issue is fixed for X11 by the following lines: if os.path.exists('/usr/lib/X11'): globbed_x11_dir = glob('/usr/lib/*/libX11.so') if globbed_x11_dir: x11_so_dir = os.path.split(globbed_x11_dir[0])[0] default_x11_lib_dirs.extend([x11_so_dir, '/usr/lib/X11']) default_x11_include_dirs.extend(['/usr/lib/X11/include', '/usr/include/X11']) in numpy/distutils/system_info.py, there is still an issue of supporting Debian multi-arch fully: http://projects.scipy.org/numpy/ticket/2150 However, I don't understand what exactly it means. Ralf, would would be a canonical example to fix? If I use for example x11, I get: In [1]: from numpy.distutils.system_info import get_info In [2]: get_info("x11", 2) /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11R6/lib64 is invalid. warnings.warn('Specified path %s is invalid.' % d) /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11R6/lib is invalid. warnings.warn('Specified path %s is invalid.' % d) /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11/lib64 is invalid. warnings.warn('Specified path %s is invalid.' % d) /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11/lib is invalid. warnings.warn('Specified path %s is invalid.' % d) /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/lib64 is invalid. warnings.warn('Specified path %s is invalid.' % d) /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11R6/include is invalid. warnings.warn('Specified path %s is invalid.' % d) /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11/include is invalid. warnings.warn('Specified path %s is invalid.' % d) /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/lib/X11/include is invalid. warnings.warn('Specified path %s is invalid.' % d) Out[2]: {'include_dirs': ['/usr/include'], 'libraries': ['X11'], 'library_dirs': ['/usr/lib/x86_64-linux-gnu']} I am using Ubuntu 12.04. Is the task to remove the warnings, or is the task to fix it for some other package from the get_info() list (which one)? Thanks, Ondrej
On Fri, Aug 31, 2012 at 3:03 AM, Ondřej Čertík <ondrej.certik@gmail.com>wrote:
On Tue, May 15, 2012 at 11:52 AM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Sat, May 12, 2012 at 9:17 PM, Ralf Gommers <
wrote:
On Sat, May 12, 2012 at 6:22 PM, Sandro Tosi <matrixhasu@gmail.com>
wrote:
Hello,
On Sat, May 5, 2012 at 8:15 PM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
Hi,
I'm pleased to announce the availability of the first release
candidate
of NumPy 1.6.2. This is a maintenance release. Due to the delay of the NumPy 1.7.0, this release contains far more fixes than a regular NumPy bugfix release. It also includes a number of documentation and build improvements.
Sources and binary installers can be found at https://sourceforge.net/projects/numpy/files/NumPy/1.6.2rc1/
Please test this release and report any issues on the numpy-discussion mailing list. ... BLD: add support for the new X11 directory structure on Ubuntu & co.
We've just discovered that this fix is not enough. Actually the new directories are due to the "multi-arch" feature of Debian systems, that allows to install libraries from other (foreign) architectures than the one the machine is (the classic example, i386 libraries on a amd64 host).
the fix included to look up in additional directories is currently only for X11, while for example Debian has fftw3 that's multi-arch-ified and thus will fail to be detected.
Could this fix be extended to include all other things that are checked? for reference the bug in Debian is [1]; there was also a patch[2] in previous versions, that was using gcc to get the multi-arch paths - you might use as a reference, or to implement something debian-systems-specific.
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=640940 [2]
It would be awesome is such support would end up in 1.6.2 .
Hardcoding some more paths to check in distutils/system_info.py should be OK, also for 1.6.2 (will require a new RC).
The --print-multiarch thing looks very questionable. As far as I can tell, it's a Debian specific gcc patch, only available in gcc 4.6 and up. Ubuntu before 11.10 release also doesn't have it. Therefore I don't think use of --print-multiarch is appropriate for numpy for now, and certainly not a change I'd like to make to distutils right before a release.
If anyone with access to a Debian/Ubuntu system could come up with a
ralf.gommers@googlemail.com> patch
which adds the right paths to system_info.py, that would be great.
Hi, if there's anyone wants to have a look at the above issue this week, that would be great.
If there's a patch by this weekend I can create a second RC, so we can still have the final release before the end of this month (needed for Debian freeze). Otherwise a second RC won't be needed.
For NumPy 1.7.0, the issue is fixed for X11 by the following lines:
if os.path.exists('/usr/lib/X11'): globbed_x11_dir = glob('/usr/lib/*/libX11.so') if globbed_x11_dir: x11_so_dir = os.path.split(globbed_x11_dir[0])[0] default_x11_lib_dirs.extend([x11_so_dir, '/usr/lib/X11']) default_x11_include_dirs.extend(['/usr/lib/X11/include', '/usr/include/X11'])
in numpy/distutils/system_info.py, there is still an issue of supporting Debian multi-arch fully:
http://projects.scipy.org/numpy/ticket/2150
However, I don't understand what exactly it means. Ralf, would would be a canonical example to fix? If I use for example x11, I get:
In [1]: from numpy.distutils.system_info import get_info
In [2]: get_info("x11", 2)
/home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11R6/lib64 is invalid. warnings.warn('Specified path %s is invalid.' % d)
/home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11R6/lib is invalid. warnings.warn('Specified path %s is invalid.' % d)
/home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11/lib64 is invalid. warnings.warn('Specified path %s is invalid.' % d)
/home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11/lib is invalid. warnings.warn('Specified path %s is invalid.' % d)
/home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/lib64 is invalid. warnings.warn('Specified path %s is invalid.' % d)
/home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11R6/include is invalid. warnings.warn('Specified path %s is invalid.' % d)
/home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/X11/include is invalid. warnings.warn('Specified path %s is invalid.' % d)
/home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551: UserWarning: Specified path /usr/lib/X11/include is invalid. warnings.warn('Specified path %s is invalid.' % d) Out[2]: {'include_dirs': ['/usr/include'], 'libraries': ['X11'], 'library_dirs': ['/usr/lib/x86_64-linux-gnu']}
I am using Ubuntu 12.04. Is the task to remove the warnings, or is the task to fix it for some other package from the get_info() list (which one)?
The idea is to review and apply the patch linked to in this thread, in order for numpy builds to still work when other libs than X11 are multi-arched in Debian (FFTW was mentioned as an example by Julian). Here's a direct link to the patch again: http://anonscm.debian.org/viewvc/python-modules/packages/numpy/trunk/debian/patches/50_search-multiarch-paths.patch?view=markup&pathrev=21168 It looks to me like showing the warning is inappropriate and there should be a second dash in "-print-multi-arch" (disclaimer: didn't test the patch), but for the rest it's good to go. Ralf
participants (3)
-
Ondřej Čertík -
Ralf Gommers -
Ralf Gommers