[Numpy-discussion] Debian/Ubuntu patch help (was: ANN: NumPy 1.6.2 release candidate 1)

Fri Sep 7 17:09:09 EDT 2012

On Fri, Aug 31, 2012 at 3:03 AM, Ondřej Čertík <ondrej.certik at gmail.com>wrote:

> On Tue, May 15, 2012 at 11:52 AM, Ralf Gommers
> <ralf.gommers at googlemail.com> wrote:
> >
> >
> > On Sat, May 12, 2012 at 9:17 PM, Ralf Gommers <
> ralf.gommers at googlemail.com>
> > wrote:
> >>
> >>
> >>
> >> On Sat, May 12, 2012 at 6:22 PM, Sandro Tosi <matrixhasu at gmail.com>
> wrote:
> >>>
> >>> Hello,
> >>>
> >>> On Sat, May 5, 2012 at 8:15 PM, Ralf Gommers
> >>> <ralf.gommers at googlemail.com> wrote:
> >>> > Hi,
> >>> >
> >>> > I'm pleased to announce the availability of the first release
> candidate
> >>> > of
> >>> > NumPy 1.6.2.  This is a maintenance release. Due to the delay of the
> >>> > NumPy
> >>> > 1.7.0, this release contains far more fixes than a regular NumPy
> bugfix
> >>> > release.  It also includes a number of documentation and build
> >>> > improvements.
> >>> >
> >>> > Sources and binary installers can be found at
> >>> > https://sourceforge.net/projects/numpy/files/NumPy/1.6.2rc1/
> >>> >
> >>> > Please test this release and report any issues on the
> numpy-discussion
> >>> > mailing list.
> >>> ...
> >>> > BLD:   add support for the new X11 directory structure on Ubuntu &
> co.
> >>>
> >>> We've just discovered that this fix is not enough. Actually the new
> >>> directories are due to the "multi-arch" feature of Debian systems,
> >>> that allows to install libraries from other (foreign) architectures
> >>> than the one the machine is (the classic example, i386 libraries on a
> >>> amd64 host).
> >>>
> >>> the fix included to look up in additional directories is currently
> >>> only for X11, while for example Debian has fftw3 that's
> >>> multi-arch-ified and thus will fail to be detected.
> >>>
> >>> Could this fix be extended to include all other things that are
> >>> checked? for reference the bug in Debian is [1]; there was also a
> >>> patch[2] in previous versions, that was using gcc to get the
> >>> multi-arch paths - you might use as a reference, or to implement
> >>> something debian-systems-specific.
> >>>
> >>> [1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=640940
> >>> [2]
> >>>
> http://anonscm.debian.org/viewvc/python-modules/packages/numpy/trunk/debian/patches/50_search-multiarch-paths.patch?view=markup&pathrev=21168
> >>>
> >>> It would be awesome is such support would end up in 1.6.2 .
> >>
> >>
> >> Hardcoding some more paths to check in distutils/system_info.py should
> be
> >> OK, also for 1.6.2 (will require a new RC).
> >>
> >> The --print-multiarch thing looks very questionable. As far as I can
> tell,
> >> it's a Debian specific gcc patch, only available in gcc 4.6 and up.
> Ubuntu
> >> before 11.10 release also doesn't have it. Therefore I don't think use
> of
> >> --print-multiarch is appropriate for numpy for now, and certainly not a
> >> change I'd like to make to distutils right before a release.
> >>
> >> If anyone with access to a Debian/Ubuntu system could come up with a
> patch
> >> which adds the right paths to system_info.py, that would be great.
> >
> >
> > Hi, if there's anyone wants to have a look at the above issue this week,
> > that would be great.
> >
> > If there's a patch by this weekend I can create a second RC, so we can
> still
> > have the final release before the end of this month (needed for Debian
> > freeze). Otherwise a second RC won't be needed.
>
> For NumPy 1.7.0, the issue is fixed for X11 by the following lines:
>
>     if os.path.exists('/usr/lib/X11'):
>         globbed_x11_dir = glob('/usr/lib/*/libX11.so')
>         if globbed_x11_dir:
>             x11_so_dir = os.path.split(globbed_x11_dir[0])[0]
>             default_x11_lib_dirs.extend([x11_so_dir, '/usr/lib/X11'])
>             default_x11_include_dirs.extend(['/usr/lib/X11/include',
>                                              '/usr/include/X11'])
>
>
> in numpy/distutils/system_info.py, there is still an issue of
> supporting Debian multi-arch fully:
>
> http://projects.scipy.org/numpy/ticket/2150
>
> However, I don't understand what exactly it means. Ralf, would would
> be a canonical example to fix?
> If I use for example x11, I get:
>
>
> In [1]: from numpy.distutils.system_info import get_info
>
> In [2]: get_info("x11", 2)
>
> /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551:
> UserWarning: Specified path /usr/X11R6/lib64 is invalid.
>   warnings.warn('Specified path %s is invalid.' % d)
>
> /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551:
> UserWarning: Specified path /usr/X11R6/lib is invalid.
>   warnings.warn('Specified path %s is invalid.' % d)
>
> /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551:
> UserWarning: Specified path /usr/X11/lib64 is invalid.
>   warnings.warn('Specified path %s is invalid.' % d)
>
> /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551:
> UserWarning: Specified path /usr/X11/lib is invalid.
>   warnings.warn('Specified path %s is invalid.' % d)
>
> /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551:
> UserWarning: Specified path /usr/lib64 is invalid.
>   warnings.warn('Specified path %s is invalid.' % d)
>
> /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551:
> UserWarning: Specified path /usr/X11R6/include is invalid.
>   warnings.warn('Specified path %s is invalid.' % d)
>
> /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551:
> UserWarning: Specified path /usr/X11/include is invalid.
>   warnings.warn('Specified path %s is invalid.' % d)
>
> /home/ondrej/repos/numpy/py27/lib/python2.7/site-packages/numpy/distutils/system_info.py:551:
> UserWarning: Specified path /usr/lib/X11/include is invalid.
>   warnings.warn('Specified path %s is invalid.' % d)
> Out[2]:
> {'include_dirs': ['/usr/include'],
>  'libraries': ['X11'],
>  'library_dirs': ['/usr/lib/x86_64-linux-gnu']}
>
>
>
> I am using Ubuntu 12.04. Is the task to remove the warnings, or is the
> task to fix it for some other package from the get_info() list (which
> one)?
>

The idea is to review and apply the patch linked to in this thread, in
order for numpy builds to still work when other libs than X11 are
multi-arched in Debian (FFTW was mentioned as an example by Julian). Here's
a direct link to the patch again:
http://anonscm.debian.org/viewvc/python-modules/packages/numpy/trunk/debian/patches/50_search-multiarch-paths.patch?view=markup&pathrev=21168

It looks to me like showing the warning is inappropriate and there should
be a second dash in "-print-multi-arch" (disclaimer:  didn't test the
patch), but for the rest it's good to go.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120907/29101a1c/attachment.html>