[Numpy-discussion] Bug with how numpy.distutils.system_info handles the site.cfg

[Chris Colbert]

On Wed, May 12, 2010 at 11:06 PM, Chris Colbert <sccolbert at gmail.com> wrote:

> I had this problem back in 2009 when building Enthought Enable, and was
> happy with a work around. It just bit me again, and I finally got around to
> drilling down to the problem.
>
> On linux, if one uses the numpy/site.cfg [default] section when building
> from source to specifiy local library directories, the x11 libs won't be
> found by NumPy.
>
> The relevant section of the site.cfg.example reads as follows:
>
> # Defaults
> # ========
> # The settings given here will apply to all other sections if not
> overridden.
> # This is a good place to add general library and include directories like
> # /usr/local/{lib,include}
> #
> #[DEFAULT]
> #library_dirs = /usr/local/lib
> #include_dirs = /usr/local/include
>
> Now, I build NumPy with Atlas and my Atlas libs are installed in
> /usr/local, so my [default] section of site.cfg looks like this (as
> suggested by the site.cfg.example):
>
> # Defaults
> # ========
> # The settings given here will apply to all other sections if not
> overridden.
> # This is a good place to add general library and include directories like
> # /usr/local/{lib,include}
> #
> [DEFAULT]
> library_dirs = /usr/local/lib:/usr/local/lib/atlas
> include_dirs = /usr/local/include
>
>
> NumPy builds and works fine with this. The problem occurs when other
> libraries use numpy.distutils.system_info.get_info('x11') (ala Enthought
> Enable). That function eventually calls
> numpy.distutils.system_info.system_info.parse_config_files which has the
> following definition:
>
> def parse_config_files(self):
>         self.cp.read(self.files)
>         if not self.cp.has_section(self.section):
>             if self.section is not None:
>                 self.cp.add_section(self.section)
>
> When self.cp is instantiated (when looking for the x11 libs), it is
> provided the following defaults:
>
> {'libraries': '', 'src_dirs': '.:/usr/local/src', 'search_static_first':
> '0', 'library_dirs':
> '/usr/X11R6/lib64:/usr/X11R6/lib:/usr/X11/lib64:/usr/X11/lib:/usr/lib64:/usr/lib',
> 'include_dirs': '/usr/X11R6/include:/usr/X11/include:/usr/include'}
>
> As is clearly seen, the 'library_dirs' contains the proper paths to find
> the x11 libs. But since the config file has [default] section, these paths
> get trampled and replaced with whatever is contained in the site.cfg
> [default] section. In my case, this is /usr/local/lib:/usr/local/lib/atlas.
> Thus, my x11 libs aren't found and the Enable build fails.
>
> The workaround is to include an [x11] section in site.cfg with the
> appropriate paths, but I don't really feel this should be necessary. Would
> the better behavior be to look for a [default] section in the config file in
> the parse_config_files method and add those paths to the already specified
> defaults?
>
>
Then again, another workaround could be to add the atlas directory paths to
the [blas_opt] and [lapack_opt] sections. This would work for my case, but
it doesn't solve the larger problem of any directories put in [default]
trouncing any of the other standard dirs that would otherwise be used.


> Changing the site.cfg [default] section to read as follows:
>
> [DEFAULT]
> library_dirs = /usr/lib:/usr/local/lib:/usr/local/lib/atlas
> include_dirs = /usr/include:/usr/local/include
>
> is not an option because then NumPy will find and use the system atlas,
> which in my case is not threaded nor optimized for my machine.
>
> If you want me to patch the parse_config_files method, just let me know.
>
> Cheers,
>
> Chris
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100512/577fea97/attachment.html>