[Numpy-discussion] ANN: Numpy 1.6.1 release candidate 1

[Bruce Southey]

On 06/19/2011 05:21 AM, Ralf Gommers wrote:
>
>
>  On Tue, Jun 14, 2011 at 5:28 AM, Bruce Southey <bsouthey at gmail.com
>  <mailto:bsouthey at gmail.com>> wrote:
>
>  On Mon, Jun 13, 2011 at 8:31 PM, Pauli Virtanen <pav at iki.fi
>  <mailto:pav at iki.fi>> wrote:
> > On Mon, 13 Jun 2011 11:08:18 -0500, Bruce Southey wrote: [clip]
> >> OSError:
> >> /usr/local/lib/python3.2/site-packages/numpy/core/multiarray.pyd:
> >> cannot open shared object file: No such file or directory
> >
> > I think that's a result of Python 3.2 changing the extension
> > module file naming scheme (IIRC to a versioned one).
> >
> > The '.pyd' ending and its Unix counterpart are IIRC hardcoded to
> > the failing test, or some support code in Numpy. Clearly, they
> > should instead ask Python how it names the extensions modules. This
> > information may be available somewhere in the `sys` module.
> >
> > _______________________________________________ NumPy-Discussion
> > mailing list NumPy-Discussion at scipy.org
> > <mailto:NumPy-Discussion at scipy.org>
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
>
>  Pauli, I am impressed yet again! It really saved a lot of time to
>  understand the reason.
>
>  A quick comparison between Python versions: Python2.7: multiarray.so
>  Python3.2: multiarray.cpython-32m.so
>  <http://multiarray.cpython-32m.so>
>
>  Search for the extension leads to PEP 3149
>  http://www.python.org/dev/peps/pep-3149/
>
>
>  That looked familiar: http://projects.scipy.org/numpy/ticket/1749
>  https://github.com/numpy/numpy/commit/ee0831a8
>
>
>  This is POSIX only which excludes windows. I tracked it down to the
>  list 'libname_ext' defined in file "numpy/ctypeslib.py" line 100:
>  libname_ext = ['%s.so' % libname, '%s.pyd' % libname]
>
>  On my 64-bit Linux system, I get the following results: $ python2.4
>  -c "from distutils import sysconfig; print
>  sysconfig.get_config_var('SO')" .so $ python2.5 -c "from distutils
>  import sysconfig; print sysconfig.get_config_var('SO')" .so $
>  python2.7 -c "from distutils import sysconfig; print
>  sysconfig.get_config_var('SO')" .so $ python3.1 -c "from distutils
>  import sysconfig; print(sysconfig.get_config_var('SO'))" .so $
>  python3.2 -c "from distutils import sysconfig;
>  print(sysconfig.get_config_var('SO'))" .cpython-32m.so
>
>  My Windows 32-bit Python2.6 install:
> >>> from distutils import sysconfig sysconfig.get_config_var('SO')
>  '.pyd'
>
>  Making the bug assumption that other OSes including 32-bit and
>  64-bit versions also provide the correct suffix, this suggest adding
>  the import and modification as:
>
>  import from distutils import sysconfig libname_ext = ['%s%s' %
>  (libname, sysconfig.get_config_var('SO'))]
>
>  Actually if that works for Mac, Sun etc. then 'load_library'
>  function could be smaller.
>
>
>  The issue is that libraries built as Python extensions have the extra
>  ".cpython-mu32", but other libraries on your system don't. And
>  sysconfig.get_config_var('SO') is used for both. I don't see another
>  config var that allows to distinguish between the two. Unless on
>  platforms where it matters there are separate return values in
>  sysconfig.get_config_vars('SO'). What do you get for that on Linux?
>
>  The logic of figuring out the right extension string should not be
>  duplicated, distutils.misc_util seems like a good place for it. Can
>  you test if this works for you:
>  https://github.com/rgommers/numpy/tree/sharedlib-ext
>
>
>  Finally the test_basic2 in "tests/test_ctypeslib.py" needs to be
>  changed as well because it is no longer correct. Just commenting out
>  the 'fix' appears to work.
>
>  It works in the case of trying to load multiarray, but the function
>  under test would still be broken.
>
>  Ralf
>
>
>  _______________________________________________ NumPy-Discussion
>  mailing list NumPy-Discussion at scipy.org
>  http://mail.scipy.org/mailman/listinfo/numpy-discussion
I copied the files but that just moves the problem. So that patch is 
incorrect.

I get the same errors on Fedora 15 supplied Python3.2 for numpy 1.6.0 
and using git from 'https://github.com/rgommers/numpy.git'.  Numpy is 
getting Fedora supplied Atlas (1.5.1 does not).

It appears that there is a misunderstanding of the PEP because 'SO' and 
'SOABI' do exactly what the PEP says on my systems:
> >> from distutils import sysconfig sysconfig.get_config_var('SO')
'.cpython-32m.so'
> >> sysconfig.get_config_var('SOABI')
'cpython-32m'

Consequently, the name, 'multiarray.pyd', created within numpy is invalid.

Looking the code, I see this line which makes no sense given that the 
second part is true under Linux:

if (not is_python_ext) and 'SOABI' in distutils.sysconfig.get_config_vars():

So I think the 'get_shared_lib_extension' function is wrong and probably 
unneeded.


Bruce


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110620/add3baa5/attachment.html>