[Numpy-discussion] Problem with numpy and distutils on OS X

Fri May 9 15:32:00 EDT 2008

Thanks, Robert. Indeed, numpy 1.0.4 does some "monkeypatching" (see the transcript
below). Interestingly, 1.0.3 did not, so I'm hoping that 1.0.5 may not also.
(I'd rather stay with released version, since I distribute my code to colleagues,
and cannot impose too many conditions on them.)

In the meantime I'll get
the numpy include directory information some other way in my setup.py. I will
also investigate if gcc could be smarter about enabling vectorization (perhaps
in a version newer than 4.0.1?), and let you know.

Cheers,

Michele

---

Python 2.5.1 (r251:54869, Apr 18 2007, 22:08:04) 
[GCC 4.0.1 (Apple
Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits"
or "license" for more information.
>>> import sys
>>> old = set(sys.modules)

>>> import numpy
>>> numpy.get_include()
'/Users/vallis/lib/python2.5/site-packages/numpy-1.0.4-py2.5-macosx-10.3-fat.egg/numpy/core/include'

>>> new = set(sys.modules)
>>> [x for x in (new - old) if 'distutils' in
x]
['numpy.distutils.misc_util', 'distutils', 'numpy.distutils.re', 'numpy.distutils.numpy',
'distutils.dep_util', 'numpy.distutils.ccompiler', 'distutils.types', 'numpy.distutils.exec_command',
'numpy.distutils.tempfile', 'distutils.re', 'numpy.distutils.info', 'numpy.distutils.distutils',
'distutils.log', 'distutils.copy', 'distutils.version', 'numpy.distutils',
'distutils.sysconfig', 'numpy.distutils.curses', 'numpy.distutils.copy', 'numpy.distutils.sys',
'numpy.distutils.__version__', 'distutils.os', 'distutils.ccompiler', 'numpy.distutils.string',
'distutils.spawn', 'distutils.sys', 'distutils.dir_util', 'numpy.distutils.unixccompiler',
'distutils.util', 'distutils.string', 'numpy.distutils.new', 'numpy.distutils.imp',
'numpy.distutils.__config__', 'distutils.distutils', 'distutils.unixccompiler',
'numpy.distutils.log', 'numpy.distutils.os', 'numpy.distutils.glob', 'numpy.distutils.atexit',
'distutils.errors', 'distutils.file_util']

--- Discussion of Numerical
Python <numpy-discussion at scipy.org wrote:
On Fri, May 9, 2008 at 11:17 AM,
Michele Vallisneri
> <vallis.35530053 at bloglines.com> wrote:
> > I'm writing
a standard distutils setup.py to compile a Python C
> > extension on OS X
10.4, and I need to specify a few special compiler
> > options to enable
vector CPU extension (altivec and SSE on i686 and
> > PPC respectively).
This compromises the generation of universal
> > binaries, because these
options are CPU-specific, and cannot be
> > passed to gcc together with "-arch
ppc -arch i386".
> >
> > I'm happy with generating a nonuniversal extension,
which I can do
> > (at least with distutils 2.4.4) by specifying an extra
"-arch i386"
> > compiler option. Distutils has some Darwin-specific code
that catches
> > that, and takes out the "-arch ppc" option that would be
there by
> > default.
> >
> > All well, but then in my setup.py I also
import numpy (to find out
> > where its C includes are). If I do that, the
behavior of distutils
> > changes, and the "-arch ppc" option is not taken
out, so compilation
> > fails again.
> >
> > So I have questions:
> >

> > - Is this an acceptable behavior for numpy to have?
> 
> No. We've
seen this before, and I thought we fixed it, but perhaps
> not. numpy.distutils
does monkeypatch distutils, but you shouldn't get
> that unless if you import
numpy.distutils.
> 
> > Should it modify
> > other modules? I understand
that there is a numpy.distutils, but
> > shouldn't I have a choice to use
it explicitly, and get the old
> > distutils behavior by using that namespace?

> 
> Yes, you should be able to avoid this by avoiding importing
> numpy.distutils.
However, I don't see why you are getting this. Just
> importing numpy and
calling numpy.get_include() does not bring in
> numpy.distutils, at least
not with the SVN version of numpy (the 1.1.x
> branch rather than the 1.2.x
trunk).
> 
> >>> import sys
> >>> old = set(sys.modules)
> >>> import
numpy
> >>> numpy.get_include()
> '/Users/rkern/svn/numpy/numpy/core/include'

> >>> new = set(sys.modules)
> >>> [x for x in (new - old) if 'distutils'
in x]
> []
> >>>
> 
> You might try something similar at the top of your
setup.py script to
> see if 1.0.4 does something different.
> 
> > - Is
there a way to avoid or disable this interference?
> 
> Possibly upgrade
to the 1.1.x branch. Check it out from here:
> 
>   http://svn.scipy.org/svn/numpy/branches/1.1.x

> 
> > - Finally, is there a way to compile universal binaries with CPU-

> > specific options?
> 
> Not to my knowledge, no. There might be a gcc
option to use in
> extra_compile_args, but you will have to check the man
page for it. If
> you find one, please post it, since I am interested in
having such a
> capability myself.
> 
> -- 
> Robert Kern
> 
> "I have
come to believe that the whole world is an enigma, a harmless
> enigma that
is made terrible by our own mad attempt to interpret it as
> though it had
an underlying truth."
>  -- Umberto Eco
> _______________________________________________

> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion

> 