Mailman 3 June 2007 - NumPy-Discussion

Problems with Numeric
by Ali Santacruz 17 Jun '07

17 Jun '07

Hi dear list, I am designing an application that uses GDAL (which seems to bind to Numeric) to read an image and convert it to an array. The code runs without problem in the console, then I compile it (apparently without problems), but when I try to launch the application, it fails and the next error appears in the .log: Traceback (most recent call last): File "ViewGdal_0.0-1.py", line 10, in ? File "gdal\__init__.pyc", line 11, in ? File "gdal\gdalnumeric.pyc", line 85, in ? #from Numeric import * File "Numeric.pyc", line 93, in ? #from Precision import * File "Precision.pyc", line 26, in ? #_code_table = _fill_table(typecodes) File "Precision.pyc", line 23, in _fill_table #table[key] = _get_precisions(value) File "Precision.pyc", line 18, in _get_precisions TypeError: data type not understood This refers to the next in Precision.py in Numeric: typecodes = {'Character':'c', 'Integer':'1sil', 'UnsignedInteger':'bwu', 'Float':'fd', 'Complex':'FD'} def _get_precisions(typecodes): lst = [] for t in typecodes: lst.append( (zeros( (1,), t ).itemsize()*8, t) ) #this is line 18 return lst As far as I understand (I’ve been working with Python for just a couple of months), there is a problem with the typecodes. What can I do to solve this problem? I’ve not found a version of GDAL that binds to Numpy. I am using GDAL from hobutools (v. 1.75), Numeric v. 24.2, Python 2.4.3 on WinXP. Any help is _really_ appreciated. Regards, Ali S. __________________________________________________ Correo Yahoo! Espacio para todos tus mensajes, antivirus y antispam ¡gratis! Regístrate ya - http://correo.espanol.yahoo.com/

2 1

Specifying compiler command line options for numpy.disutils.core
by Matthieu Brucher 16 Jun '07

16 Jun '07

Hi, I've been trying to use the Intel C Compiler for some extensions, and as a matter of fact, only numpy.distutils seems to support it... (hope that the next version of setuptools will...) Is it possible to change the compiler command line options in the commandline or in a .cfg file ? For the moment, I have only -shared, I'd like to add -xP for instance. This seems to be related to rex's last mail, but I did not find anywhere a way to specify these options in (numpy.)distutils or setuptools, even though it is available for every other non-Python "library builder". Matthieu

4 7

masked arrays and record arrays
by John Hunter 15 Jun '07

15 Jun '07

Do record arrays support masks? JDH

4 6

attribute names on record arrays
by John Hunter 15 Jun '07

15 Jun '07

I fund myself using record arrays more and more, and feature missing is the ability to do tab completion on attribute names in ipython, presumably because you are using a dict under the hood and __getattr__ to resolve o.key where o is a record array and key is a field name. How hard would it be to populate __dict__ with the attribute names so we could tab complete on them? Thanks, JDH

3 2

randint for long type (permutations)
by Will Woods 15 Jun '07

15 Jun '07

I want to choose a subset of all possible permutations of a sequence of length N, with each element of the subset unique. This is then going to be scattered across multiple machines using mpi. Since there is a one-to-one mapping between the integers in the range 0 <= x < N! and the possible permutations, one solution would be to choose M < N! integers randomly, check for uniqueness, and then scatter only the integers so that individual nodes can construct the permutations. However the integers need to be of type long, and randint doesn't work for numbers which cannot be converted to int. Any suggestions?

6 8

problem setting O type for datetime field in record arrays
by John Hunter 15 Jun '07

15 Jun '07

I can create a record array with datetime types using fromrecords if I don't specify a dtype and let numpy determine the dtype. But when I try and set the dtype at record array creation time, I get the error below In [17]: import numpy In [18]: import datetime In [19]: dt = datetime.datetime In [20]: dt1 = dt.now() In [21]: dt2 = dt.now() In [22]: numpy.core.rec.fromrecords([[dt1, 0.5], [dt2, 1.0]], names='date,height') Out[22]: recarray([(datetime.datetime(2007, 6, 14, 13, 59, 20, 753356), 0.5), (datetime.datetime(2007, 6, 14, 13, 59, 22, 946850), 1.0)], dtype=[('date', '|O4'), ('height', '<f8')]) In [23]: numpy.core.rec.fromrecords([[dt1, 0.5], [dt2, 1.0]], dtype=[('date', '|O4'), ('height', '<f8')]) ------------------------------------------------------------ Traceback (most recent call last): File "<ipython console>", line 1, in ? File "/home/titan/johnh/dev/lib/python2.4/site-packages/numpy/core/records.py", line 399, in fromrecords retval = sb.array(recList, dtype = descr) ValueError: tried to set void-array with object members using buffer. In [24]: numpy.__version__ Out[24]: '1.0.4.dev3868'

1 0

numpy with python < 2.3?
by Joël Schaerer 15 Jun '07

15 Jun '07

Hi there, I'm trying to install numpy on a system I don't control, which has python 2.2.3. When I run python setup.py build, I get: python setup.py build Running from numpy source directory. Traceback (most recent call last): File "setup.py", line 91, in ? setup_package() File "setup.py", line 61, in setup_package from numpy.distutils.core import setup File "numpy/distutils/__init__.py", line 5, in ? import ccompiler File "numpy/distutils/ccompiler.py", line 11, in ? from numpy.distutils import log File "numpy/distutils/log.py", line 4, in ? from distutils.log import * ImportError: No module named log I checked in the documentation and found out that the log module was first introduced in python 2.3. Does that mean numpy is incompatible with python < 2.3? Is there a workaround? joel PS: please CC me in the answer as I have not subscribed (yet!) to the list

3 4

Incompatability of svn 3868 distutils with v10.0 Intel compilers and MKL9.1
by rex 14 Jun '07

14 Jun '07

I recently built NumPy & SciPy from svn sources using Intel's icc 10.0, ifort 10.0, and mkl 9.1 under openSUSE 10.2. In the process, I discovered and worked around four bugs (they are just quick hacks, not proper fixes; for example, some of them break compatibilty with earlier versions of Intel compilers and MKL) in NumPy svn 3868 distutils. Bug#1: distutils/system_info.py contains the lines: else: lapack_libs = self.get_libs('lapack_libs',['mkl_lapack32','mkl_lapack64']) This will fail when MKL 9.1 is used because Intel changed the names of both 'mkl_lapack32' and 'mkl_lapack64' to 'mkl_lapack'. My work-around was to change the line to: lapack_libs = self.get_libs('lapack_libs',['mkl_lapack']) Bug#2: distutils/fcompiler/intel.py has several lines like this: 'version_cmd' : ['<F77>', None], A work-around posted by George Nurser is to change all such lines to: (may break on Visual compilers) 'version_cmd' : ['<F77>', '-V'], Bug#3: distutils/ccompiler.py has the line: compiler_class['intel'] = ('intelccompiler','IntelCCompiler', "Intel C Compiler for 32-bit applications") Unfortunately, in 10.0 Intel changed the result of icc -V to: Intel(R) C Compiler for applications running on IA-32, Version 10.0 Build 20070426 Package ID: l_cc_p_10.0.023 Copyright (C) 1985-2007 Intel Corporation. All rights reserved. FOR NON-COMMERCIAL USE ONLY My work-around was to change the above to: compiler_class['intel'] = ('intelccompiler','IntelCCompiler', "Intel(R) C Compiler for applications running on IA-32") Bug#4: distutils/fcompiler/intel.py has the same id string problem that ccompiler.py had. ifort -V returns: Intel(R) Fortran Compiler for applications running on IA-32, Version 10.0 Build 20070426 Package ID: l_fc_p_10.0.023 I changed version_match = intel_version_match('32-bit') to version_match = intel_version_match('IA-32') #5 This is not a bug, but it's important to change distutils/intelccompiler.py from cc_exe = 'icc' to match the flags for your processor. For my Core 2 Duo, the line below works: cc_exe = 'icc -g -fomit-frame-pointer -xT -fast' After these changes, NumPy and SciPy built successfully, but scipy.test() returns a number of errors. The error reports are on the SciPy-dev list at the end of the thread: "Compiling scipy with Intel ifort & MKL" Perhaps this will help someone... -rex

4 5

f2py problem under MacOsX: "g95: unrecognized option '-shared'"
by David Tremouilles 13 Jun '07

13 Jun '07

Hello, I'm running into trouble with f2py (numpy 1.0.3 on my intel MacOsX) f2py or actually f95 is complaining that -share option is not recognized: "g95: unrecognized option '-shared'" Which is actually true on an OsX platform. It should be something like -dynamic or so that should be used (but I'm not skilled enough to trouble shot this myself :-( )... Does somebody know how to solve the problem? David

2 2

Overview of extra build options in svn's numpy.distutils
by David M. Cooke 13 Jun '07

13 Jun '07

Hi, Looks like I broke more stuff than I intended when I merged my distutils-revamp branch. Not that I didn't expect something to break, this stuff is fragile! The main purpose of the merge was to allow the user to configure more stuff regarding how things are compiled with Fortran. For instance, here's my ~/.pydistutils.cfg [config_fc] fcompiler=gnu95 f77exec=gfortran-mp-4.2 f90exec=gfortran-mp-4.2 #opt = -g -Wall -O2 f77flags=-g -Wall -O (I use gfortran 4.2 on my MacBook, installed using MacPorts.) Other options to set are listed in numpy/distutils/fcompiler/__init__.py, in the FCompiler class. distutils config_fc key, [environment variable] distutils flags for config_fc section compiler - Fortran compiler to use (the numpy.distutils name, like gnu95 or intel) noopt - don't compile with optimisations noarch - don't compile with host architecture optimisations debug - compile with debug optimisations verbose - spew more stuff to the console when doing distutils stuff executables: f77exec [F77] - executable for Fortran 77 f90exec [F90] - executable for Fortran 90 ldshared [LDSHARED] - executable for shared libraries for Fortran ld [LD] - executable for linker for Fortran ar [AR] - library archive maker (for .a files) ranlib [RANLIB] - some things need ranlib run over the libraries. flags: f77flags [F77FLAGS] - compiler flags for Fortran 77 compiler f90flags [F90FLAGS] - compiler flags for Fortran 90 compiler freeflags [FREEFLAGS] - compiler flags for free-format Fortran 90 opt [FOPT] - optimisation flags for all Fortran compilers (used if noopt is false) arch [FARCH] - architecture-specific flags for Fortran (used if noarch is false) fdebug [FDEBUG] - debug-specific flags for Fortran (used if debug is true) fflags [FFLAGS] - extra compiler flags ldflags [LDFLAGS] - extra linker flags arflags [ARFLAGS] - extra library archiver flags There's also more central logic for finding executables, which should be more flexible, and takes care of, for instance, using the specified F77 compiler for the linker if the linker isn't specified, or the F90 compiler if either isn't, etc. Some of this type of stuff should be done for the C compiler, but isn't, as that would be messier with regards to hooking into Python's distutils. Personally, I think Python's distutils is a poorly laid-out framework, that is in need of serious refactoring. However, that's a lot of work, and I'm not going to do it right away... -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm(a)physics.mcmaster.ca

1 0