Mailman 3 January 2009 - NumPy-Discussion

Re: [Numpy-discussion] building numpy/scipy
by Mike Landis 02 Jan '09

02 Jan '09

1 1

Alternative to record array
by Jean-Baptiste Rudant 02 Jan '09

02 Jan '09

Hello, I like to use record arrays to access fields by their name, and because they are esay to use with pytables. But I think it's not very effiicient for what I have to do. Maybe I'm misunderstanding something. Example : import numpy as np age = np.random.randint(0, 99, 10e6) weight = np.random.randint(0, 200, 10e6) data = np.rec.fromarrays((age, weight), names='age, weight') # the kind of operations I do is : data.age += data.age + 1 # but it's far less efficient than doing : age += 1 # because I think the record array stores [(age_0, weight_0) ...(age_n, weight_n)] # and not [age0 ... age_n] then [weight_0 ... weight_n]. So I think I don't use record arrays for the right purpose. I only need something which would make me esasy to manipulate data by accessing fields by their name. Am I wrong ? Is their something in numpy for my purpose ? Do I have to implement my own class, with something like : class FieldArray: def __init__(self, array_dict): self.array_list = array_dict def __getitem__(self, field): return self.array_list[field] def __setitem__(self, field, value): self.array_list[field] = value my_arrays = {'age': age, 'weight' : weight} data = FieldArray(my_arrays) data['age'] += 1 Thank you for the help, Jean-Baptiste Rudant

5 6

building numpy/scipy
by Mike Landis 02 Jan '09

02 Jan '09

Some of the install instructions are kind of ambiguous. When a library name ends in .a or .dll, it's obvious what it is, but 'library' is sometimes used generically without indicating whether you're talking about static or dynamic, e.g. how does numpy/scipy link to MKL? Is it statically or dynamically linked? It's not clear where things should go or how things are found. After downloading numpy and putting numpy-1.2.1 into a directory named numpy-1.2.1 in python2.5/Lib/site-packages/ and running python setup.py install from the numpy-1.2.1 directory, you get a numpy directory in site-packages. Later, the install instructions refer to the numpy directory generically. Which numpy directory are you talking about? Should the numpy-1.2.1 directory not be in site-packages in the first place? Ditto for scipy. E.g. are you referring to the site.cfg in the distribution directory (numpy-1.2.1) or the installed directory (numpy without version notation). Maybe you should give names to these directories (e.g. distribution and install) and always refer to them with leading adjectives. It should be possible to create a script that keeps track of all of these extra lib configuration steps, so installs won't be so involved and error prone. I built ATLAS (they need better documentation of processor types and how to use what CPU-Z tells you) and I have MKL installed. A quarter of my MKL evaluation days have already ticked away, and I've come to the disappointing realization that 30 days is probably not going to be enough to get the scipy/MKL install right. What's compatible with what? There is mention of MKL, ATLAS, LAPACK and CLAPACK (I also downloaded CLAPACK 3.1.1.1). It seems like CLAPACK would be naturally compatible because python uses C-style matrix addressing, but does Python (or numpy) flip indexes around if you have LAPACK instead of CLAPACK, so it works anyway, if slower? There is mention of ".numpy-site.cfg" in the user's home directory (which I created), but it didn't seem to have any effect on installing. I created the following site.cfg in the numpy (install) directory: [mkl] include_dirs = D:\Programs\Intel\MKL\10.1.0.018\include library_dirs = D:\Programs\Intel\MKL\10.1.0.018\ia32\lib mkl_libs = mkl_ia32, mkl_c_dll, libguide40 lapack_libs = mkl_lapack Do numpy AND scipy each need a site.cfg in their respective install directories? If so, does one take precedence over the other? When I run 'python setup.py install' from the numpy install directory, I get: Warning: No configuration returned, assuming unavailable. blas_opt_info: blas_mkl_info: libraries mkl, vml, guide not found in d:\Programs\Python25\lib libraries mkl, vml, guide not found in C:\ libraries mkl, vml, guide not found in d:\Programs\Python25\libs NOT AVAILABLE atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries mkl, vml, guide not found in d:\Programs\Python25\lib libraries mkl, vml, guide not found in C:\ libraries mkl, vml, guide not found in d:\Programs\Python25\libs NOT AVAILABLE atlas_blas_info: libraries mkl, vml, guide not found in d:\Programs\Python25\lib libraries mkl, vml, guide not found in C:\ libraries mkl, vml, guide not found in d:\Programs\Python25\libs NOT AVAILABLE D:\Programs\Python25\lib\site-packages\numpy\disutils\system_info.py:1340: UserWarning: Atlas (http://math-atlas.sourceforgr.net/) libraries not found Directories to search for the libraries can be specified in the numpy/disutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) blas_info: libraries blas not found in d:\Programs\Python25\lib ----------------------------------------------------------------- Where is it looking for blas_opt_info, blas_mkl_info, atlas_blas_threads_info, and atlas_blas_info? Is it actually looking on sourceforge for Atlas? FYI, when I try using make in ...MKL/10.1.0.018/examples/cblas, I get a "missing separator" error on the "!INCLUDE cblas.lst" line (first significant line in the file). There are no spaces on this line. Could cygwin make be complaining about something inside the cblas.lst file, or is complaining about the line not being terminated the way it expects it to be (I see carriage return, line feed pairs in the file). HELP!!!!

3 2

PyArray_DescrConverter segfault
by Igor Sylvester 01 Jan '09

01 Jan '09

Hi, Does anyone have an insight on the following problem? PyObject* descr(PyObject* self, PyObject* args) { PyObject *d; PyArg_ParseTuple(args, "O&", PyArray_DescrConverter, &d); return d; } >>> import numpy >>> import mymodule >>> numpy.__version__ '1.2.1' >>> mymodule.descr([('a', 'i4'), ('b', 'i8')]) segmentation fault (core dumped) ipython Thanks! -Igor

2 4