Mailman 3 August 2011 - NumPy-Discussion

the build and installation process
by srean Aug. 26, 2011

Aug. 26, 2011

Hi, I would like to know a bit about how the installation process works. Could you point me to a resource. In particular I want to know how the site.cfg configuration works. Is it numpy/scipy specific or is it standard with distutils. I googled for site.cfg and distutils but did not find any authoritative document. I believe many new users trip up on the installation process, especially in trying to substitute their favourite library in place os the standard. So a canonical document explaining the process will be very helpful. http://docs.scipy.org/doc/numpy/user/install.html does cover some of the important points but its a bit sketchy, and has a "this is all that you need to know" flavor. Doesnt quite enable the reader to fix his own problems. So a resource that is somewhere in between reading up all the sources that get invoked during the installation and building, and the current install document will be very welcome. English is not my native language, but if there is anyway I can help, I would do so gladly. -- srean

2 1

How to output array with indexes to a text file?
by Paul Menzel Aug. 26, 2011

Aug. 26, 2011

Dear NumPy folks, is there an easy way to also save the indexes of an array (columns, rows or both) when outputting it to a text file. For saving an array to a file I only found `savetxt()` [1] which does not seem to have such an option. Adding indexes manually is doable but I would like to avoid that. --- minimal example (also attached) --- from numpy import * a = zeros([2, 3], int) print(a) savetxt("/tmp/test1.txt", a, fmt='%8i') # Work around for adding the indexes for the columns. a[0] = range(3) print(a) savetxt("/tmp/test2.txt", a, fmt='%8i') --- minimal example --- The output is the following. $ python output-array.py [[0 0 0] [0 0 0]] [[0 1 2] [0 0 0]] $ more /tmp/test* :::::::::::::: /tmp/test1.txt :::::::::::::: 0 0 0 0 0 0 :::::::::::::: /tmp/test2.txt :::::::::::::: 0 1 2 0 0 0 Is there a way to accomplish that task without reserving the 0th row or column to store the indexes? I want to process these text files to produce graphs and MetaPost’s [2] graph package needs these indexes. (I know about Matplotlib [3], but I would like to use MetaPost.) Thanks, Paul [1] http://docs.scipy.org/doc/numpy/reference/generated/numpy.savetxt.html [2] http://wiki.contextgarden.net/MetaPost [3] http://matplotlib.sourceforge.net/

3 2

Identifying Colinear Columns of a Matrix
by Mark Janikas Aug. 26, 2011

Aug. 26, 2011

Hello All, I am trying to identify columns of a matrix that are perfectly collinear. It is not that difficult to identify when two columns are identical are have zero variance, but I do not know how to ID when the culprit is of a higher order. i.e. columns 1 + 2 + 3 = column 4. NUM.corrcoef(matrix.T) will return NaNs when the matrix is singular, and LA.cond(matrix.T) will provide a very large condition number.... But they do not tell me which columns are causing the problem. For example: zt = numpy. array([[ 1. , 1. , 1. , 1. , 1. ], [ 0.25, 0.1 , 0.2 , 0.25, 0.5 ], [ 0.75, 0.9 , 0.8 , 0.75, 0.5 ], [ 3. , 8. , 0. , 5. , 0. ]]) How can I identify that columns 0,1,2 are the issue because: column 1 + column 2 = column 0? Any input would be greatly appreciated. Thanks much, MJ

5 10

NA mask C-API documentation
by Mark Wiebe Aug. 26, 2011

Aug. 26, 2011

I've added C-API documentation to the missingdata branch. The .rst file (beware of the github rst parser though, it drops some of the content) is here: https://github.com/m-paradox/numpy/blob/missingdata/doc/source/reference/c-… and I made a small example module which goes with it here: https://github.com/m-paradox/spdiv Cheers, Mark

2 2

saving groups of numpy arrays to disk
by Chris Withers Aug. 26, 2011

Aug. 26, 2011

Hi All, I've got a tree of nested dicts that at their leaves end in numpy arrays of identical sizes. What's the easiest way to persist these to disk so that I can pick up with them where I left off? What's the most "correct" way to do so? I'm using IPython if that makes things easier... I had wondered about PyTables, but that seems a bit too heavyweight for this, unless I'm missing something? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

8 9

array_equal and array_equiv comparison functions for structured arrays
by Derek Homeier Aug. 26, 2011

Aug. 26, 2011

Hi, as the subject says, the array_* comparison functions currently do not operate on structured/record arrays. Pull request https://github.com/numpy/numpy/pull/146 implements these comparisons. There are two commits, differing in their interpretation whether two arrays with different field names, but identical data, are equivalent; i.e. res = array_equiv(array((1,2), dtype=[('i','i4'),('v','f8')]), array((1,2), dtype=[('n','i4'),('f','f8')])) is True in the current HEAD, but False in its parent. Feedback and additional comments are invited. Cheers, Derek

1 0

lazy loading ndarray? (not from file, but from user function)
by Kibeom Kim Aug. 26, 2011

Aug. 26, 2011

Hello, Q1. Is lazy loading ndarray from user defined data supplying function possible? Q2. If possible, how can I implement it? The closest method I can think of is, (which requires c++ posix) 1. create a memory region using mmap and protect read operation by mprotect. 2. add SIGSEGV signal handler to trap read operation on the memory region, and the handler will provide appropriate user data and recover from SIGSEGV. 3. slightly modify memmap class to use the above mmap (memmap is already using mmap internally, so it's not a big deal) but obviously, recovering from SIGSEGV requires removing mprotect (see http://stackoverflow.com/questions/2663456/write-a-signal-handler-to-catch-…) and it's impossible to know when to lock the region by mprotect again. Thanks, -Kibeom Kim

2 1

NA-mask introductory documentation
by Mark Wiebe Aug. 25, 2011

Aug. 25, 2011

I've written some introductory documentation for the NA-masked arrays. The patch is here: https://github.com/m-paradox/numpy/commit/227e39c34b0e5d9dfde2bbce054b5a8ac… This is approaching the end of what I will implement for NA masks at the moment. I think the system is quite usable as is, though it is missing a number of major pieces like support for struct-NA, file I/O, and other things mentioned in the release notes. On the other hand, the C API for working with NA-masked arrays is solid and designed for future expansion to multi-NA, and many things can be done already with the implementation. It's also very stable and does not break ABI compatibility, so a NumPy release with NA masks in its current state should be perfectly reasonable. Cheers, Mark

1 0

c-info.ufunc-tutorial.rst
by srean Aug. 25, 2011

Aug. 25, 2011

Hi, I was reading this document, https://github.com/numpy/numpy/blob/master/doc/source/user/c-info.ufunc-tut… its well written and there is a good build up to exciting code examples that are coming, but I do not see the actual examples, only how they may be used. Is it located somewhere else and not linked? or is it that the c-info.ufunc-tutorial.rst document is incomplete and the examples have not been written. I suspect the former. In that case could anyone point to the code examples and may be also update the c-info.ufunc-tutorial.rst document. Thanks -- srean

4 6

memoryview shape/strides representation for ndim = 0
by Stefan Krah Aug. 25, 2011

Aug. 25, 2011

Hello, Numpy arrays and memoryview currently have different representations for shape and strides if ndim = 0: >>> from numpy import * >>> x = array(9, int32) >>> x.ndim 0 >>> x.shape () >>> x.strides () >>> m = memoryview(x) >>> m.ndim 0L >>> m.shape is None True >>> m.strides is None True I think the Numpy representation is nicer. Also, I think that memoryviews should attempt to mimic the underlying object as closely as possible. Since the ndim = 0 case probably only occurs in Numpy, it might be possible to change the representation in memoryview. Travis, was the "shape is None" representation used for compatibility with ctypes? Would it be possible or advisable to use the Numpy representation? Stefan Krah

3 3