Mailman 3 August 2011 - NumPy-Discussion

How to output array with indexes to a text file?
by Paul Menzel 26 Aug '11

26 Aug '11

Dear NumPy folks, is there an easy way to also save the indexes of an array (columns, rows or both) when outputting it to a text file. For saving an array to a file I only found `savetxt()` [1] which does not seem to have such an option. Adding indexes manually is doable but I would like to avoid that. --- minimal example (also attached) --- from numpy import * a = zeros([2, 3], int) print(a) savetxt("/tmp/test1.txt", a, fmt='%8i') # Work around for adding the indexes for the columns. a[0] = range(3) print(a) savetxt("/tmp/test2.txt", a, fmt='%8i') --- minimal example --- The output is the following. $ python output-array.py [[0 0 0] [0 0 0]] [[0 1 2] [0 0 0]] $ more /tmp/test* :::::::::::::: /tmp/test1.txt :::::::::::::: 0 0 0 0 0 0 :::::::::::::: /tmp/test2.txt :::::::::::::: 0 1 2 0 0 0 Is there a way to accomplish that task without reserving the 0th row or column to store the indexes? I want to process these text files to produce graphs and MetaPost’s [2] graph package needs these indexes. (I know about Matplotlib [3], but I would like to use MetaPost.) Thanks, Paul [1] http://docs.scipy.org/doc/numpy/reference/generated/numpy.savetxt.html [2] http://wiki.contextgarden.net/MetaPost [3] http://matplotlib.sourceforge.net/

3 2

Identifying Colinear Columns of a Matrix
by Mark Janikas 26 Aug '11

26 Aug '11

Hello All, I am trying to identify columns of a matrix that are perfectly collinear. It is not that difficult to identify when two columns are identical are have zero variance, but I do not know how to ID when the culprit is of a higher order. i.e. columns 1 + 2 + 3 = column 4. NUM.corrcoef(matrix.T) will return NaNs when the matrix is singular, and LA.cond(matrix.T) will provide a very large condition number.... But they do not tell me which columns are causing the problem. For example: zt = numpy. array([[ 1. , 1. , 1. , 1. , 1. ], [ 0.25, 0.1 , 0.2 , 0.25, 0.5 ], [ 0.75, 0.9 , 0.8 , 0.75, 0.5 ], [ 3. , 8. , 0. , 5. , 0. ]]) How can I identify that columns 0,1,2 are the issue because: column 1 + column 2 = column 0? Any input would be greatly appreciated. Thanks much, MJ

5 10

NA mask C-API documentation
by Mark Wiebe 26 Aug '11

26 Aug '11

I've added C-API documentation to the missingdata branch. The .rst file (beware of the github rst parser though, it drops some of the content) is here: https://github.com/m-paradox/numpy/blob/missingdata/doc/source/reference/c-… and I made a small example module which goes with it here: https://github.com/m-paradox/spdiv Cheers, Mark

2 2

saving groups of numpy arrays to disk
by Chris Withers 26 Aug '11

26 Aug '11

Hi All, I've got a tree of nested dicts that at their leaves end in numpy arrays of identical sizes. What's the easiest way to persist these to disk so that I can pick up with them where I left off? What's the most "correct" way to do so? I'm using IPython if that makes things easier... I had wondered about PyTables, but that seems a bit too heavyweight for this, unless I'm missing something? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

8 9

array_equal and array_equiv comparison functions for structured arrays
by Derek Homeier 26 Aug '11

26 Aug '11

Hi, as the subject says, the array_* comparison functions currently do not operate on structured/record arrays. Pull request https://github.com/numpy/numpy/pull/146 implements these comparisons. There are two commits, differing in their interpretation whether two arrays with different field names, but identical data, are equivalent; i.e. res = array_equiv(array((1,2), dtype=[('i','i4'),('v','f8')]), array((1,2), dtype=[('n','i4'),('f','f8')])) is True in the current HEAD, but False in its parent. Feedback and additional comments are invited. Cheers, Derek

1 0

lazy loading ndarray? (not from file, but from user function)
by Kibeom Kim 26 Aug '11

26 Aug '11

Hello, Q1. Is lazy loading ndarray from user defined data supplying function possible? Q2. If possible, how can I implement it? The closest method I can think of is, (which requires c++ posix) 1. create a memory region using mmap and protect read operation by mprotect. 2. add SIGSEGV signal handler to trap read operation on the memory region, and the handler will provide appropriate user data and recover from SIGSEGV. 3. slightly modify memmap class to use the above mmap (memmap is already using mmap internally, so it's not a big deal) but obviously, recovering from SIGSEGV requires removing mprotect (see http://stackoverflow.com/questions/2663456/write-a-signal-handler-to-catch-…) and it's impossible to know when to lock the region by mprotect again. Thanks, -Kibeom Kim

2 1

NA-mask introductory documentation
by Mark Wiebe 25 Aug '11

25 Aug '11

I've written some introductory documentation for the NA-masked arrays. The patch is here: https://github.com/m-paradox/numpy/commit/227e39c34b0e5d9dfde2bbce054b5a8ac… This is approaching the end of what I will implement for NA masks at the moment. I think the system is quite usable as is, though it is missing a number of major pieces like support for struct-NA, file I/O, and other things mentioned in the release notes. On the other hand, the C API for working with NA-masked arrays is solid and designed for future expansion to multi-NA, and many things can be done already with the implementation. It's also very stable and does not break ABI compatibility, so a NumPy release with NA masks in its current state should be perfectly reasonable. Cheers, Mark

1 0

c-info.ufunc-tutorial.rst
by srean 25 Aug '11

25 Aug '11

Hi, I was reading this document, https://github.com/numpy/numpy/blob/master/doc/source/user/c-info.ufunc-tut… its well written and there is a good build up to exciting code examples that are coming, but I do not see the actual examples, only how they may be used. Is it located somewhere else and not linked? or is it that the c-info.ufunc-tutorial.rst document is incomplete and the examples have not been written. I suspect the former. In that case could anyone point to the code examples and may be also update the c-info.ufunc-tutorial.rst document. Thanks -- srean

4 6

memoryview shape/strides representation for ndim = 0
by Stefan Krah 25 Aug '11

25 Aug '11

Hello, Numpy arrays and memoryview currently have different representations for shape and strides if ndim = 0: >>> from numpy import * >>> x = array(9, int32) >>> x.ndim 0 >>> x.shape () >>> x.strides () >>> m = memoryview(x) >>> m.ndim 0L >>> m.shape is None True >>> m.strides is None True I think the Numpy representation is nicer. Also, I think that memoryviews should attempt to mimic the underlying object as closely as possible. Since the ndim = 0 case probably only occurs in Numpy, it might be possible to change the representation in memoryview. Travis, was the "shape is None" representation used for compatibility with ctypes? Would it be possible or advisable to use the Numpy representation? Stefan Krah

3 3

Decimal arrays?
by Chris Withers 25 Aug '11

25 Aug '11

Hi All, What's the best type of array to use for decimal values? (ie: where I care about precision and want to avoid any possible rounding errors) cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

5 9

NumPy-Discussion August 2011