Mailman 3 October 2007 - NumPy-Discussion

More complex data types
by Neal Becker 05 Oct '07

05 Oct '07

I'm thinking (again) about using numpy for signal processing applications. One issue is that there are more data types that are commonly used in signal processing that are not available in numpy (or python). Specifically, it is frequently required to convert floating point algorithms into integer algorithms. numpy is fine for arrays of integers (of various sizes), but it is also very useful to have arrays of complex<integers>. While numpy has complex<double,float>, it doesn't have complex<int,int_64...> Has anyone thought about this?

2 5

Re: [Numpy-discussion] Numpy-discussion Digest, Vol 13, Issue 11
by Vincent Broman 05 Oct '07

05 Oct '07

On Friday, 05 October 2007, Neal Becker wrote: > I'm thinking (again) about using numpy for signal processing applications. > One issue is that there are more data types that are commonly used in > signal processing that are not available in numpy (or python). > Specifically, it is frequently required to convert floating point > algorithms into integer algorithms. numpy is fine for arrays of integers > (of various sizes), but it is also very useful to have arrays of > complex<integers>. While numpy has complex<double,float>, it doesn't have > complex<int,int_64...> Has anyone thought about this? Complex ints (or Gaussian integers) are algebraically interesting but they're just a Ring, not a Field. Before you know it, you're going to want to divide something. In signal processing, int measurements are usually not counters, they're just fixed point numbers with an implicit scale, and you only want to do integer arithmetic to save time/space, not because it is a good conceptual fit to the problem. In an interpreted language like Python, fixed point seems like a strange bird. I just use complex floats and a fast library. Vincent Broman

1 0

ERROR in installation of NumPy
by Simone Marras 05 Oct '07

05 Oct '07

Hello everyone, I am trying to install numpy on my Suse 10.2 using Python 2.5 Python is correctly installed and when I launch > python setup.py install, I get the following error: numpy/core/src/multiarraymodule.c:7604: fatal error: error writing to /tmp/ccNImg9Q.s: No space left on devicetee: /tmp/tmpLmEe5Y: No space left on device compilation terminated. _exec_command_posix failed (status=256) Can anyone help me, pelase? thanks in advance, simone

3 6

problem converting to matrix from Unicode input string
by Basilisk96 05 Oct '07

05 Oct '07

Hello all, I have the following function, with print statements inserted for debugging: import numpy def file2mat(inFile, sep=None, T=True): try: input = inFile.readlines() print "input=%s" % input except: raise finally: inFile.close() data = [line.split(sep) for line in input] print "data=%s" % data if T==True: return numpy.mat(data).astype(numpy.float64).T else: return numpy.mat(data).astype(numpy.float64) which is then tested as follows: >>> s = "-0.500 -0.500\n0.500 -0.500\n-0.500 0.500\n0.500 0.500" >>> u = unicode(s) >>> from cStringIO import StringIO >>> file2mat(StringIO(s)) input=['-0.500 -0.500\n', '0.500 -0.500\n', '-0.500 0.500\n', '0.500 0.500'] data=[['-0.500', '-0.500'], ['0.500', '-0.500'], ['-0.500', '0.500'], ['0.500', '0.500']] matrix([[-0.5, 0.5, -0.5, 0.5], [-0.5, -0.5, 0.5, 0.5]]) This is the expected result matrix... But now: >>> file2mat(StringIO(u)) input=['-\x000\x00.\x005\x000\x000\x00 \x00- \x000\x00.\x005\x000\x000\x00\n', '\x000\x00.\x005\x000\x000\x00 \x00- \x000\x00.\x005\x000\x000\x00\n', '\x00-\x000\x00.\x005\x000\x000\x00 \x000\x00.\x005\x000\x000\x00\n', '\x000\x00.\x005\x000\x000\x00 \x000\x00.\x005\x000\x000\x00'] data=[['-\x000\x00.\x005\x000\x000\x00', '\x00- \x000\x00.\x005\x000\x000\x00'], ['\x000\x00.\x005\x000\x000\x00', '\x00-\x000\x00.\x005\x000\x000\x00'], ['\x00- \x000\x00.\x005\x000\x000\x00', '\x000\x00.\x005\x000\x000\x00'], ['\x000\x00.\x005\x000\x000\x00', '\x000\x00.\x005\x000\x000\x00']] Traceback (most recent call last): ... ValueError: invalid literal for float(): - When I explicitly cast the input to 'string', I get the expected result: >>> file2mat(StringIO(str(u))) input=['-0.500 -0.500\n', '0.500 -0.500\n', '-0.500 0.500\n', '0.500 0.500'] data=[['-0.500', '-0.500'], ['0.500', '-0.500'], ['-0.500', '0.500'], ['0.500', '0.500']] matrix([[-0.5, 0.5, -0.5, 0.5], [-0.5, -0.5, 0.5, 0.5]]) Any suggestions on how to improve my code? Is this a Unicode issue, numpy issue, or both? The input string can come from an ASCII file or a GUI text control. In the case of a GUI, the control returns a Unicode string, so for now I am casting it to str(), but it seems like a hack.. BTW, the reason that I am using the astype() method of numpy.matrix is that I get a "ValueError: setting an array element with a sequence" when trying to use return numpy.mat(data, numpy.float64) in the above function. Thank you, -Basilisk96

1 0

Default value in documentation
by Pierre GM 05 Oct '07

05 Oct '07

All, I'm starting to update the documentation of maskedarray to the latest standard. How should I represent the default value of an optional parameter ? I was thinking something like def function(a, value=None) """Does something *Parameters*: a : {ndarray} Input array. value : {float} *[None]* Some value. If None, a default based on the dtype of a is used. """" Suggestions/ideas are welcome. Thanks a lot in advance P.

5 9

Best representation for array of points, or, how to distinguish a Nx1 array of points from a Nx3 array of scalars
by Edson Tadeu 04 Oct '07

04 Oct '07

I've found that there are two ways to represent an array of 3D points: # 01: N = 4 a = numpy.zeros((N, 3), dtype='f8') # 02: N = 4 b = numpy.zeros(N, dtype='3f8') The second representation seems "conceptually" better, it is really an 1D array of N elements ( i.e., b.shape == (4,) ), but, "in practice", it is not very useful, because I can't directly add it to scalar, or to another array of points, and I can't multiply it by a transformation matrix. With the first representation, I can use expressions like this: c = (a+1) * numpy.matrix([[1,0,0],[0,0,3],[0,2,0]]) - numpy.array([1, 5.5, 0]) So, I chose the second representation for my application; the problem is that its shape is (N, 3), and I haven't found a way to distinguish it from a Nx3 array of scalars, for example. This distinction is needed, because I must convert it to other array types (from other libraries), I must make proper I/O of the array, etc. Also, my application uses 2D and 3D arrays, and they can be arrays of scalars, of points or of small matrices... so, for example, a 10x7 array of 2x2 matrices of double is represented in numpy as a 10x7x2x2 array of f8 (without the proper information, it could be viewed also as a 10x7x2 array of 2D points or a 10x7x2x2 array of doubles, and I DON'T want this). My question is: * Is there any field in the NumPy object where I can keep this information (the shape of the "element"), without creeping it with the dtype='(M,N)f8' representation I explained above? Thanks, Edson

4 6

Re: [Numpy-discussion] [SciPy-dev] adopting Python Style Guide for classes
by Perry Greenfield 04 Oct '07

04 Oct '07

To follow on to my previous posting on this topic given Robert's response. As I said previously, I was never strongly committed to one approach or the other. But since the v1 release has been made, I think more care needs to be given to consideration of proposals like this before actually charging off to make the changes. 1) Even though Robert is speaking for Travis, I think given Travis's role in numpy, it is important for Travis to speak directly to this when he gets the chance. 2) API changes should only be made in major releases, not minor releases (IMHO). 3) Greater time should be provided to accommodate the transition. For example, there should not be deprecation warnings in the first version that this API appears in. The first release of this should not lead to nuisance messages for those that have other software that depends on this. (A tool that allows conditional messages would be good, but the default should be no message). The next release, sure. As a result, it means that the old API can't be removed until at least two releases after that. 4) More information should be provided as to what actually will change in the public interface. I suspect that it isn't obvious to many what will change. From the mailing list discussions there doesn't even seem to be consensus on the factory functions or type objects (more on these later). Many of the remaining objects are probably used internally (essentially private) and will affect few outside of the numpy developers. Since users typically deal mostly with factory functions, and other functions, they may not deal with classes much (outside of types). So listing the public classes so affected will help people understand what changes typical users will see, and what changes advanced users will see. While this is annoying, I think someone needs to write up an explicit list of those public classes that will be changed (and those that won't) so we all know what we will face. It may be a very small list and thus alleviate concern about the process. It may show some surprises that people hadn't thought about. Not doing this before making the changes seems very unwise. 5) In my humble opinion, we would be nuts--absolutely nuts--to change either the type classes or the factory functions. This would be foolish consistency at it's worst. We *just* went through the exercise of changing Int32 to int32 and so forth and we would have to change back again? This cannot be seriously considered. Perry Greenfield On Oct 2, 2007, at 1:24 PM, Jarrod Millman wrote: > Hello, > > For those of you not on the Numpy developer's list, we have been > talking about adopting the Python class naming convention to conform > with Guido's style guide as closely as possible: > http://www.python.org/dev/peps/pep-0008/ > According to Guido, class names should use the CapWords convention. > Most Python projects (eg, ETS, matploltlib) adhere to the Python > naming conventions and it is confusing that NumPy and SciPy don't. > > Currently, both NumPy and SciPy use either lower_underscore_separated > or CapWords for class names. > > NumPy > ====== > $ grep -r '^class [A-Z]' --include "*.py" * | wc -l > 1014 > $ grep -r '^class' --include "*.py" * | grep -v 'class [A-Z]' | wc -l > 207 > > SciPy > ===== > $ grep -r '^class [A-Z]' --include "*.py" * | wc -l > 587 > $ grep -r '^class' --include "*.py" * | grep -v 'class [A-Z]' | wc -l > 565 > > So far, there is Universal support for the basic idea of conforming to > the Python Style Guide. Since it should effect the user API, we have > already updated the TestCase classes. First, we checked in a change > to allow TestCase classes to be prefixed with > either 'test' or 'Test': > http://projects.scipy.org/scipy/numpy/changeset/4144 > Then, we updated both SciPy and NumPy to use CapWords for TestCase > classes: > http://projects.scipy.org/scipy/numpy/changeset/4151 > http://projects.scipy.org/scipy/scipy/changeset/3388 > In order to run the SciPy tests on the trunk you will need to install > NumPy from svn revision 4144 or greater. Before SciPy 0.7.0 is > released, we will release NumPy 1.0.4. > > Also, if you are adding a new class to NumPy or SciPy, please use > CapWords. Now we need to decide what to do about the remaining > lower_underscore_separated class names. Obviously, it is important > that we are careful to not break a lot of code just to bring our class > names up to standards. > > Cheers, > > -- > Jarrod Millman > Computational Infrastructure for Research Labs > 10 Giannini Hall, UC Berkeley > phone: 510.643.4014 > http://cirl.berkeley.edu/ > _______________________________________________ > Scipy-dev mailing list > Scipy-dev(a)scipy.org > http://projects.scipy.org/mailman/listinfo/scipy-dev

6 6

Possible bug in numpy.trace()
by Donald Fredkin 04 Oct '07

04 Oct '07

It appears that something peculiar is going on with the numpy function trace(). The docs say that trace(A,...) is identical to A.trace(...). Here is a test: A = arange(8).reshape((2,2,2)) A.trace(0,1,2) #[Out]# array([ 3, 11]) # which is correct trace(A,0,1,2) #[Out]# array([6, 8]) #which is wrong #Since trace() is computed by summing the result of diagonal() A.diagonal(0,1,2) #[Out]# array([[0, 3], #[Out]# [4, 7]]) #which is correct diagonal(A,0,1,2) #[Out]# array([[0, 3], #[Out]# [4, 7]]) #which is the same correct result A.trace(0,0,1) #[Out]# array([6, 8]) #this is the erroneous answer we found earlier for trace(A,0,1,2) -- Donald R. Fredkin drfredkin(a)ucsd.edu

2 1

APL2007 Update
by Steven H. Rogers 04 Oct '07

04 Oct '07

APL 2007 in Montreal (only 2 1/2 weeks away, Oct 20-22). Summary program information is now available on the APL2007 home page http://www.sigapl.org/apl2007.html with a link to the comprehensive program description at http://www.sigapl.org/apl2007-program.html#a2 Registration for APL2007 is at http://www.regmaster.com/conf/oopsla2007.html

1 0

indexing bug?
by Nadia Dencheva 03 Oct '07

03 Oct '07

Is indexing with floats really allowed in numpy? >>> a=numpy.array([1,2,3,4]) >>> a[2.99] 3 Nadia Dencheva

9 12

boolean matrix bug?
by Alan G Isaac 03 Oct '07

03 Oct '07

Shouldn't x**0 be boolean for a boolean matrix? Cheers, Alan Isaac >>> import numpy >>> numpy.__version__ '1.0.3.1' >>> x = numpy.mat('1 1;1 0',dtype='bool') >>> x**0 matrix([[ 1., 0.], [ 0., 1.]]) >>> x**1 matrix([[ True, True], [ True, False]], dtype=bool) >>> x**2 matrix([[ True, True], [ True, True]], dtype=bool) >>>

1 0

adopting Python Style Guide for classes
by Jarrod Millman 02 Oct '07

02 Oct '07

Hello, NumPy and SciPy should conform with Guido's style guide as closely as possible: http://www.python.org/dev/peps/pep-0008/ The only serious divergence that I am aware of between the NumPy and SciPy codebase and the Python recommended standards is in class naming. According to Guido, class names should use the CapWords convention. Most Python projects (eg, ETS, matploltlib) adhere to the Python naming conventions and it is confusing that NumPy and SciPy don't. Currently, both NumPy and SciPy use either lower_underscore_separated or CapWords for class names. By my rough count, NumPy has about 752 classes using CapWords and 440 using lower_underscore_separated. While SciPy has about 256 classes using CapWords and 815 using lower_underscore_separated. Moreover, a far amount of the lower_underscore_separated classes are tests. That a number of classes currently use CapWords despite the NumPy convention to use lower_underscore_separated names may indicate it would make more sense to switch to the Python convention. I am hoping that most of you agree with the general principle of bringing NumPy and SciPy into compliance with the standard naming conventions. I have already talked to a few people including Travis about this switch and so far everyone supports the change in general. Please let me know if you have any major objections to adopting the Python class naming convention. Once we have agreed to using CapWords for classes, we will need to decide what to do about our existing class names. Obviously, it is important that we are careful to not break a lot of code just to bring our class names up to standards. So at the very least, we probably won't be able to just change every class name until NumPy 1.1.0 is released. Here is what I propose for NumPy: 1. Change the names of the TestCase class names to use CapWords. I just checked in a change to allow TestCase classes to be prefixed with either 'test' or 'Test': http://projects.scipy.org/scipy/numpy/changeset/4144 If no one objects, I plan to go through all the NumPy unit tests and change their names to CapWords. Ideally, I would like to get at least this change in before NumPy 1.0.4. 2. Any one adding a new class to NumPy would use CapWords. 3. When we release NumPy 1.1, we will convert all (or almost all) class names to CapWords. There is no reason to worry about the exact details of this conversion at this time. I just would like to get a sense whether, in general, this seems like a good direction to move in. If so, then after we get steps 1 and 2 completed we can start discussing how to handle step 3. Cheers, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

8 9

Run length encoding of an ndarray
by Michael Hoffman 02 Oct '07

02 Oct '07

I am trying to do a type of run-length encoding of a 2D array by axis. I have an array of values arranged along two axes, state and position. These are many (180, 30000) uint8 arrays. I would like to have a list of tuples like (state, start_pos, end_pos, values) only separating out a set of values into a new tuple if they are all the same value in a run of at least 10 cells. Is there a clever way to do this in NumPy? I was thinking of using itertools.groupby() but it would be nicer to have something faster.

2 2

Naming a slice index?
by Eagle Jones 02 Oct '07

02 Oct '07

New to python and numpy; hopefully I'm missing something obvious. I'd like to be able to slice an array with a name. For example: _T = 6:10 _R = 10:15 A = identity(20) foo = A[_T, _R] This doesn't work. Nor does _T=range(6:10); _R = range(10:15). Any ideas? Thanks, Eagle

4 3