Mailman 3 July 2006 - NumPy-Discussion

MA bug or feature?
by Michael Sorich 04 Jul '06

04 Jul '06

When transposing a masked array of dtype '<f8' I noticed that an ndarray of dtype '|O4' was returned. I found this a little strange as the masked array did not contain any masked values. Upon closer examination (see script below) it seems that if the mask is a boolean array with all false values and the fill_value is a string this will occur. However if the fill_value is a number or the mask is simply False, the dtype stays as '<f8'. I was going to submit this as a bug but wanted to check that this was not a deliberate feature. I am using the numpy version that comes with a recent version of python enthought edition. import numpy print numpy.version.version ma1 = numpy.ma.array(((1.,2,3),(4,5,6)), mask=((0,0,0),(0,0,0))) print ma1.filled('NA').dtype print ma1.filled(-999).dtype ma2 = numpy.ma.array(((1.,2,3),(4,5,6)), mask=False) print ma2.filled('NA').dtype print ma2.filled(-999).dtype --------------- output: 0.9.9.2538 '|O4' '<f8' '<f8' '<f8'

3 8

KeyError with float96 in linalg.py: polyfit
by Jan-Matthias Braun 04 Jul '06

04 Jul '06

Hi all, I'm testing some computations with float96 at the moment and right now I have problems with polyfit raising a KeyError for the keycode 'g', which is floatxx with xx>64. I am getting a KeyError using polyfit on some float96 values. The used Routines seem to know nothing about this type. My main question is: have I missed something? Shouldn't this type be used? Below is a more detailed descripton. Thanks in advance, Jan ---------------------------------------------------------------------------- In file numpy/linalg/linalg.py, the following definitions at lines 26ff seem to be the offending ones: # Helper routines _array_kind = {'i':0, 'l': 0, 'q': 0, 'f': 0, 'd': 0, 'F': 1, 'D': 1} _array_precision = {'i': 1, 'l': 1, 'q': 1, 'f': 0, 'd': 1, 'F': 0, 'D': 1} _array_type = [['f', 'd'], ['F', 'D']] Here the new typecodes are missing. I tried # Helper routines _array_kind = {'i':0, 'l': 0, 'q': 0, 'f': 0, 'd': 0, 'g': '0', 'F': 1, 'D':1, 'G':1} _array_precision = {'i': 1, 'l': 1, 'q': 1, 'f': 0, 'd': 1, 'g': 1, 'F': 0, 'D': 1, 'G': 1} _array_type = [['f', 'd', 'g'], ['F', 'D', 'G']] which gets me a step further to a TypeError: File "lib/python2.3/site-packages/numpy/linalg/linalg.py", line 454, in lstsq bstar[:b.shape[0],:n_rhs] = b.copy() TypeError: array cannot be safely cast to required type (Question: Why only one typecode for a type which varies in bitlength on different platforms? On Opteron CPU's I've seen float128 with 'g'?)

2 1

Why is my array not contiguous?
by Jens Jørgen Mortensen 04 Jul '06

04 Jul '06

Hi! With numpy-0.9.9.2726, I do this: >>> x = arange(4) >>> y = x[newaxis, :] I would expect both arrays to be contiguous: >>> x.flags.contiguous, y.flags.contiguous (True, False) Shouldn't y be contiguous? Maybe it's because of the strange strides: >>> y.strides (0, 4) >>> y.strides = (16, 4) >>> y.flags.contiguous True Jens Jørgen

1 0

Error building numpy from svn, 2nd attempt
by John Carter 04 Jul '06

04 Jul '06

Hi, As is the way posting to a news group stirs the brain cell into activity and the problem is solved. or rather shifted. I've downloaded the candidate version of mingw32 and using that to build numpy/scipy works, or rather it builds the extensions for Python 2.3 I believe that there are still problems as both numpy and scipy fail their tests. I'm also dubious as to whether they are finding Altas and Lapack. Configuration: Windows XP SP2 Cygwin = gcc version 3.4.4 (cygming special) (gdc 0.12, using dmd 0.125) MinGW = gcc version 3.4.5 (mingw special) Python = Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on win32 scipy, numpy, Atlas, Lapack all downloaded within the last 2 days I've cut and pasted the error messages from the tests below. Any help gratefully received. John ===== numpy ===== D:.\numerical> python Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.test() Found 5 tests for numpy.distutils.misc_util Found 3 tests for numpy.lib.getlimits Found 30 tests for numpy.core.numerictypes Found 13 tests for numpy.core.umath Found 3 tests for numpy.core.scalarmath Found 8 tests for numpy.lib.arraysetops Found 42 tests for numpy.lib.type_check Found 101 tests for numpy.core.multiarray Found 36 tests for numpy.core.ma Found 10 tests for numpy.lib.twodim_base Found 10 tests for numpy.core.defmatrix Found 1 tests for numpy.lib.ufunclike Found 38 tests for numpy.lib.function_base Found 3 tests for numpy.dft.helper Found 1 tests for numpy.lib.polynomial Found 7 tests for numpy.core.records Found 26 tests for numpy.core.numeric Found 4 tests for numpy.lib.index_tricks Found 46 tests for numpy.lib.shape_base Found 0 tests for __main__ ........................................................................................................ ........................................................................................................ .........E.............................................................................................. ........................... ====================================================================== ERROR: check_simple (numpy.lib.tests.test_twodim_base.test_histogram2d) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python23\Lib\site-packages\numpy\lib\tests\test_twodim_base.py", line 137, in check_simple np.random.seed(1) File "mtrand.pyx", line 311, in mtrand.RandomState.seed SystemError: C:\sf\python\dist\src-maint23\Objects\longobject.c:240: bad argument to internal function ---------------------------------------------------------------------- Ran 387 tests in 1.391s FAILED (errors=1) <unittest.TextTestRunner object at 0x010DB8F0> >>> ===== scipy ===== D:.\numerical> python Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import scipy >>> scipy.test() Overwriting lib=<module 'scipy.lib' from 'C:\PYTHON23\lib\site-packages\scipy\lib\__init__.pyc'> from C:\PYTHON23\lib\si te-packages\scipy\lib\__init__.pyc (was <module 'numpy.lib' from 'C:\PYTHON23\lib\site-packages\numpy\lib\__init__.pyc'> from C:\PYTHON23\lib\site-packages\numpy\lib\__init__.pyc) Overwriting fft=<function fft at 0x013E7AB0> from scipy.fftpack.basic (was <function fft at 0x00AAEE30> from numpy.dft.f ftpack) Overwriting ifft=<function ifft at 0x013E7AF0> from scipy.fftpack.basic (was <function ifft at 0x00AAEE70> from numpy.df t.fftpack) Found 4 tests for scipy.io.array_import Found 128 tests for scipy.linalg.fblas Found 397 tests for scipy.ndimage Found 10 tests for scipy.integrate.quadpack Found 97 tests for scipy.stats.stats Found 47 tests for scipy.linalg.decomp Found 2 tests for scipy.integrate.quadrature Found 95 tests for scipy.sparse.sparse Found 20 tests for scipy.fftpack.pseudo_diffs Found 6 tests for scipy.optimize.optimize Found 5 tests for scipy.interpolate.fitpack Found 1 tests for scipy.interpolate Found 12 tests for scipy.io.mmio Found 10 tests for scipy.stats.morestats Found 4 tests for scipy.linalg.lapack Found 18 tests for scipy.fftpack.basic Found 4 tests for scipy.linsolve.umfpack Found 4 tests for scipy.optimize.zeros Found 41 tests for scipy.linalg.basic Found 2 tests for scipy.maxentropy.maxentropy Found 358 tests for scipy.special.basic Found 128 tests for scipy.lib.blas.fblas Found 7 tests for scipy.linalg.matfuncs **************************************************************** WARNING: clapack module is empty ----------- See scipy/INSTALL.txt for troubleshooting. Notes: * If atlas library is not found by numpy/distutils/system_info.py, then scipy uses flapack instead of clapack. **************************************************************** Found 42 tests for scipy.lib.lapack Found 1 tests for scipy.optimize.cobyla Found 16 tests for scipy.lib.blas Found 1 tests for scipy.integrate Found 14 tests for scipy.linalg.blas Found 70 tests for scipy.stats.distributions Found 4 tests for scipy.fftpack.helper Found 4 tests for scipy.signal.signaltools Found 0 tests for __main__ Don't worry about a warning regarding the number of bytes read. Warning: 1000000 bytes requested, 20 bytes read. .......caxpy:n=4 ..caxpy:n=3 ....ccopy:n=4 ..ccopy:n=3 .............cscal:n=4 ....cswap:n=4 ..cswap:n=3 .....daxpy:n=4 ..daxpy:n=3 ....dcopy:n=4 ..dcopy:n=3 .............dscal:n=4 ....dswap:n=4 ..dswap:n=3 .....saxpy:n=4 ..saxpy:n=3 ....scopy:n=4 ..scopy:n=3 .............sscal:n=4 ....sswap:n=4 ..sswap:n=3 .....zaxpy:n=4 ..zaxpy:n=3 ....zcopy:n=4 ..zcopy:n=3 .............zscal:n=4 ....zswap:n=4 ..zswap:n=3 ........................................................................................................................ ........................................................................................................................ ........................................................................................................................ ........................................................................................................................ .........................................................................Took 13 points. ..........Resizing... 16 17 24 Resizing... 20 7 35 Resizing... 23 7 47 Resizing... 24 25 58 Resizing... 28 7 68 Resizing... 28 27 73 .....Use minimum degree ordering on A'+A. ........................Use minimum degree ordering on A'+A. ...................Resizing... 16 17 24 Resizing... 20 7 35 Resizing... 23 7 47 Resizing... 24 25 58 Resizing... 28 7 68 Resizing... 28 27 73 .....Use minimum degree ordering on A'+A. .................Resizing... 16 17 24 Resizing... 20 7 35 Resizing... 23 7 47 Resizing... 24 25 58 Resizing... 28 7 68 Resizing... 28 27 73 .....Use minimum degree ordering on A'+A. ......................................C:\PYTHON23\lib\site-packages\scipy\interpolate\fitpack2.py:410: UserWarning: The coefficients of the spline returned have been computed as the minimal norm least-squares solution of a (numerically) rank deficient system (deficiency=7). If deficiency is large, the results may be inaccurate. Deficiency may strongly depend on the value of eps. warnings.warn(message) ....................Ties preclude use of exact statistic. ..Ties preclude use of exact statistic. ........ **************************************************************** WARNING: clapack module is empty ----------- See scipy/INSTALL.txt for troubleshooting. Notes: * If atlas library is not found by numpy/distutils/system_info.py, then scipy uses flapack instead of clapack. **************************************************************** ....................data-ftype: z compared to data D Calling _superlu.zgssv Use minimum degree ordering on A'+A. .data-ftype: c compared to data F Calling _superlu.cgssv Use minimum degree ordering on A'+A. .data-ftype: d compared to data d Calling _superlu.dgssv Use minimum degree ordering on A'+A. .data-ftype: s compared to data f Calling _superlu.sgssv Use minimum degree ordering on A'+A. ........................................................................................................................ ........................................................................................................................ ....................................Gegenbauer, a = 2.7337413228 ........................................................................................................................ .............caxpy:n=4 ..caxpy:n=3 ....ccopy:n=4 ..ccopy:n=3 .............cscal:n=4 ....cswap:n=4 ..cswap:n=3 .....daxpy:n=4 ..daxpy:n=3 ....dcopy:n=4 ..dcopy:n=3 .............dscal:n=4 ....dswap:n=4 ..dswap:n=3 .....saxpy:n=4 ..saxpy:n=3 ....scopy:n=4 ..scopy:n=3 .............sscal:n=4 ....sswap:n=4 ..sswap:n=3 .....zaxpy:n=4 ..zaxpy:n=3 ....zcopy:n=4 ..zcopy:n=3 .............zscal:n=4 ....zswap:n=4 ..zswap:n=3 ...Result may be inaccurate, approximate err = 4.1928136851e-009 ...Result may be inaccurate, approximate err = 7.27595761418e-012 .............................................F................Residual: 1.05006950319e-007 . **************************************************************** WARNING: cblas module is empty ----------- See scipy/INSTALL.txt for troubleshooting. Notes: * If atlas library is not found by numpy/distutils/system_info.py, then scipy uses fblas instead of cblas. **************************************************************** ............................................................................................ ====================================================================== FAIL: check_simple (scipy.optimize.tests.test_cobyla.test_cobyla) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python23\Lib\site-packages\scipy\optimize\tests\test_cobyla.py", line 20, in check_simple assert_almost_equal(x, [x0, x1], decimal=5) File "C:\Python23\Lib\site-packages\numpy\testing\utils.py", line 152, in assert_almost_equal return assert_array_almost_equal(actual, desired, decimal, err_msg) File "C:\Python23\Lib\site-packages\numpy\testing\utils.py", line 222, in assert_array_almost_equal header='Arrays are not almost equal') File "C:\Python23\Lib\site-packages\numpy\testing\utils.py", line 207, in assert_array_compare assert cond, msg AssertionError: Arrays are not almost equal (mismatch 100.0%) x: array([ 4.957975 , 0.64690335]) y: array([ 4.95535625, 0.66666667]) ---------------------------------------------------------------------- Ran 1552 tests in 5.828s FAILED (failures=1) <unittest.TextTestRunner object at 0x02671210> >>> Dr. John N. Carter. E-Mail : jnc(a)ecs.soton.ac.uk Building 1, Room 2005 http://www.ecs.soton.ac.uk/~jnc/ Information: Signals, Images, Systems Phone : +44 (0) 23 8059 2405 School of Electronics & Computer Science, Fax : +44 (0) 23 8059 4498 Southampton University, Hants, UK SO17 1BJ.

2 1

Ctypes support in NumPy
by Travis Oliphant 04 Jul '06

04 Jul '06

I've been playing a bit with ctypes and realized that with a little help, it could be made much easier to interface with NumPy arrays. Thus, I added a ctypes attribute to the NumPy array. If ctypes is installed, this attribute returns a "conversion" object otherwise an AttributeError is raised. The ctypes-conversion object has attributes which return c_types aware objects so that the information can be passed directly to c-code (as an integer, the number of dimensions can already be passed using c-types). The information available and it's corresponding c_type is data - c_void_p shape, strides - c_int * nd or c_long * nd or c_longlong * nd depending on platform -Travis

3 13

iterate along a ray: linear algebra?
by stephen emslie 04 Jul '06

04 Jul '06

I am in the process of implementing an image processing algorithm that requires following rays extending outwards from a starting point and calculating the intensity derivative at each point. The idea is to find the point where the difference in intensity goes beyond a particular threshold. Specifically I'm examining an image of an eye to find the pupil, and the edge of the pupil is a sharp change in intensity. How does one iterate along a line in a 2d matrix, and is there a better way to do this? Is this a problem that linear algebra can help with? Thanks Stephen Emslie

3 3

Args for ones, zeros, rand, eye, ones, empty (possible 1.0 change?)
by Bill Baxter 04 Jul '06

04 Jul '06

Here's another possible now or never change: fix rand(), eye(), ones(), zeros(), and empty() to ALL take either a tuple argument or plain list. I know this has been discussed before, but I really don't see why these methods can't be overloaded to accept either one. Here are some wrappers I cooked up that basically do the trick, with a few exceptions which could easily be ironed out with a little input from someone who knows python better than me. I've got versions of rand(), eye() and ones() below. empty() and zeros() should basically be the same as ones(). Actually I guess it's not a "now or never change", because this should be completely backwards compatible. But still I find myself very frequently typing things like ones(2,2), and I'm getting tired of it not working.

9 17

Research study on CVS - you can help and get paid!
by Eric Gilbert 04 Jul '06

04 Jul '06

I am a Computer Science graduate student at the University of Illinois at Urbana-Champaign. I need your help in a study I want to conduct. I have created a project called LifeSource that visualizes CVS repositories. You can find screenshots and info here: http://social.cs.uiuc.edu/people/gilbert (info on me) http://social.cs.uiuc.edu/people/gilbert/research.html (info on LifeSource, the CVS project) LifeSource appeared as a Work-in-Progress paper at ACM's CHI 2006 in Montreal (http://www.chi2006.org). I applied LifeSource to the Azureus and Gaim CVS repositories. I would like to apply the same visualization to your project. Next, I want to see if you as the visualized community find it accurate, helpful, disturbing, etc. In order to participate in the study, you will need to view the LifeSource visualization once a week (on the web) for four weeks and respond to two short online surveys. You will receive a $20 gift certificate for your participation. I can fully explain the process in the near future - just email me back. I will provide all of the information of the methodology of the study online. (We can set up a blog for study members to leave further comments.) Please let me know what you think. I plan on using the study in a long paper to CHI 2007. Cheers, Eric P.S. I apologize for the spam and will send no more.

1 0

Does zero based indexing drive anyone else crazy?
by Webb Sprague 04 Jul '06

04 Jul '06

Hi Numpeans, I have been working on a web-based scientific application for about a year, most of which had been written in either Matlab or SPLUS/R. My task has been to make it "driveable" through an online interface (if anyone cares about mortality forecasting, drop me an email and we can chat about it offline). I chose Python/Numpy for the language because Python and Numpy are both so full featured and easy to work with (except for one little thing...), and neither Matlab nor R could gracefully deal with CGI programming (misguided propaganda notwithstanding). However.... I have spent a huge amount of my time fixing and bending my head around off-by-one errors caused by trying to index matrices using 0 to n-1. The problem is two-fold (threefold if you count my limited IQ...): one, all the formulas in the literature use 1 to n indexing except for some small exceptions. Second and more important, it is far more natural to program if the indices are aligned with the counts of the elements (I think there is a way to express that idea in modern algebra but I can't recall it). This lets you say "how many are there? Three--ok, grab the third one and do whatever to it" etc. Or "how many? zero--ok don't do anything". With zero-based indexing, you are always translating between counts and indices, but such translation is never a problem in one-based indexing. Given the long history of python and its ancestry in C (for which zero based indexing made lots of sense since it dovetailed with thinking in memory offsets in systems programming), there is probably nothing to be done now. I guess I just want to vent, but also to ask if anyone has found any way to deal with this issue in their own scientific programming. Or maybe I am the only with this problem, and if I were a real programmer would translate into zero indexing without even noticing.... Anyway, thanks for listening...

5 4

Time for beta1 of NumPy 1.0
by Travis Oliphant 03 Jul '06

03 Jul '06

I think it's time for the first beta-release of NumPy 1.0 I'd like to put it out within 2 weeks. Please make any comments or voice major concerns so that the 1.0 release series can be as stable as possible. -Travis

38 86