Mailman 3 November 2010 - NumPy-Discussion

Where did the github numpy repository go?
by Charles R Harris 22 Nov '10

22 Nov '10

I keep getting page does not exist. Chuck

3 2

Yet more merges in maintenance/1.5.x
by Pauli Virtanen 21 Nov '10

21 Nov '10

Hi, There are more unnecessary merge commits in the git history. Before you do "git push", check git log --oneline --graph git log --oneline --graph upstream/maintenance/1.5.x.. so that you don't push unnecessary stuff. (Also, don't use "git pull".) $ git log --oneline --graph upstream/maintenance/1.5.x * cc8d516 BUG: Fix exception handling for python 3k. * 15b875a Merge branch 'maintenance/1.5.x' of github.com:numpy/numpy into |\ | * de969d7 REL: set version to 1.5.1. * | 3d429f1 Merge branch 'maintenance/1.5.x' of github.com:numpy/numpy in |\ \ | |/ | * 3304791 REL: set version number to 1.5.1rc2 and set released=True. | * e885a7a TST: mark longdouble tests as knownfail on OS X PPC. | * 516aa4a TST: mark longdouble tests for spacing/nextafter as knownfail o | * 08ccb3a TST: silence ldexp overflow warning. | * 8346ba0 BUG: fix issue with incorrect Fortran arch flags. Closes #1399. | * c0bd3df TST: core: disable C99 complex tests also on Solaris if it seem | * c6504f5 TST: core: mark test_ldexp_overflow as known failure on Python | * 082956b TST: core: fix test_fromfile_tofile_seeks to work on Windows (c * | f819262 Merge branch 'maintenance/1.5.x' of github.com:numpy/numpy in |\ \ | |/ | * 29c38c6 REL: set released=False again, and minor fix in paver script. | * 97cb28e REL: set version number to 1.5.1rc1, released=True. | * 4b18dc8 REL: Set start and end tags for the Changelog. | * 2aa0317 REL: make the OS X installer naming scheme correspond to what i | * 3bc3174 BUG: on Windows the sysconfig module does not contain CFLAGS in | * c9f2514 REL: add a note on #1399 to the 1.5.1 release notes. * | c035b13 DOC: recommend to turn on deprecation warnings for Python >= 2. |/ * 427d3fc Merge branch 'maintenance/1.5.x' of github.com:numpy/numpy into |\ | * b55eacd BUG: core: implement a long-int loop for ldexp, for cases where | * 0e792e6 BUG: core: adjust ComplexWarning location frame up by one, so t | * 4e177d3 BUG: get fortran arch flags from C arch flags if available. Clo | * c8e1315 BUG: DOC: fix invalid vdot documentation (cherry picked from co

2 2

problem with numpy/cython on python3, ok with python2
by Darren Dale 21 Nov '10

21 Nov '10

I just installed numpy for both python2 and 3 from an up-to-date checkout of the 1.5.x branch. I am attempting to cythonize the following code with cython-0.13: --- cimport numpy as np import numpy as np def test(): cdef np.ndarray[np.float64_t, ndim=1] ret ret_arr = np.zeros((20,), dtype=np.float64) ret = ret_arr --- I have this setup.py file: --- from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext import numpy setup( cmdclass = {'build_ext': build_ext}, ext_modules = [ Extension( "test_open", ["test_open.pyx"], include_dirs=[numpy.get_include()] ) ] ) --- When I run "python setup.py build_ext --inplace", everything is fine. When I run "python3 setup.py build_ext --inplace", I get an error: running build_ext cythoning test_open.pyx to test_open.c Error converting Pyrex file to C: ------------------------------------------------------------ ... # For use in situations where ndarray can't replace PyArrayObject*, # like PyArrayObject**. pass ctypedef class numpy.ndarray [object PyArrayObject]: cdef __cythonbufferdefaults__ = {"mode": "strided"} ^ ------------------------------------------------------------ /home/darren/.local/lib/python3.1/site-packages/Cython/Includes/numpy.pxd:173:49: "mode" is not a buffer option Error converting Pyrex file to C: ------------------------------------------------------------ ... cimport numpy as np import numpy as np def test(): cdef np.ndarray[np.float64_t, ndim=1] ret ^ ------------------------------------------------------------ /home/darren/temp/test/test_open.pyx:6:8: 'ndarray' is not a type identifier building 'test_open' extension gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/home/darren/.local/lib/python3.1/site-packages/numpy/core/include -I/usr/include/python3.1 -c test_open.c -o build/temp.linux-x86_64-3.1/test_open.o test_open.c:1: error: #error Do not use this file, it is the result of a failed Cython compilation. error: command 'gcc' failed with exit status 1 Is this a bug, or is there a problem with my example? Thanks, Darren

2 1

changing 'nan' string for np.nan in np.savetxt
by Maria Liukis 20 Nov '10

20 Nov '10

Hello everybody, I was wondering if there is an elegant way of overwriting 'nan' string representation with 'NaN' when saving numpy array containing np.nan values with numpy.savetxt() function. numpy.set_printoptions(nanstr='NaN') setting (which has default value of 'NaN' already) does not seem to have any effect when writing array to the file, only when printing to the screen: $ python Python 2.5.2 (r252:60911, Sep 30 2008, 15:42:03) [GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy as np >>> print np.__version__ 1.3.0 >>> np.get_printoptions() {'infstr': 'Inf', 'threshold': 1000, 'suppress': False, 'linewidth': 75, 'edgeitems': 3, 'precision': 8, 'nanstr': 'NaN'} >>> >>> a = np.array([1, 2, 3, np.nan]) >>> a array([ 1., 2., 3., NaN]) >>> np.savetxt('testa.txt', a) >>> $ cat testa.txt 1.000000000000000000e+00 2.000000000000000000e+00 3.000000000000000000e+00 nan $ Many thanks, Masha

1 0

seeking advice on a fast string->array conversion
by Darren Dale 20 Nov '10

20 Nov '10

I am wrapping up a small package to parse a particular ascii-encoded file format generated by a program we use heavily here at the lab. (In the unlikely event that you work at a synchrotron, and use Certified Scientific's "spec" program, and are actually interested, the code is currently available at https://github.com/darrendale/praxes/tree/specformat/praxes/io/spec/ .) I have been benchmarking the project against another python package developed by a colleague, which is an extension module written in pure C. My python/cython project takes about twice as long to parse and index a file (~0.8 seconds for 100MB), which is acceptable. However, actually converting ascii strings to numpy arrays, which is done using numpy.fromstring, takes a factor of 10 longer than the extension module. So I am wondering about the performance of np.fromstring: import time import numpy as np s = b'1 ' * 2048 *1200 d = time.time() x = np.fromstring(s) print time.time() - d

4 11

Advise for numerical programming content (New python user)
by Sachin Kumar Sharma 19 Nov '10

19 Nov '10

Users, I am an average Fortran user. I am new to python and I am currently evaluating options and functionalities of numerical programming and related 2d and 3d graphic outputs with python. Kindly share your experience in scientific programming with python like how do you like it, comparison with Fortran and C++. Which version of python + numpy+scipy are compatible with each other or if any other numerical analysis package is available (I am working on windows environment.) Does graphic output like maps, histogram, crossplot, tornado charts is good enough with basic installation or needs some additional packages? Your feedback is valuable for me to start. Thanks & Regards Sachin ************************************************************************ Sachin Kumar Sharma Senior Geomodeler - Samarang Project (IPM) Field Development & Production Services (DCS) Schlumberger Sdn. Bhd., 7th Floor, West Wing, Rohas Perkasa, No. 8 Jalan Perak, Kuala Lumpur, 50450, Malaysia Mobile: +60 12 2196443 * Email: ssharma84(a)exchange.slb.com<mailto:ssharma84@bombay.oilfield.slb.com> sachin_sharma(a)petronas.com.my

4 4

Test failures on 2.6
by T J 19 Nov '10

19 Nov '10

Hi, I'm getting a couple of test failures with Python 2.6, Numpy 1.2.0, Nose 0.10.4: nose version 0.10.4 ..........................................................................................................................................................................................................................................................................................................................................................................................................................................F................K........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................./share/home/me/usr/lib/python2.6/site-packages/numpy/lib/tests/test_io.py:68: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(c.readlines(), ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................./share/home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py:1315: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(store._mask, True) /home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py:1322: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(store._mask, True) /home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py:1989: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(test.mask, [0,1,0,0,0,0,0,0,0,0]) ...............................................E................................................................................................................................................................................ ====================================================================== ERROR: Tests the min/max functions with explicit outputs ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py", line 653, in test_minmax_funcs_with_output result = npfunc(xm,axis=0,out=nout) File "/home/me/usr/lib/python2.6/site-packages/numpy/core/fromnumeric.py", line 1525, in amin return amin(axis, out) File "/home/me/usr/lib/python2.6/site-packages/numpy/ma/core.py", line 2978, in min np.putmask(out, newmask, np.nan) ValueError: cannot convert float NaN to integer ====================================================================== FAIL: test_umath.TestComplexFunctions.test_against_cmath ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/me/usr/lib/python2.6/site-packages/nose-0.10.4-py2.6.egg/nose/case.py", line 182, in runTest self.test(*self.arg) File "/home/me/usr/lib/python2.6/site-packages/numpy/core/tests/test_umath.py", line 268, in test_against_cmath assert abs(a - b) < atol, "%s %s: %s; cmath: %s"%(fname,p,a,b) AssertionError: arcsin 2: (1.57079632679-1.31695789692j); cmath: (1.57079632679+1.31695789692j) ---------------------------------------------------------------------- Ran 1726 tests in 8.856s FAILED (KNOWNFAIL=1, errors=1, failures=1) <nose.result.TextTestResult run=1726 errors=1 failures=1>

2 1

Re: [Numpy-discussion] Test failures on 2.6
by W Bradley Knox 19 Nov '10

19 Nov '10

I'm having almost exactly the same problem, but with Python 2.6.1, Numpy 1.2.1, and Nose 0.11.3. Nobody responded to TJ the first time around, so any advice would be greatly appreciated. Thanks, Brad ---------------------------------------------------------------------- From: T J <tjhnson <at> gmail.com> Subject: Test failures on 2.6 Newsgroups: gmane.comp.python.numeric.general Date: 2008-10-05 20:53:22 GMT (2 years, 6 weeks, 1 day, 13 hours and 32 minutes ago) Hi, I'm getting a couple of test failures with Python 2.6, Numpy 1.2.0, Nose 0.10.4: nose version 0.10.4 ..........................................................................................................................................................................................................................................................................................................................................................................................................................................F................K.................................................................................................................................................................................................................................................................................................................................................................... .............................................................................................................................................................................................. ......./share/home/me/usr/lib/python2.6/site-packages/numpy/lib/tests/test_io.py:68: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(c.readlines(), ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................./share/home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py:1315: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(store._mask, True) /home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py:1322: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(store._mask, True) /home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py:1989: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(test.mask, [0,1,0,0,0,0,0,0,0,0]) ...............................................E................................................................................................................................................................................ ====================================================================== ERROR: Tests the min/max functions with explicit outputs ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/me/usr/lib/python2.6/site-packages/numpy/ma/tests/test_core.py", line 653, in test_minmax_funcs_with_output result = npfunc(xm,axis=0,out=nout) File "/home/me/usr/lib/python2.6/site-packages/numpy/core/fromnumeric.py", line 1525, in amin return amin(axis, out) File "/home/me/usr/lib/python2.6/site-packages/numpy/ma/core.py", line 2978, in min np.putmask(out, newmask, np.nan) ValueError: cannot convert float NaN to integer ====================================================================== FAIL: test_umath.TestComplexFunctions.test_against_cmath ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/me/usr/lib/python2.6/site-packages/nose-0.10.4-py2.6.egg/nose/case.py", line 182, in runTest self.test(*self.arg) File "/home/me/usr/lib/python2.6/site-packages/numpy/core/tests/test_umath.py", line 268, in test_against_cmath assert abs(a - b) < atol, "%s %s: %s; cmath: %s"%(fname,p,a,b) AssertionError: arcsin 2: (1.57079632679-1.31695789692j); cmath: (1.57079632679+1.31695789692j) ---------------------------------------------------------------------- Ran 1726 tests in 8.856s FAILED (KNOWNFAIL=1, errors=1, failures=1) <nose.result.TextTestResult run=1726 errors=1 failures=1>

1 0

How to import input data to make ndarray for batch processing?
by Venkat 19 Nov '10

19 Nov '10

Hi All, I am new to Numpy (also Scipy). I am trying to reshape my text data which is in one single column (10,000 rows). I want the data to be in 100x100 array form. I have many files to convert like this. All of them are having file names like 0, 1, 2, ....500. with out any extension. Actually, I renamed actual files so that I can import them in Matlab for batch processing. Since Matlab also new for me, I thought I will try Numpy first. Can any body help me in writing the script to do this for making batch processing. Thanks in advance, Venkat -- ******************************************************************************* D.Venkat Research Scholar Dept of Physics IISc, Bangalore India-560 012 ********************************************************************************

6 5

Returning numpy scalars in cython functions
by Keith Goodman 19 Nov '10

19 Nov '10

The cython function below returns a long int: @cython.boundscheck(False) def mysum(np.ndarray[np.int64_t, ndim=1] a): "sum of 1d numpy array with dtype=np.int64." cdef Py_ssize_t i cdef int asize = a.shape[0] cdef np.int64_t asum = 0 for i in range(asize): asum += a[i] return asum What's the best way to make it return a numpy long int, or whatever it is called, that has dtype, ndim, size, etc. class methods? The only thing I could come up with is changing the last line to return np.array(asum)[()] It works. And adds some overhead: >> a = np.arange(10) >> timeit mysum(a) 10000000 loops, best of 3: 167 ns per loop >> timeit mysum2(a) 1000000 loops, best of 3: 984 ns per loop And for scale: >> timeit np.sum(a) 100000 loops, best of 3: 3.3 us per loop I'm new to cython. Did I miss any optimizations in the mysum function above?

2 3

NumPy-Discussion November 2010