Mailman 3 June 2013 - NumPy-Discussion

f2py: Exposing parameters from "used" modules
by Jeremy Solbrig June 18, 2013

June 18, 2013

Hi all, I assume that this question is addressed somewhere, but I have spent an inordinate amount of time looking around for the answer including digging into the source code a bit. I have tried to put the basic problem in the first paragraph. The rest of the email shows a basic example of the problem. This question is also available here: http://stackoverflow.com/questions/17156125/f2py-exposing-parameters-from-u… I am attempting to compile a module that contains a "use" statement that points to another, more general, module. I would prefer to keep the used module separate so that it can be used in several "packages" as a set of general settings. When I compile the two modules using f2py everything works as advertised from the fortran side, but from the python side the use statement appears to be ignored. If I allow f2py to generate a signature file, the file contains a use statement as is appropriate, but if I complete the compilation and import from the resulting library the parameters from the used module are not available in the module that contains the use statement. Below are two modules illustrating the situation: #Contents of test.f90 MODULE test INTEGER, PARAMETER :: a = 1 END MODULE test #Contents of test2.f90 MODULE test2 USE test INTEGER, PARAMETER :: b = 2 END MODULE test2 In order to show the intermediate step I ran "f2py -h test.pyf test.f90 test2.f90". The following signature file is generated; note that the "test2" module contains "use test": ! -*- f90 -*- ! Note: the context of this file is case sensitive. python module test ! in interface ! in :test module test ! in :test:test.f90 integer, parameter,optional :: a=1 end module test module test2 ! in :test:test2.f90 use test integer, parameter,optional :: b=2 end module test2 end interface end python module test ! This file was auto-generated with f2py (version:2). ! See http://cens.ioc.ee/projects/f2py2e/ If I now compile with "f2py --fcompiler=gfortran -c test.pyf test.f90 test2.f90" I obtain test.so (same as running "f2py --fcompiler=gfortran -m test -c test.f90 test2.f90 without creating the signature file first). Importing from this library in python exposes test.test.a and test.test2.b, but does not expose test.test2.a as can be seen here: In [1]: import test In [2]: print test.test.a 1 In [3]: print test.test2.b 2 In [4]: print test.test2.a --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /users/solbrig/svn_checkouts/inversion/satmet/branches/solbrig/rootpath/data/users/GeoIPS/src/test/<ipython-input-4-bffcf464e408> in <module>() ----> 1 print test.test2.a AttributeError: a Just to illustrate that "b" is defined properly in test2 from the perspective of fortran, the following code uses test2 and prints both "a" and "b": SUBROUTINE run_test() USE test2 IMPLICIT NONE print *, "a = ", a print *, "b = ", b END SUBROUTINE run_test After compiling with "f2py -m run_test -c test.f90 test2.f90 run_test.f90" and obtaining run_test.so, run_test can be imported in python and works as expected: In [1]: import run_test In [2]: run_test.run_test() a = 1 b = 2 Any help with this issue would be greatly appreciated. Thanks, Jeremy

1 0

sampling arrays
by Sudheer Joseph June 18, 2013

June 18, 2013

Dear experts, Is it possible to sample a 4D array of numpy at given dimensions with out writing loops? ie a smart python way? for example, I have a data cube with cube=c(x,y,z,t) dimensions which are essentially latitude longitude depth and time. Is it possible to sample this array at a given dimensions ie sample1= sample(c,:,:,1,1) this is sample at surface at time 1 sample2= sample(c,:,:,1,:) this is time series of surface data sample3= sample(c,1,1,:,1) this is a profile at given location and time sample4= sample(c,1,1,:,:) this is time series at given location at all depths with best regards, Sudheer *************************************************************** Sudheer Joseph Indian National Centre for Ocean Information Services Ministry of Earth Sciences, Govt. of India POST BOX NO: 21, IDA Jeedeemetla P.O. Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55 Tel:+91-40-23886047(O),Fax:+91-40-23895011(O), Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile) E-mail:sjo.India@gmail.com;sudheer.joseph@yahoo.com Web- http://oppamthadathil.tripod.com ***************************************************************

3 5

saving the lower half of matrix
by Bala subramanian June 17, 2013

June 17, 2013

Friends, I have to save only the lower half of a symmetric matrix to a file. I used numpy.tril to extract the lower half. However when i use 'numpy.savetxt', numpy saves the whole matrix (with upper half values replaced as zeros) rather than only the lower half. Any better idea to achieve this. as an example >>>import numpy as np >>> d=np.arange(1,26,1) >>> d=d.reshape(5,5) >>> np.tril(d) array([[ 1, 0, 0, 0, 0], [ 6, 7, 0, 0, 0], [11, 12, 13, 0, 0], [16, 17, 18, 19, 0], [21, 22, 23, 24, 25]]) >>> np.savetxt( 'test', np.tril(d)) output test also contains the zeros, what i want is only the lower half like below. 1 6 7 11 12 13 - - - - etc Thanks, Bala -- C. Balasubramanian

3 2

saving 3d array
by Sudheer Joseph June 17, 2013

June 17, 2013

Dear Experts, I was trying to save results of eof analysis to an npz file and see that it is not possible to save a 3d array as npz file variable. Is this true even today or are there developments which make this possible? I am able to save all other variables except the 3d array. with best regards, Sudheer eof1.shape Out[61]: (10, 140, 180) pc1.shape Out[62]: (2192, 10) x.shape Out[63]: (140, 180) np.savez(outf,pcs=pc1,x=x,y=y,clevs=clevs) works fine but below do not np.savez(outf,eof1=eof1) --------------------------------------------------------------------------- NotImplementedError Traceback (most recent call last) /home/sjo/RAMA_20120807/adcp/<ipython-input-66-fe5940e44a77> in <module>() ----> 1 np.savez(outf,eof1=eof1) /usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/lib/npyio.pyc in savez(file, *args, **kwds) 514 515 """ --> 516 _savez(file, args, kwds, False) 517 518 def savez_compressed(file, *args, **kwds): /usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/lib/npyio.pyc in _savez(file, args, kwds, compress) 573 fid = open(tmpfile, 'wb') 574 try: --> 575 format.write_array(fid, np.asanyarray(val)) 576 fid.close() 577 fid = None /usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/lib/format.pyc in write_array(fp, array, version) 407 else: 408 if isfileobj(fp): --> 409 array.tofile(fp) 410 else: 411 # XXX: We could probably chunk this using something like /usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/ma/core.pyc in tofile(self, fid, sep, format) 5380 5381 """ -> 5382 raise NotImplementedError("Not implemented yet, sorry...") 5383 5384 def toflex(self): NotImplementedError: Not implemented yet, sorry... *************************************************************** Sudheer Joseph Indian National Centre for Ocean Information Services Ministry of Earth Sciences, Govt. of India POST BOX NO: 21, IDA Jeedeemetla P.O. Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55 Tel:+91-40-23886047(O),Fax:+91-40-23895011(O), Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile) E-mail:sjo.India@gmail.com;sudheer.joseph@yahoo.com Web- http://oppamthadathil.tripod.com ***************************************************************

5 8

please close 611, 629, 2490, 2264
by Eric Firing June 17, 2013

June 17, 2013

Github issues 611, 629, and 2490 are duplicates. 611 included patches with a test and a fix, both of which were committed long ago, so all three issues should be closed. Please see my comment on 2264 as to why that should be closed. On 1417, please remove the "component:numpy.ma" label and add the "component:matrixlib" label. Eric

2 1

bug fixes: which branch?
by Eric Firing June 16, 2013

June 16, 2013

What is the preferred strategy for handling bug fix PRs? Initial fix on master, and then a separate PR to backport to v1.7.x? Or the reverse? It doesn't look like v1.7.x is being merged into master regularly, so the matplotlib pattern (fix on maintenance, merge maintenance into master) seems not to be used here. Thanks. Eric

2 1

TypeError when multiplying float64 and a big integer in Python 3.
by Warren Weckesser June 16, 2013

June 16, 2013

With Python 3.3.2 (64 bit), and numpy master: >>> import numpy as np >>> np.__version__ '1.8.0.dev-2a5c2c8' >>> f = np.float64(1.0) >>> i = 2**65 >>> f*i Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for *: 'numpy.float64' and 'int' Is this the expected behavior? The error does not occur with integers that fit in 64 bits: >>> f*10 10.0 It also does not occur in numpy 1.7.1. Warren

1 1

Seg. fault when running tests
by Warren Weckesser June 15, 2013

June 15, 2013

I'm getting a seg. fault in master when I run the tests. I'm on Ubuntu 12.04 64 bit, with Python 3.3.2 (64 bits): $ python3 -c "import numpy as np; np.test('full')" Running unit tests for numpy NumPy version 1.8.0.dev-fa5bc1c NumPy is installed in /home/warren/local_py332/lib/python3.3/site-packages/numpy Python version 3.3.2 (default, Jun 14 2013, 12:12:22) [GCC 4.6.3] nose version 1.3.0 .............................S.........................................................................................................................................................S.......................................................................................................................................................S..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................K............................................................................................................................................................................................SSS.................................................................................................K.....................................................................................................................................................................................Segmentation fault The seg. fault is occurring in ma/tests/test_mrecords.py: $ nosetests test_mrecords.py .Segmentation fault More info: $ python3 Python 3.3.2 (default, Jun 14 2013, 12:12:22) [GCC 4.6.3] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import numpy as np >>> np.show_config() atlas_threads_info: library_dirs = ['/usr/lib/atlas-base/atlas', '/usr/lib/atlas-base'] include_dirs = ['/usr/include/atlas'] language = f77 libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas'] define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] atlas_blas_threads_info: library_dirs = ['/usr/lib/atlas-base'] include_dirs = ['/usr/include/atlas'] language = c libraries = ['ptf77blas', 'ptcblas', 'atlas'] define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] mkl_info: NOT AVAILABLE lapack_opt_info: library_dirs = ['/usr/lib/atlas-base/atlas', '/usr/lib/atlas-base'] include_dirs = ['/usr/include/atlas'] language = f77 libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas'] define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] blas_opt_info: library_dirs = ['/usr/lib/atlas-base'] include_dirs = ['/usr/include/atlas'] language = c libraries = ['ptf77blas', 'ptcblas', 'atlas'] define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] lapack_mkl_info: NOT AVAILABLE blas_mkl_info: NOT AVAILABLE >>> gdb: $ gdb python3 GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04 Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html > This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://bugs.launchpad.net/gdb-linaro/>... Reading symbols from /home/warren/local_py332/bin/python3...done. (gdb) run test_mrecords.py Starting program: /home/warren/local_py332/bin/python3 test_mrecords.py [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". . Program received signal SIGSEGV, Segmentation fault. 0x00007ffff5f080a4 in _aligned_strided_to_contig_size8_srcstride0 (dst=<optimized out>, dst_stride=<optimized out>, src=<optimized out>, __NPY_UNUSED_TAGGEDsrc_stride=<optimized out>, N=<optimized out>, __NPY_UNUSED_TAGGEDsrc_itemsize=<optimized out>, __NPY_UNUSED_TAGGEDdata=0x0) at numpy/core/src/multiarray/lowlevel_strided_loops.c.src:190 190 *((@type@ *)dst) = temp; (gdb) Warren

3 14

time to revisit NA/ma ideas
by Eric Firing June 15, 2013

June 15, 2013

A nice summary of the discussions from a year ago is here: http://www.numpy.org/NA-overview.html It provides food for thought. Eric

3 2

Re: [Numpy-discussion] Dropping support for, Accelerate/veclib?
by Nils Becker June 15, 2013

June 15, 2013

> I think for Scipy homebrew uses the Gfortran ABI: > https://trac.macports.org/browser/trunk/dports/python/py-scipy/Portfile fwiw, homebrew is not macports. it's a more recent replacement that seems to be taking over gradually.

5 6