Mailman 3 September 2011 - NumPy-Discussion

1.7.x release of NumPy
by Travis Oliphant 15 Sep '11

15 Sep '11

Hi all, Has there been a discussion of a 1.7.x release of NumPy? There are a few new features that should go into the 1.x release of NumPy, that don't require the ABI changes of 2.0. I thought I had heard Mark talk in support of such a thing. What are the plans for the release of NumPy 2.0? Between my experiences over the last 4 years of working on real-world projects, the data array summit, SciPy, and conversations with Mark Wiebe, Peter Wang, Wes McKinney and others over the past months, there is a collection of pretty concrete ideas emerging about where the next version of NumPy should go. Some of these can be folded into NumPy 2.0 (and even 1.7), but my mind is racing around NumPy 3.0 at this point. -Travis

5 4

[ANN} OpenOpt, FuncDesigner, DerApproximator, SpaceFuncs release 0.36
by Dmitrey 15 Sep '11

15 Sep '11

Hi all, new release of our free soft (OpenOpt, FuncDesigner, DerApproximator, SpaceFuncs) v. 0.36 is out: OpenOpt: > * Now solver interalg can handle all types of constraints and integration problems > * Some minor improvements and code cleanup > FuncDesigner: > * Interval analysis now can involve min, max and 1-d monotone splines R -> R of 1st and 3rd order > * Some bugfixes and improvements > SpaceFuncs: > * Some minor changes > DerApproximator: > * Some improvements for obtaining derivatives in points from R^n where left or right derivative for a variable is absent, especially for stencil > 1 > See http://openopt.org for more details. Regards, D.

1 0

Re: [Numpy-discussion] Functions for finding the relative extrema of numeric data
by Jacob Silterra 15 Sep '11

15 Sep '11

>What is your application? The most common case is looking at Fourier transforms and identifying spectral peaks. I've also analyzed images looking at 1D slices (usually very regular data) and looked for peaks there. That stackoverflow page had a nice link to a comparison of different algorithms here: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2631518/. That paper is focused on mass-spectrometry data, but the approach would generalize to any 1D data set. Unless somebody feels otherwise, I'll close this pull request and start working on an implementation of peak finding via continuous wavelet transform (the best and most computationally intensive approach of those described above). -Jacob ------------------------------ > > Message: 4 > Date: Tue, 13 Sep 2011 22:34:01 +0200 > From: Ralf Gommers <ralf.gommers(a)googlemail.com> > Subject: Re: [Numpy-discussion] Functions for finding the relative > extrema of numeric data > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: > <CABL7CQhxCX0LKFENMW6-4ZSbdieGxz04zbsrnY4bXYVxVL78Dw(a)mail.gmail.com > > > Content-Type: text/plain; charset="iso-8859-1" > > Hi Jacob, > > On Fri, Sep 9, 2011 at 11:57 PM, Jacob Silterra <jsilter(a)gmail.com> wrote: > > > Hello all, > > > > I'd like to see functions for calculating the relative extrema in a set > of > > data included in numpy. I use that functionality frequently, and always > seem > > to be writing my own version. It seems like this functionality would be > > useful to the community at large, as it's a fairly common operation. > > > > What is your application? > > > > > For numeric data (which is presumably noisy), the definition of a > relative > > extrema isn't completely obvious. The implementation I am proposing finds > a > > point in an ndarray along an axis which is larger (or smaller) than it's > > `order` nearest neighbors (`order` being an optional parameter, default > 1). > > This is likely to find more points than may be desired, which I believe > is > > preferable to the alternative. The output is formatted the same as > > numpy.where. > > > > Code available here: https://github.com/numpy/numpy/pull/154 > > > > I'm not sure whether this belongs in numpy or scipy, that question is > > somewhat debatable. More sophisticated peak-finding functions (in N > > dimensions, as opposed to 1) may also be useful to the community, and > those > > would definitely belong in scipy. > > > > I have the feeling this belongs in scipy. Although if it's just these two > functions I'm not sure where exactly to put them. Looking at the > functionality, this is quite a simple approach. For any data of the type > I'm > usually working with it will not be able to find the right local extrema. > The same is true for your alternative definition below. > > A more powerful peak detection function would be a very good addition to > scipy imho (perhaps in scipy.interpolate?). See also > > http://stackoverflow.com/questions/1713335/peak-finding-algorithm-for-pytho… > > Cheers, > Ralf > > > > An alternative implementation would be to require that function be > > continuously descending (or ascending) for `order` points, which would > > enforce a minimum width on a peak. > > > > -Jacob Silterra > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion(a)scipy.org > > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > >

4 3

load from text files Pull Request Review
by Christopher Jordan-Squire 15 Sep '11

15 Sep '11

Hi-- I've submitted a pull request for a new method for loading data from text files into a record array/masked record array. https://github.com/numpy/numpy/pull/143 Click on the link for more info, but the general idea is to create a regular expression for what entries should look like and loop over the file, updating the regular expression if it's wrong. Once the types are determined the file is loaded line by line into a pre-allocated numpy array. Compared to genfromtxt this function has several advantages/potential advantages. *More modular (genfromtxt is a rather large, nearly 500 line, monolithic function. In my pull request no individual method is longer than around 80 lines, and they're fairly self-contained.) *delimiters can be specified via regex's *missing data can be specified via regex's *it's bit simpler and has sensible defaults *it actually works on some (unfortunately proprietary) data that genfromtxt doesn't seem robust enough for *it supports datetimes *fairly extensible for the power user *makes two passes through the file, the first to determine types/sizes for strings and the second to read in the data, and pre-allocates the array for the second pass. So no giant memory bloating for reading large text files *fairly fast, though I think there is plenty of room for optimizations All that said, it's entirely possible that the innards which determine the type should be ripped out and submitted as a function on their own. I'd love suggestions for improvements, as well as suggestions for a better name. (Currently it's called loadtable, which I don't really like. It was just a working name.) -Chris Jordan-Squire

7 24

numpy.test() failure
by Nils Wagner 14 Sep '11

14 Sep '11

ERROR: test_polyfit (test_polynomial.TestDocs) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/lib/tests/test_polynomial.py", line 106, in test_polyfit weights = arange(8,1,-1)**2/7.0 NameError: global name 'arange' is not defined ====================================================================== FAIL: Tests polyfit ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/ma/tests/test_extras.py", line 622, in test_polyfit assert_almost_equal(a, a_) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/ma/testutils.py", line 155, in assert_almost_equal err_msg=err_msg, verbose=verbose) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/ma/testutils.py", line 221, in assert_array_almost_equal header='Arrays are not almost equal') File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/ma/testutils.py", line 186, in assert_array_compare verbose=verbose, header=header) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 677, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 100.0%) x: array([ 4.25134878, 1.14131297, 0.20519666, 0.01701 ]) y: array([ 1.9345248 , 0.49711011, 0.10202554, 0.00928034])

4 3

Re: [Numpy-discussion] behaviour of np.loadtxt
by Derek Homeier 14 Sep '11

14 Sep '11

On Wed, 14 Sep 2011, Davide wrote: > Dear list, > > I'm encountering a problem with np.loadtxt. > > Suppose i have a file containing three columns of data (and 10 rows), like: > > 0.001 0.003 0.005 > 0.001 0.003 0.006 > 0.002 0.004 0.002 > 0.004 0.002 0.007 > 0.001 0.003 0.006 > 0.002 0.004 0.002 > 0.004 0.002 0.007 > 0.001 0.003 0.006 > 0.002 0.004 0.002 > 0.004 0.002 0.007 > > If i give: > > len ( np.loadtxt( filename, unpack=True ) ) > > i get 3, since i have a three rows array. That's fine. > > If instead the file has a single column, i.e., > > 0.004 > 0.002 > 0.007 > 0.004 > 0.002 > 0.007 > 0.004 > 0.002 > 0.007 > 0.004 > > the command > > len ( np.loadtxt( filename, unpack=True ) ) > > returns 10, where i would expect it to return 1, to be consistent with > the behaviour when there are multiple columns. > > Is there a reason for why it is not like that? Yes, the default for loadtxt is to automatically "squeeze" the output to the minimum no. of dimensions required, thus it returns a 1D-array in the latter case. If you are using numpy 1.6 or later, you can ensure to get a consistent shape by passing the "ndmin=2" option. Cheers, Derek -- ---------------------------------------------------------------- Derek Homeier Centre de Recherche Astrophysique de Lyon ENS Lyon 46, Allée d'Italie 69364 Lyon Cedex 07, France +33 1133 47272-8894 ----------------------------------------------------------------

1 0

behaviour of np.loadtxt
by Davide 14 Sep '11

14 Sep '11

Dear list, I'm encountering a problem with np.loadtxt. Suppose i have a file containing three columns of data (and 10 rows), like: 0.001 0.003 0.005 0.001 0.003 0.006 0.002 0.004 0.002 0.004 0.002 0.007 0.001 0.003 0.006 0.002 0.004 0.002 0.004 0.002 0.007 0.001 0.003 0.006 0.002 0.004 0.002 0.004 0.002 0.007 If i give: len ( np.loadtxt( filename, unpack=True ) ) i get 3, since i have a three rows array. That's fine. If instead the file has a single column, i.e., 0.004 0.002 0.007 0.004 0.002 0.007 0.004 0.002 0.007 0.004 the command len ( np.loadtxt( filename, unpack=True ) ) returns 10, where i would expect it to return 1, to be consistent with the behaviour when there are multiple columns. Is there a reason for why it is not like that? Cheers Davide Lasagna

1 0

Re: [Numpy-discussion] Numpy - MKL - build error
by Igor Ying 14 Sep '11

14 Sep '11

Yes, they all are present in that directory. Also, I tried with root as login. -r-xr-xr-x 1 root root 26342559 Aug 9 22:19 libmkl_avx.so -r--r--r-- 1 root root 1190224 Aug 9 22:26 libmkl_blacs_ilp64.a -r--r--r-- 1 root root 1191496 Aug 9 22:25 libmkl_blacs_intelmpi_ilp64.a -r-xr-xr-x 1 root root 497597 Aug 9 22:25 libmkl_blacs_intelmpi_ilp64.so -r--r--r-- 1 root root 676206 Aug 9 22:21 libmkl_blacs_intelmpi_lp64.a -r-xr-xr-x 1 root root 267010 Aug 9 22:21 libmkl_blacs_intelmpi_lp64.so -r--r--r-- 1 root root 674926 Aug 9 22:22 libmkl_blacs_lp64.a -r--r--r-- 1 root root 1218290 Aug 9 22:28 libmkl_blacs_openmpi_ilp64.a -r--r--r-- 1 root root 703042 Aug 9 22:23 libmkl_blacs_openmpi_lp64.a -r--r--r-- 1 root root 1191152 Aug 9 22:29 libmkl_blacs_sgimpt_ilp64.a -r--r--r-- 1 root root 675854 Aug 9 22:23 libmkl_blacs_sgimpt_lp64.a -r--r--r-- 1 root root 425802 Aug 9 20:44 libmkl_blas95_ilp64.a -r--r--r-- 1 root root 421410 Aug 9 20:44 libmkl_blas95_lp64.a -r--r--r-- 1 root root 144354 Aug 9 22:29 libmkl_cdft_core.a -r-xr-xr-x 1 root root 115588 Aug 9 22:29 libmkl_cdft_core.so -r--r--r-- 1 root root 231886824 Aug 9 22:07 libmkl_core.a -r-xr-xr-x 1 root root 16730033 Aug 9 22:18 libmkl_core.so -r-xr-xr-x 1 root root 21474555 Aug 9 22:18 libmkl_def.so -r--r--r-- 1 root root 14974574 Aug 9 22:06 libmkl_gf_ilp64.a -r-xr-xr-x 1 root root 7008828 Aug 9 22:48 libmkl_gf_ilp64.so -r--r--r-- 1 root root 15140998 Aug 9 22:06 libmkl_gf_lp64.a -r-xr-xr-x 1 root root 7055304 Aug 9 22:48 libmkl_gf_lp64.so -r--r--r-- 1 root root 16435120 Aug 9 22:07 libmkl_gnu_thread.a -r-xr-xr-x 1 root root 9816940 Aug 9 22:49 libmkl_gnu_thread.so -r--r--r-- 1 root root 14968130 Aug 9 22:06 libmkl_intel_ilp64.a -r-xr-xr-x 1 root root 7008368 Aug 9 22:48 libmkl_intel_ilp64.so -r--r--r-- 1 root root 15134406 Aug 9 22:06 libmkl_intel_lp64.a -r-xr-xr-x 1 root root 7053588 Aug 9 22:48 libmkl_intel_lp64.so -r--r--r-- 1 root root 2472940 Aug 9 22:07 libmkl_intel_sp2dp.a -r-xr-xr-x 1 root root 1191479 Aug 9 22:20 libmkl_intel_sp2dp.so -r--r--r-- 1 root root 27642508 Aug 9 22:07 libmkl_intel_thread.a -r-xr-xr-x 1 root root 17516608 Aug 9 22:49 libmkl_intel_thread.so -r--r--r-- 1 root root 5350948 Aug 9 20:44 libmkl_lapack95_ilp64.a -r--r--r-- 1 root root 5413476 Aug 9 20:44 libmkl_lapack95_lp64.a -r-xr-xr-x 1 root root 29543829 Aug 9 22:19 libmkl_mc3.so -r-xr-xr-x 1 root root 25428037 Aug 9 22:19 libmkl_mc.so -r-xr-xr-x 1 root root 22888659 Aug 9 22:18 libmkl_p4n.so -r--r--r-- 1 root root 19232716 Aug 9 22:07 libmkl_pgi_thread.a -r-xr-xr-x 1 root root 12243062 Aug 9 22:49 libmkl_pgi_thread.so -r-xr-xr-x 1 root root 4984870 Aug 9 22:49 libmkl_rt.so -r--r--r-- 1 root root 10367758 Aug 9 22:49 libmkl_scalapack_ilp64.a -r-xr-xr-x 1 root root 6574928 Aug 9 22:50 libmkl_scalapack_ilp64.so -r--r--r-- 1 root root 10292432 Aug 9 22:49 libmkl_scalapack_lp64.a -r-xr-xr-x 1 root root 6452627 Aug 9 22:50 libmkl_scalapack_lp64.so -r--r--r-- 1 root root 9958444 Aug 9 22:07 libmkl_sequential.a -r-xr-xr-x 1 root root 5926347 Aug 9 22:48 libmkl_sequential.so -r--r--r-- 1 root root 1048 Aug 9 16:50 libmkl_solver_ilp64.a -r--r--r-- 1 root root 1048 Aug 9 16:50 libmkl_solver_ilp64_sequential.a -r--r--r-- 1 root root 1048 Aug 9 16:50 libmkl_solver_lp64.a -r--r--r-- 1 root root 1048 Aug 9 16:50 libmkl_solver_lp64_sequential.a -r-xr-xr-x 1 root root 6711968 Aug 9 22:48 libmkl_vml_avx.so -r-xr-xr-x 1 root root 2795928 Aug 9 22:47 libmkl_vml_def.so -r-xr-xr-x 1 root root 5476786 Aug 9 22:48 libmkl_vml_mc2.so -r-xr-xr-x 1 root root 5778052 Aug 9 22:48 libmkl_vml_mc3.so -r-xr-xr-x 1 root root 5382511 Aug 9 22:48 libmkl_vml_mc.so -r-xr-xr-x 1 root root 4235841 Aug 9 22:48 libmkl_vml_p4n.so drwxr-xr-x 3 root root 4096 Aug 18 11:43 locale Y you can reach the person managing the list at numpy-discussion-owner(a)scipy.org Message: 1 Date: Tue, 13 Sep 2011 09:58:27 -0400 From: Olivier Delalleau <shish(a)keba.be> Subject: Re: [Numpy-discussion] Numpy - MKL - build error To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> Message-ID: <CAFXk4bpDN7qcwmZe2G565gsuOntau7LMH4fGSTH_zY_R+NLbMg(a)mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Sorry if it sounds like a stupid question, but are the files listed in the error message present in that directory? If yes, maybe try running the command with sudo, just in case it would be some weird permission issue. -=- Olivier 2011/9/13 Igor Ying <igor.ying(a)yahoo.com> > Hi, > > I am very new to Numpy and trying to build Numpy 1.6.1 with Intel MKL > 10.3.6 and getting the following errors. > > $python setup.py config --compiler=intel build_clib --compiler=intel > build_ext --compiler=intel install > > Running from numpy source directory.F2PY Version 2 > blas_opt_info: > blas_mkl_info: > libraries mkl_def,mkl_intel_lp64,mkl_intel_thread,mkl_core,iomp5md not > found in /opt/intel/composer_xe_2011_sp1/mkl/lib/intel64 > NOT AVAILABLE > > I have modified the site.cfg as below. > > [mkl] > library_dirs = /opt/intel/composer_xe_2011_sp1/mkl/lib/intel64 > include_dirs = /opt/intel/composer_xe_2011_sp1/mkl/include > mkl_libs = mkl_def,mkl_intel_lp64,mkl_intel_thread,mkl_core,iomp5md > lapack_libs = mkl_lapack_lp64 > > > Environment variables are also set to point to mkl libraries. > > source /opt/intel/composer_xe_2011_sp1/bin/compilervars.sh intel64 lp64 > > export OMP_NUM_THREADS=4 > LD_LIBRARY_PATH="/opt/intel/composer_xe_2011_sp1/mkl/lib/intel64:${LD_LIBRARY_PATH}"; > > export LD_LIBRARY_PATH > export > LD_RUN_PATH=/opt/lib:/opt1/composer_xe_2011_sp1/lib/intel64:/opt/composer_xe_2011_sp1/mkl/lib/intel64 > > Can anyone help me? > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > >

2 2

upgrading to 1.6.1, lapack not used?
by Thomas K Gamble 14 Sep '11

14 Sep '11

I am trying to upgrade my numpy installation from 1.5.1 to 1.6.1 on Ubuntu 10.04. When I build, it doesn't seem to use the installed lapack libraries, but uses lapack_lite. After installation (which seems to complete without error), importing numpy fails with an error that it can't find libquadmath.so.0. Any ideas what might be wrong? Am I missing a dependent package somewhere? How can I force the build to use the installed lapack stuff? (installed via synaptic from the ubuntu debs) Numpy 1.5.1 builds and installs with no problems. -- Thomas K. Gamble Research Technologist, System/Network Administrator Chemical Diagnostics and Engineering (C-CDE) Los Alamos National Laboratory MS-E543,p:505-665-4323 f:505-665-4267 "There cannot be a crisis next week. My schedule is already full." Henry Kissinger

2 1

Functions for finding the relative extrema of numeric data
by Jacob Silterra 13 Sep '11

13 Sep '11

Hello all, I'd like to see functions for calculating the relative extrema in a set of data included in numpy. I use that functionality frequently, and always seem to be writing my own version. It seems like this functionality would be useful to the community at large, as it's a fairly common operation. For numeric data (which is presumably noisy), the definition of a relative extrema isn't completely obvious. The implementation I am proposing finds a point in an ndarray along an axis which is larger (or smaller) than it's `order` nearest neighbors (`order` being an optional parameter, default 1). This is likely to find more points than may be desired, which I believe is preferable to the alternative. The output is formatted the same as numpy.where. Code available here: https://github.com/numpy/numpy/pull/154 I'm not sure whether this belongs in numpy or scipy, that question is somewhat debatable. More sophisticated peak-finding functions (in N dimensions, as opposed to 1) may also be useful to the community, and those would definitely belong in scipy. An alternative implementation would be to require that function be continuously descending (or ascending) for `order` points, which would enforce a minimum width on a peak. -Jacob Silterra

3 2

Numpy - MKL - build error
by Igor Ying 13 Sep '11

13 Sep '11

Hi, I am very new to Numpy and trying to build Numpy 1.6.1 with Intel MKL 10.3.6 and getting the following errors. $python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install Running from numpy source directory.F2PY Version 2 blas_opt_info: blas_mkl_info: libraries mkl_def,mkl_intel_lp64,mkl_intel_thread,mkl_core,iomp5md not found in /opt/intel/composer_xe_2011_sp1/mkl/lib/intel64 NOT AVAILABLE I have modified the site.cfg as below. [mkl] library_dirs = /opt/intel/composer_xe_2011_sp1/mkl/lib/intel64 include_dirs = /opt/intel/composer_xe_2011_sp1/mkl/include mkl_libs = mkl_def,mkl_intel_lp64,mkl_intel_thread,mkl_core,iomp5md lapack_libs = mkl_lapack_lp64 Environment variables are also set to point to mkl libraries. source /opt/intel/composer_xe_2011_sp1/bin/compilervars.sh intel64 lp64 export OMP_NUM_THREADS=4 LD_LIBRARY_PATH="/opt/intel/composer_xe_2011_sp1/mkl/lib/intel64:${LD_LIBRARY_PATH}"; export LD_LIBRARY_PATH export LD_RUN_PATH=/opt/lib:/opt1/composer_xe_2011_sp1/lib/intel64:/opt/composer_xe_2011_sp1/mkl/lib/intel64 Can anyone help me?

2 1

using Matlab compiler to build Python-callable library?
by Reckoner 12 Sep '11

12 Sep '11

Hi, Has anybody ever tried using the Matlab compiler to build a standalone library that would be callable using Python? We have a lot of leftover Matlab code that we are trying to migrate. Thanks!

1 0

f2py : NotImplementedError: Only MS compiler supported with gfortran on win64
by Jim Vickroy 12 Sep '11

12 Sep '11

Hello All, I'm attempting to create a python wrapper, for a Fortran subroutine, using f2py. My system details are: >>> sys.version '2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]' >>> sys.getwindowsversion() (5, 1, 2600, 2, 'Service Pack 3') >>> scipy.__version__ '0.7.1' >>> numpy.__version__ '1.4.0' C:\>gfortran -dumpversion 4.7.0 C:\Python26\Lib\site-packages\numpy\f2py>f2py.py -c --help-fcompiler Traceback (most recent call last): File "C:\Python26\Scripts\f2py.py", line 24, in <module> main() File "C:\Python26\lib\site-packages\numpy\f2py\f2py2e.py", line 557, in main run_compile() File "C:\Python26\lib\site-packages\numpy\f2py\f2py2e.py", line 543, in run_compile setup(ext_modules = [ext]) File "C:\Python26\lib\site-packages\numpy\distutils\core.py", line 186, in setup return old_setup(**new_attr) File "C:\Python26\lib\distutils\core.py", line 138, in setup ok = dist.parse_command_line() File "C:\Python26\lib\distutils\dist.py", line 460, in parse_command_line args = self._parse_command_opts(parser, args) File "C:\Python26\lib\distutils\dist.py", line 574, in _parse_command_opts func() File "C:\Python26\lib\site-packages\numpy\distutils\command\config_compiler.py", line 13, in show_fortran_compilers show_fcompilers(dist) File "C:\Python26\lib\site-packages\numpy\distutils\fcompiler\__init__.py", line 855, in show_fcompilers c.customize(dist) File "C:\Python26\lib\site-packages\numpy\distutils\fcompiler\__init__.py", line 525, in customize self.set_libraries(self.get_libraries()) File "C:\Python26\lib\site-packages\numpy\distutils\fcompiler\gnu.py", line 306, in get_libraries raise NotImplementedError("Only MS compiler supported with gfortran on win64") NotImplementedError: Only MS compiler supported with gfortran on win64 Could someone help me to resolve this? Thanks, -- jv

4 13

Matrix addition, +=
by Jonas Wallin 12 Sep '11

12 Sep '11

Hello, I am sure this question has been answered before but I can't find the right search word to find it. Why does MuY += MuY.transpose() and MuY = MuY + MuY.transpose() give different answers? thanks /Jonas Wallin

2 1

recarrays and lists of lists
by Travis Vaught 11 Sep '11

11 Sep '11

Greetings, Is there a particular reason why a list of lists can't be passed in to create a recarray given a particular dtype? A list of tuples works fine. I keep getting bitten by this and was thinking it should be an easy check/convert for an allowance for a row to be a list _or_ a tuple? Here's a session: ~~~~~~~~~~ In [2]: import numpy as np In [3]: dt = np.dtype({'names':['a', 'b', 'c'], 'formats':[float]*3}) In [4]: dt Out[4]: dtype([('a', '<f8'), ('b', '<f8'), ('c', '<f8')]) In [5]: rows = [[1,2,3],[2,3,4],[3,4,5]] In [6]: ary = np.array(rows, dtype=dt) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /Users/travis/git/trading/<ipython-input-6-b69b5c361254> in <module>() ----> 1 ary = np.array(rows, dtype=dt) TypeError: expected a readable buffer object In [7]: rows2 = [tuple(row) for row in rows] In [8]: ary = np.array(rows2, dtype=dt) In [9]: ary Out[9]: array([(1.0, 2.0, 3.0), (2.0, 3.0, 4.0), (3.0, 4.0, 5.0)], dtype=[('a', '<f8'), ('b', '<f8'), ('c', '<f8')]) ~~~~~~~~~~ Thoughts? Best, Travis

2 3

How to apply Numpy ufunc to Scipy sparse matrices?
by Pengkui Luo 11 Sep '11

11 Sep '11

It seems that numpy universal functions only apply to ndarray (or dense matrix). Is there a way to apply them to scipy sparse matrices also? For example, suppose S is an large sparse matrix (lil_matrix format, dtype=np.float). I want to get another sparse matrix B (of the same shape) that represents the signs of corresponding entries of S. I wrote down "B=np.sign(S)" first, but it did not give me the desired output. I realized that np.sign() might not work for a sparse matrix, so I have to convert S to a dense matrix, i.e. B = np.sign( S.todense() ) However, converting a large sparse matrix to dense would easily eat up the memory. Is there a way for np.sign (as well as other ufunc) to take a sparse matrix as parameter, and return a sparse matrix? (If I recall correctly, in Matlab all functions work on both dense matrices and sparse matrices.) -- Pengkui

2 2

[ANN] ESCO2012 and PyHPC2011: Python in science in major computational science conference
by Gael Varoquaux 11 Sep '11

11 Sep '11

ESCO 2012 - European Seminar on Coupled Problems ================================================= ESCO2012 http://esco2012.femhub.com/ is the 3rd event in a series of interdisciplineary meetings dedicated to computational science challenges in multi-physics and PDEs. I was invited as ESCO last year. It was an aboslute pleasure, because it is a small conference that is very focused on discussions. I learned a lot and could sit down with people who code top notch PDE libraries such as FEniCS and have technical discussions. Besides, it is hosted in the historical brewery where the Pilsner was invented. Plenty of great beer. Application areas ------------------ Theoretical results as well as applications are welcome. Application areas include, but are not limited to: Computational electromagnetics, Civil engineering, Nuclear engineering, Mechanical engineering, Computational fluid dynamics, Computational geophysics, Geomechanics and rock mechanics, Computational hydrology, Subsurface modeling, Biomechanics, Computational chemistry, Climate and weather modeling, Wave propagation, Acoustics, Stochastic differential equations, and Uncertainty quantification. Minisymposia * Multiphysics and Multiscale Problems in Civil Engineering * Modern Numerical Methods for ODE * Porous Media Hydrodynamics * Nuclear Fuel Recycling Simulations * Adaptive Methods for Eigenproblems * Discontinuous Galerkin Methods for Electromagnetics * Undergraduate Projects in Technical Computing Software afternoon ------------------- Important part of each ESCO conference is a software afternoon featuring software projects by participants. Presented can be any computational software that has reached certain level of maturity, i.e., it is used outside of the author's institution, and it has a web page and a user documentation. Proceedings ----------- For each ESCO we strive to reserve a special issue of an international journal with impact factor. Proceedings of ESCO 2008 appeared in Math. Comput. Simul., proceedings of ESCO 2010 in CiCP and Appl. Math. Comput. Proceedings of ESCO 2012 will appear in Computing. Important Dates * December 15, 2011: Abstract submission deadline. * December 15, 2011: Minisymposia proposals. * January 15, 2012: Notification of acceptance. PyHPC: Python for High performance computing -------------------------------------------- If you are doing super computing, SC11, ( http://sc11.supercomputing.org/) the Super Computing conference is the reference conference. This year there will a workshop on high performance computing with Python: PyHPC (http://www.dlr.de/sc/desktopdefault.aspx/tabid-1183/1638_read-31733/). At the scipy conference, I was having a discussion with some of the attendees on how people often still do process management and I/O with Fortran in the big computing environment. This is counter productive. However, has success stories of supercomputing folks using high-level languages are not advertized, this is bound to stay. Come and tell us how you use Python for high performance computing! Topics * Python-based scientific applications and libraries * High performance computing * Parallel Python-based programming languages * Scientific visualization * Scientific computing education * Python performance and language issues * Problem solving environments with Python * Performance analysis tools for Python application Papers We invite you to submit a paper of up to 10 pages via the submission site. Authors are encouraged to use IEEE two column format. Important Dates * Full paper submission: September 19, 2011 * Notification of acceptance: October 7, 2011 * Camera-ready papers: October 31, 2011

1 0

strange test failure in Cython code with numpy master
by Ralf Gommers 08 Sep '11

08 Sep '11

Hi, There's a test failure in scipy/io/matlab/mio_utils that shows up with numpy master but not 1.5.1, see http://projects.scipy.org/scipy/ticket/1512 I have a fix here: https://github.com/rgommers/scipy/commit/4ade7829649b9e2c251f5f7f370781b16f…, but I don't really understand why the code is failing in the first place. This is the relevant part: - arr = np.squeeze(arr) - if not arr.shape and arr.dtype.isbuiltin: # 0d coverted to scalar - return arr.item() - return arr + arr2 = np.squeeze(arr) + if (not arr2.shape) and arr2.dtype.isbuiltin: # 0d coverted to scalar + return arr2.item() + return arr2 All it does is rename arr to arr2. It fails with both Cython 0.13 and 0.15. Any idea? Ralf

2 1

what to use in buildbot config for numpy testing
by Chris Kees 08 Sep '11

08 Sep '11

Hi, Is there a recommended way to run the numpy test suite as a buildbot test? Just run ad python -c "import numpy; numpy.test" as ShellCommand object? Thanks, Chris

3 3

Numeric 24.2 for Python 2.4 on Windows 7?
by Dietmar Schwertberger 07 Sep '11

07 Sep '11

Hi! Does anybody have Numeric 24.2 binaries for Python 2.4 which are compatible with Windows 7 SP1? Regards, Dietmar

2 2

3D array -> 2D array issue...
by Fred 06 Sep '11

06 Sep '11

Hi all, Let's say C1 is a 3D array, and q0 and k are 2D array. dim C1 = nx*ny*nz dim q0 = nx*ny = dim k I have to do the following: q0[0, 0] = C1[0, 0, k[0, 0]] q0[1, 1] = C1[1, 1, k[1, 1]] ... q0[i, j] = C1[i, j, k[i, j]] ... I tried q0 = C1[:, :, k] but this obviously does not work. How could I do this ala NumPy? TIA Cheers, -- Fred

5 11

Smoothing binary volumes
by Thiago Franco Moraes 06 Sep '11

06 Sep '11

Hi all, I've been implementing the algorithm from this paper "Reducing Aliasing Artifacts in Iso-Surfaces of Binary Volumes" from Ross T. Whitaker. Because I develop a opensource software which works with segmentation of CT and MRI medical images, and the results of segmentation is a binary volume image. The problem is that the surface generate from binary images are rough and have those staircase artifacts. Bellow some examples after and before using the algorithm: Binary ball - http://i.imgur.com/DlIwP.png Smoothed ball - http://i.imgur.com/G04zN.png Binary CT - http://i.imgur.com/Ah1LB.png Smoothed CT - http://i.imgur.com/ps1Nz.png The algorithm works in the volumetric image. After the algorithm I use the Marching Cubes (from VTK) to generate the surface. I could use a gaussian filter, but some fine features could be lost. The source-code is here [1]. The first attempt I use python, but it used a lot of memory. Now I'm using Cython, it's not completed. Maybe I will try to code directly in C. I'm sending this email to this mail-list because someone could have some interest in this, and maybe it could be inserted in some python library, like mahotas. Thanks! [1] - https://github.com/tfmoraes/binvol

1 0

Efficient way to load a 1Gb file?
by Russell E. Owen 06 Sep '11

06 Sep '11

A coworker is trying to load a 1Gb text data file into a numpy array using numpy.loadtxt, but he says it is using up all of his machine's 6Gb of RAM. Is there a more efficient way to read such text data files? -- Russell

7 13

Eigenvalues did not converge
by Rick Muller 04 Sep '11

04 Sep '11

I'm bumping into the old "Eigenvalues did not converge" error using numpy.linalg.eigh() on several different linux builds of numpy (1.4.1). The matrix is 166x166. I can compute the eigenvalues on a Macintosh build of numpy, and I can confirm that there aren't degenerate eigenvalues, and that the matrix appears to be negative definite. I've seen this before (though not for several years), and what I normally do is to build lapack with -O0. This trick did not work in the current instance. Does anyone have any tricks to getting eigh to work? Other weird things that I've noticed about this case: I can compute the eigenvalues using eigvals and eigvalsh, and can compute the eigenvals/vecs using eig(). The matrix is real symmetric, and I've tested that it's symmetric enough by forcibly symmetrizing it. Thanks in advance for any help you can offer. -- Rick Muller rpmuller(a)gmail.com 505-750-7557

3 5

converting standard array to record array
by Bryce Ready 04 Sep '11

04 Sep '11

Hello all, So i'm using numpy 1.6.0, and trying to convert a (4,4) numpy array of dtype 'f8' into a record array of this dtype: dt = np.dtype([('mat','(4,4)f8')]) > Here is the code snippet: In [21]: a = np.random.randn(4,4) > > In [22]: a.view(dt) > and the resulting error: ValueError: new type not compatible with array. > Can anyone shed some light for me on why this conversion is not possible? It is certainly technically possible, since the memory layout of the two arrays should be the same. Can anyone recommend a better way to do this conversion? Thanks in advance! -Bryce Ready

2 2