Mailman 3 June 2013 - NumPy-Discussion

new OpenOpt Suite release 0.50
by Dmitrey June 15, 2013

June 15, 2013

Hi all, > I'm glad to inform you about new OpenOpt Suite release 0.50 (2013-June-15): > > * interalg (solver with specifiable accuracy) now works many times (sometimes orders) faster on (possibly multidimensional) integration problems (IP) and on some optimization problems * Add modeling dense (MI)(QC)QP in FuncDesigner (alpha-version, rendering may work slowly yet) * Bugfix for cplex wrapper * Some improvements for FuncDesigner interval analysis (and thus interalg) * Add FuncDesigner interval analysis for tan in range(-pi/2,pi/2) * Some other bugfixes and improvements * (Proprietary) FuncDesigner stochastic addon now is available as standalone pyc-file, became available for Python3 as well > > Regards, Dmitrey. >

1 0

Not checking github
by Sebastian Berg June 15, 2013

June 15, 2013

Just to note, I disabled most github notifications for time reasons for the next months, so if something pops up that you think I should look at, use @mention. - Sebastian

2 1

Reading data from structured text
by klo uo June 15, 2013

June 15, 2013

Hi, I have data reports in text files, where first 5 lines describe the data following, which is actually continuous time series of 2048 values wrapped in 205 rows and 10 columns, and each file has 12 such sets. If I crop to first dataset and leave the headers (first 5 lines), genfromtxt(skip_header=5) raises ValueError because last row contains 8 records instead 10 like the rest. If I just try to read file as is, with same argument, genfromtxt(skip_header=5), I get many errors as genfromtxt() doesn't seem to be able to detect header present on every dataset. So can I read such file with numpy and learn for future use, or should I just write my reader?

1 0

Generating special polynomials (Chebyshev, Hermite etc.)
by Kumar Appaiah June 15, 2013

June 15, 2013

Dear Numpy Users, I am trying to find out a way by which I can easily generate the n-th order "special" polynomial, where "special" could refer to Hermite, Chebyshev etc. Numpy 1.7 introduces several methods for such polynomials, but I couldn't find a convenience function that gives me a polynomial directly based on degree. For instance, I'd like: hermite(3) to result in array([ 0., -12., 0., 8.]) hermite(6) to result in array([-120., 0., 720., 0., -480., 0., 64.]) and so on. The quickest way I could come up with for this is: def hermite(n): if n <= 0: return numpy.array([1.0]) coeff_polynomial = [0.0] * n coeff_polynomial.extend([1]) return numpy.polynomial.hermite.herm2poly(coeff_polynomial) Now, if I am missing something, please let me know. If you think this is a useful feature, I volunteer to patch all the polynomial modules to generate such polynomials, if you could tell me appropriate function names for such convenience functions. Thanks! Kumar -- Kumar Appaiah

2 4

Will transpose ever need to copy data?
by Jaime Fernández del Río June 14, 2013

June 14, 2013

I noticed today that the documentation for np.transpose states, for the return value, that "A view is returned whenever possible." Is there really any situation where swapping axes around could trigger the need to copy data, or will a view always be returned no matter what? I can't think of any such situation, and was planning on relying on that for some code: basically, I have an output array, which I would like to be contiguous. So I preallocate it with the right shape, then take a view of it moving a certain axis to the end to make computations easier, run all my computations on the modified view, then return the original array, not the view. If I started with an array with the axis at the end, and then transposed it, I would need to make a copy to make it contiguous, which is what I am trying to avoid. Is this a bad practice? Is that precaution in the documentation real? Should I check that my view's base is the original array and trigger a copy, or is it an unnecessary check? Thanks in advance, Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

4 5

Dropping support for Accelerate/veclib?
by Pauli Virtanen June 13, 2013

June 13, 2013

Hi, numpy.distutils provides support for Accelerate+veclib on OSX, but does not provide Fortran compiler flags that ensure that the Fortran ABI used is compatible. As a result ASUM, SDOT, (apparently also DDOT), and other common BLAS functions return garbage when called with the wrong ABI. This breaks parts of Scipy. (There's some code to hack around this --- it worked on earlier versions of OSX, but apparently is unreliable on 10.8, demonstrating that this approach is just too brittle.) Moreover, third-party module developers who use Fortran will not be aware of this issue, so it may also break 3rd party code. As far as I see, the options are: 1. Add -ff2c (or whatever is the correct flag for Accelerate) to the Fortran flags on OSX. 2. Drop support for Veclib/Accelerate. I think Accelerate was also incompatible with multiprocessing, which would weigh for option 2. -- Pauli Virtanen

5 9

strided copy unroll, benchmarks needed
by Julian Taylor June 13, 2013

June 13, 2013

hi, I posted a pull with a minor change instructing the GCC compiler to unroll the strided copy loops (gcc will almost never do that on its own, not even on O3). https://github.com/numpy/numpy/pull/3429 It improves performance of these copies by 20%-50% depending on the size of the data copied (if it goes out of all cpu caches you don't gain anything anymore) on a couple machines (amd phenom x4, intel core2duo, xeon 7xxx/5xxx) As overriding the compiler decision is always dodgy, I would like some numbers on a couple of cpu types to decide if its really a good idea. So if you have the time please try the pull and the benchmark in the first comment and report the difference in performance between the pull and the unchanged numpy git head in the PR. please include your cpu, gcc version and architecture (32 bit or 64 bit). The benchmark can be run with ipython: irunner --ipython bench.py Cheers, Julian

1 0

Memory leak
by Pietro Bonfa' June 13, 2013

June 13, 2013

Dear Numpy users, I have a memory leak in my code. A simple way to reproduce my problem is: import numpy class test(): def __init__(self): pass def t(self): temp = numpy.zeros([200,100,100]) A = numpy.zeros([200], dtype = numpy.float) for i in range(200): A[i] = numpy.sum( temp[i].diagonal() ) return A a = test() c = [a.t() for i in range(100)] Running this script will require 1.5 Gb of memory since the 16 mb of temp arrays are never deallocated. How can I solve this problem? Thanks in advances, Pietro Bonfa' P.S: I asked the same question also on stack overflow (http://stackoverflow.com/questions/17085197/is-this-a-memory-leak-python-nu… ) -- Pietro Bonfa' - PhD student Dipartimento di Fisica e Scienze della Terra "Macedonio Melloni" Viale delle Scienze 7A 43124 Parma - Italy

4 5

svd + multiprocessing hangs
by Uwe Schmitt June 13, 2013

June 13, 2013

Dear all, the following code hangs on my Ubuntu machine. I use self compiled numpy 1.7.1 and Python 2.7.3 ----- import numpy import numpy.linalg import multiprocessing def classify(): X = numpy.random.random((10,3)) print "before svd" numpy.linalg.svd(X) print "this message is not printed" if __name__ == "__main__": p = multiprocessing.Process(target=classify, args=()) p.start() p.join() ----- Regards, Uwe. -- Dr. rer. nat. Uwe Schmitt Leitung F/E Mathematik mineway GmbH Gebäude 4 Im Helmerswald 2 66121 Saarbrücken Telefon: +49 (0)681 8390 5334 Telefax: +49 (0)681 830 4376 uschmitt(a)mineway.de www.mineway.de Geschäftsführung: Dr.-Ing. Mathias Bauer Amtsgericht Saarbrücken HRB 12339

4 4

supporting quad precision
by Anne Archibald June 12, 2013

June 12, 2013

Hi folks, I recently came across an application I needed quad precision for (high-accuracy solution of a differential equation). I found a C++ library (odeint) that worked for the integration itself, but unfortunately it appears numpy is not able to work with quad precision arrays. For my application the quad precision is only really needed for integrator state, so I can manually convert my data to long doubles as I go to and from numpy, but it's a pain. So quad precision support would be nice. There's a thread discussing quad support: http://mail.scipy.org/pipermail/numpy-discussion/2012-February/061080.html Essentially, there isn't any, but since gcc >= 4.6 supports them on Intel hardware (in software), it should be possible. (Then the thread got bogged down in bike-shedding about what to call them.) What would it take to support quads in numpy? I looked into the numpy base dtype definitions, and it's a complex arrangement involving detection of platform support and templatized C code; in the end I couldn't really see where to start. But it seems to me all the basics are available: native C syntax for basic arithmetic, "qabs"-style versions of all the basic functions, NaNs and Infs. So how would one go about adding quad support? Thanks, Anne

5 10