Mailman 3 April 2013 - NumPy-Discussion

Proposal of new function: iteraxis()
by andrew giessel May 1, 2013

May 1, 2013

Hello all- A while back I emailed the list about function for the numpy namespace, iteraxis(), which allows you to generalize the default iteration behavior of numpy arrays over any axis. I've implemented this function more cleanly and the pull request is here: https://github.com/numpy/numpy/pull/3262, and includes passing tests and documentation. This is very simple code, which uses np.rollaxis() to bring the desired dimension to the front, and then allows you to loop over slices in this re-structured view of the array. While little more than an alias, I feel this is a very useful function because looping over iterators is a core pattern in python, and makes working with slices of any multidimensional array very pythonic. Adding this function makes this more visible for users, new and old, and I hope members of this list will agree it is worth adding to the namespace. Generalizing this to iterate over multiple axes is something that might be worthwhile, but the specifics of how to implement the axis ordering would take some thought. I'm happy to discuss and tackle this if people are really interested. Hoping for some nice feedback, ag -- Andrew Giessel, PhD Department of Neurobiology, Harvard Medical School 220 Longwood Ave Boston, MA 02115 ph: 617.432.7971 email: andrew_giessel(a)hms.harvard.edu

11 21

Raveling, reshape order keyword unnecessarily confuses index and memory ordering
by Matthew Brett April 30, 2013

April 30, 2013

Hi, On Fri, Apr 5, 2013 at 12:53 PM, Ralf Gommers <ralf.gommers(a)gmail.com> wrote: > > > > On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett <matthew.brett(a)gmail.com> > wrote: >> >> Hi, >> >> On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers <ralf.gommers(a)gmail.com> >> wrote: >> > >> > >> > >> > On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett <matthew.brett(a)gmail.com> >> > wrote: >> >> >> >> Hi, >> >> >> >> On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg >> >> <sebastian(a)sipsolutions.net> wrote: >> >> > Hey >> >> > >> >> > On Thu, 2013-04-04 at 14:20 -0700, Matthew Brett wrote: >> >> >> Hi, >> >> >> >> >> >> On Tue, Apr 2, 2013 at 4:32 AM, Nathaniel Smith <njs(a)pobox.com> >> >> >> wrote: >> >> >> <snip> >> >> >> > Maybe we should go through and rename "order" to something more >> >> >> > descriptive >> >> >> > in each case, so we'd have >> >> >> > a.reshape(..., index_order="C") >> >> >> > a.copy(memory_order="F") >> >> >> > etc.? >> >> >> >> >> >> I'd like to propose this instead: >> >> >> >> >> >> a.reshape(..., order="C") >> >> >> a.copy(layout="F") >> >> >> >> >> > >> >> > I actually like this, makes the point clearer that it has to do with >> >> > memory layout and implies contiguity, plus it is short and from the >> >> > numpy perspective copy, etc. are the ones that add additional info to >> >> > "order" and not reshape (because IMO memory order is something new >> >> > users >> >> > should not worry about at first). A and K orders will still have >> >> > their >> >> > quirks with np.array and copy=True/False, but for many functions they >> >> > are esoteric anyway. >> >> > >> >> > It will be one hell of a deprecation though, but I am +0.5 for adding >> >> > an >> >> > alias for now (maybe someone knows an even better name?), but I think >> >> > that in this case, it probably really is better to wait with actual >> >> > deprecation warnings for a few versions, since it touches a *lot* of >> >> > code. Plus I think at the point of starting deprecation warnings (and >> >> > best earlier) numpy should provide an automatic fixer script... >> >> > >> >> > The only counter point that remains for me is the difficulty of >> >> > deprecation, since I think the new name idea is very clean. And this >> >> > is >> >> > unfortunately even more invasive then the index_order proposal. >> >> >> >> I completely agree that we'd have to be gentle with the change. The >> >> problem we'd want to avoid is people innocently using 'layout' and >> >> finding to their annoyance that the code doesn't work with other >> >> people's numpy. >> >> >> >> How about: >> >> >> >> Step 1: 'order' remains as named keyword, layout added as alias, >> >> comment on the lines of "layout will become the default keyword for >> >> this option in later versions of numpy; please consider updating any >> >> code that does not need to remain backwards compatible'. >> >> >> >> Step 2: default keyword becomes 'layout' with 'order' as alias, >> >> comment like "order is an alias for 'layout' to maintain backwards >> >> compatibility with numpy <= 1.7.1', please update any code that does >> >> not need to maintain backwards compatibility with these numpy >> >> versions' >> >> >> >> Step 3: Add deprecation warning for 'order', "order will be removed as >> >> an alias in future versions of numpy" >> >> >> >> Step 4: (distant future) Remove alias >> >> >> >> ? >> > >> > >> > A very strong -1 from me. Now we're talking about deprecation warnings >> > and a >> > backwards compatibility break after all. I thought we agreed that this >> > was a >> > very bad idea, so why are you proposing it now? >> > >> > Here's how I see it: deprecation of "order" is a no go. Therefore we >> > have >> > two choices here: >> > 1. Simply document the current "order" keyword better and leave it at >> > that. >> > 2. Add a "layout" (or "index_order") keyword, and live with both "order" >> > and >> > "layout" keywords forever. >> > >> > (2) is at least as confusing as (1), more work and poor design. >> > Therefore I >> > propose to go with (1). >> >> You are saying that deprecation of 'order' at any stage in the next 10 >> years of numpy's lifetime is a no go? > > > For something like this? Yes. You are saying I think that I am wrong in thinking this is an important change that will make numpy easier to explain and use in the long term. You'd probably expect me to disagree, and I do. I think I am right in thinking the change is important - I've tried to make that case in this thread, as well as I can. >> I think that is short-sighted and I think it will damage numpy. > > > It will damage numpy to be conservative and not change a name for a little > bit of clarity for some people that avoids reading the docs maybe a little > more carefully? There's a lot of things that can damage numpy, but this > isn't even close in my book. Too few developers, continuous backwards > compatibility issues, faster alternative libraries surpassing numpy - that's > the kind of thing that causes damage. We're talked about consensus on this list. Of course it can be very hard to achieve. >> Believe me, I have as much investment in backward compatibility as you >> do. All the three libraries that I spend a long time maintaining need >> to test against old numpy versions - but - for heaven's sake - only >> back to numpy 1.2 or numpy 1.3. We don't support Python 2.5 any more, >> and I don't think we need to maintain compatibility with Numeric >> either. > > > Really? This is from 3 months ago: > http://article.gmane.org/gmane.comp.python.numeric.general/52632. It's now > 2013, we are probably dropping numarray compat in 1.8. Not exactly 10 years, > but of the same order. I am happy to make this change over the same time course if you think that is necessary. Cheers, Matthew

4 14

bug in deepcopy() of rank-zero arrays?
by Chris Barker - NOAA Federal April 30, 2013

April 30, 2013

Hi folks, In [264]: np.__version__ Out[264]: '1.7.0' I just noticed that deep copying a rank-zero array yields a scalar -- probably not what we want. In [242]: a1 = np.array(3) In [243]: type(a1), a1 Out[243]: (numpy.ndarray, array(3)) In [244]: a2 = copy.deepcopy(a1) In [245]: type(a2), a2 Out[245]: (numpy.int32, 3) regular copy.copy() seems to work fine: In [246]: a3 = copy.copy(a1) In [247]: type(a3), a3 Out[247]: (numpy.ndarray, array(3)) Higher-rank arrays seem to work fine: In [253]: a1 = np.array((3,4)) In [254]: type(a1), a1 Out[254]: (numpy.ndarray, array([3, 4])) In [255]: a2 = copy.deepcopy(a1) In [256]: type(a2), a2 Out[256]: (numpy.ndarray, array([3, 4])) Array scalars seem to work fine as well: In [257]: s1 = np.float32(3) In [258]: s2 = copy.deepcopy(s1) In [261]: type(s1), s1 Out[261]: (numpy.float32, 3.0) In [262]: type(s2), s2 Out[262]: (numpy.float32, 3.0) There are other ways to copy arrays, but in this case, I had a dict with a bunch of arrays in it, and needed a deepcopy of the dict. I was surprised to find that my rank-0 array got turned into a scalar. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

2 2

1.8 release
by Charles R Harris April 29, 2013

April 29, 2013

Hi All, I think it is time to start the runup to the 1.8 release. I don't know of any outstanding blockers but if anyone has a PR/issue that they feel needs to be in the next Numpy release now is the time to make it known. Chuck

6 9

int to binary
by josef.pktd＠gmail.com April 29, 2013

April 29, 2013

Is there a available function to convert an int to binary representation as sequence of 0 and 1? binary_repr produces strings and is not vectorized >>> np.binary_repr(5) '101' >>> np.binary_repr(5, width=4) '0101' >>> np.binary_repr(np.arange(5), width=4) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Python26\lib\site-packages\numpy\core\numeric.py", line 1732, in binary_repr if num < 0: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() ------------ That's the best I could come up with in a few minutes: >>> k = 3; int2bin(np.arange(2**k), k, roll=False) array([[ 0., 0., 0.], [ 1., 0., 0.], [ 0., 0., 1.], [ 1., 0., 1.], [ 0., 1., 0.], [ 1., 1., 0.], [ 0., 1., 1.], [ 1., 1., 1.]]) >>> k = 3; int2bin(np.arange(2**k), k, roll=True) array([[ 0., 0., 0.], [ 0., 0., 1.], [ 0., 1., 0.], [ 0., 1., 1.], [ 1., 0., 0.], [ 1., 0., 1.], [ 1., 1., 0.], [ 1., 1., 1.]]) ----------- def int2bin(x, width, roll=True): x = np.atleast_1d(x) res = np.zeros(x.shape + (width,) ) for i in range(width): x, r = divmod(x, 2) res[..., -i] = r if roll: res = np.roll(res, width-1, axis=-1) return res Josef

4 5

EuroSciPy 2013: deadline extension 5 May 2013!
by Tiziano Zito April 29, 2013

April 29, 2013

The committee of the EuroSciPy 2013 conference has extended the deadline for abstract submission to **Sunday May 5th 2013, 23:59:50 (UTC)**. Up to then, new abstracts may be submitted on http://www.euroscipy.org . We are very much looking forward to your submissions to the conference. EuroSciPy 2013 is the annual European conference for scientists using Python. It will be held August 21-25 2013 in Brussels, Belgium. Any other questions should be addressed exclusively to euroscipy-org(a)python.org -- Tiziano Zito (Program Chair)

1 0

Re: [Numpy-discussion] [Pytables-users] ANN: numexpr 2.1 (Python 3 support is here!)
by Anthony Scopatz April 27, 2013

April 27, 2013

Congrats Francesc! On Sat, Apr 27, 2013 at 5:07 AM, Francesc Alted <faltet(a)gmail.com> wrote: > ======================== > Announcing Numexpr 2.1 > ======================== > > Numexpr is a fast numerical expression evaluator for NumPy. With it, > expressions that operate on arrays (like "3*a+4*b") are accelerated > and use less memory than doing the same calculation in Python. > > It wears multi-threaded capabilities, as well as support for Intel's > VML library (included in Intel MKL), which allows an extremely fast > evaluation of transcendental functions (sin, cos, tan, exp, log...) > while squeezing the last drop of performance out of your multi-core > processors. > > Its only dependency is NumPy (MKL is optional), so it works well as an > easy-to-deploy, easy-to-use, computational kernel for projects that > don't want to adopt other solutions that require more heavy > dependencies. > > What's new > ========== > > The main feature of this version is that it adds a much needed > **compatibility for Python 3** > > Many thanks to Antonio Valentino for his fine work on this. > Also, Christoph Gohlke quickly provided feedback and binaries for > Windows and Mark Wiebe and Gaëtan de Menten provided many small > (but important!) fixes and improvements. All of you made numexpr 2.1 > the best release ever. Thanks! > > In case you want to know more in detail what has changed in this > version, see: > > http://code.google.com/p/numexpr/wiki/ReleaseNotes > > or have a look at RELEASE_NOTES.txt in the tarball. > > Where I can find Numexpr? > ========================= > > The project is hosted at Google code in: > > http://code.google.com/p/numexpr/ > > You can get the packages from PyPI as well: > > http://pypi.python.org/pypi/numexpr > > Share your experience > ===================== > > Let us know of any bugs, suggestions, gripes, kudos, etc. you may > have. > > > Enjoy data! > > Francesc Alted > > > > ------------------------------------------------------------------------------ > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr > _______________________________________________ > Pytables-users mailing list > Pytables-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/pytables-users >

1 0

ANN: NumPy 1.7.1 release
by Ondřej Čertík April 26, 2013

April 26, 2013

Hi, I'm pleased to announce the availability of the final NumPy 1.7.1 release. Sources and binary installers can be found at https://sourceforge.net/projects/numpy/files/NumPy/1.7.1/ Only three simple bugs were fixed since 1.7.1rc1 (#3166, #3179, #3187). I would like to thank everybody who contributed patches since 1.7.1rc1: Eric Fode, Nathaniel J. Smith and Charles Harris. Cheers, Ondrej P.S. I'll create the Mac binary installers in a few days. Pypi is updated. ========================= NumPy 1.7.1 Release Notes ========================= This is a bugfix only release in the 1.7.x series. Issues fixed ------------ gh-2973 Fix `1` is printed during numpy.test() gh-2983 BUG: gh-2969: Backport memory leak fix 80b3a34. gh-3007 Backport gh-3006 gh-2984 Backport fix complex polynomial fit gh-2982 BUG: Make nansum work with booleans. gh-2985 Backport large sort fixes gh-3039 Backport object take gh-3105 Backport nditer fix op axes initialization gh-3108 BUG: npy-pkg-config ini files were missing after Bento build. gh-3124 BUG: PyArray_LexSort allocates too much temporary memory. gh-3131 BUG: Exported f2py_size symbol prevents linking multiple f2py modules. gh-3117 Backport gh-2992 gh-3135 DOC: Add mention of PyArray_SetBaseObject stealing a reference gh-3134 DOC: Fix typo in fft docs (the indexing variable is 'm', not 'n'). gh-3136 Backport #3128 Checksums ========= 9e369a96b94b107bf3fab7e07fef8557 release/installers/numpy-1.7.1-win32-superpack-python2.6.exe 0ab72b3b83528a7ae79c6df9042d61c6 release/installers/numpy-1.7.1.tar.gz bb0d30de007d649757a2d6d2e1c59c9a release/installers/numpy-1.7.1-win32-superpack-python3.2.exe 9a72db3cad7a6286c0d22ee43ad9bc6c release/installers/numpy-1.7.1.zip 0842258fad82060800b8d1f0896cb83b release/installers/numpy-1.7.1-win32-superpack-python3.1.exe 1b8f29b1fa89a801f83f551adc13aaf5 release/installers/numpy-1.7.1-win32-superpack-python2.7.exe 9ca22df942e5d5362cf7154217cb4b69 release/installers/numpy-1.7.1-win32-superpack-python2.5.exe 2fd475b893d8427e26153e03ad7d5b69 release/installers/numpy-1.7.1-win32-superpack-python3.3.exe

6 9

f2py and object libraries
by Jonathan Slavin April 25, 2013

April 25, 2013

Hi all, I have recently started using f2py to access some legacy fortran code and it's mostly worked better than I expected. It handles common blocks, block data, etc. with no problems. I did need to define the type of all the arguments in subroutine and function calls, but not in the body of the code (though of course that would be good programming practice). The one thing that I haven't been able to get to work is linking to code in an object library. My normal approach to compiling my code that uses this particular code collection (RS) goes like: f77 -c (...other options) main_prog.f f77 -o main_prog main_prog.o -L/lib_dir -lRS where /lib_dir is the directory where the library resides and the library name is libRS.a The library was created by via: ar cru libRS.a file1.o file2.o (...list of object files, compiled fortran routines) ranlib libRS.a f2py does accept the -L and -l arguments but doesn't seem to be able to find the code in the libraries. Any suggestions? Jon -- ______________________________________________________________ Jonathan D. Slavin Harvard-Smithsonian CfA jslavin(a)cfa.harvard.edu 60 Garden Street, MS 83 phone: (617) 496-7981 Cambridge, MA 02138-1516 cell: (781) 363-0035 USA ______________________________________________________________

2 1

Random number generation and testing across different OS's.
by Andrew Nelson April 25, 2013

April 25, 2013

I have written a differential evolution optimiser that i use for curvefitting. As a genetic optimisation technique it is stochastic and relies heavily on random number generators to do the minimisation. As part of the module tests I would like to write a cross-platform test that checks if the fitting is being done correctly. I use an instance of numpy.random.RandomState for the generation. If I use the seed method on a single platform I get the same output, which I could use to write a test. However, I am unsure of how the seeding and RandomState works across platforms. If I use the same seed on OSX/Windows/Linux, will I get the same stream of random numbers being generated? I need to know if the test I write works across platforms. regards, Andrew. -- _____________________________________ Dr. Andrew Nelson _____________________________________

4 3