Mailman 3 September 2012 - NumPy-Discussion

ANN: NumPy 1.7.0b2 release
by Ondřej Čertík 01 Oct '12

01 Oct '12

Hi, I'm pleased to announce the availability of the second beta release of NumPy 1.7.0b2. Sources and binary installers can be found at https://sourceforge.net/projects/numpy/files/NumPy/1.7.0b2/ Please test this release and report any issues on the numpy-discussion mailing list. Since beta1, we've fixed most of the known (back then) issues, except: http://projects.scipy.org/numpy/ticket/2076 http://projects.scipy.org/numpy/ticket/2101 http://projects.scipy.org/numpy/ticket/2108 http://projects.scipy.org/numpy/ticket/2150 And many other issues that were reported since the beta1 release. The log of changes is attached. The full list of issues that we still need to work on is at: https://github.com/numpy/numpy/issues/396 Any help is welcome, the best is to send a PR fixing any of the issues -- against master, and I'll then back-port it to the release branch (unless it is something release specific, in which case just send the PR against the release branch). Cheers, Ondrej * f217517 Release 1.7.0b2 * 50f71cb MAINT: silence Cython warnings about changes dtype/ufunc size. * fcacdcc FIX: use py24-compatible version of virtualenv on Travis * d01354e FIX: loosen numerical tolerance in test_pareto() * 65ec87e TST: Add test for boolean insert * 9ee9984 TST: Add extra test for multidimensional inserts. * 8460514 BUG: Fix for issues #378 and #392 This should fix the problems with numpy.insert(), where the input values were not checked for all scalar types and where values did not get inserted properly, but got duplicated by default. * 07e02d0 BUG: fix npymath install location. * 6da087e BUG: fix custom post_check. * 095a3ab BUG: forgot to build _dotblas in bento build. * cb0de72 REF: remove unused imports in bscript. * 6e3e289 FIX: Regenerate mtrand.c with Cython 0.17 * 3dc3b1b Retain backward compatibility. Enforce C order. * 5a471b5 Improve ndindex execution speed. * 2f28db6 FIX: Add a test for Ticket #2066 * ca29849 BUG: Add a test for Ticket #2189 * 1ee4a00 BUG: Add a test for Ticket #1588 * 7b5dba0 BUG: Fix ticket #1588/gh issue #398, refcount error in clip * f65ff87 FIX: simplify the import statement * 124a608 Fix returned copy * 996a9fb FIX: bug in np.where and recarray swapping * 7583adc MAINT: silence DeprecationWarning in np.safe_eval(). * 416af9a pavement.py: rename "yop" to "atlas" * 3930881 BUG: fix bento build. * fbad4a7 Remove test_recarray_from_long_formats * 5cb80f8 Add test for long number in shape specifier of dtype string * 24da7f6 Add test for long numbers in numpy.rec.array formats string * 77da3f8 Allow long numbers in numpy.rec.array formats string * 99c9397 Use PyUnicode_DecodeUTF32() * 31660d0 Follow the C guidelines * d5d6894 Fix memory leak in concatenate. * 8141e1e FIX: Make sure the tests produce valid unicode * d67785b FIX: Fixes the PyUnicodeObject problem in py-3.3 * a022015 Re-enable unpickling optimization for large py3k bytes objects. * 470486b Copy bytes object when unpickling an array * d72280f Fix tests for empty shape, strides and suboffsets on Python 3.3 * a1561c2 [FIX] Add missing header so separate compilation works again * ea23de8 TST: set raise-on-warning behavior of NoseTester to release mode. * 28ffac7 REL: set version number to 1.7.0rc1-dev.

7 17

Memory order of array copies
by Nathaniel Smith 01 Oct '12

01 Oct '12

There are three basic Python APIs to copy an array in numpy: a.copy(): has always returned a C-contiguous array by default. has always taken an order= argument, which defaults to "C". np.array(a, copy=True): by default, produces an array with whatever memory ordering that 'a' had. Can also specify order="C", "F" to get C or Fortran contiguity instead. np.copy(a): has always been a simple alias for np.array(a, copy=True), which means that it also preserves memory ordering. BUT in current master and the 1.7 betas, an extra argument order= has been added, and this has been set to default to "C" ordering. The extra argument and behavioural change occurred in 0e1a4e95. It's not clear why; the change isn't mentioned in the commit message. The change has to be reverted for 1.7, at least, because it caused regressions in scikit-learn (and presumably other packages too). So the question is, what *should* these interface look like. Then we can figure out what kind of transition scheme is needed, if any. My gut reaction is that if we want to change this at all from it's pre-1.7 status quo, it would be the opposite of the change that was made in master... I'd expect np.copy(a) and a.copy() to return arrays that are as nearly-identical to 'a' as possible, unless I explicitly requested something different by passing an order= argument. But, I bet there is code out there that's doing something like: my_fragile_C_function_wrapper(a.copy()) when it really should be doing something more explicit like my_fragile_C_function_wrapper(np.array(a, copy=False, order="C", dtype=float)) i.e., they're depending on the current behaviour where a.copy() normalizes order. I don't see any way to detect these cases and issue a proper warning, so we may not be able to change this at all. Any ideas? Is there anything better to do than simply revert np.copy() to its traditional behaviour and accept that np.copy(a) and a.copy() will continue to have different semantics indefinitely? -n

4 4

ANN: scikits-image 0.7.0 release
by Tony Yu 30 Sep '12

30 Sep '12

Announcement: scikits-image 0.7.0 ================================= We're happy to announce the 7th version of scikits-image! Scikits-image is an image processing toolbox for SciPy that includes algorithms for segmentation, geometric transformations, color space manipulation, analysis, filtering, morphology, feature detection, and more. For more information, examples, and documentation, please visit our website http://skimage.org New Features ------------ It's been only 3 months since scikits-image 0.6 was released, but in that short time, we've managed to add plenty of new features and enhancements, including - Geometric image transforms - 3 new image segmentation routines (Felsenzwalb, Quickshift, SLIC) - Local binary patterns for texture characterization - Morphological reconstruction - Polygon approximation - CIE Lab color space conversion - Image pyramids - Multispectral support in random walker segmentation - Slicing, concatenation, and natural sorting of image collections - Perimeter and coordinates measurements in regionprops - An extensible image viewer based on Qt and Matplotlib, with plugins for edge detection, line-profiling, and viewing image collections Plus, this release adds a number of bug fixes, new examples, and performance enhancements. Contributors to this release ---------------------------- This release was only possible due to the efforts of many contributors, both new and old. - Andreas Mueller - Andreas Wuerl - Andy Wilson - Brian Holt - Christoph Gohlke - Dharhas Pothina - Emmanuelle Gouillart - Guillaume Gay - Josh Warner - James Bergstra - Johannes Schonberger - Jonathan J. Helmus - Juan Nunez-Iglesias - Leon Tietz - Marianne Corvellec - Matt McCormick - Neil Yager - Nicolas Pinto - Nicolas Poilvert - Pavel Campr - Petter Strandmark - Stefan van der Walt - Tim Sheerman-Chase - Tomas Kazmar - Tony S Yu - Wei Li

1 0

Re: [Numpy-discussion] Should abs([nan]) be supported?
by Alex Leach 30 Sep '12

30 Sep '12

Ondřej Čertík <ondrej.certik <at> gmail.com> writes: > > On Tue, Sep 4, 2012 at 8:38 PM, Travis Oliphant <travis <at> continuum.io> > wrote: > I think the test should be changed to check for RuntimeWarning on some of > the cases. This might take a little work as it looks like the code uses > generators across multiple tests and > would have to be changed to handle expecting warnings. > > > > Alternatively, the error context can be set before the test runs and then > > restored afterwords: > > > > olderr = np.seterr(invalid='ignore') > > abs(a) > > np.seterr(**olderr) > > > > > > or, using an errstate context --- > > > > with np.errstate(invalid='ignore'): > > abs(a) > > I see --- so abs([nan]) should emit a warning, but in the test we > should suppress it. > I'll work on that. > Just witnessing this error now.. I'm building numpy on 64bit Linux with Intel's icc, and on OSX Mountain Lion with clang. I thought it was a problem with the built-in abs, as I used the following test case:- $ /usr/local/bin/python >>> import numpy as np >>> abs(np.array([1e-08, 1, 1000020.0000000099] ) - \ ... np.array([0, np.nan, 1000000.0] ) ) With the OSX clang build, this returns without an error message. array([ 1.00000000e-08, nan, 2.00000000e+01]) But on Ubuntu with my icc build of Python-2.7.3, I get that FloatingPointError, and corresponding numpy test failures. I figure this is caused by a compile flag that I did or didn't use, so dug around the icc man page, and think I found the cause for it... I built my Ubuntu python with the '-fp-model strict' option, as per recommendations I've seen, but this turns on floating point exceptions, so I'm going to rebuild with '-fp-model precise -fp-model source', and see how it goes... Cheers, Alex

1 0

Re: [Numpy-discussion] Making numpy sensible: backward compatibility please
by Robert Lupton the Good 28 Sep '12

28 Sep '12

Gael puts in a plea for backward compatibility; I totally agree. Numpy sometimes goes out of its way to make this hard. For example, when the syntax of histogram were changed you got a nice DepreciationWarning about an option to switch to the new behaviour; great. But a few releases later that option went away and code carefully written to survive the transition stopped working; that didn't make me a happy user. R > Also, putting more emphasis on backward compatibility is possible. For instance, the 'order' parameter added to np.copy could have defaulted to the old behavior, 'K', for a year, with a DeprecationWarning, same thing for the casting rules.

1 0

PyData NYC 2012 Speakers and Talks announced!
by Peter Wang 28 Sep '12

28 Sep '12

Hi everyone, The PyData NYC team and Continuum Analytics are proud to announce the full lineup of talks and speakers for the PyData NYC 2012 event! We're thrilled with the exciting lineup of workshops, hands-on tutorials, and talks about real-world uses of Python for data analysis. http://nyc2012.pydata.org/schedule The list of presenters and talk abstracts are also available, and are linked from the schedule page. For those who will be in town on Thursday evening of October 25th, there will be a special PyData edition of Drinks & Data at Dewey's Flatiron. It'll be a great chance to socialize and meet with PyData presenters and other attendees. Register here: http://drinks-and-data-pydata-conf-ny.eventbrite.com/ We're also proud to be part of the NYC DataWeek: http://oreilly.com/dataweek/?cmp=tw-strata-ev-dr. The week of October 22nd is going to be a great time to be in New York! Lastly, we are still looking for sponsors! If you want to get your company recognition in front of a few hundred Python data hackers and hardcore developers, PyData will be a premier venue to showcase your products or recruit exceptional talent. Please visit http://nyc2012.pydata.org/sponsors/becoming/ to inquire about sponsorship. In addition to the conference sponsorship, charter sponsorships for dinner Friday night, as well as the Sunday Hack-a-thon event are all open. Please help us promote the conference! Tell your friends, email your meetup groups, and follow @PyDataConf on Twitter. Early registration ends in just a few weeks, so register today! http://pydata.eventbrite.com/ See you there! -Peter Wang Organizer, PyData NYC 2012

1 0

numpy.distutils, lapack, and cython
by Nathaniel Smith 28 Sep '12

28 Sep '12

Hi all, I've gotten a pull request for scikits-sparse to switch it to using numpy.distutils: https://github.com/njsmith/scikits-sparse/pull/2 Overall this seems fair enough, finding libraries is a pain and numpy.distutils has that knowledge. 1) What's the proper way to find lapack using numpy.distutils? The patch tries lapack_mkl_info and lapack_info, but this is 2 of the 6 variants that my numpy.distutils.system_info contains. Really what I probably want is a way to ask numpy how it was built (including any custom paths the user used) and to default to doing the same? Is that possible? 2) Is there a better way to build Cython files than this weird monkey-patching thing they propose? (It's still better than the horror that setuptools/distribute require, but I guess I have higher expectations...) 3) Can I use 2to3 along with numpy.distutils? -n

3 2

variable number of columns in loadtxt/genfromtxt
by Andreas Hilboll 25 Sep '12

25 Sep '12

Hi, I commonly have to deal with legacy ASCII files, which don't have a constant number of columns. The standard is 10 values per row, but sometimes, there are less columns. loadtxt doesn't support this, and in genfromtext, the rows which have less than 10 values are excluded from the resulting array. Is there any way around this? Thanks for your insight, Andreas.

3 3

API, ABI compatibility
by David Cournapeau 25 Sep '12

25 Sep '12

2 2

Long-standing issue with using numpy in embedded CPython
by Yang Zhang 25 Sep '12

25 Sep '12

It turns out that there's a long-standing problem in numpy that prevents it from being used in embedded CPython environments: http://stackoverflow.com/questions/7592565/when-embedding-cpython-in-java-w… http://mail.scipy.org/pipermail/numpy-discussion/2009-July/044046.html Is there any fix or workaround for this? Thanks. -- Yang Zhang http://yz.mit.edu/

9 15