Mailman 3 March 2015 - NumPy-Discussion

numpy pickling problem - python 2 vs. python 3
by Anrd Baecker 07 Mar '15

07 Mar '15

Dear all, when preparing the transition of our repositories from python 2 to python 3, I encountered a problem loading pytables (.h5) files generated using python 2. I suspect that it is caused by a problem with pickling numpy arrays under python 3: The code appended at the end of this mail works fine on either python 2.7 or python 3.4, however, generating the data on python 2 and trying to load them on python 3 gives some strange string ( b'(lp1\ncnumpy.core.multiarray\n_reconstruct\np2\n(cnumpy\nndarray ...) instead of [array([ 0., 1., 2., 3., 4., 5.]), array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9., 10.])] The problem sounds very similar to the one reported here https://github.com/numpy/numpy/issues/4879 which was fixed with numpy 1.9. I tried different versions/combintations of numpy (including 1.9.2) and always end up with the above result. Also I tried to reduce the problem down to the level of pure numpy and pickle (as in the above bug report): import numpy as np import pickle arr1 = np.linspace(0.0, 1.0, 2) arr2 = np.linspace(0.0, 2.0, 3) data = [arr1, arr2] p = pickle.dumps(data) print(pickle.loads(p)) p Using the resulting string for p as input string (with b added at the beginnung) under python 3 gives UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 14: ordinal not in range(128) Can someone reproduce the problem with pytables? Is there maybe work-around? (And no: I can't re-generate the "old" data files - it's hundreds of .h5 files ... ;-). Many thanks, best, Arnd ############################################################################## """Illustrate problem with pytables data - python 2 to python 3.""" from __future__ import print_function import sys import numpy as np import tables as tb def main(): """Run the example.""" print("np.__version__=", np.__version__) check_on_same_version = False arr1 = np.linspace(0.0, 5.0, 6) arr2 = np.linspace(0.0, 10.0, 11) data = [arr1, arr2] # Only generate on python 2.X or check on the same python version: if sys.version < "3.0" or check_on_same_version: fpt = tb.open_file("tstdat.h5", mode="w") fpt.set_node_attr(fpt.root, "list_of_arrays", data) fpt.close() # Load the saved file: fpt = tb.open_file("tstdat.h5", mode="r") result = fpt.get_node_attr("/", "list_of_arrays") fpt.close() print("Loaded:", result) main()

11 17

appveyor CI
by Charles R Harris 06 Mar '15

06 Mar '15

Anyone familiar with appveyor <http://www.appveyor.com/>? Is this something we could use to test/build numpy on windows machines? It is free for open source. Chuck

5 7

Adding keyword to asarray and asanyarray.
by Charles R Harris 06 Mar '15

06 Mar '15

Hi All, This is apropos gh-5634 <https://github.com/numpy/numpy/pull/5634>, a PR adding a precision keyword to asarray and asanyarray. The PR description is The precision keyword differs from the current dtype keyword in the > following way. > > - It specifies a minimum precision. If the precision of the input is > greater than the specified precision, the input precision is preserved. > - Complex types are preserved. A specifies floating precision applies > to the dtype of the real and complex parts separately. > > For example, both complex128 and float64 dtypes have the > same precision and an array of dtype float64 will be unchanged if the > specified precision is float32. > > Ideally the precision keyword would be pushed down into the array > constructor so that the resulting dtype could be determined before the > array is constructed, but that would require adding new functions as the > current constructors are part of the API and cannot have their > signatures changed. > The name of the keyword is open to discussion, as well as its acceptable values. And of course, anything else that might come to mind ;) Thoughts? Chuck

4 8

ufuncs now take a tuple of arrays as 'out' kwarg
by Jaime Fernández del Río 06 Mar '15

06 Mar '15

Hi all, There is a PR, ready to be merged, that adds the possibility of passing a tuple of arrays in the 'out' kwarg to ufuncs with multiple outputs: https://github.com/numpy/numpy/pull/5621 The new functionality is as follows: * If the ufunc has a single output, then the 'out' kwarg can either be a single array (or None) like today, or a tuple holding a single array (or None). * If the ufunc has more than one output, then the 'out' kwarg must be a tuple with one array (or None) per output argument. The old behavior, where only the first output could be specified, is now deprecated, will raise a deprecation warning, and potentially be changed to an error in the future. * In both cases, positional and keyword output arguments are incompatible. This has been made a little more strict, as the following is valid in <= 1.9.x but will now raise an error: np.add(2, 2, None, out=arr) There seemed to be a reasonable amount of agreement on the goodness of this change from the discussions on github, but I wanted to inform the larger audience, in case there are any addressable concerns. Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

1 0

ANN: scikit-image 0.11
by Stefan van der Walt 05 Mar '15

05 Mar '15

Announcement: scikit-image 0.11.0 ================================= We're happy to announce the release of scikit-image v0.11.0! scikit-image is an image processing toolbox for SciPy that includes algorithms for segmentation, geometric transformations, color space manipulation, analysis, filtering, morphology, feature detection, and more. For more information, examples, and documentation, please visit our website: http://scikit-image.org Highlights ---------- For this release, we merged over 200 pull requests with bug fixes, cleanups, improved documentation and new features. Highlights include: - Region Adjacency Graphs - Color distance RAGs (#1031) - Threshold Cut on RAGs (#1031) - Similarity RAGs (#1080) - Normalized Cut on RAGs (#1080) - RAG drawing (#1087) - Hierarchical merging (#1100) - Sub-pixel shift registration (#1066) - Non-local means denoising (#874) - Sliding window histogram (#1127) - More illuminants in color conversion (#1130) - Handling of CMYK images (#1360) - `stop_probability` for RANSAC (#1176) - Li thresholding (#1376) - Signed edge operators (#1240) - Full ndarray support for `peak_local_max` (#1355) - Improve conditioning of geometric transformations (#1319) - Standardize handling of multi-image files (#1200) - Ellipse structuring element (#1298) - Multi-line drawing tool (#1065), line handle style (#1179) - Point in polygon testing (#1123) - Rotation around a specified center (#1168) - Add `shape` option to drawing functions (#1222) - Faster regionprops (#1351) - `skimage.future` package (#1365) - More robust I/O module (#1189) API Changes ----------- - The ``skimage.filter`` subpackage has been renamed to ``skimage.filters``. - Some edge detectors returned values greater than 1--their results are now appropriately scaled with a factor of ``sqrt(2)``. Contributors to this release ---------------------------- (Listed alphabetically by last name) - Fedor Baart - Vighnesh Birodkar - François Boulogne - Nelson Brown - Alexey Buzmakov - Julien Coste - Phil Elson - Adam Feuer - Jim Fienup - Geoffrey French - Emmanuelle Gouillart - Charles Harris - Jonathan Helmus - Alexander Iacchetta - Ivana Kajić - Kevin Keraudren - Almar Klein - Gregory R. Lee - Jeremy Metz - Stuart Mumford - Damian Nadales - Pablo Márquez Neila - Juan Nunez-Iglesias - Rebecca Roisin - Jasper St. Pierre - Jacopo Sabbatini - Michael Sarahan - Salvatore Scaramuzzino - Phil Schaf - Johannes Schönberger - Tim Seifert - Arve Seljebu - Steven Silvester - Julian Taylor - Matěj Týč - Alexey Umnov - Pratap Vardhan - Stefan van der Walt - Joshua Warner - Tony S Yu

1 0

ANN: BayesPy 0.3 released
by Jaakko Luttinen 05 Mar '15

05 Mar '15

Dear all, I am pleased to announce that BayesPy 0.3 has been released. BayesPy provides tools for variational Bayesian inference. The user can easily constuct conjugate exponential family models from nodes and run approximate posterior inference. BayesPy aims to be efficient and flexible enough for experts but also accessible for casual users. ----------------------------------------------------------------------- This release adds several state-of-the-art VB features. Below is a list of significant new features in this release: * Gradient-based optimization of the nodes by using either the Euclidean or Riemannian/natural gradient. This enables, for instance, the Riemannian conjugate gradient method. * Collapsed variational inference to improve the speed of learning. * Stochastic variational inference to improve scalability. * Pattern search to improve the speed of learning. * Deterministic annealing to improve robustness against initializations. * Gaussian Markov chains can use input signals. More details about the new features can be found here: http://www.bayespy.org/user_guide/advanced.html ------------------------------------------------------ PyPI: https://pypi.python.org/pypi/bayespy/0.3 Git repository: https://github.com/bayespy/bayespy Documentation: http://www.bayespy.org/ Best regards, Jaakko

1 0

linalg.norm probems
by Charles R Harris 05 Mar '15

05 Mar '15

Hi All, This is with reference to issue #5626 <https://github.com/numpy/numpy/issues/5626>. Currently linalg.norm converts the input like so `x = asarray(x)`. This can produce integer arrays, which in turn may create problems of overflow, or the failure of the abs functions for minimum values of signed integer types. I propose to convert the input to a minimum precision of float32. However, this will be a change in behavior. I'd guess that that might not be much of a problem, as otherwise it is likely that this problem would have been reported earlier. Thoughts? Chuck

3 5

ANN: xray v0.4 released
by Stephan Hoyer 03 Mar '15

03 Mar '15

I'm pleased to announce a major release of xray, v0.4. xray is an open source project and Python package that aims to bring the labeled data power of pandas to the physical sciences, by providing N-dimensional variants of the core pandas data structures. Our goal is to provide a pandas-like and pandas-compatible toolkit for analytics on multi-dimensional arrays, rather than the tabular data for which pandas excels. Our approach adopts the Common Data Model for self-describing scientific data in widespread use in the Earth sciences: xray.Dataset is an in-memory representation of a netCDF file. Documentation: http://xray.readthedocs.org/ GitHub: https://github.com/xray/xray Highlights of this release: * Automatic alignment of index labels in arithmetic and when combining arrays or datasets. * Aggregations like mean now skip missing values by default. * Relaxed equality rules in concat and merge for variables with equal value(s) but different shapes. * New drop method for dropping variables or index labels. * Support for reindexing with a fill method like pandas. For more details, read the full release notes: http://xray.readthedocs.org/en/stable/whats-new.html Best, Stephan

1 0

quastion numpy build
by Константин Мухамеджанов 03 Mar '15

03 Mar '15

Hi,Sorry for my English(I am from Russia).I have such quastion:how can I build extension of numpy binaries using setup.py file.How can I specify the compiler program to setup.py.I had not found the option --compiler as it was ,when I had build another package extension.I have Mingw32 compiler. Regard

1 0

Congratulations to Chris Barker...
by Nathaniel Smith 03 Mar '15

03 Mar '15

...on the acceptance of his PEP! PEP 485 adds a math.isclose function to the standard library, encouraging people to do numerically more reasonable floating point comparisons. The PEP: https://www.python.org/dev/peps/pep-0485/ The pronouncement: http://thread.gmane.org/gmane.comp.python.devel/151776/focus=151778 -n -- Nathaniel J. Smith -- http://vorpus.org

4 4