Mailman 3 April 2011 - NumPy-Discussion

ANN: Numpy 1.6.0 release candidate 1
by Ralf Gommers 03 May '11

03 May '11

Hi, I am pleased to announce the availability of the first release candidate of NumPy 1.6.0. If no new problems are reported, the final release will be in one week. Sources and binaries can be found at http://sourceforge.net/projects/numpy/files/NumPy/1.6.0rc1/ For (preliminary) release notes see below. Enjoy, Ralf ========================= NumPy 1.6.0 Release Notes ========================= This release includes several new features as well as numerous bug fixes and improved documentation. It is backward compatible with the 1.5.0 release, and supports Python 2.4 - 2.7 and 3.1 - 3.2. Highlights ========== * Re-introduction of datetime dtype support to deal with dates in arrays. * A new 16-bit floating point type. * A new iterator, which improves performance of many functions. New features ============ New 16-bit floating point type ------------------------------ This release adds support for the IEEE 754-2008 binary16 format, available as the data type ``numpy.half``. Within Python, the type behaves similarly to `float` or `double`, and C extensions can add support for it with the exposed half-float API. New iterator ------------ A new iterator has been added, replacing the functionality of the existing iterator and multi-iterator with a single object and API. This iterator works well with general memory layouts different from C or Fortran contiguous, and handles both standard NumPy and customized broadcasting. The buffering, automatic data type conversion, and optional output parameters, offered by ufuncs but difficult to replicate elsewhere, are now exposed by this iterator. Legendre, Laguerre, Hermite, HermiteE polynomials in ``numpy.polynomial`` ------------------------------------------------------------------------- Extend the number of polynomials available in the polynomial package. In addition, a new ``window`` attribute has been added to the classes in order to specify the range the ``domain`` maps to. This is mostly useful for the Laguerre, Hermite, and HermiteE polynomials whose natural domains are infinite and provides a more intuitive way to get the correct mapping of values without playing unnatural tricks with the domain. Fortran assumed shape array and size function support in ``numpy.f2py`` ----------------------------------------------------------------------- F2py now supports wrapping Fortran 90 routines that use assumed shape arrays. Before such routines could be called from Python but the corresponding Fortran routines received assumed shape arrays as zero length arrays which caused unpredicted results. Thanks to Lorenz Hüdepohl for pointing out the correct way to interface routines with assumed shape arrays. In addition, f2py interprets Fortran expression ``size(array, dim)`` as ``shape(array, dim-1)`` which makes it possible to automatically wrap Fortran routines that use two argument ``size`` function in dimension specifications. Before users were forced to apply this mapping manually. Other new functions ------------------- ``numpy.ravel_multi_index`` : Converts a multi-index tuple into an array of flat indices, applying boundary modes to the indices. ``numpy.einsum`` : Evaluate the Einstein summation convention. Using the Einstein summation convention, many common multi-dimensional array operations can be represented in a simple fashion. This function provides a way compute such summations. ``numpy.count_nonzero`` : Counts the number of non-zero elements in an array. ``numpy.result_type`` and ``numpy.min_scalar_type`` : These functions expose the underlying type promotion used by the ufuncs and other operations to determine the types of outputs. These improve upon the ``numpy.common_type`` and ``numpy.mintypecode`` which provide similar functionality but do not match the ufunc implementation. Changes ======= Changes and improvements in the numpy core ------------------------------------------ ``default error handling`` -------------------------- The default error handling has been change from ``print`` to ``warn`` for all except for ``underflow``, which remains as ``ignore``. ``numpy.distutils`` ------------------- Several new compilers are supported for building Numpy: the Portland Group Fortran compiler on OS X, the PathScale compiler suite and the 64-bit Intel C compiler on Linux. ``numpy.testing`` ----------------- The testing framework gained ``numpy.testing.assert_allclose``, which provides a more convenient way to compare floating point arrays than `assert_almost_equal`, `assert_approx_equal` and `assert_array_almost_equal`. ``C API`` --------- In addition to the APIs for the new iterator and half data type, a number of other additions have been made to the C API. The type promotion mechanism used by ufuncs is exposed via ``PyArray_PromoteTypes``, ``PyArray_ResultType``, and ``PyArray_MinScalarType``. A new enumeration ``NPY_CASTING`` has been added which controls what types of casts are permitted. This is used by the new functions ``PyArray_CanCastArrayTo`` and ``PyArray_CanCastTypeTo``. A more flexible way to handle conversion of arbitrary python objects into arrays is exposed by ``PyArray_GetArrayParamsFromObject``. Deprecated features =================== The "normed" keyword in ``numpy.histogram`` is deprecated. Its functionality will be replaced by the new "density" keyword. Removed features ================ ``numpy.fft`` ------------- The functions `refft`, `refft2`, `refftn`, `irefft`, `irefft2`, `irefftn`, which were aliases for the same functions without the 'e' in the name, were removed. ``numpy.memmap`` ---------------- The `sync()` and `close()` methods of memmap were removed. Use `flush()` and "del memmap" instead. ``numpy.lib`` ------------- The deprecated functions ``numpy.unique1d``, ``numpy.setmember1d``, ``numpy.intersect1d_nu`` and ``numpy.lib.ufunclike.log2`` were removed. ``numpy.ma`` ------------ Several deprecated items were removed from the ``numpy.ma`` module:: * ``numpy.ma.MaskedArray`` "raw_data" method * ``numpy.ma.MaskedArray`` constructor "flag" keyword * ``numpy.ma.make_mask`` "flag" keyword * ``numpy.ma.allclose`` "fill_value" keyword ``numpy.distutils`` ------------------- The ``numpy.get_numpy_include`` function was removed, use ``numpy.get_include`` instead.

5 13

Less dimensions than expected with record array
by Alan Gibson 02 May '11

02 May '11

Hello all, This question may seem elementary (mostly because it is), but I can't find documentation anywhere as to why the following are true: >>> import numpy as np >>> data = [(1,2,3),(4,5,6),(7,8,9)] >>> dt = [('a',int),('b',int),('c',int)] >>> normal_array = np.array(data) >>> record_array = np.array(data, dtype=dt) >>> print "ndarray has shape %s but record array has shape %s" % \ ... (normal_array.shape, record_array.shape) ndarray has shape (3, 3) but record array has shape (3,) >>> print "ndarray has %s dimensions but record array has %s dimensions" % \ ... (normal_array.ndim, record_array.ndim) ndarray has 2 dimensions but record array has 1 dimensions This makes seemingly reasonable things, like using apply_along_axis() over a table of data with named columns, impossible: >>> np.apply_along_axis(record_array, 1, lambda x: x) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.6/dist-packages/numpy/lib/shape_base.py", line 72, in apply_along_axis % (axis,nd)) ValueError: axis must be less than arr.ndim; axis=1, rank=0. What's the reason for this behavior? Is there a way to make such operations work with record arrays? Thanks, Alan

3 2

numpy histogram : Maybe a little error in the tutorial
by tonthon 30 Apr '11

30 Apr '11

Hi, I was followong the tutorial about numpy's basics and found a little error here : http://www.scipy.org/Tentative_NumPy_Tutorial#head-aa75ec76530ff51a2e98071a… (n, bins) = numpy.histogram(v, bins=50, normed=1) # NumPy version (no plot) should be : (n, bins) = numpy.histogram(v, bins=50, normed=True) since in the 'new' version of histogram, the normed var is explicitly tested as a boolean (in numpy/lib/function_base.py line 380): if normed is False: ... elif normed is True: ... Cheers Gaston

2 1

Array of size 'n' with common difference < 1
by dileep kunjaai 29 Apr '11

29 Apr '11

Dear sir, I am trying to make an array of varies from -60 to 90 with difference 0.25. I tried the following command ... >>import numpy as N lat=N.array(xrange(-6000, 9000, 25), dtype=float) print lat/100 I know that there is another easy method...... Please give me a replay ........ Thanking you.............. -- DILEEPKUMAR. R J R F, IIT DELHI

5 7

ANN: Spyder v2.0.11
by Pierre Raybaut 29 Apr '11

29 Apr '11

Hi all, I am pleased to announced that Spyder v2.0.11 has just been released. As this is mostly a maintenance release, a lot of bugs were fixed and some minor features were added (see below). Embedding an interactive Python console into your GUI-based application (Qt): This version includes an example of application using the Spyder's internal shell as a debugging console which also demonstrates the py2exe deployment procedure. Spyder (previously known as Pydee) is a free open-source Python development environment providing MATLAB-like features in a simple and light-weighted software, available for Windows XP/Vista/7, GNU/Linux and MacOS X: http://spyderlib.googlecode.com/ Spyder is part of spyderlib, a Python module based on PyQt4, pyflakes and rope (QScintilla's dependency has been removed in version 2.0 and rope features have been integrated since this version as well). Some of Spyder basic features: * Python, C/C++, Fortran source editor with class/function browser, code completion and calltips * consoles: o open as many Python interpreters, IPython consoles or command windows as you need o code completion and calltips o variable explorer with GUI-based editors for a lot of data types (numbers, strings, lists, arrays, dictionaries, ...) * object inspector: provide documentation or source code on any Python object (class, function, module, ...) * online documentation: automatically generated html documentation on installed Python modules * find in files * file explorer * project manager * MATLAB-like PYTHONPATH management dialog box (works with all consoles) * Windows only: current user environment variables editor * direct links to documentation (Python, Qt, Matplotlib, NumPy, Scipy, etc.) * direct link to Python(x,y) launcher * direct links to QtDesigner, QtLinguist and QtAssistant (Qt documentation) Bug fixes (since v2.0.9): (Fixes Issue 616) Pylint plugin: tree widget header text was not updated when analyizing a new script (the last analyzed script name was still shown) Editor/completion widget/bugfix: pressing shift was hiding the completion combo box (Fixes Issue 630) Added missing default settings for "Spyder light" (only necessary when installing from scratch and without any remaining .spyder.ini file) Editor/Console-bugfix: info tooltips (calltips) were hidden right after being shown (i.e. when typing any character after the left parenthesis) (Fixes Issue 631) Drag and drop of files into editor on Linux was pasting path instead of opening the file (Fixes Issue 640) Editor: block comment was not working correctly at end of file Code completion widget (Editor/Console) - bugfix: parenting to the ancestor widget was necessary on Linux (Fixes Issue 546) [Contributor: Alex Fargus] C/Cpp syntax highlighting bugfix (Fixes Issue 646) IPython integration: fixed pyreadline monkey-patch for pyreadline v1.7 Enhancements (since v2.0.9): File explorer widget/plugin: improved performances (widget is now populated in a separate thread) Spyder crash dialog: warning the user about the '--reset' option (this will remove all configuration files) Cheers, Pierre

1 0

numpy.load truncates read from network file on XP
by Dan Halbert 29 Apr '11

29 Apr '11

I'm having trouble loading a large remote .npy file on Windows XP. This is on numpy-1.3.0 on Windows XP SP3: numpy.load(r'\\myserver\mydir\big.npy') will fail with this sort of error being printed: "14328000 items requested but only 54 read" and then I get this with a backtrace: "ValueError: total size of new array must be unchanged" (due to the truncated array) The file big.npy is a big 2d array, about 112MB. The same file when stored locally gives no error when read. I can also read it into an editor, or copy it, and I get the whole thing. More strangely, the same file when read from the same UNC path on Windows 7 64-bit (with the same 32-bit versions of all Python-related software) does not give an error either. The fileserver in question is a NetApp box. Any clues? My websearching hasn't netted any leads. Thanks a lot, Dan

3 3

Possible funding opportunity for numpy and scipy development
by Thouis (Ray) Jones 29 Apr '11

29 Apr '11

(Posted for a friend) This NSF opportunity just renewed last week and might interest the NumPy/SciPy community: NSF Software Infrastructure for Sustained Innovation (SI2) http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=503489 It sounds like it's not suited for end-user software in a particular domain but is intended to be more software infrastructure oriented.

1 0

Zero row in SVD's unitary matrix on some Mac's
by Rob Beezer 28 Apr '11

28 Apr '11

I am working to make many of NumPy's matrix decomposition routines available in Sage. As part of testing a new routine, we have found some odd behavior with the singular value decomposition. On certain Mac's, the numpy built in Sage will return the second of the unitary matrices with a row of all zeros, so of course, it can't be unitary. For the configurations in question, this erroneous output happens only for certain sized matrices and for those sizes it always occurs. The smallest sizes are 3 x 4, 4 x 5, 5 x 6, 5 x 7, 6 x 7, 6 x 8, 6 x 9, 7 x 8, 7 x 9, 8 x 9. The fault is not in Sage code per se, as it can be reproduced by running Sage's python and using numpy directly. It could be possible Sage is not building numpy correctly, we have not tested a standalone version of numpy since this problem seems to be limited to very few configurations. The initial report, and a confirmation, are both on Macs where Sage is built using gcc 4.0.1 and gcc 4.2.1. The test that uncovered this situation was introduced two alpha releases back, and has not failed for testers on Linux or newer Macs. The svd routine itself has been in Sage for about three years without exhibiting any problems, but maybe the cases above were not tested. I do not own a Mac, so testing out scenarios involves sending suggestions to the two folks who have reported failures. Many more details and complete transcripts are at: http://trac.sagemath.org/sage_trac/ticket/11248 Any thoughts or advice to help us understand this would be greatly appreciated. Thanks in advance. Rob -- Robert A. Beezer Professor Department of Mathematics and Computer Science University of Puget Sound 1500 N Warner Tacoma, WA 98416-1043 beezer(a)ups.edu http://buzzard.ups.edu Voice: 253.879.3564 Fax: 253.879.3522

5 18

Memmap with multiprocessing
by Thiago Franco de Moraes 27 Apr '11

27 Apr '11

Hi all, I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between <traceback> tags): <traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in <bound method memmap.__del__ of memmap([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int16)> ignored </traceback> I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not. Thanks!

6 6

reading tiff images
by Mathew Yeates 26 Apr '11

26 Apr '11

Hi What is current method of using ndiimage on a Tiff file? I've seen different methods using ndimage itself, scipy.misc and Pil. Mathew

5 6