On 4/30/2011 1:19 PM, Ralf Gommers wrote:

Hi,

I am pleased to announce the availability of the first release candidate of NumPy 1.6.0. If no new problems are reported, the final release will be in one week.

Sources and binaries can be found at http://sourceforge.net/projects/numpy/files/NumPy/1.6.0rc1/ For (preliminary) release notes see below.

Enjoy, Ralf

========================= NumPy 1.6.0 Release Notes =========================

This release includes several new features as well as numerous bug fixes and improved documentation. It is backward compatible with the 1.5.0 release, and supports Python 2.4 - 2.7 and 3.1 - 3.2.

Highlights ==========

* Re-introduction of datetime dtype support to deal with dates in arrays.

* A new 16-bit floating point type.

* A new iterator, which improves performance of many functions.

New features ============

New 16-bit floating point type ------------------------------

This release adds support for the IEEE 754-2008 binary16 format, available as the data type ``numpy.half``. Within Python, the type behaves similarly to `float` or `double`, and C extensions can add support for it with the exposed half-float API.

New iterator ------------

A new iterator has been added, replacing the functionality of the existing iterator and multi-iterator with a single object and API. This iterator works well with general memory layouts different from C or Fortran contiguous, and handles both standard NumPy and customized broadcasting. The buffering, automatic data type conversion, and optional output parameters, offered by ufuncs but difficult to replicate elsewhere, are now exposed by this iterator.

Legendre, Laguerre, Hermite, HermiteE polynomials in ``numpy.polynomial`` -------------------------------------------------------------------------

Extend the number of polynomials available in the polynomial package. In addition, a new ``window`` attribute has been added to the classes in order to specify the range the ``domain`` maps to. This is mostly useful for the Laguerre, Hermite, and HermiteE polynomials whose natural domains are infinite and provides a more intuitive way to get the correct mapping of values without playing unnatural tricks with the domain.

Fortran assumed shape array and size function support in ``numpy.f2py`` -----------------------------------------------------------------------

F2py now supports wrapping Fortran 90 routines that use assumed shape arrays. Before such routines could be called from Python but the corresponding Fortran routines received assumed shape arrays as zero length arrays which caused unpredicted results. Thanks to Lorenz Hüdepohl for pointing out the correct way to interface routines with assumed shape arrays.

In addition, f2py interprets Fortran expression ``size(array, dim)`` as ``shape(array, dim-1)`` which makes it possible to automatically wrap Fortran routines that use two argument ``size`` function in dimension specifications. Before users were forced to apply this mapping manually.

Other new functions -------------------

``numpy.ravel_multi_index`` : Converts a multi-index tuple into an array of flat indices, applying boundary modes to the indices.

``numpy.einsum`` : Evaluate the Einstein summation convention. Using the Einstein summation convention, many common multi-dimensional array operations can be represented in a simple fashion. This function provides a way compute such summations.

``numpy.count_nonzero`` : Counts the number of non-zero elements in an array.

``numpy.result_type`` and ``numpy.min_scalar_type`` : These functions expose the underlying type promotion used by the ufuncs and other operations to determine the types of outputs. These improve upon the ``numpy.common_type`` and ``numpy.mintypecode`` which provide similar functionality but do not match the ufunc implementation.

Changes =======

Changes and improvements in the numpy core ------------------------------------------

``default error handling`` --------------------------

The default error handling has been change from ``print`` to ``warn`` for all except for ``underflow``, which remains as ``ignore``.

``numpy.distutils`` -------------------

Several new compilers are supported for building Numpy: the Portland Group Fortran compiler on OS X, the PathScale compiler suite and the 64-bit Intel C compiler on Linux.

``numpy.testing`` -----------------

The testing framework gained ``numpy.testing.assert_allclose``, which provides a more convenient way to compare floating point arrays than `assert_almost_equal`, `assert_approx_equal` and `assert_array_almost_equal`.

``C API`` ---------

In addition to the APIs for the new iterator and half data type, a number of other additions have been made to the C API. The type promotion mechanism used by ufuncs is exposed via ``PyArray_PromoteTypes``, ``PyArray_ResultType``, and ``PyArray_MinScalarType``. A new enumeration ``NPY_CASTING`` has been added which controls what types of casts are permitted. This is used by the new functions ``PyArray_CanCastArrayTo`` and ``PyArray_CanCastTypeTo``. A more flexible way to handle conversion of arbitrary python objects into arrays is exposed by ``PyArray_GetArrayParamsFromObject``.

Deprecated features ===================

The "normed" keyword in ``numpy.histogram`` is deprecated. Its functionality will be replaced by the new "density" keyword.

Removed features ================

``numpy.fft`` -------------

The functions `refft`, `refft2`, `refftn`, `irefft`, `irefft2`, `irefftn`, which were aliases for the same functions without the 'e' in the name, were removed.

``numpy.memmap`` ----------------

The `sync()` and `close()` methods of memmap were removed. Use `flush()` and "del memmap" instead.

``numpy.lib`` -------------

The deprecated functions ``numpy.unique1d``, ``numpy.setmember1d``, ``numpy.intersect1d_nu`` and ``numpy.lib.ufunclike.log2`` were removed.

``numpy.ma`` ------------

Several deprecated items were removed from the ``numpy.ma`` module::

* ``numpy.ma.MaskedArray`` "raw_data" method * ``numpy.ma.MaskedArray`` constructor "flag" keyword * ``numpy.ma.make_mask`` "flag" keyword * ``numpy.ma.allclose`` "fill_value" keyword

``numpy.distutils`` -------------------

The ``numpy.get_numpy_include`` function was removed, use ``numpy.get_include`` instead. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

I get a null pointer access violation during numpy.test() with all msvc9/MKL builds for win32 (not win-amd64). The crash occurs during test_result_type() in test_numeric.py and can be reduced to the following code:

...

...

import numpy as np np.result_type(np.array([np.float32(0)]), np.complex128(0))

np.float64(0) and np.float16(0) also crash. Unfortunately the debug builds do not crash. Christoph

On Sat, Apr 30, 2011 at 5:53 PM, Charles R Harris

wrote:

<snip>

...

I get a null pointer access violation during numpy.test() with all msvc9/MKL builds for win32 (not win-amd64). The crash occurs during test_result_type() in test_numeric.py and can be reduced to the following code:

...

...

...

import numpy as np np.result_type(np.array([np.float32(0)]), np.complex128(0))

np.float64(0) and np.float16(0) also crash. Unfortunately the debug builds do not crash.

This is new, right?

Does it depend on the optimization level? Chuck

On Sat, Apr 30, 2011 at 6:50 PM, Christoph Gohlke wrote:

On 4/30/2011 4:58 PM, Charles R Harris wrote:

...

On Sat, Apr 30, 2011 at 5:53 PM, Charles R Harris mailto:charlesr.harris@gmail.com> wrote:

<snip>

I get a null pointer access violation during numpy.test() with

all

...

msvc9/MKL builds for win32 (not win-amd64). The crash occurs

during

...

test_result_type() in test_numeric.py and can be reduced to the following code:

> >> import numpy as np > >> np.result_type(np.array([np.float32(0)]), np.complex128(0))

np.float64(0) and np.float16(0) also crash. Unfortunately the

debug

...

builds do not crash.

This is new, right?

Does it depend on the optimization level?

Chuck

Yes it's new. The pure msvc9 builds without MKL also crash. The crash disapperars When compiling with /Od (disable optimization) instead of /Ox (maximum optimization; the default for distutils).

So all of np.float16(0), np.float32(0), np.float64(0), etc crash? Does it depend at all on 0 as the argument, or is it the same for 1, 0.0, etc. What about string arguments like np.float64("0"). I want to pin the location down a bit more. Too bad it doesn't crash in the debugger. Chuck

On 4/30/2011 6:37 PM, Charles R Harris wrote:

On Sat, Apr 30, 2011 at 6:50 PM, Christoph Gohlke mailto:cgohlke@uci.edu> wrote:

On 4/30/2011 4:58 PM, Charles R Harris wrote: > > > On Sat, Apr 30, 2011 at 5:53 PM, Charles R Harris > mailto:charlesr.harris@gmail.com mailto:charlesr.harris@gmail.com>> wrote: > > <snip> > > > I get a null pointer access violation during numpy.test() with all > msvc9/MKL builds for win32 (not win-amd64). The crash occurs during > test_result_type() in test_numeric.py and can be reduced to the > following code: > > > >> import numpy as np > > >> np.result_type(np.array([np.float32(0)]), np.complex128(0)) > > np.float64(0) and np.float16(0) also crash. Unfortunately the debug > builds do not crash. > > This is new, right? > > > Does it depend on the optimization level? > > Chuck >

Yes it's new. The pure msvc9 builds without MKL also crash. The crash disapperars When compiling with /Od (disable optimization) instead of /Ox (maximum optimization; the default for distutils).

So all of np.float16(0), np.float32(0), np.float64(0), etc crash? Does it depend at all on 0 as the argument, or is it the same for 1, 0.0, etc. What about string arguments like np.float64("0"). I want to pin the location down a bit more. Too bad it doesn't crash in the debugger.

Chuck

Sorry, I should have been more precise. The crash occurs in `numpy.result_type(a, b)` with the following inputs: b = numpy.complex128(0) a = numpy.array([numpy.float16(0)]) a = numpy.array([numpy.float32(0)]) a = numpy.array([numpy.float64(0)]) Christoph

On 4/30/2011 9:27 PM, Mark Wiebe wrote:

On Sat, Apr 30, 2011 at 9:11 PM, Christoph Gohlke mailto:cgohlke@uci.edu> wrote:

On 4/30/2011 6:37 PM, Charles R Harris wrote: > > > On Sat, Apr 30, 2011 at 6:50 PM, Christoph Gohlke mailto:cgohlke@uci.edu > mailto:cgohlke@uci.edu>> wrote: > > > > On 4/30/2011 4:58 PM, Charles R Harris wrote: > > > > > > On Sat, Apr 30, 2011 at 5:53 PM, Charles R Harris > > mailto:charlesr.harris@gmail.com mailto:charlesr.harris@gmail.com> > mailto:charlesr.harris@gmail.com > mailto:charlesr.harris@gmail.com>>> wrote: > > > > <snip> > > > > > > I get a null pointer access violation during numpy.test() > with all > > msvc9/MKL builds for win32 (not win-amd64). The crash > occurs during > > test_result_type() in test_numeric.py and can be reduced > to the > > following code: > > > > > >> import numpy as np > > > >> np.result_type(np.array([np.float32(0)]), np.complex128(0)) > > > > np.float64(0) and np.float16(0) also crash. Unfortunately > the debug > > builds do not crash. > > > > This is new, right? > > > > > > Does it depend on the optimization level? > > > > Chuck > > > > Yes it's new. The pure msvc9 builds without MKL also crash. The crash > disapperars When compiling with /Od (disable optimization) instead of > /Ox (maximum optimization; the default for distutils). > > > So all of np.float16(0), np.float32(0), np.float64(0), etc crash? Does > it depend at all on 0 as the argument, or is it the same for 1, 0.0, > etc. What about string arguments like np.float64("0"). I want to pin the > location down a bit more. Too bad it doesn't crash in the debugger. > > Chuck >

Sorry, I should have been more precise. The crash occurs in `numpy.result_type(a, b)` with the following inputs:

b = numpy.complex128(0) a = numpy.array([numpy.float16(0)]) a = numpy.array([numpy.float32(0)]) a = numpy.array([numpy.float64(0)])

Christoph _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org mailto:NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Can you try it with this commit backed out?

https://github.com/numpy/numpy/commit/fc0148bef18b8fd34d124d5edd90887f63e3bf...

What does the stack trace of the crash look like? Compiling with debug information still works with optimizations, just the line numbers and stack traces don't match the source as well.

https://github.com/numpy/numpy/commit/fc0148bef18b8fd34d124d5edd90887f63e3bf...-Mark

Sorry, I get no stack trace; apparently the stack is corrupted. I printf-debugged it down to the following preprocessor directive, which already caused trouble in gcc before http://projects.scipy.org/numpy/ticket/1737: #if NPY_SIZEOF_LONGLONG >= NPY_SIZEOF_CLONGDOUBLE npy_longlong value; #else npy_clongdouble value; #endif Replacing the two occasions of this code in multiarray/convert_datatype.c with `npy_longlong value[4];` solved the crash and all tests pass. Btw, where is NPY_SIZEOF_CLONGDOUBLE defined? I searched the source code and the build directory but could not find it. Christoph

I've found a few reference counting bugs running the regression tests under Valgrind. Pull request here: https://github.com/numpy/numpy/pull/79 Mike On 04/30/2011 04:19 PM, Ralf Gommers wrote:

Hi,

I am pleased to announce the availability of the first release candidate of NumPy 1.6.0. If no new problems are reported, the final release will be in one week.

Sources and binaries can be found at http://sourceforge.net/projects/numpy/files/NumPy/1.6.0rc1/ For (preliminary) release notes see below.

Enjoy, Ralf

========================= NumPy 1.6.0 Release Notes =========================

This release includes several new features as well as numerous bug fixes and improved documentation. It is backward compatible with the 1.5.0 release, and supports Python 2.4 - 2.7 and 3.1 - 3.2.

Highlights ==========

* Re-introduction of datetime dtype support to deal with dates in arrays.

* A new 16-bit floating point type.

* A new iterator, which improves performance of many functions.

New features ============

New 16-bit floating point type ------------------------------

This release adds support for the IEEE 754-2008 binary16 format, available as the data type ``numpy.half``. Within Python, the type behaves similarly to `float` or `double`, and C extensions can add support for it with the exposed half-float API.

New iterator ------------

A new iterator has been added, replacing the functionality of the existing iterator and multi-iterator with a single object and API. This iterator works well with general memory layouts different from C or Fortran contiguous, and handles both standard NumPy and customized broadcasting. The buffering, automatic data type conversion, and optional output parameters, offered by ufuncs but difficult to replicate elsewhere, are now exposed by this iterator.

Legendre, Laguerre, Hermite, HermiteE polynomials in ``numpy.polynomial`` -------------------------------------------------------------------------

Extend the number of polynomials available in the polynomial package. In addition, a new ``window`` attribute has been added to the classes in order to specify the range the ``domain`` maps to. This is mostly useful for the Laguerre, Hermite, and HermiteE polynomials whose natural domains are infinite and provides a more intuitive way to get the correct mapping of values without playing unnatural tricks with the domain.

Fortran assumed shape array and size function support in ``numpy.f2py`` -----------------------------------------------------------------------

F2py now supports wrapping Fortran 90 routines that use assumed shape arrays. Before such routines could be called from Python but the corresponding Fortran routines received assumed shape arrays as zero length arrays which caused unpredicted results. Thanks to Lorenz Hüdepohl for pointing out the correct way to interface routines with assumed shape arrays.

In addition, f2py interprets Fortran expression ``size(array, dim)`` as ``shape(array, dim-1)`` which makes it possible to automatically wrap Fortran routines that use two argument ``size`` function in dimension specifications. Before users were forced to apply this mapping manually.

Other new functions -------------------

``numpy.ravel_multi_index`` : Converts a multi-index tuple into an array of flat indices, applying boundary modes to the indices.

``numpy.einsum`` : Evaluate the Einstein summation convention. Using the Einstein summation convention, many common multi-dimensional array operations can be represented in a simple fashion. This function provides a way compute such summations.

``numpy.count_nonzero`` : Counts the number of non-zero elements in an array.

``numpy.result_type`` and ``numpy.min_scalar_type`` : These functions expose the underlying type promotion used by the ufuncs and other operations to determine the types of outputs. These improve upon the ``numpy.common_type`` and ``numpy.mintypecode`` which provide similar functionality but do not match the ufunc implementation.

Changes =======

Changes and improvements in the numpy core ------------------------------------------

``default error handling`` --------------------------

The default error handling has been change from ``print`` to ``warn`` for all except for ``underflow``, which remains as ``ignore``.

``numpy.distutils`` -------------------

Several new compilers are supported for building Numpy: the Portland Group Fortran compiler on OS X, the PathScale compiler suite and the 64-bit Intel C compiler on Linux.

``numpy.testing`` -----------------

The testing framework gained ``numpy.testing.assert_allclose``, which provides a more convenient way to compare floating point arrays than `assert_almost_equal`, `assert_approx_equal` and `assert_array_almost_equal`.

``C API`` ---------

In addition to the APIs for the new iterator and half data type, a number of other additions have been made to the C API. The type promotion mechanism used by ufuncs is exposed via ``PyArray_PromoteTypes``, ``PyArray_ResultType``, and ``PyArray_MinScalarType``. A new enumeration ``NPY_CASTING`` has been added which controls what types of casts are permitted. This is used by the new functions ``PyArray_CanCastArrayTo`` and ``PyArray_CanCastTypeTo``. A more flexible way to handle conversion of arbitrary python objects into arrays is exposed by ``PyArray_GetArrayParamsFromObject``.

Deprecated features ===================

The "normed" keyword in ``numpy.histogram`` is deprecated. Its functionality will be replaced by the new "density" keyword.

Removed features ================

``numpy.fft`` -------------

The functions `refft`, `refft2`, `refftn`, `irefft`, `irefft2`, `irefftn`, which were aliases for the same functions without the 'e' in the name, were removed.

``numpy.memmap`` ----------------

The `sync()` and `close()` methods of memmap were removed. Use `flush()` and "del memmap" instead.

``numpy.lib`` -------------

The deprecated functions ``numpy.unique1d``, ``numpy.setmember1d``, ``numpy.intersect1d_nu`` and ``numpy.lib.ufunclike.log2`` were removed.

``numpy.ma`` ------------

Several deprecated items were removed from the ``numpy.ma`` module::

* ``numpy.ma.MaskedArray`` "raw_data" method * ``numpy.ma.MaskedArray`` constructor "flag" keyword * ``numpy.ma.make_mask`` "flag" keyword * ``numpy.ma.allclose`` "fill_value" keyword

``numpy.distutils`` -------------------

The ``numpy.get_numpy_include`` function was removed, use ``numpy.get_include`` instead. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

-- Michael Droettboom Science Software Branch Space Telescope Science Institute Baltimore, Maryland, USA

On Mon, May 2, 2011 at 10:55 PM, Mark Wiebe wrote:

I applied everything, since they're all obviously bugs and the fixes look straightforward.

I tested those commits on a few python/OS combinations, all looks good. So I'll try to tag an RC2 tonight. Ralf