Mailman 3 September 2016 - NumPy-Discussion

ANN: PyTables 3.3.0
by Tom Kooij Sept. 12, 2016

Sept. 12, 2016

=========================== Announcing PyTables 3.3.0 =========================== We are happy to announce PyTables 3.3.0. What's new ========== - Single codebase Python 2 and 3 support (PR #493). - Internal Blosc version updated to 1.11.1 (closes :issue:`541`) - Full BitShuffle support for new Blosc versions (>= 1.8). - It is now possible to remove all rows from a table. - It is now possible to read reference types by dereferencing them as numpy array of objects (closes :issue:`518` and :issue:`519`). Thanks to Ehsan Azar - Fixed Windows 32 and 64-bit builds. In case you want to know more in detail what has changed in this version, please refer to: http://www.pytables.org/release_notes.html You can install it via pip or download a source package with generated PDF and HTML docs from: https://github.com/PyTables/PyTables/releases/tag/v3.3.0 For an online version of the manual, visit: http://www.pytables.org/usersguide/index.html What it is? =========== PyTables is a library for managing hierarchical datasets and designed to efficiently cope with extremely large amounts of data with support for full 64-bit file addressing. PyTables runs on top of the HDF5 library and NumPy package for achieving maximum throughput and convenient use. PyTables includes OPSI, a new indexing technology, allowing to perform data lookups in tables exceeding 10 gigarows (10**10 rows) in less than a tenth of a second. Resources ========= About PyTables: http://www.pytables.org About the HDF5 library: http://hdfgroup.org/HDF5/ About NumPy: http://numpy.scipy.org/ Acknowledgments =============== Thanks to many users who provided feature improvements, patches, bug reports, support and suggestions. See the ``THANKS`` file in the distribution package for a (incomplete) list of contributors. Most specially, a lot of kudos go to the HDF5 and NumPy makers. Without them, PyTables simply would not exist. Share your experience ===================== Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. ---- **Enjoy data!** -- The PyTables Developers .. Local Variables: .. mode: rst .. coding: utf-8 .. fill-column: 72 .. End:

1 0

New Indexing Methods Revival #N (subclasses!)
by Sebastian Berg Sept. 11, 2016

Sept. 11, 2016

Hi all, not that I am planning to spend much time on this right now, however, I did a small rebase of the stuff I had (did not push yet) on oindex and remembered the old problem ;). The one remaining issue I have with adding things like (except making the code prettier and writing tests): arr.oindex[...] # outer/orthogonal indexing arr.vindex[...] # Picking of elements (much like current) arr.lindex[...] # current behaviour for backward compat is what to do about subclasses. Now what I can do (and have currently in my branch) is to tell someone on `subclass.oindex[...]`: This won't work, the subclass implements `__getitem__` or `__setitem__` so I don't know if the result would be correct (its a bit annoying if you also warn about using those attributes, but...). However, with or without such error, we need a nice way for subclasses to define these attributes! This is important even within numpy at least for masked arrays (possibly also matrix and memmap). They (typically) do some stuff before or after the plain indexing operation, so how do we make it convenient to allow them to do the same stuff for the special indexing attributes without weird code duplication? I can think of things, but nothing too great yet so maybe you guys got an elegant idea. - Sebastian

6 29

Continued New Indexing Methods Revival #N (subclasses!)
by Sebastian Berg Sept. 10, 2016

Sept. 10, 2016

Hi all, from the discussion, I was thinking maybe something like this: class B(): def __numpy_getitem__(self, index, indexing_method="plain"): # do magic. return super().__numpy_getitem__( index, indexing_method=indexing_method) as new API. There are some issues, though. An old subclass may define `__getitem__`. Now the behaviour that would seem nice to me is: 1. No new attribute (no `__numpy_getitem__`) and also no `__getitem__`/`__setitem__`: Should just work 2. No new attribute but old attributes defined: Should at least give a warning (or an error) when using the new attributes, since the behaviour might buggy. 3. `__numpy_getitem__` defined: Will channel all indexing through it (except maybe some edge cases in python 2). Best, also avoid that use getitem in setitem trick.... If you define both (which might make sense for some edge case stuff), you should just channel it through this yourself. Now the issue I have is that for 1. and 2. to work correctly, I need to know which methods are overloaded by the subclass. Checking is a bit tedious and the method I hacked first for getitem and setitem does not work for a normal method. Can anyone think of a nicer way to do this trick that does not require quite as much hackery. Or is there an easy way to do the overloading check? - Sebastian

1 1

gmane
by Charles R Harris Sept. 9, 2016

Sept. 9, 2016

Hi All, Looks like gmane is going down <https://lars.ingebrigtsen.no/2016/07/28/the-end-of-gmane/>. Does anyone know of an alternative for searching and referencing the NumPy mail archives? Chuck

2 1

Numpy 1.11.2
by Charles R Harris Sept. 9, 2016

Sept. 9, 2016

Hi All, I would like to release Numpy 1.11.2rc1 this weekend. It will contain a few small fixes and enhancements for windows and the last Scipy release. If there are any pending PRs that you think should go in or be backported for this release, please speak up. Chuck

2 4

ANN: pandas v0.19.0rc1 - RELEASE CANDIDATE
by Joris Van den Bossche Sept. 8, 2016

Sept. 8, 2016

Hi, I'm pleased to announce the availability of the first release candidate of Pandas 0.19.0. Please try this RC and report any issues at the pandas issue tracker <https://github.com/pydata/pandas/issues>. The release candidate can be installed with conda from our development channel (builds for osx-64, linux-64 and win-64 are available for Python 2.7, 3.4 and 3.5): conda install -c pandas pandas=0.19.0rc1 or with pip from PyPI <https://pypi.python.org/pypi/pandas/0.19.0rc1> (wheels are available): pip install --pre pandas==0.19.0rc1 --- THIS IS NOT A PRODUCTION RELEASE This is a major release from 0.18.1 and includes a number of API changes, several new features, enhancements, and performance improvements along with a large number of bug fixes. Highlights include: - New method merge_asof for asof-style time-series joining, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0190…> - The .rolling() method is now time-series aware, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0190…> - read_csv now supports parsing Categorical data, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0190…> - A function union_categorical has been added for combining categoricals, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0190…> - PeriodIndex now has its own period dtype, and changed to be more consistent with other Index classes. See here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0190…> - Sparse data structures gained enhanced support of int and bool dtypes, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0190…> - Comparison operations with Series no longer ignores the index, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0190…> for an overview of the API changes. - Introduction of a pandas development API for utility functions, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0190…> . - Deprecation of Panel4D and PanelND. We recommend to represent these types of n-dimensional data with the xarray package <http://xarray.pydata.org/en/stable/>. - Removal of the previously deprecated modules pandas.io.data, pandas.io.wb, pandas.tools.rplot. See the Whatsnew <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html> file for more information. Please report any issues here <https://github.com/pydata/pandas/issues/>. A big thanks to all contributors! Joris

1 0

Which NumPy/Numpy/numpy spelling?
by mail＠telenczuk.pl Sept. 8, 2016

Sept. 8, 2016

Hi all, What is the official spelling of NumPy/Numpy/numpy? The documentation is not consistent and it mixes both NumPy and Numpy. For example, the reference manual uses both spellings in the introduction paragraph (http://docs.scipy.org/doc/numpy/reference/): "This reference manual details functions, modules, and objects included in Numpy, describing what they are and what they do. For learning how to use NumPy, see also NumPy User Guide." However, in all docs taken together "NumPy" is most frequently used (74%): % find . -name "*.rst" -exec grep Numpy -ow {} \; | wc -l 161 % find . -name "*.rst" -exec grep NumPy -ow {} \; | wc -l 471 I also reported it as an issue: https://github.com/numpy/numpy/issues/7986 Yours, Bartosz

8 9

Re: [Numpy-discussion] State-of-the-art to use a C/C++ library from Python
by Peter Creasey Sept. 7, 2016

Sept. 7, 2016

> Date: Wed, 31 Aug 2016 13:28:21 +0200 > From: Michael Bieri <mibieri(a)gmail.com> > > I'm not quite sure which approach is state-of-the-art as of 2016. How would > you do it if you had to make a C/C++ library available in Python right now? > > In my case, I have a C library with some scientific functions on matrices > and vectors. You will typically call a few functions to configure the > computation, then hand over some pointers to existing buffers containing > vector data, then start the computation, and finally read back the data. > The library also can use MPI to parallelize. > Depending on how minimal and universal you want to keep things, I use the ctypes approach quite often, i.e. treat your numpy inputs an outputs as arrays of doubles etc using the ndpointer(...) syntax. I find it works well if you have a small number of well-defined functions (not too many options) which are numerically very heavy. With this approach I usually wrap each method in python to check the inputs for contiguity, pass in the sizes etc. and allocate the numpy array for the result. Peter

7 7

Has anyone ever used f2py with Cray ftn? How?
by Paul Leopardi Sept. 7, 2016

Sept. 7, 2016

Hi, Has anyone ever used f2py with the Cray ftn compiler driver? The compiler driver can drive Cray, Gnu, Intel Fortran compilers, including necessary libraries, via loaded modules. Assuming that this has never been done, or that the existing code to do this is unavailable: To use Cray ftn with f2py do I need to change the source code under numpy/distutils/fcompiler, as suggested by this blog post? https://gehrcke.de/2014/02/building-numpy-and-scipy-with-intel-compilers-an… I would need to create a new file, cray.py under this directory, contain classes for each of the Cray, Gnu and Intel compilers as invoked by the ftn driver. What other files would I need to change? How would I package tests? How would I contribute the resulting code to NumPy? All the best, Paul -- Paul Leopardi https://sites.google.com/site/paulleopardi/

1 0

Correct error of invalid axis arguments.
by Charles R Harris Sept. 5, 2016

Sept. 5, 2016

Hi All, At the moment there are two error types raised when invalid axis arguments are encountered: IndexError and ValueError. I prefer ValueError for arguments, IndexError seems more appropriate when the bad axis value is used as an index. In any case, having mixed error types is inconvenient, but also inconvenient to change. Should we worry about that? If so, what should the error be? Note that some of the mixup arises because the axis values are not checked before use, in which case IndexError is raised. Chuck

2 1