Mailman 3 October 2018 - NumPy-Discussion

LaTeX version of boolean indexing
by Matthew Harrigan 12 Oct '18

12 Oct '18

Hello, I am documenting some code, translating the core of the algorithm to LaTeX. The style I have currently is very similar to the einsum syntax (which is awesome btw). Here <https://gist.github.com/mattharrigan/68b292e64381bba6b78a06a6f1762fa2> is an example of some of the basic operations in NumPy. One part I do not know how to capture well is boolean indexing, ie: mask = np.array([1, 0, 1]) x = np.array([1, 2, 3]) y = x[mask] Any suggestions on how to clearly, formally, and concisely show that operation? Also, are there any guides on translating NumPy to LaTeX? It might be helpful for documenting algorithms and also for people learning NumPy. Thank you, Matt

2 3

Adding a hex version like PY_VERSION_HEX
by Matti Picus 10 Oct '18

10 Oct '18

In PR 12074 https://github.com/numpy/numpy/pull/12074 I propose adding a function `version.get_numpy_version_as_hex()` which returns a hex value to represent the current NumPy version MAJOR.MINOR.MICRO where v = hex(MAJOR << 24 | MINOR << 16 | MICRO) so the current 1.15.0 would become '0x10f0000'. I also made this avaiable via C through `get_hex_version`. The hex version is based on the |PY_VERSION_HEX| macro from CPython. Currently we have a ABI version and an API version for the numpy C-API. We only increment those for updated or breaking changes in the NumPy C-API, but not for - changes in behavior, especially in python code - changes in sizes of outward-facing structures like PyArray_Desc Occasionally it is desirable to determine backward compatibility from the runtime version, rather than from the ABI or API versions, and having it as a single value makes the comparison in C easy. For instance this may be convenient when there is suspicion that older header files may have been used to create or manipulate an object directly in C (or via a cython optimization), and we want to verify the version used to create the object, or when we may want to verify de-serialized objects. The `numpy.lib._version.NumpyVersion` class enables version comparison in python, but I would prefer a single value that can be stored in a C struct as an integer type. Since this is an enhancement proposal, I am bringing the idea to the mailing list for reactions. Matti

4 4

Plans for 1.15.3 release, 1.16.x branch
by Charles R Harris 10 Oct '18

10 Oct '18

Hi All, I'm planning to do a 1.15.3 release in about two weeks, if there are fixes or regressions that you feel have slipped by without getting marked for backport, please comment. I'm planning on branching 1.16.x in mid November, which should provide enough time for 1-2 release candidates and a release before the end of the year. This is all contingent on having the numpy-wheels repo working again. The latest 0.32 release of the wheel package broke everything, so we will either need to pin the version or wait on potential fixes currently under discussion. Chuck

2 1

Fwd: Performance feature for np.isin and np.in1d
by Miles Cranmer 10 Oct '18

10 Oct '18

(Not sure what the right list is for this) Hi, I have started a PR for a "fast_integers" flag for np.isin and np.in1d which greatly increases performance when both arrays are integral. It works by creating a boolean array with elements set to 1 where the parent array (ar2) has elements and 0 otherwise. This array is then indexed by the child array (ar1) to create the output. https://github.com/numpy/numpy/pull/12065 Thoughts on this? Please let me know if you have any questions about my addition. Thank you. Best regards, Miles

1 1

Determining NPY_ABI_VERSION statically in compiled extensions
by Robert T. McGibbon 09 Oct '18

09 Oct '18

Is anyone aware of any tricks that can be played with tools like `readelf`, `nm` or `dlopen` / `dlsym` in order to statically determine what version of numpy a fully-compiled C extension (for example, found inside a wheel) was compiled against? Even if it only worked with relatively new versions of numpy, that would be fine. I'm interested in creating something similar to https://github.com/pypa/auditwheel that could statically check for compatibility between wheel files and python installations, in situations where the metadata about how they were compiled is missing. -- -Robert

2 2

ndrange, like range but multidimensiontal
by Mark Harfouche 08 Oct '18

08 Oct '18

Hi All, I've been using numpy array objects to store collections of 2D (and soon ND) variables. When iterating through these collections, I often found it useful to use `ndindex`, which for `for loops` behaves much like `range` with only a `stop` parameter. That said, it lacks a few features that are now present in `range` are missing from `ndindex`, most notably the ability to iterate over a subset of the ndindex. I found myself often writing `itertools.product(range(1, data.shapep[0]), range(3, data.shape[2]))` for custom iterations. While it does flatten out the for loop, it is arguable less readable than having 1 or 2 levels of nested for loops. It is quite possible that `nditer` would solve my problems, but unfortunately I am still not able to make sense of then numerous options it has. I propose an `ndrange` class that can be used to iterate over nd-collections mimicking the API of `range` as much as possible and adapting it to the ND case (i.e. returning tuples instead of singletons). Since this is an enhancement proposal, I am bringing the idea to the mailing list for reactions. The implementation in this PR https://github.com/numpy/numpy/pull/12094 is based on keeping track of a tuple of python `range` range objects. The `__iter__` method returns the result of `itertools.product(*self._ranges)` By leveraging python's `range` implementation, operations like `containement` `index`, `reversed`, `equality` and most importantly slicing of the ndrange object are possible to offer to the general numpy audiance. For example, iterating through a 2D collection but avoiding indexing the first and last column used to look like this: ``` c = np.empty((4, 4), dtype=object) # ... compute on c for j in range(c.shape[0]): for i in range(1, c.shape[1]-1): c[j, i] # = compute on c[j, i] that depends on the index i, j ``` With `np.ndrange` it can look something like this: ``` c = np.empty((4, 4), dtype=object) # ... compute on c for i in np.ndrange(c.shape)[:, 1:-1]: c[i] # = some operation on c[i] that depends on the index i ``` very pythonic, very familiar to numpy users Thank you for the feedback, Mark References: An issue requesting expansion to the ndindex API on github: https://github.com/numpy/numpy/issues/6393

2 1

Deactivated appveyor
by Charles R Harris 05 Oct '18

05 Oct '18

Hi All, This is just to notify everyone making PRs on github that I have deactivated the appeveyor webhook now that azure testing seems to be working for the windows tests. Azure is much faster and I expect that travis or one of the other platforms will become the testing bottleneck. I think the new tests can be activated by closing/opening PRs, and maybe it will happen automatically on updates, we will see. Finding details of failing tests is a bit of a hassle, but there is an obscure button at the bottom of the default details page that you can click for actual details, although you will still need to hunt around to find the pipeline. This is still somewhat experimental, so post your feedback and complaints here. Chuck

1 0