NumPy-Discussion October 2020

numpy-discussion@python.org

30 participants
26 discussions

Documentation Team meeting - Monday June 8th

by Melissa Mendonça

Hi all! A reminder that on Monday, June 8, we have another documentation team meeting at 3PM UTC**. If you wish to join on Zoom, you need to use this link https://zoom.us/j/420005230 Here's the permanent hackmd document with the meeting notes: https://hackmd.io/oB_boakvRqKR-_2jRV-Qjg <https://www.google.com/url?q=https%3A%2F%2Fhackmd.io%2FoB_boakvRqKR-_2jRV-Q…> Hope to see you around (especially if you want to introduce yourself or discuss ideas for Google Season of Docs). ** You can click this link to get the correct time at your timezone: https://www.timeanddate.com/worldclock/fixedtime.html?msg=NumPy+Documentati… - Melissa

5 days, 13 hours

Re: [Numpy-discussion] example reading binary Fortran file

by Neil Martinsen-Burrell

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

4 months, 1 week

NumPy 1.20.x branch in two weeks

by Charles R Harris

Hi All, Time to start planning for the 1.20.x branch. These are my thoughts at the moment: - Keep support for Python 3.6. Python 3.7 came out in June 2018, which seems too recent to be our oldest supported version. - Drop Python 3.6 for 1.21.x, that will make the oldest supported version about three years old. - Drop manylinux1 for 1.21.x. It would be nice to drop earlier, but manylinux2010 is pretty recent. There were 33 wheels in the 1.19.3 release, I think we can live with that for 1.20.x. I'm more worried about our tools aging out. After Python has settled into its yearly release cycle, I think we will end up supporting the latest 4 versions. Thoughts? Chuck

12 months

Add sliding_window_view method to numpy

by Zimmermann Klaus

Hello, I would like to draw the attention of this list to PR #17394 [1] that adds the implementation of a sliding window view to numpy. Having a sliding window view in numpy is a longstanding open issue (cf #7753 [2] from 2016). A brief summary of the discussions surrounding it can be found in the description of the PR. This PR implements a sliding window view based on stride tricks. Following the discussion in issue #7753, a first implementation was provided by Fanjin Zeng in PR #10771. After some discussion, that PR stalled and I picked up the issue in the present PR #17394. It is based on the first implementation, but follows the changed API as suggested by Eric Wieser. Code reviews have been provided by Bas van Beek, Stephen Hoyer, and Eric Wieser. Sebastian Berg added the "62 - Python API" label. Do you think this is suitable for inclusion in numpy? Do you consider the PR ready? Do you have suggestions or requests? Thanks for your time and consideration! Klaus [1] https://github.com/numpy/numpy/pull/17394 [2] https://github.com/numpy/numpy/issues/7753

12 months

start of an array (tensor) and dataframe API standardization initiative

by Ralf Gommers

Hi all, I'd like to share this announcement blog post about the creation of a consortium for array and dataframe API standardization here: https://data-apis.org/blog/announcing_the_consortium/. It's still in the beginning stages, but starting to take shape. We have participation from one or more maintainers of most array and tensor libraries - NumPy, TensorFlow, PyTorch, MXNet, Dask, JAX, Xarray. Stephan Hoyer, Travis Oliphant and myself have been providing input from a NumPy perspective. The effort is very much related to some of the interoperability work we've been doing in NumPy (e.g. it could provide an answer to what's described in https://numpy.org/neps/nep-0037-array-module.html#requesting-restricted-sub… ). At this point we're looking for feedback from maintainers at a high level (see the blog post for details). Also important: the python-record-api tooling and data in its repo has very granular API usage data, of the kind we could really use when making decisions that impact backwards compatibility. Cheers, Ralf

1 year

Ndarray static typing: Order of generic types

by bas van beek

Hey all, With the recent merging of numpy/numpy#16759<https://github.com/numpy/numpy/pull/16759> we're at the point where `ndarray` can be made generic w.r.t. its dtype and shape. An open question which yet remains is to order in which these two parameters should appear (numpy/numpy#16547<https://github.com/numpy/numpy/issues/16547>): * `ndarray[Dtype, Shape]` * `ndarray[Shape, Dtype]` There has been a some discussion about this question in issue 16547, but a consensus has not yet to be reached. Most people seem to slightly preferring one option over the other. Are there any further thoughts on this subject? Regards, Bas van Beek

1 year

Allow __getitem__ to support custom objects

by Aaron Meurer

For ndindex (https://quansight.github.io/ndindex/), the biggest issue with the API is that to use an ndindex object to actually index an array, you have to use a[idx.raw] instead of a[idx]. This is because for NumPy arrays, you cannot allow custom objects to be indices. The exception is objects that define __index__, but this only works for integer indices. If __index__ returns anything other than an integer, you get an IndexError. This is annoying because it's easy to forget to do this when working with the ndindex API, and the error message from NumPy isn't informative about what went wrong unless you know to expect it. I'd like to propose an API that would allow custom objects to define how they should be converted to a standard NumPy index, similar to __index__ but that supports all index types. I think there are two options here: - Allow __index__ to return any index type, not just integers. This is the simplest because it reuses an existing API, and __index__ is the best possible name for this API. However, I'm not sure, but this may actually conflict with the text of PEP 357 (https://www.python.org/dev/peps/pep-0357/). Also, some other APIs use __index__ to check if something is an indexable integer, which wouldn't accept generic index. For example, elements of a slice can be any object that defines __index__. - Add a new __numpy_index__ API that works like def __numpy_index__(self): return <tuple, integer, slice, newaxis, ellipsis, or integer or boolean array> In NumPy, __getitem__ and __setitem__ on ndarray would first check if the input index type is one of the known types as it currently does, then it would try __index__, and if neither of those fails, it would call __numpy_index__(index) and use that. Note: there is a more general way that NumPy arrays could allow __getitem__ to be defined on custom objects, which I am NOT proposing. Instead of an API that returns one of the current predefined index types (tuple, integer, slice, newaxis, ellipsis, or integer or boolean array), there could instead be an API that takes the array as input and returns another array (or view) as an output. This would allow an object to define itself as an index in arbitrary ways, even if such an index would not actually be possible via traditional indexing. There are definitely some interesting ideas that could be done with this, but this idea would be much more complicated, and isn't something that I need. Unless the community feels that a more general API like this would be preferred, I would suggest deferring something like it to a later discussion. What would be the best way to go about getting something like this implemented? Is it simple enough that we can just work out the details here and on a pull request, or should I write a NEP? Aaron Meurer

1 year

Multinomial random sampling

by Currurant

I wonder if there is an efficient way to draw multinomial random samples. For univariate sampling, we can do that for different parameters and desired shape, but neither of rng nor multinomial function in numpy.random is able to achieve that. To be specific, can we draw multinomial samples for different probabilities and different n? I asked the question on Stackoverflow here: https://stackoverflow.com/questions/64529620/is-there-an-efficient-way-to-g…. Seems like there isn't a function to do that. Thank you and appreciate numpy, Daniel -- Sent from: http://numpy-discussion.10968.n7.nabble.com/

1 year

Should we transfer MacPython/numpy-wheels to the NumPy org?

by Charles R Harris

Hi All, Seems that is pretty easy to transfer a github repo to another owner. Should we do this for the numpy-wheels repo? That would put all the management in one place and now might be a good time to do it before the 1.20.x branch. Chuck

1 year

NumPy 1.19.3 release

by Charles R Harris

Hi All, On behalf of the NumPy team I am pleased to announce that NumPy 1.19.3 has been released. NumPy 1.19.3 is a small maintenance release with two major improvements: - Python 3.9 binary wheels on all supported platforms, - OpenBLAS fixes for Windows 10 version 2004 fmod bug. This release supports Python 3.6-3.9 and is linked with OpenBLAS 3.12 to avoid some of the fmod problems on Windows 10 version 2004. Microsoft is aware of the problem and users should upgrade when the fix becomes available, the fix here is limited in scope. NumPy Wheels for this release can be downloaded from the PyPI <https://pypi.org/project/numpy/1.19.3/>, source archives, release notes, and wheel hashes are available on Github <https://github.com/numpy/numpy/releases/tag/v1.19.3>. Linux users will need pip >= 0.19.3 in order to install manylinux2010 and manylinux2014 wheels. *Contributors* A total of 8 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Charles Harris - Chris Brown + - Daniel Vanzo + - E. Madison Bray + - Hugo van Kemenade + - Ralf Gommers - Sebastian Berg - @danbeibei + *Pull requests merged* A total of 10 pull requests were merged for this release. - #17298: BLD: set upper versions for build dependencies - #17336: BUG: Set deprecated fields to null in PyArray_InitArrFuncs - #17446: ENH: Warn on unsupported Python 3.10+ - #17450: MAINT: Update test_requirements.txt. - #17522: ENH: Support for the NVIDIA HPC SDK nvfortran compiler - #17568: BUG: Cygwin Workaround for #14787 on affected platforms - #17647: BUG: Fix memory leak of buffer-info cache due to relaxed strides - #17652: MAINT: Backport openblas_support from master. - #17653: TST: Add Python 3.9 to the CI testing on Windows, Mac. - #17660: TST: Simplify source path names in test_extending. Cheers, Charles Harris

1 year

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

NumPy-Discussion October 2020