Mailman 3 April 2019 - NumPy-Discussion

Re: [Numpy-discussion] overhauling numpy.random and randomgen Message-ID:
by Kevin Sheppard 19 Apr '19

19 Apr '19

> Rather than "base RNG", what about calling these classes a "random source" or "random stream"? In particular, I would suggest defining two Python classes: > - np.random.Generator as a less redundant name for what is currently called RandomGenerator > - np.random.Source or np.random.Stream as an abstract base class for what are currently called "base RNGs" Naming is definitely hard. Simple RNGs are currently called basic RNGs which was inspired by mkl-random. 'source' sounds OK to me, but sort of hides the fact that these are the actual Psuedo RNGs. `stream` has a technical meaning (a single PRNG make produce multiple independent streams) and IMO should be avoided since this might lead to confusion. Perhaps source_rng (or in docs Source RNG)? RandomGenerator is actually RandomTransformer, but I didn't like the latter name. > There are also a couple of convenience attributes in the user-facing API that I would suggest refining: > - The "brng" attribute of RandomGenerator is not a very descriptive name. I would prefer "stream" or "source", or the more explicit "base_rng" if we stick with that term. > - I don't think we need the "generator" property on base RNG objects. It is fine to require writing np.random.Generator(base) instead. Looking at the implementation, .generator caches the RandomGenerator objects it creates on the base RNG, which creates a reference cycle. Yes, Python can garbage collect reference cycles, but this is still a muddled data model. The attribute name should match the final (descriptive) name, whatever it is. In RandomGen I am using the `basic_rng` attribute name, but this could be `source`. I also use a property so that the attribute can have a docstring attached for use in IPython. I think this is more user-friendly. I think dropping the `generator` property on the basic RNGs is reasonable. It was a convenience but is awkward, and I always understood that it creates a cycle. > Finally, why do we expose the np.random.gen object? I thought part of the idea with the new API was to avoid global mutable state. Module level functions are essential for quick experiments and should be provided. The only difference here is that the singleton `seed` and `state` are no longer exposed so that it isn't possible (using the exposed API) to set the seed.

3 3

Google Summer of Docs ideas?
by Ralf Gommers 18 Apr '19

18 Apr '19

Hi all, NumFOCUS has applied as an umbrella org for the inaugural Google Summer of Docs, and we're participating. We need 1-2 ideas and work them out very well (ideas need to be high quality, it's not yet sure that NumFOCUS will get accepted as an umbrella org). Guidelines are at https://developers.google.com/season-of-docs/docs/project-ideas#project-idea Any suggestions for ideas? Cheers, Ralf

2 4

overhauling numpy.random and randomgen
by Matti Picus 18 Apr '19

18 Apr '19

Thanks to the work of Kevin Sheppard, Robert Kern and others, the branch to merge randomgen https://github.com/bashtage/randomgen into numpy is ready for final review. The branch is here https://github.com/numpy/numpy/pull/13163. It is fully backward compatible: numpy.random.mtrand, numpy.random.RandomState, and the various stateful distributions from RandomState available as numpy.random.* produce the same streams as the current versions. The branch is intended to implement NEP 19 https://www.numpy.org/neps/nep-0019-rng-policy.html The biggest change is that now there are a variety of random number generators available https://6722-908607-gh.circle-artifacts.com/0/home/circleci/repo/doc/build/…, and a class numpy.random.RandomGenerator that can produce all the distributions from RandomState. A RandomGenerator instance is provided for convenience as numpy.random.gen Additional enhancements https://6722-908607-gh.circle-artifacts.com/0/home/circleci/repo/doc/build/… allow convenient use of the new constructs in CFFI, Numba, Ctypes, and Cython. There are a few things to address before merging: - Review the new constructs and other APIS - Decide which BRNGs to include in the first release - Check that your packages still work with the new implementations. You can do this by creating a new virtualenv and installing numpy via pip install git+https://github.com/mattip/numpy.git@randomgen <https://github.com/user/repo.git@branch> We will try to have a final video call about the branch during the upcoming meeting May 10-11, more details will follow once we schedule the call. The goal is to merge it for the upcoming 1.17 release. The expectation is that this first merge will be followed by implementation and documentation tweaks and improvements, but we hope to get the major pieces in place as much as possible now. Matti Notes: Sorry for the long urls, they link to the generated documentation from CI. They may not be available a few weeks from now. There is a tracking issue for further work related to the PR numpy.random https://github.com/numpy/numpy/issues/13164

2 1

NumPy 1.17.0
by Charles R Harris 17 Apr '19

17 Apr '19

Hi All, The middle of May seems about right for branching 1.17.x and making the first rc. I'm happy to make the release, but if someone else would like to give it a try that would be great. Chuck

1 0

NumPy 1.16.3 release
by Charles R Harris 16 Apr '19

16 Apr '19

Hi All, I'm planning on releasing 1.16.3 this weekend. There will be more maintenance releases, but if there is something important that you think needs to be in this release, please respond to this email. Chuck

1 0

Weekly NumPy Community Meeting -- April 17/ 2019
by Tyler Reddy 16 Apr '19

16 Apr '19

Hi All, There will be a community meeting tomorrow, April 17, at 12 pm Pacific Time. Anyone may join & contribute to the work in progress meeting document available here: https://hackmd.io/a03zhF4TSsewJp_fXXXwOQ?view Best wishes, Tyler

1 0

adding Quansight Labs as institutional partner
by Ralf Gommers 16 Apr '19

16 Apr '19

Hi all, Last week I joined Quansight. In Quansight Labs I will be working on increasing the contributions to core SciPy/PyData projects, and will also have funded time to work on NumPy and other projects myself. Hameer Abbasi has had and will continue to have funded time to work on NumPy as well. So I have submitted a pull request (gh-13289) to add Quansight Labs as an Institutional Partner (we list those at https://docs.scipy.org/doc/numpy/dev/governance/people.html#institutional-p…, currently only BIDS). Both Travis and I wrote blog posts about where we want to go with Quansight Labs. Given the relevance to NumPy I thought it would be appropriate to reference those posts here: - https://www.quansight.com/single-post/2019/04/02/Welcoming-Ralf-Gommers-as-… - https://labs.quansight.org/blog/2019/4/joining-labs/ Any feedback, suggestion or idea is very welcome. Cheers, Ralf

5 7

Behaviour of copy for structured dtypes with gaps
by Marten van Kerkwijk 12 Apr '19

12 Apr '19

Hi All, An issue [1] about the copying of arrays with structured dtype raised a question about what the expected behaviour is: does copy always preserve the dtype as is, or should it remove padding? Specifically, consider an array with a structure with many fields, say 'a' to 'z'. Since numpy 1.16, if one does a[['a', 'z']]`, a view will be returned. In this case, its dtype will include a large offset. Now, if we copy this view, should the result have exactly the same dtype, including the large offset (i.e., the copy takes as much memory as the original full array), or should the padding be removed? From the discussion so far, it seems the logic has boiled down to a choice between: (1) Copy is a contract that the dtype will not vary (e.g., we also do not change endianness); (2) Copy is a contract that any access to the data in the array will return exactly the same result, without wasting memory and possibly optimized for access with different strides. E.g., `array[::10].copy() also compacts the result. An argument in favour of (2) is that, before numpy 1.16, `a[['a', 'z']].copy()` did return an array without padding. Of course, this relied on `a[['a', 'z']]` already returning a copy without padding, but still this is a regression. More generally, there should at least be a clear way to get the compact copy. Also, it would make sense for things like `np.save` to remove any padding (it currently does not). What do people think? All the best, Marten [1] https://github.com/numpy/numpy/issues/13299

7 8

Does numpy depend upon a Fortran library?
by Marc F Paterno 11 Apr '19

11 Apr '19

Hello, I have encountered a problem with a binary incompatibility between the Fortran runtime library installed with numpy when using 'pip install --user numpy', and that used by the rest of my program, which is built using gfortran from GCC 8.2. The numpy installation uses libgfortran.5.dylib, and GCC 8.2 provides libgfortran.5.dylib. While investigating the source of this problem, I downloaded the numpy source (https://files.pythonhosted.org/packages/04/b6/d7faa70a3e3eac39f943cc6a6a64c…), and tried building it. The resulting libraries have no coupling to any Fortran library that I can find. I can find no Fortran source code files in the numpy source, except in tests or documentation. I am working on a MacBook laptop, running macOS Mojave, and so am using the Accelerate framework to supply BLAS. I do not understand why the pip installation of numpy includes a Fortran runtime library. Can someone explain to me what I am missing? thanks Marc

5 6

(no subject)
by Gheorghe Gardu 10 Apr '19

10 Apr '19

hello, Sir, I have downloaded mlxtend-0.15.0.0, and right now I have error when I compile python scripts. I receive "AttributeError: module 'numpy' has no attribute 'testing'", and it seems to be due to an installation issue. Do you have any idea why does this happen ? It is because of mlx? Someone said I should delete the numpy.py file and update it. Do you have idea where is numpy.py located ? Thank in advance. gheorghe gardu

1 0