Mailman 3 December 2015 - NumPy-Discussion

[ANN-JOB] Project Jupyter is hiring a Project Manager - position at UC Berkeley
by Fernando Perez 22 Dec '15

22 Dec '15

Hi all, [ please direct all replies directly to me ] Project Jupyter is announcing the opening of a position for a full-time project manager, who will help us coordinate our technical development, engage the open source community and work with our multiple stakeholders in academia and industry. If you have experience leading technical teams in open source communities, we'd love to hear from you! In the last few years the project has rapidly grown in multiple directions, and this presents both challenges and opportunities. We are looking for someone who can help us harness the energy and activity from our many contributors that include those funded by our research grants, our industry partners, and the entire open source community. The role of the project manager is to help us maintain this activity focused into a solid whole, so we can deliver timely and robust releases, evolve our architecture coherently, ensure our documentation and communication matches our technical foundation, and continue engaging a wide range of stakeholders to evolve the project in new, interesting and valuable directions. This position will be hosted at the Berkeley Institute for Data Science, working locally with Fernando Perez, Matthias Bussonnier, and our new postdoctoral scholars. But the scope of this role is the entire project, so we are looking for a candidate who will be regularly communicating with project stakeholders from all locations, traveling to conferences, development workshops and other project activities. For specific details on the position and to apply, you can learn more at jobs.berkeley.edu, Job ID #20975: https://hrw-vip-prod.is.berkeley.edu/psc/JOBSPROD/EMPLOYEE/HRMS/c/HRS_HRAM.… Note that while the application review date is listed as January 1, 2016, we will be considering applicants past that date (that is the cutoff for us to be allowed to look at incoming applications). The search will remain open until filled. -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail

1 0

Re: [Numpy-discussion] PR for complex np.interp, question about assert_almost_equal
by Peter Creasey 22 Dec '15

22 Dec '15

>> > assert_almost_equal >> > TypeError: Cannot cast array data from dtype('complex128') to >> dtype('float64') according to the rule 'safe' >> >> > > Hi Peter, that error is unrelated to assert_almost_equal. What happens is > that when you pass in a complex argument `fp` to your modified > `compiled_interp`, you're somewhere doing a cast that's not safe and > trigger the error at > https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/ctors.…. Thanks a lot Ralf! The build log I was looking at ( https://travis-ci.org/numpy/numpy/jobs/98198323 ) really confused me by not mentioning the function call that wrote the error, but now I think I understand and can recreate the failure in my setup. Best, Peter

1 0

PR for complex np.interp, question about assert_almost_equal
by Peter Creasey 22 Dec '15

22 Dec '15

Hi all, I submitted a PR (#6872) for using complex numbers in np.lib.interp. The tests pass on my machine, but I see that the TravisCI builds are giving assertion fails (on my own test) with python 3.3 and 3.5 of the form: > assert_almost_equal > TypeError: Cannot cast array data from dtype('complex128') to dtype('float64') according to the rule 'safe' When I was writing the test I used np.testing.assert_almost_equal with complex128 as it works in my python 2.7, however having checked the docstring I cannot tell what the expected behaviour should be (complex or no complex allowed). Should my test be changed or the assert_almost_equal? Best, Peter

3 2

Why is masked string array fill value 'N/A'
by Charles R Harris 21 Dec '15

21 Dec '15

Why, why, why is it not the empty string. you can't actually fill an `S2` type with such a string, hence one must convert masked string arrays to a list of strings to use it. That is pathetic. Chuck

1 0

numpy.power -> numpy.random.choice Probabilities don't sum to 1
by Ryan R. Rosario 19 Dec '15

19 Dec '15

Hi, I have a matrix whose entries I must raise to a certain power and then normalize by row. After I do that, when I pass some rows to numpy.random.choice, I get a ValueError: probabilities do not sum to 1. I understand that floating point is not perfect, and my matrix is so large that I cannot use np.longdouble because I will run out of RAM. As an example on a smaller matrix: np.power(mymatrix, 10, out=mymatrix) row_normalized = np.apply_along_axis(lambda x: x / np.sum(x), 1, mymatrix) sums = row_normalized.sum(axis=1) sums[np.where(sums != 1)] array([ 0.99999994, 0.99999994, 1.00000012, ..., 0.99999994, 0.99999994, 0.99999994], dtype=float32) np.random.choice(range(row_normalized.shape[0]), 1, p=row_normalized[0, :]) … ValueError: probabilities do not sum to 1 I also tried the normalize function in sklearn.preprocessing and have the same problem. Is there a way to avoid this problem without having to make manual adjustments to get the row sums to = 1? — Ryan

4 3

7th Annual Scientific Software Days Conference (25-26 February, 2016 -- Austin, TX)
by Damon McDougall 18 Dec '15

18 Dec '15

The 7th Annual Scientific Software Days Conference (SSD) <http://scisoftdays.org/> targets users and developers of scientific software. The conference will be held at the University of Texas at Austin Thursday Feb 25 - Friday Feb 26, 2016 and focuses on two themes: a) sharing best practices across scientific software communities; b) sharing the latest tools and technology relevant to scientific software. Past keynotes speakers include Greg Wilson (2008), Victoria Stodden (2009), Steve Easterbrook (2010), Fernando Perez (2011), Will Schroeder (2012), Neil Chue Hong (2013). This year's list of speakers include: - Brian Adams (Sandia, Dakota): http://www.sandia.gov/~briadam/index.html - Iain Dunning (MIT, Julia Project): http://iaindunning.com/ - Victor Eijkhout (TACC): http://pages.tacc.utexas.edu/~eijkhout/ - Robert van de Geijn (keynote, UT Austin, libflame): https://www.cs.utexas.edu/users/rvdg/ - Jeff Hammond (Intel, nwchem): https://jeffhammond.github.io/ - Mark Hoemmen (keynote, Sandia, Trilinos): https://plus.google.com/+MarkHoemmen - James Howison (UT Austin): http://james.howison.name/ - Fernando Perez (Berkeley, IPython): http://fperez.org/ - Cory Quammen (Kitware, Paraview/VTK): http://www.kitware.com/company/team/quammen.html - Ridgway Scott (UChicago, FEniCS): http://people.cs.uchicago.edu/~ridg/ - Roy Stogner (UT Austin, LibMesh): https://scholar.google.com/citations?user=XcurJI0AAAAJ In additional, we solicit poster submissions that share novel uses of scientific software. Please send an abstract of less than 250 words to ssd-organizers(a)googlegroups.com. Limited travel funding for students and early career researchers who present posters will be available. Early-bird registration fees (before Feb 10th): Students: $35 Everyone else: $50 Late registration fees (Feb 10th onwards): Students: $55 Everyone else: $70 More details, including how to register, will appear on the website in the coming weeks: http://scisoftdays.org/ Regards, S. Fomel (UTexas), T. Isaac (UChicago), M. Knepley (Rice), R. Kirby (Baylor), Y. Lai (UTexas), K. Long (Texas Tech), D. McDougall (UTexas), J. Stewart (Sandia)

1 0

Introducing outer/orthonongal indexing to numpy
by Sebastian Berg 18 Dec '15

18 Dec '15

Hello all, sorry for cross posting (discussion should go to the numpy list). But I would like to get a bit of discussion on the introduction of (mostly) two new ways to index numpy arrays. This would also define a way for code working with different array-likes, some of which implement outer indexing (i.e. xray and dask I believe), to avoid ambiguity. The new methods are (names up for discussion): 1. arr.oindex[...] 2. arr.vindex[...] The difference beeing that `oindex` will return outer/orthogonal type indexing, while `vindex` would be a (hopefully) less confusing variant of "fancy" indexing. The biggest reason for introducing this is to provide `oindex` for situations such as: >>> arr = np.arange(25).reshape((5, 5)) >>> arr[[0, 1], [1, 2]] array([1, 7]) >>> # While most might expect the result to be: >>> arr.oindex[[0, 1], [1, 2]] array([[1, 2], [6, 7]]) To provide backwards compatibility the current plan is to also introduce `arr.legacy_index[...]` or similar, with the (long term) plan to force the users to explicitly choose `oindex`, `vindex`, or `legacy_index` if the indexing operation is otherwise not well defined. There are still some open questions for me regarding, for example: * the exact time line (should we start deprecation immediately, etc.) * the handling of boolean indexing arrays * questions that might crop up about other array-likes/subclasses * Are there indexing needs that we are forgetting but are related? More details the current status of my NEP, which has a lot of examples, can be found at: https://github.com/numpy/numpy/pull/6256/files?short_path=01e4dd9#diff-01e4… and comments about are very welcome. There is a fully functional implementation available at https://github.com/numpy/numpy/pull/6075 and you can test it using (after cloning numpy): git fetch upstream pull/6075/head:pr-6075 && git checkout pr-6075; python runtests.py --ipython # Inside ipython (too see the deprecations): import warnings; warnings.simplefilter("always") My current hope for going forward is to get clear feedback of what is wanted, for the naming and generally from third party module people, so that we can polish up the NEP and the community can accept it. With good feedback, I think we may be able to get the new attributes into 1.11. So if you are interested in teaching and have suggestions for the names, or have thoughts about subclasses, or... please share your thoughts! :) Regards, Sebastian

1 0

array_equal too strict?
by Nico Schlömer 17 Dec '15

17 Dec '15

Hi everyone, I noticed a funny behavior in numpy's array_equal. The two arrays ``` a1 = numpy.array( [3.14159265358979320], dtype=numpy.float64 ) a2 = numpy.array( [3.14159265358979329], dtype=numpy.float64 ) ``` (differing the in the 18th overall digit) are reported equal by array_equal: ``` print(numpy.array_equal(a1, a2)) # output: true ``` That's expected because the difference is only in the 18th overall digit, and the mantissa length of float64 is 52 bits [1], i.e., approx 15.6 decimal digits. Moving the difference to the 17th overall digit should also be fine, however: ``` a1 = numpy.array( [3.1415926535897930], dtype=numpy.float64 ) a2 = numpy.array( [3.1415926535897939], dtype=numpy.float64 ) print(numpy.array_equal(a1, a2)) # output: false ``` It gets even more visible with float32 and its 23 mantissa bits (i.e., 6.9 decimal digits): ``` a1 = numpy.array( [3.14159260], dtype=numpy.float32 ) a2 = numpy.array( [3.14159269], dtype=numpy.float32 ) print(numpy.array_equal(a1, a2)) # output: false ``` The difference is only in the 9th decimal digit, still `array_equal_ detects the difference. I'm not sure where I'm going wrong here. Any hints? Cheers, Nico [1] https://docs.scipy.org/doc/numpy-1.10.1/user/basics.types.html

4 3

A minor clarification no why count_nonzero is faster for boolean arrays
by Raghav R V 17 Dec '15

17 Dec '15

I was just playing with `count_nonzero` and found it to be significantly faster for boolean arrays compared to integer arrays >>> a = np.random.randint(0, 2, (100, 5)) >>> a_bool = a.astype(bool) >>> %timeit np.sum(a) 100000 loops, best of 3: 5.64 µs per loop >>> %timeit np.count_nonzero(a) 1000000 loops, best of 3: 1.42 us per loop >>> %timeit np.count_nonzero(a_bool) 1000000 loops, best of 3: 279 ns per loop (but why?) I tried looking into the code and dug my way through to this line <https://github.com/numpy/numpy/blob/c0e48cfbbdef9cca954b0c4edd0052e1ec8a30a…>. I am unable to dig further. I know this is probably a trivial question, but was wondering if anyone could provide insight on why this is so? Thanks R

4 4

Re: [Numpy-discussion] performance solving system of equations in numpy and MATLAB
by Edward Richards 17 Dec '15

17 Dec '15

Thanks everyone for helping me glimpse the secret world of FORTRAN compilers. I am running a Linux machine, so I will look into MKL and openBLAS. It was easy for me to get a Intel parallel studio XE license as a student, so I have options.

1 0

NumPy-Discussion December 2015