Mailman 3 March 2023 - NumPy-Discussion

1D ndarray to java double[]
by Bill Ross 20 Mar '23

20 Mar '23

How best to write a 1D ndarray as a block of doubles, for reading in java as double[] or a stream of double? Maybe the performance of simple looping over doubles in python.write() and java.read() is fine, but maybe there are representational diffs? Maybe there's a better solution for the use case? Use case: I get the ndarray from keras, and it represents a 2D distance matrix. I want to find the top-50 matches for each item, per row and column. I'm looking at moving the top-50 task to java for its superior parallel threading. (Java doesn't fork processes with a copy of the array, which is ~5% of memory; rather one gets 1 process with e.g. 1475% CPU.) Thanks, Bill Ross -- Phobrain.com

2 5

ANN: pvlib-0.9.5 released
by Dr. Mark Alexander Mikofski PhD 20 Mar '23

20 Mar '23

Dear Pythonistas and solar power enthusiasts, The maintainers are happy to announce a new release of pvlib python: software for simulating performance of photovoltaic solar energy systems. *v0.9.5 Highlights:* * The infinite_sheds model now has options to use the hay-davies transposition model and faster vectorized calculations. * New models for irradiance decomposition (boland) and relative airmass (gueymard2003). * Model extensions for multiple strings in pvlib.snow.loss_townsend and AR coating in pvlib.iam.physical. * Updated the parameters database for the ADR inverter model. * Various other bug fixes and testing updates. For the full list of what's new, see the documentation: https://pvlib-python.readthedocs.io/en/stable/whatsnew.html *Releases are available from PyPI and the conda-forge channel:* * https://pypi.org/project/pvlib/ * https://anaconda.org/conda-forge/pvlib and https://anaconda.org/conda-forge/pvlib-python NOTE: new pvlib releases are no longer uploaded to the "pvlib" conda channel. Please install from PyPI or the conda-forge channel instead. *Read the Documentation:* * https://pvlib-python.readthedocs.io/en/stable/index.html *Report issues & contribute:* * https://github.com/pvlib/pvlib-python *Community discussion & support:* * https://groups.google.com/g/pvlib-python * https://github.com/pvlib/pvlib-python/discussions *Thank you for using pvlib python!* -- Mark Mikofski, PhD (2005) *Fiat Lux*

1 0

`dtype` parameter in `clip` method
by Sergio Callegari 17 Mar '23

17 Mar '23

I am trying to use the `clip` method to transform floats to integers with clipping. I was expecting to be able to do both the clipping and the type conversion at once, passing the `dtype` parameter to `clip`, together with a suitable `casting` param. Such expectation came from reading the documentation of the `dtype` param for u-funcs that states that this parameter "Overrides the DType of the output arrays". Unfortunately, using the `dtype` parameter with `clip` causes the type conversion to be practiced before the clipping, not after it. Is this how it is expected to operate?

3 2

15ᵗʰ Advanced Scientific Programming in Python in Heraklion, Crete, Greece, 27 August – 3 September,2023
by Tiziano Zito 17 Mar '23

17 Mar '23

ASPP2023: 15ᵗʰ Advanced Scientific Programming in Python ======================================================== a Summer School https://aspp.school Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists have been trained to use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques and best practices which are standard in the industry, but especially tailored to the needs of a programming scientist. Lectures are interactive and allow students to acquire direct hands-on experience with the topics. Students will work in pairs throughout the school and will team up to practice the newly learned skills in a real programming project — an entertaining computer game. We use the Python programming language for the entire course. Python works as a simple programming language for beginners, but more importantly, it also works great in scientific simulations and data analysis. Python is the standard tool for the programming scientist due to clean language design, ease of extensibility, and the great wealth of open source libraries for scientific computing and data visualization. This school is targeted at PhD students, postdocs and more senior researchers from all areas of science. Competence in Python or in another language such as Java, JavaScript, C/C++, MATLAB, or R is absolutely required. Basic knowledge of Python and git or another version control system is assumed. Participants without any prior experience with Python or git should work through the proposed introductory material before the course. We care for diversity and inclusion, and strive for a welcoming atmosphere to programming scientists of all levels. In particular, we have focused on recruiting an international and gender-balanced pool of students. Date & Location =============== 27 August – 3 September, 2023. Heraklion, Crete, Greece Application =========== You can apply online: https://aspp.school Application deadline: 23:59 UTC, Monday 1 May, 2023. There will be no deadline extension, so be sure to apply on time. Invitations and notification of rejection will be sent by Sunday 28 May, 2023. Participation is for free, i.e. no fee is charged! Participants however should take care of travel, living, and accommodation expenses by themselves. Program ======= • Large-scale collaborative scientific code development with git and GitHub • Best practices in data visualization • Testing and debugging scientific code • Advanced NumPy • Organizing, documenting, and distributing scientific code • Scientific programming patterns in Python • Writing parallel applications in Python • Profiling and speeding up scientific code • Programming in teams Faculty ======= • Aitor Morales-Gregorio, Theoretical Neuroanatomy, Institute of Neuroscience and Medicine (INM-6), Forschungszentrum Jülich Germany • Guillermo Aguilar, Department of Computational Psychology, Technische Universität Berlin Germany • Jakob Jordan, Department of Physiology, University of Bern Switzerland • Lisa Schwetlick, Experimental and Biological Psychology, Universität Potsdam Germany • Pamela Hathway, Orange Business, Berlin/Nürnberg Germany • Pietro Berkes, NAGRA Kudelski, Lausanne Switzerland • Rike-Benjamin Schuppner, Institute for Theoretical Biology, Humboldt-Universität zu Berlin Germany • Tiziano Zito, innoCampus, Technische Universität Berlin Germany • Verjinia Metodieva, NeuroCure, Charité – Universitätsmedizin Berlin Germany • Zbigniew Jędrzejewski-Szmek, Red Hat Inc., Warsaw Poland Organizers ========== Head of the organization for ASPP and responsible for the scientific program: • Tiziano Zito, innoCampus, Technische Universität Berlin Organization team from IMBB/FORTH: • Athanasia Papoutsi, Institute of Molecular Biology and Biotechnology of the Foundation for Research and Technology – Hellas, Heraklion Greece • Emmanouil Froudarakis, Institute of Molecular Biology and Biotechnology of the Foundation for Research and Technology – Hellas, Heraklion Sponsors ======== We are able to hold this year's ASPP school thanks to the financial support of the Tübingen AI Center. The Institute of Molecular Biology & Biotechnology of the Foundation for Research and Technology – Hellas is hosting us in Heraklion and is taking care of the local organization. We also explicitly thank Prof. Felix Wichmann at the Neural Information Processing Group, Eberhard Karls Universität Tübingen, Germany, for his invaluable help. Website: https://aspp.school Contact: info(a)aspp.school

1 0

Would it be sensible to augment `casting` policies with a `clip` policy?
by Sergio Callegari 16 Mar '23

16 Mar '23

I think it is a not so infrequent requirement to do castings from float types to integer types of from float types to lower precision float types where it is OK to have a loss of precision, but you want to be sure not to get values that are completely unrelated to the input ones because of wrapping effects. A typical application can be in sound/image processing, where you do filtering with floating point, but then you want to go back to integer types for the final result and if you have values too large for the integer type, then you want the values to saturate to the maximum ones and not certainly a positive value to become negative because of wrapping. Missing the clipping step in some cases can even be dangerous (e.g. resulting in sound signals that once played through a headset can get so loud to damage the hear). As of today, I think that this conversion can only be done with `.clip(....).astype(...)`. However, this is a bit inconvenient: on one hand, for the clip step you need to have the minimum and maximum values, which could be automatically determined from the type you want (but clip knows nothing about it yet). Secondly, this method concatenation creates temporary copies that would be unneeded if the operation could be done in just one step. Ideally, I'd like to be able to write `a.astype(np.int16, casting=`clip`).

1 0

Faster Integer operations with oneDNN
by Tilmann E. Bartsch 16 Mar '23

16 Mar '23

Hello, I noticed that integer operations in NumPy are very slow. A quick Google research shows the reason is the missing optimization for that in BLAS. I experimented a bit and found that Intel's oneDNN is a good solution for me; at least on my Intel CPU. Comparison of different int8 MMM methods in NumPy. <https://tebartsch.ai/blog/oneDNN-Adventures.html> <https://tebartsch.ai/blog/oneDNN-Adventures.html>(see: https://tebartsch.ai/blog/oneDNN-Adventures.html and https://github.com/tebartsch/numpy-int-ops) I'm thinking about building a pip package which provides fast integer operations for NumPy arrays based on oneDNN. I would like to discuss this idea with you. Maybe there is even a path of bringing something like this into NumPy itself? Cheers, Tilmann -- *Tilmann E. Bartsch* Website: tebartsch.ai <https://tebartsch.ai> Email: info(a)tebartsch.ai

1 0

backwards incompatible change to numpy.busday_count
by Pieter Eendebak 14 Mar '23

14 Mar '23

Hi, The numpy.busday_count contains a bug where the number of days returned for a call np.busday_count(begin_date, end_date) can be incorrect if end_date is before begin_data in time. The problem has been around since 2011. There are two possible ways to solve this: 1. Fix the bug. This is proposed in https://github.com/numpy/numpy/pull/23229 2. Update the documentation and accept the behaviour The issue has been discussed in the triage meeting and the proposal is to fix the bug ( https://github.com/numpy/numpy/issues/23197#issuecomment-1460610260). Since this modifies the behaviour of a method that has been around for a long time, we wanted to check whether there are any objections or concerns. - Are there objections to fixing the bug and thereby modifying the behaviour? - Would there be objections to deprecating the busday functionality and moving it to a separate package (e.g. numpy financials) With kind regards, Pieter Eendebak

1 0

Documentation Meeting - Monday March 13th, 4PM UTC
by Ross Barnowski 13 Mar '23

13 Mar '23

Hi all, We'll have the bi-weekly NumPy documentation meeting tomorrow, Monday 3/13 at 4PM UTC. *Note that this will be an hour later in the US due to daylight savings! *Check out the community calendar <https://scientific-python.org/calendars/> to see if you're affected. ~Ross

1 0

removing NUMPY_EXPERIMENTAL_ARRAY_FUNCTION env var
by Ralf Gommers 13 Mar '23

13 Mar '23

Hi all, In https://github.com/numpy/numpy/pull/23364 we touched on the NUMPY_EXPERIMENTAL_ARRAY_FUNCTION environment variable. This was a temporary feature during the introduction of `__array_function__` (see NEP 18), but we never removed it. I propose we do so now, since it is cumbersome to have around (see gh-23364 for one reason why). GitHub code search shows some usages, but that's mostly old code to explicitly enable it or print diagnostic info it looks like - none of it seemed relevant. In case there is any need for this functionality to disable `__array_function__`, then please speak up. In that case it probably applies to `__array_ufunc__` as well, and there should be a better way to do this than an undocumented environment variable with "experimental" in the name. Cheers, Ralf

3 3

ENH: add dtype and order to np.ctypeslib.as_array
by Rob Farmer 13 Mar '23

13 Mar '23

Hi, First-time contribution so I wanted to check in before pushing a PR. I'd like to add an 'order' and 'dtype' flags to np.ctypeslib.as_array. The order flag is for setting things as a Fortran array order while dtype is needed if all you have is a c_void_p (rather than a ctype array). I have a working version at: https://github.com/rjfarmer/numpy/tree/ctypelibs_as_array_flags Thoughts? Rob

1 0