Mailman 3 May 2019 - NumPy-Discussion

[ANN] pvlib-python v0.6.3: predicting power for solar energy
by Mark Mikofski 17 May '19

17 May '19

pvlib-0.6.3 has been released What's New: https://pvlib-python.readthedocs.io/en/stable/whatsnew.html#v0-6-3-may-15-2… PyPI: https://pypi.org/project/pvlib/ Read the Docs: https://pvlib-python.readthedocs.io/en/latest/ GitHub: https://github.com/pvlib/pvlib-python -- Mark Mikofski, PhD (2005) *Fiat Lux*

1 0

New Proposed Time for the NumPy Community Meeting
by Hameer Abbasi 15 May '19

15 May '19

Hello everyone! I’d like to propose that we shift the time of the NumPy community meeting to one hour earlier starting next week. The reason is twofold: One, we have an Indian student who wishes to join the meetings, and so it’d be nice to have a more reasonable time for her timezone. Second, my fast for the month of Ramadan breaks at 9:07 PM, and will only get later. I’d hate to eat during the meeting. Best Regards, Hameer Abbasi

2 1

Re: [Numpy-discussion] Style guide for numpy code?
by Joe Harrington 14 May '19

14 May '19

I have a handout for my PHZ 3150 Introduction to Numerical Computing course that includes some rules: (a) All integer-valued floating-point numbers should have decimal points after them. For example, if you have a time of 10 sec, do not use y = np.e**10 # sec use y = np.e**10. # sec instead. For example, an item count is always an integer, but a distance is always a float. A decimal in the range (-1,1) must always have a zero before the decimal point, for readability: x = 0.23 # Right! x = .23 # WRONG The purpose of this one is simply to build the decimal-point habit. In Python it's less of an issue now, but sometimes code is translated, and integer division is still out there. For that reason, in other languages, it may be desirable to use a decimal point even for counts, unless integer division is wanted. Make a comment whenever you intend integer division and the language uses the same symbol (/) for both kinds of division. (b) Use spaces around binary operations and relations (=<>+-*/). Put a space after “,”. Do not put space around “=” in keyword arguments, or around “ ** ”. (c) Do not put plt.show() in your homework file! You may put it in a comment if you like, but it is not necessary. Just save the plot. If you say plt.ion() plots will automatically show while you are working. (d) Use: import matplotlib.pyplot as plt NOT: import matplotlib.pylab as plt (e) Keep lines to 80 characters, max, except in rare cases that are well justified, such as very long strings. If you make comments on the same line as code, keep them short or break them over more than a line: code = code2 # set code equal to code2 # Longer comment requiring much more space because # I'm explaining something complicated. code = code2 code = code2 # Another way to do a very long comment, # like this one, which runs over more than # one line. (f) Keep blocks of similar lines internally lined up on decimals, comments, and = signs. This makes them easier to read and verify. There will be some cases when this is impractical. Use your judgment (you're not a computer, you control the computer!): x = 1. # this is a comment y = 378.2345 # here's another fred = chuck # note how the decimals, = signs, and # comments line up nicely... alacazamshmazooboloid = 2721 # but not always! (g) Put the units and sources of all values in comments: t_planet = 523. # K, Smith and Jones (2016, ApJ 234, 22) (h) I don't mean to start a religious war, but I emphasize the alignment of similar adjacent code lines to make differences pop out and reduce the likelihood of bugs. For example, it is much easier to verify the correctness of: a = 3 * x + 3 * 8. * short - 5. * np.exp(np.pi * omega * t) a_alt = 3 * x + 3 * 8. * anotshortvar - 5. * np.exp(np.pi * omega * t) than: a = 3 * x + 3 * 8. * short - 5. * np.exp(np.pi * omega * t) a_altvarname = 3 * x + 3*9*anotshortvar - 5. * np.exp(np.pi * omega * i) (i) Assign values to meaningful variables, and use them in formulae and functions: ny = 512 nx = 512 image = np.zeros((ny, nx)) expr1 = ny * 3 expr2 = nx * 4 Otherwise, later on when you upgrade to 2560x1440 arrays, you won't know which of the 512s are in the x direction and which are in the y direction. Or, the student you (now a senior researcher) assign to code the upgrade won't! Also, it reduces bugs arising from the order of arguments to functions if the args have meaningful names. This is not to say that you should assign all numbers to functions. This is fine: circ = 2 * np.pi * r (j) All functions assigned for grading must have full docstrings in numpy's format, as well as internal comments. Utility functions not requested in the assignment and that the user will never see can have reduced docstrings if the functions are simple and obvious, but at least give the one-line summary. (k) If you modify an existing function, you must either make a Git entry or, if it is not under revision control, include a Revision History section in your docstring and record your name, the date, the version number, your email, and the nature of the change you made. (l) Choose variable names that are meaningful and consistent in style. Document your style either at the head of a module or in a separate text file for the project. For example, if you use CamelCaps with initial capital, say that. If you reserve initial capitals for classes, say that. If you use underscores for variable subscripts and camelCaps for the base variables, say that. If you accept some other style and build on that, say that. There are too many good reasons to have such styles for only one to be the community standard. If certain kinds of values should get the same variable or base variable, such as fundamental constants or things like amplitudes, say that. (j) It's best if variables that will appear in formulae are short, so more terms can fit in one 80 character line. Overall, having and following a style makes code easier to read. And, as an added bonus, if you take care to be consistent, you will write slower, view your code more times, and catch more bugs as you write them. Thus, for codes of any significant size, writing pedantically commented and aligned code is almost always faster than blast coding, if you include debugging time. Did you catch both bugs in item h? --jh-- On 5/9/19 11:25 AM, Chris Barker - NOAA Federal <chris.barker(a)noaa.gov> wrote: > Do any of you know of a style guide for computational / numpy code? > > I don't mean code that will go into numpy itself, but rather, users > code that uses numpy (and scipy, and...) > > I know about (am a proponent of) PEP8, but it doesn’t address the > unique needs of scientific programming. > > This is mostly about variable names. In scientific code, we often want: > > - variable names that match the math notation- so single character > names, maybe upper or lower case to mean different things ( in ocean > wave mechanics, often “h” is the water depth, and “H” is the wave height) > > -to distinguish between scalar, vector, and matrix values — often > UpperCase means an array or matrix, for instance. > > But despite (or because of) these unique needs, a style guide would be > really helpful. > > Anyone have one? Or even any notes on what you do yourself? > > Thanks, > -CHB > > > > >> -- >> >> Christopher Barker, Ph.D. >> Oceanographer >> >> Emergency Response Division >> NOAA/NOS/OR&R (206) 526-6959 voice >> 7600 Sand Point Way NE (206) 526-6329 fax >> Seattle, WA 98115 (206) 526-6317 main reception >> >> Chris.Barker(a)noaa.gov <mailto:Chris.Barker@noaa.gov>

4 3

NumPy Community Meeting -- May 15/ 2019
by Tyler Reddy 14 May '19

14 May '19

Hi, There will be a community meeting at 12 pm Pacific Time on May 15/ 2019. Anyone is free to join and edit the work in progress meeting notes: https://hackmd.io/M-ef_Fu5QOOitACnyoO0kQ?view Best wishes, Tyler

1 0

Release vs development testing.
by Charles R Harris 13 May '19

13 May '19

Hi All, NumPy currently distinguishes between release and development versions when running tests. Is there a good reason to continue this practice? I ask, because with the last pytest release it would be convenient to always include `pytest.ini ` so that we can register markers. The presence of `pytest.ini` is how we distinguish betweendevelopment from release for testing purposes. Chuck

4 9

casting from datetime64
by Alex Samuel 13 May '19

13 May '19

Hi, When registering a custom cast function from datetime64 to another dtype, how can I get the units? I am calling PyArray_RegisterCastFunc from NPY_DATETIME. Ideally, I'd like to register a separate cast function for each datetime64 units (or none at all... I don't want all units to be castable). Next best thing would be to obtain the units in the cast function and dispatch accordingly. Is this possible? I glanced through the code, and it looks like there's a lot of hard-coded logic around datetime64, but I didn't go through it carefully. Thought I'd ask before drilling further down. Thanks in advance, Alex

2 2

Adding to the non-dispatched implementation of NumPy methods
by Stephan Hoyer 13 May '19

13 May '19

Hi everyone, We have a proposed a revision to NEP-18 (__array_function__). The proposal is for an adding an alias to the non-dispatched version of NumPy array functions in the __numpy_implementaiton__ function attribute: https://github.com/numpy/numpy/pull/13305 I believe this attribute improves the protocol in three ways: 1. It provides a hook that __array_function__ methods can use to call implementation intended for NumPy arrays. This allows for "partial implementations" of NumPy's API, which turns out to useful even for some array libraries that reimplement nearly everything (namely, for CuPy and JAX). 2. It allows for fast access to the non-dispatching version of NumPy functions, e.g., np.concatenate.__numpy_implementation__(list_of_all_numpy_arrays). 3. Internally, the implementation of numpy.ndarray.__array_function__ now looks identical to how we encourage outside developers to write their own __array_function__ methods. The dispatching logic no longer includes a special case for NumPy arrays. Feedback would be greatly welcomed! Best, Stephan

8 61

folds and scans
by Alan Isaac 12 May '19

12 May '19

What is the recommended (i.e., fast) way to do folds and scans across one axis of a NumPy array? (The equivalent of Mma's Fold and FoldList.) Assume an arbitrary Python function that produces one array from two arrays, where these three arrays share the same dimensions. (So the use of Python's `reduce` is certainly conceivable for the fold.) Thank you, Alan Isaac

1 0

numpy finding local tests on import?!?!
by Chris Barker 10 May '19

10 May '19

TL;DR: This issue appears to have been fixed in numpy 1.15 (at least, I didn't test 1.14) However, I also had some issues in my environment that I also fixed, so it may be that numpy's behavior hasn't changed -- I don't have the energy to test now. And it doesn't hurt to have this in the archives in case someone else runs into the problem. Read on if you care about weird behaviour with the testing package in numpy 1.13.... Numpy appears to be both running tests on import (or at lest the the runner), and finding local tests that are not numpy's I found this issue (closed without a resolution): https://github.com/numpy/numpy/issues/11457 which is related -- but it's about the import time of numpy.testing, and not about errors/issues from that import. But maybe the import process ahs been changed in newer numpys What I did, and what I got: I am trying t debug what looks like a numpy-related issue in a project. So one thing I did was try to import numpy and check __version__: python -c "import numpy; print(numpy.__version__)" very weird barf: <snip> File "/Users/chris.barker/miniconda2/envs/gridded/lib/python2.7/unittest/runner.py", line 4, in <module> import time File "time.py", line 7, in <module> import netCDF4 as nc4 File "/Users/chris.barker/miniconda2/envs/gridded/lib/python2.7/site-packages/netCDF4/__init__.py", line 3, in <module> from ._netCDF4 import * File "include/netCDF4.pxi", line 728, in init netCDF4._netCDF4 (netCDF4/_netCDF4.c:83784) AttributeError: 'module' object has no attribute 'ndarray I get the same thing if I fire up the interpreter and then import numpy as the error seemed to come from: unittest/runner.py I had a hunch. I was, in fact, running with my current working directory in the package dir of my project, and there is a test package in that dir I cd out of that, and presto! numy imports fine: $ python -c "import numpy; print(numpy.__version__)" 1.13.1 OK, that's a kinda old numpy -- but it's the minimum required by my project. (though I can probably update that -- I"ll do that soon) So it appears that the test runner is looking in the current working dir (or, I suppose sys.PATH) for packages called tests -- this seems like a broken system, unless you are runing the tests explicitly from teh command line, it shouldn't look in the cwd, and it probably shouldn't ever look in all of sys.path. BUt my bigger confusion here is -- why the heck is the test runner being run at ALL on a simple import ?!?!? If this has been fixed / changed in newer numpy's the OK -- I'll update my dependencies. -CHB - Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

1 0

My Introduction and Getting Started with Numpy.
by Joyce Tirnyuy 10 May '19

10 May '19

Hi All, I am Ngoran Clare-Joyce, an Electrical Engineer from Cameroon. I use Python and Javascript for Software Development. Over the past year, I have gained insight into Machine Learning and Data Science Algorithms. I have used Numpy, Scipy, Pandas, Pytorch, Scikit-Learn libraries. I have realized that to take my career to the next level, I need to contribute to open source as a way to gain skills, experience and proper understanding of how these libraries work. Please, I will appreciate help on how to get started, set up my development environment, some important documentation, and beginners issues so I can start contributing to Numpy. Thanks, Ngoran Clare-Joyce.

3 3