Mailman 3 October 2015 - NumPy-Discussion

Interesting discussion on copyrighting files.
by Charles R Harris 17 Oct '15

17 Oct '15

Worth a read at A&D <http://esr.ibiblio.org/?p=6867#more-6867>. Chuck

3 2

Bug
by josef.pktd＠gmail.com 16 Oct '15

16 Oct '15

1 1

how is toolchain selected for compiling (OS X with python.org build)?
by Andrew Jaffe 15 Oct '15

15 Oct '15

This isn't, strictly speaking, a numpy question, but I suspect it's something that numpy devs and users have some insight into. I am trying to compile an extension that requires a fairly advanced c++ compiler. Using the built-in apple python, it defaults to the latest clang from apple, and it works just fine. Using the python.org framework build, it still selects clang, which is in principle a new enough compiler, but for some reason it seems to end up pointing to /usr/include/c++/4.2.1/ which of course is too old, and the build fails. So the questions I have are: - *why* is it using such an old toolchain (I am pretty sure that the answer is backward compatibility, and specifically because that is the way the framework build python is itself compiled). - *how* is it selecting those tools, and in particular, that include directory? It doesn't seem to explicitly show up in the logs, until there's an error. If I just use the same clang invocation as seems to be used by the build, it is able to compile full C++-11 code... - Is there any way to still use the apple clang, but in full c++-11 mode to build extensions? The solution/workaround is to install and then explicitly select a more advanced compiler, e.g., from homebrew, using environment variables, but it would be nice if it could work out of the box, and ideally with the same behaviour as with apple's python build. -Andrew p.s. for the aficionados, this is for [healpy][1], and we're looking at it with [this issue][2]. [1]: https://github.com/healpy [2]: https://github.com/healpy/healpy/issues/284#issuecomment-148354405

2 1

A regression in numpy 1.10: VERY slow memory mapped file generation
by Nadav Horesh 15 Oct '15

15 Oct '15

I have binary files of size range between few MB to 1GB, which I read process as memory mapped files (via np.memmap). Until numpy 1.9 the creation of recarray on an existing file (without reading its content) was instantaneous, and now it takes ~6 seconds (system: archlinux on sandy bridge). A profiling (using ipython %prun) top of the list is: ncalls tottime percall cumtime percall filename:lineno(function) 21 3.037 0.145 4.266 0.203 _internal.py:372(_check_field_overlap) 3713431 1.663 0.000 1.663 0.000 _internal.py:366(<genexpr>) 3713750 0.790 0.000 0.790 0.000 {range} 3713709 0.406 0.000 0.406 0.000 {method 'update' of 'set' objects} 322 0.320 0.001 1.984 0.006 {method 'extend' of 'list' objects} Nadav.

2 2

Numpy 1.10.1 released.
by Charles R Harris 14 Oct '15

14 Oct '15

Hi All, I'm pleased to announce the release of Numpy 1.10.1. This release fixes some build problems and serves to reset the release number on pipy to something usable. As a note for future release managers, I had to upload these files from the command line, as using the file upload option at pipy resulted in a failure to parse the version. NumPy 1.10.1 Release Notes ************************** This release deals with a few build problems that showed up in 1.10.0. Most users would not have seen these problems. The differences are: * Compiling with msvc9 or msvc10 for 32 bit Windows now requires SSE2. This was the easiest fix for what looked to be some miscompiled code when SSE2 was not used. If you need to compile for 32 bit Windows systems without SSE2 support, mingw32 should still work. * Make compiling with VS2008 python2.7 SDK easier * Change Intel compiler options so that code will also be generated to support systems without SSE4.2. * Some _config test functions needed an explicit integer return in order to avoid the openSUSE rpmlinter erring out. * We ran into a problem with pipy not allowing reuse of filenames and a resulting proliferation of *.*.*.postN releases. Not only were the names getting out of hand, some packages were unable to work with the postN suffix. Numpy 1.10.1 supports Python 2.6 - 2.7 and 3.2 - 3.5. Commits: 45a3d84 DEP: Remove warning for `full` when dtype is set. 0c1a5df BLD: import setuptools to allow compile with VS2008 python2.7 sdk 04211c6 BUG: mask nan to 1 in ordered compare 826716f DOC: Document the reason msvc requires SSE2 on 32 bit platforms. 49fa187 BLD: enable SSE2 for 32-bit msvc 9 and 10 compilers dcbc4cc MAINT: remove Wreturn-type warnings from config checks d6564cb BLD: do not build exclusively for SSE4.2 processors 15cb66f BLD: do not build exclusively for SSE4.2 processors c38bc08 DOC: fix var. reference in percentile docstring 78497f4 DOC: Sync 1.10.0-notes.rst in 1.10.x branch with master. Cheers, Chuck

6 7

Make all comparisons with NaT false?
by Stephan Hoyer 14 Oct '15

14 Oct '15

Currently, NaT (not a time) does not have any special treatment when used in comparison with datetime64/timedelta64 objects. This means that it's equal to itself, and treated as the smallest possible value in comparisons, e.g., NaT == NaT and NaT < any_other_time. To me, this seems a little crazy for a value meant to denote a missing/invalid time -- NaT should really have the same comparison behavior as NaN. That is, all comparisons with NaT should be false. The good news is that updating this behavior turns out to be only a matter of adding a single conditional to umath/loops.c.src -- most of the work would be fixing tests. Whether you call this an API change or a bug fix is somewhat of a judgment call, but I believe this change is certainly consistent with the goals of datetime64. It's also consistent with how NaT is used in pandas, which uses its own wrappers around datetime64 precisely to fix these sorts of issues. So I'm raising this here to get some opinions on the right path forward: 1. Is this a bug fix that we can backport to 1.10.x? 2. Is this an API change that should wait until 1.11? 3. Is this something where we need to start issuing warnings and deprecate the existing behavior? My vote would be for option 2. I think it's really a bug fix, but it would break enough code that I wouldn't want to spring this on anybody in a bug fix release. I'd rather not wait several releases on this one because that will only exacerbate issues with being able to use datetime64 reliably. Stephan

3 2

Deprecating unitless timedelta64 and "safe" casting of integers to timedelta64
by Stephan Hoyer 13 Oct '15

13 Oct '15

As part of the datetime64 cleanup I've been working on over the past few days, I noticed that NumPy's casting rules for np.datetime64('NaT') were not working properly: https://github.com/numpy/numpy/pull/6465 This led to my discovery that NumPy currently supports unit-less timedeltas (e.g., "np.timedelta64(5)"), which indicate some sort of generic time difference. The current behavior is to take the time units from the other argument when these are used in a binary operation. Even worse, we currently support "safe" casting of integers to timedelta64, which means that integer + datetime64 and integer + timedelta64 arithmetic works: In [4]: np.datetime64('2000-01-01T00') + 10 Out[4]: numpy.datetime64('2000-01-01T10:00-0800','h') Based on the principle that NumPy's datetime support should mirror the standard library as much as possible, both of these behaviors seem like a bad idea. We have datetime types precisely to disambiguate these sort of situations. I'd like to propose deprecating such casting in NumPy 1.11, with the intent of removing it entirely as soon as practical. Stephan

3 2

Problem with os.environ.clear in numpy initialization
by Michael Behrisch 12 Oct '15

12 Oct '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi list, I encountered a problem in my code which depends on numpy. Due to an unusual variable in my environment and this python bug https://bugs.python.org/issue20658 the os.environ.clear call in numpy/core/__init__.py fails. I wrote a patch and submitted a pull request: https://github.com/behrisch/numpy/pull/1 Although it is mainly a workaround for the bug mentioned, I would be happy if it could get accepted because I have only limited control of the environment. Best regards, Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlYb+c0ACgkQPBD+ltFwpilo1wCZAbvS192vVwnWCYT0y7L5At6m TdsAoOmLjTnq7iVlh2QmnsM1qJ72PTyE =wDnI -----END PGP SIGNATURE-----

1 1

1.10 and development version docs
by Ralf Gommers 11 Oct '15

11 Oct '15

Hi, I see that there are no docs for 1.10 on docs.scipy.org yet, and the development version docs are from Nov'14. Anyone with permissions want to look at rectifying that situation? Also, building development version docs on TravisCI after each merge would be useful (for Numpy and Scipy). A little more work, but unlike updating the current docs you don't need doc server permissions so can be done by anyone. Here's a more concrete idea of how to achieve this https://github.com/scipy/scipy/issues/5343 (with links to things to script to borrow from IPython/MPL). Ralf

2 2

Re: [Numpy-discussion] method to calculate the magnitude squared
by Phillip Feldman 11 Oct '15

11 Oct '15

My apologies for the slow response; I was experiencing some technical problems with e-mail. In answer to Antoine's question, my main desire is for a numpy ndarray method, for the convenience, with a secondary goal being improved performance. I have added the function `magsq` to my library, but would like to access it as a method rather than as a function. I understand that I could create a class that inherits from NumPy and add a `magsq` method to that class, but this has a number of disadvantages. Phillip

5 9