Mailman 3 September 2016 - NumPy-Discussion

Parsing a file with dates to datetim64
by Florian Lindner 26 Sep '16

26 Sep '16

Hey, I have a file: ;;;;;;;;;Eintrittsdatum;;; ;;;;;;;;;04.03.16;;10,00 €;genehmigt ;;;;;;;;;04.03.16;;10,00 €;genehmigt which I try to parse using def dateToNumpyDate(s): s = s.decode("utf-8") ret = datetime.datetime.strptime(s, "%d.%m.%y").isoformat() return ret def generateMembers(): members = np.genfromtxt("test_CSC_Mitglieder.csv", dtype = { "names" : ["EntryDate"], "formats" : ['datetime64[D]'] }, converters = { 9 : dateToNumpyDate }, skip_header = 1, delimiter = ";", usecols = (9)) count = members.shape[0] y = np.linspace(1, count, count) print(members) print(members.dtype) plt.plot(members["EntryDate"], y) plt.show() but the datatype was ignored homehow, <U19 instead of the datetime64 ['2016-03-04T00:00:00' '2016-03-04T00:00:00'] <U19 CSC.py:76: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future plt.plot(members["EntryDate"], y) Traceback (most recent call last): File "CSC.py", line 82, in <module> generateMembers() File "CSC.py", line 76, in generateMembers plt.plot(members["EntryDate"], y) IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices I also tried to print the return of dateToNumpbyData: 2016-03-04T00:00:00 2016-03-04T00:00:00 2016-03-04T00:00:00 Is there a problem with the dtype argument? Thanks, Florian

1 0

[7949] How to handle these deprecation Warning errors
by Saurabh Mehta 26 Sep '16

26 Sep '16

Hi I am working on issue #7949, and need to use "-3" switch while running python 2.7 for my tests. python -3 -c "import numpy as np; np.test()" Several errors are reported and all all of them are DeprecationWarnings, which is ok. (https://travis-ci.org/njase/numpy/jobs/162733502) *But now these errors must be either fixed or skipped. This is where I am facing problem. Pls suggest:* 1. *Identify that python was invoked with -3 and skip these cases*: There seems no way to know if python was invoked with -3 sys.argv only reports about "-c" and ignores other switches 2. *Fix these issues:* All of them are about deprecated APIs and new APIs have been introduced in python 3. Since I am using python 2.x, I don't see a way to fix them What to do? Regards Saurabh

2 1

mail.scipy.org update
by Didrik Pinte 26 Sep '16

26 Sep '16

Hi everyone, While updating the scipy SSL certificates yesterday, it appeared that filesystem of the servers is corrupted (more than likely a hardware failure). The problem is restricted to one volume and impacts only the web services. The mailing list/mailman service works as expected. We're working on restoring all the different non-functional services. Thanks for you patience! -- Didrik

2 2

guvectorize, a helper for writing generalized ufuncs
by Stephan Hoyer 26 Sep '16

26 Sep '16

NumPy has the handy np.vectorize for turning Python code that operates on scalars into a function that vectorizes works like a ufunc, but no helper function for creating generalized ufuncs ( http://docs.scipy.org/doc/numpy/reference/c-api.generalized-ufuncs.html). np.apply_along_axis accomplishes some of this, but it only allows a single core dimension on a single argument. So I propose adding a new object, np.guvectorize(pyfunc, signature, otypes, ...), where pyfunc is defined over the core dimensions only of any inputs and signature is any valid gufunc signature (a string). Calling this object would apply the gufunc. This is inspired by the similar numba.guvectorize, which is currently the easiest way to write a gufunc in Python. In addition to be handy like vectorize, such functionality would be especially useful for with working libraries that build upon NumPy to extend the capabilities of generalized ufuncs (e.g., xarray after https://github.com/pydata/xarray/pull/964). Cheers, Stephan

3 4

testing
by Charles R Harris 23 Sep '16

23 Sep '16

Testing if this gets posted... Chuck

1 0

test
by Didrik Pinte 19 Sep '16

19 Sep '16

checking if server works as expected after SSL cert update

1 0

Swig tests failing on Array2.resize()
by Don Porges 14 Sep '16

14 Sep '16

On both osx and Linux, I am seeing 3 swig tests failing out of the entire suite. These both call Array2.resize() with 2 (non-self) arguments. Note that the last two are in fact supposed to fail, but not because of a wrong number of arguments, as happens here: ====================================================================== ERROR: testResize0 (__main__.Array2TestCase) Test Array2 resize method, size ---------------------------------------------------------------------- Traceback (most recent call last): File "testArray.py", line 177, in testResize0 self.array2.resize(newRows, newCols) TypeError: resize() takes exactly 2 arguments (3 given) ====================================================================== ERROR: testResizeBad1 (__main__.Array2TestCase) Test Array2 resize method, negative nrows ---------------------------------------------------------------------- Traceback (most recent call last): File "testArray.py", line 188, in testResizeBad1 self.assertRaises(ValueError, self.array2.resize, -5, 5) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 475, in assertRaises callableObj(*args, **kwargs) TypeError: resize() takes exactly 2 arguments (3 given) ====================================================================== ERROR: testResizeBad2 (__main__.Array2TestCase) Test Array2 resize method, negative ncols ---------------------------------------------------------------------- Traceback (most recent call last): File "testArray.py", line 192, in testResizeBad2 self.assertRaises(ValueError, self.array2.resize, 5, -5) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 475, in assertRaises callableObj(*args, **kwargs) TypeError: resize() takes exactly 2 arguments (3 given) ------------------------------------------------------------------------------------------- This is with python 2.7.x, swig 3.0.10, and numpy 1.11.1 . I note that >> import Array >> help(Array) gives, for Array2.resize, only this signature: *resize*(self, nrows) Is this currently supposed to work, or have I done something clearly wrong?

1 0

String & unicode arrays vs text loading in python 3
by Lluís Vilanova 14 Sep '16

14 Sep '16

Hi! I'm giving a shot to issue #3184 [1], based on the observation that the string dtype ('S') under python 3 uses byte arrays instead of unicode (the only readable string type in python 3). This brings two major problems: * numpy code has to go through loops to open and read files as binary data to load text into a bytes array, and does not play well with users providing string (unicode) arguments * the repr of these arrays shows strings as b'text' instead of 'text', which breaks doctests of software built on numpy What I'm trying to do is make dtypes 'S' and 'U' equivalnt (NPY_STRING and NPY_UNICODE). Now the question. Keeping 'S' and 'U' as separate dtypes (but same internal implementation) will provide the best backwards compatibility, but is more cumbersome to implement. Is it acceptable to internally just translate all appearances of 'S' (NPY_STRING) to 'U' (NPY_UNICODE) and get rid of one of the two when running in python 3? The main drawback I see is that dtype reprs would not always be as expected: # python 2 >>> np.array('foo', dtype='S') array('foo', dtype='|S3') # python 3 >>> np.array('foo', dtype='S') array('foo', dtype='<U3') [1] https://github.com/numpy/numpy/issues/3184 Cheers, Lluis

4 8

New iterator API (nditer): Overlap detection in NumPy
by Sebastian Berg 12 Sep '16

12 Sep '16

Hi all, Pauli just opened a nice pull request [1] to add overlap detection to the new iterator, this means adding a new iterator flag: `NPY_ITER_COPY_IF_OVERLAP` If passed to the iterator (also exposed in python), the iterator will copy the operands such that reading and writing should only occur for identical operands. For now this is implemented by always copying the output/writable operand (this could be improved though, so I would not say its fixed API). Since adding this flag is new API, please feel free to suggest other names/approaches or even decline the change ;). This is basically a first step, which should be easily followed by adding overlap detection to ufuncs, removing traps such as the well (or not so well known) `a += a.T`. Other parts of numpy may follow one by one. The work is based on his older awesome new memory overlap detection implementation. If there are no comments, I will probably merge it very soon, so we can look at the follow up things. - Sebastian [1] https://github.com/numpy/numpy/pull/8026

3 7

NumPy 1.11.2rc1
by Charles R Harris 12 Sep '16

12 Sep '16

Hi All, I'm pleased to announce the release of Numpy 1.11.2rc1. This release supports Python 2.6 - 2.7, and 3.2 - 3.5 and fixes bugs and regressions found in Numpy 1.11.1. Wheels for Linux, Windows, and OSX can be found on PyPI. Sources are available on both PyPI and Sourceforge <https://sourceforge.net/projects/numpy/files/NumPy/1.11.2rc1/>. Thanks to all who were involved in this release. The following pull requests have been merged. PRs overridden by later merges and trivial release notes updates have been omitted. - #7736 BUG: Many functions silently drop 'keepdims' kwarg. - #7738 ENH: Add extra kwargs and update doc of many MA methods. - #7778 DOC: Update Numpy 1.11.1 release notes. - #7793 BUG: MaskedArray.count treats negative axes incorrectly. - #7816 BUG: Fix array too big error for wide dtypes. - #7821 BUG: Make sure npy_mul_with_overflow_<type> detects overflow. - #7824 MAINT: Allocate fewer bytes for empty arrays. - #7847 MAINT,DOC: Fix some imp module uses and update f2py.compile docstring. - #7849 MAINT: Fix remaining uses of deprecated Python imp module. - #7851 BLD: Fix ATLAS version detection. - #7896 BUG: Construct ma.array from np.array which contains padding. - #7904 BUG: Fix float16 type not being called due to wrong ordering. - #7917 BUG: Production install of numpy should not require nose. - #7919 BLD: Fixed MKL detection for recent versions of this library. - #7920 BUG: Fix for issue #7835 (ma.median of 1d). - #7932 BUG: Monkey-patch _msvccompile.gen_lib_option like other compilers. - #7939 BUG: Check for HAVE_LDOUBLE_DOUBLE_DOUBLE_LE in npy_math_complex. - #7953 BUG: Guard against buggy comparisons in generic quicksort. - #7954 BUG: Use keyword arguments to initialize Extension base class. - #7955 BUG: Make sure numpy globals keep identity after reload. - #7972 BUG: MSVCCompiler grows 'lib' & 'include' env strings exponentially. - #8005 BLD: Remove __NUMPY_SETUP__ from builtins at end of setup.py. - #8010 MAINT: Remove leftover imp module imports. - #8020 BUG: Fix return of np.ma.count if keepdims is True and axis is None. - #8024 BUG: Fix numpy.ma.median. - #8031 BUG: Fix np.ma.median with only one non-masked value. - #8044 BUG: Fix bug in NpyIter buffering with discontinuous arrays. The following people contributed to this release. The '+' marks first time contributors. - Allan Haldane - Bertrand Lefebvre - Charles Harris - Julian Taylor - Loïc Estève - Marshall Bockrath-Vandegrift+ - Michael Seifert+ - Pauli Virtanen - Ralf Gommers - Sebastian Berg - Shota Kawabuchi+ - Thomas A Caswell - Valentin Valls+ - Xavier Abellan Ecija+ Chuck

1 0