deprecate updateifcopy in nditer operand flags?
I filed issue 9714 https://github.com/numpy/numpy/issues/9714 and wrote a mail in September trying to get some feedback on what to do with updateifcopy semantics and user-exposed nditer. It garnered no response, so I am trying again. For those who are unfamiliar with the issue see below for a short summary and issue 7054 for a lengthy discussion. Note that pull request 9639 which should be merged very soon changes the magical UPDATEIFCOPY into WRITEBACKIFCOPY, and hopefully will appear in NumPy 1.14. As I mention in the issue, there is a magical update done in this snippet in the next-to-the-last line: |a = np.arange(24, dtype='f8').reshape(2, 3, 4).T i = np.nditer(a, [], [['readwrite', 'updateifcopy']], casting='same_kind', op_dtypes=[np.dtype('f4')]) # Check that UPDATEIFCOPY is activated i.operands[0][2, 1, 1] = -12.5 assert a[2, 1, 1] != -12.5 i = None # magic!!! assert a[2, 1, 1] == -12.5| Not only is this magic very implicit, it relies on refcount semantics and thus does not work on PyPy. Possible solutions: 1. nditer is rarely used, just deprecate updateifcopy use on operands 2. make nditer into a context manager, so the code would become explicit |a = np.arange(24, dtype='f8').reshape(2, 3, 4).T with np.nditer(a, [], [['readwrite', 'updateifcopy']], casting='same_kind', op_dtypes=[np.dtype('f4')]) as i: # Check that WRITEBACKIFCOPY is activated i.operands[0][2, 1, 1] = -12.5 assert a[2, 1, 1] != -12.5 assert a[2, 1, 1] == -12.5 # a is modified in i.__exit__| 3. something else? Any opinions? Does anyone use nditer in production code? Matti ------------------------- what are updateifcopy semantics? When a temporary copy or work buffer is required, NumPy can (ab)use the base attribute of an ndarray by - creating a copy of the data from the base array - mark the base array read-only Then when the temporary buffer is "no longer needed" - the data is copied back - the original base array is marked read-write The trigger for the "no longer needed" decision before pull request 9639 is in the dealloc function. That is not generally a place to do useful work, especially on PyPy which can call dealloc much later. Pull request 9639 adds an explicit PyArray_ResolveWritebackIfCopy api function, and recommends calling it explicitly before dealloc. The only place this change is visible to the python-level user is in nditer. C-API users will need to adapt their code to use the new API function, with a deprecation cycle that is backwardly compatible on CPython.
participants (1)
-
Matti Picus