[Numpy-discussion] deprecate updateifcopy in nditer operand flags?

[Matti Picus]

I filed issue 9714 https://github.com/numpy/numpy/issues/9714 and wrote 
a mail in September trying to get some feedback on what to do with 
updateifcopy semantics and user-exposed nditer.
It garnered no response, so I am trying again.
For those who are unfamiliar with the issue see below for a short 
summary and issue 7054 for a lengthy discussion.
Note that pull request 9639 which should be merged very soon changes the 
magical UPDATEIFCOPY into WRITEBACKIFCOPY, and hopefully will appear in 
NumPy 1.14.

As I mention in the issue, there is a magical update done in this 
snippet in the next-to-the-last line:

|a = np.arange(24, dtype='f8').reshape(2, 3, 4).T i = np.nditer(a, [], 
[['readwrite', 'updateifcopy']], casting='same_kind', 
op_dtypes=[np.dtype('f4')]) # Check that UPDATEIFCOPY is activated 
i.operands[0][2, 1, 1] = -12.5 assert a[2, 1, 1] != -12.5 i = None # 
magic!!! assert a[2, 1, 1] == -12.5|


Not only is this magic very implicit, it relies on refcount semantics 
and thus does not work on PyPy.
Possible solutions:

1. nditer is rarely used, just deprecate updateifcopy use on operands

2. make nditer into a context manager, so the code would become explicit

|a = np.arange(24, dtype='f8').reshape(2, 3, 4).T with np.nditer(a, [], 
[['readwrite', 'updateifcopy']], casting='same_kind', 
op_dtypes=[np.dtype('f4')]) as i: # Check that WRITEBACKIFCOPY is 
activated i.operands[0][2, 1, 1] = -12.5 assert a[2, 1, 1] != -12.5 
assert a[2, 1, 1] == -12.5 # a is modified in i.__exit__|


3. something else?

Any opinions? Does anyone use nditer in production code?
Matti

-------------------------
what are updateifcopy semantics? When a temporary copy or work buffer is 
required, NumPy can (ab)use the base attribute of an ndarray by

    - creating a copy of the data from the base array

    - mark the base array read-only

Then when the temporary buffer is "no longer needed"

    - the data is copied back

    - the original base array is marked read-write

The trigger for the "no longer needed" decision before pull request 9639 
is in the dealloc function.
That is not generally a place to do useful work, especially on PyPy 
which can call dealloc much later.
Pull request 9639 adds an explicit PyArray_ResolveWritebackIfCopy api 
function, and recommends calling it explicitly before dealloc.

The only place this change is visible to the python-level user is in 
nditer.
C-API users will need to adapt their code to use the new API function, 
with a deprecation cycle that is backwardly compatible on CPython.