[Numpy-discussion] nditer and updateifcopy semantics - advice needed

Matti Picus matti.picus at gmail.com
Mon Sep 25 13:36:58 EDT 2017


I filed issue 9714 trying to get some feedback on what to do with 
updateifcopy semantics and user-exposed nditer.
For those who are unfamiliar with the issue see below for a short 
summary, issue 7054 for a lengthy discussion, or pull request 9639 
(which is still not merged).

As I mention in the issue, I am willing to put in the work to make the 
magical update done in the last line of this snippet more explicit:

a = arange(24, dtype='<i4').reshape(2, 3, 4)
i = nditer(a, ['buffered'], order='F', casting='unsafe', 
op_dtypes='>f8', buffersize=5)
j = i.copy()
i = None # <<<< HERE

but need some direction from the community. Possible solutions:

1. nditer is rarely used, just deprecate updateifcopy use on iterands 
and raise an exception

2. make nditer into a context manager, so the code would become explicit

a = arange(24, dtype='<i4').reshape(2, 3, 4)
with nditer(a, ['buffered'], order='F', casting='unsafe', 
op_dtypes='>f8', buffersize=5) as i:
     j = i.copy()

3. something else?

Any opinions?
Matti

-------------------------
what are updateifcopy semantics? When a temporary copy or work buffer is 
required, NumPy can (ab)use the base attribute of an ndarray by

    - creating a copy of the data from the base array

    - mark the base array read-only

Then when the temporary buffer is "no longer needed"

    - the data is copied back

    - the original base array is marked read-write

The trigger for the "no longer needed" decision before pull request 9639 
is in the dealloc function.
That is not generally a place to do useful work, especially on PyPy 
which can call dealloc much later.
Pull request 9639 adds an explicit PyArray_ResolveWritebackIfCopy api 
function, and recommends calling it explicitly before dealloc.

The only place this change is visible to the python-level user is in 
nditer.
C-API users will need to adapt their code to use the new API function, 
with a deprecation cycle that is backwardly compatible on CPython.


More information about the NumPy-Discussion mailing list