[Numpy-discussion] Add guaranteed no-copy to array creation and reshape?

[Ralf Gommers]

On Wed, Dec 26, 2018 at 3:29 PM Sebastian Berg <sebastian at sipsolutions.net>
wrote:

> Hi all,
>
> In https://github.com/numpy/numpy/pull/11897 I am looking into the
> addition of a `copy=np.never_copy` argument to:
>   * np.array
>   * arr.reshape/np.reshape
>   * arr.astype
>
> Which would cause an error to be raised when numpy cannot guarantee
> that the returned array is a view of the input array.
> The motivation is to easier avoid accidental copies of large data, or
> ensure that in-place manipulation will be meaningful.
>
> The copy flag API would be:
>   * `copy=True` forces a copy
>   * `copy=False` allows numpy to copy if necessary
>   * `copy=np.never_copy` will error if a copy would be necessary
>   * (almost) all other input will be deprecated.
>
> Unfortunately using `copy="never"` is tricky, because currently
> `np.array(..., copy="never")` behaves exactly the same as
> `np.array(..., copy=bool("never"))`. So that the wrong result would be
> given on old numpy versions and it would be a behaviour change.
>

I think np.never_copy is really ugly. I'd much rather simply use 'never',
and clearly document that if users start using this and they critically
rely on it really being never, then they should ensure that their code is
only used with numpy >= 1.17.0.

Note also that this would not be a backwards compatibility break, because
`copy` is now clearly documented as bool, and not bool_like or some such
thing. So we do not need to worry about the very improbable case that users
now are using `copy='never'`.

If others think `copy='never'` isn't acceptable now, there are two other
options:
1. add code first to catch `copy='never'` in 1.17.x and raise on it, then
in a later numpy version introduce it.
2. just do nothing. I'd prefer that over `np.never_copy`.

Cheers,
Ralf


> Some things that are a not so nice maybe:
>  * adding/using `np.never_copy` is not very nice
>  * Scalars need a copy and so will not be allowed
>  * For rare array-likes numpy may not be able to guarantee no-copy,
>    although it could happen (but should not).
>
>
> The history is that a long while ago I considered adding a copy flag to
> `reshape` so that it is possible to do `copy=np.never_copy` (or
> similar) to ensure that no copy is made. In these, you may want
> something like an assertion:
>
> ```
> new_arr = arr.reshape(new_shape)
> assert np.may_share_memory(arr, new_arr)
>
> # Which is sometimes -- but should not be -- written as:
> arr.shape = new_shape  # unnecessary container modification
>
> # Or:
> view = np.array(arr, order="F")
> assert np.may_share_memory(arr, new_arr)
> ```
>
> but is more readable and will not cause an intermediate copy on error.
>
>
> So what do you think? Other variants would be to not expose this for
> `np.array` and probably limit `copy="never"` to the reshape method. Or
> just to not do it at all. Or to also accept "never" for `reshape`,
> although I think I would prefer to keep it in sync and wait for a few
> years to consider that.
>
> Best,
>
> Sebastian
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181226/2d628ea9/attachment.html>