[Numpy-discussion] in the NA discussion, what can we agree on?

Lluís xscript at gmx.net
Thu Nov 3 10:28:14 EDT 2011

Nathaniel Smith writes:

> 4) There is consensus that whatever approach is taken, there should be
> a quick and convenient way to identify values that are MISSING,
> IGNORED, or both. (E.g., functions is_MISSING, is_IGNORED,
> is_MISSING_or_IGNORED, or some equivalent.)

Well, maybe it's too low level, but I'd rather decouple the two concepts into
two orthogonal properties that can be composed:

* Destructiveness: whether the previous data value is lost whenever you assign a
  "special" value.

* Propagation: whether any of these "special" values is propagated or just
  skipped when performing computations.

I think we can all agree on the definition of these two properties (where
bit-patters are destructive and masks are non-destructive), so I'd say that the
first discussion is establishing whether to expose them as separate properties
or just expose specific combinations of them:

* MISSING: destructive + propagating
* IGNORED: non-destructive + non-propagating

For example, it makes sense to me to have non-destructive + propagating.

If we take this road, then the next points to discuss should probably be how
these combinations are expressed:

* At the array level: all special values behave the same in a specific array,
  given its properties (e.g., all of them are destructive+propagating).

* At the value level: each special value conveys a specific combination of the
  aforementioned properties (e.g., assigning A is destructive+propagating and
  assigning B is non-destructive+non-propagating).

* Hybrid: e.g., all special values are destructive, but propagation depends on
  the specific special value.

I think this last decision is crucial, as it will have a direct impact on
performance, numpy code maintainability and 3rd party interface simplicity.


 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom

More information about the NumPy-Discussion mailing list