# [Numpy-discussion] in the NA discussion, what can we agree on?

Gary Strangman strang at nmr.mgh.harvard.edu
Fri Nov 4 12:31:55 EDT 2011

```>> destructive + propagating = the data point is truly missing (satellite fell into
>> the ocean; dog ate my source datasheet, or whatever), this is the nature of that
>> data point, such missingness should be replicated in elementwise operations, and
>> the missingness SHOULD interfere with reduction operations that involve that
>> datapoint (np.sum([1,MISSING])=MISSING)
>
> Right.
>
>
>> destructive + non-propagating = the data point is truly missing, this is the
>> nature of that data point, such missingness should be replicated in elementwise
>> operations, but such missingness should NOT interfere with reduction operations
>> that involve that datapoint (np.sum([1,MISSING])=1)
>
> What do you define as element-wise operations?
>
> Is a sum on an array an element-wise operation?
>
>  >>> [1, MISSING]+2
>  [1, MISSING]
>
> Or is it just a form of reduction (after shape broadcasting)?
>
>  >>> [1, MISSING]+2
>  [3, 2]
>
> For me it's the second, so the only time where special values "propagate" in a
> non-propagating scenario is when you slice an array.

Let's say I want to re-scale a column (or remove the mean from a column).
I wouldn't want that to change my "missingness". Thus, I'm thinking:

>>> x = [1,2,MISSING]
>>> x*3
[3, 6, MISSING]
>>> x = [1,2,MISSING]
>>> x - x.mean()
[-0.5, 0.5, MISSING]

To me it makes sense to have identical operations for the temporary IGNORE
case below (versus the permanent MISSING case here). Note, the reason to
independently have separate IGNORE and MISSING is so that I can (for
example) temporarily IGNORE entire rows in my 2D array (which may have
scattered MISSING elements), and when I undo the IGNORE operation the
MISSING elements are still MISSING.

The question does still remain what to do when performing operations like
those above in IGNORE cases. Perform the operation "underneath"? Or not?

>> non-destructive + propagating = I want to ignore this datapoint for now;
>> element-wise operations should replicate this "ignore" designation, and
>> missingness of this type SHOULD interfere with reduction operations that involve
>> this datapoint (np.sum([1,IGNORE])=IGNORE)
>
> Right.
>
>
>> non-destructive + non-propagating = I want to ignore this datapoint for now;
>> element-wise operations should replicate this "ignore" designation, but
>> missingness of this type SHOULD NOT interfere with reduction operations that
>> involve this datapoint (np.sum([1,IGNORE])=1)
>
> Same concerns as above.
>
>
> Lluis
>
>

The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail