On Wed, Jul 6, 2011 at 10:44 AM, Matthew Brett <matthew.brett@gmail.com>wrote:
Hi,
On Wed, Jul 6, 2011 at 6:11 PM, Benjamin Root <ben.root@ou.edu> wrote:
On Wed, Jul 6, 2011 at 12:01 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jul 6, 2011 at 5:48 PM, Peter <numpy-discussion@maubp.freeserve.co.uk> wrote:
On Wed, Jul 6, 2011 at 5:38 PM, Matthew Brett <
wrote:
Hi,
On Wed, Jul 6, 2011 at 4:40 PM, Mark Wiebe <mwwiebe@gmail.com>
wrote:
It appears to me that one of the biggest reason some of us have been talking past each other in the discussions is that different people have different definitions for the terms being used. Until this is thoroughly cleared up, I feel the design process is tilting at windmills. In the interests of clarity in our discussions, here is a starting point which is consistent with the NEP. These definitions have been added in a glossary within the NEP. If there are any ideas for amendments to these definitions that we can agree on, I will update the NEP with those amendments. Also, if I missed any important terms which need to be added, please propose definitions for them. NA (Not Available) A placeholder for a value which is unknown to computations. That value may be temporarily hidden with a mask, may have been lost due to hard drive corruption, or gone for any number of reasons. This is the same as NA in the R project.
Really? Can one implement NA with a mask in R? I thought an NA was always bitpattern in R?
I don't think that was what Mark was saying, see this bit later in
matthew.brett@gmail.com> this
email:
I think it would make a difference if there was an implementation that had conflated masking with bitpatterns in terms of API. I don't think R is an example.
Of course R is not an example of that. Nothing is. This is merely conceptual. Separate NA from np.NA in Mark's NEP, and you will see his point. Consider it the logical intersection of NA in Mark's NEP and the aNEP.
I am trying to work out what you feel you feel the points of discussion are. There's surely no point in continuing to debate things we agree on.
I don't think anyone disputes (or has ever disputed) that:
There can be missing data implemented with bitpatterns There can be missing data implemented with masks Missing data can have propagate semantics Missing data can have ignore semantics. The implementation does not in itself constrain the semantics.
So, to be clear, is your concern is that you want to be able to tell difference between whether an np.NA comes from the bit pattern or the mask in its implementation? But why would you have both the parameterized dtype and the mask implementation at the same time? They implement the same abstraction. Is your desire that the np.NA's are implemented solely through bit patterns and np.IGNORE is implemented solely through masks? So that you can think of the masks as being IGNORE flags? What if you want multiple types of IGNORE? (To ignore certain values because they're outliers, others because the data wouldn't make sense, and others because you're just focusing on a particular subgroup, for instance.) A related question is if the IGNORE values could just be another NA value? I don't understand what the specific problem would be with having several NA values, say NA(1), NA(2), ..., and then letting the user decide that NA(1) means NA in the sense discussed above and NA(2) means IGNORE. Then the ufuncs could be told whether to ignore or propagate each type of NA value. Could you explain to me if this would resolve your concerns about NA/IGNORE, or possibly give a few examples if it doesn't? Because I am still rather confused. Let's not discuss that any more; we all agree. So what do you think
is the source of the disagreement?
Or are you saying that there should be no disagreement at this stage?
Cheers,
Matthew _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion