On Mar 28, 2005, at 4:30 AM, Travis Oliphant wrote:
Scott,
Thank you for your detailed explanations. This is starting to make more sense to me. It is obvious that you understand what we are trying to do, and I pretty much agree with you in how you think it should be done. I think you do a great job of explaining things. I agree we should come up with a set of names for the interface to arrayobjects. I'm even convinced that offset should be an optional part of the interface (implied 0 if it's not there).
Just to add my two cents, I don't think I ever thought it was necessary to bundle the metadata with the memory object for the reasons Scott outlined. It isn't needed functionally, and there are cases where the same memory may be used in different contexts (as is done with our record arrays). Numarray, when it uses the buffer object, always gets a fresh pointer for the buffer object for every data access. But Scott is right that that pointer is good so long as there isn't a chance for something else to change it. In practice, I don't think that ever happens with the buffers that numarray happens to use, but it's still a flaw of the current buffer object that there is no way to ensure it won't change. I'm not sure how the support for large data sets should be handled. I generally think that it will be very awkward to handle these until Python does as well. Speaking of which... I had been in occasional contact with Martin von Loewis about his work to update Python to handle 64-bit addressing. We weren't planning to handle this in nummarray (nor Numeric3, right Travis or do I have that wrong?) until Python did. A few months ago Martin said he was mostly done. I had a chance to talk to him at Pycon about where that work stood. Unfortunately, it is not turning out to be as easy as he hoped. This is too bad. I have a feeling that this work is going to stall without help on our (numpy community) part to help make the changes or drum beating to make it a higher priority. At the moment the Numeric3 effort should be the most important focus, but I think that after that, this should become a high priority. Perry