[Numpy-discussion] Behavior of .base

[Travis Oliphant]

On Oct 1, 2012, at 9:11 AM, Jim Bosch wrote:

> On 09/30/2012 03:59 PM, Travis Oliphant wrote:
>> Hey all,
>> 
>> In a github-discussion with Gael and Nathaniel, we came up with a proposal for .base that we should put before this list.    Traditionally, .base has always pointed to None for arrays that owned their own memory and to the "most immediate" array object parent for arrays that did not own their own memory.   There was a long-standing issue related to running out of stack space that this behavior created.
>> 
>> Recently this behavior was altered so that .base always points to "the original" object holding the memory (something exposing the buffer interface).   This created some problems for users who relied on the fact that most of the time .base pointed to an instance of an array object.
>> 
>> The proposal here is to change the behavior of .base for arrays that don't own their own memory so that the .base attribute of an array points to "the most original object" that is still an instance of the type of the array.      This would go into the 1.7.0 release so as to correct the issues reported.
>> 
>> What are reactions to this proposal?
>> 
> 
> In the past, I've relied on putting arbitrary Python objects in .base in 
> my C++ to NumPy conversion code to make sure reference counting for 
> array memory works properly.  In particular, I've used Python CObjects 
> that hold boost::shared_ptrs, which don't even have a buffer interface. 
>  So it sounds like I may be a few steps behind on the rules of what 
> actually should go in .base.
> 

This should still work, nothing has been proposed to change this use-case.

> I'm very concerned that if we do demand that .base always point to a 
> NumPy array (rather than an arbitrary Python object or even just one 
> with a buffer interface), there's no longer any way for a NumPy array to 
> hold data allocated by something other than NumPy.

I don't recall a suggestion to demand that .base always point to a NumPy array.    The suggestion is that a view of a view of an array that has your boost::shared_ptr as a PyCObject pointed to by base will have it's base point to the first array instead of the PyCObject (as the recent change made). 

> 
> If I want to put external memory in a NumPy array and indicate that it's 
> owned by some non-NumPy Python object, what is the recommended way to do 
> that?

The approach you took is still the way I would recommend doing that.  There may be other suggestions. 

-Travis