[Numpy-discussion] Trying out Numeric3

[Scott Gilbert]

--- Travis Oliphant <oliphant at ee.byu.edu> wrote:
> 
>    - the PEP for a good "buffer" object (this has been called by others 
> a "byte" array which might be a good name.  Essentially, it needs to be 
> a light-weight object around a chunk of memory -- i.e. a way to allocate 
> memory through Python.  We would like to standardize on a set of meta 
> information that could be used to "understand" this memory as a numeric 
> array.   Then, other objects which used this buffer as a memory block 
> would just have to expose the meta information in order to make seamless 
> the transfer of data from one application to another.    We need to be 
> vocal about the value of the buffer object.  This PEP is one way to do 
> that.   There are some people who think buffer objects were a "bad 
> idea."  This is primarily because of a fatal flaw in some objects that 
> both expose a memory pointer through the buffer protocol AND allow the 
> object's memory to be reallocated (using realloc) --- Numeric does not 
> do this.     This problem could actually be easily fixed by a good 
> Python memory allocator that returns a simple memory object.  If people 
> who wanted memory went through it's C-API (instead of using malloc and 
> realloc), much of the problems would be alleviated.   This is what the 
> new "byte" object should be.   I think it also wise to expect the "byte" 
> object to have an attribute called "meta" that would just be a 
> dictionary of "other information" you might want to pass to something 
> using the buffer protocol.
> 


Hi Travis.

I'm curious if you find PEP-296 sufficient:

    http://www.python.org/peps/pep-0296.html

It is marked as "withdrawn by the author", but that is not really true.  A
more accurate statement would be "the author spent his allotted time
defending and revising the PEP on the Python mailing list and was not left
with sufficient time to finish the implementation on his corporate dollar".
 :-)  It's a good PEP, and while my company uses Python quite extensively,
after two weeks I had to get back to more direct goals...

Regardless, I think PEP-296 meets your needs (and several other groups in
the Python community), and it might save someone the time recreating a new
PEP from scratch.  More importantly, it might save someone some of the time
required to defend and argue the PEP on the Python mailing list.  When the
discussion cleared, Guido was very positive toward the PEP - I just never
got it implemented...

The "meta" attribute would be a small change.  It's possible to do that
with composition or inheritance instead, but that's really a just matter of
taste.

When I wrote the PEP, I had high hopes of creating a Python only "ndarray"
class out of bytes and the struct module, so it was definitely targeted at
needs similar to what I believe yours to be.  Obviously you should do what
is best for you, but I would be pleased if my wasted effort was revived and
 completed to actually be useful.


Cheers,
    -Scott