[Numpy-discussion] core library structure

[Mark Wiebe]

Moving this to a new thread.

On Thu, Feb 3, 2011 at 10:50 AM, Charles R Harris <charlesr.harris at gmail.com
> wrote:
>
> On Thu, Feb 3, 2011 at 11:07 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>
>> On Tue, Feb 1, 2011 at 10:09 PM, David <david at silveregg.co.jp> wrote:
>>
>>> On 02/02/2011 02:57 PM, Mark Wiebe wrote:
>>> [snip]
>>>
>>
>>> > I think the error handling policy
>>> > in the ufuncs could also be useful for computations in the core.
>>>
>>> At the moment, ufunc and multiarray are separate extensions. If each
>>> depends on the other API-wise, it is an issue. If there are some
>>> commonalities, then they can be put in a separate extensions. They are
>>> already way too big as they are (historical reasons), we should really
>>> fight this for maintanability.
>>>
>>> I realize the code organization and the build stuff is a bit of a mess
>>> at the moment, so if you have questions on that aspect, feel free to ask
>>> clarification or help,
>>
>>
>> Yeah, I understand it's the result of organic growth and merging from many
>> different sources. The core library should probably become layered in a
>> manner roughly as follows, with each layer depending only on the previous
>> APIs.  This is what Travis was getting at, I believe, with the generator
>> array idea affecting mainly the Storage and Iteration APIs, generalizing
>> them so that new storage and iteration methods can be plugged in.
>>
>> Data Type API: data type numbers, casting, byte-swapping, etc.
>> Array Storage API: array creation/manipulation/deletion.
>> Array Iteration API: array iterators, copying routines.
>> Array Calculation API: typedefs for various types of calculation
>> functions, common calculation idioms, ufunc creation API, etc.
>>
>> Then, the ufunc module would use the Array Calculation API to implement
>> all the ufuncs and other routines like inner, dot, trace, diag, tensordot,
>> einsum, etc.
>>
>>
> I like the lower two levels if, as I assume, they are basically aimed at
> allocating, deallocating blocks of memory (or equivalent) and doing basic
> manipulations such as dealing with endianess and casting. Where do you see
> array methods making an appearance?
>

That's correct. Currently, for example, the cast functions take array
objects as parameters, something that would no longer be the case.  The
array methods vs functions only shows up in the Python exposure, I believe.
 The above structure only affects the C library, and how its exposed to
Python could remain as it is now.

The original Numeric only had three (IIRC) rather basic methods and
> everything else was function based, an approach which is probably easier to
> maintain. The extensive use of methods came from numarray and might be
> something that could be added at a higher level so that the current ndarrays
> would be objects combining ow level arrays and ufuncs.
>

Sounds reasonable to me.

-Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110203/714a8213/attachment.html>