[Numpy-discussion] core library structure

Mark Wiebe mwwiebe at gmail.com
Thu Feb 3 14:07:18 EST 2011

Moving this to a new thread.

On Thu, Feb 3, 2011 at 10:50 AM, Charles R Harris <charlesr.harris at gmail.com
> wrote:
> On Thu, Feb 3, 2011 at 11:07 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:
>> On Tue, Feb 1, 2011 at 10:09 PM, David <david at silveregg.co.jp> wrote:
>>> On 02/02/2011 02:57 PM, Mark Wiebe wrote:
>>> [snip]
>>> > I think the error handling policy
>>> > in the ufuncs could also be useful for computations in the core.
>>> At the moment, ufunc and multiarray are separate extensions. If each
>>> depends on the other API-wise, it is an issue. If there are some
>>> commonalities, then they can be put in a separate extensions. They are
>>> already way too big as they are (historical reasons), we should really
>>> fight this for maintanability.
>>> I realize the code organization and the build stuff is a bit of a mess
>>> at the moment, so if you have questions on that aspect, feel free to ask
>>> clarification or help,
>> Yeah, I understand it's the result of organic growth and merging from many
>> different sources. The core library should probably become layered in a
>> manner roughly as follows, with each layer depending only on the previous
>> APIs.  This is what Travis was getting at, I believe, with the generator
>> array idea affecting mainly the Storage and Iteration APIs, generalizing
>> them so that new storage and iteration methods can be plugged in.
>> Data Type API: data type numbers, casting, byte-swapping, etc.
>> Array Storage API: array creation/manipulation/deletion.
>> Array Iteration API: array iterators, copying routines.
>> Array Calculation API: typedefs for various types of calculation
>> functions, common calculation idioms, ufunc creation API, etc.
>> Then, the ufunc module would use the Array Calculation API to implement
>> all the ufuncs and other routines like inner, dot, trace, diag, tensordot,
>> einsum, etc.
> I like the lower two levels if, as I assume, they are basically aimed at
> allocating, deallocating blocks of memory (or equivalent) and doing basic
> manipulations such as dealing with endianess and casting. Where do you see
> array methods making an appearance?

That's correct. Currently, for example, the cast functions take array
objects as parameters, something that would no longer be the case.  The
array methods vs functions only shows up in the Python exposure, I believe.
 The above structure only affects the C library, and how its exposed to
Python could remain as it is now.

The original Numeric only had three (IIRC) rather basic methods and
> everything else was function based, an approach which is probably easier to
> maintain. The extensive use of methods came from numarray and might be
> something that could be added at a higher level so that the current ndarrays
> would be objects combining ow level arrays and ufuncs.

Sounds reasonable to me.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110203/714a8213/attachment.html>

More information about the NumPy-Discussion mailing list