It is however useful to distinguish between input and output arrays. If your array is just an input, then the PyArray_FromAny function you mention below is good enough. If necessary they will make a well-behaved copy which can be discarded afterwards. However, if you need to modify an array 'in-place', then your well-behaved copy needs to be copied back at the end of your function. That can be done transparently as numarray does that it. You mention that you could explicitly copy back, but that means that I have to determine myself if there was a temporary made. And that could also happen if the array was byteswapped or misaligned not just if I asked for a contiguous array. The Numarray functions do all that for you. The point is, that if you use an array just as an input, you don't want to copy things back, since that is a waste of resources. So you need somehow to distinguish between input and output arrays, which numarray does by having separate functions. For me, the NA_OutputArray function has been very useful, I would like to see that functionality somehow in Numeric3. The NA_IoArray function I don't really need. Maybe I have understood the Numeric API wrong and you can do this right now, I have not used it very much lately. I know you avoid these problems if you don't modify arrays at all, but just return new arrays with your result. However, then you cannot properly implement output arguments. Many numarray and Numeric functions do _not_ allow you to pass an output array, to modify "in-place" as you can do with ufuncs. That annoys me endlessly, it is a big source of inefficiency if you have big arrays. A good example are the FFT functions that make lots of temporaries if you do a multi-dimensional transform. I am digressing now, but maybe output arrays could be supported more consistently in Numeric3? Cheers, Peter On 10 Feb 2005, at 10:25, Travis Oliphant wrote:

Timo Korvola wrote:

...

Andrew McNamara writes:

...

I know this is a loaded question, but what does numarray offer that Numeric does not (as they stand today)? A cleaner implementation?

NA_OutputArray and NA_IoArray. Very convenient for wrapping C libraries. NA_InputArray is more or less equivalent to PyArray_ContiguousFromObject but as far as I know Numeric has nothing equivalent to the other two.

I am not convinced these are really useful additional C-API calls. I've done a fair bit of wrapping of C libraries and have never needed anything but

PyArray_ContiguousFromObject and PyArray_FromOb ject

This idea of "shadowing" ala NA_IOArray is a new one. Can somebody show me an example of it's usefulness that could not be handled by simply a PyArray_ContiguousFromObject with an explicit copy back at the end? I don't like this idea of "hiding" what is going on from the user.

This kind of C-API alteration which as far as I can tell has not been well justified is an example of some of the problems I see with Numarray.

Numeric3 condenses the Numeric API calls using numarray's idea of the requires flag:

PyArray_FromAny(PyObject *op, PyArray_Typecode *typecode, int mindepth, int maxdepth, int requires)

typecode is a simple structure with integer elements (type_num, itemsize, and fortran).

For some unexplained reason, the mindepth and maxdepth arguments are not copied by the new Numarray C-API. I'm not sure why. Having them does make it easier to guarantee you are getting the right dimension for the array you want rather than having to check after the fact.

These kinds of discussions about numarray's C-API never took place. In my opinion the Numeric C-API is better tested and easier to understand.

-Travis

------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion

On 10 Feb 2005, at 14:24, Timo Korvola wrote:

Peter Verveer writes:

...

The NA_IoArray function I don't really need.

Depends on what C code you are working with. Bidirectional array arguments are not uncommon in libraries.

I agree, this is just a personal preference of me to rather keep the input and output arrays separate arguments, which gives you the same functionality plus a bit more flexibility. I would not argue to remove NA_IoArray though, if you prefer that.

...

Maybe I have understood the Numeric API wrong and you can do this right now, I have not used it very much lately.

Perhaps you can but you would have to write it all yourself. In Numarray that work has been done for you. The three functions NA_{Input,Output,Io}Array are pretty much all you need for accessing C libraries with the usual (int size, double *data) interface, whichever way the data goes.

Agreed. I don't know what the work would be to extend the Numeric API to at least have the OutputArray functionality (I guess IoArray would then be easily also added), but it seems worth having it. Peter

If Numeric3 intends to support byteswapped and misaligned data then additional functionality is clearly required; whether or not the functionality is explicitly exposed in the API is another matter; it is hidden in our compatibility API so there's no reason Numeric3 can't hide it too. But it sounds to me like people have found the numarray high level API fairly useful so the design features bear discussion even if the API itself is dropped.

Exactly, I found the NA_*Array functions better than the original Numeric functions. It seems Travis is moving in a similar direction, but you are right that it is going to need extra functionality. A question I have about the compatibility API in Numarray: In cases where a temporary is created, does that get copied back when the temporary is destroyed? That seems to be needed when you use an array as an output. That would be wasteful if you only use the array as an input. Its of course fine to do that, since it is a compability API that you cannot change, but it would point to a weakness in the original Numeric API, which should be fixed by the Numeric people if they want to move forward. It is relevant to me since I decided to move the nd_image to the Numeric compatibility API, and while doing that found myself wondering that in some cases I would be introducing a slight performance penalty.

I used NA_IoArray myself in numarray's Numeric compatibility API and also found it useful in adding f2py support. I found it necessary to get Numeric's extra packages (LinearAlgebra, FFT, RandomArray) to pass their self-tests. Don't overlook the fact that arrays are typically passed by reference so whether we like it or not I/O behavior is normally exposed. For me, even for a job as small as the compatibility layer or f2py support, NA_IoArray was an asset. It's just easier to think about and use and eliminates the chance of getting the shadowing logic wrong.

Yes, I can only agree with that.

The NumArray destructor-based array shadowing works well to keep things transparent and porting changes low. You obviously can do the explict exit copy you've discussed, but I think that's the kind of unnecessary and intrusive change we're trying to get away from. With the best code coming from experts doing their own thing on their own time, I don't think support for numarray's advanced features is going to happen unless it's as automatic and simple to use as possible.

I can confirm that the high-level API of Numarray works very nicely to hide you from things you do not want to know (i.e. byte-swapping) while at the same time allowing flexibility that does not seem to be completely present in the Numeric API, unless I don't understand the latter well enough. Peter