On Thu, 2004-07-01 at 02:33, gerard.vermeulen@grenoble.cnrs.fr wrote:
On 30 Jun 2004 17:54:19 -0400, Todd Miller wrote
So... you use the "meta" code to provide package specific ordinary (not-macro-fied) functions to keep the different versions of the Present() and isArray() macros from conflicting.
It would be nice to have a standard approach for using the same "extension enhancement code" for both numarray and Numeric. The PEP should really be expanded to provide an example of dual support for one complete and real function, guts and all, so people can see the process end-to-end; Something like a simple arrayprint. That process needs to be refined to remove as much tedium and duplication of effort as possible. The idea is to make it as close to providing one implementation to support both array packages as possible. I think it's important to illustrate how to partition the extension module into separate compilation units which correctly navigate the dual implementation mine field in the easiest possible way.
It would also be nice to add some logic to the meta-functions so that which array package gets used is configurable. We did something like that for the matplotlib plotting software at the Python level with the "numerix" layer, an idea I think we copied from Chaco. The kind of dispatch I think might be good to support configurability looks like this:
PyObject * whatsThis(PyObject *dummy, PyObject *args) { PyObject *result, *what = NULL; if (!PyArg_ParseTuple(args, "O", &what)) return 0; switch(PyArray_Which(what)) { USE_NUMERIC: result = Numeric_whatsThis(what); break; USE_NUMARRAY: result = Numarray_whatsThis(what); break; USE_SEQUENCE: result = Sequence_whatsThis(what); break; } Py_INCREF(Py_None); return Py_None; }
In the above, I'm picturing a separate .c file for Numeric_whatsThis and for Numarray_whatsThis. It would be nice to streamline that to one .c and a process which somehow (simply) produces both functions.
Or, ideally, the above would be done more like this:
PyObject * whatsThis(PyObject *dummy, PyObject *args) { PyObject *result, *what = NULL; if (!PyArg_ParseTuple(args, "O", &what)) return 0; switch(Numerix_Which(what)) { USE_NUMERIX: result = Numerix_whatsThis(what); break; USE_SEQUENCE: result = Sequence_whatsThis(what); break; } Py_INCREF(Py_None); return Py_None; }
Here, a common Numerix implementation supports both numarray and Numeric from a single simple .c. The extension module would do "#include numerix/arrayobject.h" and "import_numerix()" and otherwise just call PyArray_* functions.
The current stumbling block is that numarray is not binary compatible with Numeric... so numerix in C falls apart. I haven't analyzed every symbol and struct to see if it is really feasible... but it seems like it is *almost* feasible, at least for typical usage.
So, in a nutshell, I think the dual implementation support you demoed is important and we should work up an example and kick it around to make sure it's the best way we can think of doing it. Then we should add a section to the PEP describing dual support as well.
I would never apply numarray code to Numeric arrays and the inverse. It looks dangerous and I do not know if it is possible.
I think that's definitely the marching orders for now... but you gotta admit, it would be nice.
The first thing coming to mind is that numarray and Numeric arrays refer to different type objects (this is what my pep module uses to differentiate them). So, even if numarray and Numeric are binary compatible, any 'alien' code referring the the 'Python-standard part' of the type objects may lead to surprises. A PEP proposing hacks will raise eyebrows at least.
I'm a little surprised it took someone to talk me out of it... I'll just concede that this was probably a bad idea.
Secondly, most people use Numeric *or* numarray and not both.
A class of question which will arise for developers is this: "X works with Numeric, but X doesn't work with numaray." The reverse also happens occasionally. For this reason, being able to choose would be nice for developers.
So, I prefer: Numeric In => Numeric Out or Numarray In => Numarray Out (NINO) Of course, Numeric or numarray output can be a user option if NINO does not apply.
When I first heard it, I though NINO was a good idea, with the limitation that it doesn't apply when a function produces an array without consuming any. But... there is another problem with NINO that Perry Greenfield pointed out: with multiple arguments, there can be a mix of array types. For this reason, it makes sense to be able to coerce all the inputs to a particular array package. This form might look more like: switch(PyArray_Which(<no_parameter_at_all!>)) { case USE_NUMERIC: result = Numeric_doit(a1, a2, a3); break; case USE_NUMARRAY: result = Numarray_doit(a1, a2, a3); break; case USE_SEQUENCE: result = Sequence_doit(a1, a2, a3); break; } One last thing: I think it would be useful to be able to drive the code into sequence mode with arrays. This would enable easy benchmarking of the performance improvement.
(explicit safe conversion between Numeric and numarray is possible if really needed).
I'll try to flesh out the demo with real functions in the way you indicated (going as far as I consider safe).
The problem of coding the Numeric (or numarray) functions in more than a single source file has also be addressed.
It may take 2 weeks because I am off to a conference next week.
Excellent. See you in a couple weeks. Regards, Todd