Hi Todd, Attached is a new version of numnum (including 'topbot', an alternative implementation of numnum). The README contains some additional comments with respect to numarray and Numeric (new comments are preceeded by '+', old comments by '-'). There were still some other bugs in numnum, too. On 23 Jul 2004 13:28:47 -0400, Todd Miller wrote
I finally got to your numnum stuff today... awesome work! You've got lots of good suggestions. Here are some comments:
1. Thanks for catching the early return problem with numarray's import_array(). It's not just bad, it's wrong. It'll be fixed for 1.1.
2. That said, I think expanding the macros in-line in numnum is a mistake. It seems to me that "import_array(); PyErr_Clear();" or something like it ought to be enough... after numarray-1.1 anyway.
Indeed, but I am spoiled by C++ and was falling back on gcc -E for debugging.
3. I think there's a problem in numnum.toNP() because of numarray's array "behavior" issues. A test needs to be done to ensure that the incoming array is not byteswapped or misaligned; if it is, the easy fix is to make a numarray copy of the array before copying it to Numeric.
Done, but what would be the best function to do this? And the documentation could insist a little more on the possibility of ill-behaved arrays (see README).
4. Kudos for the LP64 stuff. numconfig is a thorn in the side of the PEP, so I'll put your techniques into numarray for 1.1. HAS_FLOAT128 is not currently used, so it might be time to ditch it. Anyway, thanks!
There is a difference between the PEP header files and internal numarray usage. I find in my CVS working copy: [packer@slow numarray]$ grep HAS_FLOAT */* Src/_ndarraymodule.c:#if HAS_FLOAT128 and [packer@slow numarray]$ grep HAS_UINT64 */* Src/buffer.ch: #if HAS_UINT64 Src/buffer.ch: #if HAS_UINT64 Src/buffer.ch: #if HAS_UINT64 Src/buffer.ch: #if HAS_UINT64 Src/buffer.ch: #if HAS_UINT64 Src/libnumarraymodule.c: #if HAS_UINT64 Src/libnumarraymodule.c: #if HAS_UINT64 Src/libnumarraymodule.c: #if HAS_UINT64 Src/libnumarraymodule.c: #if HAS_UINT64 Src/libnumarraymodule.c: #if HAS_UINT64 but that is not be true for the header files (more important for the PEP) [packer@slow Include]$ grep HAS_UINT64 */* [packer@slow Include]$ grep HAS_FLOAT128 */* numarray/arraybase.h:#if HAS_FLOAT128
5. PyArray_Present() and isArray() are superfluous *now*. I was planning to add them to Numeric.
6. The LGPL may be a problem for us and is probably an issue if we ever try to get numnum into the Python distribution. It would be better to release numnum under the modified BSD license, same as numarray.
Done, with certain regrets because I believe in (L)GPL. The minutes of the last board meeting of the PSF tipped the scale ( http://www.python.org/psf/records/board/minutes-2004-06-18.html ) What remains to be done is showing how to add numnum's functionality to a 3rd party extension by linking numnum's object files to the extension instead of importing numnum's C-API (numnum should not become another dependency) Gerard
7. Your API struct was very clean. Eventually I'll regenerate numarray like that.
8. I logged your comments and bug reports on Source Forge and eventually they'll get fixed.
A to Z the numnum/pep code is beautiful. Next stop, header PEP update.
Regards, Todd
On Sun, 2004-07-18 at 17:24, gerard.vermeulen@grenoble.cnrs.fr wrote:
Hi Todd,
This is a follow-up on the 'header pep' discussion.
The attachment numnum-0.1.tar.gz contains the sources for the extension modules pep and numnum. At least on my systems, both modules behave as described in the 'numarray header PEP' when the extension modules implementing the C-API are not present (a situation not foreseen by the macros import_array() of Numeric and especially numarray). IMO, my solution is 'bona fide', but requires further testing.
The pep module shows how to handle the colliding C-APIs of the Numeric and numarray extension modules and how to implement automagical conversion between Numeric and numarray arrays.
For a technical reason explained in the README, the hard work of doing the conversion between Numeric and numarray arrays has been delegated to the numnum module. The numnum module is useful when one needs to convert from one array type to the other to use an extension module which only exists for the other type (eg. combining numarray's image processing extensions with pygame's Numeric interface):
Python 2.3+ (#1, Jan 7 2004, 09:17:35) [GCC 3.3.1 (SuSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import numnum; import Numeric as np; import numarray as na np1 = np.array([[1, 2], [3, 4]]); na1 = numnum.toNA(np1) na2 = na.array([[1, 2, 3], [4, 5, 6]]); np2 = numnum.toNP(na2) print type(np1); np1; type(np2); np2 <type 'array'> array([[1, 2], [3, 4]]) <type 'array'> array([[1, 2, 3], [4, 5, 6]],'i') print type(na1); na1; type(na2); na2 <class 'numarray.numarraycore.NumArray'> array([[1, 2], [3, 4]]) <class 'numarray.numarraycore.NumArray'> array([[1, 2, 3], [4, 5, 6]])
The pep module shows how to implement array processing functions which use the Numeric, numarray or Sequence C-API:
static PyObject * wysiwyg(PyObject *dummy, PyObject *args) { PyObject *seq1, *seq2; PyObject *result;
if (!PyArg_ParseTuple(args, "OO", &seq1, &seq2)) return NULL;
switch(API) { case NumericAPI: { PyObject *np1 = NN_API->toNP(seq1); PyObject *np2 = NN_API->toNP(seq2); result = np_wysiwyg(np1, np2); Py_XDECREF(np1); Py_XDECREF(np2); break; } case NumarrayAPI: { PyObject *na1 = NN_API->toNA(seq1); PyObject *na2 = NN_API->toNA(seq2); result = na_wysiwyg(na1, na2); Py_XDECREF(na1); Py_XDECREF(na2); break; } case SequenceAPI: result = seq_wysiwyg(seq1, seq2); break; default: PyErr_SetString(PyExc_RuntimeError, "Should never happen"); return 0; }
return result; }
See the README for an example session using the pep module showing that it is possible pass a mix of Numeric and numarray arrays to pep.wysiwyg().
Notes:
- it is straightforward to adapt pep and numnum so that the conversion functions are linked into pep instead of imported.
- numnum is still 'proof of concept'. I am thinking about methods to make those techniques safer if the numarray (and Numeric?) header files make it never into the Python headers (or make it safer to use those techniques with Python < 2.4). In particular it would be helpful if the numerical C-APIs export an API version number, similar to the versioning scheme of shared libraries -- see the libtool->versioning info pages.
I am considering three possibilities to release a more polished version of numnum (3rd party extension writers may prefer to link rather than import numnum's functionality):
1. release it from PyQwt's project page 2. register an independent numnum project at SourceForge 3. hand numnum over to the Numerical Python project (frees me from worrying about API changes).
Regards -- Gerard Vermeulen
--
-- Open WebMail Project (http://openwebmail.org)