[Numpy-discussion] Follow-up Numarray header PEP
gerard.vermeulen at grenoble.cnrs.fr
gerard.vermeulen at grenoble.cnrs.fr
Sun Jul 25 22:49:12 EDT 2004
Hi Todd,
Attached is a new version of numnum (including 'topbot', an alternative
implementation of numnum). The README contains some additional comments
with respect to numarray and Numeric (new comments are preceeded by '+',
old comments by '-'). There were still some other bugs in numnum, too.
On 23 Jul 2004 13:28:47 -0400, Todd Miller wrote
> I finally got to your numnum stuff today... awesome work! You've got
> lots of good suggestions. Here are some comments:
>
> 1. Thanks for catching the early return problem with numarray's
> import_array(). It's not just bad, it's wrong. It'll be fixed for 1.1.
>
> 2. That said, I think expanding the macros in-line in numnum is a
> mistake. It seems to me that "import_array(); PyErr_Clear();" or
> something like it ought to be enough... after numarray-1.1 anyway.
>
Indeed, but I am spoiled by C++ and was falling back on gcc -E for
debugging.
>
> 3. I think there's a problem in numnum.toNP() because of numarray's
> array "behavior" issues. A test needs to be done to ensure that the
> incoming array is not byteswapped or misaligned; if it is, the easy
> fix is to make a numarray copy of the array before copying it to Numeric.
>
Done, but what would be the best function to do this? And the documentation
could insist a little more on the possibility of ill-behaved arrays (see
README).
>
> 4. Kudos for the LP64 stuff. numconfig is a thorn in the side of the
> PEP, so I'll put your techniques into numarray for 1.1.
> HAS_FLOAT128 is not currently used, so it might be time to ditch
> it. Anyway, thanks!
>
There is a difference between the PEP header files and internal numarray
usage. I find in my CVS working copy:
[packer at slow numarray]$ grep HAS_FLOAT */*
Src/_ndarraymodule.c:#if HAS_FLOAT128
and
[packer at slow numarray]$ grep HAS_UINT64 */*
Src/buffer.ch: #if HAS_UINT64
Src/buffer.ch: #if HAS_UINT64
Src/buffer.ch: #if HAS_UINT64
Src/buffer.ch: #if HAS_UINT64
Src/buffer.ch: #if HAS_UINT64
Src/libnumarraymodule.c: #if HAS_UINT64
Src/libnumarraymodule.c: #if HAS_UINT64
Src/libnumarraymodule.c: #if HAS_UINT64
Src/libnumarraymodule.c: #if HAS_UINT64
Src/libnumarraymodule.c: #if HAS_UINT64
but that is not be true for the header files (more important for the PEP)
[packer at slow Include]$ grep HAS_UINT64 */*
[packer at slow Include]$ grep HAS_FLOAT128 */*
numarray/arraybase.h:#if HAS_FLOAT128
>
> 5. PyArray_Present() and isArray() are superfluous *now*. I was
> planning to add them to Numeric.
>
> 6. The LGPL may be a problem for us and is probably an issue if we ever
> try to get numnum into the Python distribution. It would be better
> to release numnum under the modified BSD license, same as numarray.
>
Done, with certain regrets because I believe in (L)GPL.
The minutes of the last board meeting of the PSF tipped the scale
( http://www.python.org/psf/records/board/minutes-2004-06-18.html )
What remains to be done is showing how to add numnum's functionality
to a 3rd party extension by linking numnum's object files to the
extension instead of importing numnum's C-API (numnum should not
become another dependency)
Gerard
>
> 7. Your API struct was very clean. Eventually I'll regenerate numarray
> like that.
>
> 8. I logged your comments and bug reports on Source Forge and eventually
> they'll get fixed.
>
> A to Z the numnum/pep code is beautiful. Next stop, header PEP update.
>
> Regards,
> Todd
>
>
> On Sun, 2004-07-18 at 17:24, gerard.vermeulen at grenoble.cnrs.fr wrote:
> > Hi Todd,
> >
> > This is a follow-up on the 'header pep' discussion.
> >
> > The attachment numnum-0.1.tar.gz contains the sources for the
> > extension modules pep and numnum. At least on my systems, both
> > modules behave as described in the 'numarray header PEP' when the
> > extension modules implementing the C-API are not present (a situation
> > not foreseen by the macros import_array() of Numeric and especially
> > numarray). IMO, my solution is 'bona fide', but requires further
> > testing.
> >
> > The pep module shows how to handle the colliding C-APIs of the Numeric
> > and numarray extension modules and how to implement automagical
> > conversion between Numeric and numarray arrays.
> >
> > For a technical reason explained in the README, the hard work of doing
> > the conversion between Numeric and numarray arrays has been delegated
> > to the numnum module. The numnum module is useful when one needs to
> > convert from one array type to the other to use an extension module
> > which only exists for the other type (eg. combining numarray's image
> > processing extensions with pygame's Numeric interface):
> >
> > Python 2.3+ (#1, Jan 7 2004, 09:17:35)
> > [GCC 3.3.1 (SuSE Linux)] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> import numnum; import Numeric as np; import numarray as na
> > >>> np1 = np.array([[1, 2], [3, 4]]); na1 = numnum.toNA(np1)
> > >>> na2 = na.array([[1, 2, 3], [4, 5, 6]]); np2 = numnum.toNP(na2)
> > >>> print type(np1); np1; type(np2); np2
> > <type 'array'>
> > array([[1, 2],
> > [3, 4]])
> > <type 'array'>
> > array([[1, 2, 3],
> > [4, 5, 6]],'i')
> > >>> print type(na1); na1; type(na2); na2
> > <class 'numarray.numarraycore.NumArray'>
> > array([[1, 2],
> > [3, 4]])
> > <class 'numarray.numarraycore.NumArray'>
> > array([[1, 2, 3],
> > [4, 5, 6]])
> > >>>
> >
> > The pep module shows how to implement array processing functions which
> > use the Numeric, numarray or Sequence C-API:
> >
> > static PyObject *
> > wysiwyg(PyObject *dummy, PyObject *args)
> > {
> > PyObject *seq1, *seq2;
> > PyObject *result;
> >
> > if (!PyArg_ParseTuple(args, "OO", &seq1, &seq2))
> > return NULL;
> >
> > switch(API) {
> > case NumericAPI:
> > {
> > PyObject *np1 = NN_API->toNP(seq1);
> > PyObject *np2 = NN_API->toNP(seq2);
> > result = np_wysiwyg(np1, np2);
> > Py_XDECREF(np1);
> > Py_XDECREF(np2);
> > break;
> > }
> > case NumarrayAPI:
> > {
> > PyObject *na1 = NN_API->toNA(seq1);
> > PyObject *na2 = NN_API->toNA(seq2);
> > result = na_wysiwyg(na1, na2);
> > Py_XDECREF(na1);
> > Py_XDECREF(na2);
> > break;
> > }
> > case SequenceAPI:
> > result = seq_wysiwyg(seq1, seq2);
> > break;
> > default:
> > PyErr_SetString(PyExc_RuntimeError, "Should never happen");
> > return 0;
> > }
> >
> > return result;
> > }
> >
> > See the README for an example session using the pep module showing that
> > it is possible pass a mix of Numeric and numarray arrays to pep.wysiwyg().
> >
> > Notes:
> >
> > - it is straightforward to adapt pep and numnum so that the conversion
> > functions are linked into pep instead of imported.
> >
> > - numnum is still 'proof of concept'. I am thinking about methods to
> > make those techniques safer if the numarray (and Numeric?) header
> > files make it never into the Python headers (or make it safer to
> > use those techniques with Python < 2.4). In particular it would
> > be helpful if the numerical C-APIs export an API version number,
> > similar to the versioning scheme of shared libraries -- see the
> > libtool->versioning info pages.
> >
> > I am considering three possibilities to release a more polished
> > version of numnum (3rd party extension writers may prefer to link
> > rather than import numnum's functionality):
> >
> > 1. release it from PyQwt's project page
> > 2. register an independent numnum project at SourceForge
> > 3. hand numnum over to the Numerical Python project (frees me from
> > worrying about API changes).
> >
> >
> > Regards -- Gerard Vermeulen
>
> --
--
Open WebMail Project (http://openwebmail.org)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: numnum-0.2.tar.gz
Type: application/gzip
Size: 19729 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20040725/b5485182/attachment-0001.bin>
More information about the NumPy-Discussion
mailing list