[Numpy-discussion] 1.6.0b1 half float buffer bug?

[Mark Wiebe]

On Thu, Mar 31, 2011 at 10:40 AM, Ralf Gommers
<ralf.gommers at googlemail.com>wrote:

> On Thu, Mar 31, 2011 at 7:06 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
> > On Wed, Mar 30, 2011 at 4:42 PM, Eli Stevens (Gmail) <
> wickedgrey at gmail.com>
> > wrote:
> >>
> >> On Fri, Mar 25, 2011 at 10:00 AM, Eli Stevens (Gmail)
> >> <wickedgrey at gmail.com> wrote:
> >> > Can anyone please give me some suggestions on how to go about writing
> >> > a unit test for this?  Or should I just submit a pull request?
> >>
> >> I've gotten a bit of positive feedback to adding the 'e' type to the
> >> struct module on the python-ideas list (per my understanding, not
> >> before python 3.3, but I don't think that should hinder adoption in
> >> other libraries), so I'd like to ask again about unit testing a change
> >> like this.  Can anyone offer some advice for where to start?
> >
> > The tests for the buffer protocol are in
> > numpy/core/tests/test_multiarray.py, starting at line 1847. It does some
> > round-trip testing of all the types through the memoryview object.
> > You'll also need to change the _dtype_from_pep3118 in
> > numpy/core/_internal.py, called from the _descriptor_from_pep3118_format
> > function, so that NumPy can accept the half type from a buffer as well.
> >>
> >> Also, what kind of timeframe / cutoff am I looking at to get this into
> >> 1.6.0 or 1.6.x?
> >
> > Since the changes to NumPy are pretty small, I think it should be ok to
> get
> > into 1.6.0. How to handle it is up to Ralf, however, and it would
> probably
> > be best to get it into beta 2 which I believe he would like to release
> over
> > the weekend (please chime in to correct me).
>
> It would definitely need to get into beta 2, and even then I'm a
> little hesitant to push in such a change at the last moment. It would
> need some testing with other libraries that use the numpy buffer
> protocol. Also, reading back the thread, Pauli seemed to disagree with
> this.


My reading of Pauli's thoughts was that putting it in unilaterally is
undesirable, something I definitely agree with. I think with Eli doing the
legwork of getting input and acceptance from the relevant parties, we should
help him out as much as possible. Not getting this change into 1.6 makes the
Cython support much more difficult because of their design based around the
buffer protocol.

Looking at the thread on python-users, I see mostly concern that the type be
standard or have precedent elsewhere, which as an IEEE standard it
definitely satisfies. The other question is on the chosen letter - we picked
'e' as 'h' was taken and it is close to 'f' and 'd', the other float types.
If a letter change is deemed necessary, it would be good to get that into
1.6 as well, since this kind of change is easy now, but much more difficult
later.

For testing, Eli implementing the Cython support seems like a good start to
me. Another nice test would be to get PIL's OpenEXR support to export a half
buffer to NumPy.

-Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110331/1ea6ece8/attachment.html>