On Thu, Mar 31, 2011 at 10:40 AM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Thu, Mar 31, 2011 at 7:06 PM, Mark Wiebe <mwwiebe@gmail.com> wrote:
> On Wed, Mar 30, 2011 at 4:42 PM, Eli Stevens (Gmail) <wickedgrey@gmail.com>
> wrote:
>>
>> On Fri, Mar 25, 2011 at 10:00 AM, Eli Stevens (Gmail)
>> <wickedgrey@gmail.com> wrote:
>> > Can anyone please give me some suggestions on how to go about writing
>> > a unit test for this?  Or should I just submit a pull request?
>>
>> I've gotten a bit of positive feedback to adding the 'e' type to the
>> struct module on the python-ideas list (per my understanding, not
>> before python 3.3, but I don't think that should hinder adoption in
>> other libraries), so I'd like to ask again about unit testing a change
>> like this.  Can anyone offer some advice for where to start?
>
> The tests for the buffer protocol are in
> numpy/core/tests/test_multiarray.py, starting at line 1847. It does some
> round-trip testing of all the types through the memoryview object.
> You'll also need to change the _dtype_from_pep3118 in
> numpy/core/_internal.py, called from the _descriptor_from_pep3118_format
> function, so that NumPy can accept the half type from a buffer as well.
>>
>> Also, what kind of timeframe / cutoff am I looking at to get this into
>> 1.6.0 or 1.6.x?
>
> Since the changes to NumPy are pretty small, I think it should be ok to get
> into 1.6.0. How to handle it is up to Ralf, however, and it would probably
> be best to get it into beta 2 which I believe he would like to release over
> the weekend (please chime in to correct me).

It would definitely need to get into beta 2, and even then I'm a
little hesitant to push in such a change at the last moment. It would
need some testing with other libraries that use the numpy buffer
protocol. Also, reading back the thread, Pauli seemed to disagree with
this.

My reading of Pauli's thoughts was that putting it in unilaterally is undesirable, something I definitely agree with. I think with Eli doing the legwork of getting input and acceptance from the relevant parties, we should help him out as much as possible. Not getting this change into 1.6 makes the Cython support much more difficult because of their design based around the buffer protocol.

Looking at the thread on python-users, I see mostly concern that the type be standard or have precedent elsewhere, which as an IEEE standard it definitely satisfies. The other question is on the chosen letter - we picked 'e' as 'h' was taken and it is close to 'f' and 'd', the other float types. If a letter change is deemed necessary, it would be good to get that into 1.6 as well, since this kind of change is easy now, but much more difficult later.

For testing, Eli implementing the Cython support seems like a good start to me. Another nice test would be to get PIL's OpenEXR support to export a half buffer to NumPy.

-Mark