Removing undocumented __buffer__ attribute lookup
In digging around the code, I found a gem in PyArray_FromBuffer (exposed to python as numpy.frombuffer). If a PyObject* does not have a tp_as_buffer->bf_getbuffer function, we check if the python object has a __buffer__ attribute. If so we use that as buf in PyObject_GetBuffer(buf, ...). This seems to stem back to the original numerics code, where getBuffer would look up the attribute and call it as a method. PyArray_FromBuffer does not call the attribute as a method, it simply passes it on to PyObject_GetBuffer, which will then raise an error saying it cannot convert a method. You can try this out by creating a class with a __buffer__ method and calling numpy.frombuffer on it. I submitted a pull request to remove the code. Since it is undocumented and (as far as I can tell) broken, I do not think we need a deprecation cycle. More details, including links to the original numeric code from 2005, in the PR https://github.com/numpy/numpy/pull/13049 Any thoughts or objections? Matti
Cc-ing in Travis, because he was the original author of the buffer protocol, and this is most definitely related. Best Regards, Hameer Abbasi
On Wednesday, Feb 27, 2019 at 9:20 AM, Matti Picus <matti.picus@gmail.com (mailto:matti.picus@gmail.com)> wrote: In digging around the code, I found a gem in PyArray_FromBuffer (exposed to python as numpy.frombuffer). If a PyObject* does not have a tp_as_buffer->bf_getbuffer function, we check if the python object has a __buffer__ attribute. If so we use that as buf in PyObject_GetBuffer(buf, ...).
This seems to stem back to the original numerics code, where getBuffer would look up the attribute and call it as a method. PyArray_FromBuffer does not call the attribute as a method, it simply passes it on to PyObject_GetBuffer, which will then raise an error saying it cannot convert a method. You can try this out by creating a class with a __buffer__ method and calling numpy.frombuffer on it.
I submitted a pull request to remove the code. Since it is undocumented and (as far as I can tell) broken, I do not think we need a deprecation cycle.
More details, including links to the original numeric code from 2005, in the PR https://github.com/numpy/numpy/pull/13049
Any thoughts or objections?
Matti
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
On Wed, Feb 27, 2019 at 1:37 AM Hameer Abbasi <einstein.edison@gmail.com> wrote:
Cc-ing in Travis, because he was the original author of the buffer protocol, and this is most definitely related.
Best Regards, Hameer Abbasi
On Wednesday, Feb 27, 2019 at 9:20 AM, Matti Picus <matti.picus@gmail.com> wrote: In digging around the code, I found a gem in PyArray_FromBuffer (exposed to python as numpy.frombuffer). If a PyObject* does not have a tp_as_buffer->bf_getbuffer function, we check if the python object has a __buffer__ attribute. If so we use that as buf in PyObject_GetBuffer(buf, ...).
This seems to stem back to the original numerics code, where getBuffer would look up the attribute and call it as a method. PyArray_FromBuffer does not call the attribute as a method, it simply passes it on to PyObject_GetBuffer, which will then raise an error saying it cannot convert a method. You can try this out by creating a class with a __buffer__ method and calling numpy.frombuffer on it.
I submitted a pull request to remove the code. Since it is undocumented and (as far as I can tell) broken, I do not think we need a deprecation cycle.
I vaguely recall using that many, many years ago for some C code, but don't see anything using it these days. Chuck
participants (3)
-
Charles R Harris -
Hameer Abbasi -
Matti Picus