NumPy frombuffer giving nonsense values when reading C float array on Windows
eryk sun
eryksun at gmail.com
Tue Jul 26 18:42:41 EDT 2016
On Tue, Jul 26, 2016 at 6:31 PM, sth <urschrei at gmail.com> wrote:
>
> The restype is a ctypes Structure instance with a single __fields__ entry (coords), which
Watch the underscores with ctypes attributes. Your code spells it
correctly as "_fields_".
> is a Structure with two fields (len and data) which are the FFI array's length and the void
> pointer to its memory:
> https://github.com/urschrei/pypolyline/blob/master/pypolyline/util.py#L109-L117
_FFIArray.__init__ isn't properly keeping a reference to the wrapped
numpy array:
def __init__(self, seq, data_type = c_double):
ptr = POINTER(data_type)
nparr = np.array(seq, dtype=np.float64)
arr = nparr.ctypes.data_as(ptr)
self.data = cast(arr, c_void_p)
self.len = len(seq)
arr doesn't have a reference to nparr, so self.data doesn't have a
reference to the numpy array when it goes out of scope. For example,
we can trigger a segfault here:
>>> nparr = np.array(range(2**24), dtype=np.float64)
>>> ptr = ctypes.POINTER(ctypes.c_double)
>>> arr = nparr.ctypes.data_as(ptr)
>>> arr[0], arr[2**24-1]
(0.0, 16777215.0)
>>> del nparr
>>> arr[0], arr[2**24-1]
Segmentation fault (core dumped)
> I'm only half-following your explanation of how ctypeslib works, but it seems clear that I'm doing
> something wrong.
The library requires the data pointers wrapped in a struct with the
length, so I think what you're doing to wrap and unwrap arbitrary
sequences is generally fine. But you don't need the bytes copy from
string_at. You can cast to a double pointer and create a numpy array
from that, all without copying any data. The only copy made is for
tolist(). For example:
def _void_array_to_nested_list(res, func, args):
""" Dereference the FFI result to a list of coordinates """
try:
shape = res.coords.len, 2
ptr = cast(res.coords.data, POINTER(c_double))
array = np.ctypeslib.as_array(ptr, shape)
return array.tolist()
finally:
drop_array(res.coords)
If you're not hard coding the double data type, consider adding a
simple C type string to the _FFIArray struct. For example:
>>> ctypes.c_double._type_
'd'
You just need a dict to map type strings to simple C types.
More information about the Python-list
mailing list