[New-bugs-announce] [issue16132] ctypes incorrectly encodes .format attribute of memory views

[David Beazley]

New submission from David Beazley:

This is somewhat related to an earlier bug report concerning memory views, but as far as I can tell, ctypes is not encoding the '.format' attribute correctly in most cases.   Consider this example:

First, create a ctypes array:

>>> a = (ctypes.c_double * 3)(1,2,3)
>>> len(a)
3
>>> a[0]
1.0
>>> a[1]
2.0
>>> 

Now, create a memory view for it:

>>> m = memoryview(a)
>>> len(m)
3
>>> m.itemsize
8
>>> m.ndim
1
>>> m.shape
(3,)
>>> 

All looks well.  However, if you try to do anything with the .format or access the items, it's completely broken:

>>> m.format
'(3)<d'
>>> m[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NotImplementedError: memoryview: unsupported format (3)<d
>>> 

This is quite inconsistent with the behavior observed elsewhere. For example:

>>> import array
>>> b = array.array('d',[1,2,3])
>>> memoryview(b).format
'd'
>>> import numpy
>>> c = numpy.array([1,2,3],dtype='d')
>>> memoryview(c).format
'd'
>>> 

As you can see, array libraries are using .format to encode the format of a single array item.  ctypes is encoding the format of the entire array (all items).  ctypes also includes endianness which presents additional difficulties.

This behavior affects both Python code that wants to use memoryviews, but also C extension code that wants to use the underlying buffer protocol to work with arrays in a generic way. Essentially, it cuts the use of ctypes off entirely unless you modify the underlying buffer handling code to special case it. 

Suggested fix:  Have ctypes only encode the format for a single item in the case of arrays.  Also, for items that are encoded using the native byte ordering, don't include an endianness modifier ('<','>', etc.).  Including the byte order just complicates all of the handling code because it has to be modified to a) know what the native byte ordering is and b) to check multiple cases such as for "d" and "<d".

----------
messages: 171965
nosy: dabeaz
priority: normal
severity: normal
status: open
title: ctypes incorrectly encodes .format attribute of memory views
type: behavior
versions: Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16132>
_______________________________________