[New-bugs-announce] [issue39295] usage of bitfields in ctypes structures changed between 3.7.5 and 3.7.6

Matthew Newville report at bugs.python.org
Fri Jan 10 17:16:07 EST 2020

New submission from Matthew Newville <matt.newville at gmail.com>:

We have a library (https://github.com/pyepics/pyepics) that wraps several C structures for a communication protocol library that involves many C->Python callbacks.  One of the simpler structures we wrap with ctypes is defined with

typedef struct ca_access_rights {
    unsigned    read_access:1;
    unsigned    write_access:1; } caar;

struct  access_rights_handler_args {
    long  chanId; /* channel id */
    caar  access; /* access rights state */

which we had wrapped (perhaps naively) as 

class access_rights_handler_args(ctypes.Structure):
    "access rights arguments"
    _fields_ = [('chid', ctypes.c_long),
                ('read_access', ctypes.c_uint, 1),
                ('write_access', ctypes.c_uint, 1)]

which we would then this structure as the function argument of a callback function that the underlying library would call, using

    _Callback = ctypes.CFUNCTYPE(None, ctypes.POINTER(access_rights_handler_args))(access_rights_handler)

and the python function `access_righte_handler` would be able to unpack and use this structure.  This worked for Python 2.7, 3.3 - 3.7.5 on 64-bit Linux, Windows, and MacOS.  This code was well-tested and was used in production code on very many systems. It did not cause segfaults.

With Python 3.7.6 this raises an exception at the ctypes.CFUNCTYPE() call with

...../lib/python3.7/ctypes/__init__.py", line 99, in CFUNCTYPE
    class CFunctionType(_CFuncPtr):
TypeError: item 1 in _argtypes_ passes a struct/union with a bitfield by value, which is unsupported.

We were able to find a quick work-around this by changing the structure definition to be

class access_rights_handler_args(ctypes.Structure):
    "access rights arguments"
    _fields_ = [('chid', ctypes.c_long),
                ('access', ctypes.c_ubyte)]

and then explicitly extract the 2 desired bits from the byte. Of course, that byte is more data than is being sent in the structure, so there is trailing garbage.

This change seems to have been related to https://bugs.python.org/issue16576.

Is there any way to restore the no-really-I'm-not-making-it-up-it-was-most-definitely-working-for-us behavior of Python 3.7.5 and earlier?  

If this is not possible, what would be the right way to wrap this sort of structure? Thanks

components: ctypes
messages: 359763
nosy: Matthew Newville
priority: normal
severity: normal
status: open
title: usage of bitfields in ctypes structures changed between 3.7.5 and 3.7.6
type: behavior
versions: Python 3.7

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list