[Tutor] Passing Data to .DLL

[eryksun]

On Wed, Oct 22, 2014 at 3:50 PM, Wilson, Pete <Pete.Wilson at atmel.com> wrote:
> I don't understand the line
> rx_buf = (c_uint8 * rx_buf_size).from_buffer_copy(string_buf)

A buffer is a block of memory used to pass data between functions,
processes, or systems. Specifically its use as a 'buffer' comes from
using a block of memory to buffer I/O operations. There are also
buffers implemented in hardware. They're typically FIFO (first in,
first out) queues or circular buffers (e.g. the sample buffer in an
ADC).

In ctypes, the expression `c_uint8 * rx_buf_size` creates an array
type that consists of rx_buf_size elements of type c_uint8. Note that
an array in C is an aggregate type. It's not a pointer, even though it
degenerates to a pointer to the first element when passed as an
argument to a function (look back at how I defined it in
process_incoming_serial_data.argtypes).

An instance of this array type has a contiguous buffer that consists
of rx_buf_size elements that are each sizeof(c_uint8) bytes:

    >>> rx_buf_size = 8
    >>> rx_buf_t = c_uint8 * rx_buf_size
    >>> sizeof(c_uint8) * 8
    8
    >>> sizeof(rx_buf_t)
    8

> or where .from_buffer_copy() came from but it works...

from_buffer is a class method of ctypes data types. It creates an
instance that shares the same base address as the target buffer, but
only if it's writable and at least the same size. In this case
'buffer' is referring to Python's buffer protocol. Python byte strings
implement this protocol, but not unicode strings.

Immutable strings are read-only buffers:

>>> rx_buf_t.from_buffer('01234567')[:]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Cannot use string as modifiable buffer

A bytearray is writable:

    >>> string_buf = bytearray('01234567')
    >>> rx_buf = rx_buf_t.from_buffer(string_buf)
    >>> rx_buf[:]
    [48, 49, 50, 51, 52, 53, 54, 55]

Note that the buffer is shared by the bytearray and ctypes array:

    >>> string_buf[0] = '7'
    >>> rx_buf[:]
    [55, 49, 50, 51, 52, 53, 54, 55]

from_buffer_copy is similar, accept instead of sharing the buffer it
makes a private copy. Thus you can use it with an immutable string:

    >>> string_buf = '01234567'
    >>> rx_buf = rx_buf_t.from_buffer_copy(string_buf)
    >>> rx_buf[:]
    [48, 49, 50, 51, 52, 53, 54, 55]

In this case the buffer is *not* shared:

    >>> rx_buf[0] = ord('7')
    >>> string_buf
    '01234567'

These class methods are implemented by the metaclass, PyCArrayType.

    >>> PyCArrayType = type(rx_buf_t)
    >>> PyCArrayType
    <type '_ctypes.PyCArrayType'>

    >>> print '\n'.join(sorted(vars(PyCArrayType)))
    __doc__
    __mul__
    __new__
    __rmul__
    from_address
    from_buffer
    from_buffer_copy
    from_param
    in_dll

from_address is particularly dangerous. Misusing it will probably
crash the interpreter.

in_dll gives you access to a library's data exports. For example, the
flag Py_HashRandomizationFlag indicates whether the interpreter is
randomizing string hashes (command-line option -R). In Python 3 it's
on by default:

    >>> sys.flags.hash_randomization
    1
    >>> c_int.in_dll(pythonapi, 'Py_HashRandomizationFlag')
    c_long(1)

> I promise I will not knowingly pass Python strings to C.

It's fine if you see something such as `const char *var` in the
function prototype. The const qualifier lets you know the function
promises to not modify the buffer. In C, a promise is the best you can
get. Better still the docs will tell you whether or not the buffer
needs to be writable. This is an important distinction to make for the
shared library itself. A memory block could literally be read only
(not just by contract), such that trying to write to it will raise a
Windows access violation or POSIX segfault.