[issue10181] Problems with Py_buffer management in memoryobject.c (and elsewhere?)

Stefan Krah report at bugs.python.org
Wed Jul 6 14:39:20 CEST 2011


Stefan Krah <stefan-usenet at bytereef.org> added the comment:

Antoine is right, this needs to be fixed. I think that for *practical*
purposes, the existing release() method already behaves like a tryrelease()
method:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
BufferError: Existing exports of data: object cannot be re-sized
>>>

So while m1.release() in fact *does* release a buffer, the desired
effect (freeing up 'b' for subsequent operations) only happens
after also calling m2.release(). This applies to Python and NumPy
objects.

With this amount of complexity, it might be a good idea to separate
refcounting and exports, like so:

typedef struct {
    PyObject_HEAD
    int released;
    Py_ssize_t exports; /* total number of exports */
    Py_buffer master;
} PyManagedBufferObject;

typedef struct {
    PyObject_HEAD
    PyManagedBufferObject *mbuf;
    Py_ssize_t shape[Py_MEMORYVIEW_MAXSTATIC];
    Py_ssize_t strides[Py_MEMORYVIEW_MAXSTATIC];
    Py_ssize_t suboffsets[Py_MEMORYVIEW_MAXSTATIC];
    int released; /* for clarity in the code */
    Py_ssize_t exports; /* buffer exports */
    Py_buffer view;
} PyMemoryViewObject;

Then it is possible for a MemoryView to call mbuf_release() from
memory_release() if self->exports == 0 and --self->mbuf->exports == 0.

So, if I understand Antoine correctly, in following graph m1, m2 and
m4 could mark themselves as 'released' and decrement self->mbuf->exports.
This could happen in any order.

What should happen with m2.release()? Raising would be odd here as well,
since Pauli stated earlier that some people might actually construct
new objects from an exported buffer.

m2 could mark itself as 'release_pending', and henceforth only accept
releasebuffer requests from b1, b2 and x1. Once those are all in, it
releases itself properly and sends the message to mbuf.

'release_pending' could be expressed as (self->released && self->exports > 0).

                        ManagedBuffer (mbuf)
                             |
             -----------------------------------
             |                                 |
             m1 (private arrays)              m3 (private arrays)
             |                                 |
             m2                               m4 (private arrays)
             |
     -----------------
     |       |       |
     |      b1      b2 (regular temporary buffer exports)
     |
     x1 (new object constructed from a buffer export)

Antoine, was this roughly your suggestion?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10181>
_______________________________________


More information about the Python-bugs-list mailing list