[Python-Dev] pre-PEP: The Safe Buffer Interface

[Guido van Rossum]

> > > > I guess I still don't understand Neil's requirements.  What can't be
> > > > done with the existing buffer interface (which requires you to hold
> > > > the GIL while using the pointer)?
> > > 
> > > Processing in Python :-(.
> > 
> > Can you work out an example?
> Not sure, maybe Neil could do it better.
> 
> However, you yourself pointed out to Greg that it may be unsafe
> to even call Py_DECREF() on an unrelated object.

The safe rule is that you should grab the pointer and then do some I/O
on it and nothing else.

> >  I don't understand what you can do in
> > Python, apart from passing it to something else that takes the buffer
> > API or converting the data to a string or a bytes buffer.
> 
> Or pack it into a buffer *object* and hand it to arbitrary
> Python code. That's what we have now.

Since the object you're packing already supports the buffer API, I
don't see the point of packing it in a buffer object.

> What does 'hold the GIL' mean in this context?
> No other thread can execute: we have complete control
> over what we do. But what are we *allowed* to do?

When accessing a movable buffer, the safest rule is no Python API
calls.  There's a less restrictive safe rule, but it's messy because
the end goal is "don't do anything that could conceivably end up in
the Python interpreter main loop (ceval.c)" and there's no easy rule
for that -- anything that uses Py_DECREF can end up doing that.

--Guido van Rossum (home page: http://www.python.org/~guido/)