[Python-Dev] pre-PEP: The Safe Buffer Interface

[Scott Gilbert]

--- Neil Hodgson <nhodgson@bigpond.net.au> wrote:
> Thomas Heller:
> 
> >     The size and pointer returned must be valid as long as the object
> >     is alive (has a positive reference count).  So, only objects which
> >     never reallocate or resize the memory block are allowed to
> >     implement this interface.
> 
> I'd prefer an interface that allows for reallocation but has an explicit
> locked state during which the buffer must stay still. My motivation comes
> from the data structures implemented in Scintilla (an editor component),
> which could be exposed through this buffer interface to other code. The
> most important type in Scintilla (as in many editors) is a split (or 
> gapped) buffer. Upon receiving a lock call, it could collapse the gap and
> return a stable pointer to its contents and then revert to its normal 
> behaviour on receiving an unlock.
> 

A couple of questions come to mind:

First, could this be implemented by a gapped_buffer object that implements
the locking functionality you want, but that returns simple buffers to work
with when the object is locked.  In other words, do we need to add this
extra functionality up in the core protocol when it can be implemented
specifically the way Scintilla (cool editor by the way) wants it to be in
the Scintilla specific extension.


Second, if you are using mutexes to do this stuff, you'll have to be very
careful about deadlock.  I imagine:

  thread 1:
      grab the object lock
      grab the object pointer
      release the GIL
      do some work
      acquire the GIL # deadlock

  thread 2:
      acquire the GIL
      try to resize the object # requires no outstanding locks

Thread 2 needs to make sure no objects are holding the object lock when it
does the resize, but thread 1 can't acquire the GIL until thread 2 gives it
up.  Both are stuck.

If you choose not to implement the locks with true mutexes, then you're
probably going to end up polling and that's bad too.

Is there a way out of this?  This is part of the reason I didn't want to
put a lock state into the bytes object.







__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com