[Python-Dev] pre-PEP: The Safe Buffer Interface

[Scott Gilbert]

--- Neil Hodgson <nhodgson@bigpond.net.au> wrote:
> 
>    Would this mean that the explicit locking completely defines the
> validity of the address or is the address valid until the 'view' buffer 
> object is garbage collected? I would like the gapped_buffer to be put 
> back into gapped mode as soon as possible and depending on the lifetime 
> of a view buffer object is not that robust in the face of alternate
> Python implementations that use non-reference-counted GC implementations
> (Jython / Python .Net).
>

If you're worried about exactly when the object is released, you could add
a specific release() method to your object indicating that you don't intend
to use it anymore.

My point was that, with Thomas Heller's safe buffer protocol (or my bytes
object), you would have a pointer that could be manipulated independently
of the GIL, but that putting locking semantics into your gapped_buffer is
something you could add on top without complicating the core.

In other words, his PEP (or mine) allows you to do something you couldn't
necessarily do previously, and it doesn't sound like there is anything you
want to do that you won't be able to.
 
> 
>    By locking, I want to change state on the buffer from having a gap and
> allowing resizes to having a static size and address which will remain
> valid until an unlock. The lock and unlock are not treating the buffer as
> a mutex (I'd call the operations 'acquire' and 'release' then) although
> mutexes may be needed for safety in the lock and unlock implementations.
> It is likely that the lock and unlock would be counted (it can be locked
> twice and then won't be expandable until it is unlocked twice) and that
> exceptions would be thrown for length changing operations while locked.
> 

You could easily implement the a counting (recursive) mutex as described
above, and it might be the case that throwing an exception on the length
changing operations keeps the dead lock from occurring.  I'm still a bit
confused though.  When thread A locks (acquires) the buffer, and thread B
tries to do a resize and it generates an exception, what is thread B
supposed to do next?  I assume that the resize was due to something like
the user typing somewhere in the buffer.  From a user interface point of
view, you can't just ignore their request to insert text.  Would you just
try the same operation again after catching the exception?  How long would
you wait?

>
>    If you think my particular use is out of the scope of what you are
> trying to achieve then that is fine.
> 

It is definitely up to Thomas Heller to decide what he wants his scope to
be, and I don't want to step on his toes at all.  Especially since the
reason for his PEP getting written is that I didn't want to add this stuff
to mine. :-)

I'm just trying to point out two things:

  1) With his PEP, there is a way to get the behavior you desire with out
adding the complexity to the core of Python.  And with recursive/counting
mutexes, the behavior you want is getting more complicated.  The "safe
buffer protocol" is likely to cater to a wide class of users.  I could be
wrong, but the "lockable gapped buffer protocol" probably appeals to a much
smaller set.

  2) Any time you go from one lock (mutex, GIL, semaphore) to multiple
locks, you can introduce deadlock states.  Without my understanding your
design fully, your use case sounds to me like it either has the potential
for deadlock, or the potential for polling.  There are ways to avoid this
of course, but then everyone has to follow a more complicated set of rules
(for instance build a hierarchy describing the order of locks to acquire). 
Since Thomas's PEP doesn't introduce any new types of locks, it sidesteps
these problems.



Cheers,
    -Scott


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com