[Python-Dev] PEP-298: buffer interface widening too much?

[Martin v. Löwis]

> I've used this aspect of PyArg_ParseTuple,  but forgot about it.    I 
> was thinking that you're right,  the original object is "gone", we're 
> stuck.   But...  it's still (in all probability) lying around in the arg 
> tuple,  so some sort of PyArg_ReleaseBuffers is possible.   

That looks clumsy, and also too easy to forget unless done systematically,
in which case it will add a considerable performance cost. I think the
locked buffer interface is much simpler in comparison.

If Jack ever manages to design an alternative parse tuple interface, I
assume it will arrange to automatically release the buffer when the function
returns (or, more precisely, when the buffer is deallocated.

In fact, my PEP 286 could address this issue: the locked buffers would be
associated with the enhanced tuple, so that releasing the tuple would unlock
the buffers. This *still* has the potential for breaking code, since now
the buffers are locked while the function runs (i.e. callbacks couldn't 
expand the array anymore); it is likely that such breakage would uncover
a bug in the locking assumptions of the existing code.

Your attempt to fix your proposal indicates one of the strengths of the PEP
process: most initial proposals have flaws, and, unless completely spelled
out, readers will typically miss the flaws on initial reading. The weakness,
of course, is that a PEP might be sitting around for a long time, and nobody
would be reviewing it.

> Incidentally,  does PEP-298 solve this now?

Depends on what "this" is. Existing code will continue to work as before:
No locking in ParseTuple happens, as a result, the arguments are unlocked
when the function returns; this is desirable. Since the buffers aren't 
locked, relocation may happen. If such relocation is ever observed, people
would have to replace the "s" parser with an "O" parser, and perform the
locking themselves.

So, yes, PEP-298 addresses this - although modification of existing code is
necessary to make use of this feature (as is in your fixed proposal).

> Well,  this does sound bad,  but I was assuming the core and library 
> would be "fixed" at the first introduction of locking, and the only 
> problem would be with non-conforming extensions.    I now think that 
> "lock violations" might be better handled using the warning framework, 
> defaulting to a one time warning.  

I think we should strive for a solution that keeps the function parameters
locked while the function runs, and unlocks them automatically at function 
return time.

This shows another flaw in the wording of PEP 298: It implies without saying
that the expectation is that you should have as many release calls as you
have lock calls. 

Regards,
Martin