[Python-Dev] PEP-298: buffer interface widening too much?

[Todd Miller]

Martin v. Löwis wrote:

>Todd Miller <jmiller@stsci.edu> writes:
>
>  
>
>>I think that kind of backwards compatability just paves the wave for
>>extensions to stay broken and complicates the object structures as
>>well. I guess PEP-298 does create the possibility for creating correct
>>extensions,  I just hate the thought of 5 getpointer calls where there
>>are now 3 and should be 2.
>>    
>>
>
>I think this is the core of your objection. However, notice that all
>
You're right I think,  but I also percieve the complexity of extension 
writing going up as well.   Perhaps it's a false perception which can be 
dispelled by documentation stating that new extensions should *always* 
use the locking interface,  with the old interface preserved for the 
sole purpose of backwards compatability.  Without some thought,  the 
following was not clear:  the need for locking is a property of specific 
types,  not specific extension functions.  Therefore the existence of 
two get pointer varieties implies an ambguity which is not really there: 
 should I use the locking or non-locking variety?  I think the answer is 
always:  use the locking variety.

>types implementing have to be modified in either proposal, and that
>adding the two getpointer functions should be trivial in most cases:
>- if the type has read/write memory, implement them both the same
>  way;
>- if the type has read-only memory, set the write pointer to NULL
>- make the getpointer function a wrapper around the existing
>  getpointer, requesting only a single segment and simultaneously
>  locking the buffer.
>
>The overhead implementation-wise is really minimal; all the hard work
>will come from the locking code - see Thomas' patch to the string
>  
>
Thanks for the pointer.

>object for an example. We could even provide a Generic function which
>delegates the new get functions to the existing ones, accessing
>segment zero.
>  
>
Wouldn't this require more information about the lock implementation? 
 As PEP-298 stands,  I thought that was left up to the buffered type. 
 It might make sense to change the locking getpointer fields into lock, 
and release into unlock.  Then compose the primitive operations in the 
wrapper as you suggested.   That wouldn't fix the size_t problem though.

>  
>
>>Sounds good to me.  I hope you implement it!  If PEP-286 were here
>>now, a simplified PEP-298 would be possible. 
>>    
>>
>
>I now recall what the issue is: you can't assume that modifying the
>argument tuple object to record resources is feasible, as this breaks
>apply (where in every case the user allocates a tuple, and where this
>is a plain old tuple).
>  
>
Is the efficiency of apply() so critical that a conversion is ruled out?

>So I'm now envisioning a strategy where a thread-local
>(i.e. thread-state associated) stack is maintained - on argument tuple
>allocation, a frame is put on this stack, and all resources go into
>that frame, on tuple decrefing, the frame is cleared. There would be
>three kinds of resources: objects, void*, and locked buffers. For
>efficiency, this stack and its frames would be a flat memory block.
>  
>
Way over my head here...  I guess the point is that arg tuples aren't 
the only point of attack.

>In any case, I'm still -0 on PEP 298 as it stands: it should deal with
>much more aspects of buffers (and I now consider argument parsing to
>be one of them), when those things are added, I'll support the PEP -
>your objection that this adds too much API is of no concern to me.
>  
>
Well,  thanks for discussing it anyway.   I had a feeling this 
conversation would end this way before I ever opened my mouth,  but I 
wanted to change the balance between saying something and saying *nothing*.

>Regards,
>Martin
>
Regards,
Todd

>  
>