[Python-Dev] Choosing a best practice solution for Python/extension modules

[James Pye]

On Feb 21, 2009, at 2:17 PM, Brett Cannon wrote:
> The other approach is having pickle contain code known not to be  
> overridden by anyone, import _pypickle for stuff that may be  
> overridden, and then import _pickle for whatever is available. This  
> approach has the perk of using a standard practice for how to pull  
> in different implementation. But the drawback, thanks to how globals  
> are bound, is that any code pulled in from _pickle/_pypickle will  
> not be able to call into other optimized code; it's a take or leave  
> it once the call chain enters one of those modules as they will  
> always call the implementations in the module they originate from.


fwiw,

I believe this is the approach that I've been using when I'm faced  
with the need to optimize code, but still want to retain a pure-Python  
version. Thankfully, I haven't had a need for "implementation  
intersections"(well, it almost works. I think. ;) for access to common  
module globals as the optimizations tend to be simple transformations  
or isolated classes. However, if I were faced with the problem of  
needing some common global data/functionality, I'd probably put it in  
yet-another-module and just import it explicitly in each  
implementation. Sure, it seems like it might be annoying, but so is  
maintaining multiple implementations. ;)

Specifically:

pbuffer.py - The python implementation
buffer.c -> cbuffer.so - The c implementation
buffer.py - The "abstraction module", trying to import the contents of  
the fast one first.

And in my unittest:

if c_buffer_module is not None:
   class c_buffer(buffer_test, unittest.TestCase):
     bufferclass = c_buffer_module.pq_message_stream

class p_buffer(buffer_test, unittest.TestCase):
   bufferclass = p_buffer_module.pq_message_stream

Of course, "buffer_test" is not invoked because it's not a TestCase.


However, Aahz is probably right about this thread belonging elsewhere.?
Hrm, day old, maybe it's been moved already.. sigh. :)