[Python-Dev] Choosing a best practice solution for Python/extension modules

[Aahz]

On Sun, Feb 22, 2009, Brett Cannon wrote:
> On Sat, Feb 21, 2009 at 20:12, Aahz <aahz at pythoncraft.com> wrote:
>> On Sat, Feb 21, 2009, Brett Cannon wrote:
>>> On Sat, Feb 21, 2009 at 15:46, Aahz <aahz at pythoncraft.com> wrote:
>>>> On Sat, Feb 21, 2009, Brett Cannon wrote:
>>>>>
>>>>> I am seeing two approaches emerging. One is where pickle contains all
>>>>> Python code and then uses something like use_extension to make sure
>>>>> the original Python objects are still reachable at some point. This
>>>>> has the drawback that you have to use some function to make the
>>>>> extensions happen and there is some extra object storage.
>>>>>
>>>>> The other approach is having pickle contain code known not to
>>>>> be overridden by anyone, import _pypickle for stuff that may be
>>>>> overridden, and then import _pickle for whatever is available. This
>>>>> approach has the perk of using a standard practice for how to pull in
>>>>> different implementation. But the drawback, thanks to how globals are
>>>>> bound, is that any code pulled in from _pickle/_pypickle will not be
>>>>> able to call into other optimized code; it's a take or leave it once
>>>>> the call chain enters one of those modules as they will always call
>>>>> the implementations in the module they originate from.
>>>>
>>>> To what extent do we care about being able to select Python-only on a
>>>> per-module basis, particularly in the face of threaded imports?  That
>> is,
>>>> we could have a sys.python_only attribute that gets checked on import.
>>>> That's simple and direct, and even allows per-module switching if the
>>>> application really cares and import doesn't need to worry about threads.
>>>>
>>>> Alternatively, sys.python_only could be a set, but that gets ugly about
>>>> setting from the application.  (The module checks to see whether it's
>>>> listed in sys.python_only.)
>>>>
>>>> Maybe we should move this discussion to python-ideas for now to kick
>>>> around really oddball suggestions?
>>>
>>> This is all about testing. If a change is made to some extension code it
>>> should be mirrored in the Python code and vice-versa.
>>
>> Okay, I don't see how that is a response to my suggestion -- I can
>> imagine that someone might want to test a combination of pure-Python and
>> binary libraries.
> 
> I don't want to move it because this isn't some idea for a new feature that
> may or may not be useful; this isn't an "idea", it's needed.

That's fine, but what about my idea of using sys.python_only?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.