[Python-Dev] Choosing a best practice solution for Python/extension modules

[Brett Cannon]

On Sun, Feb 22, 2009 at 22:41, Aahz <aahz at pythoncraft.com> wrote:

> On Sun, Feb 22, 2009, Brett Cannon wrote:
> > On Sat, Feb 21, 2009 at 20:12, Aahz <aahz at pythoncraft.com> wrote:
> >> On Sat, Feb 21, 2009, Brett Cannon wrote:
> >>> On Sat, Feb 21, 2009 at 15:46, Aahz <aahz at pythoncraft.com> wrote:
> >>>> On Sat, Feb 21, 2009, Brett Cannon wrote:
> >>>>>
> >>>>> I am seeing two approaches emerging. One is where pickle contains all
> >>>>> Python code and then uses something like use_extension to make sure
> >>>>> the original Python objects are still reachable at some point. This
> >>>>> has the drawback that you have to use some function to make the
> >>>>> extensions happen and there is some extra object storage.
> >>>>>
> >>>>> The other approach is having pickle contain code known not to
> >>>>> be overridden by anyone, import _pypickle for stuff that may be
> >>>>> overridden, and then import _pickle for whatever is available. This
> >>>>> approach has the perk of using a standard practice for how to pull in
> >>>>> different implementation. But the drawback, thanks to how globals are
> >>>>> bound, is that any code pulled in from _pickle/_pypickle will not be
> >>>>> able to call into other optimized code; it's a take or leave it once
> >>>>> the call chain enters one of those modules as they will always call
> >>>>> the implementations in the module they originate from.
> >>>>
> >>>> To what extent do we care about being able to select Python-only on a
> >>>> per-module basis, particularly in the face of threaded imports?  That
> >> is,
> >>>> we could have a sys.python_only attribute that gets checked on import.
> >>>> That's simple and direct, and even allows per-module switching if the
> >>>> application really cares and import doesn't need to worry about
> threads.
> >>>>
> >>>> Alternatively, sys.python_only could be a set, but that gets ugly
> about
> >>>> setting from the application.  (The module checks to see whether it's
> >>>> listed in sys.python_only.)
> >>>>
> >>>> Maybe we should move this discussion to python-ideas for now to kick
> >>>> around really oddball suggestions?
> >>>
> >>> This is all about testing. If a change is made to some extension code
> it
> >>> should be mirrored in the Python code and vice-versa.
> >>
> >> Okay, I don't see how that is a response to my suggestion -- I can
> >> imagine that someone might want to test a combination of pure-Python and
> >> binary libraries.
> >
> > I don't want to move it because this isn't some idea for a new feature
> that
> > may or may not be useful; this isn't an "idea", it's needed.
>
> That's fine, but what about my idea of using sys.python_only?


But what is it supposed to represent? That only pure Python modules get
imported? What if the module depends on another module that is an extension
module?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090223/b6637706/attachment-0001.htm>