[Python-Dev] Extension modules, Threading, and the GIL

[Tim Peters]

[Martin v. Lowis]
> That covers the case that proper calls are possible from this thread
> (although there are still ambiguities: a single that may have multiple
> thread states, which are associated to multiple interpreters, which may
> have diffent contents of sys.modules)
>
> However, that answer does not work for the case for a thread that has
> never seen a Python call: proper calls could not run on this thread.
>
>> Then I expect the strongest that can be said is that the output you get
>> corresponds to the actual state of sys.modules at some time T
>
> The issue is that there may be multiple sys.modules in the process, at
> T. Then the question is whether to use one of the existing ones (if so:
> which one), or create a new one.

All right, you're worried about multiple interpreter states.  I'm not --
I've never used them, the Python distribution never uses them, there are no
tests of that feature, and they don't seem particularly well-defined in end
cases regardless.  I'm happy to leave them as an afterthought.  If someone
wants to champion them, they better ensure their interests are met.  As far
as I'm concerned, if a user does the all-purpose prologue dance (the "I
don't know anything, but I want to use the Python API anyway" one), then the
interpreter state in effect isn't defined.  It may use an existing
interpreter state, or an interpreter state created solely for use by this
call, or take one out of a pool of interpreter states reused for such cases,
or whatever.  Regardless, it's a little tail that I don't want wagging the
dog here.

>> I don't expect we can say anything stronger than that today either.

> Currently, the extension author must somehow come up with an interpreter
> state.  I'm uncertain whether you are proposing to leave it at that, or
> whether you require that any solution to "the problem" also provides a
> way to obtain the "right" interpreter state somehow.

I define "right" as "undefined" in this case.  Someone who cares about
multiple interpreter states should feel free to define and propose a
stronger requirement.  However, the people pushing for change here have
explicitly disavowed interest in multiple interpreter states, and I'm happy
to press on leaving them for afterthoughts.