[Python-Dev] Codecs data

[Guido van Rossum]

> Gustavo Niemeyer wrote:
> > I belive we have a problem in our current codecs implementation: it
> > doesn't work for multiple interpreter states, because the data is
> > currently statically allocated in codecs.c. That way, only the first
> > interpreter is going to have the codecs lookup working. The following
> > interpreters will try to cross the interpreter state boundary and use
> > the lookup function from the first interpreter, issuing all kinds of
> > weird errors.
> > 
> > One solution would be to store that data in the "sys" module
> > (e.g. codec_search_path, codec_search_cache, codec_inited).
> > 
> > Can you please advice on how to fix that? I'll be willing to provide
> > a fix with the proposed solution.

[MAL]
> Martin had the idea to put these globals (along with other
> similar globals) in the thread state structure. I believe that's
> the right approach or at least hints in the right direction (we
> don't want the globals to be per thread, but there doesn't
> seem to be a better place).

The thread state seems just wrong, if threads can share codecs.  I'm
not sure what the purpose of these globals is, but why can't we
augment the interpreter state to hold them?

> Also see:
> https://sourceforge.net/tracker/?func=detail&aid=663074&group_id=5470&atid=105470

[MAL again]
> Hmm, there is something called PyInterpreterState. Perhaps that's
> what we want ?! (and Martin was thinking about)

Yes!

--Guido van Rossum (home page: http://www.python.org/~guido/)