[Python-bugs-list] [ python-Bugs-663074 ] codec registry and Python embedding problem

[SourceForge.net]

Bugs item #663074, was opened at 2003-01-06 12:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=663074&group_id=5470

Category: Python Interpreter Core
Group: None
>Status: Closed
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Gustavo Niemeyer (niemeyer)
Summary: codec registry and Python embedding problem

Initial Comment:
Found by Hartmut Ring:

#include <Python.h>
int main() {
  for (int i=0; i<2; i++) {
    Py_Initialize();
    PyRun_SimpleString("a = u'\xe4'.encode('Latin-1')");
    Py_Finalize();
  }
  return 0;
}

First try runs fine, second try gives:

Traceback (most recent call last):
  File "<string>", line 1, in ?
LookupError: no codec search functions registered:
can't find encoding



----------------------------------------------------------------------

>Comment By: Gustavo Niemeyer (niemeyer)
Date: 2003-03-19 00:50

Message:
Logged In: YES 
user_id=7887

Commited as:

Include/pystate.h: 2.23
Misc/NEWS: 1.699
Python/codecs.c: 2.21
Python/pystate.c: 2.24
Python/pythonrun.c: 2.182

Thanks for your support!


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-03-07 17:36

Message:
Logged In: YES 
user_id=38388

Oh, I see what you mean. In that case, it's OK to add 
them just before the optional parts.


----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2003-03-07 17:20

Message:
Logged In: YES 
user_id=7887

No problems.

If you don't mind, I'd just to have a confirmation: are you
sure I should move the new fields to the end, and leave a
conditional member in the middle?


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-03-04 10:07

Message:
Logged In: YES 
user_id=38388

Same comments as Guido, plus:

* in the interpreter state struct you should add the new 
  fields to the end, not in the middle

* the finalization phase is moved to a later point in
  the process; this could cause problems when finalizing
  codecs -- I guess we'll just have to see whether we
  get bug reports related to this; perhaps it's better
  to ZAP the dicts before ZAPping builtins etc. ?!

Great work, Gustavo. Please check the patch in with 
the above changes.

Thanks.




----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-03-01 14:41

Message:
Logged In: YES 
user_id=6380

I can't review this beyond "it doesn't break the tests for
me"; I suggest to assign to MAL for review. One note: I got
a compiler warning

  implicit declaration of function `_PyCodecRegistry_Init'

to fix, add

  static int _PyCodecRegistry_Init(void); /* Forward */

somewhere to the top of codecs.c.

----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2003-03-01 08:07

Message:
Logged In: YES 
user_id=7887

Here is a proposed solution.

It has been tested with the standard regression tests, and
in an environment with multiple interpreters.


----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2003-02-12 13:13

Message:
Logged In: YES 
user_id=7887

As discussed in python-dev, I'll work on that.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-01-22 12:12

Message:
Logged In: YES 
user_id=38388

Good suggestion. Do you have time to cook up a patch ?

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-01-22 10:07

Message:
Logged In: YES 
user_id=21627

I think the various global variables maintained in codecs.c
(_PyCodec_SearchCache, import_encodings_called, ...) need to
become part of the interpreter state. Otherwise, a codec
registered in one interpreter will be found in another.

Also, it appears that import_encodings_called can be
eliminated. Instead, a NULL value of _PyCodec_SearchPath
could be used to indicate whether initialization has taken
place. 

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-01-06 12:18

Message:
Logged In: YES 
user_id=38388

This is due to a flag used in codecs.c:

/* Flag used for lazy import of the standard encodings
package */
static int import_encodings_called = 0;

The solution is to reset this flag in Py_Finalize().

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=663074&group_id=5470