[Python-Dev] unicode imports

[Nick Coghlan]

Phillip J. Eby wrote:
> Actually, you would want to put it in sys.path_hooks, and then instances 
> would be placed in path_importer_cache automatically.  If you are adding 
> it to the path_hooks after the fact, you should simply clear the 
> path_importer_cache.  Simply poking stuff into the path_importer_cache 
> is not a recommended approach.

Oh, I agree - poking it in directly was a desperation measure if the 
path_hooks machinery didn't like Unicode either.

I've since gone and looked, and you may be screwed either way - the standard 
import paths appear to be always put on the system path as encoded 8-bit 
strings, not as Unicode objects.

That said, it also appears that the existing machinery *should* be able to 
handle non-ASCII path items, so long as 'Py_FileSystemDefaultEncoding' is set 
correctly. If it isn't handling it, then there's something else going wrong.

Modules/getpath.c and friends don't encode the results returned by the 
platform APIs, so the strings in

Kristján, can you provide more details on the fault you get when trying to 
import from the path containing the Chinese characters? Specifically:

What is the actual file system path?
What do sys.prefix, sys.exec_prefix and sys.path contain?
What does sys.getdefaultencoding() return?
What do sys.stdin.encoding, sys.stdout.encoding and sys.stderr.encoding say?
What does "python -v" show?
Does adding the standard lib directories manually to sys.path make any difference?
Does setting PYTHONHOME to the appropriate settings make any difference?

Running something like the following would be good:

   import sys
   print "Prefixes:", sys.prefix, sys.exec_prefixes
   print "Path:", sys.path
   print "Default encoding:", sys.getdefaultencoding()
   print "Input encoding:", sys.stdin.encoding,
   print "Output encodings:", sys.stdout.encoding, sys.stderr.encoding
   try:
       import string # Make python -v do something interesting
   except ImportError:
       print "Could not find string module"
   sys.path.append(u"stdlib directory name")
   try:
       import string # Make python -v do something interesting
   except ImportError:
       print "Could not find string module"






-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org