[Python-Dev] Import and unicode: part two

[Toshio Kuratomi]

On Wed, Jan 19, 2011 at 04:40:24PM -0500, Terry Reedy wrote:
> On 1/19/2011 4:05 PM, Simon Cross wrote:
> 
> >I have no problem with non-ASCII module identifiers being valid
> >syntax. It's a question of whether attempting to translate a non-ASCII
> 
> If the names are the same, ie, produced with the same sequence of
> keystrokes in the save-as box and importing box, then there is no
> translation, at least from the user's view.
> 
> >module name into a file name (so the file can be imported) is a good
> >idea and whether these sorts of files can be safely transferred among
> >diverse filesystems.
> 
> I believe we now have the situation that a package that works on *nix
> could fail on Windows, whereas I believe that patch would *improve*
> portability.
> 
I'm not so sure about this....  You may have something that works on Windows
and on *NIX under certain circumstances but it seems likely to fail when
moving files between them (for instance, as packages downloaded from pypi).
Additionally, many unix filesystem don't specify a filesystem encoding for
filenames; they deal in legal and illegal bytes which could lead to
troubles.  This problem of which encoding to use is a problem that can be
seen on UNIX systems even now.  Try this:

  echo 'print("hi")' > café.py
  convmv -f utf-8 -t latin1 café.py
  python3 -c 'import café'

ASCII seems very sensible to me when faced with these ambiguities.

Other options I can brainstorm that could be explored:

* Specify an encoding per platform and stick to that.  (So, for instance,
  all module names on posix platforms would have to be utf-8).  Force
  translation between encoding when installing packages (But that doesn't
  help for people that are creating their modules using their own build
  scripts rather than distutils, copying the files using raw tar, etc.)
* Change import semantics to allow specifying the encoding of the module on
  the filesystem (seems really icky).

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110119/a6bcc294/attachment.pgp>