[Python-Dev] Unicode strings as filenames

[Neil Hodgson]

   Martin:

> Looks good. The posix_do_stat changes contain an error; you have put
> Python API calls inside the BEGIN_ALLOW_THREADS block. That is wrong:
> you must always hold the interpreter lock when calling Python
> API.

   OK, moved the thread stuff so no API calls are inside. However,
PyUnicode_AS_UNICODE left in as that is just a macro for accessing a field
that should be stable over the call. Or is it? Other methods don't seem to
worry that GC will move buffers, during calls.

> Also, when calling _wstati64, you might want to assert that the
> function pointer is _stati64. Likewise, the code inside posix_open
> should hold the interpreter lock.

   OK, assert in for stat.

> However, the size of your changes is really disturbing here. There
> used to be already four versions of listing a directory; now you've
> added a fifth one. And it isn't even clear whether this code works on
> W9x, is it?

   Currently it won't work on Windows 9x. That is more work and code bulk.

> There must be a way to fold the different Windows versions into a
> single one; perhaps it is acceptable to drop Win16 support. I think
> three different versions should be offered to the end user:

   Windows does this with the preprocessor - you are either building a
Unicode version or an ANSI version.

> - path is plain string, result is list of plain strings
> - path is Unicode string, result is list of Unicode strings
> - path is Unicode string, result is list of plain strings
>
> Perhaps one could argue that the third version isn't really needed:

   Sounds good to me. I'm moving back towards not using the 'utf-8' system
encoding but rather checking of Unicode arguments and handling them
explicitly even at the cost of code expansion.

> Now, os.rename will fail if you pass two Unicode strings
> referring to non-ASCII file names. posix_1str and posix_2str are like
> the stat implementation, except that you cannot know statically what
> the function pointer is.

   The code now passes both narrow and wide functions to posix_nstr and
there are two null functions to make this compile on non-Windows. Added
mkdir to allow testing the chdir and rmdir functions.

   Now handled are open, os.open, os.stat. os.listdir, os.rename, os.remove,
os.mkdir, os.chdir, os.rmdir.

   Updated files available from
http://scintilla.sourceforge.net/winunichanges.zip

   Neil