[Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ...

[M.-A. Lemburg]

"Martin v. Loewis" wrote:
> 
> > The restriction when compiling Python in wide mode on Windows
> > should be lifted: The PyUnicode_AsWideChar() API should be used
> > to convert 4-byte Unicode to wchar_t (which is 2-byte on Windows).
> 
> While I agree that this restriction ought to be removed eventually, I
> doubt that Python will be usable on Windows with a four-byte Unicode
> type in any foreseeable future.

Perhaps Neil ought to copy your notes to the PEP, so that we
don't forget about this issue.
 
> Just have a look at unicodeobject.c:PyUnicode_DecodeMBCS; it makes the
> assumption that a Py_UNICODE* is the same thing as a WCHAR*. That
> means that the "mbcs" encoding goes away on Windows if
> HAVE_USABLE_WCHAR_T does not hold anymore.
> 
> Also, I believe most of PythonWin also assumes HAVE_USABLE_WCHAR_T
> (didn't check, though).
> 
> > Why is "unicodefilenames" a function and not a constant ?
> 
> In the Windows binary, you need a run-time check to see whether this
> is DOS/W9x, or NT/W2k/XP; on DOS, the Unicode API is not available
> (you still can pass Unicode file names to open and listdir, but they
> will get converted through the MBCS encoding). So it clearly is not a
> compile time constant.

I see.
 
> I'm still not certain what the meaning of this function is, if it
> means "Unicode file names are only restricted by the file system
> conventions", then on Unix, it may change at run-time, if a user or
> the application sets an UTF-8 locale, switching from the original "C"
> locale.

Doesn't it mean: "posix functions and file() can accept Unicode file 
names" ? 

That's what I thought, at least; whether they succeed or not 
is another question and could well be handled by run-time errors
(e.g. on Unix it is not at all clear whether NFS, Samba or some
other more exotic file system can handle the encoding chosen by 
Python or the program).

Perhaps we ought to drop that function altogether and let the
various file IO functions raise run-time errors instead ?!

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/