[Patches] [ python-Patches-594001 ] PEP 277: Unicode file name support

noreply@sourceforge.net noreply@sourceforge.net
Mon, 30 Sep 2002 16:48:34 -0700


Patches item #594001, was opened at 2002-08-12 21:33
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=594001&group_id=5470

Category: Windows
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Martin v. Löwis (loewis)
>Assigned to: Martin v. Löwis (loewis)
Summary: PEP 277: Unicode file name support

Initial Comment:
This patch is in an updated version of the patch [1]
mentioned in the PEP. In addition to merging it with
the CVS, it fixes a few formatting problems.

----------------------------------------------------------------------

>Comment By: Mark Hammond (mhammond)
Date: 2002-10-01 09:48

Message:
Logged In: YES 
user_id=14198

Although SF implies otherwise, a bad filename in a
submission appears to only prevent the file being uploaded -
the rest of the comments etc come through.  Hence my
double-comments and strange status changes you can see.

So, back to Martin as I don't think he is waiting on me for
anything.  Happy to check in if you like, but would like to
see someone run a debug build of this through Linux first to
see if any asserts fire - some asserts did indeed fire in
the first version of this patch, and Linux may trigger
others, or even the 2 that I added.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 22:54

Message:
Logged In: YES 
user_id=14198

Sorry - not sure what I was thinking - obviously can't check
in a patch known to crash the test suite!

Attaching a new patch - fixes the problem I mentioned with
test_unicode_files.py, fixes an error I introduced where the
"WithFilename" error functions could be passed a NULL
filename, and adding an assert to match the one I added
previously.


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 22:52

Message:
Logged In: YES 
user_id=14198

Sorry - not sure what I was thinking - obviously can't check
in a patch known to crash the test suite!

Attaching a new patch.  This fixes the crash in
test_unicode_file I mentioned.  It also fixes another error
I introduced where PyErr_SetWithFilename functions can have
NULL filenames.  Also added an assert for symmetry with one
I added before.

Martin - back to me if you want me to check it in.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 22:21

Message:
Logged In: YES 
user_id=14198

I missed the fact this was re-assigned to me - should I
check my new patch in?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 22:20

Message:
Logged In: YES 
user_id=14198

I get an exception running test_unicode_file.py - we end up
with a MBCS encoded string passed to
PyUnicode_FromEncodedObject(), that attempts to decode using
the system default encoding rather than the file-system
default.  The error is a NULL pointer deref so should be
obvious - unfortunately the fix requires a local
reimplementation of PyUnicode_FromEncodedObject(), passing
an explicit encoding for string objects.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-09-30 20:43

Message:
Logged In: YES 
user_id=21627

Thanks for your comments, just shouting at me to correct all
these problems would have been fine :-)

In what case do you still get an exception? 

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 16:35

Message:
Logged In: YES 
user_id=14198

Back to martin for comments on my comments ;)

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 16:34

Message:
Logged In: YES 
user_id=14198

Back to Martin for comments on my comments ;)

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 16:33

Message:
Logged In: YES 
user_id=14198

Uploading text file with my comments on the patch.  Also
uploading a new patch with these corrections and even better
(?) error handling.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 11:03

Message:
Logged In: YES 
user_id=14198

Will review this instant - sorry for the delay.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-09-26 16:24

Message:
Logged In: YES 
user_id=21627

Mark, can you please indicate whether you will review this
patch?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-09-26 05:43

Message:
Logged In: YES 
user_id=31435

Sorry, Martin, bouncing back to Mark -- I really don't know 
anything about the Windows filename APIs.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-09-23 01:31

Message:
Logged In: YES 
user_id=21627

Tim, can you review this?

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-09-04 22:06

Message:
Logged In: YES 
user_id=21627

This is a new version:
- wrapped all Windows wide API code with #ifdef 
Py_WIN_WIDE_FILENAMES
- added test case

Mark, can you please take a look at this?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-12 23:38

Message:
Logged In: YES 
user_id=6380

I'd make unicode_filenames() a macro that expands to 0 on
platforms without this wart. I'd also test for wfunc!=NULL
before calling unicode_filenames().

There's a lot of hairy code here. Are you sure that there
are test cases in the test suite that exercise all of it?

Aren't there some #ifdefs missing? posix_[12]str have code
that's only relevant for Windows but isn't #ifdef'ed out
like it is elsewhere.

There should probably be a separate #define in pyport.h to
test for this that's equivalent to defined(MS_WINDOWS) &&
!defined(Py_UNICODE_WIDE), so this can be uniformly tested
to see whether this code is necessary.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=594001&group_id=5470