[Patches] [ python-Patches-594001 ] PEP 277: Unicode file name support
noreply@sourceforge.net
noreply@sourceforge.net
Thu, 03 Oct 2002 00:00:55 -0700
Patches item #594001, was opened at 2002-08-12 21:33
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=594001&group_id=5470
Category: Windows
Group: None
>Status: Open
Resolution: Fixed
Priority: 5
Submitted By: Martin v. Löwis (loewis)
Assigned to: Mark Hammond (mhammond)
Summary: PEP 277: Unicode file name support
Initial Comment:
This patch is in an updated version of the patch [1]
mentioned in the PEP. In addition to merging it with
the CVS, it fixes a few formatting problems.
----------------------------------------------------------------------
>Comment By: Mark Hammond (mhammond)
Date: 2002-10-03 17:00
Message:
Logged In: YES
user_id=14198
Oops - I missed checking in Lib/test/test_pep277.py.
With this file in place, the test suite fails for me:
test_pep277
test test_pep277 produced unexpected output:
...
However, I seem to recall that new tests do not need output
files. Should I use the old technique of "regrtest -g ..."
and check both the test and output file in, or is there
something else I should be doing?
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-10-03 15:15
Message:
Logged In: YES
user_id=14198
/cvsroot/python/python/dist/src/Include/pyerrors.h,v <--
pyerrors.h
new revision: 2.61; previous revision: 2.60
/cvsroot/python/python/dist/src/Lib/test/test_unicode_file.py,v
<-- test_unicode_file.py
new revision: 1.6; previous revision: 1.5
/cvsroot/python/python/dist/src/Modules/posixmodule.c,v <--
posixmodule.c
new revision: 2.261; previous revision: 2.260
/cvsroot/python/python/dist/src/Objects/fileobject.c,v <--
fileobject.c
new revision: 2.169; previous revision: 2.168
/cvsroot/python/python/dist/src/PC/pyconfig.h,v <-- pyconfig.h
new revision: 1.15; previous revision: 1.14
/cvsroot/python/python/dist/src/Python/errors.c,v <-- errors.c
new revision: 2.72; previous revision: 2.71
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2002-10-02 03:41
Message:
Logged In: YES
user_id=21627
The patch passes the test suite on Solaris. I've also
attached a documentation change, so I think the patch is
ready now. Back to Mark for checkin.
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-10-01 19:59
Message:
Logged In: YES
user_id=14198
The block:
+ if (!wpath1 || !wpath2)
+ return NULL;
Should include Py_XDECREF calls for wpath1 and wpath2.
Can't be bothered with a new patch just for this :)
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-10-01 09:48
Message:
Logged In: YES
user_id=14198
Although SF implies otherwise, a bad filename in a
submission appears to only prevent the file being uploaded -
the rest of the comments etc come through. Hence my
double-comments and strange status changes you can see.
So, back to Martin as I don't think he is waiting on me for
anything. Happy to check in if you like, but would like to
see someone run a debug build of this through Linux first to
see if any asserts fire - some asserts did indeed fire in
the first version of this patch, and Linux may trigger
others, or even the 2 that I added.
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 22:54
Message:
Logged In: YES
user_id=14198
Sorry - not sure what I was thinking - obviously can't check
in a patch known to crash the test suite!
Attaching a new patch - fixes the problem I mentioned with
test_unicode_files.py, fixes an error I introduced where the
"WithFilename" error functions could be passed a NULL
filename, and adding an assert to match the one I added
previously.
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 22:52
Message:
Logged In: YES
user_id=14198
Sorry - not sure what I was thinking - obviously can't check
in a patch known to crash the test suite!
Attaching a new patch. This fixes the crash in
test_unicode_file I mentioned. It also fixes another error
I introduced where PyErr_SetWithFilename functions can have
NULL filenames. Also added an assert for symmetry with one
I added before.
Martin - back to me if you want me to check it in.
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 22:21
Message:
Logged In: YES
user_id=14198
I missed the fact this was re-assigned to me - should I
check my new patch in?
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 22:20
Message:
Logged In: YES
user_id=14198
I get an exception running test_unicode_file.py - we end up
with a MBCS encoded string passed to
PyUnicode_FromEncodedObject(), that attempts to decode using
the system default encoding rather than the file-system
default. The error is a NULL pointer deref so should be
obvious - unfortunately the fix requires a local
reimplementation of PyUnicode_FromEncodedObject(), passing
an explicit encoding for string objects.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2002-09-30 20:43
Message:
Logged In: YES
user_id=21627
Thanks for your comments, just shouting at me to correct all
these problems would have been fine :-)
In what case do you still get an exception?
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 16:35
Message:
Logged In: YES
user_id=14198
Back to martin for comments on my comments ;)
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 16:34
Message:
Logged In: YES
user_id=14198
Back to Martin for comments on my comments ;)
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 16:33
Message:
Logged In: YES
user_id=14198
Uploading text file with my comments on the patch. Also
uploading a new patch with these corrections and even better
(?) error handling.
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2002-09-30 11:03
Message:
Logged In: YES
user_id=14198
Will review this instant - sorry for the delay.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2002-09-26 16:24
Message:
Logged In: YES
user_id=21627
Mark, can you please indicate whether you will review this
patch?
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2002-09-26 05:43
Message:
Logged In: YES
user_id=31435
Sorry, Martin, bouncing back to Mark -- I really don't know
anything about the Windows filename APIs.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2002-09-23 01:31
Message:
Logged In: YES
user_id=21627
Tim, can you review this?
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2002-09-04 22:06
Message:
Logged In: YES
user_id=21627
This is a new version:
- wrapped all Windows wide API code with #ifdef
Py_WIN_WIDE_FILENAMES
- added test case
Mark, can you please take a look at this?
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-12 23:38
Message:
Logged In: YES
user_id=6380
I'd make unicode_filenames() a macro that expands to 0 on
platforms without this wart. I'd also test for wfunc!=NULL
before calling unicode_filenames().
There's a lot of hairy code here. Are you sure that there
are test cases in the test suite that exercise all of it?
Aren't there some #ifdefs missing? posix_[12]str have code
that's only relevant for Windows but isn't #ifdef'ed out
like it is elsewhere.
There should probably be a separate #define in pyport.h to
test for this that's equivalent to defined(MS_WINDOWS) &&
!defined(Py_UNICODE_WIDE), so this can be uniformly tested
to see whether this code is necessary.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=594001&group_id=5470