[issue12730] Python's casemapping functions are untrustworthy due to narrow/wide build issues
Terry J. Reedy
report at bugs.python.org
Sat Aug 13 00:07:32 CEST 2011
Terry J. Reedy <tjreedy at udel.edu> added the comment:
I agree that better masking of narrow-wide build difference would be good as long as it does not severely impact normal performance. Revision of the test file (see below) shows that the 'bug' is that the .upper, .lower, and .title methods leaves the tested non-BMP chars unchanged on narrow builds.
I am not sure if this is true of all upper-plane chars and whether this is by design or simply a matter of not catching up to an ever-expanding database. Hence, I am also not sure whether this is a bug report or feature request.
I made several changes in casemap.python so I could run it and get better information:
* Rename to casemap.py. Many of us use software that recognizes and special-cases the standard .py extension. All python code files uploaded should use this.
* Remove the unused 3-rd party regex import which stops the test for most people.
* Remove the unnecessary PYTHONIOENCODING exit which stop the test on Windows and possibly elsewhere. The file seems to run fine without it.
* Rewrite the test data using \Uxxxxxxxx (8 hex chars) escapes for the non-BMP chars. That will be required for new tests for test_unicode.py. (I believe the test suite avoid literal non-ascii chars unless really necessary.) Besides which, all I see (on Windowsj) in Firefox is things like
"ð¼ð¯ð‘…ð¨ð‘‰ð¯ð»". IDLE just has empty boxes.
* Factor the tests so the output is easier to rewrite.
* Rewrite the test output to make comparisons easier. Writing the 'wrong' answer first, directly under the original, made it easy to see that the 'wrong' answer *is* the original, unchanged.
The revised version (to be uploaded separately) has the same 6 failures.
nosy: +haypo, lemburg, loewis, terry.reedy
stage: -> needs patch
versions: +Python 3.2, Python 3.3
Python tracker <report at bugs.python.org>
More information about the Python-bugs-list