[ python-Bugs-969415 ] CJK codecs list incomplete
SourceForge.net
noreply at sourceforge.net
Sat Jun 12 08:04:41 EDT 2004
Bugs item #969415, was opened at 2004-06-09 08:54
Message generated for change (Settings changed) made by lemburg
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=969415&group_id=5470
Category: Documentation
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Mike Brown (mike_j_brown)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: CJK codecs list incomplete
Initial Comment:
http://www.python.org/dev/doc/devel/whatsnew/node7.
html states that various CJK encodings have been
added, but the list given there does not match the list
on
http://www.python.org/dev/doc/devel/lib/node128.html.
In particular, missing from the latter list are all of the
aliases with hyphens:
shift-jis, shift-jisx0213, euc-jp, euc-jisx0213, iso-2022-
jp, iso-2022-jp-1, iso-2022-jp-2, iso-2022-jp-3, iso-
2022-jp-ext, euc-kr, iso-2022-kr
Since I successfully ran codecs.lookup() tests on a few
of the hyphenated aliases, I assume that the omission
of the hyphenated versions in the docs is merely an
oversight.
----------------------------------------------------------------------
Comment By: M.-A. Lemburg (lemburg)
Date: 2004-06-12 14:04
Message:
Logged In: YES
user_id=38388
I think that it might be a good idea to document of how the
standard search
function of the encodings package work at the top of that
page, namely
to normalize encoding names before doing the lookup:
"""
Normalization works as follows: all non-alphanumeric
characters except the dot used for Python package
names are
collapsed and replaced with a single underscore,
e.g. ' -;#'
becomes '_'. Leading and trailing underscores are
removed.
Note that encoding names should be ASCII only; if
they do use
non-ASCII characters, these must be Latin-1 compatible.
"""
The table should then only list normalized encoding names (which
I think is already the case).
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2004-06-12 13:59
Message:
Logged In: YES
user_id=21627
Actually, the top of the page does already say
Notice that spelling alternatives that only differ in case
or use a hyphen instead of an underscore are also valid aliases.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2004-06-12 13:54
Message:
Logged In: YES
user_id=21627
It is just not feasible to list all recognized aliases. For
example, for
ISO-8859-1, there are trivial 31 aliases, including
Iso_8859-1 and iSO-8859_1. For shift_jisx0213, there are
1023 trivial aliases.
The aliases column in the documentation should only list
non-trivial aliases, and for these, it should list a form
that people are most likely to encounter. So if "s-jis"
would be more common than "s_jis", this is what should be
listed. If s-JIS is even more common, this should be listed.
The top of the page should say that case in encoding names
does not matter, and that _ and - can be freely substituted.
----------------------------------------------------------------------
Comment By: Raymond Hettinger (rhettinger)
Date: 2004-06-12 09:05
Message:
Logged In: YES
user_id=80475
Mark, would you pronounce on this one.
----------------------------------------------------------------------
Comment By: Mike Brown (mike_j_brown)
Date: 2004-06-09 10:25
Message:
Logged In: YES
user_id=371366
I see no reason to omit any aliases that are recognized,
especially when the aliases in question are, more often than
not, the IANA's preferred MIME name as shown at
http://www.iana.org/assignments/character-sets.
I was looking in the docs to see if Python 2.4 was going to
support 'euc-jp', and was dismayed to see 'euc_jp' and
variants but no 'euc-jp'. I had to obtain and install 2.4a0 to
test to find out that it was just a documentation problem.
Please consider listing all realnames and aliases.
----------------------------------------------------------------------
Comment By: Hye-Shik Chang (perky)
Date: 2004-06-09 09:10
Message:
Logged In: YES
user_id=55188
Reopened to consider the consistence with non-cjk codecs.
All the non-cjk codecs are written with hyphen even if their
realname is with underscore. (eg. iso8859-1 and iso8859_1.py)
Will changing cjk codecs's codec/alias names to use not
underscores but hyphens make docs more friendly?
----------------------------------------------------------------------
Comment By: Hye-Shik Chang (perky)
Date: 2004-06-09 09:01
Message:
Logged In: YES
user_id=55188
All hyphens are translated as underscores in encoding lookups.
So we may not need to provide encoding list with hyphens
additionally.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=969415&group_id=5470
More information about the Python-bugs-list
mailing list