[issue6331] Add unicode script info to the unicode database

[Martin v. Löwis]

Martin v. Löwis <martin at v.loewis.de> added the comment:

I think the patch is incorrect: the default value for the script
property ought to be Unknown, not Common (despite UCD.html saying the
contrary; see UTR#24 and Scripts.txt).

I'm puzzled why you use a hard-coded list of script names. The set of
scripts will certainly change across Unicode versions, and I think it
would be better to learn the script names from Scripts.txt.

Out of curiosity: how does the addition of the script property affect
the number of distinct database records, and the total size of the database?

I think a common application would be lower-cases script names, for more
efficient comparison; UCD has also changed the spelling of the script
names over time (from being all-capital before). So I propose that
a) two functions are provided: one with the original script names, and
one with the lower-case script names
b) keep cached versions of interned script name strings in separate
arrays, to avoid PyString_FromString every time.

I'm doubtful that script names need to be provided for old database
versions, so I would be happy to not record the script for old versions,
and raise an exception if somebody tries to get the script for an old
database version - surely applications of the old database records won't
be accessing the script property, anyway.

----------
nosy: +loewis

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6331>
_______________________________________