[Python-Dev] New codecs checked in

[M.-A. Lemburg]

Martin v. Löwis wrote:
> M.-A. Lemburg wrote:
> 
>>I just left them in because I thought they wouldn't do any harm
>>and might be useful in some applications.
>>
>>Removing them where not directly needed by the codec would not
>>be a problem.
> 
> 
> I think memory usage caused is measurable (I estimated 4KiB per
> dictionary). More importantly, people apparently currently change
> the dictionaries we provide and expect the codecs to automatically
> pick up the modified mappings. It would be better if the breakage
> is explicit (i.e. they get an AttributeError on the variable) instead
> of implicit (their changes to the mapping simply have no effect
> anymore).

Agreed. I've already checked in the changes, BTW.

>>KOI8-U is not available as mapping on ftp.unicode.org and
>>I only recreated codecs from the mapping files available
>>there.
> 
> 
> I think we should come up with mapping tables for the additional
> codecs as well, and maintain them in the CVS. This also applies
> to things like rot13.

Agreed.

>>I'll rerun the creation with the above changes sometime this
>>week.
> 
> 
> I hope I can finish my encoding routine shortly, which again
> results in changes to the codecs (replacing the encoding dictionaries
> with other lookup tables).

Having seen the decode tables written as long Unicode string,
I think that this may indeed also be a good solution for
encoding - the major improvement here is that the parser
and compiler will do the work of creating the table. At
module load time, the .pyc file will only contain a long
string which is very fast to create and load (unlike dictionaries
which are set up dynamically at load time).

In general, it's better to do all the work up-front when
creating the codecs, rather than having run-time code
repeat these tasks over and over again.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 25 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::