[Python-Dev] Stateful codecs [Was: str object going in Py3K]

[M.-A. Lemburg]

Walter Dörwald wrote:
> M.-A. Lemburg wrote:
>> Walter Dörwald wrote:
>>> M.-A. Lemburg wrote:
>>>
>>>> [...]
>>>> Like I suggested in the patch discussion, such functionality could
>>>> be factored out of the implementations of StreamReaders/Writers
>>>> and put into new StatefulEncoder/Decoder classes, the objects of
>>>> which then get used by StreamReader/Writer.
>>>>
>>>> In addition to that we could extend the codec registry to also
>>>> maintain slots for the stateful encoders and decoders, if needed.
>>> We *have* to do it like this otherwise there would be no way to get a
>>> StatefulEncoder/Decoder from an encoding name.
>>>
>>> Does this mean that codecs.lookup() would have to return a 6-tuple?
>>> But this would break if someone uses codecs.lookup("foo")[-1].
>>
>> Right; though I'd much rather see that people use the direct
>> codecs module lookup APIs:
>>
>> getencoder(), getdecoder(), getreader() and getwriter()
>>
>> instead of using codecs.lookup() directly.
> 
> OK.
> 
>>> So maybe
>>> codecs.lookup() should return an instance of a subclass of tuple which
>>> has the StatefulEncoder/Decoder as attributes. But then codecs.lookup()
>>> must be able to handle old 4-tuples returned by old search functions and
>>> update those to the new 6-tuples. (But we could drop this again after
>>> several releases, once all third party codecs are updated).
>>
>> This was a design error: I should have not made
>> codecs.lookup() a documented function.
>>
>> I'd suggest we keep codecs.lookup() the way it is and
>> instead add new functions to the codecs module, e.g.
>> codecs.getencoderobject() and codecs.getdecoderobject().
>>
>> Changing the codec registration is not much of a problem:
>> we could simply allow 6-tuples to be passed into the
>> registry.
> 
> OK, so codecs.lookup() returns 4-tuples, but the registry stores
> 6-tuples and the search functions must return 6-tuples. And we add
> codecs.getencoderobject() and codecs.getdecoderobject() as well as new
> classes codecs.StatefulEncoder and codecs.StatefulDecoder. What about
> old search functions that return 4-tuples?

The registry should then simply set the missing entries to None
and the getencoderobject()/getdecoderobject() would then have
to raise an error.

Perhaps we should also deprecate codecs.lookup() in Py 2.5 ?!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 17 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::