[Python-Dev] Stateful codecs [Was: str object going in Py3K]

[Walter Dörwald]

M.-A. Lemburg wrote:
> Walter Dörwald wrote:
>> M.-A. Lemburg wrote:
>>
>>> [...]
>>> Like I suggested in the patch discussion, such functionality could
>>> be factored out of the implementations of StreamReaders/Writers
>>> and put into new StatefulEncoder/Decoder classes, the objects of
>>> which then get used by StreamReader/Writer.
>>>
>>> In addition to that we could extend the codec registry to also
>>> maintain slots for the stateful encoders and decoders, if needed.
>> We *have* to do it like this otherwise there would be no way to get a
>> StatefulEncoder/Decoder from an encoding name.
>>
>> Does this mean that codecs.lookup() would have to return a 6-tuple? 
>> But this would break if someone uses codecs.lookup("foo")[-1].
> 
> Right; though I'd much rather see that people use the direct
> codecs module lookup APIs:
> 
> getencoder(), getdecoder(), getreader() and getwriter()
> 
> instead of using codecs.lookup() directly.

OK.

>> So maybe
>> codecs.lookup() should return an instance of a subclass of tuple which
>> has the StatefulEncoder/Decoder as attributes. But then codecs.lookup()
>> must be able to handle old 4-tuples returned by old search functions and
>> update those to the new 6-tuples. (But we could drop this again after
>> several releases, once all third party codecs are updated).
> 
> This was a design error: I should have not made
> codecs.lookup() a documented function.
> 
> I'd suggest we keep codecs.lookup() the way it is and
> instead add new functions to the codecs module, e.g.
> codecs.getencoderobject() and codecs.getdecoderobject().
> 
> Changing the codec registration is not much of a problem:
> we could simply allow 6-tuples to be passed into the
> registry.

OK, so codecs.lookup() returns 4-tuples, but the registry stores 
6-tuples and the search functions must return 6-tuples. And we add 
codecs.getencoderobject() and codecs.getdecoderobject() as well as new 
classes codecs.StatefulEncoder and codecs.StatefulDecoder. What about 
old search functions that return 4-tuples?

Bye,
    Walter Dörwald