[Patches] [ python-Patches-1436130 ] Incremental codecs

[SourceForge.net]

Patches item #1436130, was opened at 2006-02-21 11:32
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1436130&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Incremental codecs

Initial Comment:
This patch extends the codec machinery to add
incremental codecs: stateful codecs that don't use a
stream API. It adds the following stuff: a class
codecs.CodecInfo (a subclass of tuple), that is used as
the return value of codecs.lookup();
codecs.IncrementalEncoder and codecs.IncrementalDecoder
(the basic interface classes),
codecs.BufferedIncrementalDecoder (a class that can be
used to implement decoders that must handle incomplete
input); codecs.iterencode() and codecs.iterdecode()
(generators that use the incremental codecs for
encoding/decoding an input iterable). On the C level
PyCodec_IncrementalEncoder() and
PyCodec_IncrementalDecoder() are added.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2006-03-15 00:01

Message:
Logged In: YES 
user_id=33168

MAL, do you have any more issues with this patch?  Should it
be assigned to Martin?

MAL, Walter, can you review these patches 1443155 1449471
which I think are related?  Should they go in?

The first alpha is coming up soon and I'd like to get these
patches in ASAP.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2006-03-03 09:39

Message:
Logged In: YES 
user_id=89016

This fourth version of the patch removes the changes to 
Lib/encodings/idna.py (only the addition of the 
IncrementalEncoder/IncrementalDecoder and the changed 
getregentry() remain). This patch to idna.py probably only 
makes sense once this patch is in.

> Is it possible to make IncrementalEncoder/Decoder
> instances iterable per-se (without the need to go
> through the helper functions iterencode/iterdecode) ?

For IncrementalEncoder/Decoder to be iterable it would have 
to have some iterable from which it gets the input. But 
this has the same limitation as the stream API: The user is 
forced to provide the input as a service that the 
encoder/decoder uses, which requires support for a certain 
API. The only change would be that now it's an iterator API 
instead of a stream API.

The incremental codecs invert the call logic: The user no 
longer has to provide a callback service to the codec, but 
calls the codec directly. This gives much more flexibility.


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2006-03-02 15:03

Message:
Logged In: YES 
user_id=38388

Very nice ! 

This is a much better approach than the feed style path you
wanted to take previously.

Minor nits:

Please separate out the non-related changes to the IDNA
codec into a new patch and assign that to Martin for review.

Is it possible to make IncrementalEncoder/Decoder instances
iterable per-se (without the need to go through the helper
functions iterencode/iterdecode) ?

Thanks.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2006-03-01 06:47

Message:
Logged In: YES 
user_id=89016

This third version of the patch fixes the bug when the
iterator in iterencode() or iterdecode() is empty and
updates the docstring in encodings/__init__.py.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2006-02-28 04:08

Message:
Logged In: YES 
user_id=89016

This second version of the patch enhances
codecs.iterencode() and codecs.iterdecode(), so that
additional keyword arguments are passed through to the
Incremental(De|En)coder constructor.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1436130&group_id=5470