[Python-Dev] Possible bug in codecs readline? It breaks lines apart.

[Irmen de Jong]

Hello
using current cvs Python on Linux, I observe this weird
behavior of the readline() method on file-like objects
returned from the codecs module:

[irmen at atlantis ypage]$ cat testfile1.txt
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
offending line: ladfj askldfj klasdj fskla dfzaskdj fasklfj laskd 
fjasklfzzzzaa%whereisthis!!!
next line.
[irmen at atlantis ypage]$ cat testfile2.txt
aaaaaaaaaaaaaaaaaaaaaaaa
bbbbbbbbbbbbbbbbbbbbbbbb
stillokay:bbbbxx
broken!!!!badbad
againokay.
[irmen at atlantis ypage]$ cat bug.py
import codecs
for name in ("testfile1.txt","testfile2.txt"):
     f=codecs.open(name,encoding="iso-8859-1")  # precise encoding doesn't matter
     print "----",name,"----"
     for line in f:
         print "LINE:"+repr(line)
[irmen at atlantis ypage]$ python25 bug.py
---- testfile1.txt ----
LINE:u'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy\r\n'
LINE:u'offendi'
LINE:u'ng line: ladfj askldfj klasdj fskla dfzaskdj fasklfj laskd fjasklfzzzzaa'
LINE:u'%whereisthis!!!\r\n'
LINE:u'next line.\r\n'
---- testfile2.txt ----
LINE:u'aaaaaaaaaaaaaaaaaaaaaaaa\n'
LINE:u'bbbbbbbbbbbbbbbbbbbbbbbb\n'
LINE:u'stillokay:bbbbxx\n'
LINE:u'broke'
LINE:u'n!!!!badbad\n'
LINE:u'againokay.\n'
[irmen at atlantis ypage]$


See how it breaks certain lines in half?
It only happens when a certain encoding is used, so regular
file objects behave as expected. Also, readlines() works fine.

Python 2.3.4 and Python 2.4 do not have this problem.

Am I missing something or is this a bug? Thanks!

--Irmen