[Python-Dev] unicode regex quickie: should a newline be the same thing as a linebreak?

Fredrik Lundh Fredrik Lundh" <effbot@telia.com
Sat, 13 May 2000 14:56:41 +0200


in the current 're' engine, a newline is chr(10) and nothing
else.

however, in the new unicode aware engine, I used the new
LINEBREAK predicate instead, but it turned out to break one
of the tests in the current test suite:

    sre.match('a\rb', 'a.b') => None

(unicode adds chr(13), chr(28), chr(29), chr(30), and also
unichr(133), unichr(8232), and unichr(8233) to the list of
line breaking codes)

what's the best way to deal with this?  I see three alter-
natives:

a) stick to the old definition, and use chr(10) also for
   unicode strings

b) use different definitions for 8-bit strings and unicode
   strings; if given an 8-bit string, use chr(10); if given
   a 16-bit string, use the LINEBREAK predicate.

c) use LINEBREAK in either case.

I think (c) is the "right thing", but it's the only that may
break existing code...

</F>