[Python-Dev] Patch submitted for cross-platform newline support

[Jack Jansen]

Hi folks,
I've just submitted a patch that allows Python to do cross-platform newline 
handling, #476814. This sort-of happened after the promised PEP didn't 
materialise, and there was discussion about this among myself, Guido and the 
pythonmac-sig, and at some point Guido said "Ok, code it!". The patch has been 
tested on SGI Irix and in MacPython only.

I'd like people to have a look at this. There's one thing I don't like, and 
that's the changes I had to make to the parser sourcefiles. They're needed, 
but I'm open to suggestions as to a better way to do this. And of course I'm 
open to any suggestions.

I'm also interested in discussing whether a patch like this is appropriate 
while we're in beta. On the one hand I would say it is, because the feature is 
disabled by default. On the other hand there are changes (albeit mainly 
cosmetic ones) in a large number of places. Another argument for allowing this 
even while in beta is that I really really want it for Mac OS X (but this 
might not be a very strong argument, I guess:-).

Here's the summary of the patch:

This patch enables Python to interpret all known newline conventions,
CR, LF or CRLF, on all platforms.

This support is enabled by configuring with --with-universal-newlines
(so by default it is off, and everything should behave as usual).

With universal newline support enabled two things happen:
- When importing or otherwise parsing .py files any newline convention
  is accepted.
- Python code can pass a new "t" mode parameter to open() which reads
  files with any newline convention. "t" cannot be combined with any
  other mode flags like "w" or "+", for obvious reasons.

File objects have a new attribute "newlines" which contains the type
of newlines encountered in the file (or None when no newline has been
seen, or "mixed" if there were various types of newlines).

Also included is a test script which tests both file I/O and parsing.

--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | ++++ see http://www.xs4all.nl/~tank/ ++++