Hi folks, I've just submitted a patch that allows Python to do cross-platform newline handling, #476814. This sort-of happened after the promised PEP didn't materialise, and there was discussion about this among myself, Guido and the pythonmac-sig, and at some point Guido said "Ok, code it!". The patch has been tested on SGI Irix and in MacPython only. I'd like people to have a look at this. There's one thing I don't like, and that's the changes I had to make to the parser sourcefiles. They're needed, but I'm open to suggestions as to a better way to do this. And of course I'm open to any suggestions. I'm also interested in discussing whether a patch like this is appropriate while we're in beta. On the one hand I would say it is, because the feature is disabled by default. On the other hand there are changes (albeit mainly cosmetic ones) in a large number of places. Another argument for allowing this even while in beta is that I really really want it for Mac OS X (but this might not be a very strong argument, I guess:-). Here's the summary of the patch: This patch enables Python to interpret all known newline conventions, CR, LF or CRLF, on all platforms. This support is enabled by configuring with --with-universal-newlines (so by default it is off, and everything should behave as usual). With universal newline support enabled two things happen: - When importing or otherwise parsing .py files any newline convention is accepted. - Python code can pass a new "t" mode parameter to open() which reads files with any newline convention. "t" cannot be combined with any other mode flags like "w" or "+", for obvious reasons. File objects have a new attribute "newlines" which contains the type of newlines encountered in the file (or None when no newline has been seen, or "mixed" if there were various types of newlines). Also included is a test script which tests both file I/O and parsing. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.cwi.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++
Jack wrote:
- Python code can pass a new "t" mode parameter to open() which reads files with any newline convention. "t" cannot be combined with any other mode flags like "w" or "+", for obvious reasons.
so "t" can be both a mode (like "r" and "w") and a mode modifier (like "t" and "b")? could be confusing. </F>
[Jack Jansen]
- Python code can pass a new "t" mode parameter to open() which reads files with any newline convention. "t" cannot be combined with any other mode flags like "w" or "+", for obvious reasons.
[/F]
so "t" can be both a mode (like "r" and "w") and a mode modifier (like "t" and "b")? could be confusing.
Well, the "t" modifier isn't std; does anyone other than Microsoft support it? MS also gives meaning to non-std "c" and "n" modifiers. Luckily, Python doesn't define which modes or modifiers it accepts <wink>.
"JJ" == Jack Jansen <jack@oratrix.nl> writes:
JJ> I'm also interested in discussing whether a patch like this is JJ> appropriate while we're in beta. On the one hand I would say JJ> it is, because the feature is disabled by default. On the JJ> other hand there are changes (albeit mainly cosmetic ones) in JJ> a large number of places. Another argument for allowing this JJ> even while in beta is that I really really want it for Mac OS JJ> X (but this might not be a very strong argument, I guess:-). I tend toward the more conservative, especially with changes that touch lots of .c files, so I'd say -1. But it looks from the patch dialog that Guido's already approved of this in principle, so I'll revise that to a -0. -Barry
I tend toward the more conservative, especially with changes that touch lots of .c files, so I'd say -1. But it looks from the patch dialog that Guido's already approved of this in principle, so I'll revise that to a -0.
I'm not at all sure on whether this should be incorporated into 2.2. I approve it (or something like it) for 2.3; for 2.2, I'm hesitant but if Jack thinks it's needed for MacOS, and it's off by default, and a thorough code review shows no problems with it as long as it's off, I would be OK with it. In other words, you're the release manager; it's your call. --Guido van Rossum (home page: http://www.python.org/~guido/)
"GvR" == Guido van Rossum <guido@python.org> writes:
>> I tend toward the more conservative, especially with changes >> that touch lots of .c files, so I'd say -1. But it looks from >> the patch dialog that Guido's already approved of this in >> principle, so I'll revise that to a -0. GvR> I'm not at all sure on whether this should be incorporated GvR> into 2.2. I approve it (or something like it) for 2.3; for GvR> 2.2, I'm hesitant but if Jack thinks it's needed for MacOS, GvR> and it's off by default, and a thorough code review shows no GvR> problems with it as long as it's off, I would be OK with it. GvR> In other words, you're the release manager; it's your call. Sounds good. I'll reserve judgement for now. -Barry
[Jack Jansen]
... I'm also interested in discussing whether a patch like this is appropriate while we're in beta. On the one hand I would say it is, because the feature is disabled by default. On the other hand there are changes (albeit mainly cosmetic ones) in a large number of places.
Not so many -- it's mostly in fileobject.c, which perhaps should not have surprised me <wink>. A prime reason we do "feature freeze" is to give adequate mindshare to the stuff already in the beta. I just spent 10 hours on a Sunday "dealing with" bugs and patches and 2.2 issues, including over an hour reviewing this patch. The 2.2 work I *wanted* to get out of the way this weekend never got started, and it's increasingly doubtful it ever will. If the patch were a no-brainer with no controversial aspects, and 2.2 were basically done in all other respects, it would be easy to say "sure"; as is, there are threading issues and (lack of) error-detection issues and (lack of) doc issues and user interface issues ... and this sure looks like something that *should* have had a PEP and wide community debate (not just in the Mac community). I like the idea of the patch. but the timing is really bad (esp. with Guido on leave this month, American holidays coming up fast, and a backlog of bug reports and patches growing in the wrong <wink> direction). OTOH, since it's off by default, "no harm done" is an arguable position. If it does go in, and you turn it on for the Mac, and we decide we need something different in 2.3, will we be doomed to play the "sorry, can't change it -- backward compatibility!" game? If so, I'm -1 on it for 2.2. If it's treated as purely experimental and subject to arbitrary potentially incompatible change, then -0 (the "-" still for mindshare-dilution reasons alone).
participants (6)
-
barry＠zope.com -
Fredrik Lundh -
Guido van Rossum -
Jack Jansen -
Tim Peters -
Tim Peters