As far as I know I've addressed all outstanding issues in PEP 278, http://python.sourceforge.net/peps/pep-0278.html and in the accompanying patch.

I'm cautiously in favor of this, but a few more things need to be addressed. I didn't study the patch too carefully, so I'll ask: When this is disabled through the configure flag, is the 'U' mode still recognized? I think it ought to be allowed then (and mean simply text mode) so that Python code opening files in universal mode doesn't have to be prepared for that situation (it can't use the newlines attribute, but that's rarely needed I expect). Before we go ahead, I'd like MvL and MAL to have a look at the patch to see if there would be interactions with their implementation plans for PEP 262. I still think that this PEP is a big hack -- but as big hacks go, it seems to have a pretty good payback. I'm hoping that eventually the parser (really the lexer) will be able to open the file in binary mode and recognize all three newline styles directly. That would solve the problems with exec, eval, and compile. Missing: - docs for the new open mode and file attributes (!!!) - docs for the --with-universal-newlines flag in README - the linecache and py_compile modules should use mode 'U' (any others?) --Guido van Rossum (home page: http://www.python.org/~guido/)

I'm proposing to accept PEP 278 and let Jack check it in. There will probably be some small issues with the code (though it passes the test suite on my Linux box), but I expect the only way to tease those out is to check it in first. Jack did an admirable job of answering all questions about the patch, adding doc patches and unit tests, and so on. Unless there's significant uproar about this within 48 hours, I'll approve the PEP. One comment for Jack: I think that the 'newlines' attribute should exist even if universal newlines are not configured; it should always be None in that case. --Guido van Rossum (home page: http://www.python.org/~guido/)

- when mixed newlines are found in a file, can the newlines attribute be a list of those that are found, instead of "mixed"?

Good idea.

- shouldn't open() mode "wU" also be illegal?

Indeed.

Other than that, +1 (it would squelch the complaints about the email package on systems who's MTA doesn't convert to native-newlines for mail program stdin).

Indeed! --Guido van Rossum (home page: http://www.python.org/~guido/)

One more:

- Why isn't "rU+" allowed? It's understandable that "+" is not allowed for output, but I can't see a good reason why it shouldn't be allowed to open file for read/write, and read with universal support (maybe some implementation detail?). Allowing this would even easy the task, since when writting to the file the programmer could consider the newlines attribute, if he wants.

This is answered by the PEP: """ A partial output implementation, where strings passed to fp.write() would be converted to use fp.newlines as their line terminated but all other output would not is far too surprising, in my view. Because there is no output support for universal newlines there is also no support for a mode "rU+": the surprise factor of the previous paragraph would hold to an even stronger degree. """ --Guido van Rossum (home page: http://www.python.org/~guido/)

This is answered by the PEP:

A partial output implementation, where strings passed to fp.write() would be converted to use fp.newlines as their line terminated but all other output would not is far too surprising, in my view.

Because there is no output support for universal newlines there is also no support for a mode "rU+": the surprise factor of the previous paragraph would hold to an even stronger degree.

I've read that, but I don't agree with the second paragraph. Universal newline support is available only for input. This sentence is enough to easily predict the behavior in every usage case, including "+" modes. If there's any intent to add support to output, then I'd understand the exclusion, since this would make backwards compatibility possible (users of "rU+" could have their code broken, since they were doing output by hand). Otherwise, it should be allowed, IMO. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ]

[PEP 278]

...

A partial output implementation, where strings passed to fp.write() would be converted to use fp.newlines as their line terminated but all other output would not is far too surprising, in my view.

Because there is no output support for universal newlines there is also no support for a mode "rU+": the surprise factor of the previous paragraph would hold to an even stronger degree.

[Gustavo]

I've read that, but I don't agree with the second paragraph. Universal newline support is available only for input. This sentence is enough to easily predict the behavior in every usage case, including "+" modes.

If there's any intent to add support to output, then I'd understand the exclusion, since this would make backwards compatibility possible (users of "rU+" could have their code broken, since they were doing output by hand). Otherwise, it should be allowed, IMO.

That's one possible reason not to do this. It could be added later, but I think it's better to disallow it for now. Think YAGNI. --Guido van Rossum (home page: http://www.python.org/~guido/)

Hi Gordon!

How can you reliably update a file if there's no obvious mapping between file position and character position?

If you're updating a text file, the relation is probably about line position, not file position. AFAICS, this would only be a problem if one wanted to insert something between CR and LF <wink>. -- Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ]

there's enough "Mode U cannot be combined" stuff in the PEP to make me think that they're slightly more than just a minor variation.

I think you can simplify the PEP somewhat by emphasizing the difference. making things simpler is never a bad idea.

...

Also, changing the mode is a morelocalized change (it's all in fileobject.c).

I'm willing to do the work necessary to provide an alternate factory/constructor. from what I can tell, all that has to be done is is to apply the patch, and do some fairly straight- forward refactoring.

(and perhaps some slight API renaming -- the functions in there are clearly too useful to deserve names like Py_UniversalNewlineFgets -- "there is no permanent place in this world for ugly names" ;-)

so +1 on checking it in as it stands, but -0 on not touching the result until 2.3 final...

OK, good. I'll leave this between you & Jack then. I'll try to make sure you two actually do something about the open items before 2.3 is released! --Guido van Rossum (home page: http://www.python.org/~guido/)

On donderdag, april 11, 2002, at 10:01 , Fredrik Lundh wrote:

so +1 on checking it in as it stands, but -0 on not touching the result until 2.3 final...

I'm +1 on reading the second half of that sentence as a statement that you volunteer to do this:-) (Though I still don't understand why you think a separate constructor is somehow cleaner than a new mode flag, even after reading your rationale. But I have enough other interesting things to do so I don't want to argue this point with you, you can have it if you do the work:-) -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -