On 4/4/11 10:35 PM, Charles R Harris wrote:
IIUC, "Ub" is undefined -- "U" means universal newlines, which makes no sense when used with "b" for binary. I looked at the code a ways back, and I can't remember the resolution order, but there isn't any checking for incompatible flags.
I'd expect that genfromtxt, being txt, and line oriented, should use 'rU'. but if it wants the raw line endings (why would it?) then rb should be fine.
"U" has been kept around for backwards compatibility, the python documentation recommends that it not be used for new code.
That is for 3.* -- the 2.7.* docs say: """ In addition to the standard fopen() values mode may be 'U' or 'rU'. Python is usually built with universal newline support; supplying 'U' opens the file as a text file, but lines may be terminated by any of the following: the Unix end-of-line convention '\n', the Macintosh convention '\r', or the Windows convention '\r\n'. All of these external representations are seen as '\n' by the Python program. If Python is built without universal newline support a mode with 'U' is the same as normal text mode. Note that file objects so opened also have an attribute called newlines which has a value of None (if no newlines have yet been seen), '\n', '\r', '\r\n', or a tuple containing all the newline types seen. Python enforces that the mode, after stripping 'U', begins with 'r', 'w' or 'a'. "" which does, in fact indicate that 'Ub' is NOT allowed. We should be using 'Ur', I think. Maybe the "python enforces" is what we saw the error from -- it didn't used to enforce anything. On 4/5/11 7:12 AM, Charles R Harris wrote:
The 'Ub' mode doesn't work for '\r' on python 3. This may be a bug in python, as it works just fine on python 2.7.
"Ub" never made any sense anywhere -- "U" means universal newline text file. "b" means binary -- combining them makes no sense. On older pythons, the behaviour of 'Ub' was undefined -- now, it looks like it is supposed to raise an error. does 'Ur' work with \r line endings on Python 3? According to my read of the docs, 'U' does nothing -- "universal" newline support is supposed to be the default: """ On input, if newline is None, universal newlines mode is enabled. Lines in the input can end in '\n', '\r', or '\r\n', and these are translated into '\n' before being returned to the caller. """
It may indeed be desirable to read the files as text, but that would require more work on both loadtxt and genfromtxt.
Why can't we just open the file with mode 'Ur'? text is text, messing with line endings shouldn't hurt anything, and it might help. If we stick with binary, then it comes down to: - will having an extra \r with Windows files hurt anything? -- probably not. - Are there many mac-style text files out there anymore? not many. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov