[Python-ideas] Iterating non-newline-separated files should be easier

[Andrew Barnert]

On Saturday, July 19, 2014 9:42 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> > Paul Moore wrote:
>>  And for that matter, how would you remove an
>>  arbitrary separator? Maybe line = line[:-1] works, but what if at some
>>  point people ask for multi-character separators

You already can't use line[:-1] today, because '\r\n' is already a valid value, and always has been.

And however people deal with newline='\r\n' will work for any crazy separator you can think of. Maybe line[:-len(nl)]. Maybe line.rstrip(nl) if it's appropriate (it isn't always, either for \r\n or for some arbitrary separator).

> If the newline mechanism is re-used, it would

> convert whatever separator is used into '\n'.


No it wouldn't.

https://docs.python.org/3/library/io.html#io.TextIOWrapper

> When reading input from the stream, if newline is None, universal newlines mode is enabled… If it is '', universal newlines mode is enabled, but line endings are returned to the caller untranslated. If it has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated.

So, making '\0' a legal value just means the '\0' line endings will be returned to the caller untranslated.

Also, remember that binary files don't do universal newline translation ever, so just letting you change the separator there wouldn't add translation.


Of course both of those could be changed as well (although with what interface, I'm not sure…), but I don't think they should be.