[Python-Dev] Re: [Csv] csv module and universal newlines

[Skip Montanaro]

    Andrew> The csv parser consumes lines from an iterator, but it also has
    Andrew> it's own idea of end-of-line conventions, which are currently
    Andrew> only used by the writer, not the reader, which is a source of
    Andrew> much confusion. The writer, by default, also attempts to emit a
    Andrew> \r\n sequence, which results in more confusion unless the file
    Andrew> is opened in binary mode.

    Andrew> I'm looking for suggestions for how we can mitigate these
    Andrew> problems (without breaking things for existing users).

You can argue that reading csv data from/writing csv data to a file on
Windows if the file isn't opened in binary mode is an error.  Perhaps we
should enforce that in situations where it matters.  Would this be a start?

    terminators = {"darwin": "\r",
                   "win32": "\r\n"}

    if (dialect.lineterminator != terminators.get(sys.platform, "\n") and
       "b" not in getattr(f, "mode", "b")):
       raise IOError, ("%s not opened in binary mode" %
                       getattr(f, "name", "???"))

The elements of the postulated terminators dictionary may already exist
somewhere within the sys or os modules (if not, perhaps they should be
added).  The idea of the check is to enforce binary mode on those objects
that support a mode if the desired line terminator doesn't match the
platform's line terminator.

Skip