[Python-3000] Proposed new language for newline parameter to TextIOBase

[Adam Olsen]

On 8/14/07, Guido van Rossum <guido at python.org> wrote:
> I thought some more about the universal newlines situation, and I
> think I can handle all the use cases with a single 'newline'
> parameter. The use cases are:
>
> (A) input use cases:
>
> (1) newline=None: input with default universal newlines mode; lines
> may end in \r, \n, or \r\n, and these are translated to \n.
>
> (2) newline='': input with untranslated universal newlines mode; lines
> may end in \r, \n, or \r\n, and these are returned untranslated.

Caveat: this mode cannot be supported by sockets.  When reading a lone
\r you need to peek ahead to ensure the next character is not a \n,
but for sockets that may block indefinitely.

I don't expect sockets to use the file API by default, but there's
enough overlap (named pipes?) that limitations like this should be
well documented (and if possible, produce an explicit error!)


> (3) newline='\r', newline='\n', newline='\r\n': input lines must end
> with the given character(s), and these are translated to \n.
>
> (B) output use cases:
>
> (1) newline=None: every \n written is translated to os.linesep.
>
> (2) newline='': no translation takes place.
>
> (3) newline='\r', newline='\n', newline='\r\n': every \n written is
> translated to the value of newline.
>
> Note that cases (2) are new, and case (3) changes from the current PEP
> and/or from the current implementation (which seems to deviate from
> the PEP).

[snip]
>
>         * If universal newlines without translation are requested on
>           input (i.e. ``newline=''``), if a system read operation
>           returns a buffer ending in ``'\r'``, another system read
>           operation is done to determine whether it is followed by
>           ``'\n'`` or not.  In universal newlines mode with
>           translation, the second system read operation may be
>           postponed until the next read request, and if the following
>           system read operation returns a buffer starting with
>           ``'\n'``, that character is simply discarded.


-- 
Adam Olsen, aka Rhamphoryncus