[Pythonmac-SIG] Re: [Python-Dev] Import hook to do end-of-line conversion?

[Tim Peters]

[Guido]
> ...
> But if we want to use our own line end convention, we can't use
> fgets() any more, so we lose big.

Well, people said "we couldn't" use fgets() for get_line() either, because
Python strings can contain embedded nulls but fgets() doesn't tell you how
many bytes it read and makes up null bytes of its own.  But I have 200 lines
of excruciating code in fileobject.c that proved them excruciatingly wrong
<wink>.  The same kind of excruciating crap could almost certainly be used to
search for alternative line endings on top of fgets() too.  We would have to
layer our own buffer on top of the hidden platform buffer to get away with
this, because while fgets() will stop at the first \n it sees, there's no way
to ask it to stop at any other character (so in general fgets() would
"over-read" when looking for a non-native line-end, and we'd have to save the
excess in our own buffer).

Hard to say how much that would cost.  I think it surprised everyone (incl.
me!) that even with all the extra buffer-filling and buffer-searching the
fgets() hackery does, that method was at worst a wash with the
getc_unlocked() method on all platforms tried.

In any case, the fgets() hack is only *needed* on Windows, so every other
platform could just make get_line()'s character-at-a-time loop search for
more end conditions.  This can't be impossible <wink>.

s/\r\n?/\n/g-ly y'rs  - tim