[Pythonmac-SIG] Re: MacPython and line-endings

[Jack Jansen]

The more I think about the problem the more I think it should be
handled at the stdio level or slightly above it.

With the current proposal of Chris Barker we get a platform
independent solution with only minimal extra C code in the core, but
we pay for that by having to re-implement the import logic in Python
(which will again, at least for MacPython introduce new machine
dependencies if we don't want to lose PYC resources and such).

Suggestion: we implement a Py_UniversalFgets(char *, int, FILE *) and
a Py_UniversalFread(char *, int, int, FILE *). These are then used
everywhere in Python (mainly fileobject.c and the parser, I think, and
I think these are the only ones used).

There are now three possible implementations of these:
- depending on a configure option they can be #defined as fgets() and
  fread() for maximal performance.
- there is a machine-independent implementation using fgetc.
- there may be a machine-dependent (or, actually,
  stdio-implementation-dependent) implementation that does horrible
  things like peeking into stdio data structures and doing something
  almost as fast as what fgets() and fread() would do.

Actually the machine-dependent implementation needn't be all that
dependent: you need routines that are really machine dependent for
getting at the buffer pointer and the number of valid bytes there, and
possibly one for getting at the pushback (ungetc) buffer, all
non-destructive. The universal routines can then examine those
buffers, call fread() upto buffer-end or newline and possibly convert
the latter, and possibly repeat.

This method would enable universal newlines for all text files,
however, which may be seen as a drawback (but which I would personally
see as an advantage).
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.cwi.nl/~jack        | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm