[Pythonmac-SIG] Re: [Python-Dev] Import hook to do end-of-line conversion?

[Guido van Rossum]

> Guido van Rossum wrote:
> > No, but (as has been explained) fixing the parser isn't enough -- all
> > tools dealing with source would have to be fixed.  Or we would have to
> > write our own C-level file object, which has its own drawbacks.
> 
> From what people have posted, it seems clear that having our own C-level
> file object is the only reasonable choice. It would take care of all the
> issues that have been brought up (both code and other text files, etc).
> Even if people have been sloppy and used binary mode for text files
> under *nix, that code would still work with *nix text files, which is
> the only way it works now anyway.
> 
> Given that something like this has been done in numerous places (JAVA,
> MATLAB, ???), It seems likely that there is some code out there that
> could be used. Hopefully there is some without licensing issues (Maybe
> Scilab or Octave, both are MATLAB clones)

I doubt that we could use anything that was done for another language,
because everybody who codes this kind of thing makes it do exactly
what their environment needs, e.g. in terms of error handling API,
functionality, and performance.

> What are the drawbacks?? (besides the below example)

The drawbacks aren't so much technical (I have a pretty good idea of
how to build such a thing), they're political and psychological.
There's the need for supporting the old way of doing things for years,
there's the need for making it easy to convert existing code to the
new way, there's the need to be no slower than the old solution,
there's the need to be at least as portable as the old solution (which
may mean implementing it *on top of* stdio since on some systems
that's all you've got).

> I'm not sure who wrote:
> > what happens under oddball systems like OpenVMS, which seem to use
> > radically different file structures for text and binary data?  I've no idea
> > what happens if you try to open a text file in binary mode under those.
> 
> Would we have to? At the Python level, you would open a text file, and
> get what you expected. The "oddball" port would have to have some
> probably very different code for the C-level file object, but that's
> presumable the case anyway. what happens when you want to read a
> non-native text file on those systems now? So you have to read it as
> binary?
> 
> By the way, does any of this tie in at all with trying to add Perl-like
> speed to processing text files?

It would be one way towards that goal.  But notice that we've already
gotten most of the way there with the recent readline changes in 2.1.

--Guido van Rossum (home page: http://www.python.org/~guido/)