[Python-Dev] Re: [Patches] [Patch #102915] xreadlines : readlines :: xrange : range

[Jeff Epler]

I'd like to speak up about this patch I've submitted on sourceforge.

I consider the xreadlines function/object to be the core of my proposal.
The addition of a method to file objects, as well as the modifications
to fileinput, are secondary in my opinion.

The desire is to iterate over file conents in a way that satisfies the
following criteria:
	* Uses the "for" syntax, because this clearly captures the
	  underlying operation. (files can be viewed as sequences of
	  lines when appropriate)
	* Consumes small amounts of memory even when the file contents
	  are large.
	* Has the lowest overhead that can reasonably be attained.

I think that it is agreed that the ability to use the "for" syntax is
important, since it was the impetus for the xrange function/object.
After all, there's a "while" statement which will give the same effect,
without introducing xrange.

The point under debate, as I see it, is the utility of speeding up the
"benchmarks" of folks who compare the speed of Python and another
language doing a very simple loop over the lines in a file.  Since this
advantage disappears once real work is beig done on the file, maybe an
XReadLines class, written in Python, would be more suitable.  In fact,
I've written such a class since I didn't know about fileinput and in
any case I find it less useful to me because of all the weird stuff it
does. (parsing argv, opening files by name, etc)

One shortcoming of my current patch, aside from the ones already named
in another person's response to the it, are that it fails when working
on a file-like class which implements .readline but not .readlines.

In any case, I wrote xreadlines to learn how to write C extensions to
Python, and submitted it at the suggestion of a fellow Python user in a
private discussion.  I'd like to extinguish one of these eternal
comp.lang.python threads with it too, but maybe it's not to be.

Happy new year, all.

Jeff