On Wednesday 17 July 2002 07:38 pm, Guido van Rossum wrote: ...

But leaving the file object as an exception to the rule helps as a reminder that it's just a rule of thumb and cannot be taken as absolute law.

The sublunar world has enough reminders of its imperfections that we need not strive to add more.

...

Still, it doesn't solve the reference-loop-between-two-deuced-things- that-don't-cooperate-with-gc problem. And I can't see how either could be made into a WEAK reference given that xreadlines objects in other contexts need to hold a strong ref to the file they work on -- we'd have to refactor xreadlines objects too, a core part holding a weak ref and a shell around it (holding a strong ref to the file) to support ordinary calls to xreadlines.xreadlines. Messy:-(.

I don't think that a weak ref to the file would be sufficient for xreadlines -- e.g.

for line in open(filename): print line,

would close the file right away.

If the iterator were the file itself, no it wouldn't, whatever kind of ref the xreadlines object had to the file. What would break without refactoring would be: for line in xreadlines.xreadlines(open(filename)): ... The refactoring would be to have a, say _xreadlines, object, with the functionality of today's xreadlines object BUT a weak ref to the file, and an xreadlines object with strong refs to the file and the _xreadlines object and delegating functionality to the latter. A bit of a mess.

Likewise, the file needs a strong ref to the xreadlines, otherwise the

Definitely! Otherwise nothing keeps the xreadlines (or _xreadlines) object around _at all_ -- it's even worse than you indicate below, it seems to me:

following would create a new iterator in the second for loop, and lose data buffered by the first iterator.

f = open(filename) it = iter(f)

...with the patch it would be "it is f", and so, I don't really get it...

for i in range(10): it.next() del it for line in f: print line,

I think I will have to reject Oren's patch because of this, and the situation with file iterators will remain as it is: once you've asked for the iterator, all operations on the file are unsafe, and the only way to get back to using the file is to abandon the file and do an

Abandon the iterator, you mean? Or am I hopelessly confused?

absolute seek on the file. (This is sort of like switching between the raw integer file descriptor and the stream object in C -- or in Python if you care to use f.fileno() and os.read() etc.)

In these cases you do get some control on the buffering, though, if you care to exercise it. Alex

On Wednesday 17 July 2002 08:07 pm, Guido van Rossum wrote:

OK, I'll wait to see if someone submits a working patch. I still find it a non-issue myself.

OK, I'm gonna give it a try -- kludging up Oren's patch so that the xreadlines object is able to hold a non-addref'd pointer to the file object (when it's for internal use of the file object) and, as long as I'm at it, also including the little further kludge that makes f.readline delegate to f.next if f is holding an xreadlines object. Oh, and dropping the xreadlines object on a seek, too. It's just a few lines' changes to two files after all, Objects/fileobject.c and Modules/xreadlines.c. A bit kludgey and tricky, admittedly, which is perhaps not the nicest thing in the world given that fileobject.c isn't the shortest, simplest, or least crucial part of Python. But anyway, I think I'll have it ready by early tomorrow my time (it's past midnight and I'm past the age for all-nighters:-). Alex

Guido> Likewise, the file needs a strong ref to the xreadlines, Guido> otherwise the following would create a new iterator in the Guido> second for loop, and lose data buffered by the first iterator. Guido> f = open(filename) Guido> it = iter(f) Guido> for i in range(10): Guido> it.next() Guido> del it Guido> for line in f: Guido> print line, Guido> I think I will have to reject Oren's patch because of this, and Guido> the situation with file iterators will remain as it is: once Guido> you've asked for the iterator, all operations on the file are Guido> unsafe, and the only way to get back to using the file is to Guido> abandon the file and do an absolute seek on the file. This implies that you don't expect the code above to work correctly, right? -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark

Guido:

Likewise, the file needs a strong ref to the xreadlines, otherwise the following would create a new iterator in the second for loop, and lose data buffered by the first iterator.

To me, these problems are screaming out that the buffer *shouldn't* be kept in the xreadlines object! Maybe the xreadlines object's buffer should be kept in the file object? Then it wouldn't matter if multiple xreadlines objects were created, as they'd all share the same buffer, and there would be no reference loops. Hmmm... then we're moving towards making the file object and the xreadlines object be the same object. What was the reason for not doing that again? Was it just to avoid changing a lot of code, or was there some reason it wouldn't work? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Alex Martelli :

Still, it doesn't solve the reference-loop-between-two-deuced-things- that-don't-cooperate-with-gc problem.

Would making them cooperate with GC be a difficult thing to do? Seems to me we should be moving towards making everything cooperate with GC, and fixing things like this whenever they come to light. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

...

Would making them cooperate with GC be a difficult thing to do? Seems to me we should be moving towards making everything cooperate with GC, and fixing things like this whenever they come to light.

Tim Peters says it wouldn't be, but I have not explored that.

But he also warned that it introduces new surprises. --Guido van Rossum (home page: http://www.python.org/~guido/)