interaction of mode 'r+', file.write(), and file.tell(): a bug or undefined behavior?

[Alf P. Steinbach]

* Anthony Tolle:
> On Jan 28, 7:12 am, Lie Ryan <lie.1... at gmail.com> wrote:
>> In the code:
>>
>> """
>> f = open('input.txt', 'r+')
>> for line in f:
>>     s = line.replace('python', 'PYTHON')
>>     # f.tell()
>>     f.write(s)
>> """
>> [snip]
> 
> My guess is that there are a few possible problems:
> 
> 1) In this case, writing to file opened with 'r+' without an explicit
> f.seek is probably not a good idea.  The file iterator (for line in f)
> uses a readahead buffer, which means you can't guarantee what the
> current file position will be.
> 
> 2) It may be necessary to do an explicit f.flush or f.close when
> writing to an 'r+' file.  In your case, the close should automatically
> happen when the f object falls out of scope, which tells me that were
> still looking at some other problem, like not using f.seek
> 
> 3) It is possible that f.tell implicitly flushes buffers used by the
> file object.  That would explain why uncommenting the f.tell causes
> the writes to show up.

As far as I understand it the behavior stems from CPython file operations being 
implemented fairly directly as forwarding to C library FILE* operations, and the 
C standard prescribes Undefined Behavior to the case above.

I think the Python language/library specification should specify the effect 
(perhaps just as UB, but anyway, specified).

For as it is, it may/will be different with different Python implementations, 
meaning that code that works OK with one implementation may fail with another 
implementation.



> What are you trying to accomplish?  Overwrite the original file, or
> append to it?  If you want to overwrite the file, it may be better to
> generate a new file, delete the old one, then rename the new one.  If
> you want to append, then it would be better to open the file with
> append mode ('a')

Cheers,

- Alf