What's wrong with >>> open(’xxx’, ’w’).write(’stuff’) ?
Hi, I encountered this quite a few times when learning pypy from internet resources: the code like this
open(’xxx’, ’w’).write(’stuff’)
This code is not working on pypy because it rely on CPython refcounting behaviour. I don't get it. Why ? I thought the code should be similar to storing the file object in temporary variable like this
f = open('xxx', 'w') f.write('stuff') del f
Also, I've tried that with both Jython and IronPython and they all work fine. Why does this cause problem to pypy ? Do I have to avoid writing code like this in the future ?
Hi. Yes, those two things are equivalent and they both work. However, if you try to read the file immediately after deleting the variable, you'll find out that the file is empty on any implementation but cpython. On Thu, Aug 19, 2010 at 6:25 AM, sakesun roykiatisak <sakesun@gmail.com> wrote:
open(’xxx’, ’w’).write(’stuff’) This code is not working on pypy because it rely on CPython refcounting behaviour. I don't get it. Why ? I thought the code should be similar to storing the file object in temporary variable like this f = open('xxx', 'w') f.write('stuff') del f Also, I've tried that with both Jython and IronPython and they all work fine. Why does this cause problem to pypy ? Do I have to avoid writing code like
Hi, I encountered this quite a few times when learning pypy from internet resources: the code like this this in the future ? _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Hi Sakesun, On Thu, Aug 19, 2010 at 11:25:42AM +0700, sakesun roykiatisak wrote:
f = open('xxx', 'w') f.write('stuff') del f
Also, I've tried that with both Jython and IronPython and they all work fine.
I guess that you didn't try exactly the same thing. If I do: arigo@tannit ~ $ jython Jython 2.2.1 on java1.6.0_20 Type "copyright", "credits" or "license" for more information. >>> open('x', 'w').write('hello') >>> Then "cat x" in another terminal shows an empty file. The file "x" is only filled when I exit Jython. It is exactly the same behavior as I get on PyPy. Maybe I missed something, and there is a different way to do things such that it works on Jython but not on PyPy; if so, can you describe it more precisely? Thanks! A bientot, Armin.
Armin: Sakesun used "del f" and it appears you did not. In Python IIRC, an explicit call to del should kick off the finalizer to flush and close the file! open('x', 'w').write('hello') alone does not imply the file instance (return value of open()) has been finalized because the garbage collector may not have hit it yet. Jython and IronPython are pretty much guaranteed to behave differently under a wide variety of circumstances when it comes to the garbage collector. Do not rely on the garbage collector for program semantics! Because Sakesun has used "del f" it should be quite a concern that the file has not been finalized properly! On Fri, Aug 20, 2010 at 5:57 AM, Armin Rigo <arigo@tunes.org> wrote:
Hi Sakesun,
On Thu, Aug 19, 2010 at 11:25:42AM +0700, sakesun roykiatisak wrote:
f = open('xxx', 'w') f.write('stuff') del f
Also, I've tried that with both Jython and IronPython and they all work fine.
I guess that you didn't try exactly the same thing. If I do:
arigo@tannit ~ $ jython Jython 2.2.1 on java1.6.0_20 Type "copyright", "credits" or "license" for more information. >>> open('x', 'w').write('hello') >>>
Then "cat x" in another terminal shows an empty file. The file "x" is only filled when I exit Jython. It is exactly the same behavior as I get on PyPy. Maybe I missed something, and there is a different way to do things such that it works on Jython but not on PyPy; if so, can you describe it more precisely? Thanks!
A bientot,
Armin. _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
It seems you too have missed the difference between deleting some reference to the object (as del does) and finalising. On 20/08/2010 8:23 PM, "Donny Viszneki" <donny.viszneki@gmail.com> wrote: Armin: Sakesun used "del f" and it appears you did not. In Python IIRC, an explicit call to del should kick off the finalizer to flush and close the file! open('x', 'w').write('hello') alone does not imply the file instance (return value of open()) has been finalized because the garbage collector may not have hit it yet. Jython and IronPython are pretty much guaranteed to behave differently under a wide variety of circumstances when it comes to the garbage collector. Do not rely on the garbage collector for program semantics! Because Sakesun has used "del f" it should be quite a concern that the file has not been finalized properly! On Fri, Aug 20, 2010 at 5:57 AM, Armin Rigo <arigo@tunes.org> wrote:
Hi Sakesun,
On Thu, Aug ... -- http://codebad.com/
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/...
Hi Donny, On Fri, Aug 20, 2010 at 06:23:26AM -0400, Donny Viszneki wrote:
Armin: Sakesun used "del f" and it appears you did not.
As explained earlier this makes no difference. E.g. in any Python version, the following code would not call the __del__ method of the object x either:
x = SomeClassWithADel() y = x del x
A bientot, Armin.
On Fri, Aug 20, 2010 at 12:23, Donny Viszneki <donny.viszneki@gmail.com> wrote:
Armin: Sakesun used "del f" and it appears you did not. Actually, he didn't either. He said "I think that open(’xxx’, ’w’).write(’stuff’)" is equivalent to using del (which he thought would work), and the equivalence was correct.
Anyway, in the _first reply_ message, he realized that using: ipy -c "open(’xxx’, ’w’).write(’stuff’)" jython -c "open(’xxx’, ’w’).write(’stuff’)" made a difference (because the interpreter exited), so that problem was solved. His mail implies that on PyPy he typed the code at the prompt, rather than at -c.
In Python IIRC, an explicit call to del should kick off the finalizer to flush and close the file!
No, as shown by Armin. del will just clear the reference, which on CPython means decreasing the refcount. Refcounting will then finalize the object immediately, a GC at some later point, if it runs at all - there's no such guarantee on Java and .NET. For Java, that's unless you do special unsafe setup (System.runFinalizersOnExit(), it's discouraged for a number of reasons, see docs). On .NET, I expect a such method to exist, too, since they were so unaware of problems wiith finalizers in .NET 1.0 to give them the syntax of destructors. But .NET 2.0 has SafeHandles, which guarantee release of critical resources if the "finalization" code follows some restriction, using _reference counting_: http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.safeh... http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.safeh...
open('x', 'w').write('hello') alone does not imply the file instance (return value of open()) has been finalized because the garbage collector may not have hit it yet.
On CPython, you have such an implication, because of refcounting semantics.
On Fri, Aug 20, 2010 at 5:57 AM, Armin Rigo <arigo@tunes.org> wrote:
Hi Sakesun,
On Thu, Aug 19, 2010 at 11:25:42AM +0700, sakesun roykiatisak wrote:
f = open('xxx', 'w') f.write('stuff') del f
Also, I've tried that with both Jython and IronPython and they all work fine.
I guess that you didn't try exactly the same thing. If I do:
arigo@tannit ~ $ jython Jython 2.2.1 on java1.6.0_20 Type "copyright", "credits" or "license" for more information. >>> open('x', 'w').write('hello') >>>
Then "cat x" in another terminal shows an empty file. The file "x" is only filled when I exit Jython. It is exactly the same behavior as I get on PyPy. Maybe I missed something, and there is a different way to do things such that it works on Jython but not on PyPy; if so, can you describe it more precisely? Thanks!
-- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/
This discussion is getting a little too long than necessary, at least for me. :) Most of pypy talk video is in pretty poor recording quality. Most of the time I try to discern barely from the slides. I always understand the difference between resource lifetime and object lifetime. Actually, in my most recent years, my sole python interpreter is the non-refcounting IronPython already. And I always wrap file operation inside try/finally or with statement. The problem is the example that claim to cause problem:
open('xxx', 'w').write('stuff')
I misinterpret that the problem is caused in the "write" methods. The above statement cause no problem, but the subsequent usage of the file will. That's what I missed. In fact, it might be more intuitive to demonstrate in a little longer sample.
open('xxx', 'w').write('stuff') assert open('xxx').read() == 'stuff' # Might fail ! The first file might not be closed yet !
Cheers. On Fri, Aug 20, 2010 at 8:39 PM, Paolo Giarrusso <p.giarrusso@gmail.com>wrote:
On Fri, Aug 20, 2010 at 12:23, Donny Viszneki <donny.viszneki@gmail.com> wrote:
Armin: Sakesun used "del f" and it appears you did not. Actually, he didn't either. He said "I think that open(’xxx’, ’w’).write(’stuff’)" is equivalent to using del (which he thought would work), and the equivalence was correct.
Anyway, in the _first reply_ message, he realized that using:
ipy -c "open(’xxx’, ’w’).write(’stuff’)" jython -c "open(’xxx’, ’w’).write(’stuff’)"
made a difference (because the interpreter exited), so that problem was solved. His mail implies that on PyPy he typed the code at the prompt, rather than at -c.
Hi Donny, On Fri, Aug 20, 2010 at 06:23:26AM -0400, Donny Viszneki wrote:
Armin: Sakesun used "del f" and it appears you did not. In Python IIRC, an explicit call to del should kick off the finalizer to flush and close the file!
No, you are wrong. Try for example: >>> f = open('xxx') >>> g = f >>> del f After this, 'g' still refers to the file, and it is still open. If you want the file to be flushed and closed, then call 'f.close()' :-) A bientot, Armin.
Obviously please close the file, ideally using something like the with-statement or at least finally. But for perhaps the convenience of scripters, and the sorrow of everyone else ;), Jython will close the file upon clean termination of the JVM via registering a closer of such files with Runtime#addShutdownHook<http://download.oracle.com/javase/6/docs/api/java/lang/Runtime.html#addShutd...> This is currently part of the most important outstanding bug<http://bugs.jython.org/issue1327>in Jython 2.5.2, and something that has to be resolved for 2.5.2 beta 2, because of how it interacts with classloaders and prevents their class GC upon reload (thus potentially exhausting permgen). On Fri, Aug 20, 2010 at 8:06 AM, Armin Rigo <arigo@tunes.org> wrote:
Hi Donny,
On Fri, Aug 20, 2010 at 06:23:26AM -0400, Donny Viszneki wrote:
Armin: Sakesun used "del f" and it appears you did not. In Python IIRC, an explicit call to del should kick off the finalizer to flush and close the file!
No, you are wrong. Try for example:
f = open('xxx') g = f del f
After this, 'g' still refers to the file, and it is still open.
If you want the file to be flushed and closed, then call 'f.close()' :-)
A bientot,
Armin. _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
participants (7)
-
Armin Rigo -
Donny Viszneki -
Jim Baker -
Maciej Fijalkowski -
Paolo Giarrusso -
sakesun roykiatisak -
William Leslie