Hi list I have found what appears to be a bug in the pickle and cPickle library in pypy. I can reproduce it both on linux and mac. Basically, I can't unpickle a file if it has been pickled with a file object returned by open(), but it works if I use the with construct. The problem is present both using pickle and cPickle. On mac, the pypy version is Python 2.7.3 (87aa9de10f9c, Nov 24 2013, 20:57:21) [PyPy 2.2.1 with GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] On linux, the pypy version is Python 2.7.3 (2.2.1+dfsg-1, Jan 24 2014, 10:12:37) [PyPy 2.2.1 with GCC 4.6.3] *#import cPickle as pickle* *import pickle* *l = range(10)* *# fails:* *try:* * pickle.dump(l, open('f', 'wb'))* * print pickle.load(open('f', 'rb'))* *except EOFError:* * print("Yup, that's an EOFError")* *#succeeds* *with open('f', 'wb') as f:* * pickle.dump(l, f)* *with open('f', 'rb') as f:* * print pickle.load(f)* If I instead of using open() inline in the argument to pickle assign it to a variable and explicitly close the file after calling pickle.dump, then the problem goes away: * f = open('f', 'wb')* * pickle.dump(l, f)* * f.close()* * print pickle.load(open('f', 'rb'))* Apparently, in the first code snippet, the file isn't closed as it should be when the object returned by open() goes out of scope after pickle.dump. Thanks, /Martin Koch
Hi Martin. This is a documented difference with CPython where files are not flushed unless either closed explicitly or garbage collected (which will happen later than you might expect). On Wed, Apr 23, 2014 at 3:39 PM, Martin Koch <mak@issuu.com> wrote:
Hi list
I have found what appears to be a bug in the pickle and cPickle library in pypy. I can reproduce it both on linux and mac. Basically, I can't unpickle a file if it has been pickled with a file object returned by open(), but it works if I use the with construct. The problem is present both using pickle and cPickle.
On mac, the pypy version is Python 2.7.3 (87aa9de10f9c, Nov 24 2013, 20:57:21) [PyPy 2.2.1 with GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)]
On linux, the pypy version is Python 2.7.3 (2.2.1+dfsg-1, Jan 24 2014, 10:12:37) [PyPy 2.2.1 with GCC 4.6.3]
#import cPickle as pickle import pickle
l = range(10) # fails: try: pickle.dump(l, open('f', 'wb')) print pickle.load(open('f', 'rb')) except EOFError: print("Yup, that's an EOFError")
#succeeds with open('f', 'wb') as f: pickle.dump(l, f)
with open('f', 'rb') as f: print pickle.load(f)
If I instead of using open() inline in the argument to pickle assign it to a variable and explicitly close the file after calling pickle.dump, then the problem goes away:
f = open('f', 'wb') pickle.dump(l, f) f.close() print pickle.load(open('f', 'rb'))
Apparently, in the first code snippet, the file isn't closed as it should be when the object returned by open() goes out of scope after pickle.dump.
Thanks, /Martin Koch
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
participants (2)
-
Maciej Fijalkowski
-
Martin Koch