cPickle fails on manually compiled and executed Python function
dieter
dieter at handshake.de
Tue Jul 18 01:07:24 EDT 2017
"Jan Gosmann" <jan at hyper-world.de> writes:
> today I came across some weird behaviour (a bug?) in Python 2.7.13 (on
> Linux) with the cPickle module. The pickle module works and so does
> the pickle module in Python 3.
>
> I have a file fn.py with a minimal function definition:
>
> ```
> def fn():
> pass
> ```
>
> The actual code that I run is in a separate file (test.py):
>
> ```
> import cPickle
> import pickle
>
> def load_pyfile(filename):
> source = ''
> with open(filename, 'r') as f:
> source += f.read()
> code = compile(source, filename, 'exec')
> loaded = {'__file__': filename}
> exec(code, loaded)
> return loaded
>
> fn = load_pyfile('fn.py')['fn']
>
> print(pickle.dumps(fn))
> print('----')
> print(cPickle.dumps(fn))
> ```
>
> The first print works fine, but the one with cPickle leads to an
> exception. Here is the output:
>
> ```
> c__main__
> fn
> p0
> .
> ----
> Traceback (most recent call last):
> File "test.py", line 17, in <module>
> print(cPickle.dumps(fn))
> TypeError: expected string or Unicode object, NoneType found
"pickle" (and "cpickle") are serializing functions as so called
"global"s, i.e. as a module reference together with a name.
This means, they cannot handle functions computed in a module
(as in your case).
I am quite convinced that "pickle" will not be able to deserialize (i.e. load)
your function (even though it appears to perform the serialization
(i.e. dump).
You are using "pickle/cPickle" for a case not anticipated (computed
functions). This means that this case is not as tested as other
cases. As a consequence, you can see different behavior between
"picke" and "cPickle".
More information about the Python-list
mailing list