[Python-bugs-list] [ python-Bugs-496873 ] cPickle / time.struct_time loop

[noreply@sourceforge.net]

Bugs item #496873, was opened at 2001-12-26 13:18
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=496873&group_id=5470

Category: Type/class unification
Group: Python 2.2.1 candidate
Status: Open
Resolution: None
Priority: 5
Submitted By: bixentxo (bixentxo)
Assigned to: Anthony Baxter (anthonybaxter)
Summary: cPickle / time.struct_time loop

Initial Comment:
The following code produces and endless loop
on python 2.2 ( ok on python 2.1 ). The
file being saved by cPickle.dump grows
indefinetely.

-----------------------------
import time, cPickle

created = time.localtime()

fd = open('/tmp/bla.pickle','w')
cPickle.dump(created,fd)
[...endless loop... ]

----------------------------------------------------------------------

Comment By: Simo Salminen (frangen)
Date: 2002-01-31 05:13

Message:
Logged In: YES 
user_id=291461

os.stat results cant be pickled either.

>>> pickle.dump(os.stat('.'), open('foo', 'w'))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/opt/local/lang/python2.2/lib/python2.2/pickle.py", 
line 969, in dump
    Pickler(file, bin).dump(object)
  File "/opt/local/lang/python2.2/lib/python2.2/pickle.py", 
line 115, in dump
    self.save(object)
  File "/opt/local/lang/python2.2/lib/python2.2/pickle.py", 
line 185, in save
    tup = reduce()
  
File "/opt/local/lang/python2.2/lib/python2.2/copy_reg.py", 
line 56, in _reduce
    state = base(self)
TypeError: constructor takes exactly 13 arguments (10 given)


----------------------------------------------------------------------

Comment By: bixentxo (bixentxo)
Date: 2001-12-27 12:31

Message:
Logged In: YES 
user_id=395354

As for me, it is not a problem anymore
as I found the 'tuple' solution straight away.

It is not that I needed pickling struct_time
is that I had no reason why not doing it.

Only that it DID break my code.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-12-27 10:02

Message:
Logged In: YES 
user_id=31435

Re "does anybody need to pickle time structs?", I think a 
better question is "does anybody pickle time.localtime() 
results?".  The existence of the bug report suggests yes, 
and the OP is right that this worked in 2.1 (and 2.0, 
and ...).

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-27 08:32

Message:
Logged In: YES 
user_id=6380

Actually, the bug was in copy_reg._reduce: it was possible
that this would not make any progress, returning an instance
of the same type that it was passed, thus causing the
infinite recursion. I've checked in a fix that raises an
exception when this situation is detected.
(copy_reg.py:1.10)

Now, the question is, does anybody need to pickle time
structs? I would recommend converting them into a tuple
before pickling: pickle.dumps(tuple(time.localtime())) works
fine.

The statvfs issue was an unrelated bug; I've checked in a
fix for that too. (posixmodule.c:2.217)

Both are 2.2.1 candidates (I only noted this in the
posixmodule.c CVS checkin message by mistake).

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-27 06:06

Message:
Logged In: YES 
user_id=6380

It gets in an infinite loop with pickle too, so I think this
is definitely a bug in the magical type used for localtime.
The stacktrack looks like this:

  File "/usr/local/lib/python2.2/pickle.py", line 370, in
save_tuple
    save(element)
  File "/usr/local/lib/python2.2/pickle.py", line 215, in
save
    self.save_reduce(callable, arg_tup, state)
  File "/usr/local/lib/python2.2/pickle.py", line 241, in
save_reduce
    save(arg_tup)
  File "/usr/local/lib/python2.2/pickle.py", line 221, in
save
    f(self, object)

repeated an infinite number of times.

Note that we should look at the other objects that use
structseq.c/h too. pickle.dumps(os.stat("/")) gives

TypeError: constructor takes exactly 13 arguments (10 given)

pickle.dumps(os.statvfs("/")) gives

pickle.PicklingError: Can't pickle <type
'posix.statvfs_result'>: it's not the same object as
posix.statvfs_result

Looks like a variety of bugs. :-(



----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2001-12-27 03:50

Message:
Logged In: YES 
user_id=6656

Another factoid: if you back out the change that renamed the 
"struct_time" type to "time.struct_time", the crash goes 
away, to be replaced with 

cPickle.PicklingError: Can't pickle <type 'struct_time'>: 
it's not found as __builtin__.struct_time

so I suspect this bug has been lurking for some time.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-12-26 22:06

Message:
Logged In: YES 
user_id=31435

Boosted priority, assigned to Guido.

A stack fault is faster to produce via the one-liner

cPickle.dumps(time.localtime())

There's unbounded recursion.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=496873&group_id=5470