pickle: maximum recursion depth exceeded

[Simon Burton]

Hi,

I am pickling big graphs of data and running into this problem:

[...]
  File "/usr/lib/python2.2/pickle.py", line 225, in save
    f(self, object)
  File "/usr/lib/python2.2/pickle.py", line 414, in save_list
    save(element)
  File "/usr/lib/python2.2/pickle.py", line 219, in save
    self.save_reduce(callable, arg_tup, state)
  File "/usr/lib/python2.2/pickle.py", line 249, in save_reduce
    save(state)
  File "/usr/lib/python2.2/pickle.py", line 225, in save
    f(self, object)
  File "/usr/lib/python2.2/pickle.py", line 447, in save_dict
    save(value)
  File "/usr/lib/python2.2/pickle.py", line 219, in save
    self.save_reduce(callable, arg_tup, state)
  File "/usr/lib/python2.2/pickle.py", line 245, in save_reduce
    save(arg_tup)
  File "/usr/lib/python2.2/pickle.py", line 225, in save
    f(self, object)
  File "/usr/lib/python2.2/pickle.py", line 374, in save_tuple
    save(element)
  File "/usr/lib/python2.2/pickle.py", line 225, in save
    f(self, object)
  File "/usr/lib/python2.2/pickle.py", line 405, in save_list
    write(self.put(memo_len))
RuntimeError: maximum recursion depth exceeded

However, it works when i try the smallest examples and use
sys.setrecursionlimit(4000)

This seems like a limitation in the pickling code. Yes?

The data is perhaps better off in some kind of DB designed for
massively interconnected objects. Any suggestions? ZODB ?

BTW, the data is path searching info for a game, and takes 1-2Mb of memory.

Thankyou,

Simon Burton.