Hi Pydeas, I encountered exceptions like File "c:\python27\lib\site-packages\filecache-0.67-py2.7.egg\filecache\__init__.py", line 82, in function_with_cache rv = function.__db[key] File "c:\python27\lib\shelve.py", line 122, in __getitem__ value = Unpickler(f).load() cPickle.UnpicklingError: could not find MARK With my filecache module which I have a feeling are related to database failures upon abrupt termination of the process. If I write the class SqliteShelf, would it be of use to anyone? Would it be a viable patch for shelve.py? Note that I don't mean to affect the shelve.py "open" function, just to add a reliable alternative class. http://docs.python.org/dev/library/shelve.html Cheers, --Yuval Greenfield
On Fri, Aug 05, 2011 at 11:36:49AM +0300, Yuval Greenfield wrote:
Something like this: http://bugs.python.org/issue3783 ? Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
Actually I was originally thinking of just directly interacting with sqlite3 in shelve.py but if you think this is the better approach (it does make sense to just use the dbm api), I'm willing to try and write up the patch. Some more shelve stack traces follow: Traceback (most recent call last): File "import_chess.py", line 282, in <module> get_all_players() File "import_chess.py", line 182, in get_all_players get_player_tourneys(first, last) File "import_chess.py", line 172, in get_player_tourneys get_tournament(tournament_id) File "import_chess.py", line 139, in get_tournament tournament_table = get_tournament_by_type(tournament_id, 3) File "import_chess.py", line 90, in get_tournament_by_type tournament_html = get(url).decode('iso-8859-1') File "c:\python27\lib\site-packages\filecache-0.67-py2.7.egg\filecache\__init_ _.py", line 82, in function_with_cache rv = function.__db[key] File "c:\python27\lib\shelve.py", line 122, in __getitem__ value = Unpickler(f).load() cPickle.UnpicklingError: invalid load key, 'x'. Traceback (most recent call last): File "import_chess.py", line 286, in <module> get_all_players() File "import_chess.py", line 186, in get_all_players get_player_tourneys(first, last) File "import_chess.py", line 174, in get_player_tourneys get_tournament(tournament_id) File "import_chess.py", line 137, in get_tournament tournament_table = get_tournament_by_type(tournament_id, 2) File "import_chess.py", line 90, in get_tournament_by_type tournament_html = get(url).decode('iso-8859-1') File "c:\python27\lib\site-packages\filecache-0.67-py2.7.egg\filecache\__init_ _.py", line 91, in function_with_cache function.__db[key] = __retval(_time.time(), _pickle.dumps(retval)) File "c:\python27\lib\shelve.py", line 133, in __setitem__ self.dict[key] = f.getvalue() File "c:\python27\lib\bsddb\__init__.py", line 279, in __setitem__ _DeadlockWrap(wrapF) # self.db[key] = value File "c:\python27\lib\bsddb\dbutils.py", line 68, in DeadlockWrap return function(*_args, **_kwargs) File "c:\python27\lib\bsddb\__init__.py", line 278, in wrapF self.db[key] = value bsddb.db.DBRunRecoveryError: (-30974, 'DB_RUNRECOVERY: Fatal error, run database recovery -- PANIC: Invalid argument') --Yuval
On Fri, Aug 05, 2011 at 11:36:49AM +0300, Yuval Greenfield wrote:
Something like this: http://bugs.python.org/issue3783 ? Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
Actually I was originally thinking of just directly interacting with sqlite3 in shelve.py but if you think this is the better approach (it does make sense to just use the dbm api), I'm willing to try and write up the patch. Some more shelve stack traces follow: Traceback (most recent call last): File "import_chess.py", line 282, in <module> get_all_players() File "import_chess.py", line 182, in get_all_players get_player_tourneys(first, last) File "import_chess.py", line 172, in get_player_tourneys get_tournament(tournament_id) File "import_chess.py", line 139, in get_tournament tournament_table = get_tournament_by_type(tournament_id, 3) File "import_chess.py", line 90, in get_tournament_by_type tournament_html = get(url).decode('iso-8859-1') File "c:\python27\lib\site-packages\filecache-0.67-py2.7.egg\filecache\__init_ _.py", line 82, in function_with_cache rv = function.__db[key] File "c:\python27\lib\shelve.py", line 122, in __getitem__ value = Unpickler(f).load() cPickle.UnpicklingError: invalid load key, 'x'. Traceback (most recent call last): File "import_chess.py", line 286, in <module> get_all_players() File "import_chess.py", line 186, in get_all_players get_player_tourneys(first, last) File "import_chess.py", line 174, in get_player_tourneys get_tournament(tournament_id) File "import_chess.py", line 137, in get_tournament tournament_table = get_tournament_by_type(tournament_id, 2) File "import_chess.py", line 90, in get_tournament_by_type tournament_html = get(url).decode('iso-8859-1') File "c:\python27\lib\site-packages\filecache-0.67-py2.7.egg\filecache\__init_ _.py", line 91, in function_with_cache function.__db[key] = __retval(_time.time(), _pickle.dumps(retval)) File "c:\python27\lib\shelve.py", line 133, in __setitem__ self.dict[key] = f.getvalue() File "c:\python27\lib\bsddb\__init__.py", line 279, in __setitem__ _DeadlockWrap(wrapF) # self.db[key] = value File "c:\python27\lib\bsddb\dbutils.py", line 68, in DeadlockWrap return function(*_args, **_kwargs) File "c:\python27\lib\bsddb\__init__.py", line 278, in wrapF self.db[key] = value bsddb.db.DBRunRecoveryError: (-30974, 'DB_RUNRECOVERY: Fatal error, run database recovery -- PANIC: Invalid argument') --Yuval
participants (2)
-
Oleg Broytman -
Yuval Greenfield