[issue23174] shelve.open fails with error "anydbm.error: db type could not be determined"
New submission from Karl Richter: `shelve.open(tempfile.mkstemp()[1])` fails with error "anydbm.error: db type could not be determined" which is not explainable with the docs. Traceback is Traceback (most recent call last): File "./cudaminer_param_checker.py", line 720, in <module> plac.call(cudaminer_param_checker) File "/usr/local/lib/python2.7/dist-packages/plac_core.py", line 309, in call cmd, result = parser_from(obj).consume(arglist) File "/usr/local/lib/python2.7/dist-packages/plac_core.py", line 195, in consume return cmd, self.func(*(args + varargs + extraopts), **kwargs) File "./cudaminer_param_checker.py", line 715, in cudaminer_param_checker visualize_cudaminer_param_checker_results_wxpython_gui() File "./cudaminer_param_checker.py", line 365, in visualize_cudaminer_param_checker_results_wxpython_gui frame = CudaminerParamChecker(None, ) File "./cudaminer_param_checker.py", line 378, in __init__ self.generator = CudaminerParamCheckerGenerator() File "./cudaminer_param_checker.py", line 160, in __init__ self.result_dict= shelve.open(storage_file_path) File "/usr/lib/python2.7/shelve.py", line 239, in open return DbfilenameShelf(filename, flag, protocol, writeback) File "/usr/lib/python2.7/shelve.py", line 223, in __init__ Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback) File "/usr/lib/python2.7/anydbm.py", line 82, in open raise error, "db type could not be determined" anydbm.error: db type could not be determined ---------- assignee: docs@python components: Documentation messages: 233488 nosy: docs@python, krichter priority: normal severity: normal status: open title: shelve.open fails with error "anydbm.error: db type could not be determined" type: behavior versions: Python 2.7 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
Karl Richter added the comment: EDIT 1: other examples, e.g. import os import shelve curdir = os.path.dirname(__file__) passwords = shelve.open(os.path.join(curdir, 'password_db')) work, so there's need for usable error messages. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
R. David Murray added the comment: You are opening a just-created empty file. The db type of the file cannot, therefore, be determined. Which is what the error message says. How would you suggest the error message be improved? ---------- nosy: +r.david.murray _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
R. David Murray added the comment: Sorry, I mean "an empty file with no recognized extension". I doubt supplying a suffix is what you want, though, since you probably want shelve to pick the persistent backend on db creation in order to be portable. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
Karl Richter added the comment: Then, let the error message say "You are opening a just-created empty file. The db type of the file cannot, therefore, be determined." which is much clearer than "anydbm.error: db type could not be determined" which sounds like a generic fallback error message in "an error occured"-style. It seems to be necessary that the filename passed to `shelve.open` has a suffix which is not very intuitive for Linux users. It'd be great to have validation of the suffix and raise a `ValueError` with an error message like `"filename '%s' doesn't contain a suffix" % (filename,)` when it isn't supplied. If there's another issue regarding the fact that the file is "just-created" and empty, this is independent of the issue above. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
R. David Murray added the comment: The problem is that this is not the only situation in which that error can be generated. If someone passed in an "old" (and possibly non-empty) file with no extension, the same code path would get executed. The shelve/anydbm API is database agnostic, so there is no way to know a-priori what the valid extensions are. The whichdbm module (which anydbm uses) could provide this information in theory, but it doesn't currently, and if it did that would be a new API. Your problem is that the shelve API is designed so that you either pass it the base name of an existing database, *or* you pass it the base name of a *non* existent database, which it then creates all the right files for (there may be more than one file). The fact that you are passing it a new existing file is the problem, and I'm not sure the error message can be effectively improved to handle that particular mistaken use of the API without breaking the message for the places it is more meaningful. The best I can think of is to add the filename to the error message and list all the extensions and (if there are any) the 'magic' type guesses that whichdbm tried, but as I said that would be a new feature and thus could only go into 3.5. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
Karl Richter added the comment: That's nice, thanks. Considering your last comment, some points * If the issue can't go into the error message, than the essence of the discussion here should go into the docs (in 0.5 to 1.5 sentences). * It'd be nice if it was clear from the error message that shelve or underlying modules check the extension in filename, i.e. do `anydbm.error: db type could not be determined by file extension '%s' in filename '%s'` rather than `anydbm.error: db type could not be determined`. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
R. David Murray added the comment: What is it you want the docs to say? For your second point, yes, that is what I was saying would be an enhancement (the extension is a list of possible extensions that varies per-platform and per how python is built/installed). ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
Karl Richter added the comment: For example, it should be clear why `shelve.open(tempfile.mkstemp()[1])` fails with the mentioned exception and `shelve.open("/tmp/bla")` fails. I still haven't figured out the constraints to create a working `shelve.Shelve` at all. It should be clear why `shelve.open("/tmp/tmphTTQLd")` fails and `shelve.open("/tmp/tmphTTQLda")` succeeds. There has to be something unrelated to extensions. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
Karl Richter added the comment: After checking the code, I think that it'd make more sense to document `whichdb.py`. It needs to be enhanced with references to criteria for the determination of the database type. Currently there're only function comments and the fact that some variables are named magic speaks for itself, i.e. it's ok for the implementation of the module to be a black box, but the requirements for the input, i.e. the potential database file, which is per se not part of the black box, needs to be specified. Then just link that in the `shelve` docs stating that `shelve.open` is basically a wrapper around `whichdb`. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
Changes by Roundup Robot <devnull@psf.upfronthosting.co.za>: ---------- pull_requests: +1291 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
Anand Reddy Pandikunta added the comment: dbm.whichdb mentions that it returns empty string ('') if the file’s format can’t be guessed. Enhancing exception message should suffice. ---------- nosy: +ChillarAnand _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
Serhiy Storchaka added the comment: Proposed error message isn't correct. The file extension is not the only method for determining db type. Current error message looks good to me and I don't see how it can be enhanced. ---------- nosy: +serhiy.storchaka _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
Raymond Hettinger added the comment: [Serhiy]
Current error message looks good to me and I don't see how it can be enhanced.
I concur with Serhiy and David. ---------- nosy: +rhettinger resolution: -> not a bug stage: -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue23174> _______________________________________
participants (6)
-
Anand Reddy Pandikunta -
Karl Richter
-
R. David Murray
-
Raymond Hettinger
-
Roundup Robot
-
Serhiy Storchaka