[issue18820] json.dump() ignores its 'default' option when serializing dictionary keys
New submission from July Tikhonov: According to documentation of json.dump(), concerning its 'default' option: default(obj) is a function that should return a serializable version of obj or raise TypeError. The default simply raises TypeError. But this function is actually never applied to serialized dictionary keys:
def default(obj): ... if isinstance(obj, bytes): ... return obj.decode('ascii') ... raise ValueError ... json.dumps(b'asdf') Traceback (most recent call last): ... TypeError: b'asdf' is not JSON serializable json.dumps(b'asdf', default=default) '"asdf"' json.dumps({b'asdf' : 1}, default=default) Traceback (most recent call last): ... TypeError: keys must be a string json.dumps({1 : b'asdf'}, default=default) '{"1": "asdf"}'
(bytes are used purely for the purpose of example) Such behavior should be either documented or corrected. Patch correcting python implementation of json attached. ---------- assignee: docs@python components: Documentation, Library (Lib) files: json-default-dict-keys.diff keywords: patch messages: 195957 nosy: docs@python, july priority: normal severity: normal status: open title: json.dump() ignores its 'default' option when serializing dictionary keys type: enhancement versions: Python 3.4 Added file: http://bugs.python.org/file31436/json-default-dict-keys.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18820> _______________________________________
July Tikhonov added the comment: Oops, my patch disables 'skipkeys' argument of dump. Another version attached. ---------- Added file: http://bugs.python.org/file31437/json-default-dict-keys.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18820> _______________________________________
Changes by July Tikhonov <july.tikh@gmail.com>: Removed file: http://bugs.python.org/file31436/json-default-dict-keys.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18820> _______________________________________
Ezio Melotti added the comment: Are there already tests that cover the changes in your patch? If not, could you add them? ---------- nosy: +ezio.melotti, pitrou, rhettinger stage: -> test needed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18820> _______________________________________
July Tikhonov added the comment: Proposed tests attached. ---------- Added file: http://bugs.python.org/file31450/json-default-tests.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18820> _______________________________________
Changes by Chris Rebert <pybugs@rebertia.com>: ---------- nosy: +cvrebert _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18820> _______________________________________
Erwin Mayer added the comment: Will this be merged? I also believe it is an unexpected behavior to not serialize dictionary keys when the default option is used. ---------- nosy: +Erwin Mayer _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18820> _______________________________________
Changes by Berker Peksag <berker.peksag@gmail.com>: ---------- assignee: docs@python -> components: -Documentation versions: -Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18820> _______________________________________
Serhiy Storchaka added the comment: See also https://github.com/simplejson/simplejson/issues/42 https://github.com/simplejson/simplejson/issues/100 https://github.com/simplejson/simplejson/issues/103 And allowing non-string keys leads to other issue: https://github.com/simplejson/simplejson/issues/77 Thus there is an argument for disallowing serializing non-string keys. ---------- nosy: +bob.ippolito, serhiy.storchaka _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18820> _______________________________________
Erwin Mayer added the comment: Regarding the issues mentioned in https://github.com/simplejson/simplejson/issues/77, they already apply with the current implementation anyway (true is serialized as 'true'), so users must already be careful. The JSONEncoder with default parameters could definitely keep rejecting complex keys, but an optional 'encode_complex_keys=False' parameter could be added to __init__ to provide this functionality. There would be zero performance impact as the parameter check would only be done if all else has failed instead of raising the TypeError (the same way _skipkeys does). In that respect, the patch of this issue would need to be amended (and the C version would also need to be changed). I am trying to fork the json core module to achieve this (to not have to wait for the next Python release, assuming it gets implemented), if you are familiar with the packaging process of core modules into standalone modules, your advice would be much appreciated (and could well help others contribute to Python): http://stackoverflow.com/questions/36498527/how-to-create-a-customized-versi... ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue18820> _______________________________________
participants (6)
-
Berker Peksag
-
Chris Rebert
-
Erwin Mayer
-
Ezio Melotti
-
July Tikhonov
-
Serhiy Storchaka