Nested Dictionary Pointers and Logic - "flatten / nested iteration" Proposal
Hello Python-ideas, I am working with lots of JSON objects in my work and need to obtain JSON pointers to particular components of a JSON object. These pointers are defined as part of the JSON-LD specifications as relative IRIs and as part of the JSON-Schema specifications. However, I am not aware of these pointers being part of the JSON specification. The pointers are really straightforward and follow the same syntax as other IRIs (URLs) or file paths. For example, the following flatten function illustrates the mapping of nested dictionaries and arrays to a single flat dictionary, where each nested key is mapped to a unique path. d = {"a": 1, "b": {"ba": 2, "bb": [{"bba": 3, "bbb": None}]}} flatten(d)
{'a': 1, 'b/ba': 2, 'b/bb/0/bba': 3, 'b/bb/0/bbb': None}
Not only does this conversion help in generating JSON pointers, but it helps with logic for nested data structures. Specifically, assume there is a dictionary, which includes nested dictionaries, lists, and tuples, and any of these elements are contained within another dictionary or that other dictionaries elements. Then it is not sufficient to simply compare the items of the two dictionaries to check whether the first is a subset of the other. However, the flat structures can be compared directly. 1. A nested dictionary is a subset of another nested dictionary. The flatten function ensures the nested dictionaries are not checked for equivalence but subset of. # Current {"a": 1, "c": {"d": 4}}.items() <= {"a": 1, "b": 2, "c": {"d": 4, "e": 5}}.items()
False # Proposed flatten({"a": 1, "c": {"d": 4}}).items() <= flatten({"a": 1, "b": 2, "c": {"d": 4, "e": 5}}).items() True
2. A nested list or tuple is a subset of a dictionary. # Current {"a": 1, "c": [3]}.items() <= {"a": 1, "b": 2, "c": [3,3]}.items()
False # Proposed flatten({"a": 1, "c": [3]}).items() <= flatten({"a": 1, "b": 2, "c": [3,3]}).items() True
Note that these examples only have one level of nesting, but the flatten function must handle any level of nesting. For the current version of the flatten function, I essentially borrowed the JSON "dumps" source code. However, the dumps code doesn't utilize any other functions and I believe there may be a missing "nestedIter" function in python. This function could be used for JSON dumps or flatten or anytime someone wants to visit all nested elements of a dictionary and perform some operation. Therefore, I think this function could be included in the JSON library if python-ideas thinks that the pointers are very specific to JSON or the itertools library, where the nested iteration function could be generally used for all nested data types. Let me know what you think of this proposal and I am looking forward to your responses. Best, Sven
You've given some examples of what flatten() does, but I still don't really understand how it's expected to work. Could you provide a specification, rather than just examples? Also, I've never needed a function like this in anything I've ever done - that may mean nothing, but it does suggest that you need to demonstrate that it's a generally useful tool (maybe by pointing to examples of "real world" code in various contexts that would be improved by using it). There's a lot of pressure to keep the stdlib for things that are widely applicable, and the burden of demonstrating that lies with the proposer. As the proposal stands here, I honestly don't see anything that suggests this wouldn't be better as a standalone function published on PyPI. Paul On Wed, 21 Jul 2021 at 13:43, Sven Voigt <svenpvoigt@gmail.com> wrote:
Hello Python-ideas,
I am working with lots of JSON objects in my work and need to obtain JSON pointers to particular components of a JSON object. These pointers are defined as part of the JSON-LD specifications as relative IRIs and as part of the JSON-Schema specifications. However, I am not aware of these pointers being part of the JSON specification. The pointers are really straightforward and follow the same syntax as other IRIs (URLs) or file paths. For example, the following flatten function illustrates the mapping of nested dictionaries and arrays to a single flat dictionary, where each nested key is mapped to a unique path.
d = {"a": 1, "b": {"ba": 2, "bb": [{"bba": 3, "bbb": None}]}} flatten(d)
{'a': 1, 'b/ba': 2, 'b/bb/0/bba': 3, 'b/bb/0/bbb': None}
Not only does this conversion help in generating JSON pointers, but it helps with logic for nested data structures. Specifically, assume there is a dictionary, which includes nested dictionaries, lists, and tuples, and any of these elements are contained within another dictionary or that other dictionaries elements. Then it is not sufficient to simply compare the items of the two dictionaries to check whether the first is a subset of the other. However, the flat structures can be compared directly.
1. A nested dictionary is a subset of another nested dictionary. The flatten function ensures the nested dictionaries are not checked for equivalence but subset of. # Current {"a": 1, "c": {"d": 4}}.items() <= {"a": 1, "b": 2, "c": {"d": 4, "e": 5}}.items()
False # Proposed flatten({"a": 1, "c": {"d": 4}}).items() <= flatten({"a": 1, "b": 2, "c": {"d": 4, "e": 5}}).items() True
2. A nested list or tuple is a subset of a dictionary. # Current {"a": 1, "c": [3]}.items() <= {"a": 1, "b": 2, "c": [3,3]}.items()
False # Proposed flatten({"a": 1, "c": [3]}).items() <= flatten({"a": 1, "b": 2, "c": [3,3]}).items() True
Note that these examples only have one level of nesting, but the flatten function must handle any level of nesting. For the current version of the flatten function, I essentially borrowed the JSON "dumps" source code. However, the dumps code doesn't utilize any other functions and I believe there may be a missing "nestedIter" function in python. This function could be used for JSON dumps or flatten or anytime someone wants to visit all nested elements of a dictionary and perform some operation. Therefore, I think this function could be included in the JSON library if python-ideas thinks that the pointers are very specific to JSON or the itertools library, where the nested iteration function could be generally used for all nested data types.
Let me know what you think of this proposal and I am looking forward to your responses.
Best, Sven _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/V2DNTL... Code of Conduct: http://python.org/psf/codeofconduct/
Paul, I agree that the mapping to a flat data structure might have limited use cases. However, like you said it is only one example. What I am proposing is a nested iteration tool that can map both keys and values to new keys and new values. It only considers nesting in dictionaries, lists, and tuples by default, but allows users to pass another function to continue searching for pointers in other types. For example: class Node: name: str="" next: 'Node' = None val: Any = None def newDefaultMapper(o) -> o: '''Converts the object o and returns it''' if isinstance(o, dict): # This would be automatically implemented in itermapper.Mapper if arg 'dictMapper' was defined return {k+"Mapped": itermapper.Mapper(v) for k, v in o.items()} if isinstance(o, Node): o.name = 'm' + o.name[1] if o.next: return itermapper.Mapper(o.next) # resorts to newDefaultMapper if it cannot handle, then the oldDefaultMapper return o return itermapper.oldDefaultMapper(o) n1 = Node(name='n1') n2 = Node(name='n2') n1.next = n2 d = {'a': 1, 'b': n1} itermapper(d, default=newDefaultMapper) print(d)
{'aMapped': 1, 'bMapped': <__main__.Node at 0x....>}
print(n1.name, n2.name)
m1 m2
This functionality partially exists in the current versions of python in the JSON dumps function. Dumps iterates over all values in a nested data structure and it also accepts a JSONEncoder <https://docs.python.org/3/library/json.html#json.JSONEncoder> with a new default function to handle arbitrary types. No such functionality exists in other parts of python to map values in a nested data structure into values of another identical nested data structure. What I am proposing goes beyond this though to also handle keys. It also doesn't return JSON strings, but a new mapped data structure. Best, Sven On Wed, Jul 21, 2021 at 9:30 AM Paul Moore <p.f.moore@gmail.com> wrote:
You've given some examples of what flatten() does, but I still don't really understand how it's expected to work. Could you provide a specification, rather than just examples? Also, I've never needed a function like this in anything I've ever done - that may mean nothing, but it does suggest that you need to demonstrate that it's a generally useful tool (maybe by pointing to examples of "real world" code in various contexts that would be improved by using it). There's a lot of pressure to keep the stdlib for things that are widely applicable, and the burden of demonstrating that lies with the proposer.
As the proposal stands here, I honestly don't see anything that suggests this wouldn't be better as a standalone function published on PyPI.
Paul
On Wed, 21 Jul 2021 at 13:43, Sven Voigt <svenpvoigt@gmail.com> wrote:
Hello Python-ideas,
I am working with lots of JSON objects in my work and need to obtain
JSON pointers to particular components of a JSON object. These pointers are defined as part of the JSON-LD specifications as relative IRIs and as part of the JSON-Schema specifications. However, I am not aware of these pointers being part of the JSON specification. The pointers are really straightforward and follow the same syntax as other IRIs (URLs) or file paths. For example, the following flatten function illustrates the mapping of nested dictionaries and arrays to a single flat dictionary, where each nested key is mapped to a unique path.
d = {"a": 1, "b": {"ba": 2, "bb": [{"bba": 3, "bbb": None}]}} flatten(d)
{'a': 1, 'b/ba': 2, 'b/bb/0/bba': 3, 'b/bb/0/bbb': None}
Not only does this conversion help in generating JSON pointers, but it
helps with logic for nested data structures. Specifically, assume there is a dictionary, which includes nested dictionaries, lists, and tuples, and any of these elements are contained within another dictionary or that other dictionaries elements. Then it is not sufficient to simply compare the items of the two dictionaries to check whether the first is a subset of the other. However, the flat structures can be compared directly.
1. A nested dictionary is a subset of another nested dictionary. The
# Current {"a": 1, "c": {"d": 4}}.items() <= {"a": 1, "b": 2, "c": {"d": 4, "e": 5}}.items()
False # Proposed flatten({"a": 1, "c": {"d": 4}}).items() <= flatten({"a": 1, "b": 2, "c": {"d": 4, "e": 5}}).items() True
2. A nested list or tuple is a subset of a dictionary. # Current {"a": 1, "c": [3]}.items() <= {"a": 1, "b": 2, "c": [3,3]}.items()
False # Proposed flatten({"a": 1, "c": [3]}).items() <= flatten({"a": 1, "b": 2, "c": [3,3]}).items() True
Note that these examples only have one level of nesting, but the flatten function must handle any level of nesting. For the current version of the flatten function, I essentially borrowed the JSON "dumps" source code. However, the dumps code doesn't utilize any other functions and I believe
flatten function ensures the nested dictionaries are not checked for equivalence but subset of. there may be a missing "nestedIter" function in python. This function could be used for JSON dumps or flatten or anytime someone wants to visit all nested elements of a dictionary and perform some operation. Therefore, I think this function could be included in the JSON library if python-ideas thinks that the pointers are very specific to JSON or the itertools library, where the nested iteration function could be generally used for all nested data types.
Let me know what you think of this proposal and I am looking forward to
your responses.
Best, Sven _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/V2DNTL...
Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Jul 22, 2021 at 1:14 AM Sven Voigt <svenpvoigt@gmail.com> wrote:
Paul,
I agree that the mapping to a flat data structure might have limited use cases. However, like you said it is only one example. What I am proposing is a nested iteration tool that can map both keys and values to new keys and new values. It only considers nesting in dictionaries, lists, and tuples by default, but allows users to pass another function to continue searching for pointers in other types.
This sounds like the sort of thing that's best coded up specifically to your purposes. There'll be myriad small variants of this kind of traversal, so it's easiest to just write the thing you want, rather than try to get the standard library to support every variant. ChrisA
I could say the same thing about JSON dumps then and most itertools functions. The proposed functionality is in line with existing functionality and extends it to the case of nested data structures. On Wed, Jul 21, 2021 at 11:25 AM Chris Angelico <rosuav@gmail.com> wrote:
On Thu, Jul 22, 2021 at 1:14 AM Sven Voigt <svenpvoigt@gmail.com> wrote:
Paul,
I agree that the mapping to a flat data structure might have limited use
cases. However, like you said it is only one example. What I am proposing is a nested iteration tool that can map both keys and values to new keys and new values. It only considers nesting in dictionaries, lists, and tuples by default, but allows users to pass another function to continue searching for pointers in other types.
This sounds like the sort of thing that's best coded up specifically to your purposes. There'll be myriad small variants of this kind of traversal, so it's easiest to just write the thing you want, rather than try to get the standard library to support every variant.
ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2IOA23... Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Jul 22, 2021 at 1:28 AM Sven Voigt <svenpvoigt@gmail.com> wrote:
I could say the same thing about JSON dumps then and most itertools functions. The proposed functionality is in line with existing functionality and extends it to the case of nested data structures.
JSON itself is standardized, heavily used, and well supported in many languages. What you're saying here about pointers is far less widely used, and the act of flattening a nested dictionary doesn't seem to be a well known one. Can you point to other languages or libraries that have this feature as a built-in or standard library function? ChrisA
Maybe your examples aren't making it clear what you're suggesting, then. I'm in agreement with Chris, your "nested iteration tool" sounds like something that would have so many configuration options and parameters that it would be harder to use than traversing "by hand". But maybe you imagine something simpler than I do. Again, can I suggest you post what you imagine the user documentation for the function would look like, and then we'll be able to stop misunderstanding each other? Paul On Wed, 21 Jul 2021 at 16:29, Sven Voigt <svenpvoigt@gmail.com> wrote:
I could say the same thing about JSON dumps then and most itertools functions. The proposed functionality is in line with existing functionality and extends it to the case of nested data structures.
On Wed, Jul 21, 2021 at 11:25 AM Chris Angelico <rosuav@gmail.com> wrote:
On Thu, Jul 22, 2021 at 1:14 AM Sven Voigt <svenpvoigt@gmail.com> wrote:
Paul,
I agree that the mapping to a flat data structure might have limited use cases. However, like you said it is only one example. What I am proposing is a nested iteration tool that can map both keys and values to new keys and new values. It only considers nesting in dictionaries, lists, and tuples by default, but allows users to pass another function to continue searching for pointers in other types.
This sounds like the sort of thing that's best coded up specifically to your purposes. There'll be myriad small variants of this kind of traversal, so it's easiest to just write the thing you want, rather than try to get the standard library to support every variant.
ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2IOA23... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BRWPYX... Code of Conduct: http://python.org/psf/codeofconduct/
Side note: there is a library, scalpl, which does quite a lot of what you want with keys. But they use a slightly different syntax for list items. They might be amenable to adding a JSON schema dialect. On Wed, 21 Jul 2021, 4:46 pm Paul Moore, <p.f.moore@gmail.com> wrote:
Maybe your examples aren't making it clear what you're suggesting, then. I'm in agreement with Chris, your "nested iteration tool" sounds like something that would have so many configuration options and parameters that it would be harder to use than traversing "by hand".
But maybe you imagine something simpler than I do. Again, can I suggest you post what you imagine the user documentation for the function would look like, and then we'll be able to stop misunderstanding each other?
Paul
On Wed, 21 Jul 2021 at 16:29, Sven Voigt <svenpvoigt@gmail.com> wrote:
I could say the same thing about JSON dumps then and most itertools
functions. The proposed functionality is in line with existing functionality and extends it to the case of nested data structures.
On Wed, Jul 21, 2021 at 11:25 AM Chris Angelico <rosuav@gmail.com>
On Thu, Jul 22, 2021 at 1:14 AM Sven Voigt <svenpvoigt@gmail.com>
wrote:
Paul,
I agree that the mapping to a flat data structure might have limited
use cases. However, like you said it is only one example. What I am
wrote: proposing is a nested iteration tool that can map both keys and values to new keys and new values. It only considers nesting in dictionaries, lists, and tuples by default, but allows users to pass another function to continue searching for pointers in other types.
This sounds like the sort of thing that's best coded up specifically to your purposes. There'll be myriad small variants of this kind of traversal, so it's easiest to just write the thing you want, rather than try to get the standard library to support every variant.
ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2IOA23... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BRWPYX... Code of Conduct: http://python.org/psf/codeofconduct/
Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/UR52XK... Code of Conduct: http://python.org/psf/codeofconduct/
I would add that this seems to me like something you need. So be all means write it, document it, maybe publish it on PyPi, and then, if it does seem to be generally useful, propose it for the stdlib. In fact, if something like this were to be included in the stdlib, we’d probably want a back-port on PyPi anyway. Meanwhile, a few thoughts: 1) it likely makes sense to support Mappings and Iterables, rather than specific types. Though special casing strings, as they are usually treated at atomic in this context. 2) you talk about a use case of flattening to be subset checking. This strikes me as a special case that will require a custom class ( or functions) In particular, how do you deal with dict ordering? 3) is there really no existing JSON-LD lib for Python? Or not a good one? If so, then that’s clearly a package for PyPi, and maybe you’ll discover whether this flattening function is generally useful after it’s been battle tested. I for one, am interested. I’ve got a system built of nested Dataclasses that serialize to/from JSON compatible nested dicts and lists. This traversing/flattening functionality could be quite useful. -CHB On Wed, Jul 21, 2021 at 8:48 AM Paul Moore <p.f.moore@gmail.com> wrote:
Maybe your examples aren't making it clear what you're suggesting, then. I'm in agreement with Chris, your "nested iteration tool" sounds like something that would have so many configuration options and parameters that it would be harder to use than traversing "by hand".
But maybe you imagine something simpler than I do. Again, can I suggest you post what you imagine the user documentation for the function would look like, and then we'll be able to stop misunderstanding each other?
Paul
On Wed, 21 Jul 2021 at 16:29, Sven Voigt <svenpvoigt@gmail.com> wrote:
I could say the same thing about JSON dumps then and most itertools
functions. The proposed functionality is in line with existing functionality and extends it to the case of nested data structures.
On Wed, Jul 21, 2021 at 11:25 AM Chris Angelico <rosuav@gmail.com>
On Thu, Jul 22, 2021 at 1:14 AM Sven Voigt <svenpvoigt@gmail.com>
wrote:
Paul,
I agree that the mapping to a flat data structure might have limited
use cases. However, like you said it is only one example. What I am
wrote: proposing is a nested iteration tool that can map both keys and values to new keys and new values. It only considers nesting in dictionaries, lists, and tuples by default, but allows users to pass another function to continue searching for pointers in other types.
This sounds like the sort of thing that's best coded up specifically to your purposes. There'll be myriad small variants of this kind of traversal, so it's easiest to just write the thing you want, rather than try to get the standard library to support every variant.
ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2IOA23... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BRWPYX... Code of Conduct: http://python.org/psf/codeofconduct/
Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/UR52XK... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
Thank you so much for all of your responses. I am not aware of the procedure for suggesting ideas but building a PyPI package with documentation and then sharing here seems like a good way to build this out (which I will get started on). Also, scalpl looks really cool, although I am not sure it does any mapping or support non-standard data types (will check it out more). Also, if this wasn't clear before, my interpretation of the JSON library's dump function was that it was a very specific implementation of an otherwise generic nested iteration technique for mapping nested data structures from a domain to the JSON domain, where the mapping may not be invertible. Once generically implemented, JSON dumps would use the nested iteration functionality to map any nested data structure onto the JSON domain, the string representation of the resulting structure would be a valid JSON string. So, this should be a generalization of an existing method to map any nested data structure between different domains, where the mapping could be altering dictionary keys, the type of a value or object, or traversing other nested data types. On Wed, Jul 21, 2021 at 12:38 PM Christopher Barker <pythonchb@gmail.com> wrote:
I would add that this seems to me like something you need.
So be all means write it, document it, maybe publish it on PyPi, and then, if it does seem to be generally useful, propose it for the stdlib.
In fact, if something like this were to be included in the stdlib, we’d probably want a back-port on PyPi anyway.
Meanwhile, a few thoughts:
1) it likely makes sense to support Mappings and Iterables, rather than specific types. Though special casing strings, as they are usually treated at atomic in this context.
2) you talk about a use case of flattening to be subset checking. This strikes me as a special case that will require a custom class ( or functions) In particular, how do you deal with dict ordering?
3) is there really no existing JSON-LD lib for Python? Or not a good one? If so, then that’s clearly a package for PyPi, and maybe you’ll discover whether this flattening function is generally useful after it’s been battle tested.
I for one, am interested. I’ve got a system built of nested Dataclasses that serialize to/from JSON compatible nested dicts and lists.
This traversing/flattening functionality could be quite useful.
-CHB
On Wed, Jul 21, 2021 at 8:48 AM Paul Moore <p.f.moore@gmail.com> wrote:
Maybe your examples aren't making it clear what you're suggesting, then. I'm in agreement with Chris, your "nested iteration tool" sounds like something that would have so many configuration options and parameters that it would be harder to use than traversing "by hand".
But maybe you imagine something simpler than I do. Again, can I suggest you post what you imagine the user documentation for the function would look like, and then we'll be able to stop misunderstanding each other?
Paul
On Wed, 21 Jul 2021 at 16:29, Sven Voigt <svenpvoigt@gmail.com> wrote:
I could say the same thing about JSON dumps then and most itertools
functions. The proposed functionality is in line with existing functionality and extends it to the case of nested data structures.
On Wed, Jul 21, 2021 at 11:25 AM Chris Angelico <rosuav@gmail.com>
On Thu, Jul 22, 2021 at 1:14 AM Sven Voigt <svenpvoigt@gmail.com>
wrote:
Paul,
I agree that the mapping to a flat data structure might have limited
use cases. However, like you said it is only one example. What I am
wrote: proposing is a nested iteration tool that can map both keys and values to new keys and new values. It only considers nesting in dictionaries, lists, and tuples by default, but allows users to pass another function to continue searching for pointers in other types.
This sounds like the sort of thing that's best coded up specifically to your purposes. There'll be myriad small variants of this kind of traversal, so it's easiest to just write the thing you want, rather than try to get the standard library to support every variant.
ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2IOA23... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BRWPYX... Code of Conduct: http://python.org/psf/codeofconduct/
Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/UR52XK... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD (Chris)
Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Thu, Jul 22, 2021 at 5:22 AM Sven Voigt <svenpvoigt@gmail.com> wrote:
Thank you so much for all of your responses. I am not aware of the procedure for suggesting ideas but building a PyPI package with documentation and then sharing here seems like a good way to build this out (which I will get started on). Also, scalpl looks really cool, although I am not sure it does any mapping or support non-standard data types (will check it out more).
Also, if this wasn't clear before, my interpretation of the JSON library's dump function was that it was a very specific implementation of an otherwise generic nested iteration technique for mapping nested data structures from a domain to the JSON domain, where the mapping may not be invertible. Once generically implemented, JSON dumps would use the nested iteration functionality to map any nested data structure onto the JSON domain, the string representation of the resulting structure would be a valid JSON string. So, this should be a generalization of an existing method to map any nested data structure between different domains, where the mapping could be altering dictionary keys, the type of a value or object, or traversing other nested data types.
Beware of the trap of overgeneralization. https://thedailywtf.com/articles/The_Inner-Platform_Effect If you try to make a generic way to map any data structure to any other, you'll end up with a function so utterly generic that the means of telling it what to do... is basically just the same code that would do the mapping manually. At best, what you'll be doing is reinventing generic tools like map(). Sometimes, it's best to just write the code you need, when you need it, instead of trying to make something for every possible use-case. ChrisA
participants (5)
-
Chris Angelico -
Christopher Barker -
Oliver Margetts -
Paul Moore -
Sven Voigt