thes mapping data type and thescfg cfg file module - review and suggestion request
If you are not aware: - Thesaurus is a mapping data type with recursive keypath map and attribute aliasing. It is a subclass of dict() and is mostly compatible as a general use dictionary replacement. - ThesaurusExtended is a subclass of Thesaurus providing additional usability methods such as recursive key and value searching. - ThesaurusCfg is a subclass of ThesaurusExtended providing a nested key configuration file parser and per key data coercion methods. The README.rsl will give a better idea: https://git.cinege.com/thesaurus/ After 7 years I might have reached the point of 'interesting' with my Thesaurus and ThesaurusCfg modules. To anyone that is overly bored, I'd appreciate your terse review and comments on how mundane and worthless they still actually are. :-) I'm primarily interested in suggestions to what I've done conceptually here. While I'm not completely ashamed of the state of the Thesaurus code, ThesaurusCfg is not far beyond the original few hours I slapped it together one day in frustration. After considering suggestions I intend to make changes towards a formal release. Thanks in advance, Dave
Hi Dave, Thesaurus looks interesting and it is obvious that you've put a lot of effort into it! This list is for the discussion of the development *of* Python itself, however, rather than development *with* Python, so it's not an appropriate place for such posts. I suggest you post this on python-list and/or python-announce, to get this in front of a wider audience. - Tal Einat On Mon, Nov 18, 2019 at 7:17 AM Dave Cinege <dave@cinege.com> wrote:
If you are not aware:
- Thesaurus is a mapping data type with recursive keypath map and attribute aliasing. It is a subclass of dict() and is mostly compatible as a general use dictionary replacement.
- ThesaurusExtended is a subclass of Thesaurus providing additional usability methods such as recursive key and value searching.
- ThesaurusCfg is a subclass of ThesaurusExtended providing a nested key configuration file parser and per key data coercion methods.
The README.rsl will give a better idea: https://git.cinege.com/thesaurus/
After 7 years I might have reached the point of 'interesting' with my Thesaurus and ThesaurusCfg modules.
To anyone that is overly bored, I'd appreciate your terse review and comments on how mundane and worthless they still actually are. :-)
I'm primarily interested in suggestions to what I've done conceptually here. While I'm not completely ashamed of the state of the Thesaurus code, ThesaurusCfg is not far beyond the original few hours I slapped it together one day in frustration.
After considering suggestions I intend to make changes towards a formal release.
Thanks in advance,
Dave _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CTXKJUTT... Code of Conduct: http://python.org/psf/codeofconduct/
Hello Tal, Yes I understand that. I posted this here because it's been suggested by those less in the know that Thesaurus and ThesaurusCfg (or some of the the concepts contained in them) might have a place in the future mainline. Remember that Thesaurus is a data type and ThesaurusCfg is a alternative to configparser. Additionally I could bring this full circle to something that integrates with or provides an alternative to argparse. (providing an integrated cfg file and command line argument solution which currently does not exist.) I hope with this in mind this thread is considered on-topic and I'd highly value any feedback from the 'serious people' on this list if they should take a look. Dave On 2019/11/18 08:51, Tal Einat wrote:
Hi Dave,
Thesaurus looks interesting and it is obvious that you've put a lot of effort into it!
This list is for the discussion of the development *of* Python itself, however, rather than development *with* Python, so it's not an appropriate place for such posts.
I suggest you post this on python-list and/or python-announce, to get this in front of a wider audience.
- Tal Einat
On Mon, Nov 18, 2019 at 7:17 AM Dave Cinege <dave@cinege.com <mailto:dave@cinege.com>> wrote:
If you are not aware:
- Thesaurus is a mapping data type with recursive keypath map and attribute aliasing. It is a subclass of dict() and is mostly compatible as a general use dictionary replacement.
- ThesaurusExtended is a subclass of Thesaurus providing additional usability methods such as recursive key and value searching.
- ThesaurusCfg is a subclass of ThesaurusExtended providing a nested key configuration file parser and per key data coercion methods.
The README.rsl will give a better idea: https://git.cinege.com/thesaurus/
After 7 years I might have reached the point of 'interesting' with my Thesaurus and ThesaurusCfg modules.
To anyone that is overly bored, I'd appreciate your terse review and comments on how mundane and worthless they still actually are. :-)
I'm primarily interested in suggestions to what I've done conceptually here. While I'm not completely ashamed of the state of the Thesaurus code, ThesaurusCfg is not far beyond the original few hours I slapped it together one day in frustration.
After considering suggestions I intend to make changes towards a formal release.
Thanks in advance,
Dave _______________________________________________ Python-Dev mailing list -- python-dev@python.org <mailto:python-dev@python.org> To unsubscribe send an email to python-dev-leave@python.org <mailto:python-dev-leave@python.org> https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CTXKJUTT... Code of Conduct: http://python.org/psf/codeofconduct/
On Mon, Nov 18, 2019 at 5:30 PM Dave Cinege <dave@cinege.com> wrote:
Hello Tal,
Yes I understand that. I posted this here because it's been suggested by those less in the know that Thesaurus and ThesaurusCfg (or some of the the concepts contained in them) might have a place in the future mainline.
Ah, that's something else then. I didn't understand that from your original post. Thanks for clarifying!
Remember that Thesaurus is a data type and ThesaurusCfg is a alternative to configparser. Additionally I could bring this full circle to something that integrates with or provides an alternative to argparse. (providing an integrated cfg file and command line argument solution which currently does not exist.)
These days, thanks to pip and PyPI, anyone can publish libraries and it is easy for developers to find and use them if they like. There is no longer a need to add things to the stdlib just to make them widely available. On the other hand, bringing something into the stdlib means undertaking an obligation by the Python core devs to maintain it for a very long period of time. It also ties their releases to the Python release cycle, which is not appropriate for many libraries. Therefore, our current policy for additions to the standard library is to add as few things as possible. We've recently been adding to the standard library only in special cases, such as wanting to use something in the of the standard library. With Thesaurus I can't currently think of a reason for such special consideration.
I hope with this in mind this thread is considered on-topic and I'd highly value any feedback from the 'serious people' on this list if they should take a look.
We are, after all, a group of people developing a language named after Monty Python. You will find very few 'serious people' here! ;) - Tal Einat
Tal, On 2019/11/18 10:59, Tal Einat wrote:
These days, thanks to pip and PyPI, anyone can publish libraries and it is easy for developers to find and use them if they like. There is no longer a need to add things to the stdlib just to make them widely available.
Fair enough, only that you possibly neglected to remember the benefits to my ego if I get something accepted into mainline. :-D Framing this a little more specifically: Aside from my original Thesaurus release in Dec 2012, there have been many many attempts at extended/recursive dictionary objects. The use case for such a thing is well established. (JSON maybe enough to mention) I would argue that a new standardized mapping datatype, possibly (some form of) Thesaurus, could be warranted in stdlib. At this stage I consider Thesaurus 'serious' and 'interesting' because: - I've been using some version of it for 7 years in my own production code; many of the concepts in it are mature. - It's tightly wrapped around and highly compatible with dict. - I've paid close attention to performance - The recent extended features for tree searches and the slicing you can do with 'keypaths' (my own term) appears novel. I am also sure that some will consider things I currently do in Thesaurus complete heresy. For now I'll point to ThesaurusCfg as an actually use case. (But still consider my use of coercion methods really cool :-) My goal is to attempt to take Thesaurus toward a standardized or de facto recursive mapping object and I welcome any dialog that results. Dave
On Tue., 19 Nov. 2019, 7:32 am Dave Cinege, <dave@cinege.com> wrote:
Tal,
On 2019/11/18 10:59, Tal Einat wrote:
These days, thanks to pip and PyPI, anyone can publish libraries and it is easy for developers to find and use them if they like. There is no longer a need to add things to the stdlib just to make them widely available.
Fair enough, only that you possibly neglected to remember the benefits to my ego if I get something accepted into mainline. :-D
Framing this a little more specifically:
Aside from my original Thesaurus release in Dec 2012, there have been many many attempts at extended/recursive dictionary objects. The use case for such a thing is well established. (JSON maybe enough to mention)
I would argue that a new standardized mapping datatype, possibly (some form of) Thesaurus, could be warranted in stdlib.
At this stage I consider Thesaurus 'serious' and 'interesting' because:
- I've been using some version of it for 7 years in my own production code; many of the concepts in it are mature.
- It's tightly wrapped around and highly compatible with dict.
- I've paid close attention to performance
- The recent extended features for tree searches and the slicing you can do with 'keypaths' (my own term) appears novel.
I think there's one potential avenue for making the case that a new JSON-centric data type would be worth including: framing it as an alternative to the None-aware operator proposal in PEP 505. It may still not be successful, but ad hoc data manipulation has been the most compelling use case presented for that PEP so far, and a standard library addition would be a less intrusive change than a syntax update. So if you wanted to pursue this idea further, my suggestions would be: * post a thread on python-ideas seeking advice on building a credible case for improving arbitrarily nested data container support in the standard library, as we currently make it easy to build them, but not so easy to destructure them later. (Replies to my suggestions below would also be better directed to python-ideas) * review PEP 505 and consider how the library might help with the use cases mentioned there * review other APIs like https://glom.readthedocs.io/en/latest/ and consider the trade-offs between a data manipulation library that can work with existing libraries that produce plain dictionaries and one that requires a new data structure * in the same vein, review whether it might be possible to use a "dict wrapper" approach over a "dict subclass" approach (many potential users will balk at a data copy operation, but would be amenable to a ChainMap style wrapper) * consider your options for tackling the naming conflict with https://libraries.io/pypi/thesaurus (I must admit, "nested data structure manipulation" is not what springs to mind when I hear "thesaurus". I'm far more likely to think of synonyms, antonyms, and natural language processing) Cheers, Nick.
It won't be easy to add this to the stdlib. One thing that bugs me in particular is that you can access key via '.attr' notation (the example shows t['a']['b']['c']['d'] as equivalent to t.a.b.c.d). This feels problematic: What should happen if the key happens to be the name of a method (e.g. .keys or .update)? The choices you have are either to mask the method or to mask the key. Neither solution seems ideal. JavaScript has this equivalence and it makes me very uncomfortable. I'm not denying that Thesaurus[Cfg] looks useful. But, like Tal, I must stress that that's not enough to consider inclusion in the stdlib. On Mon, Nov 18, 2019 at 4:25 PM Dave Cinege <dave@cinege.com> wrote:
Hello Tal,
Yes I understand that. I posted this here because it's been suggested by those less in the know that Thesaurus and ThesaurusCfg (or some of the the concepts contained in them) might have a place in the future mainline.
Remember that Thesaurus is a data type and ThesaurusCfg is a alternative to configparser. Additionally I could bring this full circle to something that integrates with or provides an alternative to argparse. (providing an integrated cfg file and command line argument solution which currently does not exist.)
I hope with this in mind this thread is considered on-topic and I'd highly value any feedback from the 'serious people' on this list if they should take a look.
Dave
Hi Dave,
Thesaurus looks interesting and it is obvious that you've put a lot of effort into it!
This list is for the discussion of the development *of* Python itself, however, rather than development *with* Python, so it's not an appropriate place for such posts.
I suggest you post this on python-list and/or python-announce, to get this in front of a wider audience.
- Tal Einat
On Mon, Nov 18, 2019 at 7:17 AM Dave Cinege <dave@cinege.com <mailto:dave@cinege.com>> wrote:
If you are not aware:
- Thesaurus is a mapping data type with recursive keypath map and attribute aliasing. It is a subclass of dict() and is mostly compatible as a general use dictionary replacement.
- ThesaurusExtended is a subclass of Thesaurus providing additional usability methods such as recursive key and value searching.
- ThesaurusCfg is a subclass of ThesaurusExtended providing a nested key configuration file parser and per key data coercion methods.
The README.rsl will give a better idea: https://git.cinege.com/thesaurus/
After 7 years I might have reached the point of 'interesting' with my Thesaurus and ThesaurusCfg modules.
To anyone that is overly bored, I'd appreciate your terse review and comments on how mundane and worthless they still actually are. :-)
I'm primarily interested in suggestions to what I've done conceptually here. While I'm not completely ashamed of the state of the Thesaurus code, ThesaurusCfg is not far beyond the original few hours I slapped it together one day in frustration.
After considering suggestions I intend to make changes towards a
On 2019/11/18 08:51, Tal Einat wrote: formal
release.
Thanks in advance,
Dave _______________________________________________ Python-Dev mailing list -- python-dev@python.org <mailto:python-dev@python.org> To unsubscribe send an email to python-dev-leave@python.org <mailto:python-dev-leave@python.org> https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/CTXKJUTT...
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VXA7EZYV... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
Guido, Thank you for your generous comments. All of them. On 2019/11/18 15:33, Guido van Rossum wrote:
It won't be easy to add this to the stdlib.
Maybe there are other existing threads that considered a new recursive mapping object for stblib that someone can point me to?
One thing that bugs me in particular is that you can access key via '.attr' notation (the example shows t['a']['b']['c']['d'] as equivalent to t.a.b.c.d). This feels problematic: What should happen if the key happens to be the name of a method (e.g. .keys or .update)? The choices you have are either to mask the method or to mask the key. Neither solution seems ideal. JavaScript has this equivalence and it makes me very uncomfortable.
Well, let's see:
t.set_path('a.b.c.update', 'Hello') t.a.b.c.update <built-in method update of Thesaurus object at 0x7f78b99df8e0> t.a.b.c['update'] 'Hello'
Attribute aliasing has limitations the coder must understand. Thesaurus attempts to deal with them gracefully. When in doubt normal key names should always work (including keys with dots in them) and if they don't I have something to fix. With the focus on being a datatype primitive I consider anything more to be the job of higher level modules. (ThesaurusCfg being an example) The benefit of course is dramatically cleaner code for heavily nested objects and recursive string replacement compatibility. (f-string, printf, format, template) And with that said, if the only obstacle to a new recursive mapping object for stblib is attribute aliasing, then it can be removed. Dave
I'm not denying that Thesaurus[Cfg] looks useful. But, like Tal, I must stress that that's not enough to consider inclusion in the stdlib.
On Mon, Nov 18, 2019 at 4:25 PM Dave Cinege <dave@cinege.com <mailto:dave@cinege.com>> wrote:
Hello Tal,
Yes I understand that. I posted this here because it's been suggested by those less in the know that Thesaurus and ThesaurusCfg (or some of the the concepts contained in them) might have a place in the future mainline.
Remember that Thesaurus is a data type and ThesaurusCfg is a alternative to configparser. Additionally I could bring this full circle to something that integrates with or provides an alternative to argparse. (providing an integrated cfg file and command line argument solution which currently does not exist.)
I hope with this in mind this thread is considered on-topic and I'd highly value any feedback from the 'serious people' on this list if they should take a look.
Dave
On 2019/11/18 08:51, Tal Einat wrote: > Hi Dave, > > Thesaurus looks interesting and it is obvious that you've put a lot of > effort into it! > > This list is for the discussion of the development *of* Python itself, > however, rather than development *with* Python, so it's not an > appropriate place for such posts. > > I suggest you post this on python-list and/or python-announce, to get > this in front of a wider audience. > > - Tal Einat > > On Mon, Nov 18, 2019 at 7:17 AM Dave Cinege <dave@cinege.com <mailto:dave@cinege.com> > <mailto:dave@cinege.com <mailto:dave@cinege.com>>> wrote: > > If you are not aware: > > - Thesaurus is a mapping data type with recursive keypath map > and attribute aliasing. It is a subclass of dict() and is mostly > compatible as a general use dictionary replacement. > > - ThesaurusExtended is a subclass of Thesaurus providing additional > usability methods such as recursive key and value searching. > > - ThesaurusCfg is a subclass of ThesaurusExtended providing a nested > key configuration file parser and per key data coercion methods. > > The README.rsl will give a better idea: > https://git.cinege.com/thesaurus/ > > > After 7 years I might have reached the point of 'interesting' with > my Thesaurus and ThesaurusCfg modules. > > To anyone that is overly bored, I'd appreciate your terse review and > comments on how mundane and worthless they still actually are. :-) > > I'm primarily interested in suggestions to what I've done conceptually > here. While I'm not completely ashamed of the state of the Thesaurus > code, ThesaurusCfg is not far beyond the original few hours I > slapped it > together one day in frustration. > > After considering suggestions I intend to make changes towards a formal > release. > > Thanks in advance, > > Dave > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org <mailto:python-dev@python.org> > <mailto:python-dev@python.org <mailto:python-dev@python.org>> > To unsubscribe send an email to python-dev-leave@python.org <mailto:python-dev-leave@python.org> > <mailto:python-dev-leave@python.org <mailto:python-dev-leave@python.org>> > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/CTXKJUTT... > Code of Conduct: http://python.org/psf/codeofconduct/ > _______________________________________________ Python-Dev mailing list -- python-dev@python.org <mailto:python-dev@python.org> To unsubscribe send an email to python-dev-leave@python.org <mailto:python-dev-leave@python.org> https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VXA7EZYV... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido <http://python.org/~guido>) /Pronouns: he/him //(why is my pronoun here?)/ <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
participants (4)
-
Dave Cinege -
Guido van Rossum -
Nick Coghlan -
Tal Einat