Hi, I first want to thank everyone in the community for the contributions over the years. I know the idea of a frozendict has been proposed before and rejected. I have a use case for a frozendict implementation that to my knowledge was not discussed during previous debates. My reasoning for a frozendict class stems from patterns I typically see arise when performing ETL or data integrations and conversions. I generally have used MappingProxyType as a way to set default mapping to a function or to set an empty mapping to a function. I've created a gist with an example use case: https://gist.github.com/pmart123/493edf84d9aa61691ca7321325ebb6ab I've included an example of what code typically looks like when using MappingProxyType and what it could look like with a frozendict implementation. I believe this use case may also be under-reported in open source code as it often crops up when integrating third-party data sources, which at times can't be open sourced due to licensing issues. I would love to hear if anyone has used MappingProxyType in a similar manner, or if this use case could help warrant a frozendict in the standard library.
How does a frozendict help in that example? It's not obvious to me. Despite not understanding that example, I'm +1 for having a frozendict. I don't think it'll increase cognitive load much, as it'll sit right next to frozenset when someone reads the builtins in alphabetical order. In my own experience, I want to use a dict as a dict key about once or twice a year. It'd be nice to have a quick way to convert to a frozendict. On Wed, Oct 10, 2018 at 10:05 AM Philip Martin <philip.martin2007@gmail.com> wrote:
Hi, I first want to thank everyone in the community for the contributions over the years. I know the idea of a frozendict has been proposed before and rejected. I have a use case for a frozendict implementation that to my knowledge was not discussed during previous debates. My reasoning for a frozendict class stems from patterns I typically see arise when performing ETL or data integrations and conversions. I generally have used MappingProxyType as a way to set default mapping to a function or to set an empty mapping to a function. I've created a gist with an example use case:
https://gist.github.com/pmart123/493edf84d9aa61691ca7321325ebb6ab
I've included an example of what code typically looks like when using MappingProxyType and what it could look like with a frozendict implementation. I believe this use case may also be under-reported in open source code as it often crops up when integrating third-party data sources, which at times can't be open sourced due to licensing issues. I would love to hear if anyone has used MappingProxyType in a similar manner, or if this use case could help warrant a frozendict in the standard library. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Hi Philiip, and welcome, On Wed, Oct 10, 2018 at 12:04:48PM -0500, Philip Martin wrote:
I generally have used MappingProxyType as a way to set default mapping to a function or to set an empty mapping to a function.
I've created a gist with an example use case:
https://gist.github.com/pmart123/493edf84d9aa61691ca7321325ebb6ab
Please try to keep the discussion in one place (i.e. here), for the benefit of the archives and for those who have email access but not unrestricted web access. Can you explain (in English) your use-case, and why MappingProxyType isn't suitable? If it *is* suitable, how does your proposal differ? If the only proposal is to rename types.MappingProxyType to a builtin "frozendict", that's one thing; if the proposal is something else, you should explain what.
I've included an example of what code typically looks like when using MappingProxyType and what it could look like with a frozendict implementation.
Wouldn't that be just: from types import MappingProxyType as frozendict d = frozendict({'spam': 1, 'eggs': 2}) versus: d = frozendict({'spam': 1, 'eggs': 2}) Apart from the initial import, how would they be different? You want a frozendict; the existing MappingProxyType provides a frozendict (with a surprising name, but never mind...). Wouldn't you use them exactly the same way? They both (I presume) offer precisely the same read-only access to the mapping interface. -- Steve
Steven, that's a great idea, and I would be 100% up for your suggestion to have types.MappingProxyType renamed to frozendict. However, the differences in the behavior of MappingProxyType's constructor versus dict's would make the API's behavior confusing IMO. For example, MappingProxyType(x=5, y=10) throws a TypeError. I don't think most people would expect this. MappingProxyType to me though does seem to be a non-obvious name compared to say frozenset as you have mentioned. Plus, it's included in a module that I would say is very low level alongside functions like prepare_class, new_class, etc. On Wed, Oct 10, 2018 at 6:24 PM Steven D'Aprano <steve@pearwood.info> wrote:
Hi Philiip, and welcome,
On Wed, Oct 10, 2018 at 12:04:48PM -0500, Philip Martin wrote:
I generally have used MappingProxyType as a way to set default mapping to a function or to set an empty mapping to a function.
I've created a gist with an example use case:
https://gist.github.com/pmart123/493edf84d9aa61691ca7321325ebb6ab
Please try to keep the discussion in one place (i.e. here), for the benefit of the archives and for those who have email access but not unrestricted web access.
Can you explain (in English) your use-case, and why MappingProxyType isn't suitable? If it *is* suitable, how does your proposal differ?
If the only proposal is to rename types.MappingProxyType to a builtin "frozendict", that's one thing; if the proposal is something else, you should explain what.
I've included an example of what code typically looks like when using MappingProxyType and what it could look like with a frozendict implementation.
Wouldn't that be just:
from types import MappingProxyType as frozendict d = frozendict({'spam': 1, 'eggs': 2})
versus:
d = frozendict({'spam': 1, 'eggs': 2})
Apart from the initial import, how would they be different? You want a frozendict; the existing MappingProxyType provides a frozendict (with a surprising name, but never mind...). Wouldn't you use them exactly the same way? They both (I presume) offer precisely the same read-only access to the mapping interface.
-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 10Oct2018 20:25, Philip Martin <philip.martin2007@gmail.com> wrote:
Steven, that's a great idea, and I would be 100% up for your suggestion to have types.MappingProxyType renamed to frozendict.
I'm not for the rename, myself. Though I'd not be against a frozendict factory in builtins, a tiny shim for MappingProxyType.
However, the differences in the behavior of MappingProxyType's constructor versus dict's would make the API's behavior confusing IMO. For example, MappingProxyType(x=5, y=10) throws a TypeError. I don't think most people would expect this.
Well, if it were called frozendict, indeed not. It should act like dict. So: def frozendict(**kw): return MappingProxyType(kw) You could make an argument for that (or a slightly heftier version accepting the various things dict accepts). Or... you could just keep such a thing in your personal kit as a trivial way to spell "frozendict". One could argue for the above as a nice example to live in the docs perhaps. But not everything needs a special name. Cheers, Cameron Simpson <cs@cskk.id.au>
On Thu, Oct 11, 2018 at 1:02 PM Cameron Simpson <cs@cskk.id.au> wrote:
On 10Oct2018 20:25, Philip Martin <philip.martin2007@gmail.com> wrote:
Steven, that's a great idea, and I would be 100% up for your suggestion to have types.MappingProxyType renamed to frozendict.
I'm not for the rename, myself. Though I'd not be against a frozendict factory in builtins, a tiny shim for MappingProxyType.
However, the differences in the behavior of MappingProxyType's constructor versus dict's would make the API's behavior confusing IMO. For example, MappingProxyType(x=5, y=10) throws a TypeError. I don't think most people would expect this.
Well, if it were called frozendict, indeed not. It should act like dict.
So:
def frozendict(**kw): return MappingProxyType(kw)
You could make an argument for that (or a slightly heftier version accepting the various things dict accepts). Or... you could just keep such a thing in your personal kit as a trivial way to spell "frozendict". One could argue for the above as a nice example to live in the docs perhaps.
But not everything needs a special name.
Point of note: A mapping proxy is NOT immutable; it is merely read-only.
from types import MappingProxyType d = {'a':1, 'b':2} p = MappingProxyType(d) p mappingproxy({'a': 1, 'b': 2}) d['a'] = 3 p mappingproxy({'a': 3, 'b': 2})
A frozendict type, if it's meant to parallel frozenset, ought to be hashable (subject to the hashability of its members, of course), but you can't just take MPT and toss in a __hash__ method. No idea how important that use-case is, but Michael Selik mentioned "want[ing] to use a dict as a dict key about once or twice a year", which MPT is not able to do. Interestingly, frozenset isn't a subclass of set. I was going to say that a frozendict ought to be a dict, but maybe that isn't so important. Which might make a simple pure-Python frozendict actually pretty easy. ChrisA
That is interesting. From my recollection, when OrderedDict was reimplemented in C, there was advice on the thread to not implement it as a subclass of dict. https://bugs.python.org/issue16991 I'm far from the right person to comment on the exact reasons, but perhaps frozenset being decoupled from set gives precedence for a frozendict not subclassing dict? The methods clear, pop, popitem, update and setdefault would not be necessary. On Wed, Oct 10, 2018 at 9:28 PM Chris Angelico <rosuav@gmail.com> wrote:
On Thu, Oct 11, 2018 at 1:02 PM Cameron Simpson <cs@cskk.id.au> wrote:
On 10Oct2018 20:25, Philip Martin <philip.martin2007@gmail.com> wrote:
Steven, that's a great idea, and I would be 100% up for your suggestion
to
have types.MappingProxyType renamed to frozendict.
I'm not for the rename, myself. Though I'd not be against a frozendict factory in builtins, a tiny shim for MappingProxyType.
However, the differences in the behavior of MappingProxyType's constructor versus dict's would make the API's behavior confusing IMO. For example, MappingProxyType(x=5, y=10) throws a TypeError. I don't think most people would expect this.
Well, if it were called frozendict, indeed not. It should act like dict.
So:
def frozendict(**kw): return MappingProxyType(kw)
You could make an argument for that (or a slightly heftier version accepting the various things dict accepts). Or... you could just keep such a thing in your personal kit as a trivial way to spell "frozendict". One could argue for the above as a nice example to live in the docs perhaps.
But not everything needs a special name.
Point of note: A mapping proxy is NOT immutable; it is merely read-only.
from types import MappingProxyType d = {'a':1, 'b':2} p = MappingProxyType(d) p mappingproxy({'a': 1, 'b': 2}) d['a'] = 3 p mappingproxy({'a': 3, 'b': 2})
A frozendict type, if it's meant to parallel frozenset, ought to be hashable (subject to the hashability of its members, of course), but you can't just take MPT and toss in a __hash__ method. No idea how important that use-case is, but Michael Selik mentioned "want[ing] to use a dict as a dict key about once or twice a year", which MPT is not able to do.
Interestingly, frozenset isn't a subclass of set. I was going to say that a frozendict ought to be a dict, but maybe that isn't so important. Which might make a simple pure-Python frozendict actually pretty easy.
ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Oct 11, 2018 at 01:27:50PM +1100, Chris Angelico wrote: [...] [Cameron Simpson]
Well, if it were called frozendict, indeed not. It should act like dict.
So:
def frozendict(**kw): return MappingProxyType(kw)
You could make an argument for that (or a slightly heftier version accepting the various things dict accepts).
How would we make the opposite argument? That [frozen]dict *shouldn't* accept the same constructions that dict actually does? Seems to me that since we've proven the utility of the various constructor signatures that regular dicts already support, the addition of "read-only" doesn't change that. Now for the limited use-cases that MappingProxyType was originally designed for, as a wrapper around an existing dict, it made historical sense for it to only support a single dict argument. After all, it is a proxy, so there needs to be a dict for it to proxy. But if we promoted it to a real mapping, not just a proxy, or introduced a new read-only mapping, it seems to me that the full dict constructor interface ought to be a no-brainer.
But not everything needs a special name.
Point of note: A mapping proxy is NOT immutable; it is merely read-only. [...] A frozendict type, if it's meant to parallel frozenset, ought to be hashable (subject to the hashability of its members, of course)
Good point! That's exactly the sort of difference between MappingProxyType and a true frozendict that I was looking for. So there's still a use-case for a true frozendict.
Interestingly, frozenset isn't a subclass of set. I was going to say that a frozendict ought to be a dict, but maybe that isn't so important.
If there were a subclass relationship between frozendict and dict, it ought to be equivalent to this: py> issubclass(collections.MutableMapping, collections.Mapping) True but having two independent types works for me too.
Which might make a simple pure-Python frozendict actually pretty easy.
There's historical precedent: sets (and frozensets?) were initially introduced as a std library module, and only later elevated to builtins. I would be +1 for a frozendict module in 3.8, and if it proves sufficient useful to elevate to a builtin, that can happen in 3.9 or above. -- Steve
Cameron, That's a good suggestion. Ultimately, if there are not enough various use cases for a frozendict class, I think we could add something like this as an example recipe similar to the recipe section in itertools. I would be hesitant to add a quick shim to the standard library as I can't think of another instance where a developer calls a function expecting a specific class, and receives a different class. I'm happy to draft up some documentation if we decide to take this route because there aren't enough use cases. It would be great though to hear what other use cases developers have for a frozendict to ultimately decide whether this is the case. On Wed, Oct 10, 2018 at 9:01 PM Cameron Simpson <cs@cskk.id.au> wrote:
On 10Oct2018 20:25, Philip Martin <philip.martin2007@gmail.com> wrote:
Steven, that's a great idea, and I would be 100% up for your suggestion to have types.MappingProxyType renamed to frozendict.
I'm not for the rename, myself. Though I'd not be against a frozendict factory in builtins, a tiny shim for MappingProxyType.
However, the differences in the behavior of MappingProxyType's constructor versus dict's would make the API's behavior confusing IMO. For example, MappingProxyType(x=5, y=10) throws a TypeError. I don't think most people would expect this.
Well, if it were called frozendict, indeed not. It should act like dict.
So:
def frozendict(**kw): return MappingProxyType(kw)
You could make an argument for that (or a slightly heftier version accepting the various things dict accepts). Or... you could just keep such a thing in your personal kit as a trivial way to spell "frozendict". One could argue for the above as a nice example to live in the docs perhaps.
But not everything needs a special name.
Cheers, Cameron Simpson <cs@cskk.id.au>
It would help over using a regular dict as a default argument to a function by preventing accidental mutation of the default or constant mapping. This is a quickly contrived example of the convert_price function now having a side effect by changing the translation_map. from unicodedata import normalize prices = [{'croissant': 1}, {'coffee': 3}] translation_map = {'apple': 'pomme', 'coffee': 'café'} def normalize_text(s): return normalize('NFD', s).encode('ascii', 'ignore').decode("utf-8") def usd_to_eur(v): return v / 1.2 def passthrough(v): return v def convert_price(record, convert_map=translation_map): # remove accents for price mapping. Oops! for key, value in convert_map.items(): convert_map[key] = normalize_text(value) record = { convert_map[k]: usd_to_eur(v) for k, v in record.items() } return record On Wed, Oct 10, 2018 at 6:24 PM Steven D'Aprano <steve@pearwood.info> wrote:
Hi Philiip, and welcome,
On Wed, Oct 10, 2018 at 12:04:48PM -0500, Philip Martin wrote:
I generally have used MappingProxyType as a way to set default mapping to a function or to set an empty mapping to a function.
I've created a gist with an example use case:
https://gist.github.com/pmart123/493edf84d9aa61691ca7321325ebb6ab
Please try to keep the discussion in one place (i.e. here), for the benefit of the archives and for those who have email access but not unrestricted web access.
Can you explain (in English) your use-case, and why MappingProxyType isn't suitable? If it *is* suitable, how does your proposal differ?
If the only proposal is to rename types.MappingProxyType to a builtin "frozendict", that's one thing; if the proposal is something else, you should explain what.
I've included an example of what code typically looks like when using MappingProxyType and what it could look like with a frozendict implementation.
Wouldn't that be just:
from types import MappingProxyType as frozendict d = frozendict({'spam': 1, 'eggs': 2})
versus:
d = frozendict({'spam': 1, 'eggs': 2})
Apart from the initial import, how would they be different? You want a frozendict; the existing MappingProxyType provides a frozendict (with a surprising name, but never mind...). Wouldn't you use them exactly the same way? They both (I presume) offer precisely the same read-only access to the mapping interface.
-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
One important difference between MappingProxyType and a "proper" frozendict, as analog to frozenset, is that MappingProxyType doesn't have any method to return mutated versions of itself. On Thu, 11 Oct 2018 at 01:24, Steven D'Aprano <steve@pearwood.info> wrote:
Hi Philiip, and welcome,
On Wed, Oct 10, 2018 at 12:04:48PM -0500, Philip Martin wrote:
I generally have used MappingProxyType as a way to set default mapping to a function or to set an empty mapping to a function.
I've created a gist with an example use case:
https://gist.github.com/pmart123/493edf84d9aa61691ca7321325ebb6ab
Please try to keep the discussion in one place (i.e. here), for the benefit of the archives and for those who have email access but not unrestricted web access.
Can you explain (in English) your use-case, and why MappingProxyType isn't suitable? If it *is* suitable, how does your proposal differ?
If the only proposal is to rename types.MappingProxyType to a builtin "frozendict", that's one thing; if the proposal is something else, you should explain what.
I've included an example of what code typically looks like when using MappingProxyType and what it could look like with a frozendict implementation.
Wouldn't that be just:
from types import MappingProxyType as frozendict d = frozendict({'spam': 1, 'eggs': 2})
versus:
d = frozendict({'spam': 1, 'eggs': 2})
Apart from the initial import, how would they be different? You want a frozendict; the existing MappingProxyType provides a frozendict (with a surprising name, but never mind...). Wouldn't you use them exactly the same way? They both (I presume) offer precisely the same read-only access to the mapping interface.
-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
In tri.struct we have a class Frozen https://github.com/TriOptima/tri.struct/blob/master/lib/tri/struct/__init__.... that can be used to freeze stuff. I think something like this would be even better in the standard library, especially now with data classes! If we had this frozendict would just be: class frozendict(dict, Frozen): pass / Anders
On 10 Oct 2018, at 19:04, Philip Martin <philip.martin2007@gmail.com> wrote:
Hi, I first want to thank everyone in the community for the contributions over the years. I know the idea of a frozendict has been proposed before and rejected. I have a use case for a frozendict implementation that to my knowledge was not discussed during previous debates. My reasoning for a frozendict class stems from patterns I typically see arise when performing ETL or data integrations and conversions. I generally have used MappingProxyType as a way to set default mapping to a function or to set an empty mapping to a function. I've created a gist with an example use case:
https://gist.github.com/pmart123/493edf84d9aa61691ca7321325ebb6ab
I've included an example of what code typically looks like when using MappingProxyType and what it could look like with a frozendict implementation. I believe this use case may also be under-reported in open source code as it often crops up when integrating third-party data sources, which at times can't be open sourced due to licensing issues. I would love to hear if anyone has used MappingProxyType in a similar manner, or if this use case could help warrant a frozendict in the standard library. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
May be the following simple prototype of frozendict could be useful? def frozen_error(): return RuntimeError("frozendict is not mutable") class frozendict(dict): # def __setitem__(self, key, val): raise frozen_error() # def setdefault(self, key, val=None): raise frozen_error() # def update(self, ob): raise frozen_error() # def pop(self): raise frozen_error() # def popitem(self, ob): raise frozen_error() # def clear(self, ob): raise frozen_error() # def __delitem__(self, key): raise frozen_error() # def __repr__(self): return "frozendict(" + dict.__repr__(self)[1:-1] + ")" # def __str__(self): return "frozendict(" + dict.__str__(self)[1:-1] + ")" # def fromkeys(self, keys, val=None): return frozendict([(key,val) for key in keys]) # def copy(self): return frozendict(self.items()) среда, 10 октября 2018 г., 20:06:03 UTC+3 пользователь Philip Martin написал:
Hi, I first want to thank everyone in the community for the contributions over the years. I know the idea of a frozendict has been proposed before and rejected. I have a use case for a frozendict implementation that to my knowledge was not discussed during previous debates. My reasoning for a frozendict class stems from patterns I typically see arise when performing ETL or data integrations and conversions. I generally have used MappingProxyType as a way to set default mapping to a function or to set an empty mapping to a function. I've created a gist with an example use case:
https://gist.github.com/pmart123/493edf84d9aa61691ca7321325ebb6ab
I've included an example of what code typically looks like when using MappingProxyType and what it could look like with a frozendict implementation. I believe this use case may also be under-reported in open source code as it often crops up when integrating third-party data sources, which at times can't be open sourced due to licensing issues. I would love to hear if anyone has used MappingProxyType in a similar manner, or if this use case could help warrant a frozendict in the standard library.
On Thu, Oct 11, 2018 at 02:18:52AM -0700, Zaur Shibzukhov wrote:
May be the following simple prototype of frozendict could be useful?
def frozen_error(): return RuntimeError("frozendict is not mutable")
class frozendict(dict):
This violates the Liskov Substitution Principle. https://en.wikipedia.org/wiki/Liskov_substitution_principle Why is that bad? Let's say you have a function that requires a dict, and following the principle of "Fail Fast", you check the input up front: def spam(adict): """Input: adict: an instance of dict """ if isinstance(adict, dict): # lots of code here # and deep inside the function, we end up calling: adict[key] = "something" I hear people objecting that isinstance checks are "unPythonic". Perhaps; however, the basic principle applies regardless of whether we are doing LBYL isinstance checks, test for the presence of a __setitem__ method, or do no up-front tests at all and rely on duck-typing and EAFP. Our spam() function now accepts dicts, and rejects non-dicts, and works well, until somebody passes a frozendict: spam(frozendict()) This passes the isinstance test (or the check for __setitem__). It passes the documented interface: frozendicts certainly are dicts, since they are a subclass of dict. But nevertheless, the call to setitem fails, and the function breaks. We can get a good idea of how this ought to work by looking at set and frozenset. frozenset is *not* a subclass of set; it is a distinct class, so that frozensets are not sets. Rather than having mutator methods which raise an exception, they simply don't provide those methods at all: py> f = frozenset() py> isinstance(f, set) False py> hasattr(f, 'add') False That's how we should design frozendict. -- Steve
Steve D'Aprano wrote
Zaur Shibzukhov wrote
class frozendict(dict): This violates the Liskov Substitution Principle. https://en.wikipedia.org/wiki/Liskov_substitution_principle
and so, Steve said, we should not make frozendict a subclass of dict. Perhaps Zaur was thinking of Don't Repeat Yourself (DRY) https://en.wikipedia.org/wiki/Don%27t_repeat_yourself Certainly, by writing class frozendict(dict): # methods Zaur was able to use unchanged superclass methods such as __init__ and __getitem__, and also to delegate most of __str__ to the superclass. I see much merit in both https://en.wikipedia.org/wiki/Liskov_substitution_principle https://en.wikipedia.org/wiki/Don%27t_repeat_yourself I wonder whether, in pure Python, we can nicely have them both, when we implement frozenset. The best I can come up with is (not tested) is class frozendict: __slots__ = ('_self',) def __init__(self, *argv, **kwargs): self._self = dict(*argv, **kwargs) def __getitem__(self, key): return self._self.__getitem__(key) # And so on. -- Jonathan
https://en.wikipedia.org/wiki/Liskov_substitution_principle https://en.wikipedia.org/wiki/Don%27t_repeat_yourself
I did an internet search for: python liskov (over the past year). The first result was a Ruby page (but principle probably the same) https://www.netguru.co/codestories/solid-principles-3-lsp The second result was "Incompatibile signature with supertype" https://github.com/python/mypy/issues/4250 And the code example was class FrozenDict(MutableMapping): # code So there's prior Python art for FrozenDict and Liskov inheritance. -- Jonathan
This violates the Liskov Substitution Principle.
If we REALLY had a time machine, then dict would subclass frozendict, and we’d be all set. But to what extent do we need to support ALL the ways to check for an interface? Personally, I think EAFTP is the most “Pythonic”, but if folks want to check up front, then isn’t that what ABCs are for? In which case , we already have Mapping and MutableMapping. So if you have code that checks for Mapping when you need it mutable, then that’s arguably a bug. And if you code checks for dict directly, then it’s arguably unpythonic. That being said, it probably is best not to break working code. Would there be unpleasant consequences to making dict a subclass of FrozenDict ? Or maybe come up with a new name: after all, lists and tuples are independent, even though you *could* think of a tuple as a FrozenList ... -CHB
On Fri, Oct 12, 2018 at 2:41 AM Chris Barker - NOAA Federal via Python-ideas <python-ideas@python.org> wrote:
This violates the Liskov Substitution Principle.
If we REALLY had a time machine, then dict would subclass frozendict, and we’d be all set.
Thanks to virtual subclassing, we can still do this. The question is, should we? Intuition tells me that a frozen dictionary is a form of dictionary that adds restrictions, not that a dictionary is a frozen dictionary that you left out to thaw. But as we see from [frozen]set, it's probably best to treat them as completely independent classes, both implementing the basic Mapping interface. ChrisA
Summary: Long post. Because of LSP, neither dict nor frozendict are a subclass of the other. Chris Barker wrote:
If we REALLY had a time machine, then dict would subclass frozendict, and we’d be all set.
Prediction is difficult, particularly when it involves the future. - Neils Bohr. And be careful about what you wish for. Perhaps in Python 4 we can fix this problem. Chris Angelico <rosuav@gmail.com> wrote:
Thanks to virtual subclassing, we can still do this. The question is, should we?
Intuition tells me that a frozen dictionary is a form of dictionary that adds restrictions, not that a dictionary is a frozen dictionary that you left out to thaw.
It's not quite so simple. Sometimes, less is more. https://www.phrases.org.uk/meanings/226400.html Suppose every instance of class A has useful property P, and B is a subclass of A. If the Liskov substitution principle (LSP) holds, then every instance of B also has useful property P. Notice that I said USEFUL property P. The negation Q of property P is also a property. Often, Q is NOT USEFUL. But what if both P and its negation Q are BOTH USEFUL? Well, if there should be some strange useful property P, whose negation is also useful then (drum roll) Liskov substitution says that neither class is a subclass of the other. (I guess that's one of the applications of LSP.) Here's an example of such a property. A tuple of integers is immutable, and so has a hash and can be used as a dictionary key. Being immutable is a useful property. A list of integers is mutable, and so can be used to record changing state. Being mutable is a useful property. Aside: Last month I prepared the show part of the show-and-tell for this. https://github.com/jfine2358/python-show-and-tell/issues/1 How do we explain this to a novice
dct = dict() point = [0, 1] dct[point] = 'red' Traceback (most recent call last): TypeError: unhashable type: 'list'
Conclusion: If we have LSP, then dict is not a subclass of frozendict, and frozendict is not a subclass of dict. Neither is a subclass of the other. One way to fix this in Python 4 would be to create a common ancestor, which has only the shared methods of dict and frozendict. Or do something similar with abstract base classes. Remark: The issues here might be related to mypy typing, which I've not used. From what I've read, it seems to rely on LSP. I've nothing to say at present about the hardest problem here, which is naming things. We have list and tuple. But dict and what name for frozen, and what name for common ancestor. https://martinfowler.com/bliki/TwoHardThings.html -- Jonathan
A link on https://en.wikipedia.org/wiki/Liskov_substitution_principle goes to http://www.engr.mun.ca/~theo/Courses/sd/5895-downloads/sd-principles-3.ppt.p... which has a very nice example (slides 14 and 15). Slide 14: Is immutable Square a behavioural subtype of immutable Rectangle? And vice versa? Slide 15: Is mutable Square a behavioural subtype of mutable Rectangle? And vice versa? And relationship between mutable and immutable Square (resp Rectangle). -- Jonathan
On Thu, Oct 11, 2018 at 9:54 AM, Jonathan Fine <jfine2358@gmail.com> wrote:
Summary: Long post. Because of LSP, neither dict nor frozendict are a subclass of the other.
given Python's dynamic typing, these issues are kinda academic :-)
Intuition tells me that a frozen dictionary is a form of dictionary
that adds restrictions, not that a dictionary is a frozen dictionary that you left out to thaw.
well, IIUC the Liskov principle correctly then a subclass is never a version of a class that does less, but rather always one that does more. Think you intuition may be driven by the choice of names and history: yes, a frozen dict sounds like a regular dict that has been altered (specifically frozen) -- but if we called them "hash_table" and "mutable_has_table", then your intuition may be different :-) As for subclassing or not, for most Python code is makes no difference -- polymorphism is not achieved through subclassing. and the "Pythonic" way to test for type is through ABCs, and we already have Mapping and MutableMapping, which kind of surprised me, as there is no builtin Mapping that isn't also a MutableMapping. Notice that I said USEFUL property P. The negation Q of property P is
also a property.
well, yeah, but I think the concept of "useful" is pretty vague. in this case, is "imutablilty" the "useful" property -- or is "hashability"?, which would want us to use abc.Hashable. So: I don't care what is or isn't a subclass of what -- I don't think that's a Pythonic question. But I do think : issubclass(frozendict, abc.Mapping) and issubclass(frozendict, abc.Hashable) would be useful. BTW, I just noticed that: A tuple, as in ``issubclass(x, (A, B, ...))``, may be given as the target to check against. This is equivalent to ``issubclass(x, A) or issubclass(x, B) or ...`` etc. which seems less useful than "and" -- at least for ABCs I suppose that API pre-dates ABCs.... One way to fix this in Python 4 would be to create a common ancestor,
which has only the shared methods of dict and frozendict.
which would be an immutable, but not hashable, mapping ?!? Or do
something similar with abstract base classes.
already done -- see above. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
BTW: In [7]: issubclass(set, frozenset) Out[7]: False In [8]: issubclass(frozenset, set) Out[8]: False no reason to do anything different here. and: In [13]: issubclass(MappingProxyType, abc.Hashable) Out[13]: False so yes, there is a need for something different -CHB On Thu, Oct 11, 2018 at 12:34 PM, Chris Barker <chris.barker@noaa.gov> wrote:
On Thu, Oct 11, 2018 at 9:54 AM, Jonathan Fine <jfine2358@gmail.com> wrote:
Summary: Long post. Because of LSP, neither dict nor frozendict are a subclass of the other.
given Python's dynamic typing, these issues are kinda academic :-)
Intuition tells me that a frozen dictionary is a form of dictionary
that adds restrictions, not that a dictionary is a frozen dictionary that you left out to thaw.
well, IIUC the Liskov principle correctly then a subclass is never a version of a class that does less, but rather always one that does more.
Think you intuition may be driven by the choice of names and history: yes, a frozen dict sounds like a regular dict that has been altered (specifically frozen) -- but if we called them "hash_table" and "mutable_has_table", then your intuition may be different :-)
As for subclassing or not, for most Python code is makes no difference -- polymorphism is not achieved through subclassing. and the "Pythonic" way to test for type is through ABCs, and we already have Mapping and MutableMapping, which kind of surprised me, as there is no builtin Mapping that isn't also a MutableMapping.
Notice that I said USEFUL property P. The negation Q of property P is
also a property.
well, yeah, but I think the concept of "useful" is pretty vague.
in this case, is "imutablilty" the "useful" property -- or is "hashability"?, which would want us to use abc.Hashable.
So:
I don't care what is or isn't a subclass of what -- I don't think that's a Pythonic question. But I do think :
issubclass(frozendict, abc.Mapping) and issubclass(frozendict, abc.Hashable)
would be useful.
BTW, I just noticed that:
A tuple, as in ``issubclass(x, (A, B, ...))``, may be given as the target to check against. This is equivalent to ``issubclass(x, A) or issubclass(x, B) or ...`` etc.
which seems less useful than "and" -- at least for ABCs
I suppose that API pre-dates ABCs....
One way to fix this in Python 4 would be to create a common ancestor,
which has only the shared methods of dict and frozendict.
which would be an immutable, but not hashable, mapping ?!?
Or do
something similar with abstract base classes.
already done -- see above.
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Thu, Oct 11, 2018 at 12:34:13PM -0700, Chris Barker via Python-ideas wrote:
I don't care what is or isn't a subclass of what -- I don't think that's a Pythonic question. But I do think :
issubclass(frozendict, abc.Mapping) and issubclass(frozendict, abc.Hashable)
would be useful.
So you do care about what is and isn't a subclass :-) It is 2018 and we've had isinstance and issubclass in the language since Python 2.2 in 2002, I really wish that we could get away from the idea that checking types is unPythonic and duck-typing is the One True Way to do things. The old, Python 1.5 form of type-checking: type(spam) is dict was not so great, since it tested only for a specific type by identity. But with ABCs and virtual subclasses and inheritance, isinstance is a great way to check that your duck both quacks and swims, rather than watching it sink and drown in the middle of your function :-) -- Steve
On Thu, Oct 11, 2018 at 3:35 PM, Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Oct 11, 2018 at 12:34:13PM -0700, Chris Barker via Python-ideas wrote:
I don't care what is or isn't a subclass of what -- I don't think that's a Pythonic question. But I do think :
issubclass(frozendict, abc.Mapping) and issubclass(frozendict, abc.Hashable)
would be useful.
So you do care about what is and isn't a subclass :-)
well, kinda -- I don't care whether dict and frozen dict have a subclassing relationship, and I don't care what class a given object is that gets passed to my code. What I *might* care about is what interface it presents, which is what ABCs are for. If Python had a way to check ABCs without issubclass, then I wouldn't care about subclasses at all. I'd actually kind of like to have: hasinterface(an_object, (ABC1, ABC2, ABC3)) Even though, it would be the same as issubclass() (though I'd like an AND relationship with the tuple of ABCs..) Maybe I'm making a distinction that isn't really there, but I see a subclass of an ABC is quite different than a subclass of another concrete class.
It is 2018 and we've had isinstance and issubclass in the language since Python 2.2 in 2002, I really wish that we could get away from the idea that checking types is unPythonic and duck-typing is the One True Way to do things. The old, Python 1.5 form of type-checking:
type(spam) is dict
was not so great, since it tested only for a specific type by identity. But with ABCs and virtual subclasses and inheritance, isinstance is a great way to check that your duck both quacks and swims, rather than watching it sink and drown in the middle of your function :-)
yeah, I'm not so sure -- I"ve been watching Python go more and more in that direction -- and I don't think I'm happy about it. Personally, the only time I check a type is the darn list-of-strings vs single-string issue -- ever since we got true_division, that's the only type error I commonly see (that isn't trivial to debug). And I still wish Python had a Character type, and then I wouldn't need to do that either :-) And actually, if you're talking "modern" Python, isn't static type checking the way to do it now ? Now that I think about it - how does MyPy/Typeshed handle the iterable_of_strings problem? Since a single string IS and iterable of strings? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On 12.10.18 01:30, Chris Barker via Python-ideas wrote:
If Python had a way to check ABCs without issubclass, then I wouldn't care about subclasses at all. I'd actually kind of like to have:
hasinterface(an_object, (ABC1, ABC2, ABC3))
I actually like your idea and could imagine using "hasinterface" as some asserts. Great idea. +1
Even though, it would be the same as issubclass() (though I'd like an AND relationship with the tuple of ABCs..)
When "hasinterface" ANDs the tuple, it's already different, isn't it? Btw., issubclass would require a class, not an instance. hasinterface, though, would work on all type of objects. Best, Sven
Even though, it would be the same as issubclass() (though I'd like an AND relationship with the tuple of ABCs..)
When "hasinterface" ANDs the tuple, it's already different, isn't it?
If it's AND, shouldn't it be "hasinterfaces" (notice the s!)? One could also imagine that isinstance and issubclass taking a keyword argument for the logical operator. Maybe just something as simple as "isinstance(foo, (a, b), all=True)" / Anders
On 18.10.18 18:49, Anders Hovmöller wrote:
If it's AND, shouldn't it be "hasinterfaces" (notice the s!)?
Yeah, could be. To be sure, we are on the same page here: "interface" refers to a set of attributes of the object in question, does it? E.g. like the __iter__ iterface. I usually don't care about the actual inheritance hierarchy but care about functionality.
One could also imagine that isinstance and issubclass taking a keyword argument for the logical operator. Maybe just something as simple as "isinstance(foo, (a, b), all=True)"
Does AND even make sense for isinstance/issubclass? Cheers, Sven
On Thu, Oct 18, 2018 at 10:12 AM, Sven R. Kunze <srkunze@mail.de> wrote:
On 18.10.18 18:49, Anders Hovmöller wrote:
If it's AND, shouldn't it be "hasinterfaces" (notice the s!)?
yeah, that would make sense. Is someone proposing something here? The point I was making, is in the case of ABCs: issubclass(a_class, an_abc) isn't "Really" testing a subclass, but really an interface. That is - the ABC exists solely to define an interface, not provide any functionality (though there is some in there), and you can, in fact, use ABCs to provide a way to check a duck-typed class that isn't an actual subclass. So do we need a new function that makes that clear? would it functionally be an different than issubclass with an ABC? I honestly don't know. Though an easy way to get the "and" behaviour -- i.e. interfaceS would be nice. -CHB To be sure, we are on the same page here: "interface" refers to a set of
attributes of the object in question, does it?
E.g. like the __iter__ iterface. I usually don't care about the actual inheritance hierarchy but care about functionality.
One could also imagine that isinstance and issubclass taking a keyword
argument for the logical operator. Maybe just something as simple as "isinstance(foo, (a, b), all=True)"
Does AND even make sense for isinstance/issubclass?
Cheers, Sven
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Fri, Oct 12, 2018 at 02:45:30AM +1100, Chris Angelico wrote:
On Fri, Oct 12, 2018 at 2:41 AM Chris Barker - NOAA Federal via Python-ideas <python-ideas@python.org> wrote:
This violates the Liskov Substitution Principle.
If we REALLY had a time machine, then dict would subclass frozendict, and we’d be all set.
Thanks to virtual subclassing, we can still do this. The question is, should we?
Intuition tells me that a frozen dictionary is a form of dictionary that adds restrictions, not that a dictionary is a frozen dictionary that you left out to thaw.
No offence Chris, but that's why we shouldn't program by intuition :-) We don't build Smart cars by starting with a Hummer and slicing off the bits that aren't needed. We already have prior art demonstrating best practice in the form of the Mapping and MutableMapping ABCs, and frozenset versus set for concrete classes. Best practice is not to subclass a type and restrict existing functionality, but to start with a restricted class and extend functionality. Or not to subclass at all. Subclassing is not the only way to DRY -- there are other ways for frozendict and dict to share code other than subclassing, although they're not necessarily efficient or easy if frozendict is written in pure Python. One way is delegation to a hidden dict (like the MappingProxyType, except we ensure the dict is private and not shared). # Just a sketch, not a full implementation. class frozendict(object): def __new__(cls, *args, **kwargs): instance = super().__new__(cls) instance.__d = dict(*args, **kwargs) return instance # Define only the methods we want, delegating to the # hidden dict. def __getitem__(self, key): return self.__d[key] def keys(self): return self.__d.keys() def items(self): return self.__d.items() The trickiest part is probably getting hashing right. I think this would work: def __hash__(self): return hash(tuple(self.items()))
But as we see from [frozen]set, it's probably best to treat them as completely independent classes, both implementing the basic Mapping interface.
Indeed. -- Steve
On Fri, Oct 12, 2018 at 9:16 AM Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Oct 12, 2018 at 02:45:30AM +1100, Chris Angelico wrote:
On Fri, Oct 12, 2018 at 2:41 AM Chris Barker - NOAA Federal via Python-ideas <python-ideas@python.org> wrote:
This violates the Liskov Substitution Principle.
If we REALLY had a time machine, then dict would subclass frozendict, and we’d be all set.
Thanks to virtual subclassing, we can still do this. The question is, should we?
Intuition tells me that a frozen dictionary is a form of dictionary that adds restrictions, not that a dictionary is a frozen dictionary that you left out to thaw.
No offence Chris, but that's why we shouldn't program by intuition :-)
But as we see from [frozen]set, it's probably best to treat them as completely independent classes, both implementing the basic Mapping interface.
Indeed.
Which was my point. They probably should NOT be in a direct hierarchy, partly because people's intuition WILL lead them to a dangerous expectation. ChrisA
Chris Barker - NOAA Federal via Python-ideas wrote:
Or maybe come up with a new name
We should call it a birdseyedict, because of this: http://www.vulture.com/2016/12/unearthing-a-rare-1971-monty-python-film-all-... -- Greg
Would a frozendict require that keys and values be hashable? It seems to me that we would need this restriction to make a reasonably universal frozendict that is, itself, hashable. With this restriction for the general case, is there still sufficient value for everyone that is asking for a frozendict? Without this restriction and without frozendict being hashable, is there still sufficient value for everyone that is asking for a frozendict? On Fri, Oct 12, 2018 at 7:31 AM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Chris Barker - NOAA Federal via Python-ideas wrote:
Or maybe come up with a new name
We should call it a birdseyedict, because of this:
http://www.vulture.com/2016/12/unearthing-a-rare-1971-monty-python-film-all-...
-- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Oct 16, 2018 at 01:02:03AM -0700, George Leslie-Waksman wrote:
Would a frozendict require that keys and values be hashable?
Keys, yes. Values, no. If the values were hashable, the frozendict itself would also be hashable. If not, then it would be like trying to hash a tuple with unhashable items: py> hash((1, 2, {}, 3)) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'dict'
It seems to me that we would need this restriction to make a reasonably universal frozendict that is, itself, hashable.
When people talk about frozendicts being hashable, they mean it in the same sense that tuples are hashable. For what it is worth, here's an incomplete, quick and dirty proof of concept frozendict, using automatic delegation: # Not good enough for production, not tested, buyer beware, etc. class frozendict: def __new__(cls, *args, **kwargs): d = dict(*args, **kwargs) proxy = types.MappingProxyType(d) instance = super().__new__(cls) instance.__proxy = proxy return instance def __hash__(self): return hash(tuple(self.items())) def __getattr__(self, name): return getattr(self.__proxy, name) def __getitem__(self, key): return self.__proxy[key] def __repr__(self): return "%s(%r)" % (type(self).__name__, dict(self)) -- Steve
вторник, 16 октября 2018 г., 12:29:55 UTC+3 пользователь Steven D'Aprano написал:
It seems to me that we would need this restriction to make a reasonably universal frozendict that is, itself, hashable.
When people talk about frozendicts being hashable, they mean it in the same sense that tuples are hashable.
For what it is worth, here's an incomplete, quick and dirty proof of concept frozendict, using automatic delegation:
# Not good enough for production, not tested, buyer beware, etc. class frozendict: def __new__(cls, *args, **kwargs): d = dict(*args, **kwargs) proxy = types.MappingProxyType(d) instance = super().__new__(cls) instance.__proxy = proxy return instance def __hash__(self): return hash(tuple(self.items())) def __getattr__(self, name): return getattr(self.__proxy, name) def __getitem__(self, key): return self.__proxy[key] def __repr__(self): return "%s(%r)" % (type(self).__name__, dict(self))
For those who need more performant variant (Cython compiled) of frozendict
frozenmap <https://pypi.org/project/frozenmap/> project can be offer.
I think it could inherit from the Mapping abc? class frozendict(Mapping): def __new__(cls, *args, **kwargs): d = dict(*args, **kwargs) proxy = MappingProxyType(d) instance = super().__new__(cls) instance.__proxy = proxy return instance def __hash__(self): return hash(tuple(self.items())) def __getattr__(self, name): return getattr(self.__proxy, name) def __getitem__(self, key): return self.__proxy[key] def __iter__(self): return self.__proxy.__iter__() def __len__(self): return len(self.__proxy) def __repr__(self): return "%s(%r)" % (type(self).__name__, dict(self)) On Tue, Oct 16, 2018 at 4:29 AM Steven D'Aprano <steve@pearwood.info> wrote:
On Tue, Oct 16, 2018 at 01:02:03AM -0700, George Leslie-Waksman wrote:
Would a frozendict require that keys and values be hashable?
Keys, yes. Values, no.
If the values were hashable, the frozendict itself would also be hashable. If not, then it would be like trying to hash a tuple with unhashable items:
py> hash((1, 2, {}, 3)) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'dict'
It seems to me that we would need this restriction to make a reasonably universal frozendict that is, itself, hashable.
When people talk about frozendicts being hashable, they mean it in the same sense that tuples are hashable.
For what it is worth, here's an incomplete, quick and dirty proof of concept frozendict, using automatic delegation:
# Not good enough for production, not tested, buyer beware, etc. class frozendict: def __new__(cls, *args, **kwargs): d = dict(*args, **kwargs) proxy = types.MappingProxyType(d) instance = super().__new__(cls) instance.__proxy = proxy return instance def __hash__(self): return hash(tuple(self.items())) def __getattr__(self, name): return getattr(self.__proxy, name) def __getitem__(self, key): return self.__proxy[key] def __repr__(self): return "%s(%r)" % (type(self).__name__, dict(self))
-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Oct 16, 2018 at 7:02 PM George Leslie-Waksman <waksman@gmail.com> wrote:
Would a frozendict require that keys and values be hashable?
Keys would already have to be hashable - regular dicts demand this, and there's no reason not to for frozendict. Values? Not so sure. Personally I would say that no, they don't HAVE to be hashable - but that the frozendict itself would then not be hashable.
It seems to me that we would need this restriction to make a reasonably universal frozendict that is, itself, hashable.
The tuple provides a good precedent here:
hash((1,2,3)) 2528502973977326415 hash((1,2,[])) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'list'
ChrisA
participants (15)
-
Anders Hovmöller -
Cameron Simpson -
Chris Angelico -
Chris Barker -
Chris Barker - NOAA Federal
-
George Leslie-Waksman -
Greg Ewing -
Jonathan Fine -
João Santos -
Michael Selik -
Philip Martin -
Serhiy Storchaka -
Steven D'Aprano -
Sven R. Kunze -
Zaur Shibzukhov