Mailman 3 should `dict` supply a default `__missing__` implementation? - Python-ideas

newer
Re: [Python-ideas] Python-ideas...

should `dict` supply a default `missing` implementation?

older
__enter__ and __exit__ on...

Matt Gilson

29 Jun 2016 29 Jun '16

5:47 p.m.

The following code raises an `AttributeError`:

...

...
...
class D(dict): ... def __missing__(self, k): ... super(D, self).__missing__(k) ... d = D() d['key'] Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in __missing__ AttributeError: 'super' object has no attribute '__missing__'

I find this behavior to be a little bit odd as I would have expected the default implementation to have a `__missing__` method and I would expect the `__missing__` method to raise an appropriate `KeyError`. I think that this would facilitate a class hierarchy where each class can decide "I don't know hot to handle this missing key, maybe something further up the MRO does". Currently, to do that (and adhere to the ["Super considered Super"](https://rhettinger.wordpress.com/2011/05/26/super-considered-super/) methodology) you'd have to add an extra base-class in the hierarchy which seems unnecessary. Obviously this is probably a pretty niche case and may not be worth considering, but I thought I'd throw it out there just in case. Also, FWIW, I believe that the current behavior is correct based on [the documentation](https://docs.python.org/2/library/stdtypes.html#dict) (under `d[key]`).

Attachments:

attachment.htm (text/html — 2.5 KB)

Show replies by thread

Ethan Furman

29 Jun 29 Jun

5:50 p.m.

On 06/29/2016 10:47 AM, Matt Gilson wrote:

...

The following code raises an `AttributeError`:

...
...
...
class D(dict): ... def __missing__(self, k): ... super(D, self).__missing__(k) ... d = D() d['key'] Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in __missing__ AttributeError: 'super' object has no attribute '__missing__'

I find this behavior to be a little bit odd as I would have expected the default implementation to have a `__missing__` method and I would expect the `__missing__` method to raise an appropriate `KeyError`. I think that this would facilitate a class hierarchy where each class can decide "I don't know hot to handle this missing key, maybe something further up the MRO does".

+1 -- ~Ethan~

Michael Selik

6:29 p.m.

On Wed, Jun 29, 2016 at 1:47 PM Matt Gilson wrote:

...

The following code raises an `AttributeError`:

...
...
...
class D(dict): ... def __missing__(self, k): ... super(D, self).__missing__(k) ...

I find this behavior to be a little bit odd as I would have expected the default implementation to have a `__missing__` method

The behavior of a dictionary key-lookup checking for __missing__ parallels the way a standard class does attribute-lookup, checking for __getattr__. I'm guessing the non-existence of __missing__ on the built-in dict type (and the non-existence of __getattr__ on the base object type) allows for some speed optimizations in the interpreter. If so, then the tradeoff of needing to provide an extra base class shim for your multiple inheritance hierarchy seems reasonable. Does anyone know the original reason for not having __missing__ defined on the builtin type?

Guido van Rossum

6:56 p.m.

The argument against adding `dict.__missing__()` is that it would just a shortcut to raise KeyError with certain parameters. The purpose of `__missing__` is to give you a hook to override how the base class handles missing keys in `__getitem__`. In order to define this hook you must necessarily subclass dict. IMO the idea that there's always a superclass you can call is silly -- you should distinguish between cases where you *override* a method vs. cases where you *define* it. In this case you are required to *define* `__missing__`. But I may be missing something... On Wed, Jun 29, 2016 at 11:29 AM, Michael Selik wrote:

...

On Wed, Jun 29, 2016 at 1:47 PM Matt Gilson wrote:

...
The following code raises an `AttributeError`:

...
...
...
class D(dict): ... def __missing__(self, k): ... super(D, self).__missing__(k) ...

I find this behavior to be a little bit odd as I would have expected the default implementation to have a `__missing__` method

The behavior of a dictionary key-lookup checking for __missing__ parallels the way a standard class does attribute-lookup, checking for __getattr__.

I'm guessing the non-existence of __missing__ on the built-in dict type (and the non-existence of __getattr__ on the base object type) allows for some speed optimizations in the interpreter. If so, then the tradeoff of needing to provide an extra base class shim for your multiple inheritance hierarchy seems reasonable. Does anyone know the original reason for not having __missing__ defined on the builtin type?

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

Émanuel Barry

7:04 p.m.

...

From: Guido van Rossum To: Michael Selik Cc: Python-Ideas Subject: Re: [Python-ideas] should `dict` supply a default `__missing__` implementation?

The argument against adding `dict.__missing__()` is that it would just a shortcut to raise KeyError with certain parameters.

Can't the interpreter optimize that out? Something like 'if ns["__missing__"] is dict.__missing__: raise KeyError' ? I figure it wouldn't be hard, doesn't Python already do stuff like that in a bunch of other places?

...

IMO the idea that there's always a superclass you can call is silly -- you should distinguish between cases where you *override* a method vs. cases where you *define* it. In this case you are required to *define* `__missing__`.

The difference is that `dict` explicitly supports __missing__ for subclasses, so that users don't have to re-define __getitem__ for those purposes. I don't know of many classes explicitly supporting a given feature but not implementing at least a method for it that does nothing. But maybe that's just me. -Emanuel

Ethan Furman

7:07 p.m.

On 06/29/2016 11:56 AM, Guido van Rossum wrote:

...

The argument against adding `dict.__missing__()` is that it would just a shortcut to raise KeyError with certain parameters.

The purpose of `__missing__` is to give you a hook to override how the base class handles missing keys in `__getitem__`. In order to define this hook you must necessarily subclass dict.

IMO the idea that there's always a superclass you can call is silly -- you should distinguish between cases where you *override* a method vs. cases where you *define* it. In this case you are required to *define* `__missing__`.

My understanding is a comprehensive base class is necessary when multiple inheritance is a concern, since the methods can't know if they are being called before or after other methods and so, to be good citizens, should call super() -- which of course will fail when the base class is searched and the method doesn't exist.

...

But I may be missing something...

I can see two possible use-cases for multiple __missing__ methods: - the default provided may be different based on the key name - different kinds of tracking/registering of key names is happening -- ~Ethan~

Guido van Rossum

7:09 p.m.

So providing the comprehensive base class is up to the user, not up to the stdlib. Is that such a big deal? On Wed, Jun 29, 2016 at 12:07 PM, Ethan Furman wrote:

...

On 06/29/2016 11:56 AM, Guido van Rossum wrote:

...
The argument against adding `dict.__missing__()` is that it would just a shortcut to raise KeyError with certain parameters.

The purpose of `__missing__` is to give you a hook to override how the base class handles missing keys in `__getitem__`. In order to define this hook you must necessarily subclass dict.

IMO the idea that there's always a superclass you can call is silly -- you should distinguish between cases where you *override* a method vs. cases where you *define* it. In this case you are required to *define* `__missing__`.

My understanding is a comprehensive base class is necessary when multiple inheritance is a concern, since the methods can't know if they are being called before or after other methods and so, to be good citizens, should call super() -- which of course will fail when the base class is searched and the method doesn't exist.

...
But I may be missing something...

I can see two possible use-cases for multiple __missing__ methods:

- the default provided may be different based on the key name - different kinds of tracking/registering of key names is happening

-- ~Ethan~

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

Matt Gilson

7:29 p.m.

On Wed, Jun 29, 2016 at 12:09 PM, Guido van Rossum wrote:

...

So providing the comprehensive base class is up to the user, not up to the stdlib. Is that such a big deal?

Nope. I honestly don't think that it is a big deal. Mostly I just found the behavior surprising for the reasons mentioned earlier (`__missing__` is explicitly supported for dict subclasses but also explicitly not supported on the superclass). FWIW, "The argument against adding `dict.__missing__()` is that it would just a shortcut to raise KeyError with certain parameters." also doesn't seem like that big of a deal to me :-). I'm not sure if there would be noticeable performance issues or if the implementation would be particularly difficult (both reasons which would be certainly compelling enough to reject this idea). But if the rational really is that it would be a hook method whose implementation is almost trivial -- That seems like the best kind of hook method to me :-). Of course, maybe I'm shooting my proposal in the foot here -- If it's not really a big deal on either side of the argument, maybe keeping the status-quo is the safest recourse...

...

On Wed, Jun 29, 2016 at 12:07 PM, Ethan Furman wrote:

...
On 06/29/2016 11:56 AM, Guido van Rossum wrote:

...
The argument against adding `dict.__missing__()` is that it would just a shortcut to raise KeyError with certain parameters.

The purpose of `__missing__` is to give you a hook to override how the base class handles missing keys in `__getitem__`. In order to define this hook you must necessarily subclass dict.

IMO the idea that there's always a superclass you can call is silly -- you should distinguish between cases where you *override* a method vs. cases where you *define* it. In this case you are required to *define* `__missing__`.

My understanding is a comprehensive base class is necessary when multiple inheritance is a concern, since the methods can't know if they are being called before or after other methods and so, to be good citizens, should call super() -- which of course will fail when the base class is searched and the method doesn't exist.

...
But I may be missing something...

I can see two possible use-cases for multiple __missing__ methods:

- the default provided may be different based on the key name - different kinds of tracking/registering of key names is happening

-- ~Ethan~

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- [image: pattern-sig.png] Matt Gilson // SOFTWARE ENGINEER E: matt@getpattern.com // P: 603.892.7736 We’re looking for beta testers. Go here https://www.getpattern.com/meetpattern to sign up!

Ethan Furman

7:30 p.m.

On 06/29/2016 12:09 PM, Guido van Rossum wrote:

...

So providing the comprehensive base class is up to the user, not up to the stdlib. Is that such a big deal?

No, it's not. But it makes for a better user experience if the base class has the __missing__ method that raises a KeyError already. Didn't we add a UserDict that could be subclassed primarily because subclassing dict directly was such a poor user experience? If adding __missing__ to dict is huge (performance hit?), we don't do it. If it's not, I think we should. Maybe add it to UserDict if performance is a concern? -- ~Ethan~

Guido van Rossum

8:57 p.m.

UserDict is superseded by MutableMapping. I don't think we should make dict a kitchen sink class. I also don't think we should particularly encourage subclassing it. So -1 on adding dict.__missing__. On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman wrote:

...

On 06/29/2016 12:09 PM, Guido van Rossum wrote:

...
So providing the comprehensive base class is up to the user, not up to the stdlib. Is that such a big deal?

No, it's not. But it makes for a better user experience if the base class has the __missing__ method that raises a KeyError already.

Didn't we add a UserDict that could be subclassed primarily because subclassing dict directly was such a poor user experience?

If adding __missing__ to dict is huge (performance hit?), we don't do it. If it's not, I think we should. Maybe add it to UserDict if performance is a concern?

-- ~Ethan~ _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

Neil Girdhar

5 Jul 5 Jul

7:18 p.m.

But neither UserDict nor MutableMapping defines __missing__ ? What is a subclasser supposed to do? On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote:

...

UserDict is superseded by MutableMapping.

I don't think we should make dict a kitchen sink class. I also don't think we should particularly encourage subclassing it. So -1 on adding dict.__missing__.

On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman javascript:> wrote:

...
On 06/29/2016 12:09 PM, Guido van Rossum wrote:

...
So providing the comprehensive base class is up to the user, not up to the stdlib. Is that such a big deal?

No, it's not. But it makes for a better user experience if the base class has the __missing__ method that raises a KeyError already.

Didn't we add a UserDict that could be subclassed primarily because subclassing dict directly was such a poor user experience?

If adding __missing__ to dict is huge (performance hit?), we don't do it. If it's not, I think we should. Maybe add it to UserDict if performance is a concern?

-- ~Ethan~ _______________________________________________ Python-ideas mailing list Python...@python.org javascript: https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python...@python.org javascript: https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Guido van Rossum

10:03 p.m.

What kind of question is that? If you subclass MutableMapping the whole feature doesn't exist (since you're not subclassing dict). It *only* exists for subclasses of dict. On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar wrote:

...

But neither UserDict nor MutableMapping defines __missing__ ? What is a subclasser supposed to do?

On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote:

...
UserDict is superseded by MutableMapping.

I don't think we should make dict a kitchen sink class. I also don't think we should particularly encourage subclassing it. So -1 on adding dict.__missing__.

On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman wrote:

...
On 06/29/2016 12:09 PM, Guido van Rossum wrote:

...
So providing the comprehensive base class is up to the user, not up to the stdlib. Is that such a big deal?

No, it's not. But it makes for a better user experience if the base class has the __missing__ method that raises a KeyError already.

Didn't we add a UserDict that could be subclassed primarily because subclassing dict directly was such a poor user experience?

If adding __missing__ to dict is huge (performance hit?), we don't do it. If it's not, I think we should. Maybe add it to UserDict if performance is a concern?

-- ~Ethan~ _______________________________________________ Python-ideas mailing list Python...@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python...@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

Neil Girdhar

10:07 p.m.

Okay, that makes sense, but why isn't __missing__ in UserDict? On Tue, Jul 5, 2016 at 6:04 PM Guido van Rossum wrote:

...

What kind of question is that? If you subclass MutableMapping the whole feature doesn't exist (since you're not subclassing dict). It *only* exists for subclasses of dict.

...
But neither UserDict nor MutableMapping defines __missing__ ? What is a subclasser supposed to do?

On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote:

...
UserDict is superseded by MutableMapping.

I don't think we should make dict a kitchen sink class. I also don't think we should particularly encourage subclassing it. So -1 on adding dict.__missing__.

On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman

wrote:

...
...
On 06/29/2016 12:09 PM, Guido van Rossum wrote:

...
So providing the comprehensive base class is up to the user, not up to the stdlib. Is that such a big deal?

No, it's not. But it makes for a better user experience if the base class has the __missing__ method that raises a KeyError already.

Didn't we add a UserDict that could be subclassed primarily because subclassing dict directly was such a poor user experience?

If adding __missing__ to dict is huge (performance hit?), we don't do it. If it's not, I think we should. Maybe add it to UserDict if

On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar wrote: performance

...
...
...
is a concern?

-- ~Ethan~ _______________________________________________ Python-ideas mailing list Python...@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python...@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

Guido van Rossum

10:10 p.m.

Because you shouldn't be using UserDict. On Tue, Jul 5, 2016 at 3:07 PM, Neil Girdhar wrote:

...

Okay, that makes sense, but why isn't __missing__ in UserDict?

On Tue, Jul 5, 2016 at 6:04 PM Guido van Rossum wrote:

...
What kind of question is that? If you subclass MutableMapping the whole feature doesn't exist (since you're not subclassing dict). It *only* exists for subclasses of dict.

On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar wrote:

...
But neither UserDict nor MutableMapping defines __missing__ ? What is a subclasser supposed to do?

On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote:

...
UserDict is superseded by MutableMapping.

I don't think we should make dict a kitchen sink class. I also don't think we should particularly encourage subclassing it. So -1 on adding dict.__missing__.

On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman wrote:

...
On 06/29/2016 12:09 PM, Guido van Rossum wrote:

...
So providing the comprehensive base class is up to the user, not up to the stdlib. Is that such a big deal?

No, it's not. But it makes for a better user experience if the base class has the __missing__ method that raises a KeyError already.

Didn't we add a UserDict that could be subclassed primarily because subclassing dict directly was such a poor user experience?

If adding __missing__ to dict is huge (performance hit?), we don't do it. If it's not, I think we should. Maybe add it to UserDict if performance is a concern?

-- ~Ethan~ _______________________________________________ Python-ideas mailing list Python...@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python...@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

-- --Guido van Rossum (python.org/~guido)

Neil Girdhar

10:13 p.m.

Is it deprecated? I've seen this question a lot on stackoverflow: http://stackoverflow.com/questions/7148419/subclass-dict-userdict-dict-or-ab... http://stackoverflow.com/questions/2390827/how-to-properly-subclass-dict-and... http://stackoverflow.com/questions/10901048/i-want-to-subclass-dict-and-set-... I still have no idea what the right answer is. On Tue, Jul 5, 2016 at 6:11 PM Guido van Rossum wrote:

...

Because you shouldn't be using UserDict.

...
Okay, that makes sense, but why isn't __missing__ in UserDict?

On Tue, Jul 5, 2016 at 6:04 PM Guido van Rossum wrote:

...
What kind of question is that? If you subclass MutableMapping the whole feature doesn't exist (since you're not subclassing dict). It *only* exists for subclasses of dict.

On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar wrote:

...
But neither UserDict nor MutableMapping defines __missing__ ? What

is a

...
...
subclasser supposed to do?

On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote:

...
UserDict is superseded by MutableMapping.

I don't think we should make dict a kitchen sink class. I also don't think we should particularly encourage subclassing it. So -1 on

adding

...
dict.__missing__.

On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman wrote:

...
On 06/29/2016 12:09 PM, Guido van Rossum wrote:

> So providing the comprehensive base class is up to the user, not up > to > the stdlib. Is that such a big deal?

No, it's not. But it makes for a better user experience if the

On Tue, Jul 5, 2016 at 3:07 PM, Neil Girdhar wrote: base

...
...
...
...
...
class has the __missing__ method that raises a KeyError already.

Didn't we add a UserDict that could be subclassed primarily because subclassing dict directly was such a poor user experience?

If adding __missing__ to dict is huge (performance hit?), we don't do it. If it's not, I think we should. Maybe add it to UserDict if performance is a concern?

-- ~Ethan~ _______________________________________________ Python-ideas mailing list Python...@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python...@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

-- --Guido van Rossum (python.org/~guido)

Michael Selik

10:18 p.m.

On Tue, Jul 5, 2016 at 6:15 PM Neil Girdhar wrote:

...

Is it deprecated?

I've seen this question a lot on stackoverflow: http://stackoverflow.com/questions/7148419/subclass-dict-userdict-dict-or-ab...

http://stackoverflow.com/questions/2390827/how-to-properly-subclass-dict-and... http://stackoverflow.com/questions/10901048/i-want-to-subclass-dict-and-set-...

I still have no idea what the right answer is.

On Tue, Jul 5, 2016 at 6:11 PM Guido van Rossum wrote:

...
Because you shouldn't be using UserDict.

Subclass dict when you only want to change exactly the methods you override, and/or implement __missing__. Subclass MutableMapping when you want to take advantage of the mixin methods interacting with your implementations of the abstract methods.

Guido van Rossum

10:19 p.m.

I think this is a question for Raymond Hettinger. On Tue, Jul 5, 2016 at 3:13 PM, Neil Girdhar wrote:

...

Is it deprecated?

I've seen this question a lot on stackoverflow: http://stackoverflow.com/questions/7148419/subclass-dict-userdict-dict-or-ab... http://stackoverflow.com/questions/2390827/how-to-properly-subclass-dict-and... http://stackoverflow.com/questions/10901048/i-want-to-subclass-dict-and-set-... I still have no idea what the right answer is.

On Tue, Jul 5, 2016 at 6:11 PM Guido van Rossum wrote:

...
Because you shouldn't be using UserDict.

On Tue, Jul 5, 2016 at 3:07 PM, Neil Girdhar wrote:

...
Okay, that makes sense, but why isn't __missing__ in UserDict?

On Tue, Jul 5, 2016 at 6:04 PM Guido van Rossum wrote:

...
What kind of question is that? If you subclass MutableMapping the whole feature doesn't exist (since you're not subclassing dict). It *only* exists for subclasses of dict.

On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar wrote:

...
But neither UserDict nor MutableMapping defines __missing__ ? What is a subclasser supposed to do?

On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote:

...
UserDict is superseded by MutableMapping.

I don't think we should make dict a kitchen sink class. I also don't think we should particularly encourage subclassing it. So -1 on adding dict.__missing__.

On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman wrote: > On 06/29/2016 12:09 PM, Guido van Rossum wrote: > >> So providing the comprehensive base class is up to the user, not >> up >> to >> the stdlib. Is that such a big deal? > > > No, it's not. But it makes for a better user experience if the > base > class > has the __missing__ method that raises a KeyError already. > > Didn't we add a UserDict that could be subclassed primarily > because > subclassing dict directly was such a poor user experience? > > If adding __missing__ to dict is huge (performance hit?), we don't > do > it. > If it's not, I think we should. Maybe add it to UserDict if > performance > is > a concern? > > > -- > ~Ethan~ > _______________________________________________ > Python-ideas mailing list > Python...@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python...@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

-- --Guido van Rossum (python.org/~guido)

-- --Guido van Rossum (python.org/~guido)

Matt Gilson

6 Jul 6 Jul

5:11 a.m.

This discussion has my brain all twisted up... I'm going to try to work my way through it to make some sense of what we're saying here. 1. It is pretty easy to provide a subclass with a base `__missing__` implementation, so why should python be responsible for providing that behavior? 2. We don't want to encourage subclassing `dict`, so let's not make working with `__missing__` slightly nicer (see 1.) 3. Would adding `__missing__` to `dict` actually make working with it slightly nicer anyway? 4. Other places in the standard library for creating mappings easily (`UserDict` and `collections.Mapping`) do not support `__missing__``, but it is easy to build that functionality yourself (see 1. and example code below) Let me know if I left anything out or misrepresented any of the comments ... It definitely wasn't my intent. As I see it, there are three paths forward. - Do nothing. The status quo probably only surprises one or two users a year, so lets not rock the boat. Additionally, we can't add _everything_ everywhere. Things just become a mess if you say "Yes" to every idea and you have to draw the line somewhere. - Add `__missing__` to `dict`. It might be a simple pass-through method for raising `KeyError`, but hey, maybe it'll simplify a code-path or two. https://hg.python.org/cpython/file/tip/Objects/dictobject.c#l1723 - Add `__missing__` support to `collections.abc.Mapping` https://hg.python.org/cpython/file/tip/Lib/_collections_abc.py#l595 (or I guess UserDict, but I'd be just as happy if that one went away completely :-). I'm sure I haven't thought through all the pitfalls, but (additional documentation aside) it seems like it could be as easy as: class Mapping(Sized, Iterable, Container): def __missing__(self, key): raise KeyError @abc.abstractmethod def __getitem__(self, key): self.__missing__(key) ... On Tue, Jul 5, 2016 at 3:19 PM, Guido van Rossum wrote:

...

I think this is a question for Raymond Hettinger.

On Tue, Jul 5, 2016 at 3:13 PM, Neil Girdhar wrote:

...
Is it deprecated?

I've seen this question a lot on stackoverflow:

http://stackoverflow.com/questions/7148419/subclass-dict-userdict-dict-or-ab...

...
http://stackoverflow.com/questions/2390827/how-to-properly-subclass-dict-and...

...
http://stackoverflow.com/questions/10901048/i-want-to-subclass-dict-and-set-...

...
I still have no idea what the right answer is.

On Tue, Jul 5, 2016 at 6:11 PM Guido van Rossum wrote:

...
Because you shouldn't be using UserDict.

On Tue, Jul 5, 2016 at 3:07 PM, Neil Girdhar wrote:

...
Okay, that makes sense, but why isn't __missing__ in UserDict?

On Tue, Jul 5, 2016 at 6:04 PM Guido van Rossum wrote:

...
What kind of question is that? If you subclass MutableMapping the whole feature doesn't exist (since you're not subclassing dict). It *only* exists for subclasses of dict.

On Tue, Jul 5, 2016 at 12:18 PM, Neil Girdhar
...
...
...
wrote:

...
But neither UserDict nor MutableMapping defines __missing__ ? What is a subclasser supposed to do?

On Wednesday, June 29, 2016 at 4:59:03 PM UTC-4, Guido van Rossum wrote: > > UserDict is superseded by MutableMapping. > > I don't think we should make dict a kitchen sink class. I also don't > think we should particularly encourage subclassing it. So -1 on > adding > dict.__missing__. > > On Wed, Jun 29, 2016 at 12:30 PM, Ethan Furman < et...@stoneleaf.us> > wrote: > > On 06/29/2016 12:09 PM, Guido van Rossum wrote: > > > >> So providing the comprehensive base class is up to the user, not > >> up > >> to > >> the stdlib. Is that such a big deal? > > > > > > No, it's not. But it makes for a better user experience if the > > base > > class > > has the __missing__ method that raises a KeyError already. > > > > Didn't we add a UserDict that could be subclassed primarily > > because > > subclassing dict directly was such a poor user experience? > > > > If adding __missing__ to dict is huge (performance hit?), we don't > > do > > it. > > If it's not, I think we should. Maybe add it to UserDict if > > performance > > is > > a concern? > > > > > > -- > > ~Ethan~ > > _______________________________________________ > > Python-ideas mailing list > > Python...@python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python...@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

-- --Guido van Rossum (python.org/~guido)

-- --Guido van Rossum (python.org/~guido) _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- [image: pattern-sig.png] Matt Gilson // SOFTWARE ENGINEER E: matt@getpattern.com // P: 603.892.7736 We’re looking for beta testers. Go here https://www.getpattern.com/meetpattern to sign up!

Ethan Furman

5:51 a.m.

On 07/05/2016 10:11 PM, Matt Gilson wrote:

...

This discussion has my brain all twisted up... I'm going to try to work my way through it to make some sense of what we're saying here.

1. It is pretty easy to provide a subclass with a base `__missing__` implementation, so why should python be responsible for providing that behavior? 2. We don't want to encourage subclassing `dict`, so let's not make working with `__missing__` slightly nicer (see 1.) 3. Would adding `__missing__` to `dict` actually make working with it slightly nicer anyway? 4. Other places in the standard library for creating mappings easily (`UserDict` and `collections.Mapping`) do not support `__missing__``, but it is easy to build that functionality yourself (see 1. and example code below)

Let me know if I left anything out or misrepresented any of the comments ... It definitely wasn't my intent. As I see it, there are three paths forward.

* Do nothing. The status quo probably only surprises one or two users a year, so lets not rock the boat. Additionally, we can't add _everything_ everywhere. Things just become a mess if you say "Yes" to every idea and you have to draw the line somewhere. * Add `__missing__` to `dict`. It might be a simple pass-through method for raising `KeyError`, but hey, maybe it'll simplify a code-path or two. https://hg.python.org/cpython/file/tip/Objects/dictobject.c#l1723 * Add `__missing__` support to `collections.abc.Mapping` https://hg.python.org/cpython/file/tip/Lib/_collections_abc.py#l595 (or I guess UserDict, but I'd be just as happy if that one went away completely :-). I'm sure I haven't thought through all the pitfalls, but (additional documentation aside) it seems like it could be as easy as:

class Mapping(Sized, Iterable, Container): def __missing__(self, key): raise KeyError

@abc.abstractmethod def __getitem__(self, key): self.__missing__(key)

...

I will admit I don't understand the push-back against adding __missing__, and I'm mystified about the comment that users shouldn't be subclassing dict. I thought one of the key advances late in the 2.x series was easy(er) subclassing of the builtins. Unless it's a practicality thing, what with dict being written in such a subclass-hostile way (for performance reasions?): class ADict(dict): def __init__(self): super(ADict, self).__init__() self.names = [] def __delitem__(self, name): if name in self.names: self.names.remove(name) super(ADict, self).__delitem__(name) def __setitem__(self, name, value): if name not in self.names: self.names.append(name) super(ADict, self).__setitem__(name, value) my_dict = ADict() my_dict['howdy'] = 'bye' my_dict.update({'hola': 'adios'}) print(my_dict.names) # ['howdy'] print(my_dict.items()) # dict_items([('howdy','bye'), ('hola','adios')]) my_dict.pop('howdy') del my_dict['hola'] print(my_dict.names) # ['howdy'] print(my_dict.items()) # dict_items([]) -- ~Ethan~

Stephen J. Turnbull

10:39 a.m.

Ethan Furman writes:

...

__missing__, and I'm mystified about the comment that users shouldn't be subclassing dict. I thought one of the key advances late in the 2.x series was easy(er) subclassing of the builtins.

But that's not what was said, it was conditional. Sometimes you want to be subclassing dict, sometimes MutableMapping. But UserDict is apparently considered to be obsolete.

Paul Moore

11:19 a.m.

On 6 July 2016 at 06:51, Ethan Furman wrote:

...

I will admit I don't understand the push-back against adding __missing__

I thought that the idea was that __missing__ *if present* defined an alternative behaviour for missing keys. If it's not present, the default behaviour (key error) applies. Now it's certainly the case that having the base class provide an implementation of __missing__ that had the default behaviour would do the same - and in a "pure object oriented" sense, that's the way something like this "should" be implemented. But historically, it's not that way, and I suspect that it was much easier to bolt on an "if there's a __missing__ use it, otherwise proceed as before" check than it would have been to pull out the existing behaviour into a new (overridable) method. Python's never been particularly concerned about following pure OOP principles - "practicality beats purity" is relevant here. And I think the problem is that there's no obvious gain to making the default behaviour a concrete __missing__ method (and it would result in code churn in one of the most central and performance-sensitive areas of Python). So the push-back is mostly in the nature of "if it ain't broke, don't fix it" - and I don't think there's been a compelling (i.e., founded in practical issues) argument that anything is broken here. The original issue involved super() - and my understanding is that super() is intended for "co-operative" multiple inheritance, not for arbitrary "pass this method on without me needing to know the class hierarchy" boilerplate. Specifically, in class D(dict): def __missing__(self, k): super(D, self).__missing__(k) it's not clear why D needs a __missing__ method at all, if all it's doing is delegating to dict - we know dict has no __missing__ so D can also simply omit the method. (Possibly if D is being designed as a MI base class, that's not acceptable - but in which case there's not enough information here about the requirement to comment - maybe manually raising KeyError here is enough). Paul

2850

Age (days ago)

2857

Last active (days ago)

List overview

Download

20 comments

8 participants

participants (8)

Ethan Furman
Guido van Rossum
Matt Gilson
Michael Selik
Neil Girdhar
Paul Moore
Stephen J. Turnbull
Émanuel Barry

should `dict` supply a default `__missing__` implementation?

Matt Gilson

Matt Gilson

Matt Gilson

tags

participants (8)

should `dict` supply a default `missing` implementation?