Classes with ordered namespaces

There are two relevant class namespaces in this proposal: definition namespace and __dict__. Currently both are dicts. For class definition namespaces, I'd like to make the default an OrderedDict. With the implementation in issue16991, the change is trivial (basically 1-liners in 2 spots). This change would make it unnecessary to write a custom metaclass just for a __prepare__(). PEP 422 alleviates that problem somewhat. However, I expect OrderedDict is by far the most common type returned by __prepare__(), so having it be the default would proportionately reduce the need for people to write metaclasses or learn about the PEP 422 API. You may ask what is the point if they aren't using a metaclass. That leads to the other half of the proposal. Once I have a class, I'd like to know the definition order without needing a metaclass. Unfortunately it's not as simple as using the C OrderedDict (issue16991 again) for tp_dict, etc. That change is actually pretty trivial. However, it causes problems because the concrete C API (PyDict_*) doesn't play nice with subclasses and the concrete API is used on class __dict__ all over the place. The alternative I propose is to add __definition_order__ or similar to classes. It would be a list of the names from the definition namespace, in definition order (making use of the new default there). Having a class's __dict__ be ordered isn't important if the definition order is available somewhere. Again, using the C OrderedDict, the change to add __definition_order__ is pretty trivial. FWIW, Guido already expressed some approval[1]. -eric [1] http://mail.python.org/pipermail/python-ideas/2013-February/019704.html

On 27 June 2013 16:03, Eric Snow <ericsnowcurrently@gmail.com> wrote:
There are two relevant class namespaces in this proposal: definition namespace and __dict__. Currently both are dicts.
For class definition namespaces, I'd like to make the default an OrderedDict. With the implementation in issue16991, the change is trivial (basically 1-liners in 2 spots). This change would make it unnecessary to write a custom metaclass just for a __prepare__(). PEP 422 alleviates that problem somewhat. However, I expect OrderedDict is by far the most common type returned by __prepare__(), so having it be the default would proportionately reduce the need for people to write metaclasses or learn about the PEP 422 API. You may ask what is the point if they aren't using a metaclass.
I'd be tempted to kill PEP 422 as not worth the hassle if we did this. Yes, I count that as a point in favour of the idea :)
That leads to the other half of the proposal.
Once I have a class, I'd like to know the definition order without needing a metaclass. Unfortunately it's not as simple as using the C OrderedDict (issue16991 again) for tp_dict, etc. That change is actually pretty trivial. However, it causes problems because the concrete C API (PyDict_*) doesn't play nice with subclasses and the concrete API is used on class __dict__ all over the place. The alternative I propose is to add __definition_order__ or similar to classes. It would be a list of the names from the definition namespace, in definition order (making use of the new default there). Having a class's __dict__ be ordered isn't important if the definition order is available somewhere. Again, using the C OrderedDict, the change to add __definition_order__ is pretty trivial.
I think the main concern I would have is whether other implementations are happy they can provide a suitable ordered dictionary for class namespace execution. It's also worth considering what would have to happen for dynamically created types where the namespace passed in *wasn't* ordered. It's a plausible way forward, though. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Thu, Jun 27, 2013 at 2:48 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I think the main concern I would have is whether other implementations are happy they can provide a suitable ordered dictionary for class namespace execution.
It's also worth considering what would have to happen for dynamically created types where the namespace passed in *wasn't* ordered.
Good points. In either case there is no definition order available. I'm okay with that. Would it be better to represent that as None (and the attribute is always defined) or by having the attribute undefined? I'd rather always have the attribute, but I expect the significantly simpler solution is to leave the attribute undefined when definition order is not available. So I'd go with the latter. -eric

On 06/27/2013 08:18 AM, Eric Snow wrote:
On Thu, Jun 27, 2013 at 2:48 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I think the main concern I would have is whether other implementations are happy they can provide a suitable ordered dictionary for class namespace execution.
It's also worth considering what would have to happen for dynamically created types where the namespace passed in *wasn't* ordered.
Good points. In either case there is no definition order available. I'm okay with that. Would it be better to represent that as None (and the attribute is always defined) or by having the attribute undefined? I'd rather always have the attribute, but I expect the significantly simpler solution is to leave the attribute undefined when definition order is not available. So I'd go with the latter.
So in Python space the options are either: if some_class.__definition_order__ is not None: or if getattr(some_class, '__definition_order__', None) is not None: ? Seems like always defined would be easier. -- ~Ethan~

On Thu, Jun 27, 2013 at 9:35 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 06/27/2013 08:18 AM, Eric Snow wrote:
Good points. In either case there is no definition order available. I'm okay with that. Would it be better to represent that as None (and the attribute is always defined) or by having the attribute undefined? I'd rather always have the attribute, but I expect the significantly simpler solution is to leave the attribute undefined when definition order is not available. So I'd go with the latter.
So in Python space the options are either:
if some_class.__definition_order__ is not None:
or
if getattr(some_class, '__definition_order__', None) is not None:
?
Seems like always defined would be easier.
Certainly it's easier to use if you need the definition order. However, I expect it wouldn't be used often enough that that would make a big difference. Furthermore, if it's always there then it will always show up when you dir(SomeClass) or look at SomeClass.__dict__. People would see that on every class, whether it actually provided definition order or not. I'm not sure that is as helpful as only having the attribute around when order is defined. This got me thinking. The attribute should be a class-only attribute (i.e. a property on type) like __name__ is. That would keep it from showing up on instance lookups and on every vars(obj). The big thing for me, though, is that optionally having the attribute accommodates Python implementations that can't (or don't) preserve definition order. They don't have to worry about adding a dummy attribute to every class just for the sake of complying with the spec. It also means the implementation for dynamic types like Nick mentioned don't necesarily need to be touched. The more I think about it the more the optionally set attribute approach is the way to go. -eric

On 28 Jun 2013 07:51, "Eric Snow" <ericsnowcurrently@gmail.com> wrote:
On Thu, Jun 27, 2013 at 9:35 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 06/27/2013 08:18 AM, Eric Snow wrote:
Good points. In either case there is no definition order available. I'm okay with that. Would it be better to represent that as None (and the attribute is always defined) or by having the attribute undefined? I'd rather always have the attribute, but I expect the significantly simpler solution is to leave the attribute undefined when definition order is not available. So I'd go with the latter.
So in Python space the options are either:
if some_class.__definition_order__ is not None:
or
if getattr(some_class, '__definition_order__', None) is not None:
?
Seems like always defined would be easier.
Certainly it's easier to use if you need the definition order. However, I expect it wouldn't be used often enough that that would make a big difference. Furthermore, if it's always there then it will always show up when you dir(SomeClass) or look at SomeClass.__dict__. People would see that on every class, whether it actually provided definition order or not. I'm not sure that is as helpful as only having the attribute around when order is defined.
This got me thinking. The attribute should be a class-only attribute (i.e. a property on type) like __name__ is. That would keep it from showing up on instance lookups and on every vars(obj).
The big thing for me, though, is that optionally having the attribute accommodates Python implementations that can't (or don't) preserve definition order. They don't have to worry about adding a dummy attribute to every class just for the sake of complying with the spec. It also means the implementation for dynamic types like Nick mentioned don't necesarily need to be touched. The more I think about it the more the optionally set attribute approach is the way to go.
My experience with maybe set, maybe not attributes for modules is that they're *always* a PITA to deal with and just not worth the hassle. Better to just expose it as a read-only attribute on class objects (that isn't accessible through instances) and set it to None on construction if order info is not available. This will also block accidental inheritance of the attribute from an ordered base class. A new keyword-only argument for the dynamic type construction APIs in the "types" module would also be appropriate, and would allow the creation of ordered classes even if the implementation used unordered namespaces for class syntax. Cheers, Nick.
-eric _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com

On Thu, Jun 27, 2013 at 4:48 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I'd be tempted to kill PEP 422 as not worth the hassle if we did this.
I assume you mean the "namespace" keyword part of PEP 422, since PEP 422's class initialization feature is entirely orthogonal to definition order or namespace customization. (Indeed, I cannot recall a single instance of class initialization in my code that actually *cared* about definition order. Certainly I haven't had any situations where a pre-existing definition order would've eliminated the need for a class-level initialization.)

On 28 Jun 2013 07:46, "PJ Eby" <pje@telecommunity.com> wrote:
On Thu, Jun 27, 2013 at 4:48 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I'd be tempted to kill PEP 422 as not worth the hassle if we did this.
I assume you mean the "namespace" keyword part of PEP 422, since PEP 422's class initialization feature is entirely orthogonal to definition order or namespace customization. (Indeed, I cannot recall a single instance of class initialization in my code that actually *cared* about definition order. Certainly I haven't had any situations where a pre-existing definition order would've eliminated the need for a class-level initialization.)
That and the second argument to the new initialisation hook. So you're right, the consequences for PEP 422 would be "greatly simplify" rather than "kill". Cheers, Nick.
participants (4)
-
Eric Snow
-
Ethan Furman
-
Nick Coghlan
-
PJ Eby