[Python-Dev] PEP520 and absence of __definition_order__
Guido van Rossum
guido at python.org
Sat Sep 10 17:26:58 EDT 2016
On Sat, Sep 10, 2016 at 10:57 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 11 September 2016 at 03:08, Guido van Rossum <guido at python.org> wrote:
>> So I'm happy to continue thinking about this, but I expect this is not
>> such a big deal as you fear. Anyway, let's see if someone comes up
>> with a more convincing argument by beta 2!
>
> For CPython specifically, I don't have anything more convincing than
> Ethan's Enum example (where the way the metaclass works means most of
> the interesting attributes don't live directly in the class dict, they
> live in private data structures stored in the class dict, making
> "list(MyEnum.__dict__)" inherently uninteresting, regardless of
> whether it's ordered or not).
But that would only matter if we also defined a helper utility that
used __definition_order__. I expect that the implementation of Enum
could be simplified somewhat in Python 3.6 since it can trust that the
namespace passed into __new__ is ordered (so it doesn't have to switch
it to an OrderedDict in __prepare__, perhaps).
In any case the most likely way to use __definition_order__ in general
was always to filter its contents through some other condition (e.g.
"isn't a method and doesn't start with underscore") -- you can do the
same with keys(). Classes that want to provide a custom list of
"interesting" attributes can provide that using whatever class method
or attribute they want -- it's just easier to keep those attributes
ordered because the namespace is always ordered.
> The proxy use cases I'm aware of (wrapt, weakref.proxy) tend to be
> used to wrap normal instances rather than class objects themselves, so
> they shouldn't be affected.
>
> With ordered-by-default class namespaces, both heap types and non-heap
> types should also mostly be populated in the "logical order" (i.e. the
> order names appear in the relevant C arrays), but that would formally
> be an implementation detail at this point, rather than something we
> commit to providing.
>
> The only other argument that occurs to me is one that didn't come up
> in the earlier PEP 520 discussions: how a not-quite-Python
> implementation (or a Python 3.5 compatible implementation that doesn't
> offer order-preserving behaviour the way PyPy does) can make sure that
> code that relies on ordered class namespaces *fails* in an informative
> way when run on that implementation.
Is that a real use case? It sounds like you're just constructing an
artificial example that would be less convenient without
__definition_order__.
> With __definition_order__, that's straightforward - the code that
> needs it will fail with AttributeError, and searching for the
> attribute named in the exception will lead affected users directly to
> PEP 520 and the Python 3.6 What's New guide.
But that code would have to be written to use __definition_order__. It
could just as easily be written to assert that sys.version_info() >=
(3, 6).
> With implicitly ordered class namespaces, you don't get an exception
> if the namespace isn't actually order preserving - you get attributes
> in an arbitrary order instead. Interpreters can't detect that the user
> specifically wanted order preserving behaviour, and library and
> application authors can't readily detect whether or not the runtime
> offers order preserving behaviour (since they may just get lucky on
> that particular run).
That sounds very philosophical. You still can't check whether *dict*
is order-preserving -- all you can do is checking whether a *class*
preserves its order. Since PEP 520 is accepted only for Python 3.6,
checking for the presence of __definition_order__ is no different than
checking the version.
> Even if we added a new flag to sys.implementation that indicated the
> use of order preserving class namespaces, there'd still be plenty of
> scope for subtle bugs where libraries and frameworks weren't checking
> that flag before relying on the new behaviour.
OK, I'm beginning to see the argument here. You want all code that
relies on the order to be explicitly declaring that it does so by
using a new API.
Unfortunately the mere presence of __definition_order__ doesn't really
help here -- since all dicts are order-preserving, there's still
nothing (apart from documentation) to stop apps from relying on the
ordering of the class __dict__ directly.
> Cheers,
> Nick.
>
> P.S. I'd actually love it if we could skip __definition_order__ -
> there really is a whole lot of runtime clutter on class objects, and
> we're adding __annotations__ as well. Unfortunately, I also think we
> made the right call the first time around in thinking it would still
> be necessary even if class namespaces became order preserving :)
Note that __annotations__ is only added when there are annotations, so
its presence could be used as a flag of sorts. (However you shouldn't
use it directly -- each class in the MRO has its own __annotations__,
and you should use typing.get_type_hints(cls) to coalesce all of
them.)
--
--Guido van Rossum (python.org/~guido)
More information about the Python-Dev
mailing list