[Python-ideas] Dict literal use for custom dict classes

[Andrew Barnert]

On Dec 17, 2015, at 10:37, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> 
>> On Wed, Dec 16, 2015 at 2:50 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Since requests to be able to access the order of values,
>> parameters and definitions in source code come up rather
>> often, perhaps it'd better to provide Python application
>> with a standard access mechanism to this order rather than
>> trying to push use of OrderedDict and the like into the
>> runtime, causing unnecessary performance overhead.
>> 
>> The parser does have access to this information in the AST
>> and some of it is partially copied into code object attributes,
>> but there's no general purpose access to the information.
>> 
>> Based on the source code order, you could do lots of
>> things, e.g. avoid hacks to map class attributes to
>> column definitions for ORMs, make it possible to write
>> OrderedDict(a=x, b=y) and have the literal order preserved,
>> have NamedTuple(a=x, b=y) work without additional tricks,
>> etc.
>> 
>> What's important here is that the runtime performance
>> would not change. The code objects would
>> gain some additional tuples, which store the order of
>> the literals used in their AST, so only the memory
>> consumption would increase.
> 
> +1 from me.  That's essentially the goal of PEP 468. [1]  While the
> proposed solution focuses on OrderedDict, note the various
> alternatives at the bottom of the PEP.  Also note that OrderedDict now
> has a C implementation that doesn't suffer from the same performance
> penalty. [2]

It seems like whatever the resolution of this discussion (unless it's "people mostly agree that X would be acceptable, doable, and probably desirable, but nobody's going to do it") you may want to rewrite and repush PEP 468.

After all, if Python 3.6 changes to make all dicts ordered, or make dict literals preserve literal order at least until first mutation (or anything in between--as Bruce Leban pointed out, there are at least three sensible things in between rather than just the one I suggested, and of courseall of them are good enough here), or to stash AST links on code objects, etc., you should be able to show how PEP 468 only adds a trivial requirement to any implementation of Python 3.6, and then the concerns come down to "maybe on some implementations it would be slightly faster to construct the kwdict in reverse, but now it will have to not do that".

Conversely if someone convinces everyone that there is no solution that works for dict literals, you'd probably need to explain why that same problem doesn't apply to kwargs to keep the PEP alive.

> 
> -eric
> 
> [1] http://legacy.python.org/dev/peps/pep-0468/
> [2] OrderedDict is actually faster for iteration, the same speed for
> other non-mutation operations, not much slower for most mutation
> operations, and 4x slower in the worst case.  That is a drastic
> improvement over the pure Python OrderedDict.

IIRC, the one concern of Guido's that you couldn't answer was that if someone keeps the kwdict and adds to it, he could end up wasting a lot of space, not just time. If OrderedDict is still 150-200% bigger than dict, as in the pure Python version, that's still a problem.