[New-bugs-announce] [issue40116] Regression in memory use of shared key dictionaries for "compact dicts"

[Mark Shannon]

New submission from Mark Shannon <mark at hotpy.org>:

The current implementation of dicts prevents keys from being shared when the order of attribute differs from the first instance created.
This can potentially use a considerably larger amount of memory than expected.

Consider the class:

class C:

   opt = DEFAULT

   def __init__(self, attr, optional=None):
       if optional:
           self.opt = optional
       self.attr = attr

This is a reasonable way to write a class, but has unpredictable memory use.
In the attached example, per-instance dict size goes from 104 bytes to 232 bytes when sharing is prevented.

The language specification says that the dicts maintain insertion order, but the wording implies that this only to explicit dictionaries, not instance attribute or other namespace dicts.

Either we should allow key sharing in these cases, or clarify the documentation.

----------
components: Interpreter Core
files: compact_dict_prevents_key_sharing.py
messages: 365319
nosy: Mark.Shannon, inada.naoki
priority: normal
severity: normal
stage: test needed
status: open
title: Regression in memory use of shared key dictionaries for "compact dicts"
type: behavior
Added file: https://bugs.python.org/file49013/compact_dict_prevents_key_sharing.py

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40116>
_______________________________________