[Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

[MRAB]

On 2016-09-13 11:44, Nick Coghlan wrote:
> On 13 September 2016 at 08:25, Gregory P. Smith <greg at krypto.org> wrote:
>> At this point I think coding up an example patch against beta1 offering a
>> choice of disordered iteration capability that does not increase memory or
>> iteration overhead in any significant way is needed.
>>
>> The problem is... I don't know how to express this as an API. Which sinks my
>> whole though process and tables the idea.
>>
>> A parameter to .items(), .keys() and .values() is undesirable as it isn't
>> backwards compatible [meaning it'll never be used] and .keys() needs to
>> match __iter__ which can't have one anyways. A parameter on dict
>> construction is similarly infeasible.
>>
>> Requiring the use of an orderdict like type in order to get the behavior is
>> undesirable. Effectively I'm asking for some boolean state in each dict as
>> to if it should iterate in order or not and a way to expose that to pure
>> Python code in a way that namespace dicts iterate in order by default and
>> others do not unless explicitly configured to do so.
>>
>> oh well.  end thought process on my end.  it was good while it lasted.
>
> I think this is looking at the compatibility testing problem from the
> wrong direction anyway, as rather than making it difficult for people
> to implicitly depend on the default key ordering, the scenario we
> would want to help with is this one:
>
> 1. Library developer inadvertently depends on the
> dicts-are-ordered-by-default implementation detail
> 2. Library user reports "your library isn't working for me on <other
> implementation without that behaviour>"
> 3. Library developer figures out the problem, and would like to update
> their test suite to deliberately provoke the misbehaviour
> 4. ???
>
> That is, it falls into the same category as folks depending on
> CPython's reference counting for prompt resource cleanup, where we
> offer ResourceWarning to detect such cases, and context managers to
> clean them up more explicitly.
>
> For dict ordering dependence, anyone regularly testing against CPython
> 2.7 and CPython 3.5 will already have a good chance of detecting key
> order reliance just through hash randomisation (e.g. I hit an
> "inconsistent key order in generated JSON makes line-based diffing
> unreadable" one myself last week with a 3-entry dict for md5, sha1 and
> sha256 hashes - it was relatively rare to get the same key order two
> runs in a row)
>
> That means the only problematic case is the one where the only CPython
> version a project supports is 3.6+ *and* they want to support
> alternate implementations that don't preserve order in their default
> dict implementation.
>
> Given that current alternate implementations are still in the process
> of catching up to *3.5* (or Python 3 at all in the case of Jython and
> IronPython), I think we still have a good few years to ponder the
> question before this particular concern starts cropping up in practise
> :)
>
The recommended way of dealing with features across different versions 
of Python is to check for them and see if they raise NameError or 
whatever, but I wonder if there would be any benefit to recording such 
things somewhere, e.g. sys.features['ordered_args'] returns True if 
arguments are passed in an ordered dict.