[Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

Nick Coghlan ncoghlan at gmail.com
Sat Apr 19 04:31:29 CEST 2014

After spending some time talking to the folks at the PyCon Twisted
sprints, they persuaded me that adding back the iterkeys/values/items
methods for mapping objects would be a nice way to eliminate a key
porting hassle for them (and likely others), without significantly
increasing the complexity of Python 3.

I personally put this one in the same category as PEP 414 - not
particularly useful from a Python 3 perspective, but not really
harmful either, and helpful enough from a transition perspective to be
worth doing.


PEP: 469
Title: Simplified migration of iterator-based mapping code to Python 3
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2014-04-18
Python-Version: 3.5
Post-History: 2014-04-18


For Python 3, PEP 3106 changed the design of the ``dict`` builtin and the
mapping API in general to replace the separate list based and iterator based
APIs in Python 2 with a merged, memory efficient set and multiset view
based API.

This means that Python 3 code always requires an additional qualifier to
reliably reproduce classic Python 2 mapping semantics:

    * List based (e.g. ``d.keys()``): ``list(d.keys())``
    * Iterator based (e.g. ``d.iterkeys()``): ``iter(d.keys())``

Some Python 2 code that uses ``d.keys()`` may be migrated to Python 3
(or the common subset of Python 2 and Python 3) without alteration, but
*all* code using the iterator based API requires modification. Code that
is migrating to the common subset of Python 2 and 3 and needs to retain the
memory efficient implementation that avoids creating an unnecessary list
object must switch away from using a method to instead using a helper
function (such as those provided by the ``six`` module)

To simplify the process of migrating Python 2 code that uses the existing
iterator based APIs to Python 3, this PEP proposes the reintroduction
of the Python 2 spelling of the iterator based semantics in Python 3.5, by
restoring the following methods to the builtin ``dict`` API and the
``collections.abc.Mapping`` ABC definition:

    * ``iterkeys()``
    * ``itervalues()``
    * ``iteritems()``


Methods with the following exact semantics will be added to the builtin
``dict`` type and ``collections.abc.Mapping`` ABC::

    def iterkeys(self):
        return iter(self.keys())

    def itervalues(self):
        return iter(self.values())

    def iteritems(self):
        return iter(self.items())

These semantics ensure that the methods also work as expected for subclasses
of these base types.


Similar in spirit to PEP 414 (which restored explicit Unicode literal
support in Python 3.3), this PEP is aimed primarily at helping users
that currently feel punished for making use of a feature that needed to be
requested explicitly in Python 2, but was effectively made the default
behaviour in Python 3.

Users of list-based iteration in Python 2 that aren't actually relying on
those semantics get a free memory efficiency improvement when migrating to
Python 3, and face no additional difficulties when migrating via the common
subset of Python 2 and 3.

By contrast, users that actually want the increased efficiency may have
faced a three phase migration process by the time they have fully migrated
to Python 3:

* original migration to the iterator based APIs after they were added in
  Python 2.2
* migration to a separate function based API in order to run in the common
  subset of Python 2 and 3
* eventual migration back to unprefixed method APIs when finally dropping
  Python 2.7 support at some point in the future

The view based APIs that were added to Python 2.7 don't actually help with
the transition process, as they don't exist in Python 3 and hence aren't
part of the common subset of Python 2 and Python 3, and also aren't supported
by most Python 2 mappings (including the collection ABCs).

This PEP proposes to just eliminate all that annoyance by making the iterator
based APIs work again in Python 3.5+. As with the restoration of Unicode
literals, it does add a bit of additional noise to the definition of Python
3, but it does so while bringing a significant benefit in increasing the size
of the common subset of Python 2 and Python 3 and so simplifying the process
of migrating to Python 3 for affected Python 2 users.


Thanks to the folks at the Twisted sprint table at PyCon for a very
vigorous discussion of this idea (and several other topics), and especially
to Hynek Schlawack for acting as a moderator when things got a little too
heated :)


This document has been placed in the public domain.

Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-Dev mailing list