[Python-Dev] PEP 469: Restoring the iterkeys/values/items() methods

[Steven D'Aprano]

On Fri, Apr 18, 2014 at 10:31:29PM -0400, Nick Coghlan wrote:
> After spending some time talking to the folks at the PyCon Twisted
> sprints, they persuaded me that adding back the iterkeys/values/items
> methods for mapping objects would be a nice way to eliminate a key
> porting hassle for them (and likely others), without significantly
> increasing the complexity of Python 3.

It would also considerable add to the cruft of Python 3. One motive for 
going through the pain of Python 2 to 3 migration was to remove cruft. 
Adding it back in again is not just an aid to porting but actively 
making Python 3 a worse (well, "less better") experience.

In the case of u'' syntax for strings, that was (somewhat) unavoidable, 
as you get a syntax error in Python 3 otherwise. So you cannot write:

    if sys.version < "3":
        s = u"êπд"
    else:
        s = "êπд"

it simply doesn't work. But you can write:

    if sys.version < "3":
        items = mydict.iteritems()  # or viewitems
    else:
        items = mydict.items()

Feature discovery is better than explicit version checks:

    try:
        items = mydict.iteritems()
    except AttributeError:
        items = mydict.items()


In my experience, writing polyglot 2+3 code can be easily handled with a 
few helper functions, which is not the case with unicode string 
literals. (Non-polygot code of course can just use the methods 
directly.) I don't see this is a problem to be solved, or even much of a 
nuisance. Polyglot code is never going to be quite as straightforward or 
idiomic as non-polyglot code, and that's a good thing, as it reminds the 
reader that they are dealing with polyglot code.

So while I'm sympathetic to wanting to ease the 2/3 transition, even at 
the expense of re-introducing some cruft when unavoidable, I don't think 
that the difficulty of dealing with dict items|iteritems|viewitems etc. 
justifies re-adding cruft.


[...] 
> Rationale
> =========
> 
> Similar in spirit to PEP 414 (which restored explicit Unicode literal
> support in Python 3.3), this PEP is aimed primarily at helping users
> that currently feel punished for making use of a feature that needed to be
> requested explicitly in Python 2, but was effectively made the default
> behaviour in Python 3.

"Feel punished"? That's awfully strong language. It may even be true, in 
the sense that some people *feel* that they are being punished, but I 
think the barrier to doing something about that needs to be a bit 
higher, namely that they *actually are* being punished.

I think that if "write a helper function" is punishment, then nearly 
every programmer is being punished *all the time*. That's part of 
programming. And if every 2+3 helper is seen as punishment that needs to 
be reversed, then we'll end up with Python 3.5 or 3.6 being virtually 
the same as Python 2.7 only with a few extra features.


> Users of list-based iteration in Python 2 that aren't actually relying on
> those semantics get a free memory efficiency improvement when migrating to
> Python 3, and face no additional difficulties when migrating via the common
> subset of Python 2 and 3.
> 
> By contrast, users that actually want the increased efficiency may have
> faced a three phase migration process by the time they have fully migrated
> to Python 3:
> 
> * original migration to the iterator based APIs after they were added in
>   Python 2.2

That was a long time ago. Surely we're not counting new features 
introduced a decade ago as part of the cost of migrating to Python 3?


> * migration to a separate function based API in order to run in the common
>   subset of Python 2 and 3
> * eventual migration back to unprefixed method APIs when finally dropping
>   Python 2.7 support at some point in the future

I'm not actually seeing the problem here. Forget 2+3, what you've 
described is surely part of the process of dealing with nearly any 
multi-version code. Back when I was still writing code to support Python 
2.3 through 2.5, I had helper functions like this:

def sort(alist, key=None):
    if key is None:
        alist.sort()
    else:
        try:
            alist.sort(key=key)
        except TypeError:
            # Python too old. Use the DSU idiom.
            ...

When I dropped support for 2.3, I eventually (but not at first) moved 
back from sort(mylist) to mylist.sort(). I don't see that the situation 
with iterkeys etc. is terribly different.


> This PEP proposes to just eliminate all that annoyance by making the iterator
> based APIs work again in Python 3.5+. As with the restoration of Unicode
> literals, it does add a bit of additional noise to the definition of Python
> 3, but it does so while bringing a significant benefit in increasing the size
> of the common subset of Python 2 and Python 3 and so simplifying the process
> of migrating to Python 3 for affected Python 2 users.

To me, this feels more like re-adding cruft for trivial benefit. I would 
want to see justification for why this gives *significant* benefit 
before I could support it. As the PEP stands now, I'm -1. I might be 
persuaded to change to 0 if the dict.iter* methods were added and 
immediately deprecated, rather than sold as a positive feature to be 
added to the Mapping API.



-- 
Steven