[Python-3000] Iterators for dict keys, values, and items == annoying :)

[Tim Peters]

[Jim Fulton]
> ...
> I'd be interested to hear if other people who have experience working
> with ZODB BTrees have been as annoyed as I've been.

Not since the first week ;-), no.  Two things exacerbate this in ZODB:

- BTrees are a mapping type, but unlike other mapping types its keys()
  (etc) methods don't return a list.  This makes the result of BTree.keys()
  surprising to many, just because it's different from the way other
  keys() methods work.

- ZODB Buckets are also mapping types, and pretty much interchangeable
  with BTrees (they support the same extended (relative to the base mapping
  interface) set of methods with the same meanings), _except_ that
  Bucket.keys() (etc) returns a list.  Sometimes operations on BTrees even
  return Buckets, so from one line of code to the next it's hard to remember
  whether keys() (etc) will return a list or an iterator.

That said, it doesn't much matter, since BTrees.keys() (etc) returns a
particularly rich kind of iterator (as you know, it supports, e.g.,
__len__ and indexing, much like a list).  There were only two ways I
got surprised:

- Typing, e.g.,

      >>> b.keys()

  at an interactive shell to see the keys, and getting back

      <OOBTreeItems object at 0x00AD9C00>

- Writing, e.g.,

      self.assertEqual(b.keys(), [1, 2, 3])

  in a Bucket unit test, forgetting that it was in a test class that was also
  (re)used to test BTrees.

"The solution" in both cases was to wrap the method result in list(),
and stop caring that this would make a "needless" copy when `b` was in
fact a Bucket.

If dict.keys() (etc) had also returned an iterator, I doubt anyone
would have been surprised by any of the above.

There's one other common surprise in Zope-land, namely that

    for key in b.keys():
        ... possibly try to delete `key` from `b`

"doesn't work" when `b` is a BTree.  The _expectation_, derived from
experience with Python dicts, is that it's bulletproof, but that's
again because dict.keys() has returned a distinct list and BTrees were
just different that way.  It's a little nastier for  BTrees because
they can't reliably detect a size change during iteration, so BTree
users _usually_ don't get the

>>> d = {1: 2, 3: 4}
>>> for key in d:
...     del d[key]
...
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
RuntimeError: dictionary changed size during iteration

they're accustomed to when they try to mutate a dict that changes size
during iteration.