[Tutor] How to override getting items from a list for iteration

[Oscar Benjamin]

On 10 February 2013 14:32, Walter Prins <wprins at gmail.com> wrote:
[snip
>
> This worked mostly fine, however yesterday I ran into a slightly unexpected
> problem when I found that when the list contents is iterated over and values
> retrieved that way rather than via [], then __getitem__ is in fact *not*
> called on the list to read the item values from the list, and consequently I
> get back the "not yet calculated" entries in the list, without the
> calculation routine being automatically called as is intended.
>
> Here's a test application that demonstrates the issue:
>
> class NotYetCalculated:
>     pass
>
> class CalcList(list):
>     def __init__(self, calcitem):
>         super(CalcList, self).__init__()
>         self.calcitem = calcitem
>
>     def __getitem__(self, key):
>         """Override __getitem__ to call self.calcitem() if needed"""
>         print "CalcList.__getitem__(): Enter"
>         value = super(CalcList, self).__getitem__(key)
>         if value is NotYetCalculated:
>             print "CalcList.__getitem__(): calculating"
>             value = self.calcitem(key)
>             self[key] = value
>         print "CalcList.__getitem__(): return"
>         return value
>
> def calcitem(key):
>     # Demo: return square of index
>     return key*key
>
> What's the best way to fix this problem?  Do I need to maybe override
> another method, perhaps provide my own iterator implementation? For that
> matter, why doesn't iterating over the list contents fall back to calling
> __getitem__?

It would use __getitem__ if __iter__ wasn't defined. e.g.:

>>> class A(object):
...   def __getitem__(self, index):
...     if index > 4:
...       raise IndexError
...     return index ** 2
...
>>> a = A()
>>> for x in a:
...   print(x)
...
0
1
4
9
16

The problem is that by subclassing list you inherit its __iter__
method. A for loop begins by calling iter()  on the iterable. The iter
function is roughly like:

def iterseq(seq):
  count = 0
  while True:
    try:
       yield seq[count]
    except IndexError:
      return
    count += 1

def iter(iterable):
  if hasattr(iterable, '__iter__'):
     return iterable.__iter__()
  elif hasattr(iterable, '__getitem__'):
    return iterseq(iterable)
  else:
    raise TypeError


Oscar