<div dir="ltr">Hello,<div><br></div><div style>I have a program where I'm overriding the retrieval of items from a list. As background: The data held by the lists are calculated but then read potentially many times thereafter, so in order to prevent needless re-calculating the same value over and over, and to remove checking/caching code from the calculation logic code, I therefore created a subclass of list that will automatically calculate the value in a given slot automatically if not yet calculated. (So differently put, I'm implemented a kind of list specific caching/memoization with the intent that it should be transparent to the client code.) </div>
<div style><br></div><div style>The way I've implemented this so far was to simply override list.__getitem__(self, key) to check if the value needs to be calculated or not and call a calculation method if required, after which the value is returned as normal. On subsequent calls __getitem__ then directly returns the value without calculating it again.</div>
<div style><br></div><div style>This worked mostly fine, however yesterday I ran into a slightly unexpected problem when I found that when the list contents is iterated over and values retrieved that way rather than via [], then __getitem__ is in fact *not* called on the list to read the item values from the list, and consequently I get back the "not yet calculated" entries in the list, without the calculation routine being automatically called as is intended. </div>
<div style><br></div><div style>Here's a test application that demonstrates the issue:<br></div><div style><br></div><div style><div><div>class NotYetCalculated:</div><div> pass</div><div><br></div><div>class CalcList(list):</div>
<div> def __init__(self, calcitem):</div><div> super(CalcList, self).__init__()</div><div> self.calcitem = calcitem</div><div><br></div><div> def __getitem__(self, key):</div><div> """Override __getitem__ to call self.calcitem() if needed"""</div>
<div> print "CalcList.__getitem__(): Enter"</div><div> value = super(CalcList, self).__getitem__(key)</div><div> if value is NotYetCalculated:</div><div> print "CalcList.__getitem__(): calculating"</div>
<div> value = self.calcitem(key)</div><div> self[key] = value</div><div> print "CalcList.__getitem__(): return"</div><div> return value</div><div><br></div><div>def calcitem(key):</div>
<div> # Demo: return square of index</div><div> return key*key</div><div><br></div><div><br></div><div>def main():</div><div> # Create a list that calculates its contents via a given </div><div> # method/fn onece only</div>
<div> l = CalcList(calcitem)</div><div> # Extend with few entries to demonstrate issue:</div><div> l.extend([NotYetCalculated, NotYetCalculated, NotYetCalculated, </div><div> NotYetCalculated])</div>
<div> </div><div> print "1) Directly getting values from list works as expected: __getitem__ is called:"</div><div> print "Retrieving value [2]:\n", l[2] </div><div> print</div><div> print "Retrieving value [3]:\n", l[3]</div>
<div> print</div><div> print "Retrieving value [2] again (no calculation this time):\n", l[2]</div><div> print</div><div> </div><div> print "Retrieving values via an iterator doesn't work as expected:"</div>
<div> print "(__getitem__ is not called and the code returns "</div><div> print " NotYetCalcualted entries without calling __getitem__. How do I fix this?)"</div><div> print "List contents:"</div>
<div> for x in l: print x</div><div> </div><div> </div><div>if __name__ == "__main__":</div><div> main()</div></div><div style><br></div><div style>To reiterate:</div><div style><br></div><div style>
What should happen: In test 2) above all entries should be automatically calculated and output should be numbers only.</div><div style><br></div><div style>What actually happens: In test 2) above the first 2 list entries corresponding to list indexes 0 and 1 are output as "NotYetCalculated" and calcitem is not called when required. </div>
<div><br></div></div><div style>What's the best way to fix this problem? Do I need to maybe override another method, perhaps provide my own iterator implementation? For that matter, why doesn't iterating over the list contents fall back to calling __getitem__?<br>
</div><div style><div><br></div></div><div style>Thanks in advance,</div><div style><br></div><div style>Walter</div><div style><br></div></div>