[Baypiggies] Tricky dictionary iteration problem

Hy Carrinski hcarrinski at gmail.com
Fri Jan 6 07:21:54 CET 2012


Here are a few brief thoughts and then documentation. I appreciate
consideration of the limits of writing a note rapidly on my mobile device.

1. I think it will be hard to develop maintainable clean code by
subclassing a dictionary to make it function like a dictionary but produce
alternate output.

2. When one opens IDLE and chooses "open module" from the "File" menu, one
can look at the collections module. Several of the methods of the
OrderedDict depend on __iter__() so overriding it may produce interesting
results (see info on subclassing in Python below).

3. In the collections module one can see the prevalence of yield
statements. An iterator rarely benefits from returning a list.

>From http://docs.python.org/tutorial/classes.html

"Derived classes may override methods of their base classes. Because
methods have no special privileges when calling other methods of the same
object, a method of a base class that calls another method defined in the
same base class may end up calling a method of a derived class that
overrides it."
For reference, here is an example of the __iter__ method being overridden.
Apologies for removing comments/license when transcribing Python 2.7 source.

def __iter__():
    NEXT, KEY = 1, 2
    root = self.__root
    curr = root[NEXT]
    while curr is not root:
        yield curr[KEY]
        curr = curr[NEXT]

Hope these opinions and information add to the dialog.

Hy

On Jan 5, 2012, at 9:06 PM, "corey.coughlin at comcast.net" <
corey.coughlin at comcast.net> wrote:

Thanks, that looks pretty good.  It still fails doing a .copy(), so I'll
have to rewrite that.  And see what else I need to fix.

As far as why I want to do it goes, well, I originally had an IndexedList
class, which contained a list and a dictionary.  All the objects in the
list had a .reference string attribute, and it worked mostly like a list
with an index on it.  I wrote getitem to check the index to see if it were
an integer or a string, and if it was a string use the dictionary reference
for a fast look up of the object.  The problem there is that it eats a lot
of memory for big data sets, so I was looking at using an OrderedDict based
object to emulate that behavior.  Works pretty well, except that I need to
change all my loops from:
for obj in indexlist:
to
for obj in indexlist.values():
and that's kind of a pain.  The simpler way to pull it off is to not
inherit from OrderedDict, but just have an object attribute that holds an
ordered dict and add all the mapping and iteration function calls
manually.  That works, but I was looking for a simpler way to do it.  Looks
like either way, I'll have to rewrite some functions.  Thanks for the help!

------- Corey

------------------------------
*From: *"Robert Zuber" <rob at zuber.net>
*To: *"corey coughlin" <corey.coughlin at comcast.net>
*Cc: *baypiggies at python.org
*Sent: *Thursday, January 5, 2012 5:22:14 PM
*Subject: *Re: [Baypiggies] Tricky dictionary iteration problem

using super to avoid the recursion seems to work:

>>> class od2(collections.OrderedDict):
...     def __iter__(self):
...         return (v for v in super(collections.OrderedDict,
self).itervalues())
...
>>> od = od2()
>>> od['a'] = 'b'
>>> od['c'] = 'd'
>>> [v for v in od]
['b', 'd']



On Jan 5, 2012, at 4:57 PM, corey.coughlin at comcast.net wrote:

> Hi all,
>    Sorry to bug everybody, but I'm having some trouble with a somewhat
tricky class I'm trying to make.  I have a class that inherits from
collections.OrderedDict, in an effort to create an ordered dictionary with
some extra functions.  Now what I'd like to do is to change the iteration
so that when you iterate over it, it iterates over values instead of keys.
 This is turning out to be harder than I thought.  First, I tried this:
>
> class od2(collections.OrderedDict):
>     def __iter__(self):
>         return iter(self.values())
>
> Now the code I'm running creates a dict and tries to do a .copy on it,
but I'd expect the problem to come up with most kind of dictionary access.
 The problem is that it goes into an infinite recursion with messages like
this
>
> File "...", line 57, in __iter__
>   return iter(self.values())
> File ".../python/2.7.1/linux64/lib/python2.7/_abcoll.py" line 372 in
values
>   return [self[key] for key in self]
>
> It looks like the values() function uses the iterator to generate the
values.  OK, so maybe I'll try this:
>
> class od2(collections.OrderedDict):
>     def __iter__(self):
>         values = [self[key] for key in self.keys()]
>         return iter(values)
>
> that gives me messages like this:
>
> File "...", line 56, in __iter__
>   values = [self[key] for key in self.keys()]
> File ".../python/2.7.1/linux64/lib/python2.7/_abcoll.py" line 366 in keys
>   return list(self)
>
> and list() uses the iterator.  Gah.  So I'm out of ideas here, even if I
dig into OrderedDict and find some way to pull the keys out, I'll probably
have to rewrite a bunch of access functions for it to still work like a
dictionary.  Am I just tilting at windmills, or is doing something like
this actually possible?  Or am I just being dense, and there's a super
obvious way to do this? Thanks for any help!
>
> (Oh, and forgive me if there's a known fix for this, googling for
anything along these lines just sent me to tons of tutorials about using
the .values() function.  No help there.)
>
>       ------- Corey
> _______________________________________________
> Baypiggies mailing list
> Baypiggies at python.org
> To change your subscription options or unsubscribe:
> http://mail.python.org/mailman/listinfo/baypiggies

_______________________________________________
Baypiggies mailing list
Baypiggies at python.org
To change your subscription options or unsubscribe:
http://mail.python.org/mailman/listinfo/baypiggies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/baypiggies/attachments/20120106/08d09c1e/attachment.html>


More information about the Baypiggies mailing list