[Python-Dev] Iterating over dictionaries and objects in general

[M.-A. Lemburg]

[Followup to a discussion on psa-members about iterating over
 dictionaries without creating intermediate lists]

Jim Fulton wrote:
> 
> "M.-A. Lemburg" wrote:
> >
> > Jim Fulton wrote:
> > >
> > > > The problem with the PyDict_Next() approach is that it will only
> > > > work reliably from within a single C call. You can't return
> > > > to Python between calls to PyDict_Next(), because those could
> > > > modify the dictionary causing the next PyDict_Next() call to
> > > > fail or core dump.
> > >
> > > I do this all the time without problem.  Basically, you provide an
> > > index and  if the index is out of range, you simply get an end-of-data return.
> > > The only downside of this approach is that you might get "incorrect"
> > > results if the dictionary is modified between calls.  This isn't
> > > all that different from iterating over a list with an index.
> >
> > Hmm, that's true... but what if the dictionary gets resized
> > in between iterations ? The item layout is then likely to
> > change, so you could potentially get complet bogus.
> 
> I think I said that. :)

Just wanted to verify my understanding ;-)

> > Even iterating over items twice may then occur, I guess.
> 
> Yup.
> 
> Again, this is not so different from iterating over
> a list using a range:
> 
>   l=range(10)
>   for i in range.len(l):
>     l.insert(0,'Bruce')
>     print l[i]
> 
> This always outputs 'Bruce'. :)

Ok, so the "risk" is under user control. Fine with me...
 
> > Or perhaps via a special dictionary iterator, so that the following
> > works:
> >
> > for item in dictrange(d):
> >    ...
> 
> Yup.
> 
> > The iterator could then also take some extra actions to insure
> > that the dictionary hasn't been resized.
> 
> I don't think it should do that. It should simply
> stop when it has run out of items.

I think I'll give such an iterator a spin. Would be a nice
extension to mxTools.

BTW, a generic type slot for iterating over types would probably
be a nice feature too. The type slot could provide hooks of the
form it_first, it_last, it_next, it_prev which all work integer
index based, e.g. in pseudo code:

int i;
PyObject *item;

/* set up i and item to point to the first item */
if (obj.it_first(&i,&item) < 0)
   goto onError;
while (1) {
   PyObject_Print(item);
   /* move i and item to the next item; an IndexError is raised
      in case there are no more items */
   if (obj.it_next(&i,&item) < 0) {
	PyErr_Clear();
	break;
   }
}

These slots would cover all problem instances where iteration
over non-sequences or non-uniform sequences (i.e. sequences like
objects which don't provide konvex index sets, e.g. 1,2,3,6,7,8,11,12)
is required, e.g. dictionaries, multi-segment buffers

-- 
Marc-Andre Lemburg
______________________________________________________________________
Y2000:                                                   127 days left
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/