[Python-Dev] LinkedHashSet/LinkedHashMap equivalents

Delaney, Timothy C (Timothy) tdelaney at avaya.com
Thu Mar 10 06:00:20 CET 2005

Steven Bethard wrote:

> def filterdups(iterable):
>      seen = set()
>      for item in iterable:
>          if item not in seen:
>              seen.add(item)
>              yield item
> Adding this to, say, itertools would cover all my use cases.  And as
> long as you don't have too many duplicates, filterdups as above should
> keep memory consumption down better.

Thinking about this further - memory usage would be almost identical. By
the time you completed the iterable, you would have built up exactly the
same set internally - although probably not as memory efficient since it
would be being built piecemeal. OTOH, an ordered set has a bit of extra
memory for maintaining the order, so it's going to be pretty close.

The only thing this gains you (and it's significant) is the ability to
work on any iterable lazily.

Tim Delaney

More information about the Python-Dev mailing list