remove duplicates from list *preserving order*
Steven Bethard
steven.bethard at gmail.com
Thu Feb 3 15:39:49 EST 2005
I'm sorry, I assume this has been discussed somewhere already, but I
found only a few hits in Google Groups... If you know where there's a
good summary, please feel free to direct me there.
I have a list[1] of objects from which I need to remove duplicates. I
have to maintain the list order though, so solutions like set(lst), etc.
will not work for me. What are my options? So far, I can see:
def filterdups(iterable):
result = []
for item in iterable:
if item not in result:
result.append(item)
return result
def filterdups(iterable):
result = []
seen = set()
for item in iterable:
if item not in seen:
result.append(item)
seen.add(item)
return result
def filterdups(iterable):
seen = set()
for item in iterable:
if item not in seen:
seen.add(item)
yield item
Does anyone have a better[2] solution?
STeve
[1] Well, actually it's an iterable of objects, but I can convert it to
a list if that's helpful.
[2] Yes I know, "better" is ambiguous. If it helps any, for my
particular situation, speed is probably more important than memory, so
I'm leaning towards the second or third implementation.
More information about the Python-list
mailing list