[Tutor] reducing lists within list to their set of unique values

[Oscar Benjamin]

On 21 May 2013 14:31, Treder, Robert <Robert.Treder at morganstanley.com> wrote:
> Steven wrote:
>>
>> py> L = ['b', 'd', 'c', 'a', 'b']
>> py> list(set(L))
>> ['a', 'c', 'b', 'd']
>>
>>
>> If keeping the order is important, you cannot use set, and you'll need another way to extract only the unique values. Ask if you need help on
>> that.
>
> Thanks, Steven. Very helpful. It looks like the order is only changed on the inner list that
> set() is applied to, not on the outer list since the outer list order is controlled by index.
> For this application I don't care about the order of the inner lists. However there are other
> applications where that will be import. Can you please describe the alternate method for
> extracting the unique values that maintains order.

There isn't necessarily a uniquely defined ordering. Here's a function
that preserves the order of the first occurrences of each element in
each list:

def uniquify(original):
    new = []
    seen = set()
    for item in original:
        if item not in seen:
            new.append(item)
            seen.add(item)
    return new

>>> uniquify([1, 2, 3, 1, 2, 5])
[1, 2, 3, 5]


Oscar