removing duplication from a huge list.

[Tim Rowe]

2009/2/27 odeits <odeits at gmail.com>:

> How big of a list are we talking about? If the list is so big that the
> entire list cannot fit in memory at the same time this approach wont
> work e.g. removing duplicate lines from a very large file.

We were told in the original question: more than 15 million records,
and it won't all fit into memory. So your observation is pertinent.

-- 
Tim Rowe