[Numpy-discussion] List/location of consecutive integers (2)

[Christopher Barker]

Andrea Gavana wrote:
>     I have tried the solutions proposed in the previous thread and it
> looks like Chris' one is the fastest for my purposes.

whoo hoo! What do I win? ;-)

> Splitting the reading process between 4 processes will require the
> exchange of 5-20 MB from the child processes to the main one: do you
> think my script will benefit from using multiprocessing?

If you are talking about multiprocessing to read the data in -- I don't 
think so -- that's probably IO bound anyway. You can't make your disks 
faster with multiple processors.

> Should I try another approach?

I don't know it will do anything for performance, but you might want to 
look at memory mapped arrays -- it's a very cool way to work with data 
files too big to want to bring into memory all at once.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov