[Numpy-discussion] IDL vs Python parallel computing

Siegfried Gonzi siegfried.gonzi at ed.ac.uk
Thu May 8 02:27:58 EDT 2014

On 08/05/2014 04:00, numpy-discussion-request at scipy.org wrote:
> Send NumPy-Discussion mailing list submissions to
> 	numpy-discussion at scipy.org
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://mail.scipy.org/mailman/listinfo/numpy-discussion
> or, via email, send a message with subject or body 'help' to
> 	numpy-discussion-request at scipy.org
> You can reach the person managing the list at
> 	numpy-discussion-owner at scipy.org
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of NumPy-Discussion digest..."
> ----------------------------------------------------------------------
> Message: 1
> Date: Wed, 07 May 2014 20:11:13 +0200
> From: Sturla Molden <sturla.molden at gmail.com>
> Subject: Re: [Numpy-discussion] IDL vs Python parallel computing
> To: numpy-discussion at scipy.org
> Message-ID: <lkdt01$jrc$1 at ger.gmane.org>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> On 03/05/14 23:56, Siegfried Gonzi wrote:
>   > I noticed IDL uses at least 400% (4 processors or cores) out of the box
>   > for simple things like reading and processing files, calculating the
>   > mean etc.
> The DMA controller is working at its own pace, regardless of what the
> CPU is doing. You cannot get data faster off the disk by burning the
> CPU. If you are seeing 100 % CPU usage while doing file i/o there is
> something very bad going on. If you did this to an i/o intensive server
> it would go up in a ball of smoke... The purpose of high-performance
> asynchronous i/o systems such as epoll, kqueue, IOCP is actually to keep
> the CPU usage to a minimum.

It is probbaly not so much about reading in files. But I just noticed 
(top command) it for simple things like processing say 4 dimensional 
fields (longitute, latitude, altitutde, time) and calculating column 
means or moment statistics over grid boxes and writing the fields  out 
again and things like that.

But it never uses more than 400%.

  I haven't done any thorough testing of where and why the 400% really 
kicks in and if IDL is cheating here or not.

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

More information about the NumPy-Discussion mailing list