[Numpy-discussion] IDL vs Python parallel computing
siegfried.gonzi at ed.ac.uk
Thu May 8 02:27:58 EDT 2014
On 08/05/2014 04:00, numpy-discussion-request at scipy.org wrote:
> Send NumPy-Discussion mailing list submissions to
> numpy-discussion at scipy.org
> To subscribe or unsubscribe via the World Wide Web, visit
> or, via email, send a message with subject or body 'help' to
> numpy-discussion-request at scipy.org
> You can reach the person managing the list at
> numpy-discussion-owner at scipy.org
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of NumPy-Discussion digest..."
> Message: 1
> Date: Wed, 07 May 2014 20:11:13 +0200
> From: Sturla Molden <sturla.molden at gmail.com>
> Subject: Re: [Numpy-discussion] IDL vs Python parallel computing
> To: numpy-discussion at scipy.org
> Message-ID: <lkdt01$jrc$1 at ger.gmane.org>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> On 03/05/14 23:56, Siegfried Gonzi wrote:
> > I noticed IDL uses at least 400% (4 processors or cores) out of the box
> > for simple things like reading and processing files, calculating the
> > mean etc.
> The DMA controller is working at its own pace, regardless of what the
> CPU is doing. You cannot get data faster off the disk by burning the
> CPU. If you are seeing 100 % CPU usage while doing file i/o there is
> something very bad going on. If you did this to an i/o intensive server
> it would go up in a ball of smoke... The purpose of high-performance
> asynchronous i/o systems such as epoll, kqueue, IOCP is actually to keep
> the CPU usage to a minimum.
It is probbaly not so much about reading in files. But I just noticed
(top command) it for simple things like processing say 4 dimensional
fields (longitute, latitude, altitutde, time) and calculating column
means or moment statistics over grid boxes and writing the fields out
again and things like that.
But it never uses more than 400%.
I haven't done any thorough testing of where and why the 400% really
kicks in and if IDL is cheating here or not.
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
More information about the NumPy-Discussion