Optimizing if statement check over a numpy value
Heli Nix
hemla21 at gmail.com
Wed Jul 29 10:23:07 EDT 2015
On Thursday, July 23, 2015 at 1:43:00 PM UTC+2, Jeremy Sanders wrote:
> Heli Nix wrote:
>
> > Is there any way that I can optimize this if statement.
>
> Array processing is much faster in numpy. Maybe this is close to what you
> want
>
> import numpy as N
> # input data
> vals = N.array([42, 1, 5, 3.14, 53, 1, 12, 11, 1])
> # list of items to exclude
> exclude = [1]
> # convert to a boolean array
> exclbool = N.zeros(vals.shape, dtype=bool)
> exclbool[exclude] = True
> # do replacement
> ones = vals==1.0
> # Note: ~ is numpy.logical_not
> vals[ones & (~exclbool)] = 1e-20
>
> I think you'll have to convert your HDF array into a numpy array first,
> using numpy.array().
>
> Jeremy
Dear all,
I tried the sorted python list, but this did not really help the runtime.
I havenĀ“t had time to check the sorted collections. I solved my runtime problem by using the script from Jeremy up here.
It was a life saviour and it is amazing how powerful numpy is. Thanks a lot Jeremy for this. By the way, I did not have to do any array conversion. The array read from hdf5 file using h5py is already a numpy array.
The runtime over an array of around 16M reduced from around 12 hours (previous script) to 3 seconds using numpy on the same machine.
Thanks alot for your help,
More information about the Python-list
mailing list