Accumulate values that are below threshold
Hello, Sometimes the hardest part of a problem is articulating it. Hopefully I can describe what I am trying to do - at least enough to get some help. I am trying to compare values to a threshold and when the values are lower than the threshold they are added to the value in my set until the threshold is reached. Everytime the threshold is reached I want the index and value (accumulated). Hopefully the example below will help threshold =1.0 for indx,val in enumerate(Q): print indx,val 0 100.0 1 20.0 2 16.0 3 7.0 4 3.0 5 1.5 6 0.8 7 0.6 8 0.5 9 0.2 10 0.2 11 0.1 12 0.1 The output I would like is (number of elements and value) 0 100.0 1 20.0 2 16.0 3 7.0 4 3.0 5 1.5 7 1.4 11 1.0 The 1st 6 elements are easy as they are all greater than or equal to the threshold(1.0). Once the values drop below the threshold the next value is added until the threshold is reached. Any help is appreciated, Bevan Jenkins
Hi Bevan Since the number of output elements are unknown, I don't think you can implement this efficiently using arrays. If your dataset isn't too large, a for-loop should do the trick. Otherwise, you may have to run your code through Cython, which optimises for-loops around Python lists. thresh = 1.0 carry = 0 output = [] for idx, val in data: carry += val if (carry - thresh) >= -1e-15: output.append((idx, carry)) carry = 0 The comparison line above, "(carry - thresh0 >= -1e-15", may look strange -- it basically just does "carry >= thresh". For some reason I don't quite understand, when accumulating floats, it sometimes happens that "1.0 != 1.0", so I use 1e-15 as protection. Regards Stéfan 2009/1/8 Bevan Jenkins <bevan07@gmail.com>:
Hello,
Sometimes the hardest part of a problem is articulating it. Hopefully I can describe what I am trying to do - at least enough to get some help.
I am trying to compare values to a threshold and when the values are lower than the threshold they are added to the value in my set until the threshold is reached. Everytime the threshold is reached I want the index and value (accumulated).
Hopefully the example below will help
threshold =1.0 for indx,val in enumerate(Q): print indx,val
0 100.0 1 20.0 2 16.0 3 7.0 4 3.0 5 1.5 6 0.8 7 0.6 8 0.5 9 0.2 10 0.2 11 0.1 12 0.1
The output I would like is (number of elements and value) 0 100.0 1 20.0 2 16.0 3 7.0 4 3.0 5 1.5 7 1.4 11 1.0
The 1st 6 elements are easy as they are all greater than or equal to the threshold(1.0). Once the values drop below the threshold the next value is added until the threshold is reached.
Any help is appreciated, Bevan Jenkins
Stéfan van der Walt <stefan <at> sun.ac.za> writes:
Hi Bevan
Since the number of output elements are unknown, I don't think you can implement this efficiently using arrays. If your dataset isn't too large, a for-loop should do the trick. Otherwise, you may have to run your code through Cython, which optimises for-loops around Python lists.
thresh = 1.0 carry = 0 output = [] for idx, val in data: carry += val if (carry - thresh) >= -1e-15: output.append((idx, carry)) carry = 0
The comparison line above, "(carry - thresh0 >= -1e-15", may look strange -- it basically just does "carry >= thresh". For some reason I don't quite understand, when accumulating floats, it sometimes happens that "1.0 != 1.0", so I use 1e-15 as protection.
Regards Stéfan
2009/1/8 Bevan Jenkins <bevan07 <at> gmail.com>:
Stefan, Thanks for your solution, it does exactly what i require. At the moment I am just running through proof of concept stuff so the dataset is small. If I start to run into issues with the real data, then that might be the push I need to look into Cython. Bevan
participants (2)
-
Bevan Jenkins -
Stéfan van der Walt