Re: [Numpy-discussion] Using logical function on more than 2 arrays, availability of a "between" function ?
Auto-answer, sorry, Well, actually I made a mistake lower... that you may have noticed... On the faster (your) solution, even with a cleaner use of the out parameter, the fact that the all has then to be used with parameter axis=0 takes more time and makes it actually slower than the initial solution... So I may go for the "multiplier" solution. Regards, Matthieu On Tuesday 20 March 2012 13:13:15 you wrote:
Hi Richard,
Thanks for your answer and the related help !
In fact, I was hoping to have a less memory and more speed solution. Something equivalent to a "raster calculator" for numpy. Wouldn't it make sense to have some optimized function to work on more than 2 arrays for numpy anyway ?
At the end, I am rather interested by more speed.
I tried first a code-sparing version : array = numpy.asarray([(aBlueChannel < 1.0),(aNirChannel > aBlueChannel * 1.0),(aNirChannel < aBlueChannel * 1.8)]).all()
But this one is at the end more than 2 times slower than : array1 = numpy.empty([3,6566,6682], dtype=numpy.bool) numpy.less(aBlueChannel, 1.0, out=array1[0]) numpy.greater(aNirChannel, (aBlueChannel * 1.0), out=array1[1]) numpy.less(aNirChannel, (aBlueChannel * 1.8), out=array1[2]) array = array1.all()
(and this solution is about 30% faster than the original one)
I could find another way which was fine for me too: array = (aBlueChannel < 1.0) * (aNirChannel > (aBlueChannel * 1.0)) * (aNirChannel < (aBlueChannel * 1.8))
But this one is only 5-10% faster than the original solution, even if probably using less memory than the 2 previous ones. (same was possible with operator +, but slower than operator *)
Regards, Matthieu Rigal
On Monday 19 March 2012 18:00:02 numpy-discussion-request@scipy.org wrote:
Message: 2 Date: Mon, 19 Mar 2012 13:20:23 +0000 From: Richard Hattersley <rhattersley@gmail.com> Subject: Re: [Numpy-discussion] Using logical function on more than 2 arrays, availability of a "between" function ? To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID:
<CAP=RS9=UBOc6Kmtmnne7W093t19w=T=oSrXUAW0WF8B49hqcXQ@mail.gmail.com
Content-Type: text/plain; charset=ISO-8859-1
What do you mean by "efficient"? Are you trying to get it execute faster? Or using less memory? Or have more concise source code?
Less memory: - numpy.vectorize would let you get to the end result without any intermediate arrays but will be slow. - Using the "out" parameter of numpy.logical_and will let you avoid one of the intermediate arrays.
More speed?: Perhaps putting all three boolean temporary results into a single boolean array (using the "out" parameter of numpy.greater, etc) and using numpy.all might benefit from logical short-circuiting.
And watch out for divide-by-zero from "aNirChannel/aBlueChannel".
Regards, Richard Hattersley
RapidEye AG Molkenmarkt 30 14776 Brandenburg an der Havel Germany Follow us on Twitter! www.twitter.com/rapideye_ag Head Office/Sitz der Gesellschaft: Brandenburg an der Havel Management Board/Vorstand: Ryan Johnson Chairman of Supervisory Board/Vorsitzender des Aufsichtsrates: Robert Johnson Commercial Register/Handelsregister Potsdam HRB 24742 P Tax Number/Steuernummer: 048/100/00053 VAT-Ident-Number/Ust.-ID: DE 199331235 DIN EN ISO 9001 certified
I didn't try it, but I think that Theano and numexpr should be able to make them faster. [1] http://deeplearning.net/software/theano/ [2] https://code.google.com/p/numexpr/ Fred On Tue, Mar 20, 2012 at 9:05 AM, Matthieu Rigal <rigal@rapideye.net> wrote:
Auto-answer, sorry,
Well, actually I made a mistake lower... that you may have noticed... On the faster (your) solution, even with a cleaner use of the out parameter, the fact that the all has then to be used with parameter axis=0 takes more time and makes it actually slower than the initial solution...
So I may go for the "multiplier" solution.
Regards, Matthieu
On Tuesday 20 March 2012 13:13:15 you wrote:
Hi Richard,
Thanks for your answer and the related help !
In fact, I was hoping to have a less memory and more speed solution. Something equivalent to a "raster calculator" for numpy. Wouldn't it make sense to have some optimized function to work on more than 2 arrays for numpy anyway ?
At the end, I am rather interested by more speed.
I tried first a code-sparing version : array = numpy.asarray([(aBlueChannel < 1.0),(aNirChannel > aBlueChannel * 1.0),(aNirChannel < aBlueChannel * 1.8)]).all()
But this one is at the end more than 2 times slower than : array1 = numpy.empty([3,6566,6682], dtype=numpy.bool) numpy.less(aBlueChannel, 1.0, out=array1[0]) numpy.greater(aNirChannel, (aBlueChannel * 1.0), out=array1[1]) numpy.less(aNirChannel, (aBlueChannel * 1.8), out=array1[2]) array = array1.all()
(and this solution is about 30% faster than the original one)
I could find another way which was fine for me too: array = (aBlueChannel < 1.0) * (aNirChannel > (aBlueChannel * 1.0)) * (aNirChannel < (aBlueChannel * 1.8))
But this one is only 5-10% faster than the original solution, even if probably using less memory than the 2 previous ones. (same was possible with operator +, but slower than operator *)
Regards, Matthieu Rigal
On Monday 19 March 2012 18:00:02 numpy-discussion-request@scipy.org wrote:
Message: 2 Date: Mon, 19 Mar 2012 13:20:23 +0000 From: Richard Hattersley <rhattersley@gmail.com> Subject: Re: [Numpy-discussion] Using logical function on more than 2 arrays, availability of a "between" function ? To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID:
<CAP=RS9=UBOc6Kmtmnne7W093t19w=T=oSrXUAW0WF8B49hqcXQ@mail.gmail.com
Content-Type: text/plain; charset=ISO-8859-1
What do you mean by "efficient"? Are you trying to get it execute faster? Or using less memory? Or have more concise source code?
Less memory: - numpy.vectorize would let you get to the end result without any intermediate arrays but will be slow. - Using the "out" parameter of numpy.logical_and will let you avoid one of the intermediate arrays.
More speed?: Perhaps putting all three boolean temporary results into a single boolean array (using the "out" parameter of numpy.greater, etc) and using numpy.all might benefit from logical short-circuiting.
And watch out for divide-by-zero from "aNirChannel/aBlueChannel".
Regards, Richard Hattersley
RapidEye AG Molkenmarkt 30 14776 Brandenburg an der Havel Germany
Follow us on Twitter! www.twitter.com/rapideye_ag
Head Office/Sitz der Gesellschaft: Brandenburg an der Havel Management Board/Vorstand: Ryan Johnson Chairman of Supervisory Board/Vorsitzender des Aufsichtsrates: Robert Johnson Commercial Register/Handelsregister Potsdam HRB 24742 P Tax Number/Steuernummer: 048/100/00053 VAT-Ident-Number/Ust.-ID: DE 199331235 DIN EN ISO 9001 certified
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (2)
-
Frédéric Bastien -
Matthieu Rigal