loop through values in a array and find maximum as looping
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like that value to replace it. e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0] where b>a replace with value in b, so the new a should be : a=[1,1,3,2] 2,1,2,2 1,1,2,2] and then keep looping through many arrays and replace whenever value is larger. I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask] -=- Olivier 2011/12/6 questions anon <questions.anon@gmail.com>
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like that value to replace it.
e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0]
where b>a replace with value in b, so the new a should be :
a=[1,1,3,2] 2,1,2,2 1,1,2,2]
and then keep looping through many arrays and replace whenever value is larger.
I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> wrote:
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask]
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like that value to replace it.
e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0]
where b>a replace with value in b, so the new a should be :
a=[1,1,3,2] 2,1,2,2 1,1,2,2]
and then keep looping through many arrays and replace whenever value is larger.
I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
if I understand correctly it's a minimum.reduce numpy
a = np.concatenate((np.arange(5)[::-1], np.arange(5)))*np.ones((4,3,1)) np.minimum.reduce(a, axis=2) array([[ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.]]) a.T.shape (10, 3, 4)
python with iterable
reduce(np.maximum, a.T) array([[ 4., 4., 4., 4.], [ 4., 4., 4., 4.], [ 4., 4., 4., 4.]]) reduce(np.minimum, a.T) array([[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]])
Josef
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi Olivier, No that does not seem to do anything am I missing another step whereever b is greater than a replace b with a? thanks On Wed, Dec 7, 2011 at 11:55 AM, Olivier Delalleau <shish@keba.be> wrote:
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask]
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like that value to replace it.
e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0]
where b>a replace with value in b, so the new a should be :
a=[1,1,3,2] 2,1,2,2 1,1,2,2]
and then keep looping through many arrays and replace whenever value is larger.
I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
thanks for responding Josef but that is not really what I am looking for, I have a multidimensional array and if the next array has any values greater than what is in my first array I want to replace them. The data are contained in netcdf files. I can achieve what I want if I combine all of my arrays using numpy concatenate and then using the command numpy.max(myarray, axis=0) but because I have so many arrays I end up with a memory error so I need to find a way to get the maximum while looping. On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote:
On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> wrote:
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask]
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like that value to replace it.
e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0]
where b>a replace with value in b, so the new a should be :
a=[1,1,3,2] 2,1,2,2 1,1,2,2]
and then keep looping through many arrays and replace whenever value is larger.
I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
if I understand correctly it's a minimum.reduce
numpy
a = np.concatenate((np.arange(5)[::-1], np.arange(5)))*np.ones((4,3,1)) np.minimum.reduce(a, axis=2) array([[ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.]]) a.T.shape (10, 3, 4)
python with iterable
reduce(np.maximum, a.T) array([[ 4., 4., 4., 4.], [ 4., 4., 4., 4.], [ 4., 4., 4., 4.]]) reduce(np.minimum, a.T) array([[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]])
Josef
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Weird, it worked for me (with a and b two 1d numpy arrays). Anyway, Josef's solution is probably much more efficient (especially if you can put all your arrays into a single tensor). -=- Olivier 2011/12/6 questions anon <questions.anon@gmail.com>
Hi Olivier, No that does not seem to do anything am I missing another step whereever b is greater than a replace b with a? thanks
On Wed, Dec 7, 2011 at 11:55 AM, Olivier Delalleau <shish@keba.be> wrote:
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask]
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like that value to replace it.
e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0]
where b>a replace with value in b, so the new a should be :
a=[1,1,3,2] 2,1,2,2 1,1,2,2]
and then keep looping through many arrays and replace whenever value is larger.
I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I have 2d numpy arrays On Wed, Dec 7, 2011 at 1:05 PM, Olivier Delalleau <shish@keba.be> wrote:
Weird, it worked for me (with a and b two 1d numpy arrays). Anyway, Josef's solution is probably much more efficient (especially if you can put all your arrays into a single tensor).
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
Hi Olivier, No that does not seem to do anything am I missing another step whereever b is greater than a replace b with a? thanks
On Wed, Dec 7, 2011 at 11:55 AM, Olivier Delalleau <shish@keba.be> wrote:
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask]
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like that value to replace it.
e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0]
where b>a replace with value in b, so the new a should be :
a=[1,1,3,2] 2,1,2,2 1,1,2,2]
and then keep looping through many arrays and replace whenever value is larger.
I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
If you need to do them one after the other, numpy.maximum(a, b) will do it (it won't work in-place on 'a' though, it'll make a new copy). -=- Olivier 2011/12/6 questions anon <questions.anon@gmail.com>
thanks for responding Josef but that is not really what I am looking for, I have a multidimensional array and if the next array has any values greater than what is in my first array I want to replace them. The data are contained in netcdf files. I can achieve what I want if I combine all of my arrays using numpy concatenate and then using the command numpy.max(myarray, axis=0) but because I have so many arrays I end up with a memory error so I need to find a way to get the maximum while looping.
On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote:
On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> wrote:
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask]
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like that value to replace it.
e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0]
where b>a replace with value in b, so the new a should be :
a=[1,1,3,2] 2,1,2,2 1,1,2,2]
and then keep looping through many arrays and replace whenever value is larger.
I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
if I understand correctly it's a minimum.reduce
numpy
a = np.concatenate((np.arange(5)[::-1], np.arange(5)))*np.ones((4,3,1)) np.minimum.reduce(a, axis=2) array([[ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.]]) a.T.shape (10, 3, 4)
python with iterable
reduce(np.maximum, a.T) array([[ 4., 4., 4., 4.], [ 4., 4., 4., 4.], [ 4., 4., 4., 4.]]) reduce(np.minimum, a.T) array([[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]])
Josef
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I think you want np.maximum(a, b, out=a) - Nathaniel On Dec 6, 2011 9:04 PM, "questions anon" <questions.anon@gmail.com> wrote:
thanks for responding Josef but that is not really what I am looking for, I have a multidimensional array and if the next array has any values greater than what is in my first array I want to replace them. The data are contained in netcdf files. I can achieve what I want if I combine all of my arrays using numpy concatenate and then using the command numpy.max(myarray, axis=0) but because I have so many arrays I end up with a memory error so I need to find a way to get the maximum while looping.
On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote:
On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> wrote:
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask]
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like that value to replace it.
e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0]
where b>a replace with value in b, so the new a should be :
a=[1,1,3,2] 2,1,2,2 1,1,2,2]
and then keep looping through many arrays and replace whenever value is larger.
I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
if I understand correctly it's a minimum.reduce
numpy
a = np.concatenate((np.arange(5)[::-1], np.arange(5)))*np.ones((4,3,1)) np.minimum.reduce(a, axis=2) array([[ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.]]) a.T.shape (10, 3, 4)
python with iterable
reduce(np.maximum, a.T) array([[ 4., 4., 4., 4.], [ 4., 4., 4., 4.], [ 4., 4., 4., 4.]]) reduce(np.minimum, a.T) array([[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]])
Josef
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Thanks, I didn't know you could specify the out array :) (to the OP: my initial suggestion, although probably not very efficient, seems to work with 2D arrays too, so I have no idea why it didn't work for you -- but Nathaniel's one seems to be the ideal one anyway). -=- Olivier 2011/12/6 Nathaniel Smith <njs@pobox.com>
I think you want np.maximum(a, b, out=a)
- Nathaniel On Dec 6, 2011 9:04 PM, "questions anon" <questions.anon@gmail.com> wrote:
thanks for responding Josef but that is not really what I am looking for, I have a multidimensional array and if the next array has any values greater than what is in my first array I want to replace them. The data are contained in netcdf files. I can achieve what I want if I combine all of my arrays using numpy concatenate and then using the command numpy.max(myarray, axis=0) but because I have so many arrays I end up with a memory error so I need to find a way to get the maximum while looping.
On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote:
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask]
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like
On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> wrote: that
value to replace it.
e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0]
where b>a replace with value in b, so the new a should be :
a=[1,1,3,2] 2,1,2,2 1,1,2,2]
and then keep looping through many arrays and replace whenever value is larger.
I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
if I understand correctly it's a minimum.reduce
numpy
a = np.concatenate((np.arange(5)[::-1], np.arange(5)))*np.ones((4,3,1)) np.minimum.reduce(a, axis=2) array([[ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.]]) a.T.shape (10, 3, 4)
python with iterable
reduce(np.maximum, a.T) array([[ 4., 4., 4., 4.], [ 4., 4., 4., 4.], [ 4., 4., 4., 4.]]) reduce(np.minimum, a.T) array([[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]])
Josef
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
thanks for all of your help, that does look appropriate but I am not sure how to loop it over thousands of files. I need to keep the first array to compare with but replace any greater values as I loop through each array comparing back to the same array. does that make sense? On Wed, Dec 7, 2011 at 1:12 PM, Olivier Delalleau <shish@keba.be> wrote:
Thanks, I didn't know you could specify the out array :)
(to the OP: my initial suggestion, although probably not very efficient, seems to work with 2D arrays too, so I have no idea why it didn't work for you -- but Nathaniel's one seems to be the ideal one anyway).
-=- Olivier
2011/12/6 Nathaniel Smith <njs@pobox.com>
I think you want np.maximum(a, b, out=a)
- Nathaniel On Dec 6, 2011 9:04 PM, "questions anon" <questions.anon@gmail.com> wrote:
thanks for responding Josef but that is not really what I am looking for, I have a multidimensional array and if the next array has any values greater than what is in my first array I want to replace them. The data are contained in netcdf files. I can achieve what I want if I combine all of my arrays using numpy concatenate and then using the command numpy.max(myarray, axis=0) but because I have so many arrays I end up with a memory error so I need to find a way to get the maximum while looping.
On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote:
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask]
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
I would like to produce an array with the maximum values out of many (10000s) of arrays. I need to loop through many multidimentional arrays and if a value is larger (in the same place as the previous array) then I would like
On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> wrote: that
value to replace it.
e.g. a=[1,1,2,2 11,2,2 1,1,2,2] b=[1,1,3,2 2,1,0,0 1,1,2,0]
where b>a replace with value in b, so the new a should be :
a=[1,1,3,2] 2,1,2,2 1,1,2,2]
and then keep looping through many arrays and replace whenever value is larger.
I have tried numpy.putmask but that results in TypeError: putmask() argument 1 must be numpy.ndarray, not list Any other ideas? Thanks
if I understand correctly it's a minimum.reduce
numpy
> a = np.concatenate((np.arange(5)[::-1], np.arange(5)))*np.ones((4,3,1)) > np.minimum.reduce(a, axis=2) array([[ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.]]) > a.T.shape (10, 3, 4)
python with iterable
> reduce(np.maximum, a.T) array([[ 4., 4., 4., 4.], [ 4., 4., 4., 4.], [ 4., 4., 4., 4.]]) > reduce(np.minimum, a.T) array([[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]])
Josef
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
The "out=a" keyword will ensure your first array will keep being updated. So you can do something like: a = my_list_of_arrays[0] for b in my_list_of_arrays[1:]: numpy.maximum(a, b, out=a) -=- Olivier 2011/12/6 questions anon <questions.anon@gmail.com>
thanks for all of your help, that does look appropriate but I am not sure how to loop it over thousands of files. I need to keep the first array to compare with but replace any greater values as I loop through each array comparing back to the same array. does that make sense?
On Wed, Dec 7, 2011 at 1:12 PM, Olivier Delalleau <shish@keba.be> wrote:
Thanks, I didn't know you could specify the out array :)
(to the OP: my initial suggestion, although probably not very efficient, seems to work with 2D arrays too, so I have no idea why it didn't work for you -- but Nathaniel's one seems to be the ideal one anyway).
-=- Olivier
2011/12/6 Nathaniel Smith <njs@pobox.com>
I think you want np.maximum(a, b, out=a)
- Nathaniel On Dec 6, 2011 9:04 PM, "questions anon" <questions.anon@gmail.com> wrote:
thanks for responding Josef but that is not really what I am looking for, I have a multidimensional array and if the next array has any values greater than what is in my first array I want to replace them. The data are contained in netcdf files. I can achieve what I want if I combine all of my arrays using numpy concatenate and then using the command numpy.max(myarray, axis=0) but because I have so many arrays I end up with a memory error so I need to find a way to get the maximum while looping.
On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote:
It may not be the most efficient way to do this, but you can do: mask = b > a a[mask] = b[mask]
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com> > > I would like to produce an array with the maximum values out of many > (10000s) of arrays. > I need to loop through many multidimentional arrays and if a value is > larger (in the same place as the previous array) then I would like
On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> wrote: that
> value to replace it. > > e.g. > a=[1,1,2,2 > 11,2,2 > 1,1,2,2] > b=[1,1,3,2 > 2,1,0,0 > 1,1,2,0] > > where b>a replace with value in b, so the new a should be : > > a=[1,1,3,2] > 2,1,2,2 > 1,1,2,2] > > and then keep looping through many arrays and replace whenever value is > larger. > > I have tried numpy.putmask but that results in > TypeError: putmask() argument 1 must be numpy.ndarray, not list > Any other ideas? Thanks
if I understand correctly it's a minimum.reduce
numpy
>> a = np.concatenate((np.arange(5)[::-1], np.arange(5)))*np.ones((4,3,1)) >> np.minimum.reduce(a, axis=2) array([[ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.]]) >> a.T.shape (10, 3, 4)
python with iterable
>> reduce(np.maximum, a.T) array([[ 4., 4., 4., 4.], [ 4., 4., 4., 4.], [ 4., 4., 4., 4.]]) >> reduce(np.minimum, a.T) array([[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]])
Josef
> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Dec 6, 2011 at 9:36 PM, Olivier Delalleau <shish@keba.be> wrote:
The "out=a" keyword will ensure your first array will keep being updated. So you can do something like:
a = my_list_of_arrays[0] for b in my_list_of_arrays[1:]: numpy.maximum(a, b, out=a)
I didn't think of the out argument which makes it more efficient, but in my example I used Python's reduce which takes an iterable and not one huge array. Josef
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
thanks for all of your help, that does look appropriate but I am not sure how to loop it over thousands of files. I need to keep the first array to compare with but replace any greater values as I loop through each array comparing back to the same array. does that make sense?
On Wed, Dec 7, 2011 at 1:12 PM, Olivier Delalleau <shish@keba.be> wrote:
Thanks, I didn't know you could specify the out array :)
(to the OP: my initial suggestion, although probably not very efficient, seems to work with 2D arrays too, so I have no idea why it didn't work for you -- but Nathaniel's one seems to be the ideal one anyway).
-=- Olivier
2011/12/6 Nathaniel Smith <njs@pobox.com>
I think you want np.maximum(a, b, out=a)
- Nathaniel
On Dec 6, 2011 9:04 PM, "questions anon" <questions.anon@gmail.com> wrote:
thanks for responding Josef but that is not really what I am looking for, I have a multidimensional array and if the next array has any values greater than what is in my first array I want to replace them. The data are contained in netcdf files. I can achieve what I want if I combine all of my arrays using numpy concatenate and then using the command numpy.max(myarray, axis=0) but because I have so many arrays I end up with a memory error so I need to find a way to get the maximum while looping.
On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote:
On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> wrote: > It may not be the most efficient way to do this, but you can do: > mask = b > a > a[mask] = b[mask] > > -=- Olivier > > 2011/12/6 questions anon <questions.anon@gmail.com> >> >> I would like to produce an array with the maximum values out of >> many >> (10000s) of arrays. >> I need to loop through many multidimentional arrays and if a value >> is >> larger (in the same place as the previous array) then I would like >> that >> value to replace it. >> >> e.g. >> a=[1,1,2,2 >> 11,2,2 >> 1,1,2,2] >> b=[1,1,3,2 >> 2,1,0,0 >> 1,1,2,0] >> >> where b>a replace with value in b, so the new a should be : >> >> a=[1,1,3,2] >> 2,1,2,2 >> 1,1,2,2] >> >> and then keep looping through many arrays and replace whenever >> value is >> larger. >> >> I have tried numpy.putmask but that results in >> TypeError: putmask() argument 1 must be numpy.ndarray, not list >> Any other ideas? Thanks
if I understand correctly it's a minimum.reduce
numpy
>>> a = np.concatenate((np.arange(5)[::-1], >>> np.arange(5)))*np.ones((4,3,1)) >>> np.minimum.reduce(a, axis=2) array([[ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.]]) >>> a.T.shape (10, 3, 4)
python with iterable
>>> reduce(np.maximum, a.T) array([[ 4., 4., 4., 4.], [ 4., 4., 4., 4.], [ 4., 4., 4., 4.]]) >>> reduce(np.minimum, a.T) array([[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]])
Josef
>> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
thanks again my only problem though is that the out=a in the loop does not seem to replace my a= outside the loop so my final a is whatever I started with for a. Not sure what I am doing wrong whether it is something with the loop or with the command. On Wed, Dec 7, 2011 at 1:44 PM, <josef.pktd@gmail.com> wrote:
On Tue, Dec 6, 2011 at 9:36 PM, Olivier Delalleau <shish@keba.be> wrote:
The "out=a" keyword will ensure your first array will keep being updated. So you can do something like:
a = my_list_of_arrays[0] for b in my_list_of_arrays[1:]: numpy.maximum(a, b, out=a)
I didn't think of the out argument which makes it more efficient, but in my example I used Python's reduce which takes an iterable and not one huge array.
Josef
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
thanks for all of your help, that does look appropriate but I am not
how to loop it over thousands of files. I need to keep the first array to compare with but replace any greater values as I loop through each array comparing back to the same array. does that make sense?
On Wed, Dec 7, 2011 at 1:12 PM, Olivier Delalleau <shish@keba.be> wrote:
Thanks, I didn't know you could specify the out array :)
(to the OP: my initial suggestion, although probably not very
efficient,
seems to work with 2D arrays too, so I have no idea why it didn't work for you -- but Nathaniel's one seems to be the ideal one anyway).
-=- Olivier
2011/12/6 Nathaniel Smith <njs@pobox.com>
I think you want np.maximum(a, b, out=a)
- Nathaniel
On Dec 6, 2011 9:04 PM, "questions anon" <questions.anon@gmail.com> wrote:
thanks for responding Josef but that is not really what I am looking for, I have a multidimensional array and if the next array has any
values
greater than what is in my first array I want to replace them. The data are contained in netcdf files. I can achieve what I want if I combine all of my arrays using numpy concatenate and then using the command numpy.max(myarray, axis=0) but because I have so many arrays I end up with a memory error so I need to find a way to get the maximum while looping.
On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote: > > On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> > wrote: > > It may not be the most efficient way to do this, but you can do: > > mask = b > a > > a[mask] = b[mask] > > > > -=- Olivier > > > > 2011/12/6 questions anon <questions.anon@gmail.com> > >> > >> I would like to produce an array with the maximum values out of > >> many > >> (10000s) of arrays. > >> I need to loop through many multidimentional arrays and if a value > >> is > >> larger (in the same place as the previous array) then I would
sure like
> >> that > >> value to replace it. > >> > >> e.g. > >> a=[1,1,2,2 > >> 11,2,2 > >> 1,1,2,2] > >> b=[1,1,3,2 > >> 2,1,0,0 > >> 1,1,2,0] > >> > >> where b>a replace with value in b, so the new a should be : > >> > >> a=[1,1,3,2] > >> 2,1,2,2 > >> 1,1,2,2] > >> > >> and then keep looping through many arrays and replace whenever > >> value is > >> larger. > >> > >> I have tried numpy.putmask but that results in > >> TypeError: putmask() argument 1 must be numpy.ndarray, not list > >> Any other ideas? Thanks > > if I understand correctly it's a minimum.reduce > > numpy > > >>> a = np.concatenate((np.arange(5)[::-1], > >>> np.arange(5)))*np.ones((4,3,1)) > >>> np.minimum.reduce(a, axis=2) > array([[ 0., 0., 0.], > [ 0., 0., 0.], > [ 0., 0., 0.], > [ 0., 0., 0.]]) > >>> a.T.shape > (10, 3, 4) > > python with iterable > > >>> reduce(np.maximum, a.T) > array([[ 4., 4., 4., 4.], > [ 4., 4., 4., 4.], > [ 4., 4., 4., 4.]]) > >>> reduce(np.minimum, a.T) > array([[ 0., 0., 0., 0.], > [ 0., 0., 0., 0.], > [ 0., 0., 0., 0.]]) > > Josef > > >> > >> _______________________________________________ > >> NumPy-Discussion mailing list > >> NumPy-Discussion@scipy.org > >> http://mail.scipy.org/mailman/listinfo/numpy-discussion > >> > > > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion@scipy.org > > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Is 'a' a regular numpy array or something fancier? -=- Olivier 2011/12/6 questions anon <questions.anon@gmail.com>
thanks again my only problem though is that the out=a in the loop does not seem to replace my a= outside the loop so my final a is whatever I started with for a. Not sure what I am doing wrong whether it is something with the loop or with the command.
On Wed, Dec 7, 2011 at 1:44 PM, <josef.pktd@gmail.com> wrote:
On Tue, Dec 6, 2011 at 9:36 PM, Olivier Delalleau <shish@keba.be> wrote:
The "out=a" keyword will ensure your first array will keep being updated. So you can do something like:
a = my_list_of_arrays[0] for b in my_list_of_arrays[1:]: numpy.maximum(a, b, out=a)
I didn't think of the out argument which makes it more efficient, but in my example I used Python's reduce which takes an iterable and not one huge array.
Josef
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
thanks for all of your help, that does look appropriate but I am not
how to loop it over thousands of files. I need to keep the first array to compare with but replace any greater values as I loop through each array comparing back to the same array. does that make sense?
On Wed, Dec 7, 2011 at 1:12 PM, Olivier Delalleau <shish@keba.be> wrote:
Thanks, I didn't know you could specify the out array :)
(to the OP: my initial suggestion, although probably not very
efficient,
seems to work with 2D arrays too, so I have no idea why it didn't work for you -- but Nathaniel's one seems to be the ideal one anyway).
-=- Olivier
2011/12/6 Nathaniel Smith <njs@pobox.com>
I think you want np.maximum(a, b, out=a)
- Nathaniel
On Dec 6, 2011 9:04 PM, "questions anon" <questions.anon@gmail.com> wrote: > > thanks for responding Josef but that is not really what I am looking > for, I have a multidimensional array and if the next array has any
values
> greater than what is in my first array I want to replace them. The data are > contained in netcdf files. > I can achieve what I want if I combine all of my arrays using numpy > concatenate and then using the command numpy.max(myarray, axis=0) but > because I have so many arrays I end up with a memory error so I need to find > a way to get the maximum while looping. > > > > On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote: >> >> On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> >> wrote: >> > It may not be the most efficient way to do this, but you can do: >> > mask = b > a >> > a[mask] = b[mask] >> > >> > -=- Olivier >> > >> > 2011/12/6 questions anon <questions.anon@gmail.com> >> >> >> >> I would like to produce an array with the maximum values out of >> >> many >> >> (10000s) of arrays. >> >> I need to loop through many multidimentional arrays and if a value >> >> is >> >> larger (in the same place as the previous array) then I would
sure like
>> >> that >> >> value to replace it. >> >> >> >> e.g. >> >> a=[1,1,2,2 >> >> 11,2,2 >> >> 1,1,2,2] >> >> b=[1,1,3,2 >> >> 2,1,0,0 >> >> 1,1,2,0] >> >> >> >> where b>a replace with value in b, so the new a should be : >> >> >> >> a=[1,1,3,2] >> >> 2,1,2,2 >> >> 1,1,2,2] >> >> >> >> and then keep looping through many arrays and replace whenever >> >> value is >> >> larger. >> >> >> >> I have tried numpy.putmask but that results in >> >> TypeError: putmask() argument 1 must be numpy.ndarray, not list >> >> Any other ideas? Thanks >> >> if I understand correctly it's a minimum.reduce >> >> numpy >> >> >>> a = np.concatenate((np.arange(5)[::-1], >> >>> np.arange(5)))*np.ones((4,3,1)) >> >>> np.minimum.reduce(a, axis=2) >> array([[ 0., 0., 0.], >> [ 0., 0., 0.], >> [ 0., 0., 0.], >> [ 0., 0., 0.]]) >> >>> a.T.shape >> (10, 3, 4) >> >> python with iterable >> >> >>> reduce(np.maximum, a.T) >> array([[ 4., 4., 4., 4.], >> [ 4., 4., 4., 4.], >> [ 4., 4., 4., 4.]]) >> >>> reduce(np.minimum, a.T) >> array([[ 0., 0., 0., 0.], >> [ 0., 0., 0., 0.], >> [ 0., 0., 0., 0.]]) >> >> Josef >> >> >> >> >> _______________________________________________ >> >> NumPy-Discussion mailing list >> >> NumPy-Discussion@scipy.org >> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> >> > >> > >> > _______________________________________________ >> > NumPy-Discussion mailing list >> > NumPy-Discussion@scipy.org >> > http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Something fancier I think, I am able to compare the result with my previous method so I can easily see I am doing something wrong. see code below: all_TSFC=[] for (path, dirs, files) in os.walk(MainFolder): for dir in dirs: print dir path=path+'/' for ncfile in files: if ncfile[-3:]=='.nc': print "dealing with ncfiles:", ncfile ncfile=os.path.join(path,ncfile) ncfile=Dataset(ncfile, 'r+', 'NETCDF4') TSFC=ncfile.variables['T_SFC'][:] fillvalue=ncfile.variables['T_SFC']._FillValue TSFC=MA.masked_values(TSFC, fillvalue) ncfile.close() all_TSFC.append(TSFC) a=TSFC[0] for b in TSFC[1:]: N.maximum(a,b,out=a) big_array=N.ma.concatenate(all_TSFC) Max=big_array.max(axis=0) print "max is", Max,"a is", a On Wed, Dec 7, 2011 at 2:34 PM, Olivier Delalleau <shish@keba.be> wrote:
Is 'a' a regular numpy array or something fancier?
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
thanks again my only problem though is that the out=a in the loop does not seem to replace my a= outside the loop so my final a is whatever I started with for a. Not sure what I am doing wrong whether it is something with the loop or with the command.
On Wed, Dec 7, 2011 at 1:44 PM, <josef.pktd@gmail.com> wrote:
On Tue, Dec 6, 2011 at 9:36 PM, Olivier Delalleau <shish@keba.be> wrote:
The "out=a" keyword will ensure your first array will keep being updated. So you can do something like:
a = my_list_of_arrays[0] for b in my_list_of_arrays[1:]: numpy.maximum(a, b, out=a)
I didn't think of the out argument which makes it more efficient, but in my example I used Python's reduce which takes an iterable and not one huge array.
Josef
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
thanks for all of your help, that does look appropriate but I am not
how to loop it over thousands of files. I need to keep the first array to compare with but replace any greater values as I loop through each array comparing back to the same array. does that make sense?
On Wed, Dec 7, 2011 at 1:12 PM, Olivier Delalleau <shish@keba.be> wrote:
Thanks, I didn't know you could specify the out array :)
(to the OP: my initial suggestion, although probably not very
efficient,
seems to work with 2D arrays too, so I have no idea why it didn't work for you -- but Nathaniel's one seems to be the ideal one anyway).
-=- Olivier
2011/12/6 Nathaniel Smith <njs@pobox.com> > > I think you want > np.maximum(a, b, out=a) > > - Nathaniel > > On Dec 6, 2011 9:04 PM, "questions anon" <questions.anon@gmail.com> > wrote: >> >> thanks for responding Josef but that is not really what I am looking >> for, I have a multidimensional array and if the next array has any values >> greater than what is in my first array I want to replace them. The data are >> contained in netcdf files. >> I can achieve what I want if I combine all of my arrays using numpy >> concatenate and then using the command numpy.max(myarray, axis=0) but >> because I have so many arrays I end up with a memory error so I need to find >> a way to get the maximum while looping. >> >> >> >> On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote: >>> >>> On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be> >>> wrote: >>> > It may not be the most efficient way to do this, but you can do: >>> > mask = b > a >>> > a[mask] = b[mask] >>> > >>> > -=- Olivier >>> > >>> > 2011/12/6 questions anon <questions.anon@gmail.com> >>> >> >>> >> I would like to produce an array with the maximum values out of >>> >> many >>> >> (10000s) of arrays. >>> >> I need to loop through many multidimentional arrays and if a value >>> >> is >>> >> larger (in the same place as the previous array) then I would
sure like
>>> >> that >>> >> value to replace it. >>> >> >>> >> e.g. >>> >> a=[1,1,2,2 >>> >> 11,2,2 >>> >> 1,1,2,2] >>> >> b=[1,1,3,2 >>> >> 2,1,0,0 >>> >> 1,1,2,0] >>> >> >>> >> where b>a replace with value in b, so the new a should be : >>> >> >>> >> a=[1,1,3,2] >>> >> 2,1,2,2 >>> >> 1,1,2,2] >>> >> >>> >> and then keep looping through many arrays and replace whenever >>> >> value is >>> >> larger. >>> >> >>> >> I have tried numpy.putmask but that results in >>> >> TypeError: putmask() argument 1 must be numpy.ndarray, not list >>> >> Any other ideas? Thanks >>> >>> if I understand correctly it's a minimum.reduce >>> >>> numpy >>> >>> >>> a = np.concatenate((np.arange(5)[::-1], >>> >>> np.arange(5)))*np.ones((4,3,1)) >>> >>> np.minimum.reduce(a, axis=2) >>> array([[ 0., 0., 0.], >>> [ 0., 0., 0.], >>> [ 0., 0., 0.], >>> [ 0., 0., 0.]]) >>> >>> a.T.shape >>> (10, 3, 4) >>> >>> python with iterable >>> >>> >>> reduce(np.maximum, a.T) >>> array([[ 4., 4., 4., 4.], >>> [ 4., 4., 4., 4.], >>> [ 4., 4., 4., 4.]]) >>> >>> reduce(np.minimum, a.T) >>> array([[ 0., 0., 0., 0.], >>> [ 0., 0., 0., 0.], >>> [ 0., 0., 0., 0.]]) >>> >>> Josef >>> >>> >> >>> >> _______________________________________________ >>> >> NumPy-Discussion mailing list >>> >> NumPy-Discussion@scipy.org >>> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>> >> >>> > >>> > >>> > _______________________________________________ >>> > NumPy-Discussion mailing list >>> > NumPy-Discussion@scipy.org >>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion >>> > >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@scipy.org >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I *think* it may work better if you replace the last 3 lines in your loop by: a=all_TSFC[0] if len(all_TSFC) > 1: N.maximum(a, TSFC, out=a) Not 100% sure that would work though, as I'm not entirely confident I understand your code. -=- Olivier 2011/12/6 questions anon <questions.anon@gmail.com>
Something fancier I think, I am able to compare the result with my previous method so I can easily see I am doing something wrong. see code below:
all_TSFC=[] for (path, dirs, files) in os.walk(MainFolder): for dir in dirs: print dir path=path+'/' for ncfile in files: if ncfile[-3:]=='.nc': print "dealing with ncfiles:", ncfile ncfile=os.path.join(path,ncfile) ncfile=Dataset(ncfile, 'r+', 'NETCDF4') TSFC=ncfile.variables['T_SFC'][:] fillvalue=ncfile.variables['T_SFC']._FillValue TSFC=MA.masked_values(TSFC, fillvalue) ncfile.close() all_TSFC.append(TSFC) a=TSFC[0] for b in TSFC[1:]: N.maximum(a,b,out=a)
big_array=N.ma.concatenate(all_TSFC) Max=big_array.max(axis=0) print "max is", Max,"a is", a
On Wed, Dec 7, 2011 at 2:34 PM, Olivier Delalleau <shish@keba.be> wrote:
Is 'a' a regular numpy array or something fancier?
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
thanks again my only problem though is that the out=a in the loop does not seem to replace my a= outside the loop so my final a is whatever I started with for a. Not sure what I am doing wrong whether it is something with the loop or with the command.
On Wed, Dec 7, 2011 at 1:44 PM, <josef.pktd@gmail.com> wrote:
On Tue, Dec 6, 2011 at 9:36 PM, Olivier Delalleau <shish@keba.be> wrote:
The "out=a" keyword will ensure your first array will keep being updated. So you can do something like:
a = my_list_of_arrays[0] for b in my_list_of_arrays[1:]: numpy.maximum(a, b, out=a)
I didn't think of the out argument which makes it more efficient, but in my example I used Python's reduce which takes an iterable and not one huge array.
Josef
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
thanks for all of your help, that does look appropriate but I am not
how to loop it over thousands of files. I need to keep the first array to compare with but replace any greater values as I loop through each array comparing back to the same array. does that make sense?
On Wed, Dec 7, 2011 at 1:12 PM, Olivier Delalleau <shish@keba.be> wrote: > > Thanks, I didn't know you could specify the out array :) > > (to the OP: my initial suggestion, although probably not very efficient, > seems to work with 2D arrays too, so I have no idea why it didn't work for > you -- but Nathaniel's one seems to be the ideal one anyway). > > -=- Olivier > > > 2011/12/6 Nathaniel Smith <njs@pobox.com> >> >> I think you want >> np.maximum(a, b, out=a) >> >> - Nathaniel >> >> On Dec 6, 2011 9:04 PM, "questions anon" <questions.anon@gmail.com
>> wrote: >>> >>> thanks for responding Josef but that is not really what I am looking >>> for, I have a multidimensional array and if the next array has any values >>> greater than what is in my first array I want to replace them. The data are >>> contained in netcdf files. >>> I can achieve what I want if I combine all of my arrays using numpy >>> concatenate and then using the command numpy.max(myarray, axis=0) but >>> because I have so many arrays I end up with a memory error so I need to find >>> a way to get the maximum while looping. >>> >>> >>> >>> On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote: >>>> >>>> On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau <shish@keba.be
>>>> wrote: >>>> > It may not be the most efficient way to do this, but you can do: >>>> > mask = b > a >>>> > a[mask] = b[mask] >>>> > >>>> > -=- Olivier >>>> > >>>> > 2011/12/6 questions anon <questions.anon@gmail.com> >>>> >> >>>> >> I would like to produce an array with the maximum values out of >>>> >> many >>>> >> (10000s) of arrays. >>>> >> I need to loop through many multidimentional arrays and if a value >>>> >> is >>>> >> larger (in the same place as the previous array) then I would
>>>> >> that >>>> >> value to replace it. >>>> >> >>>> >> e.g. >>>> >> a=[1,1,2,2 >>>> >> 11,2,2 >>>> >> 1,1,2,2] >>>> >> b=[1,1,3,2 >>>> >> 2,1,0,0 >>>> >> 1,1,2,0] >>>> >> >>>> >> where b>a replace with value in b, so the new a should be : >>>> >> >>>> >> a=[1,1,3,2] >>>> >> 2,1,2,2 >>>> >> 1,1,2,2] >>>> >> >>>> >> and then keep looping through many arrays and replace whenever >>>> >> value is >>>> >> larger. >>>> >> >>>> >> I have tried numpy.putmask but that results in >>>> >> TypeError: putmask() argument 1 must be numpy.ndarray, not
sure like list
>>>> >> Any other ideas? Thanks >>>> >>>> if I understand correctly it's a minimum.reduce >>>> >>>> numpy >>>> >>>> >>> a = np.concatenate((np.arange(5)[::-1], >>>> >>> np.arange(5)))*np.ones((4,3,1)) >>>> >>> np.minimum.reduce(a, axis=2) >>>> array([[ 0., 0., 0.], >>>> [ 0., 0., 0.], >>>> [ 0., 0., 0.], >>>> [ 0., 0., 0.]]) >>>> >>> a.T.shape >>>> (10, 3, 4) >>>> >>>> python with iterable >>>> >>>> >>> reduce(np.maximum, a.T) >>>> array([[ 4., 4., 4., 4.], >>>> [ 4., 4., 4., 4.], >>>> [ 4., 4., 4., 4.]]) >>>> >>> reduce(np.minimum, a.T) >>>> array([[ 0., 0., 0., 0.], >>>> [ 0., 0., 0., 0.], >>>> [ 0., 0., 0., 0.]]) >>>> >>>> Josef >>>> >>>> >> >>>> >> _______________________________________________ >>>> >> NumPy-Discussion mailing list >>>> >> NumPy-Discussion@scipy.org >>>> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>>> >> >>>> > >>>> > >>>> > _______________________________________________ >>>> > NumPy-Discussion mailing list >>>> > NumPy-Discussion@scipy.org >>>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion >>>> > >>>> _______________________________________________ >>>> NumPy-Discussion mailing list >>>> NumPy-Discussion@scipy.org >>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>> >>> >>> >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@scipy.org >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 07.12.2011, at 5:07AM, Olivier Delalleau wrote:
I *think* it may work better if you replace the last 3 lines in your loop by:
a=all_TSFC[0] if len(all_TSFC) > 1: N.maximum(a, TSFC, out=a)
Not 100% sure that would work though, as I'm not entirely confident I understand your code.
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com> Something fancier I think, I am able to compare the result with my previous method so I can easily see I am doing something wrong. see code below:
all_TSFC=[] for (path, dirs, files) in os.walk(MainFolder): for dir in dirs: print dir path=path+'/' for ncfile in files: if ncfile[-3:]=='.nc': print "dealing with ncfiles:", ncfile ncfile=os.path.join(path,ncfile) ncfile=Dataset(ncfile, 'r+', 'NETCDF4') TSFC=ncfile.variables['T_SFC'][:] fillvalue=ncfile.variables['T_SFC']._FillValue TSFC=MA.masked_values(TSFC, fillvalue) ncfile.close() all_TSFC.append(TSFC) a=TSFC[0] for b in TSFC[1:]: N.maximum(a,b,out=a)
I also understood TSFC is already the array you want to work on, so above you'd just take a slice and overwrite the result in the next file iteration anyway. Iterating over the list all_TSFC should be correct, but I understood you don't want to load the entire input into memory in you working code. Then you can simply skip the list, just need to take care of initial conditions - something like the following should do: path=path+'/' a = None for ncfile in files: if ncfile[-3:]=='.nc': print "dealing with ncfiles:", ncfile ncfile=os.path.join(path,ncfile) ncfile=Dataset(ncfile, 'r+', 'NETCDF4') TSFC=ncfile.variables['T_SFC'][:] fillvalue=ncfile.variables['T_SFC']._FillValue TSFC=MA.masked_values(TSFC, fillvalue) ncfile.close() if not is instance(a,N.ndarray): a=TSFC else: N.maximum(a, TSFC, out=a) HTH, Derek
big_array=N.ma.concatenate(all_TSFC) Max=big_array.max(axis=0) print "max is", Max,"a is", a
On 07/12/2011, at 1:49 PM, questions anon wrote:
fillvalue=ncfile.variables['T_SFC']._FillValue TSFC=MA.masked_values(TSFC, fillvalue)
You can probably also eliminate the above two lines from your code.
TSFC=ncfile.variables['T_SFC'][:]
If your NetCDF files are properly structured, the above line will give you a masked array. If you really need to put a fill value in to go to a non-masked array, better to do this just once after the maximums have been determined. Tim
sorry the 'all_TSFC' is for my other check of maximum using concatenate and N.max, I know that works so I am comparing it to this method. The only reason I need another method is for memory error issues. I like the code I have written so far as it makes sense to me. I can't get the extra examples I have been given to work and that is most likely because I don't understand them, these are the errors I get : Traceback (most recent call last): File "d:\plot_summarystats\test_plot_remove_memoryerror_max.py", line 46, in <module> N.maximum(a,TSFC,out=a) ValueError: non-broadcastable output operand with shape (106,193) doesn't match the broadcast shape (721,106,193) and Traceback (most recent call last): File "d:\plot_summarystats\test_plot_remove_memoryerror_max.py", line 45, in <module> if not instance(a, N.ndarray): NameError: name 'instance' is not defined On Wed, Dec 7, 2011 at 3:07 PM, Olivier Delalleau <shish@keba.be> wrote:
I *think* it may work better if you replace the last 3 lines in your loop by:
a=all_TSFC[0] if len(all_TSFC) > 1: N.maximum(a, TSFC, out=a)
Not 100% sure that would work though, as I'm not entirely confident I understand your code.
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
Something fancier I think, I am able to compare the result with my previous method so I can easily see I am doing something wrong. see code below:
all_TSFC=[] for (path, dirs, files) in os.walk(MainFolder): for dir in dirs: print dir path=path+'/' for ncfile in files: if ncfile[-3:]=='.nc': print "dealing with ncfiles:", ncfile ncfile=os.path.join(path,ncfile) ncfile=Dataset(ncfile, 'r+', 'NETCDF4') TSFC=ncfile.variables['T_SFC'][:] fillvalue=ncfile.variables['T_SFC']._FillValue TSFC=MA.masked_values(TSFC, fillvalue) ncfile.close() all_TSFC.append(TSFC) a=TSFC[0] for b in TSFC[1:]: N.maximum(a,b,out=a)
big_array=N.ma.concatenate(all_TSFC) Max=big_array.max(axis=0) print "max is", Max,"a is", a
On Wed, Dec 7, 2011 at 2:34 PM, Olivier Delalleau <shish@keba.be> wrote:
Is 'a' a regular numpy array or something fancier?
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com>
thanks again my only problem though is that the out=a in the loop does not seem to replace my a= outside the loop so my final a is whatever I started with for a. Not sure what I am doing wrong whether it is something with the loop or with the command.
On Wed, Dec 7, 2011 at 1:44 PM, <josef.pktd@gmail.com> wrote:
On Tue, Dec 6, 2011 at 9:36 PM, Olivier Delalleau <shish@keba.be> wrote:
The "out=a" keyword will ensure your first array will keep being updated. So you can do something like:
a = my_list_of_arrays[0] for b in my_list_of_arrays[1:]: numpy.maximum(a, b, out=a)
I didn't think of the out argument which makes it more efficient, but in my example I used Python's reduce which takes an iterable and not one huge array.
Josef
-=- Olivier
2011/12/6 questions anon <questions.anon@gmail.com> > > thanks for all of your help, that does look appropriate but I am
> how to loop it over thousands of files. > I need to keep the first array to compare with but replace any greater > values as I loop through each array comparing back to the same array. does > that make sense? > > > On Wed, Dec 7, 2011 at 1:12 PM, Olivier Delalleau <shish@keba.be> wrote: >> >> Thanks, I didn't know you could specify the out array :) >> >> (to the OP: my initial suggestion, although probably not very efficient, >> seems to work with 2D arrays too, so I have no idea why it didn't work for >> you -- but Nathaniel's one seems to be the ideal one anyway). >> >> -=- Olivier >> >> >> 2011/12/6 Nathaniel Smith <njs@pobox.com> >>> >>> I think you want >>> np.maximum(a, b, out=a) >>> >>> - Nathaniel >>> >>> On Dec 6, 2011 9:04 PM, "questions anon" < questions.anon@gmail.com> >>> wrote: >>>> >>>> thanks for responding Josef but that is not really what I am looking >>>> for, I have a multidimensional array and if the next array has any values >>>> greater than what is in my first array I want to replace them. The data are >>>> contained in netcdf files. >>>> I can achieve what I want if I combine all of my arrays using numpy >>>> concatenate and then using the command numpy.max(myarray, axis=0) but >>>> because I have so many arrays I end up with a memory error so I need to find >>>> a way to get the maximum while looping. >>>> >>>> >>>> >>>> On Wed, Dec 7, 2011 at 12:36 PM, <josef.pktd@gmail.com> wrote: >>>>> >>>>> On Tue, Dec 6, 2011 at 7:55 PM, Olivier Delalleau < shish@keba.be> >>>>> wrote: >>>>> > It may not be the most efficient way to do this, but you can do: >>>>> > mask = b > a >>>>> > a[mask] = b[mask] >>>>> > >>>>> > -=- Olivier >>>>> > >>>>> > 2011/12/6 questions anon <questions.anon@gmail.com> >>>>> >> >>>>> >> I would like to produce an array with the maximum values out of >>>>> >> many >>>>> >> (10000s) of arrays. >>>>> >> I need to loop through many multidimentional arrays and if a value >>>>> >> is >>>>> >> larger (in the same place as the previous array) then I would like >>>>> >> that >>>>> >> value to replace it. >>>>> >> >>>>> >> e.g. >>>>> >> a=[1,1,2,2 >>>>> >> 11,2,2 >>>>> >> 1,1,2,2] >>>>> >> b=[1,1,3,2 >>>>> >> 2,1,0,0 >>>>> >> 1,1,2,0] >>>>> >> >>>>> >> where b>a replace with value in b, so the new a should be : >>>>> >> >>>>> >> a=[1,1,3,2] >>>>> >> 2,1,2,2 >>>>> >> 1,1,2,2] >>>>> >> >>>>> >> and then keep looping through many arrays and replace whenever >>>>> >> value is >>>>> >> larger. >>>>> >> >>>>> >> I have tried numpy.putmask but that results in >>>>> >> TypeError: putmask() argument 1 must be numpy.ndarray, not
not sure list
>>>>> >> Any other ideas? Thanks >>>>> >>>>> if I understand correctly it's a minimum.reduce >>>>> >>>>> numpy >>>>> >>>>> >>> a = np.concatenate((np.arange(5)[::-1], >>>>> >>> np.arange(5)))*np.ones((4,3,1)) >>>>> >>> np.minimum.reduce(a, axis=2) >>>>> array([[ 0., 0., 0.], >>>>> [ 0., 0., 0.], >>>>> [ 0., 0., 0.], >>>>> [ 0., 0., 0.]]) >>>>> >>> a.T.shape >>>>> (10, 3, 4) >>>>> >>>>> python with iterable >>>>> >>>>> >>> reduce(np.maximum, a.T) >>>>> array([[ 4., 4., 4., 4.], >>>>> [ 4., 4., 4., 4.], >>>>> [ 4., 4., 4., 4.]]) >>>>> >>> reduce(np.minimum, a.T) >>>>> array([[ 0., 0., 0., 0.], >>>>> [ 0., 0., 0., 0.], >>>>> [ 0., 0., 0., 0.]]) >>>>> >>>>> Josef >>>>> >>>>> >> >>>>> >> _______________________________________________ >>>>> >> NumPy-Discussion mailing list >>>>> >> NumPy-Discussion@scipy.org >>>>> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>>>> >> >>>>> > >>>>> > >>>>> > _______________________________________________ >>>>> > NumPy-Discussion mailing list >>>>> > NumPy-Discussion@scipy.org >>>>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion >>>>> > >>>>> _______________________________________________ >>>>> NumPy-Discussion mailing list >>>>> NumPy-Discussion@scipy.org >>>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>>> >>>> >>>> >>>> _______________________________________________ >>>> NumPy-Discussion mailing list >>>> NumPy-Discussion@scipy.org >>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>>> >>> >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@scipy.org >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>> >> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 07.12.2011, at 5:54AM, questions anon wrote:
sorry the 'all_TSFC' is for my other check of maximum using concatenate and N.max, I know that works so I am comparing it to this method. The only reason I need another method is for memory error issues. I like the code I have written so far as it makes sense to me. I can't get the extra examples I have been given to work and that is most likely because I don't understand them, these are the errors I get :
Traceback (most recent call last): File "d:\plot_summarystats\test_plot_remove_memoryerror_max.py", line 46, in <module> N.maximum(a,TSFC,out=a) ValueError: non-broadcastable output operand with shape (106,193) doesn't match the broadcast shape (721,106,193)
and
OK, then it seems we did not indeed grasp the entire scope of the problem - since you have initialised a from the previous array TSFC (not from TSFC[0]?!), this can only mean the arrays read in come in different shapes? I don't quite understand how the previous version did not raise an error then; but if you only want the (106,193)-subarray you have indeed to keep the loop for b in TSFC[:]: N.maximum(a,b,out=a) But you would have to find some way to distinguish between ndim=2 and ndim=3 input, if really both can occur...
Traceback (most recent call last): File "d:\plot_summarystats\test_plot_remove_memoryerror_max.py", line 45, in <module> if not instance(a, N.ndarray): NameError: name 'instance' is not defined
Sorry, typing error (or devious auto-correct?) - this should be 'isinstance()' Cheers, Derek
thanks for all your responses. I think I have FINALLY worked it out with all of your help. I just assigned one array from one ncfile to "a" at the beginning of my code and then ran the loop and it worked!! sorry for all the questions but I learn so much playing and getting ideas from others. Thanks again. code below for anyone else that needs to do the same. onefile=Dataset("E:/01/IDZ00026_T_SFC.nc", 'r+', 'NETCDF4') oneTSFC=onefile.variables['T_SFC'][:] a=oneTSFC[0] for (path, dirs, files) in os.walk(MainFolder): for dir in dirs: print dir path=path+'/' for ncfile in files: if ncfile[-3:]=='.nc': ncfile=os.path.join(path,ncfile) ncfile=Dataset(ncfile, 'r+', 'NETCDF4') TSFC=ncfile.variables['T_SFC'][:] ncfile.close() for b in TSFC[:]: N.maximum(a,b, out=a) print a On Wed, Dec 7, 2011 at 4:11 PM, Derek Homeier < derek@astro.physik.uni-goettingen.de> wrote:
On 07.12.2011, at 5:54AM, questions anon wrote:
sorry the 'all_TSFC' is for my other check of maximum using concatenate and N.max, I know that works so I am comparing it to this method. The only reason I need another method is for memory error issues. I like the code I have written so far as it makes sense to me. I can't get the extra examples I have been given to work and that is most likely because I don't understand them, these are the errors I get :
Traceback (most recent call last): File "d:\plot_summarystats\test_plot_remove_memoryerror_max.py", line 46, in <module> N.maximum(a,TSFC,out=a) ValueError: non-broadcastable output operand with shape (106,193) doesn't match the broadcast shape (721,106,193)
and
OK, then it seems we did not indeed grasp the entire scope of the problem - since you have initialised a from the previous array TSFC (not from TSFC[0]?!), this can only mean the arrays read in come in different shapes? I don't quite understand how the previous version did not raise an error then; but if you only want the (106,193)-subarray you have indeed to keep the loop for b in TSFC[:]: N.maximum(a,b,out=a)
But you would have to find some way to distinguish between ndim=2 and ndim=3 input, if really both can occur...
Traceback (most recent call last): File "d:\plot_summarystats\test_plot_remove_memoryerror_max.py", line
45, in <module>
if not instance(a, N.ndarray): NameError: name 'instance' is not defined
Sorry, typing error (or devious auto-correct?) - this should be 'isinstance()'
Cheers, Derek
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (6)
-
Derek Homeier -
josef.pktd＠gmail.com -
Nathaniel Smith -
Olivier Delalleau -
questions anon -
Tim Burgess