looking for code advice
Hi First the disclaimer: This is my first numpy experience, so I have next to no idea what I'm doing. I've muddled through and managed to put together some code for my current problem and now that I have it going I'd like to hear any comments people may have on both my solution and other ways of approaching the problem. I have two goals here, I'd like to make the process run faster and I'd like to broaden my understanding of numpy as I can see from my brief use of it that it is a remarkably powerful tool. Now to the problem at hand. I find this difficult to explain but will try as best I can. The best word I have for the process is decimation. The input and output are both 3 dimensional arrays of uint8's. The output is half the size of the input along each dimension. Each cell [x,y,z] in the output corresponds to the 2x2x2 block [2*x:2*x+2, 2*y:2*y+2, 2*z:2*z+2] in the input. The tricky bit is in how the correspondence works. If all the cells in the input block have the same value then the cell in the output block will also have that value. Otherwise the output cell will have the value MIXED. Here is my current solution, from my limited testing it seems to produce the result I'm after. def decimate(data_in): in_x, in_y, in_z = data_in.shape out_x = in_x / 2 out_y = in_y / 2 out_z = in_z / 2 out_shape = out_x, out_y, out_z out_size = product(out_shape) # figure out which chunks are homogeneous reshaped_array = data_in.reshape(out_x, 2, out_y, 2, out_z, 2).transpose(0,2,4,1,3,5).reshape(out_x, out_y, out_z, 8) min_array = numpy.amin(reshaped_array, axis=3) max_array = numpy.amax(reshaped_array, axis=3) equal_array = numpy.equal(min_array, max_array) # select the actual value for the homogeneous chunks and MIXED for the heterogeneous decimated_array = data_in[::2,::2,::2] mixed_array = numpy.tile(MIXED, out_size).reshape(out_shape) data_out = numpy.where(equal_array, decimated_array, mixed_array) return data_out For the curious this is will be used to build a voxel octtree for a 3d graphics application. The final setup will be more complicated, this is the minimum that will let me get up and running. Regards Gordon P.S. congrats on numpy, it is a very impressive tool, I've only scraped the surface and it's already impressed me several times over.
On Wed, Sep 29, 2010 at 8:25 AM, Gordon Wrigley
Hi First the disclaimer: This is my first numpy experience, so I have next to no idea what I'm doing. I've muddled through and managed to put together some code for my current problem and now that I have it going I'd like to hear any comments people may have on both my solution and other ways of approaching the problem. I have two goals here, I'd like to make the process run faster and I'd like to broaden my understanding of numpy as I can see from my brief use of it that it is a remarkably powerful tool. Now to the problem at hand. I find this difficult to explain but will try as best I can. The best word I have for the process is decimation. The input and output are both 3 dimensional arrays of uint8's. The output is half the size of the input along each dimension. Each cell [x,y,z] in the output corresponds to the 2x2x2 block [2*x:2*x+2, 2*y:2*y+2, 2*z:2*z+2] in the input. The tricky bit is in how the correspondence works. If all the cells in the input block have the same value then the cell in the output block will also have that value. Otherwise the output cell will have the value MIXED. Here is my current solution, from my limited testing it seems to produce the result I'm after. def decimate(data_in): in_x, in_y, in_z = data_in.shape out_x = in_x / 2 out_y = in_y / 2 out_z = in_z / 2 out_shape = out_x, out_y, out_z out_size = product(out_shape) # figure out which chunks are homogeneous reshaped_array = data_in.reshape(out_x, 2, out_y, 2, out_z, 2).transpose(0,2,4,1,3,5).reshape(out_x, out_y, out_z, 8) min_array = numpy.amin(reshaped_array, axis=3) max_array = numpy.amax(reshaped_array, axis=3) equal_array = numpy.equal(min_array, max_array) # select the actual value for the homogeneous chunks and MIXED for the heterogeneous decimated_array = data_in[::2,::2,::2] mixed_array = numpy.tile(MIXED, out_size).reshape(out_shape) data_out = numpy.where(equal_array, decimated_array, mixed_array)
data_out = numpy.where(equal_array, decimated_array, MIXED) should work I don't see anything else, unless there is something in scipy.ndimage. I have to remember your reshape trick for 3d. (I don't know how many temporary arrays this creates.) Josef
return data_out For the curious this is will be used to build a voxel octtree for a 3d graphics application. The final setup will be more complicated, this is the minimum that will let me get up and running. Regards Gordon P.S. congrats on numpy, it is a very impressive tool, I've only scraped the surface and it's already impressed me several times over.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Sep 29, 2010 at 9:19 AM,
On Wed, Sep 29, 2010 at 8:25 AM, Gordon Wrigley
wrote: Hi First the disclaimer: This is my first numpy experience, so I have next to no idea what I'm doing. I've muddled through and managed to put together some code for my current problem and now that I have it going I'd like to hear any comments people may have on both my solution and other ways of approaching the problem. I have two goals here, I'd like to make the process run faster and I'd like to broaden my understanding of numpy as I can see from my brief use of it that it is a remarkably powerful tool. Now to the problem at hand. I find this difficult to explain but will try as best I can. The best word I have for the process is decimation. The input and output are both 3 dimensional arrays of uint8's. The output is half the size of the input along each dimension. Each cell [x,y,z] in the output corresponds to the 2x2x2 block [2*x:2*x+2, 2*y:2*y+2, 2*z:2*z+2] in the input. The tricky bit is in how the correspondence works. If all the cells in the input block have the same value then the cell in the output block will also have that value. Otherwise the output cell will have the value MIXED. Here is my current solution, from my limited testing it seems to produce the result I'm after. def decimate(data_in): in_x, in_y, in_z = data_in.shape out_x = in_x / 2 out_y = in_y / 2 out_z = in_z / 2 out_shape = out_x, out_y, out_z out_size = product(out_shape) # figure out which chunks are homogeneous reshaped_array = data_in.reshape(out_x, 2, out_y, 2, out_z, 2).transpose(0,2,4,1,3,5).reshape(out_x, out_y, out_z, 8) min_array = numpy.amin(reshaped_array, axis=3) max_array = numpy.amax(reshaped_array, axis=3) equal_array = numpy.equal(min_array, max_array)
maybe ptp==0 is faster numpy.ptp(a, axis=None, out=None) Range of values (maximum - minimum) along an axis. Josef
# select the actual value for the homogeneous chunks and MIXED for the heterogeneous decimated_array = data_in[::2,::2,::2] mixed_array = numpy.tile(MIXED, out_size).reshape(out_shape) data_out = numpy.where(equal_array, decimated_array, mixed_array)
data_out = numpy.where(equal_array, decimated_array, MIXED) should work
I don't see anything else, unless there is something in scipy.ndimage.
I have to remember your reshape trick for 3d. (I don't know how many temporary arrays this creates.)
Josef
return data_out For the curious this is will be used to build a voxel octtree for a 3d graphics application. The final setup will be more complicated, this is the minimum that will let me get up and running. Regards Gordon P.S. congrats on numpy, it is a very impressive tool, I've only scraped the surface and it's already impressed me several times over.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi Josef Thanks for your comments.
numpy.ptp(a, axis=None, out=None) Range of values (maximum - minimum) along an axis.
I feel silly for not having noticed that function, it's right next to amin
and amax in the docs.
Both your suggestions improved the performance, but only a little,
regardless they make the code simpler which is always a good thing.
This is what I have currently:
def decimate(data_in):
in_x, in_y, in_z = data_in.shape
out_x = in_x / 2
out_y = in_y / 2
out_z = in_z / 2
out_shape = out_x, out_y, out_z
out_size = product(out_shape)
# figure out which chunks are homogeneous
reshaped_array = data_in.reshape(out_x, 2, out_y, 2, out_z,
2).transpose(0,2,4,1,3,5).reshape(out_x, out_y, out_z, 8)
not_equal_array = numpy.ptp(reshaped_array, axis=3)
# select the actual value for the homogeneous chunks and MIXED for the
heterogeneous
decimated_array = reshaped_array[:,:,:,0]
data_out = numpy.where(not_equal_array, MIXED, decimated_array)
return data_out
Regards Gordon
On Thu, Sep 30, 2010 at 2:36 AM,
On Wed, Sep 29, 2010 at 8:25 AM, Gordon Wrigley
wrote: Hi First the disclaimer: This is my first numpy experience, so I have next to no idea what I'm doing. I've muddled through and managed to put together some code for my current problem and now that I have it going I'd like to hear any comments
may have on both my solution and other ways of approaching the problem. I have two goals here, I'd like to make the process run faster and I'd
to broaden my understanding of numpy as I can see from my brief use of it that it is a remarkably powerful tool. Now to the problem at hand. I find this difficult to explain but will
best I can. The best word I have for the process is decimation. The input and output are both 3 dimensional arrays of uint8's. The output is half the size of the input along each dimension. Each cell [x,y,z] in the output corresponds to the 2x2x2 block [2*x:2*x+2, 2*y:2*y+2, 2*z:2*z+2] in the input. The
bit is in how the correspondence works. If all the cells in the input block have the same value then the cell in the output block will also have
value. Otherwise the output cell will have the value MIXED. Here is my current solution, from my limited testing it seems to produce
On Wed, Sep 29, 2010 at 9:19 AM,
wrote: people like try as tricky that the result I'm after. def decimate(data_in): in_x, in_y, in_z = data_in.shape out_x = in_x / 2 out_y = in_y / 2 out_z = in_z / 2 out_shape = out_x, out_y, out_z out_size = product(out_shape) # figure out which chunks are homogeneous reshaped_array = data_in.reshape(out_x, 2, out_y, 2, out_z, 2).transpose(0,2,4,1,3,5).reshape(out_x, out_y, out_z, 8) min_array = numpy.amin(reshaped_array, axis=3) max_array = numpy.amax(reshaped_array, axis=3) equal_array = numpy.equal(min_array, max_array)
maybe ptp==0 is faster
numpy.ptp(a, axis=None, out=None) Range of values (maximum - minimum) along an axis.
Josef
# select the actual value for the homogeneous chunks and MIXED for
the
heterogeneous decimated_array = data_in[::2,::2,::2] mixed_array = numpy.tile(MIXED, out_size).reshape(out_shape) data_out = numpy.where(equal_array, decimated_array, mixed_array)
data_out = numpy.where(equal_array, decimated_array, MIXED) should work
I don't see anything else, unless there is something in scipy.ndimage.
I have to remember your reshape trick for 3d. (I don't know how many temporary arrays this creates.)
Josef
return data_out For the curious this is will be used to build a voxel octtree for a 3d graphics application. The final setup will be more complicated, this is
minimum that will let me get up and running. Regards Gordon P.S. congrats on numpy, it is a very impressive tool, I've only scraped
the the
surface and it's already impressed me several times over.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (2)
-
Gordon Wrigley
-
josef.pktd@gmail.com