OT: A Way to Approximate and Compress a 3D Surface
Hi Everyone, This is off topic for this mailing list but I don't know where else to ask. I have N tabulated data points { (x_i, y_i, z_i) } that describes a 3D surface. The surface is pretty "smooth." However, the number of data points is too large to be stored and manipulated efficiently. To make it easier to deal with, I am looking for an easy method to compress and approximate the data. Maybe the approximation can be described by far fewer number of coefficients. If you can give me some hints about possible numpy or nonnumpy solutions or let me know where is better to ask this kind of question, I would really appreciate it. Many thanks, Geoffrey
A Tuesday 20 November 2007, Geoffrey Zhu escrigué:
Hi Everyone,
This is off topic for this mailing list but I don't know where else to ask.
I have N tabulated data points { (x_i, y_i, z_i) } that describes a 3D surface. The surface is pretty "smooth." However, the number of data points is too large to be stored and manipulated efficiently. To make it easier to deal with, I am looking for an easy method to compress and approximate the data. Maybe the approximation can be described by far fewer number of coefficients.
If you can give me some hints about possible numpy or nonnumpy solutions or let me know where is better to ask this kind of question, I would really appreciate it.
First, a good and easy try would be to use PyTables. It does support ontheflight compression, that is, allows you to access compressed dataset slices without decompressing the complete dataset. This, in combination with a handy 'shuffle' filter (also included), allows for pretty good compression ratios on numerical data. See [1] [2] for a discussion on how to use and what you can expect from a compressor/shuffle process on PyTables. Also, if you can afford lossy compression, you may want to try truncation (quantization) before compressing as it does benefit the compression rate quite a lot. Feel free to experiment with the next function (Jeffrey Whittaker was the original author): def _quantize(data,least_significant_digit): """quantize data to improve compression. data is quantized using around(scale*data)/scale, where scale is 2**bits, and bits is determined from the least_significant_digit. For example, if least_significant_digit=1, bits will be 4.""" precision = 10.**least_significant_digit exp = math.log(precision,10) if exp < 0: exp = int(math.floor(exp)) else: exp = int(math.ceil(exp)) bits = math.ceil(math.log(10.**exp,2)) scale = 2.**bits return numpy.around(scale*data)/scale [1] http://www.pytables.org/docs/manual/ch05.html#compressionIssues [2] http://www.pytables.org/docs/manual/ch05.html#ShufflingOptim Cheers, 
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data ""
On 20/11/2007, Geoffrey Zhu <zyzhu2000@gmail.com> wrote:
I have N tabulated data points { (x_i, y_i, z_i) } that describes a 3D surface. The surface is pretty "smooth." However, the number of data points is too large to be stored and manipulated efficiently. To make it easier to deal with, I am looking for an easy method to compress and approximate the data. Maybe the approximation can be described by far fewer number of coefficients.
If you can give me some hints about possible numpy or nonnumpy solutions or let me know where is better to ask this kind of question, I would really appreciate it.
This is an important task in computer graphics, in particular, in the field of multiresolution modelling. If you look up "surface simplification" you'll find many references to articles. I don't know of a library offhand that does it, let alone one that is accessible from python, but you could try looking at a toolkit that does isosurface visualization  these are surfaces that can often be simplified enormously. In particular it looks like VTK might be able to do what you want. Anne
Anne Archibald wrote:
In particular it looks like VTK might be able to do what you want.
yes, it can. The way I've seen is to triangulate the surface, then do "decimation"  remove the triangles that don't add "much" to the detail of the surface. I don't know of a way to do something similar keeping the Cartesian grid  but you may not need to. Google "python triangle decimation" and VTK comes out on top, but there are some other things there that might be of interest too. Chris  Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 5266959 voice 7600 Sand Point Way NE (206) 5266329 fax Seattle, WA 98115 (206) 5266317 main reception Chris.Barker@noaa.gov
participants (4)

Anne Archibald

Christopher Barker

Francesc Altet

Geoffrey Zhu