Re: [Numpy-discussion] effectively computing variograms with numpy
On 6/25/07, Hanno Klemm <klemm@phys.ethz.ch> wrote:
Tim,
Thank you very much, the code does what's it expected to do. Unfortunately the thing is still pretty slow on large data sets.
This does seem like the kind of thing that there should be a faster way to compute, particularly since you are binning the results up. One approach would be to just bin the data before doing the computation, however, that loses a lot of accuracy. It does seem like there should be some moment-like approach that would allow you to bin the data before you do the computation, computing the first and second moments, or something similar and then computing the results from the binned moments. I don't know that that would work -- it's just a vague hunch. I don't know if I'll have time to try it out, but I thought I would mention it. I will probably now look for ways to calculate the variogram from some random samples of my data. Thanks for the observation regarding the square array, that would have bitten me, later. Your welcome, I hope this helps some. Regards, -- . __ . |-\ . . tim.hochberg@ieee.org [SNIP]
I will try and dig a bit more in the literature, maybe I find something. Hanno On Jun 25, 2007, at 4:59 PM, Timothy Hochberg wrote:
On 6/25/07, Hanno Klemm < klemm@phys.ethz.ch> wrote:
Tim,
Thank you very much, the code does what's it expected to do. Unfortunately the thing is still pretty slow on large data sets. This does seem like the kind of thing that there should be a faster way to compute, particularly since you are binning the results up. One approach would be to just bin the data before doing the computation, however, that loses a lot of accuracy. It does seem like there should be some moment-like approach that would allow you to bin the data before you do the computation, computing the first and second moments, or something similar and then computing the results from the binned moments. I don't know that that would work -- it's just a vague hunch. I don't know if I'll have time to try it out, but I thought I would mention it.
Your welcome, I hope this helps some.
Regards,
-- . __ . |-\ . . tim.hochberg@ieee.org
[SNIP]
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Hanno Klemm hanno.klemm@xs4all.nl http://www.mth.kcl.ac.uk/~klemm
On Mon, 2007-06-25 at 23:09 +0200, Hanno Klemm wrote:
I will try and dig a bit more in the literature, maybe I find something.
Hanno
I don't know if it can help. We started a project to convert BMELib (a matlab library) into Python. It's still bound with Numeric but it should be pretty straigthforward to convert it to numpy. The translation was not finished but the variogram methods worked pretty fine and the benchmarks between the Python and Matlab version were very interesting. If you want to investigate it, see http://bmelibpy.sourceforge.net (variograms are in the statlib file). Didrik
Didrik, thanks, I'll definitely will have a look at this. Hanno Didrik Pinte <dpinte@itae.be> said:
--=-aUNlfGW7wc8MzGzdSDGo Content-Type: text/plain Content-Transfer-Encoding: quoted-printable
On Mon, 2007-06-25 at 23:09 +0200, Hanno Klemm wrote:
I will try and dig a bit more in the literature, maybe I find
something.
=20 Hanno
I don't know if it can help. We started a project to convert BMELib (a matlab library) into Python. It's still bound with Numeric but it should be pretty straigthforward to convert it to numpy. The translation was not finished but the variogram methods worked pretty fine and the benchmarks between the Python and Matlab version were very interesting.
If you want to investigate it, see http://bmelibpy.sourceforge.net (variograms are in the statlib file).
Didrik
--=-aUNlfGW7wc8MzGzdSDGo Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQBGgNcS9Rlh4Zs4yBMRAh2JAJ4qlmltml+/b/wYo+JohjN0NpzK/ACfWWpL eealZmRl1Cw6BfTRVBXy1TE= =xuLt -----END PGP SIGNATURE-----
--=-aUNlfGW7wc8MzGzdSDGo--
-- Hanno Klemm klemm@phys.ethz.ch
participants (3)
-
Didrik Pinte -
Hanno Klemm -
Timothy Hochberg