Thanks, Matt, John, some follow-up
"In any case, the difference in the total quantity is about a few percent
it looks like. Are you attempting to quantify something with that level of
sensitivity?"
- I'm trying to compare star particle mass, and if I use TotalQuantity and
StarMassMsun comparing to summing the star particle's mass. For small
halos that are only a few cells, the difference can be quite high ~O(10%).
So from the perspective of actually particles inside the halo, I should
use the particle list inside the 3D object, right? Same thing with dark
matter particle mass?
"The output isn't what you might expect, because numpy uses the dtype of
the array for the accumulation."
- I don't quite understand why would float32 accumulation with sum() not
equal to the actual thing but casting it as float64 does (isn't 2^32 <
512^3 so no overflow should occur)? Should I always use
arr.sum(dtype='float64') when using sum() with possible large datasets?
From
G.S.
On Sat, Dec 21, 2013 at 5:43 AM, Matthew Turk
Hi Geoffrey,
On Fri, Dec 20, 2013 at 9:02 PM, Geoffrey So
wrote: Hi all,
I found that using a Enzo dataset I was getting slightly different numbers when 1) using the list of dark matter particles selected by creation_time < 0.0
In [97]: sph_dm = sph['creation_time'] < 0.0 In [98]: print "%12.12e" % (sph['ParticleMassMsun'][sph_dm]).sum() 1.211311468567e+11
2) compared to summing the dark matter particles inside a 3D container with TotalQuantity
In [101]: print "%12.12e" % (sph.quantities['TotalQuantity']('Dark_Matter_Density')[0]*vol/Msun) 1.188937185993e+11
I'm wondering if the field and particles are handled differently when being counted as inside or outside the 3D container?
I think John's answer is completely correct, and likely the dominant problem, but there's a funny thing that happens if you're not extra careful with big arrays. As an example, something that I ran into a couple years ago:
# This is an array filled with ones. arr = numpy.ones((512, 512, 512), dtype="float32") print arr.sum() print arr.size print arr.sum(dtype = "float64")
The output isn't what you might expect, because numpy uses the dtype of the array for the accumulation. The quantities -- thanks to Doug Rudd -- are uniformly careful about upgrading to 64 bits the accumulators used for quantities. Again, I don't think this is dominant, but it is something to be aware of.
-Matt
From G.S.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org