I am all for binary, especially since storage is a big problem for some of the bigger simulations. I am just curious, though, which binary are the other simulation groups pushing for? Is there a consistent way of storing binary data in YT? I am asking because currently sometimes I pickle my own data or store them in hdf5 .h5 binary or in the .yt files, and I didn't realize the different binaries are not compatible until recently. Are we looking for a universal YT way of storing binaries? One big advantage would be everyone being able to read the binary if we all store them the same way, and I'm sure there's many other advantages. But the headache is everyone agreeing to store them whichever way. From G.S. On Mon, Aug 15, 2011 at 6:02 AM, Matthew Turk <matthewturk@gmail.com> wrote:
Hi Stephen and Geoffrey,
I would prefer we stick with the longer IO output. The reason is not as much that we believe that a halo truly does exist with that specified precision, but to do our very best to ensure that we communicate between sessions the precise location. This may also come into play with very high precision runs.
My personal preference would be to utilize an all-binary storage format as our *primary* storage format and then allow ASCII for secondary, caveat emptor purposes. I believe that both the IRATE group and the Galacticus group are pushing forward with halo cataloging methods that will be binary.
-Matt
On Sun, Aug 14, 2011 at 9:24 AM, Stephen Skory <s@skory.us> wrote:
Hi all,
With the current setting, the halo attributes are outputted with 9 decimal points, but the ellipsoid parameters determined using the particle's position (when the data is 64 bit) has 16 decimals.
just to clarify, what I've done is to add the option to the halos.write_out() function (that outputs the HopAnalysis.out file) to add 5 or so extra columns for the ellipsoid information. So what Geoffrey is thinking about is increasing the precision of all the floats in that text file.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org