Hi all, I've adapted yt's gdf_writer.py into a standalone class called pygdf ( https://bitbucket.org/jsoishi/pygdf). I've done this so that any python code can now save data in the gdf format. While doing this, I came across something in the yt gdf frontend that I'm not quite sure I understand. In the grid metadata, yt expects /grid_particle_count to be a 2-D array, but the gdf standard clearly states this should be a 1-D (int64, N) array, where N is the number of grids. Further complicating the issue is the fact that if I create a 1-D array for /grid_particle_count, the failure point in the following script from yt.mods import * pf = load("/tmp/blah.gdf") sp = SlicePlot(pf, 2, "Density") is actually in data_objects/grid_patch.py, not in the gdf frontend itself. The file /tmp/blah.gdf can be generated by running the test.py script in pygdf. The obvious workaround is to simply make /grid_particle_count a 2D array (this is especially so since I'm not actually using particles at the moment), but I'm not sure why this should be so. The total count of particles in N grids seems to be better fit in a 1-D array to me. However, I'd like to understand better what's going on. Any pointers or clarification would be very helpful. Also, any feedback on pygdf would be most welcome! thanks, j
I might be missing something, but can't we just reshape/transform the array from GDF into the format yt wants? John ZuHone Laboratory for High-Energy Astrophysics NASA/Goddard Space Flight Center 8800 Greenbelt Rd., Mail Code 662 Greenbelt, MD 20771 (w) 301-286-2531 (m) 781-708-5004 john.zuhone@nasa.gov jzuhone@gmail.com
On Jan 29, 2014, at 5:22 PM, j s oishi <jsoishi@gmail.com> wrote:
Hi all,
I've adapted yt's gdf_writer.py into a standalone class called pygdf (https://bitbucket.org/jsoishi/pygdf). I've done this so that any python code can now save data in the gdf format.
While doing this, I came across something in the yt gdf frontend that I'm not quite sure I understand. In the grid metadata, yt expects /grid_particle_count to be a 2-D array, but the gdf standard clearly states this should be a 1-D (int64, N) array, where N is the number of grids. Further complicating the issue is the fact that if I create a 1-D array for /grid_particle_count, the failure point in the following script
from yt.mods import * pf = load("/tmp/blah.gdf")
sp = SlicePlot(pf, 2, "Density")
is actually in data_objects/grid_patch.py, not in the gdf frontend itself. The file /tmp/blah.gdf can be generated by running the test.py script in pygdf.
The obvious workaround is to simply make /grid_particle_count a 2D array (this is especially so since I'm not actually using particles at the moment), but I'm not sure why this should be so. The total count of particles in N grids seems to be better fit in a 1-D array to me. However, I'd like to understand better what's going on. Any pointers or clarification would be very helpful.
Also, any feedback on pygdf would be most welcome!
thanks,
j _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi John, Sure, but I guess my bigger question was: why does yt expect this to be a 2D array? j On Wed, Jan 29, 2014 at 8:08 PM, John ZuHone <jzuhone@gmail.com> wrote:
I might be missing something, but can't we just reshape/transform the array from GDF into the format yt wants?
John ZuHone Laboratory for High-Energy Astrophysics NASA/Goddard Space Flight Center 8800 Greenbelt Rd., Mail Code 662 Greenbelt, MD 20771 (w) 301-286-2531 (m) 781-708-5004 john.zuhone@nasa.gov jzuhone@gmail.com
On Jan 29, 2014, at 5:22 PM, j s oishi <jsoishi@gmail.com> wrote:
Hi all,
I've adapted yt's gdf_writer.py into a standalone class called pygdf ( https://bitbucket.org/jsoishi/pygdf). I've done this so that any python code can now save data in the gdf format.
While doing this, I came across something in the yt gdf frontend that I'm not quite sure I understand. In the grid metadata, yt expects /grid_particle_count to be a 2-D array, but the gdf standard clearly states this should be a 1-D (int64, N) array, where N is the number of grids. Further complicating the issue is the fact that if I create a 1-D array for /grid_particle_count, the failure point in the following script
from yt.mods import * pf = load("/tmp/blah.gdf")
sp = SlicePlot(pf, 2, "Density")
is actually in data_objects/grid_patch.py, not in the gdf frontend itself. The file /tmp/blah.gdf can be generated by running the test.py script in pygdf.
The obvious workaround is to simply make /grid_particle_count a 2D array (this is especially so since I'm not actually using particles at the moment), but I'm not sure why this should be so. The total count of particles in N grids seems to be better fit in a 1-D array to me. However, I'd like to understand better what's going on. Any pointers or clarification would be very helpful.
Also, any feedback on pygdf would be most welcome!
thanks,
j
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
On Thu, Jan 30, 2014 at 10:15 AM, j s oishi <jsoishi@gmail.com> wrote:
Hi John,
Sure, but I guess my bigger question was: why does yt expect this to be a 2D array?
Historical reasons we should discard. All the arrays in the original Enzo frontend were 2D where one dimension might be of size 1. That being said, we should revisit this now that we better support multiple particle species. So it should be changed now, but perhaps in the future fixed. An easy way to fix it is to have the GDF reader *fill* an array rather than create one; this is what we do elsewhere. some_array[:,0] = other_array[:] -Matt
j
On Wed, Jan 29, 2014 at 8:08 PM, John ZuHone <jzuhone@gmail.com> wrote:
I might be missing something, but can't we just reshape/transform the array from GDF into the format yt wants?
John ZuHone Laboratory for High-Energy Astrophysics NASA/Goddard Space Flight Center 8800 Greenbelt Rd., Mail Code 662 Greenbelt, MD 20771 (w) 301-286-2531 (m) 781-708-5004 john.zuhone@nasa.gov jzuhone@gmail.com
On Jan 29, 2014, at 5:22 PM, j s oishi <jsoishi@gmail.com> wrote:
Hi all,
I've adapted yt's gdf_writer.py into a standalone class called pygdf (https://bitbucket.org/jsoishi/pygdf). I've done this so that any python code can now save data in the gdf format.
While doing this, I came across something in the yt gdf frontend that I'm not quite sure I understand. In the grid metadata, yt expects /grid_particle_count to be a 2-D array, but the gdf standard clearly states this should be a 1-D (int64, N) array, where N is the number of grids. Further complicating the issue is the fact that if I create a 1-D array for /grid_particle_count, the failure point in the following script
from yt.mods import * pf = load("/tmp/blah.gdf")
sp = SlicePlot(pf, 2, "Density")
is actually in data_objects/grid_patch.py, not in the gdf frontend itself. The file /tmp/blah.gdf can be generated by running the test.py script in pygdf.
The obvious workaround is to simply make /grid_particle_count a 2D array (this is especially so since I'm not actually using particles at the moment), but I'm not sure why this should be so. The total count of particles in N grids seems to be better fit in a 1-D array to me. However, I'd like to understand better what's going on. Any pointers or clarification would be very helpful.
Also, any feedback on pygdf would be most welcome!
thanks,
j
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Interesting!! I will have a look over this, I wrote a parallel compatible version of this code a while back I will try and get it working within your module. Stuart On 29 January 2014 22:22, j s oishi <jsoishi@gmail.com> wrote:
Hi all,
I've adapted yt's gdf_writer.py into a standalone class called pygdf (https://bitbucket.org/jsoishi/pygdf). I've done this so that any python code can now save data in the gdf format.
While doing this, I came across something in the yt gdf frontend that I'm not quite sure I understand. In the grid metadata, yt expects /grid_particle_count to be a 2-D array, but the gdf standard clearly states this should be a 1-D (int64, N) array, where N is the number of grids. Further complicating the issue is the fact that if I create a 1-D array for /grid_particle_count, the failure point in the following script
from yt.mods import * pf = load("/tmp/blah.gdf")
sp = SlicePlot(pf, 2, "Density")
is actually in data_objects/grid_patch.py, not in the gdf frontend itself. The file /tmp/blah.gdf can be generated by running the test.py script in pygdf.
The obvious workaround is to simply make /grid_particle_count a 2D array (this is especially so since I'm not actually using particles at the moment), but I'm not sure why this should be so. The total count of particles in N grids seems to be better fit in a 1-D array to me. However, I'd like to understand better what's going on. Any pointers or clarification would be very helpful.
Also, any feedback on pygdf would be most welcome!
thanks,
j
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Stuart, Wow, a parallel gdf writer? I was planning to add that sometime next week or so. I'd like to know how you approached it. I was simply thinking of having a flag to only write a data-only "sidecar" file that would use HDF5 links in the master file for grids on other cores. Please let me know if I can help integrate what you've done into this framework. Jeff On Thu, Jan 30, 2014 at 11:09 AM, Stuart Mumford <stuart@mumford.me.uk>wrote:
Interesting!!
I will have a look over this, I wrote a parallel compatible version of this code a while back I will try and get it working within your module.
Stuart
On 29 January 2014 22:22, j s oishi <jsoishi@gmail.com> wrote:
Hi all,
I've adapted yt's gdf_writer.py into a standalone class called pygdf (https://bitbucket.org/jsoishi/pygdf). I've done this so that any python code can now save data in the gdf format.
While doing this, I came across something in the yt gdf frontend that I'm not quite sure I understand. In the grid metadata, yt expects /grid_particle_count to be a 2-D array, but the gdf standard clearly states this should be a 1-D (int64, N) array, where N is the number of grids. Further complicating the issue is the fact that if I create a 1-D array for /grid_particle_count, the failure point in the following script
from yt.mods import * pf = load("/tmp/blah.gdf")
sp = SlicePlot(pf, 2, "Density")
is actually in data_objects/grid_patch.py, not in the gdf frontend itself. The file /tmp/blah.gdf can be generated by running the test.py script in pygdf.
The obvious workaround is to simply make /grid_particle_count a 2D array (this is especially so since I'm not actually using particles at the moment), but I'm not sure why this should be so. The total count of particles in N grids seems to be better fit in a 1-D array to me. However, I'd like to understand better what's going on. Any pointers or clarification would be very helpful.
Also, any feedback on pygdf would be most welcome!
thanks,
j
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (4)
-
j s oishi -
John ZuHone -
Matthew Turk -
Stuart Mumford