Need help reading in the multiple particle hdf5 data from the FLASH code.
Hi, I referred to the relevant part of the documentation about FLASH particle data on, http://yt-project.org/doc/analyzing/particle_trajectories.html and managed to post process a series of data files that have just 1 single particle in it.
When I used multiple particles in the simulation, at the step where we do the following,
ds = yt.load(my_fns[0]) dd = ds.all_data() indices = dd["particle_index"].astype("int")
I got the following error:
~/.local/lib/python3.6/site-packages/yt/frontends/flash/io.py in <listcomp>(.0) 171 self._particle_fields = determine_particle_fields(self._handle) 172 self._position_fields = [self._particle_fields["particle_pos%s" % ax] --> 173 for ax in 'xyz'] 174 self._chunksize = 32**3 175
KeyError: 'particle_posx'
I am new to YT in general and facing a difficulty in understanding what's the problem. As per the documentation, the syntax should work for multiple particles. Do you think that it could be FLASH issue? What are the ways you would go about debugging it? Thanks.
Hi,
What version of FLASH is your file from?
Also, do you have a file that we could test that has this problem? You can upload files this way:
https://docs.hub.yt/services.html#curldrop
Best,
John
Dr. John ZuHone Astrophysicist, Chandra X-ray Center Center for Astrophysics | Harvard & Smithsonian
Office: (617) 496 1816 | Cell: (781) 708 5004 60 Garden Street | MS 03 | Cambridge, MA 02138 john.zuhone@cfa.harvard.edu http://hea-www.cfa.harvard.edu/~jzuhone
On Dec 15, 2018, at 1:38 PM, saurabhisreachable@gmail.com wrote:
Hi, I referred to the relevant part of the documentation about FLASH particle data on, http://yt-project.org/doc/analyzing/particle_trajectories.html and managed to post process a series of data files that have just 1 single particle in it.
When I used multiple particles in the simulation, at the step where we do the following,
ds = yt.load(my_fns[0]) dd = ds.all_data() indices = dd["particle_index"].astype("int")
I got the following error:
~/.local/lib/python3.6/site-packages/yt/frontends/flash/io.py in <listcomp>(.0) 171 self._particle_fields = determine_particle_fields(self._handle) 172 self._position_fields = [self._particle_fields["particle_pos%s" % ax] --> 173 for ax in 'xyz'] 174 self._chunksize = 32**3 175
KeyError: 'particle_posx'
I am new to YT in general and facing a difficulty in understanding what's the problem. As per the documentation, the syntax should work for multiple particles. Do you think that it could be FLASH issue? What are the ways you would go about debugging it? Thanks. _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
Hello there,
What is my_fns pointing to? When you want to load a FLASH particle set in yt, there are two methods.
1. If there is both a plot and particle file with the same number (i.e. flash_hdf5_plt_cnt_0000 and flash_hdf5_part_0000) together in the directory, and you load the plot file, yt is smart enough to grab the particle data at the same time (it silently loads the particle file for you). Of course if you load a checkpoint, you also get both. 2. Just load the particle file directly using yt.load("/path/to/data/flash_hdf5_part_0000"). Note when using this method, some information that is only stored in the plot file won't be available for you to get to.
Joshua Wall
Doctoral Candidate Department of Physics Drexel University
On Sat, Dec 15, 2018 at 1:38 PM saurabhisreachable@gmail.com wrote:
Hi, I referred to the relevant part of the documentation about FLASH particle data on, http://yt-project.org/doc/analyzing/particle_trajectories.html and managed to post process a series of data files that have just 1 single particle in it.
When I used multiple particles in the simulation, at the step where we do the following,
ds = yt.load(my_fns[0]) dd = ds.all_data() indices = dd["particle_index"].astype("int")
I got the following error:
~/.local/lib/python3.6/site-packages/yt/frontends/flash/io.py in <listcomp>(.0) 171 self._particle_fields = determine_particle_fields(self._handle) 172 self._position_fields = [self._particle_fields["particle_pos%s" % ax] --> 173 for ax in 'xyz'] 174 self._chunksize = 32**3 175
KeyError: 'particle_posx'
I am new to YT in general and facing a difficulty in understanding what's the problem. As per the documentation, the syntax should work for multiple particles. Do you think that it could be FLASH issue? What are the ways you would go about debugging it? Thanks. _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
One more thing occurs to me (I myself run FLASH with multiple particles and analyze the data with FLASH)... don't forget if you are creating particles during the simulation (i.e. using the sink particles, which form when the Jeans criterion are met) that even though there are particle files early in the run, there aren't actually any particles in there, and yt will throw exactly this error if you try and access any particle property.
To get around this when I process thousands of files with yt, I do the following check in my loop:
# Test for the presence of particles. If none, jump to else if (getattr(ds.fields, 'io', None) != None): Do fun yt analysis with particles else: Do other analysis without particles, or even simply continue to the next loop.
Joshua Wall
Doctoral Candidate Department of Physics Drexel University
On Sat, Dec 15, 2018 at 1:52 PM Joshua Wall joshua.e.wall@gmail.com wrote:
Hello there,
What is my_fns pointing to? When you want to load a FLASH particle setin yt, there are two methods.
- If there is both a plot and particle file with the same number (i.e.
flash_hdf5_plt_cnt_0000 and flash_hdf5_part_0000) together in the directory, and you load the plot file, yt is smart enough to grab the particle data at the same time (it silently loads the particle file for you). Of course if you load a checkpoint, you also get both. 2. Just load the particle file directly using yt.load("/path/to/data/flash_hdf5_part_0000"). Note when using this method, some information that is only stored in the plot file won't be available for you to get to.
Joshua Wall
Doctoral Candidate Department of Physics Drexel University
On Sat, Dec 15, 2018 at 1:38 PM saurabhisreachable@gmail.com wrote:
Hi, I referred to the relevant part of the documentation about FLASH particle data on, http://yt-project.org/doc/analyzing/particle_trajectories.html and managed to post process a series of data files that have just 1 single particle in it.
When I used multiple particles in the simulation, at the step where we do the following,
ds = yt.load(my_fns[0]) dd = ds.all_data() indices = dd["particle_index"].astype("int")
I got the following error:
~/.local/lib/python3.6/site-packages/yt/frontends/flash/io.py in <listcomp>(.0) 171 self._particle_fields = determine_particle_fields(self._handle) 172 self._position_fields = [self._particle_fields["particle_pos%s" % ax] --> 173 for ax in 'xyz'] 174 self._chunksize = 32**3 175
KeyError: 'particle_posx'
I am new to YT in general and facing a difficulty in understanding what's the problem. As per the documentation, the syntax should work for multiple particles. Do you think that it could be FLASH issue? What are the ways you would go about debugging it? Thanks. _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
Hi Josh,
This is a good point. I am fairly(?) sure that we accounted for this in the design of ParticleTrajectories, but I could be wrong.
Sent from my iPad
On Dec 15, 2018, at 2:45 PM, Joshua Wall joshua.e.wall@gmail.com wrote:
One more thing occurs to me (I myself run FLASH with multiple particles and analyze the data with FLASH)... don't forget if you are creating particles during the simulation (i.e. using the sink particles, which form when the Jeans criterion are met) that even though there are particle files early in the run, there aren't actually any particles in there, and yt will throw exactly this error if you try and access any particle property.
To get around this when I process thousands of files with yt, I do the following check in my loop:
# Test for the presence of particles. If none, jump to else if (getattr(ds.fields, 'io', None) != None): Do fun yt analysis with particles else: Do other analysis without particles, or even simply continue to the next loop.
Joshua Wall
Doctoral Candidate Department of Physics Drexel University
On Sat, Dec 15, 2018 at 1:52 PM Joshua Wall joshua.e.wall@gmail.com wrote:
Hello there,
What is my_fns pointing to? When you want to load a FLASH particle set in yt, there are two methods.
- If there is both a plot and particle file with the same number (i.e. flash_hdf5_plt_cnt_0000 and flash_hdf5_part_0000) together in the directory, and you load the plot file, yt is smart enough to grab the particle data at the same time (it silently loads the particle file for you). Of course if you load a checkpoint, you also get both.
- Just load the particle file directly using yt.load("/path/to/data/flash_hdf5_part_0000"). Note when using this method, some information that is only stored in the plot file won't be available for you to get to.
Joshua Wall
Doctoral Candidate Department of Physics Drexel University
On Sat, Dec 15, 2018 at 1:38 PM saurabhisreachable@gmail.com wrote: Hi, I referred to the relevant part of the documentation about FLASH particle data on, http://yt-project.org/doc/analyzing/particle_trajectories.html and managed to post process a series of data files that have just 1 single particle in it.
When I used multiple particles in the simulation, at the step where we do the following,
ds = yt.load(my_fns[0]) dd = ds.all_data() indices = dd["particle_index"].astype("int")
I got the following error:
~/.local/lib/python3.6/site-packages/yt/frontends/flash/io.py in <listcomp>(.0) 171 self._particle_fields = determine_particle_fields(self._handle) 172 self._position_fields = [self._particle_fields["particle_pos%s" % ax] --> 173 for ax in 'xyz'] 174 self._chunksize = 32**3 175
KeyError: 'particle_posx'
I am new to YT in general and facing a difficulty in understanding what's the problem. As per the documentation, the syntax should work for multiple particles. Do you think that it could be FLASH issue? What are the ways you would go about debugging it? Thanks. _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
participants (3)
-
John ZuHone -
Joshua Wall -
saurabhisreachable＠gmail.com