Hello, I've looked at the documentation and it seems that octree datasets are loadable for a few specific codes, but not in general. I have an octree that represents galaxy simulation data, but it's produced as an output of an analysis script. It would be nice to take advantage of yt's capabilities to do analysis. Is there a simple way to load an generic octree or would I have to write a front-end myself (as described in the documentation and by looking at ART, for example)? Thanks! Paco Holguin
Dear Paco, You may be able to achieve what you want using `yt.load_octree` (see https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...), though I never tried it myself. The function expects the octs to be fed in a depth-first manner (see https://en.wikipedia.org/wiki/Depth-first_search). Out of curiosity, what format has the data you are trying to load? Cheers, Corentin On 26/09/2020 21:14, Paco Holguin wrote:
Hello,
I've looked at the documentation and it seems that octree datasets are loadable for a few specific codes, but not in general. I have an octree that represents galaxy simulation data, but it's produced as an output of an analysis script. It would be nice to take advantage of yt's capabilities to do analysis.
Is there a simple way to load an generic octree or would I have to write a front-end myself (as described in the documentation and by looking at ART, for example)?
Thanks!
Paco Holguin
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: contact@cphyc.me
Thanks for the link. I'll take a look at how the data is structured and try the "load_octree" function. The data is in hdf5 format. On Sun, Sep 27, 2020 at 5:49 AM Corentin CADIOU <contact@cphyc.me> wrote:
Dear Paco,
You may be able to achieve what you want using `yt.load_octree` (see https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...), though I never tried it myself.
The function expects the octs to be fed in a depth-first manner (see https://en.wikipedia.org/wiki/Depth-first_search).
Out of curiosity, what format has the data you are trying to load?
Cheers, Corentin On 26/09/2020 21:14, Paco Holguin wrote:
Hello,
I've looked at the documentation and it seems that octree datasets are loadable for a few specific codes, but not in general. I have an octree that represents galaxy simulation data, but it's produced as an output of an analysis script. It would be nice to take advantage of yt's capabilities to do analysis.
Is there a simple way to load an generic octree or would I have to write a front-end myself (as described in the documentation and by looking at ART, for example)?
Thanks!
Paco Holguin
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.orghttps://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: contact@cphyc.me
I tried following the link: https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures... The "ds" object is created successfully, and fields such as 'y' seem to output correctly. However, when I try to access "gas","density" I get the following error, which occurs even if I create some random particle data and include it in the list of "quantities" Traceback (most recent call last): File "load_tree.py", line 83, in <module> ad['gas','density'] File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__ self.get_data(f) File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 1331, in get_data particles, self, self._current_chunk) File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 227, in _read_particle_fields fields_to_read) File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 199, in _read_particle_selection psize = self._count_particles_chunks(chunks, ptf, selector) File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 170, in _count_particles_chunks for ptype, (x, y, z) in self._read_particle_coords(chunks, ptf): AttributeError: 'IOHandlerStreamOctree' object has no attribute '_read_particle_coords' I tried looking at "data_containers.py" but I wasn't able to figure out where to look from there. I'm wondering if I could remove the particle part of this line, or something similar. # The _read method will figure out which fields it needs to get from # disk, and return a dict of those fields along with the fields that # need to be generated. read_fluids, gen_fluids = self.index._read_fluid_fields( fluids, self, self._current_chunk) for f, v in read_fluids.items(): self.field_data[f] = self.ds.arr(v, input_units = finfos[f].units) self.field_data[f].convert_to_units(finfos[f].output_units) read_particles, gen_particles = self.index._read_particle_fields( particles, self, self._current_chunk) I would appreciate any insight into what is going on. Thanks! On Sun, Sep 27, 2020 at 8:10 PM Paco Holguin <opaco450@gmail.com> wrote:
Thanks for the link. I'll take a look at how the data is structured and try the "load_octree" function. The data is in hdf5 format.
On Sun, Sep 27, 2020 at 5:49 AM Corentin CADIOU <contact@cphyc.me> wrote:
Dear Paco,
You may be able to achieve what you want using `yt.load_octree` (see https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...), though I never tried it myself.
The function expects the octs to be fed in a depth-first manner (see https://en.wikipedia.org/wiki/Depth-first_search).
Out of curiosity, what format has the data you are trying to load?
Cheers, Corentin On 26/09/2020 21:14, Paco Holguin wrote:
Hello,
I've looked at the documentation and it seems that octree datasets are loadable for a few specific codes, but not in general. I have an octree that represents galaxy simulation data, but it's produced as an output of an analysis script. It would be nice to take advantage of yt's capabilities to do analysis.
Is there a simple way to load an generic octree or would I have to write a front-end myself (as described in the documentation and by looking at ART, for example)?
Thanks!
Paco Holguin
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.orghttps://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: contact@cphyc.me
Hi Paco, What is the command you use to load the octree, specifically? i.e., what you feed in to load_octree ? On Mon, Sep 28, 2020 at 2:56 PM Paco Holguin <opaco450@gmail.com> wrote:
I tried following the link: https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...
The "ds" object is created successfully, and fields such as 'y' seem to output correctly. However, when I try to access "gas","density" I get the following error, which occurs even if I create some random particle data and include it in the list of "quantities"
Traceback (most recent call last):
File "load_tree.py", line 83, in <module>
ad['gas','density']
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__
self.get_data(f)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 1331, in get_data
particles, self, self._current_chunk)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 227, in _read_particle_fields
fields_to_read)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 199, in _read_particle_selection
psize = self._count_particles_chunks(chunks, ptf, selector)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 170, in _count_particles_chunks
for ptype, (x, y, z) in self._read_particle_coords(chunks, ptf):
AttributeError: 'IOHandlerStreamOctree' object has no attribute '_read_particle_coords'
I tried looking at "data_containers.py" but I wasn't able to figure out where to look from there. I'm wondering if I could remove the particle part of this line, or something similar.
# The _read method will figure out which fields it needs to get from
# disk, and return a dict of those fields along with the fields that
# need to be generated.
read_fluids, gen_fluids = self.index._read_fluid_fields(
fluids, self, self._current_chunk)
for f, v in read_fluids.items():
self.field_data[f] = self.ds.arr(v, input_units = finfos[f].units)
self.field_data[f].convert_to_units(finfos[f].output_units)
read_particles, gen_particles = self.index._read_particle_fields(
particles, self, self._current_chunk)
I would appreciate any insight into what is going on.
Thanks!
On Sun, Sep 27, 2020 at 8:10 PM Paco Holguin <opaco450@gmail.com> wrote:
Thanks for the link. I'll take a look at how the data is structured and try the "load_octree" function. The data is in hdf5 format.
On Sun, Sep 27, 2020 at 5:49 AM Corentin CADIOU <contact@cphyc.me> wrote:
Dear Paco,
You may be able to achieve what you want using `yt.load_octree` (see https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...), though I never tried it myself.
The function expects the octs to be fed in a depth-first manner (see https://en.wikipedia.org/wiki/Depth-first_search).
Out of curiosity, what format has the data you are trying to load?
Cheers, Corentin On 26/09/2020 21:14, Paco Holguin wrote:
Hello,
I've looked at the documentation and it seems that octree datasets are loadable for a few specific codes, but not in general. I have an octree that represents galaxy simulation data, but it's produced as an output of an analysis script. It would be nice to take advantage of yt's capabilities to do analysis.
Is there a simple way to load an generic octree or would I have to write a front-end myself (as described in the documentation and by looking at ART, for example)?
Thanks!
Paco Holguin
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.orghttps://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: contact@cphyc.me
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
I loaded the data with h5py, as "f" and then "density = np.array( f['rho'], dtype='f8') and "oct_mask" in a similar way, with the array containing 0's for leaf cells. From there: quantities = {} quantities['gas', 'density'] = density ds = yt.load_octree(oct_mask, data = quantities, over_refine_facotor = 0, partial_coverage=1) On Mon, Sep 28, 2020 at 3:58 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco,
What is the command you use to load the octree, specifically? i.e., what you feed in to load_octree ?
On Mon, Sep 28, 2020 at 2:56 PM Paco Holguin <opaco450@gmail.com> wrote:
I tried following the link: https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...
The "ds" object is created successfully, and fields such as 'y' seem to output correctly. However, when I try to access "gas","density" I get the following error, which occurs even if I create some random particle data and include it in the list of "quantities"
Traceback (most recent call last):
File "load_tree.py", line 83, in <module>
ad['gas','density']
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__
self.get_data(f)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 1331, in get_data
particles, self, self._current_chunk)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 227, in _read_particle_fields
fields_to_read)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 199, in _read_particle_selection
psize = self._count_particles_chunks(chunks, ptf, selector)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 170, in _count_particles_chunks
for ptype, (x, y, z) in self._read_particle_coords(chunks, ptf):
AttributeError: 'IOHandlerStreamOctree' object has no attribute '_read_particle_coords'
I tried looking at "data_containers.py" but I wasn't able to figure out where to look from there. I'm wondering if I could remove the particle part of this line, or something similar.
# The _read method will figure out which fields it needs to get from
# disk, and return a dict of those fields along with the fields that
# need to be generated.
read_fluids, gen_fluids = self.index._read_fluid_fields(
fluids, self, self._current_chunk)
for f, v in read_fluids.items():
self.field_data[f] = self.ds.arr(v, input_units = finfos[f].units)
self.field_data[f].convert_to_units(finfos[f].output_units)
read_particles, gen_particles = self.index._read_particle_fields(
particles, self, self._current_chunk)
I would appreciate any insight into what is going on.
Thanks!
On Sun, Sep 27, 2020 at 8:10 PM Paco Holguin <opaco450@gmail.com> wrote:
Thanks for the link. I'll take a look at how the data is structured and try the "load_octree" function. The data is in hdf5 format.
On Sun, Sep 27, 2020 at 5:49 AM Corentin CADIOU <contact@cphyc.me> wrote:
Dear Paco,
You may be able to achieve what you want using `yt.load_octree` (see https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...), though I never tried it myself.
The function expects the octs to be fed in a depth-first manner (see https://en.wikipedia.org/wiki/Depth-first_search).
Out of curiosity, what format has the data you are trying to load?
Cheers, Corentin On 26/09/2020 21:14, Paco Holguin wrote:
Hello,
I've looked at the documentation and it seems that octree datasets are loadable for a few specific codes, but not in general. I have an octree that represents galaxy simulation data, but it's produced as an output of an analysis script. It would be nice to take advantage of yt's capabilities to do analysis.
Is there a simple way to load an generic octree or would I have to write a front-end myself (as described in the documentation and by looking at ART, for example)?
Thanks!
Paco Holguin
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.orghttps://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: contact@cphyc.me
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
Hi Paco, OK, this does look like what I would have expected. What is likely happening, I *think*, is that it's getting "gas","density" marked as a particle field rather than a fluid field. What happens if you change it so that instead of being "gas","density" you are either "stream","density" or just "density"? On Mon, Sep 28, 2020 at 3:12 PM Paco Holguin <opaco450@gmail.com> wrote:
I loaded the data with h5py, as "f" and then "density = np.array( f['rho'], dtype='f8') and "oct_mask" in a similar way, with the array containing 0's for leaf cells.
From there:
quantities = {} quantities['gas', 'density'] = density ds = yt.load_octree(oct_mask, data = quantities, over_refine_facotor = 0, partial_coverage=1)
On Mon, Sep 28, 2020 at 3:58 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco,
What is the command you use to load the octree, specifically? i.e., what you feed in to load_octree ?
On Mon, Sep 28, 2020 at 2:56 PM Paco Holguin <opaco450@gmail.com> wrote:
I tried following the link: https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...
The "ds" object is created successfully, and fields such as 'y' seem to output correctly. However, when I try to access "gas","density" I get the following error, which occurs even if I create some random particle data and include it in the list of "quantities"
Traceback (most recent call last):
File "load_tree.py", line 83, in <module>
ad['gas','density']
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__
self.get_data(f)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 1331, in get_data
particles, self, self._current_chunk)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 227, in _read_particle_fields
fields_to_read)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 199, in _read_particle_selection
psize = self._count_particles_chunks(chunks, ptf, selector)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 170, in _count_particles_chunks
for ptype, (x, y, z) in self._read_particle_coords(chunks, ptf):
AttributeError: 'IOHandlerStreamOctree' object has no attribute '_read_particle_coords'
I tried looking at "data_containers.py" but I wasn't able to figure out where to look from there. I'm wondering if I could remove the particle part of this line, or something similar.
# The _read method will figure out which fields it needs to get from
# disk, and return a dict of those fields along with the fields that
# need to be generated.
read_fluids, gen_fluids = self.index._read_fluid_fields(
fluids, self, self._current_chunk)
for f, v in read_fluids.items():
self.field_data[f] = self.ds.arr(v, input_units = finfos[f].units)
self.field_data[f].convert_to_units(finfos[f].output_units)
read_particles, gen_particles = self.index._read_particle_fields(
particles, self, self._current_chunk)
I would appreciate any insight into what is going on.
Thanks!
On Sun, Sep 27, 2020 at 8:10 PM Paco Holguin <opaco450@gmail.com> wrote:
Thanks for the link. I'll take a look at how the data is structured and try the "load_octree" function. The data is in hdf5 format.
On Sun, Sep 27, 2020 at 5:49 AM Corentin CADIOU <contact@cphyc.me> wrote:
Dear Paco,
You may be able to achieve what you want using `yt.load_octree` (see https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...), though I never tried it myself.
The function expects the octs to be fed in a depth-first manner (see https://en.wikipedia.org/wiki/Depth-first_search).
Out of curiosity, what format has the data you are trying to load?
Cheers, Corentin On 26/09/2020 21:14, Paco Holguin wrote:
Hello,
I've looked at the documentation and it seems that octree datasets are loadable for a few specific codes, but not in general. I have an octree that represents galaxy simulation data, but it's produced as an output of an analysis script. It would be nice to take advantage of yt's capabilities to do analysis.
Is there a simple way to load an generic octree or would I have to write a front-end myself (as described in the documentation and by looking at ART, for example)?
Thanks!
Paco Holguin
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.orghttps://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: contact@cphyc.me
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
Trying "stream, "density" or just "density" results in the same error. When I change the quantities dictionary to just "density" the field appears as 'io', 'density' which seems to indicate that it's being parked as a particle field as you said. On Mon, Sep 28, 2020 at 4:18 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco,
OK, this does look like what I would have expected. What is likely happening, I *think*, is that it's getting "gas","density" marked as a particle field rather than a fluid field.
What happens if you change it so that instead of being "gas","density" you are either "stream","density" or just "density"?
On Mon, Sep 28, 2020 at 3:12 PM Paco Holguin <opaco450@gmail.com> wrote:
I loaded the data with h5py, as "f" and then "density = np.array( f['rho'], dtype='f8') and "oct_mask" in a similar way, with the array containing 0's for leaf cells.
From there:
quantities = {} quantities['gas', 'density'] = density ds = yt.load_octree(oct_mask, data = quantities, over_refine_facotor = 0, partial_coverage=1)
On Mon, Sep 28, 2020 at 3:58 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco,
What is the command you use to load the octree, specifically? i.e., what you feed in to load_octree ?
On Mon, Sep 28, 2020 at 2:56 PM Paco Holguin <opaco450@gmail.com> wrote:
I tried following the link: https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...
The "ds" object is created successfully, and fields such as 'y' seem to output correctly. However, when I try to access "gas","density" I get the following error, which occurs even if I create some random particle data and include it in the list of "quantities"
Traceback (most recent call last):
File "load_tree.py", line 83, in <module>
ad['gas','density']
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__
self.get_data(f)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 1331, in get_data
particles, self, self._current_chunk)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 227, in _read_particle_fields
fields_to_read)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 199, in _read_particle_selection
psize = self._count_particles_chunks(chunks, ptf, selector)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 170, in _count_particles_chunks
for ptype, (x, y, z) in self._read_particle_coords(chunks, ptf):
AttributeError: 'IOHandlerStreamOctree' object has no attribute '_read_particle_coords'
I tried looking at "data_containers.py" but I wasn't able to figure out where to look from there. I'm wondering if I could remove the particle part of this line, or something similar.
# The _read method will figure out which fields it needs to get from
# disk, and return a dict of those fields along with the fields that
# need to be generated.
read_fluids, gen_fluids = self.index._read_fluid_fields(
fluids, self, self._current_chunk)
for f, v in read_fluids.items():
self.field_data[f] = self.ds.arr(v, input_units = finfos[f].units)
self.field_data[f].convert_to_units(finfos[f].output_units)
read_particles, gen_particles = self.index._read_particle_fields(
particles, self, self._current_chunk)
I would appreciate any insight into what is going on.
Thanks!
On Sun, Sep 27, 2020 at 8:10 PM Paco Holguin <opaco450@gmail.com> wrote:
Thanks for the link. I'll take a look at how the data is structured and try the "load_octree" function. The data is in hdf5 format.
On Sun, Sep 27, 2020 at 5:49 AM Corentin CADIOU <contact@cphyc.me> wrote:
Dear Paco,
You may be able to achieve what you want using `yt.load_octree` (see https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...), though I never tried it myself.
The function expects the octs to be fed in a depth-first manner (see https://en.wikipedia.org/wiki/Depth-first_search).
Out of curiosity, what format has the data you are trying to load?
Cheers, Corentin On 26/09/2020 21:14, Paco Holguin wrote:
Hello,
I've looked at the documentation and it seems that octree datasets are loadable for a few specific codes, but not in general. I have an octree that represents galaxy simulation data, but it's produced as an output of an analysis script. It would be nice to take advantage of yt's capabilities to do analysis.
Is there a simple way to load an generic octree or would I have to write a front-end myself (as described in the documentation and by looking at ART, for example)?
Thanks!
Paco Holguin
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.orghttps://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: contact@cphyc.me
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
I think I found the solution. Indeed, the data was being marked as a particle field because 'load_octree' calls 'set_particle_types' on the data, and if len(data[key].shape) == 1, it gets marked as a particle type. For the density array that I pass to the quantities dictionary, I added an empty dimension via np.newaxis, and that seems to set the field to the correct type. On Mon, Sep 28, 2020 at 4:59 PM Paco Holguin <opaco450@gmail.com> wrote:
Trying "stream, "density" or just "density" results in the same error. When I change the quantities dictionary to just "density" the field appears as 'io', 'density' which seems to indicate that it's being parked as a particle field as you said.
On Mon, Sep 28, 2020 at 4:18 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco,
OK, this does look like what I would have expected. What is likely happening, I *think*, is that it's getting "gas","density" marked as a particle field rather than a fluid field.
What happens if you change it so that instead of being "gas","density" you are either "stream","density" or just "density"?
On Mon, Sep 28, 2020 at 3:12 PM Paco Holguin <opaco450@gmail.com> wrote:
I loaded the data with h5py, as "f" and then "density = np.array( f['rho'], dtype='f8') and "oct_mask" in a similar way, with the array containing 0's for leaf cells.
From there:
quantities = {} quantities['gas', 'density'] = density ds = yt.load_octree(oct_mask, data = quantities, over_refine_facotor = 0, partial_coverage=1)
On Mon, Sep 28, 2020 at 3:58 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco,
What is the command you use to load the octree, specifically? i.e., what you feed in to load_octree ?
On Mon, Sep 28, 2020 at 2:56 PM Paco Holguin <opaco450@gmail.com> wrote:
I tried following the link: https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...
The "ds" object is created successfully, and fields such as 'y' seem to output correctly. However, when I try to access "gas","density" I get the following error, which occurs even if I create some random particle data and include it in the list of "quantities"
Traceback (most recent call last):
File "load_tree.py", line 83, in <module>
ad['gas','density']
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__
self.get_data(f)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 1331, in get_data
particles, self, self._current_chunk)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 227, in _read_particle_fields
fields_to_read)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 199, in _read_particle_selection
psize = self._count_particles_chunks(chunks, ptf, selector)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 170, in _count_particles_chunks
for ptype, (x, y, z) in self._read_particle_coords(chunks, ptf):
AttributeError: 'IOHandlerStreamOctree' object has no attribute '_read_particle_coords'
I tried looking at "data_containers.py" but I wasn't able to figure out where to look from there. I'm wondering if I could remove the particle part of this line, or something similar.
# The _read method will figure out which fields it needs to get from
# disk, and return a dict of those fields along with the fields that
# need to be generated.
read_fluids, gen_fluids = self.index._read_fluid_fields(
fluids, self, self._current_chunk)
for f, v in read_fluids.items():
self.field_data[f] = self.ds.arr(v, input_units = finfos[f].units)
self.field_data[f].convert_to_units(finfos[f].output_units)
read_particles, gen_particles = self.index._read_particle_fields(
particles, self, self._current_chunk)
I would appreciate any insight into what is going on.
Thanks!
On Sun, Sep 27, 2020 at 8:10 PM Paco Holguin <opaco450@gmail.com> wrote:
Thanks for the link. I'll take a look at how the data is structured and try the "load_octree" function. The data is in hdf5 format.
On Sun, Sep 27, 2020 at 5:49 AM Corentin CADIOU <contact@cphyc.me> wrote:
> Dear Paco, > > You may be able to achieve what you want using `yt.load_octree` (see > https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...), > though I never tried it myself. > > The function expects the octs to be fed in a depth-first manner (see > https://en.wikipedia.org/wiki/Depth-first_search). > > Out of curiosity, what format has the data you are trying to load? > > Cheers, > Corentin > On 26/09/2020 21:14, Paco Holguin wrote: > > Hello, > > I've looked at the documentation and it seems that octree datasets > are loadable for a few specific codes, but not in general. I have an > octree that represents galaxy simulation data, but it's produced as an > output of an analysis script. It would be nice to take advantage of yt's > capabilities to do analysis. > > Is there a simple way to load an generic octree or would I have to > write a front-end myself (as described in the documentation and by looking > at ART, for example)? > > Thanks! > > > Paco Holguin > > _______________________________________________ > yt-users mailing list -- yt-users@python.org > To unsubscribe send an email to yt-users-leave@python.orghttps://mail.python.org/mailman3/lists/yt-users.python.org/ > Member address: contact@cphyc.me > > _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
Hi Paco, sorry for not replying sooner! I am happy that seemed to work for you. Can you share if things were problematic or not afterward? On Sun, Oct 11, 2020 at 4:28 PM Paco Holguin <opaco450@gmail.com> wrote:
I think I found the solution. Indeed, the data was being marked as a particle field because 'load_octree' calls 'set_particle_types' on the data, and if len(data[key].shape) == 1, it gets marked as a particle type.
For the density array that I pass to the quantities dictionary, I added an empty dimension via np.newaxis, and that seems to set the field to the correct type.
On Mon, Sep 28, 2020 at 4:59 PM Paco Holguin <opaco450@gmail.com> wrote:
Trying "stream, "density" or just "density" results in the same error. When I change the quantities dictionary to just "density" the field appears as 'io', 'density' which seems to indicate that it's being parked as a particle field as you said.
On Mon, Sep 28, 2020 at 4:18 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco,
OK, this does look like what I would have expected. What is likely happening, I *think*, is that it's getting "gas","density" marked as a particle field rather than a fluid field.
What happens if you change it so that instead of being "gas","density" you are either "stream","density" or just "density"?
On Mon, Sep 28, 2020 at 3:12 PM Paco Holguin <opaco450@gmail.com> wrote:
I loaded the data with h5py, as "f" and then "density = np.array( f['rho'], dtype='f8') and "oct_mask" in a similar way, with the array containing 0's for leaf cells.
From there:
quantities = {} quantities['gas', 'density'] = density ds = yt.load_octree(oct_mask, data = quantities, over_refine_facotor = 0, partial_coverage=1)
On Mon, Sep 28, 2020 at 3:58 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco,
What is the command you use to load the octree, specifically? i.e., what you feed in to load_octree ?
On Mon, Sep 28, 2020 at 2:56 PM Paco Holguin <opaco450@gmail.com> wrote:
I tried following the link: https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...
The "ds" object is created successfully, and fields such as 'y' seem to output correctly. However, when I try to access "gas","density" I get the following error, which occurs even if I create some random particle data and include it in the list of "quantities"
Traceback (most recent call last):
File "load_tree.py", line 83, in <module>
ad['gas','density']
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__
self.get_data(f)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 1331, in get_data
particles, self, self._current_chunk)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 227, in _read_particle_fields
fields_to_read)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 199, in _read_particle_selection
psize = self._count_particles_chunks(chunks, ptf, selector)
File "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", line 170, in _count_particles_chunks
for ptype, (x, y, z) in self._read_particle_coords(chunks, ptf):
AttributeError: 'IOHandlerStreamOctree' object has no attribute '_read_particle_coords'
I tried looking at "data_containers.py" but I wasn't able to figure out where to look from there. I'm wondering if I could remove the particle part of this line, or something similar.
# The _read method will figure out which fields it needs to get from
# disk, and return a dict of those fields along with the fields that
# need to be generated.
read_fluids, gen_fluids = self.index._read_fluid_fields(
fluids, self, self._current_chunk)
for f, v in read_fluids.items():
self.field_data[f] = self.ds.arr(v, input_units = finfos[f].units)
self.field_data[f].convert_to_units(finfos[f].output_units)
read_particles, gen_particles = self.index._read_particle_fields(
particles, self, self._current_chunk)
I would appreciate any insight into what is going on.
Thanks!
On Sun, Sep 27, 2020 at 8:10 PM Paco Holguin <opaco450@gmail.com> wrote:
> Thanks for the link. I'll take a look at how the data is structured > and try the "load_octree" function. The data is in hdf5 format. > > On Sun, Sep 27, 2020 at 5:49 AM Corentin CADIOU <contact@cphyc.me> > wrote: > >> Dear Paco, >> >> You may be able to achieve what you want using `yt.load_octree` >> (see >> https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...), >> though I never tried it myself. >> >> The function expects the octs to be fed in a depth-first manner >> (see https://en.wikipedia.org/wiki/Depth-first_search). >> >> Out of curiosity, what format has the data you are trying to load? >> >> Cheers, >> Corentin >> On 26/09/2020 21:14, Paco Holguin wrote: >> >> Hello, >> >> I've looked at the documentation and it seems that octree datasets >> are loadable for a few specific codes, but not in general. I have an >> octree that represents galaxy simulation data, but it's produced as an >> output of an analysis script. It would be nice to take advantage of yt's >> capabilities to do analysis. >> >> Is there a simple way to load an generic octree or would I have to >> write a front-end myself (as described in the documentation and by looking >> at ART, for example)? >> >> Thanks! >> >> >> Paco Holguin >> >> _______________________________________________ >> yt-users mailing list -- yt-users@python.org >> To unsubscribe send an email to yt-users-leave@python.orghttps://mail.python.org/mailman3/lists/yt-users.python.org/ >> Member address: contact@cphyc.me >> >> _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
No problem! I was able to create data containers and 1-d and 2-d profile plots. I still have trouble with LinePlot and Slice plots. When I try to make a slice plot, it looks like only the top 2x2 grid. If I increase the over_refine_factor, I get a finer grid (over_refine_factor = 1, I get 4x4, etc.), but for an over_refine_factor greater than 2, I get a memory error. I'm not sure if the slice plot issue is completely related to the octree, as I was trying to plot the original snapshot from which the octree is made from (GIZMO data), and I am having similar issues with that. I was going to play around more with the slice plots of the original snapshot and see if I can understand that issue and if it's related to the octree slice plot issue. On Thu, Oct 22, 2020 at 12:19 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco, sorry for not replying sooner! I am happy that seemed to work for you. Can you share if things were problematic or not afterward?
On Sun, Oct 11, 2020 at 4:28 PM Paco Holguin <opaco450@gmail.com> wrote:
I think I found the solution. Indeed, the data was being marked as a particle field because 'load_octree' calls 'set_particle_types' on the data, and if len(data[key].shape) == 1, it gets marked as a particle type.
For the density array that I pass to the quantities dictionary, I added an empty dimension via np.newaxis, and that seems to set the field to the correct type.
On Mon, Sep 28, 2020 at 4:59 PM Paco Holguin <opaco450@gmail.com> wrote:
Trying "stream, "density" or just "density" results in the same error. When I change the quantities dictionary to just "density" the field appears as 'io', 'density' which seems to indicate that it's being parked as a particle field as you said.
On Mon, Sep 28, 2020 at 4:18 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco,
OK, this does look like what I would have expected. What is likely happening, I *think*, is that it's getting "gas","density" marked as a particle field rather than a fluid field.
What happens if you change it so that instead of being "gas","density" you are either "stream","density" or just "density"?
On Mon, Sep 28, 2020 at 3:12 PM Paco Holguin <opaco450@gmail.com> wrote:
I loaded the data with h5py, as "f" and then "density = np.array( f['rho'], dtype='f8') and "oct_mask" in a similar way, with the array containing 0's for leaf cells.
From there:
quantities = {} quantities['gas', 'density'] = density ds = yt.load_octree(oct_mask, data = quantities, over_refine_facotor = 0, partial_coverage=1)
On Mon, Sep 28, 2020 at 3:58 PM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Paco,
What is the command you use to load the octree, specifically? i.e., what you feed in to load_octree ?
On Mon, Sep 28, 2020 at 2:56 PM Paco Holguin <opaco450@gmail.com> wrote:
> I tried following the link: > https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures... > > The "ds" object is created successfully, and fields such as 'y' seem > to output correctly. However, when I try to access "gas","density" I get > the following error, which occurs even if I create some random particle > data and include it in the list of "quantities" > > > > Traceback (most recent call last): > > File "load_tree.py", line 83, in <module> > > ad['gas','density'] > > File > "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", > line 282, in __getitem__ > > self.get_data(f) > > File > "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", > line 1331, in get_data > > particles, self, self._current_chunk) > > File > "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", > line 227, in _read_particle_fields > > fields_to_read) > > File > "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", > line 199, in _read_particle_selection > > psize = self._count_particles_chunks(chunks, ptf, selector) > > File > "/home/opaco/yt_dir/yt-conda/lib/python3.7/site-packages/yt/utilities/io_handler.py", > line 170, in _count_particles_chunks > > for ptype, (x, y, z) in self._read_particle_coords(chunks, ptf): > > AttributeError: 'IOHandlerStreamOctree' object has no attribute > '_read_particle_coords' > > > > > > I tried looking at "data_containers.py" but I wasn't able to figure > out where to look from there. I'm wondering if I could remove the particle > part of this line, or something similar. > > # The _read method will figure out which fields it needs to > get from > > # disk, and return a dict of those fields along with the > fields that > > # need to be generated. > > read_fluids, gen_fluids = self.index._read_fluid_fields( > > fluids, self, > self._current_chunk) > > for f, v in read_fluids.items(): > > self.field_data[f] = self.ds.arr(v, input_units = > finfos[f].units) > > > self.field_data[f].convert_to_units(finfos[f].output_units) > > > read_particles, gen_particles = > self.index._read_particle_fields( > > particles, self, > self._current_chunk) > > > > I would appreciate any insight into what is going on. > > Thanks! > > > On Sun, Sep 27, 2020 at 8:10 PM Paco Holguin <opaco450@gmail.com> > wrote: > >> Thanks for the link. I'll take a look at how the data is structured >> and try the "load_octree" function. The data is in hdf5 format. >> >> On Sun, Sep 27, 2020 at 5:49 AM Corentin CADIOU <contact@cphyc.me> >> wrote: >> >>> Dear Paco, >>> >>> You may be able to achieve what you want using `yt.load_octree` >>> (see >>> https://yt-project.org/doc/reference/api/yt.frontends.stream.data_structures...), >>> though I never tried it myself. >>> >>> The function expects the octs to be fed in a depth-first manner >>> (see https://en.wikipedia.org/wiki/Depth-first_search). >>> >>> Out of curiosity, what format has the data you are trying to load? >>> >>> Cheers, >>> Corentin >>> On 26/09/2020 21:14, Paco Holguin wrote: >>> >>> Hello, >>> >>> I've looked at the documentation and it seems that octree datasets >>> are loadable for a few specific codes, but not in general. I have an >>> octree that represents galaxy simulation data, but it's produced as an >>> output of an analysis script. It would be nice to take advantage of yt's >>> capabilities to do analysis. >>> >>> Is there a simple way to load an generic octree or would I have to >>> write a front-end myself (as described in the documentation and by looking >>> at ART, for example)? >>> >>> Thanks! >>> >>> >>> Paco Holguin >>> >>> _______________________________________________ >>> yt-users mailing list -- yt-users@python.org >>> To unsubscribe send an email to yt-users-leave@python.orghttps://mail.python.org/mailman3/lists/yt-users.python.org/ >>> Member address: contact@cphyc.me >>> >>> _______________________________________________ > yt-users mailing list -- yt-users@python.org > To unsubscribe send an email to yt-users-leave@python.org > https://mail.python.org/mailman3/lists/yt-users.python.org/ > Member address: matthewturk@gmail.com > _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: matthewturk@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: opaco450@gmail.com
participants (3)
-
Corentin CADIOU -
Matthew Turk -
Paco Holguin