advice on implementing halo member particle loading
Hi everyone, Myself and one other who should be joining this list soon are interested in implementing some new functionality and I would like to solicit some advice. What we have are halo catalog datasets that, for the most part, are already loadable in yt. They are loadable in that each individual halo is a particle such that querying the mass field returns the mass of each halo. However, these datasets also store the particles from the simulation snapshot that are the members of each halo. We currently do not have the machinery to query them and this is what we would like to add. It seems the most sensible idea is to create a halo data container such that something like the following returns the mass of all the particles in a halo: my_halo = ds.halo(some_id) my_halo["particle_mass"] This is somewhat different from the existing data containers as it is not geometric. The member particles are explicitly known. I don't have much experience with making data containers so I can't tell if this makes things easier or harder. Anyway, this is the plan. If anyone has reasons this may be too difficult or simply not work, I would appreciate hearing them. Any general advice or other ideas for making this functionality work would also be welcome. Thanks! Britton _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
+1 for this idea. I have essentially been using yt.sphere data contains
with the virial radius as my way of querying the particles within the halo.
When in fact depending on the halo finder, halos do not necessary have to
be spherical. Likewise rockstar and other halo finders also have machinery
to tell you what particles are currently in the halo too.
On topic of halo finders, how easy do you guys think would it be to
implement other halo finders? Or at least be able to read in the results of
other halo finders? With machinery like this, it would potentially be
easier to "at least" include the results of other halo finders too. I have
been using AHF for a lot of my work recently but am currently working with
a custom version of the pynbody halo catalog (pretty much a direct rip but
using YT to handle the units upon initialisation). I imagine with machinery
like this it would be easier to do so.
Ben.
On Thu, May 7, 2015 at 11:57 AM, Britton Smith
Hi everyone,
Myself and one other who should be joining this list soon are interested in implementing some new functionality and I would like to solicit some advice.
What we have are halo catalog datasets that, for the most part, are already loadable in yt. They are loadable in that each individual halo is a particle such that querying the mass field returns the mass of each halo. However, these datasets also store the particles from the simulation snapshot that are the members of each halo. We currently do not have the machinery to query them and this is what we would like to add.
It seems the most sensible idea is to create a halo data container such that something like the following returns the mass of all the particles in a halo:
my_halo = ds.halo(some_id) my_halo["particle_mass"]
This is somewhat different from the existing data containers as it is not geometric. The member particles are explicitly known. I don't have much experience with making data containers so I can't tell if this makes things easier or harder. Anyway, this is the plan.
If anyone has reasons this may be too difficult or simply not work, I would appreciate hearing them. Any general advice or other ideas for making this functionality work would also be welcome.
Thanks!
Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Ben,
I have added frontends for the outputs of a couple different halo finders
now and found it to be pretty straightforward. If you're interested in
doing this, you should have a look at either the rockstar or owls_subfind
frontends.
Britton
On Thu, May 7, 2015 at 11:08 AM, Ben Thompson
+1 for this idea. I have essentially been using yt.sphere data contains with the virial radius as my way of querying the particles within the halo. When in fact depending on the halo finder, halos do not necessary have to be spherical. Likewise rockstar and other halo finders also have machinery to tell you what particles are currently in the halo too.
On topic of halo finders, how easy do you guys think would it be to implement other halo finders? Or at least be able to read in the results of other halo finders? With machinery like this, it would potentially be easier to "at least" include the results of other halo finders too. I have been using AHF for a lot of my work recently but am currently working with a custom version of the pynbody halo catalog (pretty much a direct rip but using YT to handle the units upon initialisation). I imagine with machinery like this it would be easier to do so.
Ben.
On Thu, May 7, 2015 at 11:57 AM, Britton Smith
wrote: Hi everyone,
Myself and one other who should be joining this list soon are interested in implementing some new functionality and I would like to solicit some advice.
What we have are halo catalog datasets that, for the most part, are already loadable in yt. They are loadable in that each individual halo is a particle such that querying the mass field returns the mass of each halo. However, these datasets also store the particles from the simulation snapshot that are the members of each halo. We currently do not have the machinery to query them and this is what we would like to add.
It seems the most sensible idea is to create a halo data container such that something like the following returns the mass of all the particles in a halo:
my_halo = ds.halo(some_id) my_halo["particle_mass"]
This is somewhat different from the existing data containers as it is not geometric. The member particles are explicitly known. I don't have much experience with making data containers so I can't tell if this makes things easier or harder. Anyway, this is the plan.
If anyone has reasons this may be too difficult or simply not work, I would appreciate hearing them. Any general advice or other ideas for making this functionality work would also be welcome.
Thanks!
Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Britton.
Probably should not be too difficult to port across my code for that.
Sorry to also distract from the original topic too. I think the idea is
good, but I don't have much experience on the data selector side of things
myself, so I do not know how much help I would be on that part in
particular.
Ben.
On Thu, May 7, 2015 at 12:11 PM, Britton Smith
Hi Ben,
I have added frontends for the outputs of a couple different halo finders now and found it to be pretty straightforward. If you're interested in doing this, you should have a look at either the rockstar or owls_subfind frontends.
Britton
On Thu, May 7, 2015 at 11:08 AM, Ben Thompson
wrote: +1 for this idea. I have essentially been using yt.sphere data contains with the virial radius as my way of querying the particles within the halo. When in fact depending on the halo finder, halos do not necessary have to be spherical. Likewise rockstar and other halo finders also have machinery to tell you what particles are currently in the halo too.
On topic of halo finders, how easy do you guys think would it be to implement other halo finders? Or at least be able to read in the results of other halo finders? With machinery like this, it would potentially be easier to "at least" include the results of other halo finders too. I have been using AHF for a lot of my work recently but am currently working with a custom version of the pynbody halo catalog (pretty much a direct rip but using YT to handle the units upon initialisation). I imagine with machinery like this it would be easier to do so.
Ben.
On Thu, May 7, 2015 at 11:57 AM, Britton Smith
wrote: Hi everyone,
Myself and one other who should be joining this list soon are interested in implementing some new functionality and I would like to solicit some advice.
What we have are halo catalog datasets that, for the most part, are already loadable in yt. They are loadable in that each individual halo is a particle such that querying the mass field returns the mass of each halo. However, these datasets also store the particles from the simulation snapshot that are the members of each halo. We currently do not have the machinery to query them and this is what we would like to add.
It seems the most sensible idea is to create a halo data container such that something like the following returns the mass of all the particles in a halo:
my_halo = ds.halo(some_id) my_halo["particle_mass"]
This is somewhat different from the existing data containers as it is not geometric. The member particles are explicitly known. I don't have much experience with making data containers so I can't tell if this makes things easier or harder. Anyway, this is the plan.
If anyone has reasons this may be too difficult or simply not work, I would appreciate hearing them. Any general advice or other ideas for making this functionality work would also be welcome.
Thanks!
Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
I think this is a good idea and would be helpful to a lot of people! I have
a couple (minor) concerns/questions
1. The difference between accessing the attributes of a halo catalog and
accessing the attributes of a halo's particles needs to be really well
documented and continuously restated throughout any docs examples. In those
two lines (below), it took me a little bit to think about whether I'd
expect that to return the halo's mass (considering the halo as a
"particle") or a list of the masses of the particles that make up the halo
- and I've used this machinery a lot!
my_halo = ds.halo(some_id)
my_halo["particle_mass"]
2. On a similar note, we should be more deliberate with the naming
conventions of the various fields in a halo catalog. At the moment, the
default mass of the particle is stored as 'particle_mass', which seems
misleading if thats also what we want to call the mass of constituent
particle field. Maybe something like "particle_mass" vs "halo_mass" and
"particle_position_x" vs "halo_position_x"? I haven't completely thought
this through, but its at least a question we should consider.
By a similar (but pretty unrelated) token, we also should be more careful
about what we store as the halo radius. Right now we store a field
"virial_radius" in the halo catalog by default, which I think confuses a
lot of users, because of its similarity to the virial_quantities callback.
3. Is a data container the right way to go? Is there enough overlap between
the functionality we would want from a halo and the functionality of a
(generally) geometric subset of some original dataset? Given my lack of
familiarity with that infrastructure, I have absolutely no idea.
-Hilary
On Thu, May 7, 2015 at 9:15 AM, Ben Thompson
Hi Britton.
Probably should not be too difficult to port across my code for that.
Sorry to also distract from the original topic too. I think the idea is good, but I don't have much experience on the data selector side of things myself, so I do not know how much help I would be on that part in particular.
Ben.
On Thu, May 7, 2015 at 12:11 PM, Britton Smith
wrote: Hi Ben,
I have added frontends for the outputs of a couple different halo finders now and found it to be pretty straightforward. If you're interested in doing this, you should have a look at either the rockstar or owls_subfind frontends.
Britton
On Thu, May 7, 2015 at 11:08 AM, Ben Thompson
wrote: +1 for this idea. I have essentially been using yt.sphere data contains with the virial radius as my way of querying the particles within the halo. When in fact depending on the halo finder, halos do not necessary have to be spherical. Likewise rockstar and other halo finders also have machinery to tell you what particles are currently in the halo too.
On topic of halo finders, how easy do you guys think would it be to implement other halo finders? Or at least be able to read in the results of other halo finders? With machinery like this, it would potentially be easier to "at least" include the results of other halo finders too. I have been using AHF for a lot of my work recently but am currently working with a custom version of the pynbody halo catalog (pretty much a direct rip but using YT to handle the units upon initialisation). I imagine with machinery like this it would be easier to do so.
Ben.
On Thu, May 7, 2015 at 11:57 AM, Britton Smith
wrote: Hi everyone,
Myself and one other who should be joining this list soon are interested in implementing some new functionality and I would like to solicit some advice.
What we have are halo catalog datasets that, for the most part, are already loadable in yt. They are loadable in that each individual halo is a particle such that querying the mass field returns the mass of each halo. However, these datasets also store the particles from the simulation snapshot that are the members of each halo. We currently do not have the machinery to query them and this is what we would like to add.
It seems the most sensible idea is to create a halo data container such that something like the following returns the mass of all the particles in a halo:
my_halo = ds.halo(some_id) my_halo["particle_mass"]
This is somewhat different from the existing data containers as it is not geometric. The member particles are explicitly known. I don't have much experience with making data containers so I can't tell if this makes things easier or harder. Anyway, this is the plan.
If anyone has reasons this may be too difficult or simply not work, I would appreciate hearing them. Any general advice or other ideas for making this functionality work would also be welcome.
Thanks!
Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Britton,
I like this idea in general. Just to confirm, the output of your code
snippet would return a list of particle masses, one for each particle
contained in the halo?
Cameron
On Thu, May 7, 2015 at 7:57 AM, Britton Smith
Hi everyone,
Myself and one other who should be joining this list soon are interested in implementing some new functionality and I would like to solicit some advice.
What we have are halo catalog datasets that, for the most part, are already loadable in yt. They are loadable in that each individual halo is a particle such that querying the mass field returns the mass of each halo. However, these datasets also store the particles from the simulation snapshot that are the members of each halo. We currently do not have the machinery to query them and this is what we would like to add.
It seems the most sensible idea is to create a halo data container such that something like the following returns the mass of all the particles in a halo:
my_halo = ds.halo(some_id) my_halo["particle_mass"]
This is somewhat different from the existing data containers as it is not geometric. The member particles are explicitly known. I don't have much experience with making data containers so I can't tell if this makes things easier or harder. Anyway, this is the plan.
If anyone has reasons this may be too difficult or simply not work, I would appreciate hearing them. Any general advice or other ideas for making this functionality work would also be welcome.
Thanks!
Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Britton,
I think this solution may work, but may also be tricky. To implement a
data selector, these methods are needed:
select_sphere
select_point (a special case of sphere with radius zero)
select_bbox
select_cell (degenerate with select_bbox)
Selectors get called by a variety of cutout methods; one possibility, which
we use for the Grid selector, is to special-case the IO routines for Halo
selectors. If the particles can be read in individually, that would work.
Selecting bbox and cell would allow you to then compute things like
deposited mass or something, if that's useful, and could be done by
identifying if any particles are within a given bbox.
-Matt
On Thu, May 7, 2015 at 9:57 AM, Britton Smith
Hi everyone,
Myself and one other who should be joining this list soon are interested in implementing some new functionality and I would like to solicit some advice.
What we have are halo catalog datasets that, for the most part, are already loadable in yt. They are loadable in that each individual halo is a particle such that querying the mass field returns the mass of each halo. However, these datasets also store the particles from the simulation snapshot that are the members of each halo. We currently do not have the machinery to query them and this is what we would like to add.
It seems the most sensible idea is to create a halo data container such that something like the following returns the mass of all the particles in a halo:
my_halo = ds.halo(some_id) my_halo["particle_mass"]
This is somewhat different from the existing data containers as it is not geometric. The member particles are explicitly known. I don't have much experience with making data containers so I can't tell if this makes things easier or harder. Anyway, this is the plan.
If anyone has reasons this may be too difficult or simply not work, I would appreciate hearing them. Any general advice or other ideas for making this functionality work would also be welcome.
Thanks!
Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (5)
-
Ben Thompson
-
Britton Smith
-
Cameron Hummels
-
Hilary Egan
-
Matthew Turk