Hi all, I didn't read my email until this morning, when I saw the text halos thread had exploded. I had no idea how strongly people felt about halos! Here is a proposal for a solution that I think reduces confusion and also adds capability. Using Christine's request as an example, what I would do is add to the LoadTextHaloes a supplementary data column dict: pf = load("data") halos = LoadTextHaloes(pf, "file.txt", columns = {'x':0, 'y':1, 'z':2, 'r':3}, supp = {'m':4}) And for each halo object there would be a supp dict hanging off of it: h0 = halos[0] h0.supp['m'] This means that h0.total_mass() will correspond to the sphere, while the 'm' above to whatever's in the text file. This supp dict could be used to store temporary data for each halo object outside the LoadTextHaloes function, and if I hear that it might be useful, it could be included in a halo.dump() save. One issue is what happens to the supp columns? Do I float() them, or do I keep them strings, which allows users to convert them if needed? If I keep them strings, 'type' string columns would be useful. Or do I have yet another dict that specifies type in the call to LoadTextHaloes? -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Hi,
Could we just check if 'm' exists in the columns dict, and not have a supp
dict?
Sam
On Tue, Jul 10, 2012 at 10:18 AM, Stephen Skory wrote:
Hi all,
I didn't read my email until this morning, when I saw the text halos thread had exploded. I had no idea how strongly people felt about halos!
Here is a proposal for a solution that I think reduces confusion and also adds capability. Using Christine's request as an example, what I would do is add to the LoadTextHaloes a supplementary data column dict:
pf = load("data") halos = LoadTextHaloes(pf, "file.txt", columns = {'x':0, 'y':1, 'z':2, 'r':3}, supp = {'m':4})
And for each halo object there would be a supp dict hanging off of it:
h0 = halos[0] h0.supp['m']
This means that h0.total_mass() will correspond to the sphere, while the 'm' above to whatever's in the text file. This supp dict could be used to store temporary data for each halo object outside the LoadTextHaloes function, and if I hear that it might be useful, it could be included in a halo.dump() save.
One issue is what happens to the supp columns? Do I float() them, or do I keep them strings, which allows users to convert them if needed? If I keep them strings, 'type' string columns would be useful. Or do I have yet another dict that specifies type in the call to LoadTextHaloes?
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Sam,
I think Stephen's idea is a bit more general than that. If you look
in halo_objects.py, you can se that the number of properties of a halo
object has 1) exploded 2) no standardization. Having a dictionary of
properties, which can vary in name, fits more with the yt philosophy.
I like this idea. We could expand it to "derived properties" as well,
and maybe reduce te number of by-default calculated properties of a
halo.
If we start thinking of a future where we can load halos without
necessarily affiliating them with simulation data, this could be kind
of cool. What do you think?
-Matt
On Tue, Jul 10, 2012 at 9:25 AM, Sam Skillman
Hi,
Could we just check if 'm' exists in the columns dict, and not have a supp dict?
Sam
On Tue, Jul 10, 2012 at 10:18 AM, Stephen Skory
wrote:Hi all,
I didn't read my email until this morning, when I saw the text halos thread had exploded. I had no idea how strongly people felt about halos!
Here is a proposal for a solution that I think reduces confusion and also adds capability. Using Christine's request as an example, what I would do is add to the LoadTextHaloes a supplementary data column dict:
pf = load("data") halos = LoadTextHaloes(pf, "file.txt", columns = {'x':0, 'y':1, 'z':2, 'r':3}, supp = {'m':4})
And for each halo object there would be a supp dict hanging off of it:
h0 = halos[0] h0.supp['m']
This means that h0.total_mass() will correspond to the sphere, while the 'm' above to whatever's in the text file. This supp dict could be used to store temporary data for each halo object outside the LoadTextHaloes function, and if I hear that it might be useful, it could be included in a halo.dump() save.
One issue is what happens to the supp columns? Do I float() them, or do I keep them strings, which allows users to convert them if needed? If I keep them strings, 'type' string columns would be useful. Or do I have yet another dict that specifies type in the call to LoadTextHaloes?
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
I guess I don't understand why we have two dictionaries. I would think we
could define a standard for the normal properties, then read in all others
as desired by the user.
i would think LoadTextHaloes(pf, "file.txt", columns = {'x':0, 'y':1,
'z':2, 'r':3, 'm':4, 'dinosaur_age':5}) could be possible. Maybe you would
want the value for each key to be (column, type) if you cared about
non-float types.
Sam
On Tue, Jul 10, 2012 at 10:28 AM, Matthew Turk
Hi Sam,
I think Stephen's idea is a bit more general than that. If you look in halo_objects.py, you can se that the number of properties of a halo object has 1) exploded 2) no standardization. Having a dictionary of properties, which can vary in name, fits more with the yt philosophy. I like this idea. We could expand it to "derived properties" as well, and maybe reduce te number of by-default calculated properties of a halo.
If we start thinking of a future where we can load halos without necessarily affiliating them with simulation data, this could be kind of cool. What do you think?
-Matt
On Tue, Jul 10, 2012 at 9:25 AM, Sam Skillman
wrote: Hi,
Could we just check if 'm' exists in the columns dict, and not have a supp dict?
Sam
On Tue, Jul 10, 2012 at 10:18 AM, Stephen Skory
wrote:Hi all,
I didn't read my email until this morning, when I saw the text halos thread had exploded. I had no idea how strongly people felt about halos!
Here is a proposal for a solution that I think reduces confusion and also adds capability. Using Christine's request as an example, what I would do is add to the LoadTextHaloes a supplementary data column dict:
pf = load("data") halos = LoadTextHaloes(pf, "file.txt", columns = {'x':0, 'y':1, 'z':2, 'r':3}, supp = {'m':4})
And for each halo object there would be a supp dict hanging off of it:
h0 = halos[0] h0.supp['m']
This means that h0.total_mass() will correspond to the sphere, while the 'm' above to whatever's in the text file. This supp dict could be used to store temporary data for each halo object outside the LoadTextHaloes function, and if I hear that it might be useful, it could be included in a halo.dump() save.
One issue is what happens to the supp columns? Do I float() them, or do I keep them strings, which allows users to convert them if needed? If I keep them strings, 'type' string columns would be useful. Or do I have yet another dict that specifies type in the call to LoadTextHaloes?
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Sam & Matt,
I guess I don't understand why we have two dictionaries. I would think we could define a standard for the normal properties, then read in all others as desired by the user.
Yeah, you're probably right that we can have a bit more intelligence and have only one dict.
If we start thinking of a future where we can load halos without necessarily affiliating them with simulation data, this could be kind of cool. What do you think?
I think it's reasonable to redesign the halo object so that it doesn't need to point to a dataset. If it's made available, that's great, and if not, it will just not allow you to do certain operations. But there would have to be some value-added aspect to this - how do we make yt work on unaffiliated halo objects in a way that's more useful than a simple python/perl/awk script on a bunch of text? One idea is to make merger tree operations streamlined and simplified with halo objects. For example, one can imagine a time series halo time_halo['m'], time_halo['t'] where the first gives mass of the halo at each time given by the second. This would involve some fairly invasive changes to halos and the merger tree, and I haven't fully formed these ideas. Thoughts from the crowd? -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
I like this idea too. I don't know what's the best way to organize the data structure, but giving the user more flexibility over this sounds great. Making this more flexible will also give the code room to grow as people want to experiment and do more sophisticated things. On Tue, 2012-07-10 at 09:28 -0700, Matthew Turk wrote:
If you look in halo_objects.py, you can se that the number of properties of a halo object has 1) exploded 2) no standardization. Having a dictionary of properties, which can vary in name, fits more with the yt philosophy. I like this idea. We could expand it to "derived properties" as well, and maybe reduce te number of by-default calculated properties of a halo.
If we start thinking of a future where we can load halos without necessarily affiliating them with simulation data, this could be kind of cool. What do you think?
participants (4)
-
Christine Simpson
-
Matthew Turk
-
Sam Skillman
-
Stephen Skory