Hi again. Thank for helping me Britton! I think I've gotten the halo catalog saved to file now, but I'm having problems loading it using HaloCatalog. I've used the code you provided, and everything seems to be going fine until I try loading the catalog. If I try this: save_as_dataset(ds,'test.h5', data, field_types=ftypes, extra_attrs=extra_attrs) ds_halo = load('test.h5') hc = HaloCatalog(data_ds=ds, halos_ds=ds_halo) This returns IOError: Unable to open file (Unable to open file: name = '.0.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0) I tried using save_as_dataset(ds,'test.0.h5', data, field_types=ftypes, extra_attrs=extra_attrs) ds_halo = load('test.0.h5') hc = HaloCatalog(data_ds=ds, halos_ds=ds_halo) instead, but then I get IOError: Unable to open file (Unable to open file: name = 'test.1.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0) It seems it's trying to open more than one file for some reason? Did I misunderstand something? I've uploaded the code again to http://paste.yt-project.org/show/7062/ Best, Andreas Ellewsen On Thu, 2 Mar 2017 at 19:37 Britton Smith wrote: Hi Andreas, I think all this can be done very cleanly using yt's halo analysis toolkit. I'd be happy to help you with this. Documentation for the halo analysis, also called the HaloCatalog, is here: http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_catalogs.html... The HaloCatalog works with the simulation snapshot and the halo finder output in tandem to run a pipeline of analysis that you define. yt comes with a number of functions out of the box for doing profiling, etc, including scaling of profiles to r_vir. Before that, you will need to convert your halo finder output into a form that yt can read in using yt.load. This should be relatively straightforward. I've sketched out how to do this here: http://paste.yt-project.org/show/7061/ The code snippet there assumes some of the variables you've got in the script you posted. There may be some hiccups still, but I'm happy to work with you to get it going. Once that works, you should be able to use the HaloCatalog as documented to do the rest, but feel free to ask any other questions you might have. Britton On Wed, Mar 1, 2017 at 5:16 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote: Hi everyone. I'm currently working on making temperature and density profiles from some ramses simulations. I'd like to make these profiles the average radial profile of the halos in the simulation. I think the included rockstar in yt supports this in some form, but my halo catalog is made using AHF and then imported from an ascii file. To do this I need help with two things: The first one is how to scale the radius of each profile to the corresponding virial radius. The other is how to actually average all these profiles. Is this something that can be done? I will be trying to compare the resulting profile with the same for different simulations in the end, but I don't expect to have any big problems with that. The code I'm currently using for this can be found at http://paste.yt-project.org/show/7058/ Best, Andreas Ellewsen _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi again, Thanks again for helping me Britton. That actually made a difference. Unfortunately I now get a RuntimeError from hc = HaloCatalog(data_ds=ds, halo_ds=ds_halo) RuntimeError: HaloCatalog quantity must be a registered function or a field of a known type. Best, Andreas Ellewsen On Fri, 3 Mar 2017 at 19:16 Britton Smith wrote: Hi Andreas, I forgot about the constraints of the naming conventions for halo catalog files. They need to be named in the following convention: <prefix>.<integer>.h5 where the integer should start at 0. That integer is in case your catalog is split over multiple files. In examining the code, I can see that it counts the number of files by basically doing glob(<prefix>*h5), so if you've got a file named test.h5 and test.0.h5 in the same directory, it will fool the code into thinking there should be test.0.h5 and test.1.h5. This is clearly a bit of a silly shortcoming of this approach, and I will submit a pull request to make this more intelligent. In the mean time, I think you can fix this by making sure that there is only one file, test.0.h5 in the directory. Let me know if that doesn't fix it. Britton On Fri, Mar 3, 2017 at 6:28 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote: Hi again. Thank for helping me Britton! I think I've gotten the halo catalog saved to file now, but I'm having problems loading it using HaloCatalog. I've used the code you provided, and everything seems to be going fine until I try loading the catalog. If I try this: save_as_dataset(ds,'test.h5', data, field_types=ftypes, extra_attrs=extra_attrs) ds_halo = load('test.h5') hc = HaloCatalog(data_ds=ds, halos_ds=ds_halo) This returns IOError: Unable to open file (Unable to open file: name = '.0.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0) I tried using save_as_dataset(ds,'test.0.h5', data, field_types=ftypes, extra_attrs=extra_attrs) ds_halo = load('test.0.h5') hc = HaloCatalog(data_ds=ds, halos_ds=ds_halo) instead, but then I get IOError: Unable to open file (Unable to open file: name = 'test.1.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0) It seems it's trying to open more than one file for some reason? Did I misunderstand something? I've uploaded the code again to http://paste.yt-project.org/show/7062/ Best, Andreas Ellewsen On Thu, 2 Mar 2017 at 19:37 Britton Smith wrote: Hi Andreas, I think all this can be done very cleanly using yt's halo analysis toolkit. I'd be happy to help you with this. Documentation for the halo analysis, also called the HaloCatalog, is here: http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_catalogs.html... The HaloCatalog works with the simulation snapshot and the halo finder output in tandem to run a pipeline of analysis that you define. yt comes with a number of functions out of the box for doing profiling, etc, including scaling of profiles to r_vir. Before that, you will need to convert your halo finder output into a form that yt can read in using yt.load. This should be relatively straightforward. I've sketched out how to do this here: http://paste.yt-project.org/show/7061/ The code snippet there assumes some of the variables you've got in the script you posted. There may be some hiccups still, but I'm happy to work with you to get it going. Once that works, you should be able to use the HaloCatalog as documented to do the rest, but feel free to ask any other questions you might have. Britton On Wed, Mar 1, 2017 at 5:16 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote: Hi everyone. I'm currently working on making temperature and density profiles from some ramses simulations. I'd like to make these profiles the average radial profile of the halos in the simulation. I think the included rockstar in yt supports this in some form, but my halo catalog is made using AHF and then imported from an ascii file. To do this I need help with two things: The first one is how to scale the radius of each profile to the corresponding virial radius. The other is how to actually average all these profiles. Is this something that can be done? I will be trying to compare the resulting profile with the same for different simulations in the end, but I don't expect to have any big problems with that. The code I'm currently using for this can be found at http://paste.yt-project.org/show/7058/ Best, Andreas Ellewsen _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi again everyone. Sorry for not getting back to you faster. I've tried implementing what you said Britton, and it seems to work. However I'm still doing something wrong. I'm using this script to make the halo catalog: http://paste.yt-project.org/show/7077/ And this one to load the sim and the catalog, and making radial temperature profiles: http://paste.yt-project.org/show/7078/ I'm simply copying the example found here: http://yt-project.org/doc/cookbook/halo_analysis_example.html#Finding-Radial... In the example we calculate the virial radius instead of just using the one I already have in the catalog. I'm not sure how to use the one I already have so any tips on that is also appreciated. Still, it is extremely slow and returns the following: yt : [INFO ] 2017-03-10 11:55:17,007 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-10 12:23:13,269 Calculating virial quantities for halo 1. yt : [INFO ] 2017-03-10 12:23:13,270 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-10 12:23:13,270 Skipping halo 1 since data_object is None. yt : [INFO ] 2017-03-10 12:23:13,272 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-10 12:40:49,659 Calculating virial quantities for halo 2. yt : [INFO ] 2017-03-10 12:40:49,660 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-10 12:40:49,660 Skipping halo 2 since data_object is None. yt : [INFO ] 2017-03-10 12:40:49,662 Calculating 1D profile for halo 3. yt : [INFO ] 2017-03-10 12:46:30,547 Calculating virial quantities for halo 3. yt : [INFO ] 2017-03-10 12:46:30,548 Calculating 1D profile for halo 3. yt : [INFO ] 2017-03-10 12:46:30,548 Skipping halo 3 since data_object is None. yt : [INFO ] 2017-03-10 12:46:30,550 Calculating 1D profile for halo 4. yt : [INFO ] 2017-03-10 12:54:56,886 Calculating virial quantities for halo 4. yt : [INFO ] 2017-03-10 12:54:56,888 Calculating 1D profile for halo 4. yt : [INFO ] 2017-03-10 12:54:56,888 Skipping halo 4 since data_object is None. yt : [INFO ] 2017-03-10 12:54:56,890 Calculating 1D profile for halo 5. yt : [INFO ] 2017-03-10 13:01:48,086 Calculating virial quantities for halo 5. yt : [INFO ] 2017-03-10 13:01:48,087 Calculating 1D profile for halo 5. yt : [INFO ] 2017-03-10 13:01:48,087 Skipping halo 5 since data_object is None. yt : [INFO ] 2017-03-10 13:01:48,089 Calculating 1D profile for halo 6. I at this point I killed it. Thank you for any insight you can provide on this. Best, Andreas Ellewsen On Fri, 3 Mar 2017 at 20:09 Britton Smith wrote: Hi Andreas, Ah, I think I see the problem. It wants the halos_ds to have halo ids and masses. If it's possible, could you add fields to your save_as_dataset script called "particle_identifier" for halo ids and "particle_mass" for masses. If you don't have those fields specifically, you can probably just make up values. When I fix the other issues, I'll make sure to make this a little smarter as well. Britton On Fri, Mar 3, 2017 at 10:46 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote: Hi again, Thanks again for helping me Britton. That actually made a difference. Unfortunately I now get a RuntimeError from hc = HaloCatalog(data_ds=ds, halo_ds=ds_halo) RuntimeError: HaloCatalog quantity must be a registered function or a field of a known type. Best, Andreas Ellewsen On Fri, 3 Mar 2017 at 19:16 Britton Smith wrote: Hi Andreas, I forgot about the constraints of the naming conventions for halo catalog files. They need to be named in the following convention: <prefix>.<integer>.h5 where the integer should start at 0. That integer is in case your catalog is split over multiple files. In examining the code, I can see that it counts the number of files by basically doing glob(<prefix>*h5), so if you've got a file named test.h5 and test.0.h5 in the same directory, it will fool the code into thinking there should be test.0.h5 and test.1.h5. This is clearly a bit of a silly shortcoming of this approach, and I will submit a pull request to make this more intelligent. In the mean time, I think you can fix this by making sure that there is only one file, test.0.h5 in the directory. Let me know if that doesn't fix it. Britton On Fri, Mar 3, 2017 at 6:28 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote: Hi again. Thank for helping me Britton! I think I've gotten the halo catalog saved to file now, but I'm having problems loading it using HaloCatalog. I've used the code you provided, and everything seems to be going fine until I try loading the catalog. If I try this: save_as_dataset(ds,'test.h5', data, field_types=ftypes, extra_attrs=extra_attrs) ds_halo = load('test.h5') hc = HaloCatalog(data_ds=ds, halos_ds=ds_halo) This returns IOError: Unable to open file (Unable to open file: name = '.0.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0) I tried using save_as_dataset(ds,'test.0.h5', data, field_types=ftypes, extra_attrs=extra_attrs) ds_halo = load('test.0.h5') hc = HaloCatalog(data_ds=ds, halos_ds=ds_halo) instead, but then I get IOError: Unable to open file (Unable to open file: name = 'test.1.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0) It seems it's trying to open more than one file for some reason? Did I misunderstand something? I've uploaded the code again to http://paste.yt-project.org/show/7062/ Best, Andreas Ellewsen On Thu, 2 Mar 2017 at 19:37 Britton Smith wrote: Hi Andreas, I think all this can be done very cleanly using yt's halo analysis toolkit. I'd be happy to help you with this. Documentation for the halo analysis, also called the HaloCatalog, is here: http://yt-project.org/docs/dev/analyzing/analysis_modules/halo_catalogs.html... The HaloCatalog works with the simulation snapshot and the halo finder output in tandem to run a pipeline of analysis that you define. yt comes with a number of functions out of the box for doing profiling, etc, including scaling of profiles to r_vir. Before that, you will need to convert your halo finder output into a form that yt can read in using yt.load. This should be relatively straightforward. I've sketched out how to do this here: http://paste.yt-project.org/show/7061/ The code snippet there assumes some of the variables you've got in the script you posted. There may be some hiccups still, but I'm happy to work with you to get it going. Once that works, you should be able to use the HaloCatalog as documented to do the rest, but feel free to ask any other questions you might have. Britton On Wed, Mar 1, 2017 at 5:16 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote: Hi everyone. I'm currently working on making temperature and density profiles from some ramses simulations. I'd like to make these profiles the average radial profile of the halos in the simulation. I think the included rockstar in yt supports this in some form, but my halo catalog is made using AHF and then imported from an ascii file. To do this I need help with two things: The first one is how to scale the radius of each profile to the corresponding virial radius. The other is how to actually average all these profiles. Is this something that can be done? I will be trying to compare the resulting profile with the same for different simulations in the end, but I don't expect to have any big problems with that. The code I'm currently using for this can be found at http://paste.yt-project.org/show/7058/ Best, Andreas Ellewsen _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Andreas, My guess is that there is an issue with the units of the spheres' radii. If you have yt installed from source, then you can stick a python debugger around line 89 of yt/analysis_modules/halo_analysis/halo_callbacks.py Just add the following: import pdb ; pdb.set_trace() If that's not possible, then you might just try creating a single sphere object from the first object in your halo list. The reason I suspect this is because if yt tries to create a sphere that does not contain any elements, the YTSphereTooSmall exception will be raised. This exception is caught by the sphere halo callback, causing it to set the sphere object to None, as per the error message you're seeing. If this is the problem, it will just be a matter of making sure that your radius field has the correct units on it. Let us know if you need any help looking into this. Britton On Tue, Mar 14, 2017 at 9:38 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

Hi again folks

I think I'm close to getting this to work now. Hopefully you can tell me what I'm doing wrong.

I figured out why it was so slow. Calculating the mean inside the overdensity field is not a good idea..

At this point I have the same code as before to convert from AHF to something yt understands. And the code for making the profiles is http://paste.yt-project. org/show/7085/

When running that code, the ouput you get is the following:

yt : [INFO ] 2017-03-14 17:26:17,543 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-14 17:26:20,124 Calculating virial quantities for halo 1. yt : [INFO ] 2017-03-14 17:26:20,125 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-14 17:26:20,125 Skipping halo 1 since data_object is None. yt : [INFO ] 2017-03-14 17:26:20,125 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-14 17:26:20,125 Skipping halo 1 since data_object is None. yt : [INFO ] 2017-03-14 17:26:20,128 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-14 17:26:21,869 Calculating virial quantities for halo 2. yt : [INFO ] 2017-03-14 17:26:21,869 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-14 17:26:21,869 Skipping halo 2 since data_object is None. yt : [INFO ] 2017-03-14 17:26:21,870 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-14 17:26:21,870 Skipping halo 2 since data_object is None.

So for some reason it's skipping the halos. I don't see why it would do that.

Any ideas? Thanks for the great help!

Best, Andreas Ellewsen

On Fri, 10 Mar 2017 at 13:30 Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

...

Hi again everyone. Sorry for not getting back to you faster.

I've tried implementing what you said Britton, and it seems to work. However I'm still doing something wrong.

I'm using this script to make the halo catalog: http://paste.yt-project.org/show/7077/ And this one to load the sim and the catalog, and making radial temperature profiles: http://paste.yt-project.org/show/7078/

I'm simply copying the example found here: http://yt-project.org/doc/ cookbook/halo_analysis_example.html#Finding-Radial-Profiles

In the example we calculate the virial radius instead of just using the one I already have in the catalog. I'm not sure how to use the one I already have so any tips on that is also appreciated.

Still, it is extremely slow and returns the following: yt : [INFO ] 2017-03-10 11:55:17,007 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-10 12:23:13,269 Calculating virial quantities for halo 1. yt : [INFO ] 2017-03-10 12:23:13,270 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-10 12:23:13,270 Skipping halo 1 since data_object is None. yt : [INFO ] 2017-03-10 12:23:13,272 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-10 12:40:49,659 Calculating virial quantities for halo 2. yt : [INFO ] 2017-03-10 12:40:49,660 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-10 12:40:49,660 Skipping halo 2 since data_object is None. yt : [INFO ] 2017-03-10 12:40:49,662 Calculating 1D profile for halo 3. yt : [INFO ] 2017-03-10 12:46:30,547 Calculating virial quantities for halo 3. yt : [INFO ] 2017-03-10 12:46:30,548 Calculating 1D profile for halo 3. yt : [INFO ] 2017-03-10 12:46:30,548 Skipping halo 3 since data_object is None. yt : [INFO ] 2017-03-10 12:46:30,550 Calculating 1D profile for halo 4. yt : [INFO ] 2017-03-10 12:54:56,886 Calculating virial quantities for halo 4. yt : [INFO ] 2017-03-10 12:54:56,888 Calculating 1D profile for halo 4. yt : [INFO ] 2017-03-10 12:54:56,888 Skipping halo 4 since data_object is None. yt : [INFO ] 2017-03-10 12:54:56,890 Calculating 1D profile for halo 5. yt : [INFO ] 2017-03-10 13:01:48,086 Calculating virial quantities for halo 5. yt : [INFO ] 2017-03-10 13:01:48,087 Calculating 1D profile for halo 5. yt : [INFO ] 2017-03-10 13:01:48,087 Skipping halo 5 since data_object is None. yt : [INFO ] 2017-03-10 13:01:48,089 Calculating 1D profile for halo 6.

I at this point I killed it. Thank you for any insight you can provide on this.

Best, Andreas Ellewsen

On Fri, 3 Mar 2017 at 20:09 Britton Smith wrote:

Hi Andreas,

Ah, I think I see the problem. It wants the halos_ds to have halo ids and masses. If it's possible, could you add fields to your save_as_dataset script called "particle_identifier" for halo ids and "particle_mass" for masses. If you don't have those fields specifically, you can probably just make up values. When I fix the other issues, I'll make sure to make this a little smarter as well.

Britton

On Fri, Mar 3, 2017 at 10:46 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

Hi again,

Thanks again for helping me Britton. That actually made a difference. Unfortunately I now get a RuntimeError from hc = HaloCatalog(data_ds=ds, halo_ds=ds_halo)

RuntimeError: HaloCatalog quantity must be a registered function or a field of a known type.

Best, Andreas Ellewsen

On Fri, 3 Mar 2017 at 19:16 Britton Smith wrote:

Hi Andreas,

I forgot about the constraints of the naming conventions for halo catalog files. They need to be named in the following convention: <prefix>.<integer>.h5 where the integer should start at 0. That integer is in case your catalog is split over multiple files. In examining the code, I can see that it counts the number of files by basically doing glob(<prefix>*h5), so if you've got a file named test.h5 and test.0.h5 in the same directory, it will fool the code into thinking there should be test.0.h5 and test.1.h5. This is clearly a bit of a silly shortcoming of this approach, and I will submit a pull request to make this more intelligent. In the mean time, I think you can fix this by making sure that there is only one file, test.0.h5 in the directory.

Let me know if that doesn't fix it.

Britton

On Fri, Mar 3, 2017 at 6:28 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

Hi again.

Thank for helping me Britton!

I think I've gotten the halo catalog saved to file now, but I'm having problems loading it using HaloCatalog. I've used the code you provided, and everything seems to be going fine until I try loading the catalog.

If I try this: save_as_dataset(ds,'test.h5', data, field_types=ftypes, extra_attrs=extra_attrs) ds_halo = load('test.h5') hc = HaloCatalog(data_ds=ds, halos_ds=ds_halo)

This returns IOError: Unable to open file (Unable to open file: name = '.0.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

I tried using save_as_dataset(ds,'test.0.h5', data, field_types=ftypes, extra_attrs=extra_attrs) ds_halo = load('test.0.h5') hc = HaloCatalog(data_ds=ds, halos_ds=ds_halo) instead, but then I get

IOError: Unable to open file (Unable to open file: name = 'test.1.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

It seems it's trying to open more than one file for some reason? Did I misunderstand something?

I've uploaded the code again to http://paste.yt-project.org/show/7062/

Best, Andreas Ellewsen

On Thu, 2 Mar 2017 at 19:37 Britton Smith wrote:

Hi Andreas,

I think all this can be done very cleanly using yt's halo analysis toolkit. I'd be happy to help you with this.

Documentation for the halo analysis, also called the HaloCatalog, is here: http://yt-project.org/docs/dev/analyzing/analysis_ modules/halo_catalogs.html#halo-catalog-analysis

The HaloCatalog works with the simulation snapshot and the halo finder output in tandem to run a pipeline of analysis that you define. yt comes with a number of functions out of the box for doing profiling, etc, including scaling of profiles to r_vir.

Before that, you will need to convert your halo finder output into a form that yt can read in using yt.load. This should be relatively straightforward. I've sketched out how to do this here: http://paste.yt-project.org/show/7061/ The code snippet there assumes some of the variables you've got in the script you posted.

There may be some hiccups still, but I'm happy to work with you to get it going. Once that works, you should be able to use the HaloCatalog as documented to do the rest, but feel free to ask any other questions you might have.

Britton

On Wed, Mar 1, 2017 at 5:16 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

Hi everyone.

I'm currently working on making temperature and density profiles from some ramses simulations. I'd like to make these profiles the average radial profile of the halos in the simulation.

I think the included rockstar in yt supports this in some form, but my halo catalog is made using AHF and then imported from an ascii file.

To do this I need help with two things: The first one is how to scale the radius of each profile to the corresponding virial radius. The other is how to actually average all these profiles.

Is this something that can be done?

I will be trying to compare the resulting profile with the same for different simulations in the end, but I don't expect to have any big problems with that.

The code I'm currently using for this can be found at http://paste.yt-project.org/show/7058/

Best, Andreas Ellewsen

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Andreas, I think you just need to change the instances of "radius_200" to "virial_radius" in your callback call. Britton On Wed, Mar 15, 2017 at 9:12 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

Hi Britton,

So I've tried what you suggested now, and when checking what the radius is, it works fine for the first sphere. However when making the second sphere using hc.add_callback('sphere', radius_field='radius_200', factor=5, field_parameters=dict(virial_radius=('quantity', 'radius_200'))) I find that radius = 0.0 cm which raises the error you mentioned.

Which must mean that the radius_200 field is just zeros. So I assume the problem happens when calculating the virial quantities.

I tried moving the debugging line down to that function, and find that none of my halos have an overdensity above 200(the critical overdensity), so basically it can't find r_200 since it doesn't exist. I'll have to look into what the problem with this is. AHF must have calculated the virial radius in another way than I thought.

However I must be misunderstanding something, because I don't understand why it wants to calculate the virial radius again, when I already have one in the halo catalog I've imported. That virial radius is even used to make the first sphere. Is there some way to just construct the 'virial_radius_fraction' for each sphere just using the 'radius' field of the sphere and the corresponding value in the 'virial_radius' field contained in the Halo Catalog?

Best, Andreas Ellewsen

On Tue, 14 Mar 2017 at 22:03 Britton Smith wrote:

Hi Andreas,

My guess is that there is an issue with the units of the spheres' radii. If you have yt installed from source, then you can stick a python debugger around line 89 of yt/analysis_modules/halo_analysis/halo_callbacks.py

Just add the following: import pdb ; pdb.set_trace()

If that's not possible, then you might just try creating a single sphere object from the first object in your halo list. The reason I suspect this is because if yt tries to create a sphere that does not contain any elements, the YTSphereTooSmall exception will be raised. This exception is caught by the sphere halo callback, causing it to set the sphere object to None, as per the error message you're seeing. If this is the problem, it will just be a matter of making sure that your radius field has the correct units on it. Let us know if you need any help looking into this.

Britton

On Tue, Mar 14, 2017 at 9:38 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

Hi again folks

I think I'm close to getting this to work now. Hopefully you can tell me what I'm doing wrong.

I figured out why it was so slow. Calculating the mean inside the overdensity field is not a good idea..

At this point I have the same code as before to convert from AHF to something yt understands. And the code for making the profiles is http://paste.yt-project. org/show/7085/

When running that code, the ouput you get is the following:

yt : [INFO ] 2017-03-14 17:26:17,543 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-14 17:26:20,124 Calculating virial quantities for halo 1. yt : [INFO ] 2017-03-14 17:26:20,125 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-14 17:26:20,125 Skipping halo 1 since data_object is None. yt : [INFO ] 2017-03-14 17:26:20,125 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-14 17:26:20,125 Skipping halo 1 since data_object is None. yt : [INFO ] 2017-03-14 17:26:20,128 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-14 17:26:21,869 Calculating virial quantities for halo 2. yt : [INFO ] 2017-03-14 17:26:21,869 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-14 17:26:21,869 Skipping halo 2 since data_object is None. yt : [INFO ] 2017-03-14 17:26:21,870 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-14 17:26:21,870 Skipping halo 2 since data_object is None.

So for some reason it's skipping the halos. I don't see why it would do that.

Any ideas? Thanks for the great help!

Best, Andreas Ellewsen

On Fri, 10 Mar 2017 at 13:30 Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

Hi again everyone. Sorry for not getting back to you faster.

I've tried implementing what you said Britton, and it seems to work. However I'm still doing something wrong.

I'm using this script to make the halo catalog: http://paste.yt-project.org/show/7077/ And this one to load the sim and the catalog, and making radial temperature profiles: http://paste.yt-project.org/show/7078/

I'm simply copying the example found here: http://yt-project.org/doc/ cookbook/halo_analysis_example.html#Finding-Radial-Profiles

In the example we calculate the virial radius instead of just using the one I already have in the catalog. I'm not sure how to use the one I already have so any tips on that is also appreciated.

Still, it is extremely slow and returns the following: yt : [INFO ] 2017-03-10 11:55:17,007 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-10 12:23:13,269 Calculating virial quantities for halo 1. yt : [INFO ] 2017-03-10 12:23:13,270 Calculating 1D profile for halo 1. yt : [INFO ] 2017-03-10 12:23:13,270 Skipping halo 1 since data_object is None. yt : [INFO ] 2017-03-10 12:23:13,272 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-10 12:40:49,659 Calculating virial quantities for halo 2. yt : [INFO ] 2017-03-10 12:40:49,660 Calculating 1D profile for halo 2. yt : [INFO ] 2017-03-10 12:40:49,660 Skipping halo 2 since data_object is None. yt : [INFO ] 2017-03-10 12:40:49,662 Calculating 1D profile for halo 3. yt : [INFO ] 2017-03-10 12:46:30,547 Calculating virial quantities for halo 3. yt : [INFO ] 2017-03-10 12:46:30,548 Calculating 1D profile for halo 3. yt : [INFO ] 2017-03-10 12:46:30,548 Skipping halo 3 since data_object is None. yt : [INFO ] 2017-03-10 12:46:30,550 Calculating 1D profile for halo 4. yt : [INFO ] 2017-03-10 12:54:56,886 Calculating virial quantities for halo 4. yt : [INFO ] 2017-03-10 12:54:56,888 Calculating 1D profile for halo 4. yt : [INFO ] 2017-03-10 12:54:56,888 Skipping halo 4 since data_object is None. yt : [INFO ] 2017-03-10 12:54:56,890 Calculating 1D profile for halo 5. yt : [INFO ] 2017-03-10 13:01:48,086 Calculating virial quantities for halo 5. yt : [INFO ] 2017-03-10 13:01:48,087 Calculating 1D profile for halo 5. yt : [INFO ] 2017-03-10 13:01:48,087 Skipping halo 5 since data_object is None. yt : [INFO ] 2017-03-10 13:01:48,089 Calculating 1D profile for halo 6.

I at this point I killed it. Thank you for any insight you can provide on this.

Best, Andreas Ellewsen

On Fri, 3 Mar 2017 at 20:09 Britton Smith wrote:

Hi Andreas,

Ah, I think I see the problem. It wants the halos_ds to have halo ids and masses. If it's possible, could you add fields to your save_as_dataset script called "particle_identifier" for halo ids and "particle_mass" for masses. If you don't have those fields specifically, you can probably just make up values. When I fix the other issues, I'll make sure to make this a little smarter as well.

Britton

On Fri, Mar 3, 2017 at 10:46 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

Hi again,

Thanks again for helping me Britton. That actually made a difference. Unfortunately I now get a RuntimeError from hc = HaloCatalog(data_ds=ds, halo_ds=ds_halo)

RuntimeError: HaloCatalog quantity must be a registered function or a field of a known type.

Best, Andreas Ellewsen

On Fri, 3 Mar 2017 at 19:16 Britton Smith wrote:

Hi Andreas,

I forgot about the constraints of the naming conventions for halo catalog files. They need to be named in the following convention: <prefix>.<integer>.h5 where the integer should start at 0. That integer is in case your catalog is split over multiple files. In examining the code, I can see that it counts the number of files by basically doing glob(<prefix>*h5), so if you've got a file named test.h5 and test.0.h5 in the same directory, it will fool the code into thinking there should be test.0.h5 and test.1.h5. This is clearly a bit of a silly shortcoming of this approach, and I will submit a pull request to make this more intelligent. In the mean time, I think you can fix this by making sure that there is only one file, test.0.h5 in the directory.

Let me know if that doesn't fix it.

Britton

On Fri, Mar 3, 2017 at 6:28 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

Hi again.

Thank for helping me Britton!

I think I've gotten the halo catalog saved to file now, but I'm having problems loading it using HaloCatalog. I've used the code you provided, and everything seems to be going fine until I try loading the catalog.

If I try this: save_as_dataset(ds,'test.h5', data, field_types=ftypes, extra_attrs=extra_attrs) ds_halo = load('test.h5') hc = HaloCatalog(data_ds=ds, halos_ds=ds_halo)

This returns IOError: Unable to open file (Unable to open file: name = '.0.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

I tried using save_as_dataset(ds,'test.0.h5', data, field_types=ftypes, extra_attrs=extra_attrs) ds_halo = load('test.0.h5') hc = HaloCatalog(data_ds=ds, halos_ds=ds_halo) instead, but then I get

IOError: Unable to open file (Unable to open file: name = 'test.1.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

It seems it's trying to open more than one file for some reason? Did I misunderstand something?

I've uploaded the code again to http://paste.yt-project.org/show/7062/

Best, Andreas Ellewsen

On Thu, 2 Mar 2017 at 19:37 Britton Smith wrote:

Hi Andreas,

I think all this can be done very cleanly using yt's halo analysis toolkit. I'd be happy to help you with this.

Documentation for the halo analysis, also called the HaloCatalog, is here: http://yt-project.org/docs/dev/analyzing/analysis_ modules/halo_catalogs.html#halo-catalog-analysis

The HaloCatalog works with the simulation snapshot and the halo finder output in tandem to run a pipeline of analysis that you define. yt comes with a number of functions out of the box for doing profiling, etc, including scaling of profiles to r_vir.

Before that, you will need to convert your halo finder output into a form that yt can read in using yt.load. This should be relatively straightforward. I've sketched out how to do this here: http://paste.yt-project.org/show/7061/ The code snippet there assumes some of the variables you've got in the script you posted.

There may be some hiccups still, but I'm happy to work with you to get it going. Once that works, you should be able to use the HaloCatalog as documented to do the rest, but feel free to ask any other questions you might have.

Britton

On Wed, Mar 1, 2017 at 5:16 AM, Thor Andreas Seiff Ellewsen < tellewsen@gmail.com> wrote:

Hi everyone.

I'm currently working on making temperature and density profiles from some ramses simulations. I'd like to make these profiles the average radial profile of the halos in the simulation.

I think the included rockstar in yt supports this in some form, but my halo catalog is made using AHF and then imported from an ascii file.

To do this I need help with two things: The first one is how to scale the radius of each profile to the corresponding virial radius. The other is how to actually average all these profiles.

Is this something that can be done?

I will be trying to compare the resulting profile with the same for different simulations in the end, but I don't expect to have any big problems with that.

The code I'm currently using for this can be found at http://paste.yt-project.org/show/7058/

Best, Andreas Ellewsen

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org