Hi Britton (and yt-users), Thanks for your help on this a few months back. Your last email finally got things working with rockstar. However, even after redoing the rockstar analysis, then redoing the simulation itself and the rockstar analysis, I am still finding there seems to be some lower limit to finding halos below 1e+8 Msun/h. I get the same range of smallest halos whether I have a particle mass of 8.41072e+06 Msun/h or 1.31418e+05 Msun/h. The smallest halo I see in any output is about 1.4e+08 Msun/h. What could possibly be preventing me from finding smaller halos? Thanks, Carla Am Do., 29. Apr. 2021 um 13:19 Uhr schrieb Britton Smith < brittonsmith@gmail.com>:
Hi Carla,
I apologize for this, there is a necessary line missing from the documentation. After this line:
es = yt.load_simulation(sys.argv[1], "Enzo", find_outputs=True)
you need to add the following:
es.get_time_series()
I will correct the documentation as soon as possible. Sorry for the delay and the inconvenience.
Britton
On Thu, Apr 22, 2021 at 6:01 PM C Bernhardt <bernhardt.phd@gmail.com> wrote:
Hi Britton,
I apologize for the slow response. Another tech guy on my server managed to get yt installed from source and linked it with the yt_astro_analysis from source. When I try to run a simple rockstar analysis:
import sys,yt
from yt.extensions.astro_analysis.halo_analysis import HaloCatalog
yt.enable_parallelism()
es = yt.load_simulation(sys.argv[1], "Enzo", find_outputs=True)
hc = HaloCatalog(data_ds=es, finder_method='rockstar', finder_kwargs={ "num_readers": 2, "num_writers": 4})
hc.create()
When the script gets to "hc.create()", this error arises: "AttributeError: 'EnzoSimulation' object has no attribute 'kwargs'". This is coming from yt/data_objects/time_series.py. I looked at the code with our tech guy and I agree with him that kwargs may not be properly defined. Here is the full series of calls <https://pastebin.com/kDeUDpRX>.
Has anyone here run rockstar with yt and yt_astro_analysis from source? Does it work for you??
Best, Carla
Am Mo., 12. Apr. 2021 um 12:19 Uhr schrieb Britton Smith < brittonsmith@gmail.com>:
Hi Carla,
I'm sorry to hear that. The last stable releases of both yt and yt_astro_analysis should work together. If you start from a fresh conda environment and do: pip install yt pip install yt-astro-analysis
that should give you the older version. Unfortunately, I'm unable to provide a version of the yt-astro documentation from that version. You can build that locally if you do the following in a different conda environment:
git clone https://github.com/yt-project/yt_astro_analysis git checkout yt_astro_analysis-1.0.0 pip install -e .[dev] cd doc make html
If you then open the file in your yt-astro repo doc/build/html/index.html in a web browser, you'll get the old docs.
Britton
On Mon, Apr 12, 2021 at 9:46 AM C Bernhardt <bernhardt.phd@gmail.com> wrote:
Hi Britton,
Thanks for all your help. I have now tried several times and ways of installing yt from source and had no success, even our IT support couldn't make it work. Is there a way I can go to an older version of rockstar to be able to use it with the stable version of yt? I still want to resolve the original issue of not getting small enough halos ...
Thanks, Carla
Am Mo., 29. März 2021 um 17:03 Uhr schrieb Britton Smith < brittonsmith@gmail.com>:
Hi Carla,
The version of rockstar that yt uses is now the latest rockstar-galaxies, which supports multi-mass simulations. You should no longer need to create particle filters to select only the high-resolution particles.
The other error you're getting is from a recent change in yt's configuration system. If you update your source yt installation, that should take care of it. If you're not currently using a source installation, there is more information here on that:
https://yt-project.org/docs/dev/installing.html#installing-yt-from-source
Britton
On Mon, Mar 29, 2021 at 1:42 PM C Bernhardt <bernhardt.phd@gmail.com> wrote:
Hi Britton,
Thank you for helping me along with this. I don't see much in the documentation on any different approach for using rockstar through the HaloCatalog with nested grids, is there any extra treatment needed (similar to the necessary particle filters [i.e. "enzo_2_dark_matter_nested"] added to RockstarHaloFinder)?
I tried with
es = yt.simulation(sys.argv[1], "Enzo", find_outputs=True)
hc = HaloCatalog(data_ds=es, finder_method='rockstar', finder_kwargs={"num_readers": 2, "num_writers": 4})
hc.create()
and get this error:
File "/users/carla/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py", line 224, in _run
self.finder_method(self)
File "/users/carla/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 39, in __call__
return self.function(hc, *self.args, **self.kwargs)
File "/users/carla/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py", line 87, in _rockstar_method
rh = RockstarHaloFinder(ds, **finder_kwargs)
File "/users/carla/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 214, in __init__
self.runner = InlineRunner()
File "/users/carla/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py", line 44, in __init__
psize = ytcfg.get("yt", "internals", "global_parallel_size")
File "/users/carla/yt/lib/python3.8/site-packages/yt/config.py", line 137, in get
val = super(YTConfigParser, self).get(section, option, *args, **kwargs)
TypeError: get() takes 3 positional arguments but 4 were given
Thanks, Carla
Am Mi., 24. März 2021 um 12:33 Uhr schrieb Britton Smith < brittonsmith@gmail.com>:
> Hi Carla, > > Sorry, the halo finding call and import have changed. You now use > the HaloCatalog for all halo finding, including rockstar. It should now > look like this: > import yt > from yt.extensions.astro_analysis.halo_analysis import HaloCatalog > > my_sim = yt.load_simulation('enzo_tiny_cosmology/32Mpc_32.enzo', > 'Enzo') > hc = HaloCatalog(data_ds=my_sim, finder_method='rockstar', > finder_kwargs={"num_readers": 1, "num_writers": 1}) > hc.create() > > See the link below for more info > https://yt-astro-analysis.readthedocs.io/en/latest/halo_finding.html > > Britton > > On Wed, Mar 24, 2021 at 9:57 AM C Bernhardt <bernhardt.phd@gmail.com> > wrote: > >> Hi Britton, >> >> I updated my yt_astro_analysis package, but now I get this error: >> When trying to run >> >> from yt.extensions.astro_analysis.halo_finding.rockstar.api >> import RockstarHaloFinder >> >> Traceback (most recent call last): >> >> File "<stdin>", line 1, in <module> >> >> File >> "/users/carla/yt/lib/python3.8/site-packages/yt/exthook.py", line 88, in >> load_module >> >> raise ImportError('No module named %s' % fullname) >> >> ImportError: No module named >> yt.extensions.astro_analysis.halo_finding >> >> >> My first thought was that yt_astro_analysis wasn't finding my >> installation of rockstar-galaxies, but I still have the file rockstar.cfg >> with the correct path to rockstar in my cloned repository of >> yt_astro_analysis. I am using the rockstar-galaxies repository from John >> Wise, which did not throw this error before I updated my yt_astro_analysis. >> Do you use the same repository or the one from Peter Behroozi? >> >> Thanks, >> Carla >> >> Am Mo., 22. März 2021 um 17:15 Uhr schrieb Britton Smith < >> brittonsmith@gmail.com>: >> >>> Hi Carla, >>> >>> Ok, this new version was only merged on Friday, so if you >>> installed before then, you won't have it. >>> >>> Britton >>> >>> On Mon, Mar 22, 2021 at 4:09 PM C Bernhardt < >>> bernhardt.phd@gmail.com> wrote: >>> >>>> Hi Britton, >>>> >>>> Thanks for clarifying. I installed yt_astro_analysis as part of >>>> the process of installing rockstar-galaxies, so I think this shouldn't be >>>> the issue. I did not have this installed in my previous version, so I did >>>> not need to worry about older versions getting in the way. >>>> >>>> Cheers, >>>> Carla >>>> >>>> Am Mo., 22. März 2021 um 15:29 Uhr schrieb Britton Smith < >>>> brittonsmith@gmail.com>: >>>> >>>>> Hi Carla, >>>>> >>>>> Ok, great. Just to be clear, though, it's not rockstar you need >>>>> to update, but the yt_astro_analysis package. If you have cloned the >>>>> repository, you would need to go into it and do: >>>>> >>>>> git pull origin master >>>>> ./clean.sh >>>>> pip install -e . >>>>> >>>>> On Mon, Mar 22, 2021 at 2:12 PM C Bernhardt < >>>>> bernhardt.phd@gmail.com> wrote: >>>>> >>>>>> Hi Britton, >>>>>> >>>>>> Thanks for the feedback. I updated rockstar-galaxies about a >>>>>> month ago, but I can try running with fof or hop and see if I get better >>>>>> results. Thanks. >>>>>> >>>>>> Best, >>>>>> Carla >>>>>> >>>>>> Am Mo., 22. März 2021 um 14:00 Uhr schrieb Britton Smith < >>>>>> brittonsmith@gmail.com>: >>>>>> >>>>>>> Hi Carla, >>>>>>> >>>>>>> I would have expected some smaller halos, too, but I have no >>>>>>> reason to believe the halo finder is not doing what it should be. You might >>>>>>> try using fof or hop instead to see if things change. By the way, the halo >>>>>>> finders were very recently updated. I would suggest pulling in the latest >>>>>>> changes. The documentation has also been substantially improved. >>>>>>> >>>>>>> https://yt-astro-analysis.readthedocs.io/en/latest/halo_finding.html >>>>>>> >>>>>>> Britton >>>>>>> >>>>>>> On Mon, Mar 22, 2021 at 9:36 AM C Bernhardt < >>>>>>> bernhardt.phd@gmail.com> wrote: >>>>>>> >>>>>>>> Hi Britton, >>>>>>>> >>>>>>>> Thank you for your reply. Adding the keyword "min_halo_size" >>>>>>>> changed the parameter in my rockstar config file, but when I redid the >>>>>>>> analysis, the halo sizes were still larger than expected. With a Particle >>>>>>>> mass of 1.31418e+05 Msun/h and MIN_HALO_OUTPUT_SIZE = 10, I >>>>>>>> was expecting to get halos on the order of 1e7 Msun/h. The smallest halos I >>>>>>>> get now are 1.2e8 Msun/h. Is that reasonable to expect and if so, what do I >>>>>>>> need to change to see those smaller halos? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Carla >>>>>>>> >>>>>>>> Am Di., 9. März 2021 um 17:08 Uhr schrieb Britton Smith < >>>>>>>> brittonsmith@gmail.com>: >>>>>>>> >>>>>>>>> Hi Carla, >>>>>>>>> >>>>>>>>> It looks like RockstarHaloFinder accepts a keyword called >>>>>>>>> "min_halo_size" which sets the minimum particle number for halos. The >>>>>>>>> RockstarHaloFinder code writes its own config file, which is then passed to >>>>>>>>> rockstar itself, so that's why modifying that file by hand doesn't do >>>>>>>>> anything. >>>>>>>>> >>>>>>>>> Britton >>>>>>>>> >>>>>>>>> On Mon, Mar 8, 2021 at 9:29 AM C Bernhardt < >>>>>>>>> bernhardt.phd@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Dear yt and rockstar users, >>>>>>>>>> >>>>>>>>>> I recently installed the newest version of >>>>>>>>>> rockstar-galaxies as I had been using a much older version. Although the >>>>>>>>>> output reports a particle mass of 1.31418e+05 Msun/h, I don't see any halos >>>>>>>>>> smaller than 1e+08 Msun/h. I was expecting to get the minimum masses an >>>>>>>>>> order of magnitude lower. >>>>>>>>>> >>>>>>>>>> I see in the config includes MIN_HALO_OUTPUT_SIZE = 25, >>>>>>>>>> set as a default. I tried to change this parameter to 10 and restart from >>>>>>>>>> the beginning, but I run rockstar within yt with RockstarHaloFinder ( >>>>>>>>>> yt.extensions.astro_analysis.halo_finding.rockstar.api), >>>>>>>>>> but my config file was overwritten by the defaults. Is this even the right >>>>>>>>>> approach? If so, how do I restart rockstar (more detailed than just >>>>>>>>>> rh.run(restart=True), please) with customized parameters >>>>>>>>>> within yt's RockstarHaloFinder? >>>>>>>>>> >>>>>>>>>> Thanks in advance, >>>>>>>>>> -- >>>>>>>>>> Carla Bernhardt >>>>>>>>>> PhD Student >>>>>>>>>> Universität Heidelberg >>>>>>>>>> ZAH Institut für Theoretische Astrophysik >>>>>>>>>> _______________________________________________ >>>>>>>>>> yt-users mailing list -- yt-users@python.org >>>>>>>>>> To unsubscribe send an email to yt-users-leave@python.org >>>>>>>>>> https://mail.python.org/mailman3/lists/yt-users.python.org/ >>>>>>>>>> Member address: brittonsmith@gmail.com >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> yt-users mailing list -- yt-users@python.org >>>>>>>>> To unsubscribe send an email to yt-users-leave@python.org >>>>>>>>> https://mail.python.org/mailman3/lists/yt-users.python.org/ >>>>>>>>> Member address: bernhardt.phd@gmail.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Carla Bernhardt >>>>>>>> PhD Student >>>>>>>> Universität Heidelberg >>>>>>>> ZAH Institut für Theoretische Astrophysik >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> yt-users mailing list -- yt-users@python.org >>>>>>> To unsubscribe send an email to yt-users-leave@python.org >>>>>>> https://mail.python.org/mailman3/lists/yt-users.python.org/ >>>>>>> Member address: bernhardt.phd@gmail.com >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Carla Bernhardt >>>>>> PhD Student >>>>>> Universität Heidelberg >>>>>> ZAH Institut für Theoretische Astrophysik >>>>>> _______________________________________________ >>>>>> yt-users mailing list -- yt-users@python.org >>>>>> To unsubscribe send an email to yt-users-leave@python.org >>>>>> https://mail.python.org/mailman3/lists/yt-users.python.org/ >>>>>> Member address: brittonsmith@gmail.com >>>>>> >>>>> _______________________________________________ >>>>> yt-users mailing list -- yt-users@python.org >>>>> To unsubscribe send an email to yt-users-leave@python.org >>>>> https://mail.python.org/mailman3/lists/yt-users.python.org/ >>>>> Member address: bernhardt.phd@gmail.com >>>>> >>>> >>>> >>>> -- >>>> Carla Bernhardt >>>> PhD Student >>>> Universität Heidelberg >>>> ZAH Institut für Theoretische Astrophysik >>>> _______________________________________________ >>>> yt-users mailing list -- yt-users@python.org >>>> To unsubscribe send an email to yt-users-leave@python.org >>>> https://mail.python.org/mailman3/lists/yt-users.python.org/ >>>> Member address: brittonsmith@gmail.com >>>> >>> _______________________________________________ >>> yt-users mailing list -- yt-users@python.org >>> To unsubscribe send an email to yt-users-leave@python.org >>> https://mail.python.org/mailman3/lists/yt-users.python.org/ >>> Member address: bernhardt.phd@gmail.com >>> >> >> >> -- >> Carla Bernhardt >> PhD Student >> Universität Heidelberg >> ZAH Institut für Theoretische Astrophysik >> _______________________________________________ >> yt-users mailing list -- yt-users@python.org >> To unsubscribe send an email to yt-users-leave@python.org >> https://mail.python.org/mailman3/lists/yt-users.python.org/ >> Member address: brittonsmith@gmail.com >> > _______________________________________________ > yt-users mailing list -- yt-users@python.org > To unsubscribe send an email to yt-users-leave@python.org > https://mail.python.org/mailman3/lists/yt-users.python.org/ > Member address: bernhardt.phd@gmail.com >
-- Carla Bernhardt PhD Student Universität Heidelberg ZAH Institut für Theoretische Astrophysik _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: brittonsmith@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: bernhardt.phd@gmail.com
-- Carla Bernhardt PhD Student Universität Heidelberg ZAH Institut für Theoretische Astrophysik _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: brittonsmith@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: bernhardt.phd@gmail.com
-- Carla Bernhardt PhD Student Universität Heidelberg ZAH Institut für Theoretische Astrophysik _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: brittonsmith@gmail.com
_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org https://mail.python.org/mailman3/lists/yt-users.python.org/ Member address: bernhardt.phd@gmail.com
-- Carla Bernhardt PhD Student Universität Heidelberg ZAH Institut für Theoretische Astrophysik