Hi Nick, Yes, I think you are right. Looking quickly through the rockstar-galaxies source, it is doing something with the particle_mass variable, but I agree it does not seem to change the value for the individual particles. I've just added a new change to the open pull request that I think fixes this. I've added a new rockstar keyword argument called "mass_field" which will allow you to specify the field that sets the mass, instead of it being hard-coded to "particle_mass". I have confirmed this gets all the way to the particle reading, so I think this should work. In order to use this, you will have to create a new field to represent the proper mass of the particle and see that this gets added to the dataset correctly. Here is an example of how I did it: http://paste.yt-project.org/show/346/ In your case, I believe you can use the setup_function keyword directly with the DatasetSeries object, e.g. ts = yt.DatasetSeries(sets, setup_function=setup_ds) If you update your repo with the fetch/checkout commands, you should have it. NOTE: this last change modified the cython code, so you'll have to rerun "pip install -e ." for it to take effect. Hopefully, this time! Britton On Thu, Sep 16, 2021 at 2:39 PM Nick Gnedin <gnedin@fnal.gov> wrote:
Britton,
I tried your methods 1 and 2 and RS (Rockstar-Galaxies Version: 0.99.9-RC3-newA) runs with both to the end. However, I discovered a weird thing - the actual value for the particle mass has no effect. I ran it both with the default value and with the changed value, both values are correctly listed in corresponding out_0.list files: #Particle mass: 4.22991e+06 Msun/h [default value] #Particle mass: 5.02229e+06 Msun/h [assigned correct value]
However, the produced mass functions are the same in both runs, and they differ by the OmegaM/OmegaDM factor (the ratio of 5.02229e+06/4.22991e+06) from the mass function that I consider correct (obtained with the old RS (Rockstar Version: 0.99.9-RC3) as implemented in yt '3.7.dev0' in yt.analysis_modules.halo_finding.rockstar.api, which also agrees with the analytical fit from hmf.py).
Perhaps yt sets the RS particle mass correctly but feeds the actual particle data with the default mass?
Nick
On 9/10/2021 4:51 AM, Britton Smith wrote:
Hi Nick,
Thanks for sending the script; I can see why my solution didn't work. The time series object does not have a unit registry like a loaded dataset. I issued a pull request to yt_astro_analysis which should fix this by allowing you to do what you should have been able to all along. https://github.com/yt-project/yt_astro_analysis/pull/84 < https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_yt-2Dproject_yt-5Fastro-5Fanalysis_pull_84&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=B-7bLI_gVkNbVTTL6WLE_jBPCCUxcykB81atg_VPNZk&s=W5CyZgXekUT6Tj6afPdEBapaE3i0lR5cnwbDmUCiaoU&e=
With this PR, you can now give the particle mass in three ways: 1. as a regular float. This will assume units of Msun/h. This was the original way that was broken and should now be fixed again. 2. as a tuple of (value, unit) for example (5.02229e+06,"Msun/h"). 3. as a unyt_quantity
To make the 3rd option work, you would do ts[0].quan(5.02229e+06,"Msun/h"), since ts[0] will be the first dataset. The first two should work as is.
Let me know if this works and I'll get the PR merged.
Britton
On Thu, Sep 9, 2021 at 4:25 PM Nick Gnedin <gnedin@fnal.gov <mailto:gnedin@fnal.gov>> wrote:
Britton,
Actually, your suggestion does not work:
File "rs-hfc.py", line 19, in <module> ts.unit_registry.add("h", 0.6814, dimensionless) P010 yt : [ERROR ] 2021-09-09 10:23:11,397 AttributeError: 'DatasetSeries' object has no attribute 'unit_registry'
Here is my script (based on your suggestion from a couple of years back):
sets = glob.glob(root+"/"+dir+"/rei20c1_a*/*.art") sets.sort() ts = yt.DatasetSeries(sets) ts.unit_registry.add("h", 0.6814, dimensionless) rh = HaloCatalog(data_ds=ts, finder_method='rockstar', finder_kwargs={"num_readers":n, "num_writers":4, "outbase":root+"/"+dir+"/rstest1", "particle_type":"N-BODY_0", "particle_mass":yt.YTQuantity(5.02229e+06,"Msun/h")}) rh.create() del rh del ts
On 9/9/2021 9:01 AM, Britton Smith wrote: > Hi Nick, > > Rockstar wants particle masses in units of Msun/h, so it's trying to do > that conversion. If your dataset doesn't have h defined, you can do > something like this: > > from unyt.dimensions import dimensionless > > ds = yt.load(...) > ds.unit_registry.add("h", VALUE, dimensionless) > > Hopefully, that does the trick. > > Britton > > > On Tue, Sep 7, 2021 at 8:57 PM Nick Gnedin <gnedin@fnal.gov <mailto:gnedin@fnal.gov> > <mailto:gnedin@fnal.gov <mailto:gnedin@fnal.gov>>> wrote: > > > Britton, > > You were right, when I updated yt and yt_astro_analysis to the latest > versions (and downgraded matplotlib to circumvent the matplotlib._png > bug), Rockstar now runs, but I encounter a unit error: > > File "rs-hfc.py", line 21, in <module> > rh.create() > File "/home/gnedin/ART/yt_astro_analysis/yt_astro_analysis > ... > File >
"/data/gnedin/soft/anaconda3/envs/yt/lib/python3.8/site-packages/unyt/unit_object.py",
> > line 972, in _get_unit_data_from_expr > return _lookup_unit_symbol(unit_expr.name <
> <
https://urldefense.proofpoint.com/v2/url?u=http-3A__unit-5Fexpr.name&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=cMSuy1wk_MnvGpsaSr0UGNuBY7jsbP9G0peOd1GSxxo&s=fQnOScv-s6bWFuDrdnIHjeC3QNVz06iWIzq9luu8NMY&e= < https://urldefense.proofpoint.com/v2/url?u=http-3A__unit-5Fexpr.name&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=cMSuy1wk_MnvGpsaSr0UGNuBY7jsbP9G0peOd1GSxxo&s=fQnOScv-s6bWFuDrdnIHjeC3QNVz06iWIzq9luu8NMY&e=
, > unit_symbol_lut) > File >
"/data/gnedin/soft/anaconda3/envs/yt/lib/python3.8/site-packages/unyt/unit_registry.py",
> > line 330, in _lookup_unit_symbol > raise UnitParseError( > P011 yt : [ERROR ] 2021-09-07 14:39:16,965 UnitParseError: Could not > find unit symbol 'h' in the provided symbols. > > I removed any reference to h in my script, so I am flabbergasted by > that > error. Here is my script: > > import yt, yt.funcs > import glob, sys, os.path > yt.enable_parallelism() > from yt.extensions.astro_analysis.halo_analysis import HaloCatalog > > root = > "/data/gnedin/REI/D/Cai.B20.N128L2.sf=1_uv=0.15_bw=10_res=100.WC1" > n = 16 > dirs = ( "A", ) > for dir in dirs: > sets = glob.glob(root+"/"+dir+"/rei20c1_a*/*.art") > sets.sort() > ts = yt.DatasetSeries(sets) > rh = HaloCatalog(data_ds=ts, finder_method='rockstar', > finder_kwargs={"num_readers":n, "num_writers":4, > "outbase":root+"/"+dir+"/rscor", "particle_type":"N-BODY_0", > "particle_mass":yt.YTQuantity(5.02229e+06/0.6814,"Msun")}) > rh.create() > del rh > del ts > ## > > > Nick > > > On 9/6/2021 10:00 AM, Britton Smith wrote: > > Hi Nick, > > > > I was not able to reproduce this error using the latest dev yt and > > yt_astro_analysis installed. The call to ytcfg.get looks > correct to me. > > The error looks to me like the older yt config parser syntax, so > the yt > > installation may be out of date. > > > > Britton > > > > On Thu, Sep 2, 2021 at 1:40 AM Nick Gnedin <gnedin@fnal.gov <mailto:gnedin@fnal.gov> > <mailto:gnedin@fnal.gov <mailto:gnedin@fnal.gov>> > > <mailto:gnedin@fnal.gov <mailto:gnedin@fnal.gov> <mailto:gnedin@fnal.gov <mailto:gnedin@fnal.gov>>>> wrote: > > > > > > Folks, > > > > I am trying to run rockstar using your instructions on > > > > >
https://yt-astro-analysis.readthedocs.io/en/latest/halo_finding.html#rocksta...
<
> <
https://urldefense.proofpoint.com/v2/url?u=https-3A__yt-2Dastro-2Danalysis.readthedocs.io_en_latest_halo-5Ffinding.html-23rockstar-2Dgalaxies&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=cMSuy1wk_MnvGpsaSr0UGNuBY7jsbP9G0peOd1GSxxo&s=9YZq_c61MCYtqrvrjbgYZdjNx1lLYQheFyuRa6YIf_Q&e= < https://urldefense.proofpoint.com/v2/url?u=https-3A__yt-2Dastro-2Danalysis.readthedocs.io_en_latest_halo-5Ffinding.html-23rockstar-2Dgalaxies&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=cMSuy1wk_MnvGpsaSr0UGNuBY7jsbP9G0peOd1GSxxo&s=9YZq_c61MCYtqrvrjbgYZdjNx1lLYQheFyuRa6YIf_Q&e=
> > > < https://urldefense.proofpoint.com/v2/url?u=https-3A__yt-2Dastro-2Danalysis.readthedocs.io_en_latest_halo-5Ffinding.html-23rockstar-2Dgalaxies&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=zkLfKb5t7rANzwoPnoBa3eM_eYdF_1zIuOiSO4ALaDY&e= < https://urldefense.proofpoint.com/v2/url?u=https-3A__yt-2Dastro-2Danalysis.readthedocs.io_en_latest_halo-5Ffinding.html-23rockstar-2Dgalaxies&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=zkLfKb5t7rANzwoPnoBa3eM_eYdF_1zIuOiSO4ALaDY&e=> < https://urldefense.proofpoint.com/v2/url?u=https-3A__yt-2Dastro-2Danalysis.readthedocs.io_en_latest_halo-5Ffinding.html-23rockstar-2Dgalaxies&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=zkLfKb5t7rANzwoPnoBa3eM_eYdF_1zIuOiSO4ALaDY&e= < https://urldefense.proofpoint.com/v2/url?u=https-3A__yt-2Dastro-2Danalysis.readthedocs.io_en_latest_halo-5Ffinding.html-23rockstar-2Dgalaxies&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=zkLfKb5t7rANzwoPnoBa3eM_eYdF_1zIuOiSO4ALaDY&e=
> > > > and I am getting the following error almost immediately: > > > > Traceback (most recent call last): > > File "rs-hfc.py", line 24, in <module> > > hc.create() > > File > > >
"/home/gnedin/ART/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py",
> > > > line 333, in create > > self._run(save_halos, save_output, > > File > > >
"/data/gnedin/soft/anaconda3/lib/python3.8/site-packages/yt/utilities/parallel_tools/parallel_analysis_interface.py",
> > > > line 301, in barrierize > > return func(*args, **kwargs) > > File > > >
"/home/gnedin/ART/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_catalog.py",
> > > > line 224, in _run > > self.finder_method(self) > > File > > >
"/home/gnedin/ART/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py",
> > > > line 39, in __call__ > > return self.function(hc, *self.args,
**self.kwargs)
> > File > > >
"/home/gnedin/ART/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_catalog/halo_finding_methods.py",
> > > > line 87, in _rockstar_method > > rh = RockstarHaloFinder(ds, **finder_kwargs) > > File > > >
"/home/gnedin/ART/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py",
> > > > line 214, in __init__ > > self.runner = InlineRunner() > > File > > >
"/home/gnedin/ART/yt_astro_analysis/yt_astro_analysis/halo_analysis/halo_finding/rockstar/rockstar.py",
> > > > line 44, in __init__ > > psize = ytcfg.get("yt", "internals", > "global_parallel_size") > > File > > >
"/data/gnedin/soft/anaconda3/lib/python3.8/site-packages/yt/config.py",
> > line 137, in get > > val = super(YTConfigParser, self).get(section, option, > *args, > > **kwargs) > > TypeError: get() takes 3 positional arguments but 4 were given > > > > Nick Gnedin > > > > > > -- > > This email has been checked for viruses by AVG. > > https://www.avg.com <
> <
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.avg.com&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=cMSuy1wk_MnvGpsaSr0UGNuBY7jsbP9G0peOd1GSxxo&s=9YNi-CdZn2SuCsqFpMThMQD1XsCvxXHPeWWRh7JTOQk&e= < https://urldefense.proofpoint.com/v2/url?u=https-3A__www.avg.com&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=cMSuy1wk_MnvGpsaSr0UGNuBY7jsbP9G0peOd1GSxxo&s=9YNi-CdZn2SuCsqFpMThMQD1XsCvxXHPeWWRh7JTOQk&e=
> > > < https://urldefense.proofpoint.com/v2/url?u=https-3A__www.avg.com&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=a1Qa5dFGi8B4e2_dD7eGB_nlkGuO5Zvb9dpFAQBr8hk&e= < https://urldefense.proofpoint.com/v2/url?u=https-3A__www.avg.com&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=a1Qa5dFGi8B4e2_dD7eGB_nlkGuO5Zvb9dpFAQBr8hk&e=> < https://urldefense.proofpoint.com/v2/url?u=https-3A__www.avg.com&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=a1Qa5dFGi8B4e2_dD7eGB_nlkGuO5Zvb9dpFAQBr8hk&e= < https://urldefense.proofpoint.com/v2/url?u=https-3A__www.avg.com&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=a1Qa5dFGi8B4e2_dD7eGB_nlkGuO5Zvb9dpFAQBr8hk&e=
> > > > _______________________________________________ > > yt-users mailing list -- yt-users@python.org <mailto:yt-users@python.org> > <mailto:yt-users@python.org <mailto:yt-users@python.org>> > > <mailto:yt-users@python.org <mailto:yt-users@python.org> <mailto:yt-users@python.org <mailto:yt-users@python.org>>> > > To unsubscribe send an email to yt-users-leave@python.org <mailto:yt-users-leave@python.org> > <mailto:yt-users-leave@python.org <mailto:yt-users-leave@python.org>> > > <mailto:yt-users-leave@python.org <mailto:yt-users-leave@python.org> > <mailto:yt-users-leave@python.org <mailto:yt-users-leave@python.org>>> > > https://mail.python.org/mailman3/lists/yt-users.python.org/ < https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_yt-2Dusers.python.org_&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=B-7bLI_gVkNbVTTL6WLE_jBPCCUxcykB81atg_VPNZk&s=V_j108N1sOS-2fsepRgM7c2WSGWnQ19kfh7mXnJX3WI&e=
> < https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_yt-2Dusers.python.org_&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=cMSuy1wk_MnvGpsaSr0UGNuBY7jsbP9G0peOd1GSxxo&s=AKRiliuuNdwhHlh4Xs3JUvVlOLQH_7xAm_Cqy8NgNfI&e= < https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_yt-2Dusers.python.org_&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=cMSuy1wk_MnvGpsaSr0UGNuBY7jsbP9G0peOd1GSxxo&s=AKRiliuuNdwhHlh4Xs3JUvVlOLQH_7xAm_Cqy8NgNfI&e=
> > > < https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_yt-2Dusers.python.org_&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=bV-HFqSpv91LxTtwsbbtzAXW2cG8xTTUUlnwZ9f0MKI&e= < https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_yt-2Dusers.python.org_&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=bV-HFqSpv91LxTtwsbbtzAXW2cG8xTTUUlnwZ9f0MKI&e=> < https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_yt-2Dusers.python.org_&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=bV-HFqSpv91LxTtwsbbtzAXW2cG8xTTUUlnwZ9f0MKI&e= < https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman3_lists_yt-2Dusers.python.org_&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=91a-YxIolKtcnV0IqQEf1J21gbPSR5m03nK26IzbbmU&s=bV-HFqSpv91LxTtwsbbtzAXW2cG8xTTUUlnwZ9f0MKI&e=
> > Member address: brittonsmith@gmail.com <mailto:brittonsmith@gmail.com> > <mailto:brittonsmith@gmail.com <mailto:brittonsmith@gmail.com>> <mailto:brittonsmith@gmail.com <mailto:brittonsmith@gmail.com> > <mailto:brittonsmith@gmail.com <mailto:brittonsmith@gmail.com
> > >
-- This email has been checked for viruses by AVG. https://www.avg.com < https://urldefense.proofpoint.com/v2/url?u=https-3A__www.avg.com&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=7jSLZBQB_2gqGQ3elA922w&m=B-7bLI_gVkNbVTTL6WLE_jBPCCUxcykB81atg_VPNZk&s=qySzAAeYpHXyTxMtF86gyWB3S8Xov3hDRCYG1DnlRZw&e=