[RAMSES] trouble depositing a particle field which has been filtered based on a derived field?
Hi, I am using YT to process the outputs of a non-standard RAMSES-RT simulation (non-standard in the sense that it uses custom particle/hydro variables), and it works excellently for almost everything. However I am now trying to make plots of the surface star formation rate density, and have hit an obstacle which I can't seem to surmount. Specifically I want to estimate the SFRD by looking at all star particles that formed recently, which requires correctly processing the particle birth times. This seems to be a tricky business in RAMSES, as depending on the type of the simulation run (cosmological/RT/etc) the particle time units are different. Luckily for this particular case I have separate output files which contain the star particle formation times in the correct physical units (e.g. Gyr). What I would like to do is load these in, override the "star_age" field of the star particles, and then use that to filter out the star particles that formed recently. I have tried this and it seems to work well, up until I want to deposit this filtered particle field. At this point the code fails with the NeedsGridType and YTIllDefinedFilter exceptions. Here's a minimal working example: ################################################################################ #!/usr/bin/env python3 import yt from yt.data_objects.particle_filters import add_particle_filter from yt import YTArray import numpy as np from scipy.io import FortranFile # Load outputs of non-standard RAMSES-RT run particles = [("particle_birth_time", "d"), ("particle_metallicity", "d"), ("particle_initial_mass", "d")] fields = ["Density", "x-velocity", "y-velocity", "z-velocity", "nontp", "Pressure", "Metallicity", "HI", "HII", "HeII", "HeIII", "ivar_ref"] ds = yt.load("output_00050/info_00050.txt", fields=fields, extra_particle_fields=particles) # Add initial filter to distinguish star particles def _stars(pfilter, data): return data[("io", "conformal_birth_time")] != 0 # star particles add_particle_filter("stars", function=_stars, filtered_type="all", requires=["conformal_birth_time"]) ds.add_particle_filter("stars") # Get star formation times in Gyr # (Based on pynbody's get_tform, read in star formation time from converted # output) def convert_times(): ages = np.array([]) base = "output_00050/birth_00050." ncpu = 1024 nstar = 0 for i in range(ncpu): fname = base + "out{:05d}".format(i + 1) with FortranFile(fname) as birth_file: read_ages = birth_file.read_reals(np.float64) new = np.where(read_ages > 0)[0] # star particles only nstar += len(new) ages = np.append(ages, read_ages[new]) return nstar, ages nstar, stellar_ages = convert_times() # Override existing ("stars", "star_age") field with correct values def _stellar_age(field, data): return data.ds.current_time - YTArray(stellar_ages, "Gyr") ds.add_field(("stars", "star_age"), function=_stellar_age, units="Myr", particle_type=True, force_override=True) # Filter stars born within past 10 Myr def _new_stars(pfilter, data): age = data[(pfilter.filtered_type, "star_age")].in_units("Myr") mask = np.logical_and(age.in_units("Myr") <= 10, age > 0.0) return mask add_particle_filter("new_stars", function=_new_stars, filtered_type="stars", requires=["star_age"]) ds.add_particle_filter("new_stars") # Create new deposit field with estimate of recent star formation rate density def _star_formation(field, data): stellar_density = data[("deposit", "new_stars_density")] stellar_age = ds.quan(10, "Myr") return stellar_density / stellar_age ds.add_field(("deposit", "sfr"), function=_star_formation, units="Msun/kpc**3/yr", sampling_type="cell") # Define a region of interest to project over centre = ds.arr([0.5, 0.5, 0.5], "code_length") fwidth = ds.quan(0.1, "code_length") left_edge = centre - (fwidth/2)*1.1 right_edge = centre + (fwidth/2)*1.1 box = ds.box(left_edge=left_edge, right_edge=right_edge) proj = ds.proj(field=("deposit", "sfr"), axis="z", weight_field=None, center=centre, data_source=box) ################################################################################ The short summary of the error output is as follow: ################################################################################ yt.fields.field_exceptions.NeedsGridType: (0, None) During handling of the above exception, another exception occurred: yt.utilities.exceptions.YTIllDefinedFilter: Filter '<yt.data_objects.particle_filters.ParticleFilter object at 0x7f04591b2390>' ill-defined. Applied to shape (0, 3) but is shape (14150626,). ################################################################################ I was wondering if I am making a mistake somewhere, or if perhaps I'm simply going about this in the wrong way? The issue seems to revolve around using data[("deposit", "new_stars_density")]; if for example I instead use data[("deposit", "stars_density")] in the _star_formation() function then it all works fine. Many thanks in advance for any help! Lewis
Hi Lewis, I haven’t looked at your code in detail yet, but this is complicated enough that it would help me to understand what’s going on if you can make your script locally runnable by me so I can poke at it with a debugger. That means making your script use one of the data files we have for this purpose at yt-project.org/data or by uploading one of your outputs to a publicly visible location: $ yt upload some_file.tar.gz You’ll need to share the link the above command prints out. If you’re uncomfortable sharing your data publicly we can try off-list. Nathan On Tue, Feb 5, 2019 at 7:13 AM <lhw28@cam.ac.uk> wrote:
Hi,
I am using YT to process the outputs of a non-standard RAMSES-RT simulation (non-standard in the sense that it uses custom particle/hydro variables), and it works excellently for almost everything. However I am now trying to make plots of the surface star formation rate density, and have hit an obstacle which I can't seem to surmount.
Specifically I want to estimate the SFRD by looking at all star particles that formed recently, which requires correctly processing the particle birth times. This seems to be a tricky business in RAMSES, as depending on the type of the simulation run (cosmological/RT/etc) the particle time units are different. Luckily for this particular case I have separate output files which contain the star particle formation times in the correct physical units (e.g. Gyr). What I would like to do is load these in, override the "star_age" field of the star particles, and then use that to filter out the star particles that formed recently. I have tried this and it seems to work well, up until I want to deposit this filtered particle field. At this point the code fails with the NeedsGridType and YTIllDefinedFilter exceptions.
Here's a minimal working example:
################################################################################ #!/usr/bin/env python3 import yt from yt.data_objects.particle_filters import add_particle_filter from yt import YTArray import numpy as np from scipy.io import FortranFile
# Load outputs of non-standard RAMSES-RT run particles = [("particle_birth_time", "d"), ("particle_metallicity", "d"), ("particle_initial_mass", "d")] fields = ["Density", "x-velocity", "y-velocity", "z-velocity", "nontp", "Pressure", "Metallicity", "HI", "HII", "HeII", "HeIII", "ivar_ref"] ds = yt.load("output_00050/info_00050.txt", fields=fields, extra_particle_fields=particles)
# Add initial filter to distinguish star particles def _stars(pfilter, data): return data[("io", "conformal_birth_time")] != 0 # star particles add_particle_filter("stars", function=_stars, filtered_type="all", requires=["conformal_birth_time"]) ds.add_particle_filter("stars")
# Get star formation times in Gyr # (Based on pynbody's get_tform, read in star formation time from converted # output) def convert_times(): ages = np.array([]) base = "output_00050/birth_00050." ncpu = 1024 nstar = 0 for i in range(ncpu): fname = base + "out{:05d}".format(i + 1) with FortranFile(fname) as birth_file: read_ages = birth_file.read_reals(np.float64) new = np.where(read_ages > 0)[0] # star particles only nstar += len(new) ages = np.append(ages, read_ages[new]) return nstar, ages nstar, stellar_ages = convert_times()
# Override existing ("stars", "star_age") field with correct values def _stellar_age(field, data): return data.ds.current_time - YTArray(stellar_ages, "Gyr") ds.add_field(("stars", "star_age"), function=_stellar_age, units="Myr", particle_type=True, force_override=True)
# Filter stars born within past 10 Myr def _new_stars(pfilter, data): age = data[(pfilter.filtered_type, "star_age")].in_units("Myr") mask = np.logical_and(age.in_units("Myr") <= 10, age > 0.0) return mask add_particle_filter("new_stars", function=_new_stars, filtered_type="stars", requires=["star_age"]) ds.add_particle_filter("new_stars")
# Create new deposit field with estimate of recent star formation rate density def _star_formation(field, data): stellar_density = data[("deposit", "new_stars_density")] stellar_age = ds.quan(10, "Myr") return stellar_density / stellar_age ds.add_field(("deposit", "sfr"), function=_star_formation, units="Msun/kpc**3/yr", sampling_type="cell")
# Define a region of interest to project over centre = ds.arr([0.5, 0.5, 0.5], "code_length") fwidth = ds.quan(0.1, "code_length") left_edge = centre - (fwidth/2)*1.1 right_edge = centre + (fwidth/2)*1.1 box = ds.box(left_edge=left_edge, right_edge=right_edge)
proj = ds.proj(field=("deposit", "sfr"), axis="z", weight_field=None, center=centre, data_source=box)
################################################################################
The short summary of the error output is as follow:
################################################################################ yt.fields.field_exceptions.NeedsGridType: (0, None) During handling of the above exception, another exception occurred: yt.utilities.exceptions.YTIllDefinedFilter: Filter '<yt.data_objects.particle_filters.ParticleFilter object at 0x7f04591b2390>' ill-defined. Applied to shape (0, 3) but is shape (14150626,).
################################################################################
I was wondering if I am making a mistake somewhere, or if perhaps I'm simply going about this in the wrong way? The issue seems to revolve around using data[("deposit", "new_stars_density")]; if for example I instead use data[("deposit", "stars_density")] in the _star_formation() function then it all works fine.
Many thanks in advance for any help! Lewis _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
I just looked over the code quickly and don't see anything obviously wrong just reading the code. It would also help to see the full traceback along with the error though. On Tue, Feb 5, 2019 at 8:01 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi Lewis,
I haven’t looked at your code in detail yet, but this is complicated enough that it would help me to understand what’s going on if you can make your script locally runnable by me so I can poke at it with a debugger.
That means making your script use one of the data files we have for this purpose at yt-project.org/data or by uploading one of your outputs to a publicly visible location:
$ yt upload some_file.tar.gz
You’ll need to share the link the above command prints out.
If you’re uncomfortable sharing your data publicly we can try off-list.
Nathan
On Tue, Feb 5, 2019 at 7:13 AM <lhw28@cam.ac.uk> wrote:
Hi,
I am using YT to process the outputs of a non-standard RAMSES-RT simulation (non-standard in the sense that it uses custom particle/hydro variables), and it works excellently for almost everything. However I am now trying to make plots of the surface star formation rate density, and have hit an obstacle which I can't seem to surmount.
Specifically I want to estimate the SFRD by looking at all star particles that formed recently, which requires correctly processing the particle birth times. This seems to be a tricky business in RAMSES, as depending on the type of the simulation run (cosmological/RT/etc) the particle time units are different. Luckily for this particular case I have separate output files which contain the star particle formation times in the correct physical units (e.g. Gyr). What I would like to do is load these in, override the "star_age" field of the star particles, and then use that to filter out the star particles that formed recently. I have tried this and it seems to work well, up until I want to deposit this filtered particle field. At this point the code fails with the NeedsGridType and YTIllDefinedFilter exceptions.
Here's a minimal working example:
################################################################################ #!/usr/bin/env python3 import yt from yt.data_objects.particle_filters import add_particle_filter from yt import YTArray import numpy as np from scipy.io import FortranFile
# Load outputs of non-standard RAMSES-RT run particles = [("particle_birth_time", "d"), ("particle_metallicity", "d"), ("particle_initial_mass", "d")] fields = ["Density", "x-velocity", "y-velocity", "z-velocity", "nontp", "Pressure", "Metallicity", "HI", "HII", "HeII", "HeIII", "ivar_ref"] ds = yt.load("output_00050/info_00050.txt", fields=fields, extra_particle_fields=particles)
# Add initial filter to distinguish star particles def _stars(pfilter, data): return data[("io", "conformal_birth_time")] != 0 # star particles add_particle_filter("stars", function=_stars, filtered_type="all", requires=["conformal_birth_time"]) ds.add_particle_filter("stars")
# Get star formation times in Gyr # (Based on pynbody's get_tform, read in star formation time from converted # output) def convert_times(): ages = np.array([]) base = "output_00050/birth_00050." ncpu = 1024 nstar = 0 for i in range(ncpu): fname = base + "out{:05d}".format(i + 1) with FortranFile(fname) as birth_file: read_ages = birth_file.read_reals(np.float64) new = np.where(read_ages > 0)[0] # star particles only nstar += len(new) ages = np.append(ages, read_ages[new]) return nstar, ages nstar, stellar_ages = convert_times()
# Override existing ("stars", "star_age") field with correct values def _stellar_age(field, data): return data.ds.current_time - YTArray(stellar_ages, "Gyr") ds.add_field(("stars", "star_age"), function=_stellar_age, units="Myr", particle_type=True, force_override=True)
# Filter stars born within past 10 Myr def _new_stars(pfilter, data): age = data[(pfilter.filtered_type, "star_age")].in_units("Myr") mask = np.logical_and(age.in_units("Myr") <= 10, age > 0.0) return mask add_particle_filter("new_stars", function=_new_stars, filtered_type="stars", requires=["star_age"]) ds.add_particle_filter("new_stars")
# Create new deposit field with estimate of recent star formation rate density def _star_formation(field, data): stellar_density = data[("deposit", "new_stars_density")] stellar_age = ds.quan(10, "Myr") return stellar_density / stellar_age ds.add_field(("deposit", "sfr"), function=_star_formation, units="Msun/kpc**3/yr", sampling_type="cell")
# Define a region of interest to project over centre = ds.arr([0.5, 0.5, 0.5], "code_length") fwidth = ds.quan(0.1, "code_length") left_edge = centre - (fwidth/2)*1.1 right_edge = centre + (fwidth/2)*1.1 box = ds.box(left_edge=left_edge, right_edge=right_edge)
proj = ds.proj(field=("deposit", "sfr"), axis="z", weight_field=None, center=centre, data_source=box)
################################################################################
The short summary of the error output is as follow:
################################################################################ yt.fields.field_exceptions.NeedsGridType: (0, None) During handling of the above exception, another exception occurred: yt.utilities.exceptions.YTIllDefinedFilter: Filter '<yt.data_objects.particle_filters.ParticleFilter object at 0x7f04591b2390>' ill-defined. Applied to shape (0, 3) but is shape (14150626,).
################################################################################
I was wondering if I am making a mistake somewhere, or if perhaps I'm simply going about this in the wrong way? The issue seems to revolve around using data[("deposit", "new_stars_density")]; if for example I instead use data[("deposit", "stars_density")] in the _star_formation() function then it all works fine.
Many thanks in advance for any help! Lewis _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
Hi Nathan, Thanks a lot for looking at the code. I will have a look at your public datasets at http://yt-project.org/data and try running it on them, as well as uploading a locally runnable version of the script. In the meantime the full traceback is: Traceback (most recent call last): File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 308, in _generate_fluid_field finfo.check_available(gen_obj) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/derived_field.py", line 204, in check_available validator(data) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/derived_field.py", line 470, in __call__ raise NeedsGridType(self.ghost_zones,self.fields) yt.fields.field_exceptions.NeedsGridType: (0, None) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "yt_issue.py", line 74, in <module> center=centre, data_source=box) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/construction_data_containers.py", line 270, in __init__ self.get_data(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/construction_data_containers.py", line 344, in get_data self._handle_chunk(chunk, fields, tree) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/construction_data_containers.py", line 450, in _handle_chunk d = chunk[field] * dl File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 255, in __getitem__ self.get_data(f) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1382, in get_data self._generate_fields(fields_to_generate) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1402, in _generate_fields fd = self._generate_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 292, in _generate_field tr = self._generate_fluid_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 310, in _generate_fluid_field rv = self._generate_spatial_fluid(field, ngt_exception.ghost_zones) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 330, in _generate_spatial_fluid ind += o.select(self.selector, self[field], rv, ind) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 255, in __getitem__ self.get_data(f) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1382, in get_data self._generate_fields(fields_to_generate) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1402, in _generate_fields fd = self._generate_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 292, in _generate_field tr = self._generate_fluid_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 312, in _generate_fluid_field rv = finfo(gen_obj) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/derived_field.py", line 256, in __call__ dd = self._function(self, data) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/particle_fields.py", line 133, in particle_density pos = data[ptype, coord_name].convert_to_units("code_length") File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/octree_subset.py", line 73, in __getitem__ tr = super(OctreeSubset, self).__getitem__(key) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 255, in __getitem__ self.get_data(f) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1325, in get_data self.get_data(apply_fields[filter_type]) File "/opt/ioa/software/anaconda/anaconda36/envs/20181213_2_py36/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/particle_filters.py", line 53, in apply raise YTIllDefinedFilter(self, tr.shape, filter.shape) yt.utilities.exceptions.YTIllDefinedFilter: Filter '<yt.data_objects.particle_filters.ParticleFilter object at 0x7f04591b2390>' ill-defined. Applied to shape (0, 3) but is shape (14150626,). Thanks again, Lewis
Hi again Nathan, I have modified my script to run on the output_00080 cosmological RAMSES run from http://yt-project.org/data/. The script is uploaded with link: http://use.yt/upload/9d91ea29. When I run this example I get a similar traceback: Traceback (most recent call last): File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 308, in _generate_fluid_field finfo.check_available(gen_obj) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/derived_field.py", line 204, in check_available validator(data) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/derived_field.py", line 470, in __call__ raise NeedsGridType(self.ghost_zones,self.fields) yt.fields.field_exceptions.NeedsGridType: (0, None) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "yt_issue.py", line 58, in <module> center=centre, data_source=box) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/construction_data_containers.py", line 270, in __init__ self.get_data(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/construction_data_containers.py", line 344, in get_data self._handle_chunk(chunk, fields, tree) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/construction_data_containers.py", line 450, in _handle_chunk d = chunk[field] * dl File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 255, in __getitem__ self.get_data(f) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1382, in get_data self._generate_fields(fields_to_generate) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1402, in _generate_fields fd = self._generate_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 292, in _generate_field tr = self._generate_fluid_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 310, in _generate_fluid_field rv = self._generate_spatial_fluid(field, ngt_exception.ghost_zones) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 330, in _generate_spatial_fluid ind += o.select(self.selector, self[field], rv, ind) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 255, in __getitem__ self.get_data(f) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1382, in get_data self._generate_fields(fields_to_generate) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1402, in _generate_fields fd = self._generate_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 292, in _generate_field tr = self._generate_fluid_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 312, in _generate_fluid_field rv = finfo(gen_obj) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/derived_field.py", line 256, in __call__ dd = self._function(self, data) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/particle_fields.py", line 133, in particle_density pos = data[ptype, coord_name].convert_to_units("code_length") File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/octree_subset.py", line 73, in __getitem__ tr = super(OctreeSubset, self).__getitem__(key) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 255, in __getitem__ self.get_data(f) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1325, in get_data self.get_data(apply_fields[filter_type]) File "/opt/ioa/software/anaconda/anaconda36/envs/20181213_2_py36/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/particle_filters.py", line 53, in apply raise YTIllDefinedFilter(self, tr.shape, filter.shape) yt.utilities.exceptions.YTIllDefinedFilter: Filter '<yt.data_objects.particle_filters.ParticleFilter object at 0x7f25c23609e8>' ill-defined. Applied to shape (1175, 3) but is shape (31990,). I realised I never mentioned what version of YT I am using. In case it's important, it's 3.5.0. (And I'm using Python 3.6.3). Thanks for your help, Lewis
OK, I see what's happening. In your script you're reading the ages from a file with a fixed shape and then passing that fixed shape array back to yt whenever yt calls the _stellar_age field function. That doesn't work because yt field functions operate on chunks of the dataset, not the full dataset, so in order for your field function to work, you need a way to map from particle IDs in the original ramses output file to the particle indexes in the data file you're reading. The error you're getting is telling you that the size of the chunk (1170 particles) does not agree with the size of the stellar_age field returned by your field function (31990). I'm not sure how you're generating your original fortran file so I don't know if this will necessarily work for you, but I was able to get your output_00080 test script to work by modifying your `_stellar_age` function to look like this: def _stellar_age(field, data): data_ids = data['stars', 'particle_identity'] _, mask, _ = np.intersect1d(ids, data_ids, return_indices=True) return YTArray(ages[mask], "Myr") ds.add_field(("stars", "star_age"), function=_stellar_age, units="Myr", particle_type=True, force_override=True) See the documentation for intersect1d to see what I'm doing here to get the indices of the overlaps: https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.intersect1... On Wed, Feb 6, 2019 at 3:40 AM <lhw28@cam.ac.uk> wrote:
Hi again Nathan,
I have modified my script to run on the output_00080 cosmological RAMSES run from http://yt-project.org/data/. The script is uploaded with link: http://use.yt/upload/9d91ea29.
When I run this example I get a similar traceback:
Traceback (most recent call last): File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 308, in _generate_fluid_field finfo.check_available(gen_obj) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/derived_field.py", line 204, in check_available validator(data) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/derived_field.py", line 470, in __call__ raise NeedsGridType(self.ghost_zones,self.fields) yt.fields.field_exceptions.NeedsGridType: (0, None)
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "yt_issue.py", line 58, in <module> center=centre, data_source=box) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/construction_data_containers.py", line 270, in __init__ self.get_data(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/construction_data_containers.py", line 344, in get_data self._handle_chunk(chunk, fields, tree) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/construction_data_containers.py", line 450, in _handle_chunk d = chunk[field] * dl File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 255, in __getitem__ self.get_data(f) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1382, in get_data self._generate_fields(fields_to_generate) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1402, in _generate_fields fd = self._generate_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 292, in _generate_field tr = self._generate_fluid_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 310, in _generate_fluid_field rv = self._generate_spatial_fluid(field, ngt_exception.ghost_zones) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 330, in _generate_spatial_fluid ind += o.select(self.selector, self[field], rv, ind) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 255, in __getitem__ self.get_data(f) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1382, in get_data self._generate_fields(fields_to_generate) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1402, in _generate_fields fd = self._generate_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 292, in _generate_field tr = self._generate_fluid_field(field) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 312, in _generate_fluid_field rv = finfo(gen_obj) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/derived_field.py", line 256, in __call__ dd = self._function(self, data) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/fields/particle_fields.py", line 133, in particle_density pos = data[ptype, coord_name].convert_to_units("code_length") File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/octree_subset.py", line 73, in __getitem__ tr = super(OctreeSubset, self).__getitem__(key) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 255, in __getitem__ self.get_data(f) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1325, in get_data self.get_data(apply_fields[filter_type]) File "/opt/ioa/software/anaconda/anaconda36/envs/20181213_2_py36/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/lhw28/.local/lib/python3.6/site-packages/yt/data_objects/particle_filters.py", line 53, in apply raise YTIllDefinedFilter(self, tr.shape, filter.shape) yt.utilities.exceptions.YTIllDefinedFilter: Filter '<yt.data_objects.particle_filters.ParticleFilter object at 0x7f25c23609e8>' ill-defined. Applied to shape (1175, 3) but is shape (31990,).
I realised I never mentioned what version of YT I am using. In case it's important, it's 3.5.0. (And I'm using Python 3.6.3).
Thanks for your help, Lewis _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
Hi Nathan, Ah thank you! That makes sense, I was naively thinking in terms of the whole dataset, not in terms of chunks. I made the changes you suggested to my original code and it works perfectly now, filtering as expected! Thanks a lot for your help! Lewis P.S. I'm sure you hear this a lot but YT is an amazing library, it really makes analysing complicated simulation data so much more straightforward. Thank you (and the other devs) for working on it!
participants (3)
-
Lewis Weinberger -
lhw28＠cam.ac.uk
-
Nathan Goldbaum