Mailman 3 May 2018 - yt-users

Adding particle fields
by Nick Gnedin May 27, 2018

May 27, 2018

Guys, I am trying to add a new particle field, and apparently I am doing something wrong, but I could find the documentation for this: import yt def NewField(field,data): q = data[('STAR','age')] print("BBB",data) return q d = yt.load("rei10_a0.1001/rei10_a0.1001.art") print('AAA',d.all_data()[("STAR","age")]) d.add_field(("STAR","new-field"),function=NewField,dimensions="time",sampling_type="particle") The result is here: AAA [ 1.35689106e+08 1.29578798e+08 1.15241829e+08 ..., 2.97175878e+07 2.04648908e+07 1.35365980e+07] yr BBB defaultdict(<function FieldDetector.__init__.<locals>.<lambda> at 0x2b9e698e3598>, {('STAR', 'BIRTH_TIME'): YTArray([ 1.]) (dimensionless), ('STAR', 'creation_time'): YTArray([ 1.]) (dimensionless), ('STAR', 'age'): YTArray([ 1.]) (dimensionless)}) It appears that the data object sent to NewField(field,data) has all its fields set to dimensionless. I tried omitting sampling_type="particle", but it does not help, just generates an incomprehensible warning. n

2 1

Re: location of max(field)
by Nick Gnedin May 25, 2018

May 25, 2018

I am using 3.4.1. I get the same problem with wrong units with the data set I uploaded: d = yt.load("rei10_a0.1001/rei10_a0.1001.art") s = d.sphere([0,0,0],(10,'kpc')) loc = s.quantities.max_location(('gas', 'density')) print(loc) > [7.042448094432284e-28 g/cm**3, 0.3125 cm, 0.0625 cm, 63.8125 cm] On 05/25/2018 12:49 PM, Nathan Goldbaum wrote: > > > On Fri, May 25, 2018 at 12:25 PM, Nick Gnedin <gnedin(a)fnal.gov > <mailto:gnedin@fnal.gov>> wrote: > > > Guys, > > Sorry for multiple question, here is another simple one. > > I have a dataset d for which I create a sphere: > > s = d.sphere([x0,y0,z0],r0) > vcen = s.quantities.bulk_velocity() > s.set_field_parameter("bulk_velocity",vcen) > > It is located somewhere in the box: > > print(x0,y0,z0) > > 0.04789762673404803 Mpc 4.396788395087413 Mpc 0.9398923037682868 Mpc > > Now I would like to find the location of the max density for some > field. When I try this: > > print(s.quantities.max_location(('deposit', 'STAR_density'))) > print(s.quantities.max_location(('gas', 'density'))) > print(s.quantities.max_location(('deposit', 'N-BODY_density'))) > > I get non-nonsensical values: > > [1.1195988056992701e-21 g/cm**3, 1.3408203125 cm, 117.7998046875 cm, > 25.3603515625 cm] > [1.3058663046178781e-20 g/cm**3, 1.3408203125 cm, 117.8017578125 cm, > 25.3603515625 cm] > [1.73680472946578e-20 g/cm**3, 1.3896484375 cm, 117.7919921875 cm, > 25.3271484375 cm] > > Density values are ok, but positions surely cannot be measured in a > small number of cm. Given that in code units the box size is 128, it > seems like locations are returned in code units but interpreted as cm. > > > Hmm, I agree that's probably what's happening. That said, I can't seem > to reproduce the issue on yt 3.4.1, with the following script: > > import yt > ds = > yt.load('sizmbhloz-clref04SNth-rs9_a0.9011/sizmbhloz-clref04SNth-rs9_a0.9011.art') > > c = [0.04789762673404803*Mpc, 4.396788395087413*Mpc, > 0.9398923037682868*Mpc] > s = ds.sphere(c, 50*Mpc) > print(s.quantities.max_location(('gas', 'density'))) > > I get: > > [3.4612530065401464e-22 g/cm**3, 63.9970703125 code_length, > 63.9990234375 code_length, 63.9970703125 code_length] > > If you're not already using yt 3.4.1, can you try updating? You can see > your current yt version with: > > $ python -c "import yt; print(yt.__version__)" > > If you're on yt 3.4.1 and are still seeing the issue, I guess it has > something to do with the dataset you're using or the precise details of > the code you're running. Is this the same dataset you sent me in the > other thread? > > -Nathan > > > n > _______________________________________________ > yt-users mailing list -- yt-users(a)python.org > <mailto:yt-users@python.org> > To unsubscribe send an email to yt-users-leave(a)python.org > <mailto:yt-users-leave@python.org> > >

2 1

Hello
by Edward Buie II May 25, 2018

May 25, 2018

Hello, I installed yt with wget and then ran the install_script.sh. I then installed miniconda3 and put both paths in my .bashrc. After this I tried to run my code which imports yt and uses a simple plotting function, however I get a runtime error because the file that I am plotting is a HDF5 file and h5py is not installed. I checked and h5py is installed but I also re-installed it just in case but I still get the same error. Thank you for your help ahead of time. Edward Buie II

2 1

location of max(field)
by Nick Gnedin May 25, 2018

May 25, 2018

Guys, Sorry for multiple question, here is another simple one. I have a dataset d for which I create a sphere: s = d.sphere([x0,y0,z0],r0) vcen = s.quantities.bulk_velocity() s.set_field_parameter("bulk_velocity",vcen) It is located somewhere in the box: print(x0,y0,z0) > 0.04789762673404803 Mpc 4.396788395087413 Mpc 0.9398923037682868 Mpc Now I would like to find the location of the max density for some field. When I try this: print(s.quantities.max_location(('deposit', 'STAR_density'))) print(s.quantities.max_location(('gas', 'density'))) print(s.quantities.max_location(('deposit', 'N-BODY_density'))) I get non-nonsensical values: [1.1195988056992701e-21 g/cm**3, 1.3408203125 cm, 117.7998046875 cm, 25.3603515625 cm] [1.3058663046178781e-20 g/cm**3, 1.3408203125 cm, 117.8017578125 cm, 25.3603515625 cm] [1.73680472946578e-20 g/cm**3, 1.3896484375 cm, 117.7919921875 cm, 25.3271484375 cm] Density values are ok, but positions surely cannot be measured in a small number of cm. Given that in code units the box size is 128, it seems like locations are returned in code units but interpreted as cm. n

2 1

Re: Fwd: ParticleProjectionPlot error
by Nick Gnedin May 25, 2018

May 25, 2018

Nathan, Here is the uploaded data set (tarred together - sorry, I do not have anything smaller): http://use.yt/upload/6679c658 and here is the script: import yt d = yt.load("rei10_a0.1001/rei10_a0.1001.art") p1 = yt.ParticleProjectionPlot(d,'z',fields=[('N-BODY',"MASS")],center=[0,0,0],width=(50,'kpc')) p1.save() p2 = yt.ParticleProjectionPlot(d,'z',center=[0,0,0],width=(50,'kpc')) p2.save() The first call completes and the second returns a key error. n On 05/25/2018 10:49 AM, Nathan Goldbaum wrote: > > > On Fri, May 25, 2018 at 10:44 AM, Nick Gnedin <gnedin(a)fnal.gov > <mailto:gnedin@fnal.gov>> wrote: > > > Nathan, > > I digged a bit deeper, and the problem seems to be in file > yt/frontends/artio/data_structures.py, in function > fill_particles(self, fields): > > species_data = self.oct_handler.fill_sfc_particles(art_fields) > ... > for s, f in fields: > ... > count += species_data[i, f].size > ... > v = species_data.pop((i, f)) > > The function is entered with field = > > [('N-BODY', 'POSITION_X'), ('N-BODY', 'POSITION_X'), ('N-BODY', > 'POSITION_X'), ('STAR', 'POSITION_X'), ('N-BODY', 'POSITION_Y'), > ('N-BODY', 'POSITION_Z'), ('STAR', 'POSITION_Y'), ('STAR', > 'POSITION_Z')] > > while species_data is set as > > {(0, 'POSITION_X'): array([], dtype=float64), (0, 'POSITION_Y'): > array([], dtype=float64), (0, 'POSITION_Z'): array([], > dtype=float64), (3, 'POSITION_X'): array([], dtype=float64), (3, > 'POSITION_Y'): array([], dtype=float64), (3, 'POSITION_Z'): > array([], dtype=float64)} > > After the first iteration of the loop ('N-BODY', 'POSITION_X') is > popped out of the dictionary, so the second iteration returns the > key error. > > It looks like there is a bug in artio backend for file sets that > contain more than 1 DM species (my file set contains 3, hence 3 > separate ('N-BODY', 'POSITION_X') inside fields list. > > I tried replacing > > for s, f in fields: > > with > > for s, f in set(fields): > > but then it causes a problem further down the line with arrays not > being compatible, which I cannot decipher. > > Curiously, if I call ParticleProjectionPlot as > > yt.ParticleProjectionPlot(d,'z',fields=[('N-BODY',"MASS")],center=[x0,y0,z0],width=(50,'kpc')) > > (with fields argument specified explicitly), the call completes. In > this case it appears that the fields argument selects just one of > the three DM species for plotting. > > I can provide you a file set and a code snippet that illustrates the > problem if you think it is worth debugging further. > > > Yeah, that would be great. I think you've identified the root issue: > currently we don't have support for reading in multiple DM species from > artio data. I'll be honest I'm not terribly familiar with the internals > of the artio frontend (that was mostly taken care of by Doug Rudd, and > we haven't really had any substantial issues reported with it since he > left the field) but I'll be happy to take a look as long as you're ok > fielding questions I have if any come up :) > > > n > > > > > > On 05/24/2018 10:06 PM, Nathan Goldbaum wrote: > > Hi Nick, > > I seem to be able to make this plot using the public artio test > dataset we have at yt-project.org/data > <http://yt-project.org/data> <http://yt-project.org/data>: > > http://yt-project.org/data/sizmbhloz-clref04SNth-rs9_a0.9011.tar.gz > <http://yt-project.org/data/sizmbhloz-clref04SNth-rs9_a0.9011.tar.gz> > > Using that dataset I'm able to run the following script: > > ```python > import yt > ds = > yt.load('sizmbhloz-clref04SNth-rs9_a0.9011/sizmbhloz-clref04SNth-rs9_a0.9011.art') > p = yt.ParticleProjectionPlot(ds, 'z') > p.save() > ``` > > which produces this image: https://i.imgur.com/0lt31xR.png > <https://i.imgur.com/0lt31xR.png> > > So I think there's something about the dataset you're working > with that's breaking the artio frontend somehow. Judging from > the traceback, yt is trying to get the particle counts to > allocate the numpy arrays that will contain the particle > position fields needed by the ParticleProjectionPlot machinery. > Somehow when we ask the artio octree handler how many particles > there are, we get a dict back that's either missing some of > those counts or is empty. It's hard for me to say more about > what's going wrong without being able to debug on a live dataset > triggering the issue. > > Is there any chance you can share either the dataset you're > working with or some other dataset that triggers this error? > There's something different about the dataset you're working > with compared with the test dataset we have which will help to > explain what's going wrong. > > -Nathan > > On Thu, May 24, 2018 at 7:02 PM, Cameron Hummels > <chummels(a)gmail.com <mailto:chummels@gmail.com> > <mailto:chummels@gmail.com <mailto:chummels@gmail.com>>> wrote: > > Hi Nick, > > I think you need to explicitly set the fields that you're > plotting, > like ("Stars", "particle_position_x") and so forth. Here > is an example: > > http://yt-project.org/docs/dev/visualizing/plots.html#particle-plots > <http://yt-project.org/docs/dev/visualizing/plots.html#particle-plots> > > <http://yt-project.org/docs/dev/visualizing/plots.html#particle-plots > <http://yt-project.org/docs/dev/visualizing/plots.html#particle-plots>> > > You can see what fields are available to plot by looking at the > "derived_field_list" hanging off of your dataset: > > ds = yt.load("dataset") > print(ds.derived_field_list) > > Look for particle-based ones that have > "particle_position_x" and > "particle_position_y" present. I'm not sure the field_types > associated with ARTIO. > > Hope this helps! > > Cameron > > On Thu, May 24, 2018 at 3:25 PM, Nick Gnedin > <gnedin(a)fnal.gov <mailto:gnedin@fnal.gov> > <mailto:gnedin@fnal.gov <mailto:gnedin@fnal.gov>>> wrote: > > > Guys, > > I get an error trying to use a simple > ParticleProjectionPlot on > a ARTIO dataset. It looks like ParticleProjectionPlot > expect > particular names for particle fields that the reader > does not > create. > > imp = > > yt.ParticleProjectionPlot(d,'z',center=[x0,y0,z0],width=(10,'kpc')) > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/particle_plots.py", > line 249, in __init__ > splat_color=splat_color) > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 660, in __init__ > PlotWindow.__init__(self, *args, **kwargs) > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 219, in __init__ > self._setup_plots() > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 748, in _setup_plots > self._recreate_frb() > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 272, in _recreate_frb > self._frb._get_data_source_fields() > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", > line 646, in _get_data_source_fields > self[f] > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", > line 586, in __getitem__ > x_data = self.data_source.dd[ftype, self.x_field] > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", > line 282, in __getitem__ > self.get_data(f) > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", > line 1331, in get_data > particles, self, self._current_chunk) > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/geometry/geometry_handler.py", > line 227, in _read_particle_fields > fields_to_read) > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/utilities/io_handler.py", > line 219, in _read_particle_selection > for field_r, vals in > self._read_particle_fields(chunks, > ptf, selector): > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/frontends/artio/io.py", > line 63, in _read_particle_fields > rv = dict(**subset.fill_particles(fields)) > File > > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/frontends/artio/data_structures.py", > line 107, in fill_particles > count += species_data[i, f].size > KeyError: (0, 'POSITION_X') > _______________________________________________ > yt-users mailing list -- yt-users(a)python.org > <mailto:yt-users@python.org> > <mailto:yt-users@python.org <mailto:yt-users@python.org>> > To unsubscribe send an email to > yt-users-leave(a)python.org <mailto:yt-users-leave@python.org> > <mailto:yt-users-leave@python.org > <mailto:yt-users-leave@python.org>> > > > > > -- Cameron Hummels > NSF Postdoctoral Fellow > Department of Astronomy > California Institute of Technology > http://chummels.org > > _______________________________________________ > yt-users mailing list -- yt-users(a)python.org > <mailto:yt-users@python.org> > <mailto:yt-users@python.org <mailto:yt-users@python.org>> > To unsubscribe send an email to yt-users-leave(a)python.org > <mailto:yt-users-leave@python.org> > <mailto:yt-users-leave@python.org > <mailto:yt-users-leave@python.org>> > > >

1 0

Re: Fwd: ParticleProjectionPlot error
by Nick Gnedin May 25, 2018

May 25, 2018

Nathan, I digged a bit deeper, and the problem seems to be in file yt/frontends/artio/data_structures.py, in function fill_particles(self, fields): species_data = self.oct_handler.fill_sfc_particles(art_fields) ... for s, f in fields: ... count += species_data[i, f].size ... v = species_data.pop((i, f)) The function is entered with field = [('N-BODY', 'POSITION_X'), ('N-BODY', 'POSITION_X'), ('N-BODY', 'POSITION_X'), ('STAR', 'POSITION_X'), ('N-BODY', 'POSITION_Y'), ('N-BODY', 'POSITION_Z'), ('STAR', 'POSITION_Y'), ('STAR', 'POSITION_Z')] while species_data is set as {(0, 'POSITION_X'): array([], dtype=float64), (0, 'POSITION_Y'): array([], dtype=float64), (0, 'POSITION_Z'): array([], dtype=float64), (3, 'POSITION_X'): array([], dtype=float64), (3, 'POSITION_Y'): array([], dtype=float64), (3, 'POSITION_Z'): array([], dtype=float64)} After the first iteration of the loop ('N-BODY', 'POSITION_X') is popped out of the dictionary, so the second iteration returns the key error. It looks like there is a bug in artio backend for file sets that contain more than 1 DM species (my file set contains 3, hence 3 separate ('N-BODY', 'POSITION_X') inside fields list. I tried replacing for s, f in fields: with for s, f in set(fields): but then it causes a problem further down the line with arrays not being compatible, which I cannot decipher. Curiously, if I call ParticleProjectionPlot as yt.ParticleProjectionPlot(d,'z',fields=[('N-BODY',"MASS")],center=[x0,y0,z0],width=(50,'kpc')) (with fields argument specified explicitly), the call completes. In this case it appears that the fields argument selects just one of the three DM species for plotting. I can provide you a file set and a code snippet that illustrates the problem if you think it is worth debugging further. n On 05/24/2018 10:06 PM, Nathan Goldbaum wrote: > Hi Nick, > > I seem to be able to make this plot using the public artio test dataset > we have at yt-project.org/data <http://yt-project.org/data>: > > http://yt-project.org/data/sizmbhloz-clref04SNth-rs9_a0.9011.tar.gz > > Using that dataset I'm able to run the following script: > > ```python > import yt > ds = > yt.load('sizmbhloz-clref04SNth-rs9_a0.9011/sizmbhloz-clref04SNth-rs9_a0.9011.art') > p = yt.ParticleProjectionPlot(ds, 'z') > p.save() > ``` > > which produces this image: https://i.imgur.com/0lt31xR.png > > So I think there's something about the dataset you're working with > that's breaking the artio frontend somehow. Judging from the traceback, > yt is trying to get the particle counts to allocate the numpy arrays > that will contain the particle position fields needed by the > ParticleProjectionPlot machinery. Somehow when we ask the artio octree > handler how many particles there are, we get a dict back that's either > missing some of those counts or is empty. It's hard for me to say more > about what's going wrong without being able to debug on a live dataset > triggering the issue. > > Is there any chance you can share either the dataset you're working with > or some other dataset that triggers this error? There's something > different about the dataset you're working with compared with the test > dataset we have which will help to explain what's going wrong. > > -Nathan > > On Thu, May 24, 2018 at 7:02 PM, Cameron Hummels <chummels(a)gmail.com > <mailto:chummels@gmail.com>> wrote: > > Hi Nick, > > I think you need to explicitly set the fields that you're plotting, > like ("Stars", "particle_position_x") and so forth. Here is an example: > > http://yt-project.org/docs/dev/visualizing/plots.html#particle-plots > <http://yt-project.org/docs/dev/visualizing/plots.html#particle-plots> > > You can see what fields are available to plot by looking at the > "derived_field_list" hanging off of your dataset: > > ds = yt.load("dataset") > print(ds.derived_field_list) > > Look for particle-based ones that have "particle_position_x" and > "particle_position_y" present. I'm not sure the field_types > associated with ARTIO. > > Hope this helps! > > Cameron > > On Thu, May 24, 2018 at 3:25 PM, Nick Gnedin <gnedin(a)fnal.gov > <mailto:gnedin@fnal.gov>> wrote: > > > Guys, > > I get an error trying to use a simple ParticleProjectionPlot on > a ARTIO dataset. It looks like ParticleProjectionPlot expect > particular names for particle fields that the reader does not > create. > > imp = > yt.ParticleProjectionPlot(d,'z',center=[x0,y0,z0],width=(10,'kpc')) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/particle_plots.py", > line 249, in __init__ > splat_color=splat_color) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 660, in __init__ > PlotWindow.__init__(self, *args, **kwargs) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 219, in __init__ > self._setup_plots() > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 748, in _setup_plots > self._recreate_frb() > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 272, in _recreate_frb > self._frb._get_data_source_fields() > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", > line 646, in _get_data_source_fields > self[f] > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", > line 586, in __getitem__ > x_data = self.data_source.dd[ftype, self.x_field] > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", > line 282, in __getitem__ > self.get_data(f) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", > line 1331, in get_data > particles, self, self._current_chunk) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/geometry/geometry_handler.py", > line 227, in _read_particle_fields > fields_to_read) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/utilities/io_handler.py", > line 219, in _read_particle_selection > for field_r, vals in self._read_particle_fields(chunks, > ptf, selector): > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/frontends/artio/io.py", > line 63, in _read_particle_fields > rv = dict(**subset.fill_particles(fields)) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/frontends/artio/data_structures.py", > line 107, in fill_particles > count += species_data[i, f].size > KeyError: (0, 'POSITION_X') > _______________________________________________ > yt-users mailing list -- yt-users(a)python.org > <mailto:yt-users@python.org> > To unsubscribe send an email to yt-users-leave(a)python.org > <mailto:yt-users-leave@python.org> > > > > > -- > Cameron Hummels > NSF Postdoctoral Fellow > Department of Astronomy > California Institute of Technology > http://chummels.org > > _______________________________________________ > yt-users mailing list -- yt-users(a)python.org > <mailto:yt-users@python.org> > To unsubscribe send an email to yt-users-leave(a)python.org > <mailto:yt-users-leave@python.org> > >

2 1

Re: Fwd: ParticleProjectionPlot error
by Nick Gnedin May 25, 2018

May 25, 2018

Cameron, These fields are set for ParticlePlot, not ParticleProjectionPlot. The former function does not take center and width arguments, so I cannot use it. n On 5/24/2018 7:02 PM, Cameron Hummels wrote: > Hi Nick, > > I think you need to explicitly set the fields that you're plotting, like > ("Stars", "particle_position_x") and so forth. Here is an example: > > http://yt-project.org/docs/dev/visualizing/plots.html#particle-plots > > You can see what fields are available to plot by looking at the > "derived_field_list" hanging off of your dataset: > > ds = yt.load("dataset") > print(ds.derived_field_list) > > Look for particle-based ones that have "particle_position_x" and > "particle_position_y" present. I'm not sure the field_types associated > with ARTIO. > > Hope this helps! > > Cameron > > On Thu, May 24, 2018 at 3:25 PM, Nick Gnedin <gnedin(a)fnal.gov > <mailto:gnedin@fnal.gov>> wrote: > > > Guys, > > I get an error trying to use a simple ParticleProjectionPlot on a > ARTIO dataset. It looks like ParticleProjectionPlot expect > particular names for particle fields that the reader does not create. > > imp = > yt.ParticleProjectionPlot(d,'z',center=[x0,y0,z0],width=(10,'kpc')) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/particle_plots.py", > line 249, in __init__ > splat_color=splat_color) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 660, in __init__ > PlotWindow.__init__(self, *args, **kwargs) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 219, in __init__ > self._setup_plots() > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 748, in _setup_plots > self._recreate_frb() > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", > line 272, in _recreate_frb > self._frb._get_data_source_fields() > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", > line 646, in _get_data_source_fields > self[f] > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", > line 586, in __getitem__ > x_data = self.data_source.dd[ftype, self.x_field] > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", > line 282, in __getitem__ > self.get_data(f) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", > line 1331, in get_data > particles, self, self._current_chunk) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/geometry/geometry_handler.py", > line 227, in _read_particle_fields > fields_to_read) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/utilities/io_handler.py", > line 219, in _read_particle_selection > for field_r, vals in self._read_particle_fields(chunks, ptf, > selector): > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/frontends/artio/io.py", > line 63, in _read_particle_fields > rv = dict(**subset.fill_particles(fields)) > File > "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/frontends/artio/data_structures.py", > line 107, in fill_particles > count += species_data[i, f].size > KeyError: (0, 'POSITION_X') > _______________________________________________ > yt-users mailing list -- yt-users(a)python.org > <mailto:yt-users@python.org> > To unsubscribe send an email to yt-users-leave(a)python.org > <mailto:yt-users-leave@python.org> > > > > > -- > Cameron Hummels > NSF Postdoctoral Fellow > Department of Astronomy > California Institute of Technology > http://chummels.org

1 0

Fwd: ParticleProjectionPlot error
by Nick Gnedin May 25, 2018

May 25, 2018

Guys, I get an error trying to use a simple ParticleProjectionPlot on a ARTIO dataset. It looks like ParticleProjectionPlot expect particular names for particle fields that the reader does not create. imp = yt.ParticleProjectionPlot(d,'z',center=[x0,y0,z0],width=(10,'kpc')) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/particle_plots.py", line 249, in __init__ splat_color=splat_color) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", line 660, in __init__ PlotWindow.__init__(self, *args, **kwargs) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", line 219, in __init__ self._setup_plots() File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", line 748, in _setup_plots self._recreate_frb() File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/plot_window.py", line 272, in _recreate_frb self._frb._get_data_source_fields() File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", line 646, in _get_data_source_fields self[f] File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", line 586, in __getitem__ x_data = self.data_source.dd[ftype, self.x_field] File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 282, in __getitem__ self.get_data(f) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/data_objects/data_containers.py", line 1331, in get_data particles, self, self._current_chunk) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/geometry/geometry_handler.py", line 227, in _read_particle_fields fields_to_read) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/utilities/io_handler.py", line 219, in _read_particle_selection for field_r, vals in self._read_particle_fields(chunks, ptf, selector): File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/frontends/artio/io.py", line 63, in _read_particle_fields rv = dict(**subset.fill_particles(fields)) File "/data/gnedin/soft/anaconda3/lib/python3.6/site-packages/yt/frontends/artio/data_structures.py", line 107, in fill_particles count += species_data[i, f].size KeyError: (0, 'POSITION_X')

3 2

Postdoc opportunity at NCSA: help grow the yt community
by Nathan Goldbaum May 24, 2018

May 24, 2018

Hello all, The DXL lab at the National Center for Supercomputing Applications and the University of Illinois at Urbana-Champaign are hiring a postdoc for a one year term with a possibility to extend for another two years. The primary focus of this postdoc will be improving yt (yt-project.org, a python toolkit for analyzing and visualizing volumetric data) and adding support in yt for data used in fields across the physical sciences. We are particularly looking for candidates with a background in Earth science fields such as geophysics, seismology, oceanography, and meteorology, but interested candidates with a background in plasma physics, astronomy, nuclear engineering, or computational fluid dynamics are also encouraged to apply. This is in support of a funded NSF grant which you can read more about here: https://doi.org/10.6084/m9.figshare.2061465.v1 Please see the job ad for more details, copied below: http://www.ncsa.illinois.edu/about/jobs/pra_yt To learn more about the DXL lab see our website: https://data-exp-lab.github.io/ Best, Nathan Goldbaum --- NCSA invites applications for a postdoctoral fellow position to develop scientific software for analyzing and visualizing volumetric data based on the yt project, an open source, community-developed python package. The successful candidate is expected to contribute to the development of systems and software in support of a newly awarded National Science Foundation grant to expand the yt community to new fields in the physical sciences. Key responsibilities Develop community models and tools for use in computational and data-intensive science and engineering more generally. Conduct software development for community tools such as yt to extend support for data from domains such as weather, seismology, oceanography, and engineering. Conduct outreach to domain research groups and conferences to demonstrate the capabilities of yt for working with common data formats and research data. Conduct research in related areas of physical and computational science. Releasing open source community software tools as well as publications describing those tools. Participating in grant proposal writing. Researchers are encouraged to further their own research interests in collaboration with the broad range of projects going on at NCSA. Interact with teams at NCSA, including specifically the ISDA, ISL, and visualization teams, as well with the national projects led by NCSA including the National Data Service and the Blue Waters team on development of scalable algorithms and codes at the petascale, and the XSEDE team in the development and deployment of computing and data-services and tools for communities in the physical sciences. Connect to faculty, postdocs, and students on campus, especially in the School of Information Sciences, Computer Science, Physics, Geology, Astronomy and Atmospheric Sciences Departments. Required education and experience A recent PhD in meteorology, seismology, oceanography, physics, astronomy, computer science, information science, or a related field. Experience working with large, complex datasets and research data formats common in the applicant’s field of study. Experience with the Python programming language. Analytical problem solving ability. Strong written and oral communication skills. Good interpersonal and teamwork skills. Preferred experience Experience visualizing or analyzing volumetric datasets in the context of computational fluid dynamics simulations or tomographic reconstruction of remote sensing observations. Prior experience with one or more of the following computational approaches: Adaptive mesh refinement (AMR). Lagrangian methods, including N-body methods, smoothed particle hydrodynamics, or other meshless hydrodynamics methods. Finite element methods and unstructured meshes. Hydrodynamics on curvilinear or geographically referenced meshes. Familiar with yt, xarray, numba, dask, pandas, vaex, matplotlib or other python data science libraries. Experience with C, C++, Cython, Rust, or other low-level compiled languages is also encouraged. Experience working in the open on free and open source software using Git and GitHub. The expected start date is August 16, 2018. The appointment will be for a fixed term of one year with the possibility of extension. Salary is competitive and commensurate with experience and relevant research. For full consideration, interested applicants should send their application materials (cover letter, curriculum vitae, research statement, links to recent publications, names/contact information of 3 references) to postdoc-search(a)ncsa.illinois.edu and include "NCSA SI2 Postdoc Application" in the subject line. Interviews and hires may take place prior to the closing date and early applications are encouraged; however, full consideration will be given to complete applications received by the closing date or until the search closes. The University of Illinois conducts criminal background checks on all job candidates upon acceptance of a contingent offer.

1 0

Re: Newbie question: ParticlePlot(...)
by Nick Gnedin May 16, 2018

May 16, 2018

Nathan, Thank you. I am sorry - I did not notice that the image was indeed created. n On 05/16/2018 01:07 PM, Nathan Goldbaum wrote: > So this isn't really a bug (or a problem at all) - matplotlib is just > warning that the image yt is passing to matplotlib's imshow command > contains NaN values. > > This was actually done on purpose in yt, you can see what the code looks > like here: > > https://github.com/yt-project/yt/blob/master/yt/visualization/fixed_resolut… > > That was added in this commit: > > https://github.com/yt-project/yt/commit/6f378028f7a1eedd553f7641be719762f0c… > > And you can see a test script that explains why we did this change in > this PR: > > https://bitbucket.org/yt_analysis/yt/pull-requests/2565/remove-values-in-no… > > (I can't remember why that was declined, I think it was manually merged > in right around when we we moved to GitHub) > > I guess in principle we could suppress that warning on the yt side, > although that might be a case of using a cudgel when all we need is a > hammer. We could also think about ways to mask out images without > relying on NaN in matplotlib. One could also argue that passing NaN to > matplotlib as a mask is a common enough thing that matplotlib itself > should be doing special handing at the line raising the warning to avoid > generating it. > > It's not really a problem that matplotlib is generating the warning. You > can safely ignore it. > > > > On Wed, May 16, 2018 at 10:49 AM, Nick Gnedin <ngnedin(a)gmail.com > <mailto:ngnedin@gmail.com>> wrote: > > > Nathan, > > Done (I tared 4 files together): > > /data/gnedin/REI/TMP >tar cvf gnedin_hop.tar hop.* > hop.0.h5 > hop.1.h5 > hop.2.h5 > hop.3.h5 > /data/gnedin/REI/TMP >yt upload gnedin_hop.tar > Stream body handler: received 9687040 bytes > http://use.yt/upload/318d837f > > The files are available on BlueWaters at: > /mnt/c/scratch/sciteam/ngnedin/PERM/B40/A.DMO/a=1.0008/hop/hop.*.h5 > > Thanks a lot. > > Nick > > > > On 05/16/2018 09:38 AM, Nathan Goldbaum wrote: > > Hi, > > Is there any chance you can share the hdf5 file? I'm not able to > reproduce this warning using a test dataset I have so it might > have something to do with the data. > > You can share a dataset with "yt upload path/to/data.h5". That > will upload to a server at NCSA and then print a link out to > share here once its done. > > If the data cannot be shared publicly you can send me an e-mail > off-list with e.g. a dropbox link. > > -Nathan > > On Wed, May 16, 2018 at 10:32 AM, Nick Gnedin <ngnedin(a)gmail.com > <mailto:ngnedin@gmail.com> <mailto:ngnedin@gmail.com > <mailto:ngnedin@gmail.com>>> wrote: > > > I am trying to make a plot of all halos found by HOP. I > copied your > example from > http://yt-project.org/doc/cookbook/simple_plots.html > <http://yt-project.org/doc/cookbook/simple_plots.html> > <http://yt-project.org/doc/cookbook/simple_plots.html > <http://yt-project.org/doc/cookbook/simple_plots.html>> > > If I just plot partile positions: > > --------------------- script ------------------- > import yt > import numpy as np > > # load the dataset > ds = yt.load('../B40/A.DMO/a=1.0008/hop/hop.0.h5') > > # create our plot > p = yt.ParticlePlot(ds, 'particle_position_x', > 'particle_position_y') > > m = ds.all_data()[('all', 'particle_mass')] > print(np.amin(m),np.amax(m)) > > # pick some appropriate units > p.set_axes_unit('Mpc') > p.set_unit('particle_mass', 'Msun') > > #save result > p.save() > ----------------------------------------------- > > then everything works fine. However, if I use mass weighting: > > p = yt.ParticlePlot(ds, 'particle_position_x', > 'particle_position_y', 'particle_mass') > > I get the following error: > > ... > yt : [INFO ] 2018-05-16 09:27:39,004 Allocating for > 1.997e+05 > particles (index particle type 'all') > yt : [INFO ] 2018-05-16 09:27:39,381 Identified > 5.972e+04 octs > yt : [INFO ] 2018-05-16 09:27:40,021 xlim = 0.000000 > 199229564535610884216586240.000000 > yt : [INFO ] 2018-05-16 09:27:40,021 ylim = 0.000000 > 199229564535610884216586240.000000 > yt : [INFO ] 2018-05-16 09:27:40,023 xlim = 0.000000 > 199229564535610884216586240.000000 > yt : [INFO ] 2018-05-16 09:27:40,023 ylim = 0.000000 > 199229564535610884216586240.000000 > yt : [INFO ] 2018-05-16 09:27:40,042 Splatting (('all', > 'particle_mass')) onto a 800 by 800 mesh > 4.1036023233207635e+41 g 1.921697915571756e+47 g > yt : [INFO ] 2018-05-16 09:27:44,084 Saving plot > hop.0.h5_Particle_z_particle_mass.png > > /u/sciteam/ngnedin/.local/lib/python3.5/site-packages/matplotlib/colors.py:1020: > RuntimeWarning: invalid value encountered in less_equal > mask |= resdat <= 0 > > Notice, that there are no nonpositive masses in the catalog > print(np.amin(m),np.amax(m)): > 4.1036023233207635e+41 g 1.921697915571756e+47 g > > > > > _______________________________________________ > yt-users mailing list -- yt-users(a)python.org > <mailto:yt-users@python.org> > <mailto:yt-users@python.org <mailto:yt-users@python.org>> > To unsubscribe send an email to yt-users-leave(a)python.org > <mailto:yt-users-leave@python.org> > <mailto:yt-users-leave@python.org > <mailto:yt-users-leave@python.org>> > > >

1 0