Hey,

Indeed, the previous version was actually able to read these files ­— even if invalid. The code should have never worked in the first place if it wanted to be compliant with the binary format RAMSES uses. I think that the easiest fix is to use a version of yt prior to commit #016c7e04bdf443d16b0431369be364dbb152e4ed (https://github.com/yt-project/yt/commit/016c7e04bdf443d16b0431369be364dbb152e4ed). The latest release that do not include this commit is yt 3.4.1. If you installed yt using git, you can easily switch to yt 3.4.1 by running in the folder that contains yt

git checkout yt-3.4.1
pip install .

If installed via conda, simpy run conda install -c conda-forge yt=3.4.1.

Cheers,

Corentin

On 03/05/2019 15.33, Vadlamani Samhitha wrote:

I see. Thank you. But I’m pretty positive I didn’t face this issue with the version I had before ( 3.4.0 ) as I was able to get the required plots. It’s just that when I upgraded and trying to do the same plots that I face this issue and I am unable get back to the older version now. Is there a possibility that outputs could have been corrupted during the upgrade somehow? 

On Fri, 3 May 2019 at 3:25 PM, Corentin CADIOU <contact@cphyc.me> wrote:

Hi,

The issue is that your amr file does not contain any value for the field ‘mass_sph’. In fact, yt should not haver never been able to read it because the structure is somehow broken.

Here’s what I get from your file (as read by yt)

ncpu [4, 30, 4]
ndim [4, 3, 4]
nx [12, 3, 3, 3, 12]
nlevelmax [4, 11, 4]
ngridmax [4, 2000000, 4]
nboundary [4, 6, 4]
ngrid_current [4, 8722, 4]
boxlen [8, 2000.0, 8]
nout [12, 1, 1, 2, 12]
tout [8, 50000.0, 8]
aout [8, 1.100000023841858, 8]
t [8, 0.0, 8]
dtold [88, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 88]
dtnew [88, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 88]
nstep [8, 0, 0, 8]
stat [24, 0.0, 0.0, 0.0, 24]
cosm [56, 1.0, 0.0, 0.0, 0.0, 1.0, 10.0, 0.0, 56]
timing [40, 1.0, 0.0, 1.0, 0.0, 0.0, 40]
mass_sph [0, 2.801034444074e-311, 0]

The first and last item of each list are the length of each record (in bytes). For example the first record that contains ncpu has a length of 4 bytes and contains the value 30 (ncpu=30). However, the last record (mass_sph) has a length of 0 even though one even though it is not empty. I am not sure how that could happen, but it’s definitely on RAMSES side. The last line should read mass_sph [8, 2.801034444074e-311, 8].

I have to think about how that could be fixed.

On 03/05/2019 14.56, Vadlamani Samhitha wrote:

Hi, 

If it's only one of the air files, I'm attaching it below. Let me know if you would need the entire output folder too. Thank you.

On Fri, May 3, 2019 at 2:52 PM Corentin CADIOU <contact@cphyc.me> wrote:

Hey Vadlamani,

This looks like a bug to me. Could you share one of your file amr_xxxxx.out00001? It looks like yt has troubles reading the headers.

This can either be due to a backward incompatible change on yt side (which would be a bug) or maybe your file became corrupted at some point.

Cheers,
Corentin

On 02/05/2019 18.03, Vadlamani Samhitha wrote:

Hi, 

I was able to upgrade to the latest version and still have an issue with annotating particles on density map. I'm getting the following error:

Traceback (most recent call last):

  File "density.py", line 47, in <module>

    plot = yt.ProjectionPlot(ds, 'z', 'density', width=(800, "kpc"), center="c")

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/visualization/plot_window.py", line 1464, in __init__

    validate_mesh_fields(test_data_source, fields)

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/visualization/plot_window.py", line 132, in validate_mesh_fields

    canonical_fields = data_source._determine_fields(fields)

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/data_objects/data_containers.py", line 1165, in _determine_fields

    finfo = self.ds._get_field_info("unknown", fname)

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/data_objects/static_output.py", line 756, in _get_field_info

    self.index

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/data_objects/static_output.py", line 509, in index

    self, dataset_type=self.dataset_type)

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/frontends/ramses/data_structures.py", line 236, in __init__

    super(RAMSESIndex, self).__init__(ds, dataset_type)

File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 50, in __init__

    self._setup_geometry()

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/geometry/oct_geometry_handler.py", line 25, in _setup_geometry

    self._initialize_oct_handler()

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/frontends/ramses/data_structures.py", line 245, in _initialize_oct_handler

    for i in cpu_list]

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/frontends/ramses/data_structures.py", line 245, in <listcomp>

    for i in cpu_list]

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/frontends/ramses/data_structures.py", line 82, in __init__

    self._read_amr_header()

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/frontends/ramses/data_structures.py", line 141, in _read_amr_header

    hvals.update(f.read_attrs(header))

  File "yt/utilities/cython_fortran_utils.pyx", line 223, in yt.utilities.cython_fortran_utils.FortranFile.read_attrs

IndexError: index 0 is out of bounds for axis 0 with size 0

On Tue, Apr 30, 2019 at 4:39 AM Vadlamani Samhitha <vadlamani.samhitha@gmail.com> wrote:

Downgraded to initial version, but the 'Index error' still persists. Did a conda installation from scratch too. Reported it as a bug.

On Mon, Apr 29, 2019 at 6:00 PM Nathan Goldbaum <nathan12343@gmail.com> wrote:

Hmm, I'm not sure, perhaps try uninstalling pyembree?

You can also try creating a new conda environment from scratch.

On Mon, Apr 29, 2019 at 10:56 AM Vadlamani Samhitha <vadlamani.samhitha@gmail.com> wrote:

Thanks again, sorry but looks like I've messed up the installation. The installation wasn't right and I don't get the issue.

Upon doing yt --help I get:

Traceback (most recent call last):

  File "/home/samvad/yt-conda/bin/yt", line 4, in <module>

    import yt.utilities.command_line

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/__init__.py", line 102, in <module>

    from yt.visualization.api import \

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/api.py", line 22, in <module>

    from .particle_plots import \

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/particle_plots.py", line 18, in <module>

    from yt.visualization.fixed_resolution import \

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", line 23, in <module>

    from .volume_rendering.api import off_axis_projection

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/volume_rendering/api.py", line 23, in <module>

    from .camera import Camera

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/volume_rendering/camera.py", line 21, in <module>

    from .utils import data_source_or_all

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/volume_rendering/utils.py", line 11, in <module>

    from yt.utilities.lib import mesh_traversal

  File "rtcore_scene.pxd", line 61, in init yt.utilities.lib.mesh_traversal

ValueError: pyembree.rtcore_scene.EmbreeScene has the wrong size, try recompiling. Expected 32, got 24

-bash-4.2$ yt version

Traceback (most recent call last):

  File "/home/samvad/yt-conda/bin/yt", line 4, in <module>

    import yt.utilities.command_line

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/__init__.py", line 102, in <module>

    from yt.visualization.api import \

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/api.py", line 22, in <module>

    from .particle_plots import \

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/particle_plots.py", line 18, in <module>

    from yt.visualization.fixed_resolution import \

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", line 23, in <module>

    from .volume_rendering.api import off_axis_projection

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/volume_rendering/api.py", line 23, in <module>

    from .camera import Camera

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/volume_rendering/camera.py", line 21, in <module>

    from .utils import data_source_or_all

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/volume_rendering/utils.py", line 11, in <module>

    from yt.utilities.lib import mesh_traversal

  File "rtcore_scene.pxd", line 61, in init yt.utilities.lib.mesh_traversal

ValueError: pyembree.rtcore_scene.EmbreeScene has the wrong size, try recompiling. Expected 32, got 24

-bash-4.20.9169398134926968nbsp;

On Mon, Apr 29, 2019 at 5:30 PM Nathan Goldbaum <nathan12343@gmail.com> wrote:

Sure, you can use conda to install any version of any package. Just specify the version when you install:

conda install -c conda-forge yt=3.4.1

On Mon, Apr 29, 2019 at 10:27 AM Vadlamani Samhitha <vadlamani.samhitha@gmail.com> wrote:

Will do, but is it possible to revert to the old version for now since none of my scripts are working with the latest version? A fresh installation of 3.4.0?

On Mon, Apr 29, 2019 at 3:25 PM Nathan Goldbaum <nathan12343@gmail.com> wrote:

Hmm, not sure what’s happening there, you might need to report this as a bug on yt’s github, if you can please include a data file and script that triggers this error.

On Mon, Apr 29, 2019 at 8:00 AM Vadlamani Samhitha <vadlamani.samhitha@gmail.com> wrote:

Thank you, have done the upgrade but the following error is popping up:

Traceback (most recent call last):

  File "density.py", line 43, in <module>

    ds.add_field(name=("gas", "density_threshold"), function=density_threshold, units="H/cm**3")

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/data_objects/static_output.py", line 1227, in add_field

    self.index

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/data_objects/static_output.py", line 509, in index

    self, dataset_type=self.dataset_type)

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/frontends/ramses/data_structures.py", line 236, in __init__

    super(RAMSESIndex, self).__init__(ds, dataset_type)

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/geometry/geometry_handler.py", line 50, in __init__

    self._setup_geometry()

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/geometry/oct_geometry_handler.py", line 25, in _setup_geometry

    self._initialize_oct_handler()

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/frontends/ramses/data_structures.py", line 245, in _initialize_oct_handler

    for i in cpu_list]

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/frontends/ramses/data_structures.py", line 245, in <listcomp>

    for i in cpu_list]

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/frontends/ramses/data_structures.py", line 82, in __init__

    self._read_amr_header()

  File "/home/samvad/yt-conda/lib/python3.7/site-packages/yt/frontends/ramses/data_structures.py", line 141, in _read_amr_header

    hvals.update(f.read_attrs(header))

  File "yt/utilities/cython_fortran_utils.pyx", line 223, in yt.utilities.cython_fortran_utils.FortranFile.read_attrs

IndexError: index 0 is out of bounds for axis 0 with size 0

On Mon, Apr 29, 2019 at 1:51 PM Nathan Goldbaum <nathan12343@gmail.com> wrote:

Yes, use conda-forge:

conda update -c conda-forge yt

On Mon, Apr 29, 2019 at 6:32 AM Vadlamani Samhitha <vadlamani.samhitha@gmail.com> wrote:

I installed yt from conda so, yt update is only being done to 3.4.1 version with 'yt conda update' and with this my scripts are throwing assertion errors. Is there a way I could do a conda update to the latest version specifically?

On Mon, Apr 29, 2019 at 1:05 PM Nathan Goldbaum <nathan12343@gmail.com> wrote:

The latest stable release is 3.5.1, please try updating your yt installation.

On Mon, Apr 29, 2019 at 5:58 AM Vadlamani Samhitha <vadlamani.samhitha@gmail.com> wrote:

3.4.0

On Mon, Apr 29, 2019 at 12:52 PM Nathan Goldbaum <nathan12343@gmail.com> wrote:

What yt version are you using?

On Mon, Apr 29, 2019 at 5:34 AM Vadlamani Samhitha <vadlamani.samhitha@gmail.com> wrote:

Hi, I did try that as well, but doing so gives 'unexpected keyword' error.

Traceback (most recent call last):

  File "density.py", line 57, in <module>

    plot.save(fil)

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_container.py", line 94, in newfunc

    args[0]._setup_plots()

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_window.py", line 948, in _setup_plots

    self.run_callbacks()

  File "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_window.py", line 997, in run_callbacks

    callback = CallbackMaker(*args[1:], **kwargs)

TypeError: __init__() got an unexpected keyword argument 'data_source'

On Mon, Apr 29, 2019 at 12:18 PM Wise, John H <jwise@physics.gatech.edu> wrote:

Hi,

Because there are many optional arguments to annotate_particles, you
have to specify that dd is the data_source. Try

plot.annotate_particles(1.0, data_source=dd)

John

On 4/27/2019 15:46, Vadlamani Samhitha wrote:
> Hi,
>
> I am trying to overlap two different projection plots (particles and
> density) with two different color bars. However, I couldn't figure out a
> way to do that but found over plotting particles over fluids.
>
> So, I have RAMSES data which can't distinguish between dark matter and
> stars through particle types but only with mass cut_off. So I create a
> filter for the particles accordingly and supply it as the 'data_source'
> argument in 'annotate_particles'. So, I do the following:
>
>     defstars(pfilter, data):
>
>     # age = (data.ds.current_time - data[pfilter.filtered_type,
>     "particle_birth_time"]).in_units('Myr')
>
>              stars_only = data[pfilter.filtered_type,
>     "particle_mass"].in_units('Msun')
>
>     filter= (stars_only<4e6)
>
>     returnfilter
>
>     yt.add_particle_filter("stars", function=stars, filtered_type='io',
>     requires=["particle_mass"])
>
>
>     ini=1
>
>     fin=2
>
>     forx inrange(ini, fin):
>
>     filename="/lunarc/nobackup/users/samvad/SQ-8-5/output/output_%0*d/info_%0*d.txt"
>     % (5,x,5,x)
>
>              ds = yt.load(filename)
>
>              ds.define_unit("H", (1.674*10**(-24), "g"))
>
>     defdensity_threshold(field, data):
>
>                      dens=data["gas", "density"].in_units('H/cm**3')
>
>                      dens[dens < 0] = 0
>
>     returndens
>
>              ds.add_field(name=("gas", "density_threshold"),
>     function=density_threshold, units="H/cm**3")
>
>              ad=ds.all_data()
>
>              denst=ad["gas", "density_threshold"].in_units('H/cm**3')
>
>     #plot = yt.SlicePlot(ds, 'z', 'density_threshold', width=(400, "kpc"))
>
>              plot = yt.ProjectionPlot(ds, 'z', 'density',
>     weight_field='density',  width=(800, "kpc"), center="c")
>
>              ds.add_particle_filter('stars')
>
>              dd=ds.all_data()
>
>              plot.annotate_particles(1.0, dd)
>
>              plot.set_cmap(field='density', cmap='Blues')
>
>              plot.set_unit(("gas", "density"), "H/cm**3")
>
>              plot.set_zlim(("gas", "density"), 1e-4, 1e-1)
>
>     plot.annotate_timestamp()
>
>              fil="density_%0*d.png"  % (5,x)
>
>              plot.save(fil)
>
>
> But ending up with the error:
>
> Traceback (most recent call last):
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_window.py",
> line 999, in run_callbacks
>
>      callback(cbw)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_modifications.py",
> line 53, in _check_geometry
>
>      return func(self, plot)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_modifications.py",
> line 1563, in __call__
>
>      s=self.p_size, c=self.color,alpha=self.alpha)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/matplotlib/__init__.py",
> line 1717, in inner
>
>      return func(ax, *args, **kwargs)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/matplotlib/axes/_axes.py",
> line 4032, in scatter
>
>      alpha=alpha
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/matplotlib/collections.py",
> line 880, in __init__
>
>      self.set_sizes(sizes)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/matplotlib/collections.py",
> line 853, in set_sizes
>
>      scale = np.sqrt(self._sizes) * dpi / 72.0 * self._factor
>
> AttributeError: 'YTRegion' object has no attribute 'sqrt'
>
>
> During handling of the above exception, another exception occurred:
>
>
> Traceback (most recent call last):
>
>    File "density.py", line 57, in <module>
>
>      plot.save(fil)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_container.py",
> line 94, in newfunc
>
>      args[0]._setup_plots()
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_window.py",
> line 948, in _setup_plots
>
>      self.run_callbacks()
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_window.py",
> line 1005, in run_callbacks
>
>      sys.exc_info()[2])
>
>    File "/home/samvad/yt-conda/lib/python3.6/site-packages/six.py", line
> 692, in reraise
>
>      raise value.with_traceback(tb)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_window.py",
> line 999, in run_callbacks
>
>      callback(cbw)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_modifications.py",
> line 53, in _check_geometry
>
>      return func(self, plot)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/yt/visualization/plot_modifications.py",
> line 1563, in __call__
>
>      s=self.p_size, c=self.color,alpha=self.alpha)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/matplotlib/__init__.py",
> line 1717, in inner
>
>      return func(ax, *args, **kwargs)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/matplotlib/axes/_axes.py",
> line 4032, in scatter
>
>      alpha=alpha
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/matplotlib/collections.py",
> line 880, in __init__
>
>      self.set_sizes(sizes)
>
>    File
> "/home/samvad/yt-conda/lib/python3.6/site-packages/matplotlib/collections.py",
> line 853, in set_sizes
>
>      scale = np.sqrt(self._sizes) * dpi / 72.0 * self._factor
>
>     yt.utilities.exceptions.YTPlotCallbackError: annotate_particles
>     callback failed with the following error: 'YTRegion' object has no
>     attribute 'sqrt'
>
> I don't seem to

​