It might also be interesting to explore whether it makes sense to define some kind of chained data object that can combine data from different datasets. So you could do:

dobj = yt.chained_data([ds1.all_data(), ds2.all_data(), ...])
profile = yt.create_profile(dobj, ['density', 'temperature'], 'cell_mass')

This would be set up such that

dobj['density'] == uconcatenate([ds1.all_data()['density'], ds2.all_data()['density'], ...])

I'm not sure how generally useful this would be but then again I've never been in a situation where I wanted to combine data from multiple datasets like YT is trying to do right now.

On Fri, Nov 3, 2017 at 6:38 PM, Britton Smith <brittonsmith@gmail.com> wrote:
Hi YT,

Nathan and I have fleshed this out a bit in the discussion on the issue he created.

The main issue is that phase plots need to have all fields to be the same shape.  So the x and y fields also have to be 128x128.  You can make this happen in the following way:
Given a profile object called "profile":
x = yt.YTArray([profile.x]*128)
y = yt.YTArray([profile.y]*128).transpose()

If you then use those x and y in your save_as_dataset call, you should be able to use the loaded data with phaseplot.

Britton

On Fri, Nov 3, 2017 at 4:34 PM, ｔyuta <y0u1t1a5.t@gmail.com> wrote:
I was saving data just because I don't know how to sum up field values (in this case, "cell_mass") in other way. What I ultimately want to do is to make one phase plot which has "density", "temperature" bin fields, and the "cell_mass" is summed up through all different datasets (like, IsolatedGalaxy/galaxy0030/galaxy0030, IsolatedGalaxy/galaxy0029/galaxy0029, IsolatedGalaxy/galaxy0028/galaxy0028, ...).
Is it possible to sum up "prof_2d_ds.data ["cell_mass"]" in the example from different datasets?

2017-11-03 19:05 GMT-04:00 Nathan Goldbaum <nathan12343@gmail.com>:
So what's happening is that when you're save_as_dataset at the end of your notebook, you save data with incompatible shapes. In particular, 'density' and 'temperature' have shape (128) while 'cell_mass' has shape (128, 128). We should probably error out in cases like this because the resulting dataset is corrupt - all fields in a single dataset should have the same shape. The error you're seeing happens when yt tries to create an empty array to hold the cell_mass data from your saved dataset by using the shape of one of the bin fields. This leads to an error because the cell_mass field in your saved dataset has a different shape.

One thing that might be immediately helpful is to save your individual profile objects you want to sum, reload those, and then combine the data? You can save profile objects with save_as_dataset:

http://yt-project.org/docs/dev/analyzing/saving_data.html#profiles

Another option would be to eschew using the ytdata frontend entirely and just save the density, temperature, and cell_mass profile fields you want save using e.g. h5py and then construct the plot you want to make using those data.

It's not clear to me what you ultimately want to do with your saved data - maybe if you explain that in a little more detail I can suggest what you might want to do to restructure your workflow.

On Fri, Nov 3, 2017 at 5:53 PM, ｔyuta <y0u1t1a5.t@gmail.com> wrote:
I tried on "Isolatedgalaxy" data. I made all the outputs to comments.




2017-11-03 18:29 GMT-04:00 Nathan Goldbaum <nathan12343@gmail.com>:
So the error is coming from inside yt. Somehow yt is creating an array with shape (128) and trying to fill it with values from an array with shape (128, 128), causing numpy to complain. Unfortunately I'm still not sure what the underlying issue is just from looking at your code and the traceback. It's possible it's a yt bug, it's also possible that it's an issue in your notebook.

Is there any chance you can make a script I can run that triggers the issue? That will make it much easier to understand what's going wrong. You could also just share your full notebook.

If you can, please make use of a dataset from yt-project.org/data in your script. If that's not possible, you can also upload a dataset to the yt curldrop using the "yt upload" command line tool:

$ yt upload /path/to/dataset.tar.gz

and then include the link to the dataset that yt upload will print out once it's finished uploading the data.

-Nathan

On Fri, Nov 3, 2017 at 5:15 PM, ｔyuta <y0u1t1a5.t@gmail.com> wrote:
Dear Nathan,

Sorry for the incompleteness.
This is the error message from jupyter notebook:

ValueErrorTraceback (most recent call last)
<ipython-input-63-383c90a16917> in <module>()
     11                             fields=["cell_mass"],
     12                             weight_field=None,
---> 13                             extrema=dict(density=(1e-30, 1e-20), temperature=(1e1, 1e8))
     14                            )
     15 

/home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in create_profile(data_source, bin_fields, fields, n_bins, extrema, logs, units, weight_field, accumulation, fractional, deposition)
   1093     setattr(obj, "fractional", fractional)
   1094     if fields is not None:
-> 1095         obj.add_fields([field for field in fields])
   1096     for field in fields:
   1097         if fractional:

/home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in add_fields(self, fields)
    125         citer = self.data_source.chunks([], "io")
    126         for chunk in parallel_objects(citer):
--> 127             self._bin_chunk(chunk, fields, temp_storage)
    128         self._finalize_storage(fields, temp_storage)
    129 

/home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in _bin_chunk(self, chunk, fields, storage)
    565 
    566     def _bin_chunk(self, chunk, fields, storage):
--> 567         rv = self._get_data(chunk, fields)
    568         if rv is None: return
    569         fdata, wdata, (bf_x, bf_y) = rv

/home/ytarumi/.local/yt/yt/data_objects/profiles.pyc in _get_data(self, chunk, fields)
    253         for i, field in enumerate(fields):
    254             units = chunk.ds.field_info[field].output_units
--> 255             arr[:,i] = chunk[field][filter].in_units(units)
    256         if self.weight_field is not None:
    257             units = chunk.ds.field_info[self.weight_field].output_units

ValueError: could not broadcast input array from shape (128,128) into shape (128)

2017-11-03 17:08 GMT-04:00 Nathan Goldbaum <nathan12343@gmail.com>:
Can you share the full traceback along with the error message you're seeing? You can use gist.github.com to avoid cluttering your e-mail.

On Fri, Nov 3, 2017 at 3:54 PM, ｔyuta <y0u1t1a5.t@gmail.com> wrote:
Dear yt-users,

Hi, I want to ask you about phase plot. Some days ago, I asked how to sum up different datasets to make one profile plot, and it worked fine. This time, I tried to do the same thing in phase plot, but it did not work fine. Could you help me find out what the problem is?

(inside for loop):
     profile_medium = yt.create_profile(
             data_source=medium_region,
             bin_fields=["density", "temperature"],
             fields=["cell_mass"],
             n_bins=[128, 128],
             units=dict(),
             logs=dict(),
             weight_field=None,
             extrema=dict(density=(1e-30, 1e-20), temperature=(1e1, 1e8))
          )
     plot_med = yt.ProfilePlot.from_profiles(profile_medium)
     profile_med = plot_med.profiles[0]
     profiles_sum_med = profiles_sum_med + profile_med["cell_mass"]
    

my_data_med = {"density": profile_med.x,   
           "temperature": profile_med.y, 
              "cell_mass": profiles_sum_med}
fake_ds_med = {"current_time": yt.YTQuantity(10, "Myr")} 
yt.save_as_dataset(fake_ds_med, "phaseplots_med_all.h5", my_data_med)

ds = yt.load("phaseplots_med_all.h5")
ad = ds.data

profile = yt.create_profile(ad,
                            ["density", "temperature"],
                            n_bins=[128, 128],
                            fields=["cell_mass"], 
                            weight_field=None, 
                            extrema=dict(density=(1e-30, 1e-20), temperature=(1e1, 1e8))
                           )

and I got this error:

could not broadcast input array from shape (128,128) into shape (128)

Sincerely,
Y.T.

2017-10-26 16:15 GMT-04:00 ｔyuta <y0u1t1a5.t@gmail.com>:
Hi Suoqing Ji,

Thanks! I'll try that.

2017-10-26 15:59 GMT-04:00 Suoqing Ji <suoqing@physics.ucsb.edu>:
Hi t yuta,

What you can do is to get the data as numpy array directly from the profiles you’ve generated, and sum them up by yourself. You can follow the last example at: http://yt-project.org/docs/dev/visualizing/plots.html#d-profile-plots

For each dataset, you can have one profile:
profile = plot.profiles[0]

Then you can sum up profile[‘cell_mass'] over different datasets, and plot it against profile.x

Best wishes,
—
Suoqing Ji
Ph.D Candidate
Department of Physics
University of California, Santa Barbara

On Oct 26, 2017, 12:51 PM -0700, ｔyuta <y0u1t1a5.t@gmail.com>, wrote:
Dear yt-users,

Hi, I want to ask you about profile plot. I'm analyzing Enzo data, and I'm trying to plot metallicity-cellmass of different 12 datasets into one curve. I can plot 12 curves on one image, but that's not what I want. Does anyone know some good way, or links to documentation?
I've attached the 12-curve plot. I want to sum up all the curves to get one curve.

Thanks,
Y.T.埋め込み画像 2
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org




_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org