derived field and TotalQuantity from two sources possible?
Hi, I was wondering if it is possible to specify two different sources of input to create a derived field. Currently with a single source, I have a derived field defined as: def _HII_HFraction(field, data): return data["HII_Density"]/(data["HI_Density"]+data["HII_Density"]) add_field("HII_HFraction", function=_HII_HFraction, units=r"\frac{\rho_{HII}}{\rho_H}") And I can access the total quantity in parallel pf=load(file) dd = pf.h.all_data() dd.quantities["TotalQuantity"]("HII_HFraction")[0] But I'm dealing with pf1=load(file1) pf2=load(file2) pf1 has the HI_Density field data and pf2 has the HII_Density field data. Is there a way to create a derived field and use TotalQuantity to operate on the data in parallel? From G.S. PS. I guess my alternative is to glue the two HDF5 files into one, but want to avoid it if possible.
Hi G.S., This is pretty far outside of how fields are designed to work in yt. The access of data in yt isn't currently designed to work with more than one pf. I don't actually see a way to do this without combining the data so that it can be seen as a single pf. If parallel wasn't an issue and this were only a unigrid run, you might be able to read in all the data at once from both pfs and do some of this by hand. If this is AMR, then I think this may not be possible at all, considering the hierarchy is unlikely to be exactly the same between two different datasets. Britton On Mon, Jul 18, 2011 at 5:27 PM, <gso@physics.ucsd.edu> wrote:
Hi,
I was wondering if it is possible to specify two different sources of input to create a derived field. Currently with a single source, I have a derived field defined as:
def _HII_HFraction(field, data): return data["HII_Density"]/(data["HI_Density"]+data["HII_Density"])
add_field("HII_HFraction", function=_HII_HFraction, units=r"\frac{\rho_{HII}}{\rho_H}")
And I can access the total quantity in parallel
pf=load(file)
dd = pf.h.all_data()
dd.quantities["TotalQuantity"]("HII_HFraction")[0]
But I'm dealing with
pf1=load(file1) pf2=load(file2)
pf1 has the HI_Density field data and pf2 has the HII_Density field data. Is there a way to create a derived field and use TotalQuantity to operate on the data in parallel?
From G.S.
PS. I guess my alternative is to glue the two HDF5 files into one, but want to avoid it if possible.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Geoffrey, depending on the specifics of your dataset, you may be best off making a covering grid for both datasets. If the covering grid is small enough to fit into memory, then you can do the rest by hand, as Britton suggests. -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Hi Geoffrey, A question: is this actually an Enzo dataset (as in, .cpuNNNN files, with the parameter, hierarchy, etc. files as well), or are you talking about monolothic HDF5 files containing a single field that was created by some third party? If it's the latter, you're better off just using, say, h5py to read each file in separately, and divide them by each other. I don't think that the yt machinery is designed for brick-of-bytes HDF5 files (though I may very well be wrong). --B On Mon, Jul 18, 2011 at 5:27 PM, <gso@physics.ucsd.edu> wrote:
Hi,
I was wondering if it is possible to specify two different sources of input to create a derived field. Currently with a single source, I have a derived field defined as:
def _HII_HFraction(field, data): return data["HII_Density"]/(data["HI_Density"]+data["HII_Density"])
add_field("HII_HFraction", function=_HII_HFraction, units=r"\frac{\rho_{HII}}{\rho_H}")
And I can access the total quantity in parallel
pf=load(file)
dd = pf.h.all_data()
dd.quantities["TotalQuantity"]("HII_HFraction")[0]
But I'm dealing with
pf1=load(file1) pf2=load(file2)
pf1 has the HI_Density field data and pf2 has the HII_Density field data. Is there a way to create a derived field and use TotalQuantity to operate on the data in parallel?
From G.S.
PS. I guess my alternative is to glue the two HDF5 files into one, but want to avoid it if possible.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
The Enzo files are in different folders separated by fields like: RD0015_HI_Density/RD0000.cpuNNNN and RD0015_HII_Density/RD0000.cpuNNNN So I think I'll just glue the two fields into one and put them in another folder. Thanks for the responses. From G.S.
Hi Geoffrey,
A question: is this actually an Enzo dataset (as in, .cpuNNNN files, with the parameter, hierarchy, etc. files as well), or are you talking about monolothic HDF5 files containing a single field that was created by some third party? If it's the latter, you're better off just using, say, h5py to read each file in separately, and divide them by each other. I don't think that the yt machinery is designed for brick-of-bytes HDF5 files (though I may very well be wrong).
--B
On Mon, Jul 18, 2011 at 5:27 PM, <gso@physics.ucsd.edu> wrote:
Hi,
I was wondering if it is possible to specify two different sources of input to create a derived field. Currently with a single source, I have a derived field defined as:
def _HII_HFraction(field, data): return data["HII_Density"]/(data["HI_Density"]+data["HII_Density"])
add_field("HII_HFraction", function=_HII_HFraction, units=r"\frac{\rho_{HII}}{\rho_H}")
And I can access the total quantity in parallel
pf=load(file)
dd = pf.h.all_data()
dd.quantities["TotalQuantity"]("HII_HFraction")[0]
But I'm dealing with
pf1=load(file1) pf2=load(file2)
pf1 has the HI_Density field data and pf2 has the HII_Density field data. Is there a way to create a derived field and use TotalQuantity to operate on the data in parallel?
From G.S.
PS. I guess my alternative is to glue the two HDF5 files into one, but want to avoid it if possible.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Geoffrey, You could use the Stream handler to do this. There's an example on the blog. Alternately, you could hack together an IO modification that handles it, or something. It could be done. But, it looks like you are using a private version of Enzo that writes out a single field at a time into different files, or you are using modified/processed Enzo outputs. Either way, it's not something that we have access to, as yt developers, so we aren't really in a place to be able to help out that much. This type of change and fix is certainly tractable in yt, as it is evident from your final message that these are really a *single* parameter file that has been split, but it's not something we have any experience with. -Matt On Mon, Jul 18, 2011 at 4:07 PM, <gso@physics.ucsd.edu> wrote:
The Enzo files are in different folders separated by fields like: RD0015_HI_Density/RD0000.cpuNNNN and RD0015_HII_Density/RD0000.cpuNNNN
So I think I'll just glue the two fields into one and put them in another folder.
Thanks for the responses.
From G.S.
Hi Geoffrey,
A question: is this actually an Enzo dataset (as in, .cpuNNNN files, with the parameter, hierarchy, etc. files as well), or are you talking about monolothic HDF5 files containing a single field that was created by some third party? If it's the latter, you're better off just using, say, h5py to read each file in separately, and divide them by each other. I don't think that the yt machinery is designed for brick-of-bytes HDF5 files (though I may very well be wrong).
--B
On Mon, Jul 18, 2011 at 5:27 PM, <gso@physics.ucsd.edu> wrote:
Hi,
I was wondering if it is possible to specify two different sources of input to create a derived field. Currently with a single source, I have a derived field defined as:
def _HII_HFraction(field, data): return data["HII_Density"]/(data["HI_Density"]+data["HII_Density"])
add_field("HII_HFraction", function=_HII_HFraction, units=r"\frac{\rho_{HII}}{\rho_H}")
And I can access the total quantity in parallel
pf=load(file)
dd = pf.h.all_data()
dd.quantities["TotalQuantity"]("HII_HFraction")[0]
But I'm dealing with
pf1=load(file1) pf2=load(file2)
pf1 has the HI_Density field data and pf2 has the HII_Density field data. Is there a way to create a derived field and use TotalQuantity to operate on the data in parallel?
From G.S.
PS. I guess my alternative is to glue the two HDF5 files into one, but want to avoid it if possible.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (5)
-
Brian O'Shea -
Britton Smith -
gso＠physics.ucsd.edu -
Matthew Turk -
Stephen Skory