Hello, I am an astronomy graduate student at Columbia University, and am trying to use YT to create a 3D visualization of GADGET simulations. Matt Turk suggested I send an email to this list while he is traveling. I have seen that the source code for visualizing GADGET data is in yt/frontends/sph/data_structures.py, but am not sure how to access/use this? I have looked at: http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/e209c55b6aaa4a9... and http://yt-project.org/doc/visualizing/volume_rendering.html#generating-a-hom... but remain unsure of where to find the relevant source code/what I do with it after I find it. Matt explained that yt/frontends/sph/data_structures.py is "where the data frontend is. The broader yt codebase uses that to index and manage the data but it uses generic interfaces on top," and encouraged me to email the list for more information. Where is the relevant code? Also, where does the generic vs. frontend interface distinction appear? Best, Andrew Weis
Hi Andrew, Thanks for writing. I'm responding since I've had some experience working with yt's SPH frontends. Matt and others might have more information as well. Support for Gadget data is still not finished. In the yt-3.0 branch of the development repository, you'll should be able to load the data and do some basic visualization and analysis tasks. The data is available both in its raw form as particles and also by depositing the particle data onto an octree and then visualization and analyzing the octree. The first notebook you linked to describes the basic of loading, visualizating, and analyzing SPH data. If you want to work with a more stable codebase (although one that is not being actively developed) you should be able to do many analysis and viz tasks using the yt-3.0 branch of the main development repository: https://bitbucket.org/yt_analysis/yt The 3.0 branch was a big refactoring of the underlying data selection algorithms yt uses to load data off disk. This made it possible to present the same user interface for visualizing datasets from particle codes like Gadget, octree AMR codes like Ramses, and patch-based AMR codes like Enzo. Support for SPH smoothing is available in a separate repository. In this experimental development repository we've refactored the code to use a symbolic units library to handle unit conversions and detect code bugs using dimensional analysis. We've also completely refactored the way frontends are written and fields are set up and detected. If you want to dive in to the code, I'd suggest starting with the experimental version of yt. This may be a bit more rocky at first - there might be bugs - but will be more rewarding in the end as this is the direction the codebase is going. The work is ongoing in the yt-3.0 branch on Matt Turk's fork of yt: https://bitbucket.org/MatthewTurk/yt In both cases there is unfortunately not a lot of documentation at the moment. This is something that we're working on right now. Mailing list archives as well as the YTEP listing (http://ytep.readthedocs.org/) might prove to be useful. As for your questions about yt internals, I've written some basic description below. I would also encourage you to read the yt method paper and to take a look YTEP-0001 and YTEP-0005, which describe the new geometry system. The basic data structure yt uses to represent an on-disk dataset is the StaticOutput class. The SPH frontend defines a GadgetStaticOutput class as well as a GadgetHDF5StaticOutput subclass to represent HDF5 Gadget datasets. You can create a new StaticOutput instance using the 'load' function or by directly instantiating an instance of a StaticOutput subclass defined in one of the frontends. You'll need to look at the parameters of the __init__ method to figure out exactly how to load the data. For Gadget, the 'load' convenience function only works with HDF5 datasets, so if you are have data written in Gadget's binary format, you'll need to load your data by calling GadgetStaticOutput directly. StaticOutput instances have as an attribute an instance of GeometryHandler. This class handles the indexing and selection of data. This is really the heart of yt's hard-core numerics, and is written in a way that is very accessible at a high level. If you want to dive into the algorithms, I think it would help to look over the GeometryHandler class as well as its subclass the ParticleGeometryHandler. That said, yt's interface is more or less agnostic to the underlying algorithm used to index the data. Do you have specific questions about how to load and visualize your datasets? Hope that's helpful and not too much of a manifesto :) -Nathan
Thank you for the detailed response, Nathan. To clarify, this is the YT method paper? http://iopscience.iop.org/0067-0049/192/1/9/pdf/0067-0049_192_1_9.pdf Does the fact that these simulations only contain dark matter affect what you say at all? I will probably have more questions in the near future; at the moment, though, I am confused about where in the source code to find certain functions. For instance, the load function appears to be called after we do: from yt.mods import * but when I try this line in the terminal, I get an error that says there is "no module named yt.mods." Why might this be? I downloaded and installed the source code from: http://hg.yt-project.org/yt/raw/yt-3.0/doc/install_script.sh but I cannot find yt.mods or yt/frontends. Where would those be? Is there further software I may need to install? Thanks again, Andrew Weis On Mon, Feb 24, 2014 at 4:58 PM, Nathan Goldbaum <nathan12343@gmail.com>wrote:
Hi Andrew,
Thanks for writing. I'm responding since I've had some experience working with yt's SPH frontends. Matt and others might have more information as well.
Support for Gadget data is still not finished. In the yt-3.0 branch of the development repository, you'll should be able to load the data and do some basic visualization and analysis tasks. The data is available both in its raw form as particles and also by depositing the particle data onto an octree and then visualization and analyzing the octree. The first notebook you linked to describes the basic of loading, visualizating, and analyzing SPH data.
If you want to work with a more stable codebase (although one that is not being actively developed) you should be able to do many analysis and viz tasks using the yt-3.0 branch of the main development repository: https://bitbucket.org/yt_analysis/yt
The 3.0 branch was a big refactoring of the underlying data selection algorithms yt uses to load data off disk. This made it possible to present the same user interface for visualizing datasets from particle codes like Gadget, octree AMR codes like Ramses, and patch-based AMR codes like Enzo.
Support for SPH smoothing is available in a separate repository. In this experimental development repository we've refactored the code to use a symbolic units library to handle unit conversions and detect code bugs using dimensional analysis. We've also completely refactored the way frontends are written and fields are set up and detected.
If you want to dive in to the code, I'd suggest starting with the experimental version of yt. This may be a bit more rocky at first - there might be bugs - but will be more rewarding in the end as this is the direction the codebase is going. The work is ongoing in the yt-3.0 branch on Matt Turk's fork of yt: https://bitbucket.org/MatthewTurk/yt
In both cases there is unfortunately not a lot of documentation at the moment. This is something that we're working on right now. Mailing list archives as well as the YTEP listing (http://ytep.readthedocs.org/) might prove to be useful.
As for your questions about yt internals, I've written some basic description below. I would also encourage you to read the yt method paper and to take a look YTEP-0001 and YTEP-0005, which describe the new geometry system.
The basic data structure yt uses to represent an on-disk dataset is the StaticOutput class. The SPH frontend defines a GadgetStaticOutput class as well as a GadgetHDF5StaticOutput subclass to represent HDF5 Gadget datasets. You can create a new StaticOutput instance using the 'load' function or by directly instantiating an instance of a StaticOutput subclass defined in one of the frontends. You'll need to look at the parameters of the __init__ method to figure out exactly how to load the data. For Gadget, the 'load' convenience function only works with HDF5 datasets, so if you are have data written in Gadget's binary format, you'll need to load your data by calling GadgetStaticOutput directly.
StaticOutput instances have as an attribute an instance of GeometryHandler. This class handles the indexing and selection of data. This is really the heart of yt's hard-core numerics, and is written in a way that is very accessible at a high level. If you want to dive into the algorithms, I think it would help to look over the GeometryHandler class as well as its subclass the ParticleGeometryHandler. That said, yt's interface is more or less agnostic to the underlying algorithm used to index the data.
Do you have specific questions about how to load and visualize your datasets?
Hope that's helpful and not too much of a manifesto :)
-Nathan _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Did you 'activate' the yt installation? The install script builds an isolated environment using its own python interpreter. More detail here: http://yt-project.org/docs/dev/installing.html#activating-your-installation The load function is in convenience.py: https://bitbucket.org/yt_analysis/yt/src/61e6b84f875cc8fcf25b5b1e67ddd501a19... Cheers, Nathan On Tue, Feb 25, 2014 at 12:03 PM, Andrew Philip Weis <apw2133@columbia.edu> wrote:
Thank you for the detailed response, Nathan. To clarify, this is the YT method paper? http://iopscience.iop.org/0067-0049/192/1/9/pdf/0067-0049_192_1_9.pdf
Does the fact that these simulations only contain dark matter affect what you say at all?
I will probably have more questions in the near future; at the moment, though, I am confused about where in the source code to find certain functions. For instance, the load function appears to be called after we do:
from yt.mods import *
but when I try this line in the terminal, I get an error that says there is "no module named yt.mods." Why might this be? I downloaded and installed the source code from: http://hg.yt-project.org/yt/raw/yt-3.0/doc/install_script.sh but I cannot find yt.mods or yt/frontends. Where would those be? Is there further software I may need to install? Thanks again,
Andrew Weis
On Mon, Feb 24, 2014 at 4:58 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi Andrew,
Thanks for writing. I'm responding since I've had some experience working with yt's SPH frontends. Matt and others might have more information as well.
Support for Gadget data is still not finished. In the yt-3.0 branch of the development repository, you'll should be able to load the data and do some basic visualization and analysis tasks. The data is available both in its raw form as particles and also by depositing the particle data onto an octree and then visualization and analyzing the octree. The first notebook you linked to describes the basic of loading, visualizating, and analyzing SPH data.
If you want to work with a more stable codebase (although one that is not being actively developed) you should be able to do many analysis and viz tasks using the yt-3.0 branch of the main development repository: https://bitbucket.org/yt_analysis/yt
The 3.0 branch was a big refactoring of the underlying data selection algorithms yt uses to load data off disk. This made it possible to present the same user interface for visualizing datasets from particle codes like Gadget, octree AMR codes like Ramses, and patch-based AMR codes like Enzo.
Support for SPH smoothing is available in a separate repository. In this experimental development repository we've refactored the code to use a symbolic units library to handle unit conversions and detect code bugs using dimensional analysis. We've also completely refactored the way frontends are written and fields are set up and detected.
If you want to dive in to the code, I'd suggest starting with the experimental version of yt. This may be a bit more rocky at first - there might be bugs - but will be more rewarding in the end as this is the direction the codebase is going. The work is ongoing in the yt-3.0 branch on Matt Turk's fork of yt: https://bitbucket.org/MatthewTurk/yt
In both cases there is unfortunately not a lot of documentation at the moment. This is something that we're working on right now. Mailing list archives as well as the YTEP listing (http://ytep.readthedocs.org/) might prove to be useful.
As for your questions about yt internals, I've written some basic description below. I would also encourage you to read the yt method paper and to take a look YTEP-0001 and YTEP-0005, which describe the new geometry system.
The basic data structure yt uses to represent an on-disk dataset is the StaticOutput class. The SPH frontend defines a GadgetStaticOutput class as well as a GadgetHDF5StaticOutput subclass to represent HDF5 Gadget datasets. You can create a new StaticOutput instance using the 'load' function or by directly instantiating an instance of a StaticOutput subclass defined in one of the frontends. You'll need to look at the parameters of the __init__ method to figure out exactly how to load the data. For Gadget, the 'load' convenience function only works with HDF5 datasets, so if you are have data written in Gadget's binary format, you'll need to load your data by calling GadgetStaticOutput directly.
StaticOutput instances have as an attribute an instance of GeometryHandler. This class handles the indexing and selection of data. This is really the heart of yt's hard-core numerics, and is written in a way that is very accessible at a high level. If you want to dive into the algorithms, I think it would help to look over the GeometryHandler class as well as its subclass the ParticleGeometryHandler. That said, yt's interface is more or less agnostic to the underlying algorithm used to index the data.
Do you have specific questions about how to load and visualize your datasets?
Hope that's helpful and not too much of a manifesto :)
-Nathan _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Sorry for the delayed reply. I had forgotten that step, as it turns out. Thank you for those links. I activated the installation, and endeavored to follow the steps at http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/e209c55b6aaa4a9... Most of the steps appeared to work, but when I tried to display the projection plot with pw.show(), nothing appeared. I think the issue may lie with the definition of center; is the definition on the website a general definition, or specific to one simulation? I am referring to the lines: # This is where the halo appears to be centered, although the halo center on the Agora website is somewhat different center=np.array([29.754, 32.1, 28.29]) What do these numbers mean? It is possible they may be different for my simulations, I would think, but how do we know that a particular halo is centered there? Thanks again, Andrew Weis On Tue, Feb 25, 2014 at 3:16 PM, Nathan Goldbaum <nathan12343@gmail.com>wrote:
Did you 'activate' the yt installation? The install script builds an isolated environment using its own python interpreter.
More detail here: http://yt-project.org/docs/dev/installing.html#activating-your-installation
The load function is in convenience.py:
https://bitbucket.org/yt_analysis/yt/src/61e6b84f875cc8fcf25b5b1e67ddd501a19...
Cheers,
Nathan
On Tue, Feb 25, 2014 at 12:03 PM, Andrew Philip Weis <apw2133@columbia.edu> wrote:
Thank you for the detailed response, Nathan. To clarify, this is the YT method paper? http://iopscience.iop.org/0067-0049/192/1/9/pdf/0067-0049_192_1_9.pdf
Does the fact that these simulations only contain dark matter affect what you say at all?
I will probably have more questions in the near future; at the moment, though, I am confused about where in the source code to find certain functions. For instance, the load function appears to be called after we do:
from yt.mods import *
but when I try this line in the terminal, I get an error that says there is "no module named yt.mods." Why might this be? I downloaded and installed the source code from: http://hg.yt-project.org/yt/raw/yt-3.0/doc/install_script.sh but I cannot find yt.mods or yt/frontends. Where would those be? Is there further software I may need to install? Thanks again,
Andrew Weis
On Mon, Feb 24, 2014 at 4:58 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi Andrew,
Thanks for writing. I'm responding since I've had some experience working with yt's SPH frontends. Matt and others might have more information as well.
Support for Gadget data is still not finished. In the yt-3.0 branch of the development repository, you'll should be able to load the data and do some basic visualization and analysis tasks. The data is available both in its raw form as particles and also by depositing the particle data onto an octree and then visualization and analyzing the octree. The first notebook you linked to describes the basic of loading, visualizating, and analyzing SPH data.
If you want to work with a more stable codebase (although one that is not being actively developed) you should be able to do many analysis and viz tasks using the yt-3.0 branch of the main development repository: https://bitbucket.org/yt_analysis/yt
The 3.0 branch was a big refactoring of the underlying data selection algorithms yt uses to load data off disk. This made it possible to present the same user interface for visualizing datasets from particle codes like Gadget, octree AMR codes like Ramses, and patch-based AMR codes like Enzo.
Support for SPH smoothing is available in a separate repository. In this experimental development repository we've refactored the code to use a symbolic units library to handle unit conversions and detect code bugs using dimensional analysis. We've also completely refactored the way frontends are written and fields are set up and detected.
If you want to dive in to the code, I'd suggest starting with the experimental version of yt. This may be a bit more rocky at first - there might be bugs - but will be more rewarding in the end as this is the direction the codebase is going. The work is ongoing in the yt-3.0 branch on Matt Turk's fork of yt: https://bitbucket.org/MatthewTurk/yt
In both cases there is unfortunately not a lot of documentation at the moment. This is something that we're working on right now. Mailing list archives as well as the YTEP listing (http://ytep.readthedocs.org/) might prove to be useful.
As for your questions about yt internals, I've written some basic description below. I would also encourage you to read the yt method paper and to take a look YTEP-0001 and YTEP-0005, which describe the new geometry system.
The basic data structure yt uses to represent an on-disk dataset is the StaticOutput class. The SPH frontend defines a GadgetStaticOutput class as well as a GadgetHDF5StaticOutput subclass to represent HDF5 Gadget datasets. You can create a new StaticOutput instance using the 'load' function or by directly instantiating an instance of a StaticOutput subclass defined in one of the frontends. You'll need to look at the parameters of the __init__ method to figure out exactly how to load the data. For Gadget, the 'load' convenience function only works with HDF5 datasets, so if you are have data written in Gadget's binary format, you'll need to load your data by calling GadgetStaticOutput directly.
StaticOutput instances have as an attribute an instance of GeometryHandler. This class handles the indexing and selection of data. This is really the heart of yt's hard-core numerics, and is written in a way that is very accessible at a high level. If you want to dive into the algorithms, I think it would help to look over the GeometryHandler class as well as its subclass the ParticleGeometryHandler. That said, yt's interface is more or less agnostic to the underlying algorithm used to index the data.
Do you have specific questions about how to load and visualize your datasets?
Hope that's helpful and not too much of a manifesto :)
-Nathan _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
On Sun, Mar 2, 2014 at 4:32 PM, Andrew Weis <aweis@astro.columbia.edu> wrote:
Sorry for the delayed reply. I had forgotten that step, as it turns out. Thank you for those links. I activated the installation, and endeavored to follow the steps at http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/e209c55b6aaa4a9...
Most of the steps appeared to work, but when I tried to display the projection plot with pw.show(), nothing appeared.
Were you running those steps inside of an IPython notebook? If so, what happens when you add '%matplotlib inline' to the top of the notebook? If you weren't running of a notebook, that is the expected behavior. You can save the plot to disk using pw.save() instead.
I think the issue may lie with the definition of center; is the definition on the website a general definition, or specific to one simulation? I am referring to the lines:
# This is where the halo appears to be centered, although the halo center on the Agora website is somewhat different
center=np.array([29.754, 32.1, 28.29])
What do these numbers mean? It is possible they may be different for my simulations, I would think, but how do we know that a particular halo is centered there? Thanks again,
My guess is that Matt chose those as the location of an interesting halo in the dataset he was looking at. You will need to supply a different center if you are looking at a different dataset. If you're looking at a cosmology simulation, you will need to run a halo finder to get the positions of the halos you want to look at.
Andrew Weis
On Tue, Feb 25, 2014 at 3:16 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Did you 'activate' the yt installation? The install script builds an isolated environment using its own python interpreter.
More detail here:
http://yt-project.org/docs/dev/installing.html#activating-your-installation
The load function is in convenience.py:
https://bitbucket.org/yt_analysis/yt/src/61e6b84f875cc8fcf25b5b1e67ddd501a19...
Cheers,
Nathan
On Tue, Feb 25, 2014 at 12:03 PM, Andrew Philip Weis <apw2133@columbia.edu> wrote:
Thank you for the detailed response, Nathan. To clarify, this is the YT method paper? http://iopscience.iop.org/0067-0049/192/1/9/pdf/0067-0049_192_1_9.pdf
Does the fact that these simulations only contain dark matter affect what you say at all?
I will probably have more questions in the near future; at the moment, though, I am confused about where in the source code to find certain functions. For instance, the load function appears to be called after we do:
from yt.mods import *
but when I try this line in the terminal, I get an error that says there is "no module named yt.mods." Why might this be? I downloaded and installed the source code from: http://hg.yt-project.org/yt/raw/yt-3.0/doc/install_script.sh but I cannot find yt.mods or yt/frontends. Where would those be? Is there further software I may need to install? Thanks again,
Andrew Weis
On Mon, Feb 24, 2014 at 4:58 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi Andrew,
Thanks for writing. I'm responding since I've had some experience working with yt's SPH frontends. Matt and others might have more information as well.
Support for Gadget data is still not finished. In the yt-3.0 branch of the development repository, you'll should be able to load the data and do some basic visualization and analysis tasks. The data is available both in its raw form as particles and also by depositing the particle data onto an octree and then visualization and analyzing the octree. The first notebook you linked to describes the basic of loading, visualizating, and analyzing SPH data.
If you want to work with a more stable codebase (although one that is not being actively developed) you should be able to do many analysis and viz tasks using the yt-3.0 branch of the main development repository: https://bitbucket.org/yt_analysis/yt
The 3.0 branch was a big refactoring of the underlying data selection algorithms yt uses to load data off disk. This made it possible to present the same user interface for visualizing datasets from particle codes like Gadget, octree AMR codes like Ramses, and patch-based AMR codes like Enzo.
Support for SPH smoothing is available in a separate repository. In this experimental development repository we've refactored the code to use a symbolic units library to handle unit conversions and detect code bugs using dimensional analysis. We've also completely refactored the way frontends are written and fields are set up and detected.
If you want to dive in to the code, I'd suggest starting with the experimental version of yt. This may be a bit more rocky at first - there might be bugs - but will be more rewarding in the end as this is the direction the codebase is going. The work is ongoing in the yt-3.0 branch on Matt Turk's fork of yt: https://bitbucket.org/MatthewTurk/yt
In both cases there is unfortunately not a lot of documentation at the moment. This is something that we're working on right now. Mailing list archives as well as the YTEP listing (http://ytep.readthedocs.org/) might prove to be useful.
As for your questions about yt internals, I've written some basic description below. I would also encourage you to read the yt method paper and to take a look YTEP-0001 and YTEP-0005, which describe the new geometry system.
The basic data structure yt uses to represent an on-disk dataset is the StaticOutput class. The SPH frontend defines a GadgetStaticOutput class as well as a GadgetHDF5StaticOutput subclass to represent HDF5 Gadget datasets. You can create a new StaticOutput instance using the 'load' function or by directly instantiating an instance of a StaticOutput subclass defined in one of the frontends. You'll need to look at the parameters of the __init__ method to figure out exactly how to load the data. For Gadget, the 'load' convenience function only works with HDF5 datasets, so if you are have data written in Gadget's binary format, you'll need to load your data by calling GadgetStaticOutput directly.
StaticOutput instances have as an attribute an instance of GeometryHandler. This class handles the indexing and selection of data. This is really the heart of yt's hard-core numerics, and is written in a way that is very accessible at a high level. If you want to dive into the algorithms, I think it would help to look over the GeometryHandler class as well as its subclass the ParticleGeometryHandler. That said, yt's interface is more or less agnostic to the underlying algorithm used to index the data.
Do you have specific questions about how to load and visualize your datasets?
Hope that's helpful and not too much of a manifesto :)
-Nathan _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Ok thanks. I am indeed looking at a cosmology simulation; I am trying to locate the dark matter filaments in the simulation, using the ORIGAMI code, and am essentially trying to use YT to confirm that the filament particles are appropriately identified. I hope that makes things clearer? Andrew Weis On Sun, Mar 2, 2014 at 8:07 PM, Nathan Goldbaum <nathan12343@gmail.com>wrote:
On Sun, Mar 2, 2014 at 4:32 PM, Andrew Weis <aweis@astro.columbia.edu> wrote:
Sorry for the delayed reply. I had forgotten that step, as it turns out. Thank you for those links. I activated the installation, and endeavored to follow the steps at
http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/e209c55b6aaa4a9...
Most of the steps appeared to work, but when I tried to display the projection plot with pw.show(), nothing appeared.
Were you running those steps inside of an IPython notebook? If so, what happens when you add '%matplotlib inline' to the top of the notebook?
If you weren't running of a notebook, that is the expected behavior. You can save the plot to disk using pw.save() instead.
I think the issue may lie with the definition of center; is the definition on the website a general definition, or specific to one simulation? I am referring to the lines:
# This is where the halo appears to be centered, although the halo center on the Agora website is somewhat different
center=np.array([29.754, 32.1, 28.29])
What do these numbers mean? It is possible they may be different for my simulations, I would think, but how do we know that a particular halo is centered there? Thanks again,
My guess is that Matt chose those as the location of an interesting halo in the dataset he was looking at. You will need to supply a different center if you are looking at a different dataset.
If you're looking at a cosmology simulation, you will need to run a halo finder to get the positions of the halos you want to look at.
Andrew Weis
On Tue, Feb 25, 2014 at 3:16 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Did you 'activate' the yt installation? The install script builds an isolated environment using its own python interpreter.
More detail here:
The load function is in convenience.py:
https://bitbucket.org/yt_analysis/yt/src/61e6b84f875cc8fcf25b5b1e67ddd501a19...
Cheers,
Nathan
On Tue, Feb 25, 2014 at 12:03 PM, Andrew Philip Weis <apw2133@columbia.edu> wrote:
Thank you for the detailed response, Nathan. To clarify, this is the
YT
method paper? http://iopscience.iop.org/0067-0049/192/1/9/pdf/0067-0049_192_1_9.pdf
Does the fact that these simulations only contain dark matter affect what you say at all?
I will probably have more questions in the near future; at the moment, though, I am confused about where in the source code to find certain functions. For instance, the load function appears to be called after we do:
from yt.mods import *
but when I try this line in the terminal, I get an error that says
is "no module named yt.mods." Why might this be? I downloaded and installed the source code from: http://hg.yt-project.org/yt/raw/yt-3.0/doc/install_script.sh but I cannot find yt.mods or yt/frontends. Where would those be? Is there further software I may need to install? Thanks again,
Andrew Weis
On Mon, Feb 24, 2014 at 4:58 PM, Nathan Goldbaum < nathan12343@gmail.com> wrote:
Hi Andrew,
Thanks for writing. I'm responding since I've had some experience working with yt's SPH frontends. Matt and others might have more information as well.
Support for Gadget data is still not finished. In the yt-3.0 branch of the development repository, you'll should be able to load the data and do some basic visualization and analysis tasks. The data is available both in its raw form as particles and also by depositing
particle data onto an octree and then visualization and analyzing the octree. The first notebook you linked to describes the basic of loading, visualizating, and analyzing SPH data.
If you want to work with a more stable codebase (although one that is not being actively developed) you should be able to do many analysis and viz tasks using the yt-3.0 branch of the main development repository: https://bitbucket.org/yt_analysis/yt
The 3.0 branch was a big refactoring of the underlying data selection algorithms yt uses to load data off disk. This made it possible to present the same user interface for visualizing datasets from
codes like Gadget, octree AMR codes like Ramses, and patch-based AMR codes like Enzo.
Support for SPH smoothing is available in a separate repository. In this experimental development repository we've refactored the code to use a symbolic units library to handle unit conversions and detect code bugs using dimensional analysis. We've also completely refactored the way frontends are written and fields are set up and detected.
If you want to dive in to the code, I'd suggest starting with the experimental version of yt. This may be a bit more rocky at first - there might be bugs - but will be more rewarding in the end as this is the direction the codebase is going. The work is ongoing in the yt-3.0 branch on Matt Turk's fork of yt: https://bitbucket.org/MatthewTurk/yt
In both cases there is unfortunately not a lot of documentation at
moment. This is something that we're working on right now. Mailing list archives as well as the YTEP listing (http://ytep.readthedocs.org/) might prove to be useful.
As for your questions about yt internals, I've written some basic description below. I would also encourage you to read the yt method paper and to take a look YTEP-0001 and YTEP-0005, which describe the new geometry system.
The basic data structure yt uses to represent an on-disk dataset is the StaticOutput class. The SPH frontend defines a GadgetStaticOutput class as well as a GadgetHDF5StaticOutput subclass to represent HDF5 Gadget datasets. You can create a new StaticOutput instance using
http://yt-project.org/docs/dev/installing.html#activating-your-installation there the particle the the
'load' function or by directly instantiating an instance of a StaticOutput subclass defined in one of the frontends. You'll need to look at the parameters of the __init__ method to figure out exactly how to load the data. For Gadget, the 'load' convenience function only works with HDF5 datasets, so if you are have data written in Gadget's binary format, you'll need to load your data by calling GadgetStaticOutput directly.
StaticOutput instances have as an attribute an instance of GeometryHandler. This class handles the indexing and selection of data. This is really the heart of yt's hard-core numerics, and is written in a way that is very accessible at a high level. If you want to dive into the algorithms, I think it would help to look over the GeometryHandler class as well as its subclass the ParticleGeometryHandler. That said, yt's interface is more or less agnostic to the underlying algorithm used to index the data.
Do you have specific questions about how to load and visualize your datasets?
Hope that's helpful and not too much of a manifesto :)
-Nathan _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Realized I hadn't sent this from my astro account, so sending it from that one, just to be sure it sends. Andrew Weis On Sat, Mar 8, 2014 at 2:22 PM, Andrew Philip Weis <apw2133@columbia.edu>wrote:
Ok thanks. I am indeed looking at a cosmology simulation; I am trying to locate the dark matter filaments in the simulation, using the ORIGAMI code, and am essentially trying to use YT to confirm that the filament particles are appropriately identified. I hope that makes things clearer?
Andrew Weis
On Sun, Mar 2, 2014 at 8:07 PM, Nathan Goldbaum <nathan12343@gmail.com>wrote:
On Sun, Mar 2, 2014 at 4:32 PM, Andrew Weis <aweis@astro.columbia.edu> wrote:
Sorry for the delayed reply. I had forgotten that step, as it turns out. Thank you for those links. I activated the installation, and endeavored to follow the steps at
http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/e209c55b6aaa4a9...
Most of the steps appeared to work, but when I tried to display the projection plot with pw.show(), nothing appeared.
Were you running those steps inside of an IPython notebook? If so, what happens when you add '%matplotlib inline' to the top of the notebook?
If you weren't running of a notebook, that is the expected behavior. You can save the plot to disk using pw.save() instead.
I think the issue may lie with the definition of center; is the definition on the website a general definition, or specific to one simulation? I am referring to the lines:
# This is where the halo appears to be centered, although the halo center on the Agora website is somewhat different
center=np.array([29.754, 32.1, 28.29])
What do these numbers mean? It is possible they may be different for my simulations, I would think, but how do we know that a particular halo is centered there? Thanks again,
My guess is that Matt chose those as the location of an interesting halo in the dataset he was looking at. You will need to supply a different center if you are looking at a different dataset.
If you're looking at a cosmology simulation, you will need to run a halo finder to get the positions of the halos you want to look at.
Andrew Weis
On Tue, Feb 25, 2014 at 3:16 PM, Nathan Goldbaum <nathan12343@gmail.com
wrote:
Did you 'activate' the yt installation? The install script builds an isolated environment using its own python interpreter.
More detail here:
The load function is in convenience.py:
https://bitbucket.org/yt_analysis/yt/src/61e6b84f875cc8fcf25b5b1e67ddd501a19...
Cheers,
Nathan
On Tue, Feb 25, 2014 at 12:03 PM, Andrew Philip Weis <apw2133@columbia.edu> wrote:
Thank you for the detailed response, Nathan. To clarify, this is
method paper?
http://iopscience.iop.org/0067-0049/192/1/9/pdf/0067-0049_192_1_9.pdf
Does the fact that these simulations only contain dark matter affect what you say at all?
I will probably have more questions in the near future; at the
moment,
though, I am confused about where in the source code to find certain functions. For instance, the load function appears to be called after we do:
from yt.mods import *
but when I try this line in the terminal, I get an error that says
is "no module named yt.mods." Why might this be? I downloaded and installed the source code from: http://hg.yt-project.org/yt/raw/yt-3.0/doc/install_script.sh but I cannot find yt.mods or yt/frontends. Where would those be? Is there further software I may need to install? Thanks again,
Andrew Weis
On Mon, Feb 24, 2014 at 4:58 PM, Nathan Goldbaum < nathan12343@gmail.com> wrote:
Hi Andrew,
Thanks for writing. I'm responding since I've had some experience working with yt's SPH frontends. Matt and others might have more information as well.
Support for Gadget data is still not finished. In the yt-3.0 branch of the development repository, you'll should be able to load the
data
and do some basic visualization and analysis tasks. The data is available both in its raw form as particles and also by depositing
particle data onto an octree and then visualization and analyzing
octree. The first notebook you linked to describes the basic of loading, visualizating, and analyzing SPH data.
If you want to work with a more stable codebase (although one that is not being actively developed) you should be able to do many analysis and viz tasks using the yt-3.0 branch of the main development repository: https://bitbucket.org/yt_analysis/yt
The 3.0 branch was a big refactoring of the underlying data selection algorithms yt uses to load data off disk. This made it possible to present the same user interface for visualizing datasets from
codes like Gadget, octree AMR codes like Ramses, and patch-based AMR codes like Enzo.
Support for SPH smoothing is available in a separate repository. In this experimental development repository we've refactored the code to use a symbolic units library to handle unit conversions and detect code bugs using dimensional analysis. We've also completely refactored the way frontends are written and fields are set up and detected.
If you want to dive in to the code, I'd suggest starting with the experimental version of yt. This may be a bit more rocky at first - there might be bugs - but will be more rewarding in the end as this is the direction the codebase is going. The work is ongoing in the yt-3.0 branch on Matt Turk's fork of yt: https://bitbucket.org/MatthewTurk/yt
In both cases there is unfortunately not a lot of documentation at
moment. This is something that we're working on right now. Mailing list archives as well as the YTEP listing (http://ytep.readthedocs.org/) might prove to be useful.
As for your questions about yt internals, I've written some basic description below. I would also encourage you to read the yt method paper and to take a look YTEP-0001 and YTEP-0005, which describe the new geometry system.
The basic data structure yt uses to represent an on-disk dataset is the StaticOutput class. The SPH frontend defines a GadgetStaticOutput class as well as a GadgetHDF5StaticOutput subclass to represent HDF5 Gadget datasets. You can create a new StaticOutput instance using
http://yt-project.org/docs/dev/installing.html#activating-your-installation the YT there the the particle the the
'load' function or by directly instantiating an instance of a StaticOutput subclass defined in one of the frontends. You'll need to look at the parameters of the __init__ method to figure out exactly how to load the data. For Gadget, the 'load' convenience function only works with HDF5 datasets, so if you are have data written in Gadget's binary format, you'll need to load your data by calling GadgetStaticOutput directly.
StaticOutput instances have as an attribute an instance of GeometryHandler. This class handles the indexing and selection of data. This is really the heart of yt's hard-core numerics, and is written in a way that is very accessible at a high level. If you want to dive into the algorithms, I think it would help to look over the GeometryHandler class as well as its subclass the ParticleGeometryHandler. That said, yt's interface is more or less agnostic to the underlying algorithm used to index the data.
Do you have specific questions about how to load and visualize your datasets?
Hope that's helpful and not too much of a manifesto :)
-Nathan _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (3)
-
Andrew Philip Weis -
Andrew Weis -
Nathan Goldbaum