Hi all, Can someone point me to an example (docs, notebooks, emails, whatever) of defining a custom particle filter for yt-3.0? I'm trying to do this and I know I've seen discussion of this before, but I can't find where. Thanks, Britton
Hey Britton,
On Mon, Dec 2, 2013 at 10:15 AM, Britton Smith
Hi all,
Can someone point me to an example (docs, notebooks, emails, whatever) of defining a custom particle filter for yt-3.0? I'm trying to do this and I know I've seen discussion of this before, but I can't find where.
Yup. Here's an example: http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/534fd19b933e4a3... Looks like the nbviewer is down though. Sorry I haven't done much documenting of this. Particle filters and particle unions are two really cool things that are currently only in example form. Here's another example of how to define one: https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66... and you have to add this by name to each pf you open up: https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66... Note that you have to specify *all* the required filtering fields in the constructor for the function. -Matt
Thanks, Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Matt,
Thanks, that's very helpful.
I have an additional question. When I do:
ad = pf.h.all_data()
ad['particle_mass'] or
ad['all', 'particle_mass']
I get numbers that are ~1e44, which sounds about right if it's in grams.
However, when I do:
ad['DarkMatter', 'particle_mass']
I get numbers that are ~1e-5, which I am guessing is the unconverted
particle_mass straight out of the file. How can get the filtered particles
in proper units as well?
Thanks again,
Britton
On Mon, Dec 2, 2013 at 3:21 PM, Matthew Turk
Hey Britton,
On Mon, Dec 2, 2013 at 10:15 AM, Britton Smith
wrote: Hi all,
Can someone point me to an example (docs, notebooks, emails, whatever) of defining a custom particle filter for yt-3.0? I'm trying to do this and I know I've seen discussion of this before, but I can't find where.
Yup. Here's an example:
http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/534fd19b933e4a3...
Looks like the nbviewer is down though.
Sorry I haven't done much documenting of this. Particle filters and particle unions are two really cool things that are currently only in example form. Here's another example of how to define one:
https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66...
and you have to add this by name to each pf you open up:
https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66...
Note that you have to specify *all* the required filtering fields in the constructor for the function.
-Matt
Thanks, Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
On Mon, Dec 2, 2013 at 10:29 AM, Britton Smith
Hi Matt,
Thanks, that's very helpful.
I have an additional question. When I do: ad = pf.h.all_data() ad['particle_mass'] or ad['all', 'particle_mass']
I get numbers that are ~1e44, which sounds about right if it's in grams. However, when I do: ad['DarkMatter', 'particle_mass']
I get numbers that are ~1e-5, which I am guessing is the unconverted particle_mass straight out of the file. How can get the filtered particles in proper units as well?
Hm, that looks like a bug to me. What's your filter look like, so I can reproduce this here?
Thanks again, Britton
On Mon, Dec 2, 2013 at 3:21 PM, Matthew Turk
wrote: Hey Britton,
On Mon, Dec 2, 2013 at 10:15 AM, Britton Smith
wrote: Hi all,
Can someone point me to an example (docs, notebooks, emails, whatever) of defining a custom particle filter for yt-3.0? I'm trying to do this and I know I've seen discussion of this before, but I can't find where.
Yup. Here's an example:
http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/534fd19b933e4a3...
Looks like the nbviewer is down though.
Sorry I haven't done much documenting of this. Particle filters and particle unions are two really cool things that are currently only in example form. Here's another example of how to define one:
https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66...
and you have to add this by name to each pf you open up:
https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66...
Note that you have to specify *all* the required filtering fields in the constructor for the function.
-Matt
Thanks, Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
I am getting this with just "DarkMatter" and "CenOstriker", which are not
exactly filters (or maybe they are), but the particle types as they are
stored on disk by Enzo-3.0.
On Mon, Dec 2, 2013 at 3:30 PM, Matthew Turk
Hi Matt,
Thanks, that's very helpful.
I have an additional question. When I do: ad = pf.h.all_data() ad['particle_mass'] or ad['all', 'particle_mass']
I get numbers that are ~1e44, which sounds about right if it's in grams. However, when I do: ad['DarkMatter', 'particle_mass']
I get numbers that are ~1e-5, which I am guessing is the unconverted particle_mass straight out of the file. How can get the filtered
On Mon, Dec 2, 2013 at 10:29 AM, Britton Smith
wrote: particles in proper units as well?
Hm, that looks like a bug to me. What's your filter look like, so I can reproduce this here?
Thanks again, Britton
On Mon, Dec 2, 2013 at 3:21 PM, Matthew Turk
wrote: Hey Britton,
On Mon, Dec 2, 2013 at 10:15 AM, Britton Smith
wrote: Hi all,
Can someone point me to an example (docs, notebooks, emails, whatever) of defining a custom particle filter for yt-3.0? I'm trying to do this
and
I know I've seen discussion of this before, but I can't find where.
Yup. Here's an example:
http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/534fd19b933e4a3...
Looks like the nbviewer is down though.
Sorry I haven't done much documenting of this. Particle filters and particle unions are two really cool things that are currently only in example form. Here's another example of how to define one:
https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66...
and you have to add this by name to each pf you open up:
https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66...
Note that you have to specify *all* the required filtering fields in the constructor for the function.
-Matt
Thanks, Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
That looks like a failure to convert correctly. What goes on during
instantiation is that for each particle type, the function
_setup_particle_type is called on EnzoStaticOutput. Then the
_setup_particle_fields from enzo/fields.py gets called, which should
set the "particle_mass" function to convert based on Density * cm**3.
(Note that we now -- in _read_particle_fields -- multiply by the
grid.dds.prod, so it doesn't need that anywhere else anymore.)
Can you check what pf._get_field_info("DarkMatter", "particle_mass")
has for its _convert_function ? It should be a function called
_convertParticleMass.
I have to go for a few hours, but I can look at an E3 dataset when I get back.
-Matt
On Mon, Dec 2, 2013 at 10:33 AM, Britton Smith
I am getting this with just "DarkMatter" and "CenOstriker", which are not exactly filters (or maybe they are), but the particle types as they are stored on disk by Enzo-3.0.
On Mon, Dec 2, 2013 at 3:30 PM, Matthew Turk
wrote: On Mon, Dec 2, 2013 at 10:29 AM, Britton Smith
wrote: Hi Matt,
Thanks, that's very helpful.
I have an additional question. When I do: ad = pf.h.all_data() ad['particle_mass'] or ad['all', 'particle_mass']
I get numbers that are ~1e44, which sounds about right if it's in grams. However, when I do: ad['DarkMatter', 'particle_mass']
I get numbers that are ~1e-5, which I am guessing is the unconverted particle_mass straight out of the file. How can get the filtered particles in proper units as well?
Hm, that looks like a bug to me. What's your filter look like, so I can reproduce this here?
Thanks again, Britton
On Mon, Dec 2, 2013 at 3:21 PM, Matthew Turk
wrote: Hey Britton,
On Mon, Dec 2, 2013 at 10:15 AM, Britton Smith
wrote: Hi all,
Can someone point me to an example (docs, notebooks, emails, whatever) of defining a custom particle filter for yt-3.0? I'm trying to do this and I know I've seen discussion of this before, but I can't find where.
Yup. Here's an example:
http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/534fd19b933e4a3...
Looks like the nbviewer is down though.
Sorry I haven't done much documenting of this. Particle filters and particle unions are two really cool things that are currently only in example form. Here's another example of how to define one:
https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66...
and you have to add this by name to each pf you open up:
https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66...
Note that you have to specify *all* the required filtering fields in the constructor for the function.
-Matt
Thanks, Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Matt,
Moving this to the correct thread (with additions):
The change you suggested solved the problem of
ad['DarkMatter', 'particle_mass']
and
ad['CenOstriker', 'particle_mass']
However, when I do:
ad['all', 'particle_mass']
I get the following:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/Users/britton/Documents/work/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py",
line 189, in __getitem__
self.get_data(f)
File
"/Users/britton/Documents/work/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py",
line 535, in get_data
particles, self, self._current_chunk)
File
"/Users/britton/Documents/work/yt/yt-x86_64/src/yt-hg/yt/geometry/geometry_handler.py",
line 507, in _read_particle_fields
fields_to_read)
File
"/Users/britton/Documents/work/yt/yt-x86_64/src/yt-hg/yt/utilities/io_handler.py",
line 176, in _read_particle_selection
rv[field_f][my_ind:my_ind + vals.shape[0],...] = vals
ValueError: could not broadcast input array from shape (4851) into shape
(4361)
Additionally, if I do:
ad['io', 'particle_mass']
I get this (which I got before this change as well):
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/Users/britton/Documents/work/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py",
line 183, in __getitem__
f = self._determine_fields(key)[0]
File
"/Users/britton/Documents/work/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py",
line 406, in _determine_fields
raise YTFieldTypeNotFound(ftype)
yt.utilities.exceptions.YTFieldTypeNotFound: Could not find field 'io'.
The 'io' issue seems to come from line 848 in enzo/data_structures.py:
# If this is the case, then we know we should have a DarkMatter
# particle type, and we don't need the "io" type.
self.particle_types = ["DarkMatter"]
Would it be better to switch that to an append and keep the "io" type
around for generality? I don't know if that would actually be a good idea
since in reality it is just the dark matter for Enzo-3.0 data.
Britton
On Mon, Dec 2, 2013 at 3:41 PM, Matthew Turk
That looks like a failure to convert correctly. What goes on during instantiation is that for each particle type, the function _setup_particle_type is called on EnzoStaticOutput. Then the _setup_particle_fields from enzo/fields.py gets called, which should set the "particle_mass" function to convert based on Density * cm**3. (Note that we now -- in _read_particle_fields -- multiply by the grid.dds.prod, so it doesn't need that anywhere else anymore.)
Can you check what pf._get_field_info("DarkMatter", "particle_mass") has for its _convert_function ? It should be a function called _convertParticleMass.
I have to go for a few hours, but I can look at an E3 dataset when I get back.
-Matt
I am getting this with just "DarkMatter" and "CenOstriker", which are not exactly filters (or maybe they are), but the particle types as they are stored on disk by Enzo-3.0.
On Mon, Dec 2, 2013 at 3:30 PM, Matthew Turk
wrote: On Mon, Dec 2, 2013 at 10:29 AM, Britton Smith
wrote: Hi Matt,
Thanks, that's very helpful.
I have an additional question. When I do: ad = pf.h.all_data() ad['particle_mass'] or ad['all', 'particle_mass']
I get numbers that are ~1e44, which sounds about right if it's in
grams.
However, when I do: ad['DarkMatter', 'particle_mass']
I get numbers that are ~1e-5, which I am guessing is the unconverted particle_mass straight out of the file. How can get the filtered particles in proper units as well?
Hm, that looks like a bug to me. What's your filter look like, so I can reproduce this here?
Thanks again, Britton
On Mon, Dec 2, 2013 at 3:21 PM, Matthew Turk
wrote: Hey Britton,
On Mon, Dec 2, 2013 at 10:15 AM, Britton Smith <
brittonsmith@gmail.com>
wrote:
Hi all,
Can someone point me to an example (docs, notebooks, emails, whatever) of defining a custom particle filter for yt-3.0? I'm trying to do
On Mon, Dec 2, 2013 at 10:33 AM, Britton Smith
wrote: this and I know I've seen discussion of this before, but I can't find where.
Yup. Here's an example:
http://nbviewer.ipython.org/url/hub.yt-project.org/notebooks/534fd19b933e4a3...
Looks like the nbviewer is down though.
Sorry I haven't done much documenting of this. Particle filters and particle unions are two really cool things that are currently only in example form. Here's another example of how to define one:
https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66...
and you have to add this by name to each pf you open up:
https://bitbucket.org/mornkr/agora-analysis-script/src/1200830cd1e8d7dc60f66...
Note that you have to specify *all* the required filtering fields in the constructor for the function.
-Matt
Thanks, Britton
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (2)
-
Britton Smith
-
Matthew Turk