Hi, Is there a routine in yt for returning the nearest neighbours to a cell? I was looking at Sam's amr_kdtree and it mentions bricks.... what's a brick? Is a cell a brick? Is a brick a cell? Is that a deep question? If it's horribly complex, I can work on an extracted uniform mesh, I just thought it would be cool not to. Elizabeth
Elizabeth,
Is there a routine in yt for returning the nearest neighbours to a cell? I was looking at Sam's amr_kdtree and it mentions bricks.... what's a brick? Is a cell a brick? Is a brick a cell? Is that a deep question?
Sam's stuff wouldn't be appropriate for what I'm guessing you want, but you could probably use the same kdtree that the halo finder uses. Let me know if you're still interested. -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Hey Stephen, How complex would that be? Currently, I've cut out an extracted region and I'm working around the neighbours of cells to decide if they deserve to stay in what I'm defining as a "cloud". It's basically fine, I just like the idea of working directly with the raw data because it seems the less steps there are between raw and the result, the less numerical errors there are likely to be. That said, including different sized grid cells might make my potential field bumpy so it might not necessarily work! It also seems to me that having a routine to find the neighbours of a grid cell, independent of whether they are on the same grid, would be a useful general analysis tool. So ... how hard are we talking? :) Elizabeth On 2011-09-12, at 9:28 PM, Stephen Skory wrote:
Elizabeth,
Is there a routine in yt for returning the nearest neighbours to a cell? I was looking at Sam's amr_kdtree and it mentions bricks.... what's a brick? Is a cell a brick? Is a brick a cell? Is that a deep question?
Sam's stuff wouldn't be appropriate for what I'm guessing you want, but you could probably use the same kdtree that the halo finder uses. Let me know if you're still interested.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Elizabeth,
So ... how hard are we talking? :)
The thing is, in all honesty the kdtree we're using in yt (in parallelHF and in a few other places) is not very convenient. It's written in fortran, so the array orderings are backwards from the rest of python (which is C based). It's also confusing to use, it has a weird wrapper to access the functions that isn't very easy to understand. The only reason we're using it is in my tests I found this kdtree implementation to be significantly faster than the alternatives out there. Perhaps if I knew a bit more about what you wanted to do I could help you come up with a solution that avoids the kdtree. Are you interested in only the nearest 6 touching cells to a cell? Or do you want to look at the edges and the corners? Farther out? Do you care about periodicity? You mentioned looking at grids of coarser resolution, do you still think you'd like to have neighbor cells returned from a lower resolution grid? -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Hi Stephen, Honestly, I was just hoping there was some magically pre-programmed way to grab the 26 most refined neighbours of a cell, regardless of which grid they were on or level that happened to be. Mapping to an extracted region is no hassle, I was just checking there wasn't a smart, quick way of doing this already. Thanks for the info! Elizabeth Stephen Skory wrote:
Elizabeth,
So ... how hard are we talking? :)
The thing is, in all honesty the kdtree we're using in yt (in parallelHF and in a few other places) is not very convenient. It's written in fortran, so the array orderings are backwards from the rest of python (which is C based). It's also confusing to use, it has a weird wrapper to access the functions that isn't very easy to understand. The only reason we're using it is in my tests I found this kdtree implementation to be significantly faster than the alternatives out there.
Perhaps if I knew a bit more about what you wanted to do I could help you come up with a solution that avoids the kdtree. Are you interested in only the nearest 6 touching cells to a cell? Or do you want to look at the edges and the corners? Farther out? Do you care about periodicity? You mentioned looking at grids of coarser resolution, do you still think you'd like to have neighbor cells returned from a lower resolution grid?
Elizabeth, If you request ghost zones you can do this. In particular, requested unsmoothed ghost zones will ensure no interpolation occurs. You can do this either by yourself, grid by grid with grid_obj.retrieve_ghost_zones(...) or you can use ValidateSpatial in a field definition. It might help if you shared what kind of operation you want to do once you have the 26. If you have the index of a cell, you can just access: grid[field][index_i+1, index_j+1, index_k+1] and do the +/- and 0/1 modifications. -Matt On Tue, Sep 13, 2011 at 5:04 PM, Elizabeth Tasker <taskere@mcmaster.ca> wrote:
Hi Stephen,
Honestly, I was just hoping there was some magically pre-programmed way to grab the 26 most refined neighbours of a cell, regardless of which grid they were on or level that happened to be. Mapping to an extracted region is no hassle, I was just checking there wasn't a smart, quick way of doing this already.
Thanks for the info!
Elizabeth
Stephen Skory wrote:
Elizabeth,
So ... how hard are we talking? :)
The thing is, in all honesty the kdtree we're using in yt (in parallelHF and in a few other places) is not very convenient. It's written in fortran, so the array orderings are backwards from the rest of python (which is C based). It's also confusing to use, it has a weird wrapper to access the functions that isn't very easy to understand. The only reason we're using it is in my tests I found this kdtree implementation to be significantly faster than the alternatives out there.
Perhaps if I knew a bit more about what you wanted to do I could help you come up with a solution that avoids the kdtree. Are you interested in only the nearest 6 touching cells to a cell? Or do you want to look at the edges and the corners? Farther out? Do you care about periodicity? You mentioned looking at grids of coarser resolution, do you still think you'd like to have neighbor cells returned from a lower resolution grid?
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Elizabeth, Preface: Matt's idea of a covering grid is probably easier than what I'm proposing. I just saw his email come in. I think you might be able to use the AMRKDTree for this operation. At some point I wrote a locate_cell function (that may not be in the repo but I'm happy to find it), that you could then use to search for the 26 neighbors by modifying the current cell position by combinations of it's dx and the smallest dx in the dataset. Let me know if you want to try something like this, and I can dig around in the next day or so. Sam On Tue, Sep 13, 2011 at 5:10 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Elizabeth,
If you request ghost zones you can do this. In particular, requested unsmoothed ghost zones will ensure no interpolation occurs. You can do this either by yourself, grid by grid with grid_obj.retrieve_ghost_zones(...) or you can use ValidateSpatial in a field definition.
It might help if you shared what kind of operation you want to do once you have the 26. If you have the index of a cell, you can just access:
grid[field][index_i+1, index_j+1, index_k+1]
and do the +/- and 0/1 modifications.
-Matt
Hi Stephen,
Honestly, I was just hoping there was some magically pre-programmed way to grab the 26 most refined neighbours of a cell, regardless of which grid
On Tue, Sep 13, 2011 at 5:04 PM, Elizabeth Tasker <taskere@mcmaster.ca> wrote: they
were on or level that happened to be. Mapping to an extracted region is no hassle, I was just checking there wasn't a smart, quick way of doing this already.
Thanks for the info!
Elizabeth
Stephen Skory wrote:
Elizabeth,
So ... how hard are we talking? :)
The thing is, in all honesty the kdtree we're using in yt (in parallelHF and in a few other places) is not very convenient. It's written in fortran, so the array orderings are backwards from the rest of python (which is C based). It's also confusing to use, it has a weird wrapper to access the functions that isn't very easy to understand. The only reason we're using it is in my tests I found this kdtree implementation to be significantly faster than the alternatives out there.
Perhaps if I knew a bit more about what you wanted to do I could help you come up with a solution that avoids the kdtree. Are you interested in only the nearest 6 touching cells to a cell? Or do you want to look at the edges and the corners? Farther out? Do you care about periodicity? You mentioned looking at grids of coarser resolution, do you still think you'd like to have neighbor cells returned from a lower resolution grid?
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Matt and Sam, I just want to do a friends-of-friends scheme where I add cells to an ordered list if their properties (density, Potential field etc) satisfy my criteria. Unfortunately, that does mean that I might walk completely off one grid, ghost zones and all, and onto another one, so I think what Matt suggests below only buys me a few extra steps. Sam, your routine probably was what I was thinking of, but my question wasn't supposed to cause extra work! It was more a "does this exist and everyone is using it apart from me?" query. I'll keep with extracted grids and then if I run into issues or this later morphs into a useful analysis routine that people want to use or parallelise, perhaps we'll revisit the question of using trees. Thanks again, Elizabeth Matthew Turk wrote:
Elizabeth,
If you request ghost zones you can do this. In particular, requested unsmoothed ghost zones will ensure no interpolation occurs. You can do this either by yourself, grid by grid with grid_obj.retrieve_ghost_zones(...) or you can use ValidateSpatial in a field definition.
It might help if you shared what kind of operation you want to do once you have the 26. If you have the index of a cell, you can just access:
grid[field][index_i+1, index_j+1, index_k+1]
and do the +/- and 0/1 modifications.
-Matt
On Tue, Sep 13, 2011 at 5:04 PM, Elizabeth Tasker <taskere@mcmaster.ca> wrote:
Hi Stephen,
Honestly, I was just hoping there was some magically pre-programmed way to grab the 26 most refined neighbours of a cell, regardless of which grid they were on or level that happened to be. Mapping to an extracted region is no hassle, I was just checking there wasn't a smart, quick way of doing this already.
Thanks for the info!
Elizabeth
Stephen Skory wrote:
Elizabeth,
So ... how hard are we talking? :)
The thing is, in all honesty the kdtree we're using in yt (in parallelHF and in a few other places) is not very convenient. It's written in fortran, so the array orderings are backwards from the rest of python (which is C based). It's also confusing to use, it has a weird wrapper to access the functions that isn't very easy to understand. The only reason we're using it is in my tests I found this kdtree implementation to be significantly faster than the alternatives out there.
Perhaps if I knew a bit more about what you wanted to do I could help you come up with a solution that avoids the kdtree. Are you interested in only the nearest 6 touching cells to a cell? Or do you want to look at the edges and the corners? Farther out? Do you care about periodicity? You mentioned looking at grids of coarser resolution, do you still think you'd like to have neighbor cells returned from a lower resolution grid?
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Elizabeth, We have an "extract_connected_sets" routine that may work for what you are doing. This is what the clump finder is built on. You can supply it a threshold, and it will conduct FOF on the cells, returning objects (and boundary values) that satisfy it. Unfortunately, right now it only works with threshholding in one dimension. So, what you can do is define a field that returns 1/0 for whether it should be included, and then perform the contouring with extract_connected_sets on your initial data object. -Matt On Tue, Sep 13, 2011 at 5:22 PM, Elizabeth Tasker <taskere@mcmaster.ca> wrote:
Hi Matt and Sam,
I just want to do a friends-of-friends scheme where I add cells to an ordered list if their properties (density, Potential field etc) satisfy my criteria.
Unfortunately, that does mean that I might walk completely off one grid, ghost zones and all, and onto another one, so I think what Matt suggests below only buys me a few extra steps.
Sam, your routine probably was what I was thinking of, but my question wasn't supposed to cause extra work! It was more a "does this exist and everyone is using it apart from me?" query. I'll keep with extracted grids and then if I run into issues or this later morphs into a useful analysis routine that people want to use or parallelise, perhaps we'll revisit the question of using trees.
Thanks again,
Elizabeth
Matthew Turk wrote:
Elizabeth,
If you request ghost zones you can do this. In particular, requested unsmoothed ghost zones will ensure no interpolation occurs. You can do this either by yourself, grid by grid with grid_obj.retrieve_ghost_zones(...) or you can use ValidateSpatial in a field definition.
It might help if you shared what kind of operation you want to do once you have the 26. If you have the index of a cell, you can just access:
grid[field][index_i+1, index_j+1, index_k+1]
and do the +/- and 0/1 modifications.
-Matt
On Tue, Sep 13, 2011 at 5:04 PM, Elizabeth Tasker <taskere@mcmaster.ca> wrote:
Hi Stephen,
Honestly, I was just hoping there was some magically pre-programmed way to grab the 26 most refined neighbours of a cell, regardless of which grid they were on or level that happened to be. Mapping to an extracted region is no hassle, I was just checking there wasn't a smart, quick way of doing this already.
Thanks for the info!
Elizabeth
Stephen Skory wrote:
Elizabeth,
So ... how hard are we talking? :)
The thing is, in all honesty the kdtree we're using in yt (in parallelHF and in a few other places) is not very convenient. It's written in fortran, so the array orderings are backwards from the rest of python (which is C based). It's also confusing to use, it has a weird wrapper to access the functions that isn't very easy to understand. The only reason we're using it is in my tests I found this kdtree implementation to be significantly faster than the alternatives out there.
Perhaps if I knew a bit more about what you wanted to do I could help you come up with a solution that avoids the kdtree. Are you interested in only the nearest 6 touching cells to a cell? Or do you want to look at the edges and the corners? Farther out? Do you care about periodicity? You mentioned looking at grids of coarser resolution, do you still think you'd like to have neighbor cells returned from a lower resolution grid?
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Matt, I actually use that to get my initial sweep of objects and it works great. This next step through it more complex than just a contour, since it's not obvious what my boundary line will be: it depends on the environment of each object which gives saddle points in the potential field. (This is why this program is taking me a while. *One day* it will be complete and work and I will win at life. Yes.) I'm good with covering grids, honest. I was just seeing if I could be smarter easily. Elizabeth Matthew Turk wrote:
Hi Elizabeth,
We have an "extract_connected_sets" routine that may work for what you are doing. This is what the clump finder is built on. You can supply it a threshold, and it will conduct FOF on the cells, returning objects (and boundary values) that satisfy it. Unfortunately, right now it only works with threshholding in one dimension. So, what you can do is define a field that returns 1/0 for whether it should be included, and then perform the contouring with extract_connected_sets on your initial data object.
-Matt
On Tue, Sep 13, 2011 at 5:22 PM, Elizabeth Tasker <taskere@mcmaster.ca> wrote:
Hi Matt and Sam,
I just want to do a friends-of-friends scheme where I add cells to an ordered list if their properties (density, Potential field etc) satisfy my criteria.
Unfortunately, that does mean that I might walk completely off one grid, ghost zones and all, and onto another one, so I think what Matt suggests below only buys me a few extra steps.
Sam, your routine probably was what I was thinking of, but my question wasn't supposed to cause extra work! It was more a "does this exist and everyone is using it apart from me?" query. I'll keep with extracted grids and then if I run into issues or this later morphs into a useful analysis routine that people want to use or parallelise, perhaps we'll revisit the question of using trees.
Thanks again,
Elizabeth
Matthew Turk wrote:
Elizabeth,
If you request ghost zones you can do this. In particular, requested unsmoothed ghost zones will ensure no interpolation occurs. You can do this either by yourself, grid by grid with grid_obj.retrieve_ghost_zones(...) or you can use ValidateSpatial in a field definition.
It might help if you shared what kind of operation you want to do once you have the 26. If you have the index of a cell, you can just access:
grid[field][index_i+1, index_j+1, index_k+1]
and do the +/- and 0/1 modifications.
-Matt
On Tue, Sep 13, 2011 at 5:04 PM, Elizabeth Tasker <taskere@mcmaster.ca> wrote:
Hi Stephen,
Honestly, I was just hoping there was some magically pre-programmed way to grab the 26 most refined neighbours of a cell, regardless of which grid they were on or level that happened to be. Mapping to an extracted region is no hassle, I was just checking there wasn't a smart, quick way of doing this already.
Thanks for the info!
Elizabeth
Stephen Skory wrote:
Elizabeth,
So ... how hard are we talking? :)
The thing is, in all honesty the kdtree we're using in yt (in parallelHF and in a few other places) is not very convenient. It's written in fortran, so the array orderings are backwards from the rest of python (which is C based). It's also confusing to use, it has a weird wrapper to access the functions that isn't very easy to understand. The only reason we're using it is in my tests I found this kdtree implementation to be significantly faster than the alternatives out there.
Perhaps if I knew a bit more about what you wanted to do I could help you come up with a solution that avoids the kdtree. Are you interested in only the nearest 6 touching cells to a cell? Or do you want to look at the edges and the corners? Farther out? Do you care about periodicity? You mentioned looking at grids of coarser resolution, do you still think you'd like to have neighbor cells returned from a lower resolution grid?
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Elizabeth, Since conferences are good times for very short projects, I've written up a go at the nearest neighbors. See the commit message for a bit more info. https://bitbucket.org/yt_analysis/yt/changeset/dfb88152f5e0 There are two functions. One that takes a position and will return the neighbor cells around that position. The other takes the grid object and cell indices and does the same thing. They are AMRKDTree.get_nearest_neighbors_from_position and AMRKDTree.get_nearest_neighbors Check out the pastebin for an example: http://paste.yt-project.org/show/1803/ Let me know if it works/if it is insanely slow/how it could be improved! Best, Sam On Tue, Sep 13, 2011 at 5:35 PM, Elizabeth Tasker <taskere@mcmaster.ca>wrote:
Hi Matt,
I actually use that to get my initial sweep of objects and it works great. This next step through it more complex than just a contour, since it's not obvious what my boundary line will be: it depends on the environment of each object which gives saddle points in the potential field. (This is why this program is taking me a while. *One day* it will be complete and work and I will win at life. Yes.)
I'm good with covering grids, honest. I was just seeing if I could be smarter easily.
Elizabeth
Matthew Turk wrote:
Hi Elizabeth,
We have an "extract_connected_sets" routine that may work for what you are doing. This is what the clump finder is built on. You can supply it a threshold, and it will conduct FOF on the cells, returning objects (and boundary values) that satisfy it. Unfortunately, right now it only works with threshholding in one dimension. So, what you can do is define a field that returns 1/0 for whether it should be included, and then perform the contouring with extract_connected_sets on your initial data object.
-Matt
On Tue, Sep 13, 2011 at 5:22 PM, Elizabeth Tasker <taskere@mcmaster.ca> wrote:
Hi Matt and Sam,
I just want to do a friends-of-friends scheme where I add cells to an ordered list if their properties (density, Potential field etc) satisfy my criteria.
Unfortunately, that does mean that I might walk completely off one grid, ghost zones and all, and onto another one, so I think what Matt suggests below only buys me a few extra steps.
Sam, your routine probably was what I was thinking of, but my question wasn't supposed to cause extra work! It was more a "does this exist and everyone is using it apart from me?" query. I'll keep with extracted grids and then if I run into issues or this later morphs into a useful analysis routine that people want to use or parallelise, perhaps we'll revisit the question of using trees.
Thanks again,
Elizabeth
Matthew Turk wrote:
Elizabeth,
If you request ghost zones you can do this. In particular, requested unsmoothed ghost zones will ensure no interpolation occurs. You can do this either by yourself, grid by grid with grid_obj.retrieve_ghost_zones(**...) or you can use ValidateSpatial in a field definition.
It might help if you shared what kind of operation you want to do once you have the 26. If you have the index of a cell, you can just access:
grid[field][index_i+1, index_j+1, index_k+1]
and do the +/- and 0/1 modifications.
-Matt
On Tue, Sep 13, 2011 at 5:04 PM, Elizabeth Tasker <taskere@mcmaster.ca> wrote:
Hi Stephen,
Honestly, I was just hoping there was some magically pre-programmed way to grab the 26 most refined neighbours of a cell, regardless of which grid they were on or level that happened to be. Mapping to an extracted region is no hassle, I was just checking there wasn't a smart, quick way of doing this already.
Thanks for the info!
Elizabeth
Stephen Skory wrote:
Elizabeth,
> So ... how hard are we talking? :) > > > > The thing is, in all honesty the kdtree we're using in yt (in parallelHF and in a few other places) is not very convenient. It's written in fortran, so the array orderings are backwards from the rest of python (which is C based). It's also confusing to use, it has a weird wrapper to access the functions that isn't very easy to understand. The only reason we're using it is in my tests I found this kdtree implementation to be significantly faster than the alternatives out there.
Perhaps if I knew a bit more about what you wanted to do I could help you come up with a solution that avoids the kdtree. Are you interested in only the nearest 6 touching cells to a cell? Or do you want to look at the edges and the corners? Farther out? Do you care about periodicity? You mentioned looking at grids of coarser resolution, do you still think you'd like to have neighbor cells returned from a lower resolution grid?
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
Wow -- that's fantastic. I'll give it a test run this week. Thanks so much, Sam! Elizabeth Sam Skillman wrote:
Hi Elizabeth,
Since conferences are good times for very short projects, I've written up a go at the nearest neighbors. See the commit message for a bit more info.
https://bitbucket.org/yt_analysis/yt/changeset/dfb88152f5e0
There are two functions. One that takes a position and will return the neighbor cells around that position. The other takes the grid object and cell indices and does the same thing.
They are AMRKDTree.get_nearest_neighbors_from_position and AMRKDTree.get_nearest_neighbors
Check out the pastebin for an example: http://paste.yt-project.org/show/1803/
Let me know if it works/if it is insanely slow/how it could be improved!
Best, Sam
On Tue, Sep 13, 2011 at 5:35 PM, Elizabeth Tasker <taskere@mcmaster.ca <mailto:taskere@mcmaster.ca>> wrote:
Hi Matt,
I actually use that to get my initial sweep of objects and it works great. This next step through it more complex than just a contour, since it's not obvious what my boundary line will be: it depends on the environment of each object which gives saddle points in the potential field. (This is why this program is taking me a while. *One day* it will be complete and work and I will win at life. Yes.)
I'm good with covering grids, honest. I was just seeing if I could be smarter easily.
Elizabeth
Matthew Turk wrote:
Hi Elizabeth,
We have an "extract_connected_sets" routine that may work for what you are doing. This is what the clump finder is built on. You can supply it a threshold, and it will conduct FOF on the cells, returning objects (and boundary values) that satisfy it. Unfortunately, right now it only works with threshholding in one dimension. So, what you can do is define a field that returns 1/0 for whether it should be included, and then perform the contouring with extract_connected_sets on your initial data object.
-Matt
On Tue, Sep 13, 2011 at 5:22 PM, Elizabeth Tasker <taskere@mcmaster.ca <mailto:taskere@mcmaster.ca>> wrote:
Hi Matt and Sam,
I just want to do a friends-of-friends scheme where I add cells to an ordered list if their properties (density, Potential field etc) satisfy my criteria.
Unfortunately, that does mean that I might walk completely off one grid, ghost zones and all, and onto another one, so I think what Matt suggests below only buys me a few extra steps.
Sam, your routine probably was what I was thinking of, but my question wasn't supposed to cause extra work! It was more a "does this exist and everyone is using it apart from me?" query. I'll keep with extracted grids and then if I run into issues or this later morphs into a useful analysis routine that people want to use or parallelise, perhaps we'll revisit the question of using trees.
Thanks again,
Elizabeth
Matthew Turk wrote:
Elizabeth,
If you request ghost zones you can do this. In particular, requested unsmoothed ghost zones will ensure no interpolation occurs. You can do this either by yourself, grid by grid with grid_obj.retrieve_ghost_zones(...) or you can use ValidateSpatial in a field definition.
It might help if you shared what kind of operation you want to do once you have the 26. If you have the index of a cell, you can just access:
grid[field][index_i+1, index_j+1, index_k+1]
and do the +/- and 0/1 modifications.
-Matt
On Tue, Sep 13, 2011 at 5:04 PM, Elizabeth Tasker <taskere@mcmaster.ca <mailto:taskere@mcmaster.ca>> wrote:
Hi Stephen,
Honestly, I was just hoping there was some magically pre-programmed way to grab the 26 most refined neighbours of a cell, regardless of which grid they were on or level that happened to be. Mapping to an extracted region is no hassle, I was just checking there wasn't a smart, quick way of doing this already.
Thanks for the info!
Elizabeth
Stephen Skory wrote:
Elizabeth,
So ... how hard are we talking? :)
The thing is, in all honesty the kdtree we're using in yt (in parallelHF and in a few other places) is not very convenient. It's written in fortran, so the array orderings are backwards from the rest of python (which is C based). It's also confusing to use, it has a weird wrapper to access the functions that isn't very easy to understand. The only reason we're using it is in my tests I found this kdtree implementation to be significantly faster than the alternatives out there.
Perhaps if I knew a bit more about what you wanted to do I could help you come up with a solution that avoids the kdtree. Are you interested in only the nearest 6 touching cells to a cell? Or do you want to look at the edges and the corners? Farther out? Do you care about periodicity? You mentioned looking at grids of coarser resolution, do you still think you'd like to have neighbor cells returned from a lower resolution grid?
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
------------------------------------------------------------------------
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Elizabeth, Just a quick update. First, I documented the source a bit so that should help. This is just in the help(AMRKDTree.locate_neighbors). I also modified the neighbor finding so that it should be periodic. Before if you were doing a while loop or something there was a chance you could end up stuck on a wall or something. Anyways, I think this works. I did some checks on what happens if you always walk in a given direction, and everything seems okay. Additionally, I did a fun experiment of an AMR random walk (though in this case I force it to move at least some direction) : http://paste.yt-project.org/show/1804/ This is kinda fun, and neat that it seems to get "stuck" in the dense areas since the density of AMR cells is also higher. Anyways, let us know if this works and all that. Cheers, Sam On Wed, Sep 14, 2011 at 5:30 PM, Elizabeth Tasker <taskere@mcmaster.ca>wrote:
Wow -- that's fantastic. I'll give it a test run this week.
Thanks so much, Sam!
Elizabeth
Sam Skillman wrote:
Hi Elizabeth,
Since conferences are good times for very short projects, I've written up a go at the nearest neighbors. See the commit message for a bit more info.
There are two functions. One that takes a position and will return the neighbor cells around that position. The other takes the grid object and cell indices and does the same thing.
They are AMRKDTree.get_nearest_**neighbors_from_position and AMRKDTree.get_nearest_**neighbors
Check out the pastebin for an example: http://paste.yt-project.org/**show/1803/<http://paste.yt-project.org/show/1803/>
Let me know if it works/if it is insanely slow/how it could be improved!
Best, Sam
On Tue, Sep 13, 2011 at 5:35 PM, Elizabeth Tasker <taskere@mcmaster.ca<mailto: taskere@mcmaster.ca>> wrote:
Hi Matt,
I actually use that to get my initial sweep of objects and it works great. This next step through it more complex than just a contour, since it's not obvious what my boundary line will be: it depends on the environment of each object which gives saddle points in the potential field. (This is why this program is taking me a while. *One day* it will be complete and work and I will win at life. Yes.)
I'm good with covering grids, honest. I was just seeing if I could be smarter easily.
Elizabeth
Matthew Turk wrote:
Hi Elizabeth,
We have an "extract_connected_sets" routine that may work for what you are doing. This is what the clump finder is built on. You can supply it a threshold, and it will conduct FOF on the cells, returning objects (and boundary values) that satisfy it. Unfortunately, right now it only works with threshholding in one dimension. So, what you can do is define a field that returns 1/0 for whether it should be included, and then perform the contouring with extract_connected_sets on your initial data object.
-Matt
On Tue, Sep 13, 2011 at 5:22 PM, Elizabeth Tasker <taskere@mcmaster.ca <mailto:taskere@mcmaster.ca>> wrote:
Hi Matt and Sam,
I just want to do a friends-of-friends scheme where I add cells to an ordered list if their properties (density, Potential field etc) satisfy my criteria.
Unfortunately, that does mean that I might walk completely off one grid, ghost zones and all, and onto another one, so I think what Matt suggests below only buys me a few extra steps.
Sam, your routine probably was what I was thinking of, but my question wasn't supposed to cause extra work! It was more a "does this exist and everyone is using it apart from me?" query. I'll keep with extracted grids and then if I run into issues or this later morphs into a useful analysis routine that people want to use or parallelise, perhaps we'll revisit the question of using trees.
Thanks again,
Elizabeth
Matthew Turk wrote:
Elizabeth,
If you request ghost zones you can do this. In particular, requested unsmoothed ghost zones will ensure no interpolation occurs. You can do this either by yourself, grid by grid with grid_obj.retrieve_ghost_zones(**...) or you can use ValidateSpatial in a field definition.
It might help if you shared what kind of operation you want to do once you have the 26. If you have the index of a cell, you can just access:
grid[field][index_i+1, index_j+1, index_k+1]
and do the +/- and 0/1 modifications.
-Matt
On Tue, Sep 13, 2011 at 5:04 PM, Elizabeth Tasker <taskere@mcmaster.ca <mailto:taskere@mcmaster.ca>>
wrote:
Hi Stephen,
Honestly, I was just hoping there was some magically pre-programmed way to grab the 26 most refined neighbours of a cell, regardless of which grid they were on or level that happened to be. Mapping to an extracted region is no hassle, I was just checking there wasn't a smart, quick way of doing this already.
Thanks for the info!
Elizabeth
Stephen Skory wrote:
Elizabeth,
So ... how hard are we talking? :)
The thing is, in all honesty the kdtree we're using in yt (in parallelHF and in a few other places) is not very convenient. It's written in fortran, so the array orderings are backwards from the rest of python (which is C based). It's also confusing to use, it has a weird wrapper to access the functions that isn't very easy to understand. The only reason we're using it is in my tests I found this kdtree implementation to be significantly faster than the alternatives out there.
Perhaps if I knew a bit more about what you wanted to do I could help you come up with a solution that avoids the kdtree. Are you interested in only the nearest 6 touching cells to a cell? Or do you want to look at the edges and the corners? Farther out? Do you care about periodicity? You mentioned looking at grids of coarser resolution, do you still think you'd like to have neighbor cells returned from a lower resolution grid?
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.**spacepope.org<yt-users@lists.spacepope.org>
http://lists.spacepope.org/**listinfo.cgi/yt-users-** spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.**spacepope.org<yt-users@lists.spacepope.org>
http://lists.spacepope.org/**listinfo.cgi/yt-users-** spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.**spacepope.org<yt-users@lists.spacepope.org>
http://lists.spacepope.org/**listinfo.cgi/yt-users-** spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.** spacepope.org <yt-users@lists.spacepope.org>>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org <mailto:yt-users@lists.**spacepope.org<yt-users@lists.spacepope.org>
------------------------------**------------------------------** ------------
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
participants (4)
-
Elizabeth Tasker -
Matthew Turk -
Sam Skillman -
Stephen Skory