Mailman 3 Should support for octree data in the AMRKDTree be a blocker for 3.3? - yt-dev

Should support for octree data in the AMRKDTree be a blocker for 3.3?

Nathan Goldbaum

3 May 2016 3 May '16

1:26 a.m.

Hi all, I'm moving some discussion from the comments in issue #1212 over to the mailing list: https://bitbucket.org/yt_analysis/yt/issues/1212 I wanted to check with the list since Cameron objected a bit in the issue discussion. The issue Cameron identifies is pretty serious, but fully fixing it will require pretty substantial work - adding support for octree data to the AMRKDTree. I've gone ahead and marked it for version 3.4 - i.e. that it shouldn't be a blocker for the 3.3 release. I've also created issue 1215: https://bitbucket.org/yt_analysis/yt/issues/1215 which will be fixed when yt produces a useful error message rather than seg faulting when trying to construct an AMRKDTree for unsupported data. I think that requiring someone to add support for octree data to the AMRKDTree before we release 3.3 is effectively delaying the 3.3 release arbitrarily far into the future, since no one is currently willing to work on that project. That would be unfortunate, because there's lots of cool stuff people have worked on that deserves a stable release. -Nathan

Attachments:

attachment.html (text/html — 1.4 KB)

Show replies by thread

Kacper Kowalik

3 May 3 May

2:18 a.m.

On 05/02/2016 06:26 PM, Nathan Goldbaum wrote:

...

Hi all,

I'm moving some discussion from the comments in issue #1212 over to the mailing list:

https://bitbucket.org/yt_analysis/yt/issues/1212

I wanted to check with the list since Cameron objected a bit in the issue discussion.

The issue Cameron identifies is pretty serious, but fully fixing it will require pretty substantial work - adding support for octree data to the AMRKDTree.

I've gone ahead and marked it for version 3.4 - i.e. that it shouldn't be a blocker for the 3.3 release. I've also created issue 1215:

https://bitbucket.org/yt_analysis/yt/issues/1215

which will be fixed when yt produces a useful error message rather than seg faulting when trying to construct an AMRKDTree for unsupported data.

I think that requiring someone to add support for octree data to the AMRKDTree before we release 3.3 is effectively delaying the 3.3 release arbitrarily far into the future, since no one is currently willing to work on that project. That would be unfortunate, because there's lots of cool stuff people have worked on that deserves a stable release.

-Nathan

Hi, the rule of thumb I've always used within other open source project is that bugs that are not regressions shouldn't block release/stabilization of new versions. If we adopted such a rule, it would be a matter of answering the question whether #1212 applies to 3.2 as well, or was it introduced in 3.3? Cheers, Kacper

Nathan Goldbaum

2:20 a.m.

On Mon, May 2, 2016 at 7:18 PM, Kacper Kowalik wrote:

...

On 05/02/2016 06:26 PM, Nathan Goldbaum wrote:

...
Hi all,

I'm moving some discussion from the comments in issue #1212 over to the mailing list:

https://bitbucket.org/yt_analysis/yt/issues/1212

I wanted to check with the list since Cameron objected a bit in the issue discussion.

The issue Cameron identifies is pretty serious, but fully fixing it will require pretty substantial work - adding support for octree data to the AMRKDTree.

I've gone ahead and marked it for version 3.4 - i.e. that it shouldn't be a blocker for the 3.3 release. I've also created issue 1215:

https://bitbucket.org/yt_analysis/yt/issues/1215

which will be fixed when yt produces a useful error message rather than seg faulting when trying to construct an AMRKDTree for unsupported data.

I think that requiring someone to add support for octree data to the AMRKDTree before we release 3.3 is effectively delaying the 3.3 release arbitrarily far into the future, since no one is currently willing to work on that project. That would be unfortunate, because there's lots of cool stuff people have worked on that deserves a stable release.

-Nathan

Hi, the rule of thumb I've always used within other open source project is that bugs that are not regressions shouldn't block release/stabilization of new versions.

If we adopted such a rule, it would be a matter of answering the question whether #1212 applies to 3.2 as well, or was it introduced in 3.3?

I think it's been an issue since we began to support octree data in yt 3.0.

...

Cheers, Kacper _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Cameron Hummels

2:21 a.m.

I wasn't entirely objecting to punting it, but I think it warrants discussion here. Until I tried to build some Off axis projections and VRs for SPH data, I didn't realize that they weren't supported. In fact, which frontends are actually supported for doing VRs and OffAxisProjections? If Octtree-based code isn't, then that knocks out ART, ARTIO, and Ramses, and without SPH, it knocks out Gadget, Tipsy, Gizmo, and Gasoline, which leaves Enzo, Flash, and Athena? Others? I honestly don't have any idea how much work it would take to construct a KDTree from octree data. I figured this was in the realm of CS and would probably have been solved elsewhere, but maybe not. Anyway, I'm not trying to be too obstructionist here, just trying to have a discussion, since there were at least 4 other issues filed regarding this in the past, so it seems relevant for a lot of users. Cameron On Mon, May 2, 2016 at 4:26 PM, Nathan Goldbaum wrote:

...

Hi all,

I'm moving some discussion from the comments in issue #1212 over to the mailing list:

https://bitbucket.org/yt_analysis/yt/issues/1212

I wanted to check with the list since Cameron objected a bit in the issue discussion.

The issue Cameron identifies is pretty serious, but fully fixing it will require pretty substantial work - adding support for octree data to the AMRKDTree.

I've gone ahead and marked it for version 3.4 - i.e. that it shouldn't be a blocker for the 3.3 release. I've also created issue 1215:

https://bitbucket.org/yt_analysis/yt/issues/1215

which will be fixed when yt produces a useful error message rather than seg faulting when trying to construct an AMRKDTree for unsupported data.

I think that requiring someone to add support for octree data to the AMRKDTree before we release 3.3 is effectively delaying the 3.3 release arbitrarily far into the future, since no one is currently willing to work on that project. That would be unfortunate, because there's lots of cool stuff people have worked on that deserves a stable release.

-Nathan

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

Matthew Turk

2:33 a.m.

Hi Cameron, On Mon, May 2, 2016 at 7:21 PM, Cameron Hummels wrote:

...

I wasn't entirely objecting to punting it, but I think it warrants discussion here. Until I tried to build some Off axis projections and VRs for SPH data, I didn't realize that they weren't supported. In fact, which frontends are actually supported for doing VRs and OffAxisProjections? If Octtree-based code isn't, then that knocks out ART, ARTIO, and Ramses, and without SPH, it knocks out Gadget, Tipsy, Gizmo, and Gasoline, which leaves Enzo, Flash, and Athena? Others?

I honestly don't have any idea how much work it would take to construct a KDTree from octree data. I figured this was in the realm of CS and would probably have been solved elsewhere, but maybe not.

Yes, it absolutely has. When you asked me about this over private message or public message on Slack a while back (I forget which) I sent back information about how to do this. I think it would be relatively straightforward to implement. You need to insert a couple "fake" levels in the octree so that it is a BSP rather than the way it functions now, but these cutting planes can be inserted at the existing cell refinement divisions. It would also be straightforward to implement a ray traversal that natively worked with Octrees, which would only require mocking PartitionedGrid objects and sorting the octs in the right order for the traversal. (This would result in an identical operation, I think.) My understanding is that when over_refine_factor > 1, it works for particle data sets. Desika had success with this. I may no longer be right. Even without any new development, I have not heard of anyone attempting to debug why and where the segfault occurs. My suspicion is that it's a fencepost error that shows up because of the binary division in the octree. That would be a good first place to look.

...

Anyway, I'm not trying to be too obstructionist here, just trying to have a discussion, since there were at least 4 other issues filed regarding this in the past, so it seems relevant for a lot of users.

I think this would be a great use of resources, but I don't think it should block 3.3. I do not know when I would be able to devote the time to it; if someone else is willing to volunteer that time, I think it would be great. Would you be willing to put the time in to ensure that it could happen? If so, I think making it a blocker on 3.3 and setting a deadline for it would be a good compromise; if it isn't done by that deadline, punt until the next release. -Matt

...

Cameron

On Mon, May 2, 2016 at 4:26 PM, Nathan Goldbaum wrote:

...
Hi all,

I'm moving some discussion from the comments in issue #1212 over to the mailing list:

https://bitbucket.org/yt_analysis/yt/issues/1212

I wanted to check with the list since Cameron objected a bit in the issue discussion.

The issue Cameron identifies is pretty serious, but fully fixing it will require pretty substantial work - adding support for octree data to the AMRKDTree.

I've gone ahead and marked it for version 3.4 - i.e. that it shouldn't be a blocker for the 3.3 release. I've also created issue 1215:

https://bitbucket.org/yt_analysis/yt/issues/1215

which will be fixed when yt produces a useful error message rather than seg faulting when trying to construct an AMRKDTree for unsupported data.

I think that requiring someone to add support for octree data to the AMRKDTree before we release 3.3 is effectively delaying the 3.3 release arbitrarily far into the future, since no one is currently willing to work on that project. That would be unfortunate, because there's lots of cool stuff people have worked on that deserves a stable release.

-Nathan

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Cameron Hummels

4 May 4 May

12:51 a.m.

I don't believe I can take this project on right now to make the VR interface work with particle-based and/or octree-based frontends. I don't usually use these frontends, so I can't really justify it to myself to put a bunch of time in on it when there is little research benefit for me, and I've already got my plate full. This may change in the future. Perhaps one of the representatives of one of the affected frontends might be interested in taking this on to assure VR will work with their data? Cameron On Mon, May 2, 2016 at 5:33 PM, Matthew Turk wrote:

...

Hi Cameron,

On Mon, May 2, 2016 at 7:21 PM, Cameron Hummels wrote:

...
I wasn't entirely objecting to punting it, but I think it warrants discussion here. Until I tried to build some Off axis projections and VRs for SPH data, I didn't realize that they weren't supported. In fact, which frontends are actually supported for doing VRs and OffAxisProjections? If Octtree-based code isn't, then that knocks out ART, ARTIO, and Ramses, and without SPH, it knocks out Gadget, Tipsy, Gizmo, and Gasoline, which leaves Enzo, Flash, and Athena? Others?

I honestly don't have any idea how much work it would take to construct a KDTree from octree data. I figured this was in the realm of CS and would probably have been solved elsewhere, but maybe not.

Yes, it absolutely has. When you asked me about this over private message or public message on Slack a while back (I forget which) I sent back information about how to do this. I think it would be relatively straightforward to implement. You need to insert a couple "fake" levels in the octree so that it is a BSP rather than the way it functions now, but these cutting planes can be inserted at the existing cell refinement divisions. It would also be straightforward to implement a ray traversal that natively worked with Octrees, which would only require mocking PartitionedGrid objects and sorting the octs in the right order for the traversal. (This would result in an identical operation, I think.)

My understanding is that when over_refine_factor > 1, it works for particle data sets. Desika had success with this. I may no longer be right.

Even without any new development, I have not heard of anyone attempting to debug why and where the segfault occurs. My suspicion is that it's a fencepost error that shows up because of the binary division in the octree. That would be a good first place to look.

...
Anyway, I'm not trying to be too obstructionist here, just trying to have a discussion, since there were at least 4 other issues filed regarding this in the past, so it seems relevant for a lot of users.

I think this would be a great use of resources, but I don't think it should block 3.3. I do not know when I would be able to devote the time to it; if someone else is willing to volunteer that time, I think it would be great. Would you be willing to put the time in to ensure that it could happen? If so, I think making it a blocker on 3.3 and setting a deadline for it would be a good compromise; if it isn't done by that deadline, punt until the next release.

-Matt

...
Cameron

On Mon, May 2, 2016 at 4:26 PM, Nathan Goldbaum wrote:

...
Hi all,

I'm moving some discussion from the comments in issue #1212 over to the mailing list:

https://bitbucket.org/yt_analysis/yt/issues/1212

I wanted to check with the list since Cameron objected a bit in the issue discussion.

The issue Cameron identifies is pretty serious, but fully fixing it will require pretty substantial work - adding support for octree data to the AMRKDTree.

I've gone ahead and marked it for version 3.4 - i.e. that it shouldn't be a blocker for the 3.3 release. I've also created issue 1215:

https://bitbucket.org/yt_analysis/yt/issues/1215

which will be fixed when yt produces a useful error message rather than seg faulting when trying to construct an AMRKDTree for unsupported data.

I think that requiring someone to add support for octree data to the AMRKDTree before we release 3.3 is effectively delaying the 3.3 release arbitrarily far into the future, since no one is currently willing to work on that project. That would be unfortunate, because there's lots of cool stuff people have worked on that deserves a stable release.

-Nathan

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

Nathan Goldbaum

12:56 a.m.

On Tue, May 3, 2016 at 5:51 PM, Cameron Hummels wrote:

...

I don't believe I can take this project on right now to make the VR interface work with particle-based and/or octree-based frontends. I don't usually use these frontends, so I can't really justify it to myself to put a bunch of time in on it when there is little research benefit for me, and I've already got my plate full. This may change in the future. Perhaps one of the representatives of one of the affected frontends might be interested in taking this on to assure VR will work with their data?

Well, we've been waiting for a couple years now for someone to volunteer to do that. I'll happily continue waiting, but I still don't think that this should block 3.3

...

Cameron

On Mon, May 2, 2016 at 5:33 PM, Matthew Turk wrote:

...
Hi Cameron,

On Mon, May 2, 2016 at 7:21 PM, Cameron Hummels wrote:

...
I wasn't entirely objecting to punting it, but I think it warrants discussion here. Until I tried to build some Off axis projections and VRs for SPH data, I didn't realize that they weren't supported. In fact, which frontends are actually supported for doing VRs and OffAxisProjections? If Octtree-based code isn't, then that knocks out ART, ARTIO, and Ramses, and without SPH, it knocks out Gadget, Tipsy, Gizmo, and Gasoline, which leaves Enzo, Flash, and Athena? Others?

I honestly don't have any idea how much work it would take to construct a KDTree from octree data. I figured this was in the realm of CS and would probably have been solved elsewhere, but maybe not.

Yes, it absolutely has. When you asked me about this over private message or public message on Slack a while back (I forget which) I sent back information about how to do this. I think it would be relatively straightforward to implement. You need to insert a couple "fake" levels in the octree so that it is a BSP rather than the way it functions now, but these cutting planes can be inserted at the existing cell refinement divisions. It would also be straightforward to implement a ray traversal that natively worked with Octrees, which would only require mocking PartitionedGrid objects and sorting the octs in the right order for the traversal. (This would result in an identical operation, I think.)

My understanding is that when over_refine_factor > 1, it works for particle data sets. Desika had success with this. I may no longer be right.

Even without any new development, I have not heard of anyone attempting to debug why and where the segfault occurs. My suspicion is that it's a fencepost error that shows up because of the binary division in the octree. That would be a good first place to look.

...
Anyway, I'm not trying to be too obstructionist here, just trying to have a discussion, since there were at least 4 other issues filed regarding this in the past, so it seems relevant for a lot of users.

I think this would be a great use of resources, but I don't think it should block 3.3. I do not know when I would be able to devote the time to it; if someone else is willing to volunteer that time, I think it would be great. Would you be willing to put the time in to ensure that it could happen? If so, I think making it a blocker on 3.3 and setting a deadline for it would be a good compromise; if it isn't done by that deadline, punt until the next release.

-Matt

...
Cameron

On Mon, May 2, 2016 at 4:26 PM, Nathan Goldbaum wrote:

...
Hi all,

I'm moving some discussion from the comments in issue #1212 over to the mailing list:

https://bitbucket.org/yt_analysis/yt/issues/1212

I wanted to check with the list since Cameron objected a bit in the issue discussion.

The issue Cameron identifies is pretty serious, but fully fixing it will require pretty substantial work - adding support for octree data to the AMRKDTree.

I've gone ahead and marked it for version 3.4 - i.e. that it shouldn't be a blocker for the 3.3 release. I've also created issue 1215:

https://bitbucket.org/yt_analysis/yt/issues/1215

which will be fixed when yt produces a useful error message rather than seg faulting when trying to construct an AMRKDTree for unsupported data.

I think that requiring someone to add support for octree data to the AMRKDTree before we release 3.3 is effectively delaying the 3.3 release arbitrarily far into the future, since no one is currently willing to work on that project. That would be unfortunate, because there's lots of cool stuff people have worked on that deserves a stable release.

-Nathan

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Cameron Hummels

1:28 a.m.

I didn't even know it was something that needed to be done until I tried last week, perhaps there are others who weren't aware this didn't work? Anyway, I'm OK on it not being a 3.3 blocker, but the other issue you filed should be a blocker for 3.3--updating the docs to indicate which frontends work for OAProjs and VR and invitation to devs to help make it work. Cameron On Tue, May 3, 2016 at 3:56 PM, Nathan Goldbaum wrote:

...

On Tue, May 3, 2016 at 5:51 PM, Cameron Hummels wrote:

...
I don't believe I can take this project on right now to make the VR interface work with particle-based and/or octree-based frontends. I don't usually use these frontends, so I can't really justify it to myself to put a bunch of time in on it when there is little research benefit for me, and I've already got my plate full. This may change in the future. Perhaps one of the representatives of one of the affected frontends might be interested in taking this on to assure VR will work with their data?

Well, we've been waiting for a couple years now for someone to volunteer to do that. I'll happily continue waiting, but I still don't think that this should block 3.3

...
Cameron

On Mon, May 2, 2016 at 5:33 PM, Matthew Turk wrote:

...
Hi Cameron,

On Mon, May 2, 2016 at 7:21 PM, Cameron Hummels wrote:

...
I wasn't entirely objecting to punting it, but I think it warrants discussion here. Until I tried to build some Off axis projections and VRs for SPH data, I didn't realize that they weren't supported. In fact, which frontends are actually supported for doing VRs and OffAxisProjections? If Octtree-based code isn't, then that knocks out ART, ARTIO, and Ramses, and without SPH, it knocks out Gadget, Tipsy, Gizmo, and Gasoline, which leaves Enzo, Flash, and Athena? Others?

I honestly don't have any idea how much work it would take to construct a KDTree from octree data. I figured this was in the realm of CS and would probably have been solved elsewhere, but maybe not.

Yes, it absolutely has. When you asked me about this over private message or public message on Slack a while back (I forget which) I sent back information about how to do this. I think it would be relatively straightforward to implement. You need to insert a couple "fake" levels in the octree so that it is a BSP rather than the way it functions now, but these cutting planes can be inserted at the existing cell refinement divisions. It would also be straightforward to implement a ray traversal that natively worked with Octrees, which would only require mocking PartitionedGrid objects and sorting the octs in the right order for the traversal. (This would result in an identical operation, I think.)

My understanding is that when over_refine_factor > 1, it works for particle data sets. Desika had success with this. I may no longer be right.

Even without any new development, I have not heard of anyone attempting to debug why and where the segfault occurs. My suspicion is that it's a fencepost error that shows up because of the binary division in the octree. That would be a good first place to look.

...
Anyway, I'm not trying to be too obstructionist here, just trying to have a discussion, since there were at least 4 other issues filed regarding this in the past, so it seems relevant for a lot of users.

I think this would be a great use of resources, but I don't think it should block 3.3. I do not know when I would be able to devote the time to it; if someone else is willing to volunteer that time, I think it would be great. Would you be willing to put the time in to ensure that it could happen? If so, I think making it a blocker on 3.3 and setting a deadline for it would be a good compromise; if it isn't done by that deadline, punt until the next release.

-Matt

...
Cameron

On Mon, May 2, 2016 at 4:26 PM, Nathan Goldbaum wrote:

...
Hi all,

I'm moving some discussion from the comments in issue #1212 over to the mailing list:

https://bitbucket.org/yt_analysis/yt/issues/1212

I wanted to check with the list since Cameron objected a bit in the issue discussion.

The issue Cameron identifies is pretty serious, but fully fixing it will require pretty substantial work - adding support for octree data to the AMRKDTree.

I've gone ahead and marked it for version 3.4 - i.e. that it shouldn't be a blocker for the 3.3 release. I've also created issue 1215:

https://bitbucket.org/yt_analysis/yt/issues/1215

which will be fixed when yt produces a useful error message rather than seg faulting when trying to construct an AMRKDTree for unsupported data.

I think that requiring someone to add support for octree data to the AMRKDTree before we release 3.3 is effectively delaying the 3.3 release arbitrarily far into the future, since no one is currently willing to work on that project. That would be unfortunate, because there's lots of cool stuff people have worked on that deserves a stable release.

-Nathan

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

Nathan Goldbaum

1:40 a.m.

On Tue, May 3, 2016 at 6:28 PM, Cameron Hummels wrote:

...

I didn't even know it was something that needed to be done until I tried last week, perhaps there are others who weren't aware this didn't work?

Anyway, I'm OK on it not being a 3.3 blocker, but the other issue you filed should be a blocker for 3.3--updating the docs to indicate which frontends work for OAProjs and VR and invitation to devs to help make it work.

it is

...

Cameron

On Tue, May 3, 2016 at 3:56 PM, Nathan Goldbaum wrote:

...
On Tue, May 3, 2016 at 5:51 PM, Cameron Hummels wrote:

...
I don't believe I can take this project on right now to make the VR interface work with particle-based and/or octree-based frontends. I don't usually use these frontends, so I can't really justify it to myself to put a bunch of time in on it when there is little research benefit for me, and I've already got my plate full. This may change in the future. Perhaps one of the representatives of one of the affected frontends might be interested in taking this on to assure VR will work with their data?

Well, we've been waiting for a couple years now for someone to volunteer to do that. I'll happily continue waiting, but I still don't think that this should block 3.3

...
Cameron

On Mon, May 2, 2016 at 5:33 PM, Matthew Turk wrote:

...
Hi Cameron,

On Mon, May 2, 2016 at 7:21 PM, Cameron Hummels wrote:

...
I wasn't entirely objecting to punting it, but I think it warrants discussion here. Until I tried to build some Off axis projections and VRs for SPH data, I didn't realize that they weren't supported. In fact, which frontends are actually supported for doing VRs and OffAxisProjections? If Octtree-based code isn't, then that knocks out ART, ARTIO, and Ramses, and without SPH, it knocks out Gadget, Tipsy, Gizmo, and Gasoline, which leaves Enzo, Flash, and Athena? Others?

I honestly don't have any idea how much work it would take to construct a KDTree from octree data. I figured this was in the realm of CS and would probably have been solved elsewhere, but maybe not.

Yes, it absolutely has. When you asked me about this over private message or public message on Slack a while back (I forget which) I sent back information about how to do this. I think it would be relatively straightforward to implement. You need to insert a couple "fake" levels in the octree so that it is a BSP rather than the way it functions now, but these cutting planes can be inserted at the existing cell refinement divisions. It would also be straightforward to implement a ray traversal that natively worked with Octrees, which would only require mocking PartitionedGrid objects and sorting the octs in the right order for the traversal. (This would result in an identical operation, I think.)

My understanding is that when over_refine_factor > 1, it works for particle data sets. Desika had success with this. I may no longer be right.

Even without any new development, I have not heard of anyone attempting to debug why and where the segfault occurs. My suspicion is that it's a fencepost error that shows up because of the binary division in the octree. That would be a good first place to look.

...
Anyway, I'm not trying to be too obstructionist here, just trying to have a discussion, since there were at least 4 other issues filed regarding this in the past, so it seems relevant for a lot of users.

I think this would be a great use of resources, but I don't think it should block 3.3. I do not know when I would be able to devote the time to it; if someone else is willing to volunteer that time, I think it would be great. Would you be willing to put the time in to ensure that it could happen? If so, I think making it a blocker on 3.3 and setting a deadline for it would be a good compromise; if it isn't done by that deadline, punt until the next release.

-Matt

...
Cameron

On Mon, May 2, 2016 at 4:26 PM, Nathan Goldbaum
...
wrote:

...
Hi all,

I'm moving some discussion from the comments in issue #1212 over to the mailing list:

https://bitbucket.org/yt_analysis/yt/issues/1212

I wanted to check with the list since Cameron objected a bit in the issue discussion.

The issue Cameron identifies is pretty serious, but fully fixing it will require pretty substantial work - adding support for octree data to the AMRKDTree.

I've gone ahead and marked it for version 3.4 - i.e. that it shouldn't be a blocker for the 3.3 release. I've also created issue 1215:

https://bitbucket.org/yt_analysis/yt/issues/1215

which will be fixed when yt produces a useful error message rather than seg faulting when trying to construct an AMRKDTree for unsupported data.

I think that requiring someone to add support for octree data to the AMRKDTree before we release 3.3 is effectively delaying the 3.3 release arbitrarily far into the future, since no one is currently willing to work on that project. That would be unfortunate, because there's lots of cool stuff people have worked on that deserves a stable release.

-Nathan

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

2914

Age (days ago)

2915

Last active (days ago)

List overview

Download

8 comments

4 participants

participants (4)

Cameron Hummels
Kacper Kowalik
Matthew Turk
Nathan Goldbaum

Should support for octree data in the AMRKDTree be a blocker for 3.3?

tags

participants (4)