Mailman 3 Make DomainSubsets a subclass of YTSelectionContainer? - yt-dev

Make DomainSubsets a subclass of YTSelectionContainer?

Matthew Turk

17 Apr 2013 17 Apr '13

8:24 p.m.

Hi all, Today in IRC, Chris and I went back and forth on the particle deposition stuff in advance of the meeting next week. My task right now is to come up with spatial field chunking for Octs in a consistent way; of course I'll write this up into the extant YTEP, which will allow for feedback particularly from ARTIO, ART and RAMSES folks, but as a quick first pass I was wondering what everyone thought of: 1) Unifying DomainSubsets into a single object with subclasses 2) Making that new base class a subclass, itself, of YTSelectionContainer This would mean that this new base class would be very similar to the GridPatch object -- which it already is -- and would implement, most crucially, the __getitem__ accessor pattern. This would return a 2x2x2xNoct field. [+-][01]? Specifically, Doug, Chris, and SamL? Sidenote: I discovered that we will not have to rewrite spatial fields that currently work for grids; my understanding of the numpy broadcasting system was not complete, and I didn't realize that this script does *exactly what I want it to. http://paste.yt-project.org/show/3391/ This means that our Oct spatial fields will be (2+NGZ*2, 2+NGZ*2, 2+NGZ*2, NOCT) shaped.

Show replies by date

Douglas Harvey Rudd

17 Apr 17 Apr

8:35 p.m.

Hi Matt, I'm +1 on specifying the DomainSubset interface though subclasses, and it makes sense to integrate it with selection. One question I have relating to deposition in particular is whether the spatial fields are expected to impose any sort of order on the octs they return. If we select a region including both particles and octs, but the two are completely unordered, matching one set to the other may be very expensive. Do the deposition routines already use some organization scheme (oct tree, say) to match particles to grids? Doug On Apr 17, 2013, at 3:24 PM, Matthew Turk wrote:

...

Hi all,

Today in IRC, Chris and I went back and forth on the particle deposition stuff in advance of the meeting next week. My task right now is to come up with spatial field chunking for Octs in a consistent way; of course I'll write this up into the extant YTEP, which will allow for feedback particularly from ARTIO, ART and RAMSES folks, but as a quick first pass I was wondering what everyone thought of:

1) Unifying DomainSubsets into a single object with subclasses 2) Making that new base class a subclass, itself, of YTSelectionContainer

This would mean that this new base class would be very similar to the GridPatch object -- which it already is -- and would implement, most crucially, the __getitem__ accessor pattern. This would return a 2x2x2xNoct field.

[+-][01]? Specifically, Doug, Chris, and SamL?

Sidenote: I discovered that we will not have to rewrite spatial fields that currently work for grids; my understanding of the numpy broadcasting system was not complete, and I didn't realize that this script does *exactly what I want it to.

http://paste.yt-project.org/show/3391/

This means that our Oct spatial fields will be (2+NGZ*2, 2+NGZ*2, 2+NGZ*2, NOCT) shaped. _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Christopher Moody

8:38 p.m.

Hi Matt, +1 for subclassing a DomainSubset common to ARTIO, ART and RAMSES. chris On Wed, Apr 17, 2013 at 1:35 PM, Douglas Harvey Rudd wrote:

...

Hi Matt,

I'm +1 on specifying the DomainSubset interface though subclasses, and it makes sense to integrate it with selection.

One question I have relating to deposition in particular is whether the spatial fields are expected to impose any sort of order on the octs they return. If we select a region including both particles and octs, but the two are completely unordered, matching one set to the other may be very expensive.

Do the deposition routines already use some organization scheme (oct tree, say) to match particles to grids?

Doug

On Apr 17, 2013, at 3:24 PM, Matthew Turk wrote:

...
Hi all,

Today in IRC, Chris and I went back and forth on the particle deposition stuff in advance of the meeting next week. My task right now is to come up with spatial field chunking for Octs in a consistent way; of course I'll write this up into the extant YTEP, which will allow for feedback particularly from ARTIO, ART and RAMSES folks, but as a quick first pass I was wondering what everyone thought of:

1) Unifying DomainSubsets into a single object with subclasses 2) Making that new base class a subclass, itself, of YTSelectionContainer

This would mean that this new base class would be very similar to the GridPatch object -- which it already is -- and would implement, most crucially, the __getitem__ accessor pattern. This would return a 2x2x2xNoct field.

[+-][01]? Specifically, Doug, Chris, and SamL?

Sidenote: I discovered that we will not have to rewrite spatial fields that currently work for grids; my understanding of the numpy broadcasting system was not complete, and I didn't realize that this script does *exactly what I want it to.

http://paste.yt-project.org/show/3391/

This means that our Oct spatial fields will be (2+NGZ*2, 2+NGZ*2, 2+NGZ*2, NOCT) shaped. _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Sam Leitner

8:58 p.m.

Hi Matt, All of that sounds good to me as well. Sam On Wednesday, April 17, 2013, Christopher Moody wrote:

...

Hi Matt, +1 for subclassing a DomainSubset common to ARTIO, ART and RAMSES.

chris

On Wed, Apr 17, 2013 at 1:35 PM, Douglas Harvey Rudd

...
wrote:

...
Hi Matt,

I'm +1 on specifying the DomainSubset interface though subclasses, and it makes sense to integrate it with selection.

One question I have relating to deposition in particular is whether the spatial fields are expected to impose any sort of order on the octs they return. If we select a region including both particles and octs, but the two are completely unordered, matching one set to the other may be very expensive.

Do the deposition routines already use some organization scheme (oct tree, say) to match particles to grids?

Doug

On Apr 17, 2013, at 3:24 PM, Matthew Turk

...
wrote:

...
Hi all,

Today in IRC, Chris and I went back and forth on the particle deposition stuff in advance of the meeting next week. My task right now is to come up with spatial field chunking for Octs in a consistent way; of course I'll write this up into the extant YTEP, which will allow for feedback particularly from ARTIO, ART and RAMSES folks, but as a quick first pass I was wondering what everyone thought of:

1) Unifying DomainSubsets into a single object with subclasses 2) Making that new base class a subclass, itself, of YTSelectionContainer

This would mean that this new base class would be very similar to the GridPatch object -- which it already is -- and would implement, most crucially, the __getitem__ accessor pattern. This would return a 2x2x2xNoct field.

[+-][01]? Specifically, Doug, Chris, and SamL?

Sidenote: I discovered that we will not have to rewrite spatial fields that currently work for grids; my understanding of the numpy broadcasting system was not complete, and I didn't realize that this script does *exactly what I want it to.

http://paste.yt-project.org/show/3391/

This means that our Oct spatial fields will be (2+NGZ*2, 2+NGZ*2, 2+NGZ*2, NOCT) shaped. _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Matthew Turk

8:42 p.m.

On Wed, Apr 17, 2013 at 4:35 PM, Douglas Harvey Rudd wrote:

...

Hi Matt,

I'm +1 on specifying the DomainSubset interface though subclasses, and it makes sense to integrate it with selection.

Awesome.

...

One question I have relating to deposition in particular is whether the spatial fields are expected to impose any sort of order on the octs they return. If we select a region including both particles and octs, but the two are completely unordered, matching one set to the other may be very expensive.

That's a *really* good point ,and one that I had hoped we could address through chunking. I *definitely* want to avoid an NxM search of particles to octs, so I was thinking two prongs to this: 1) This will happen inside a chunked iteration if at all possible, which reduces the search space for particles considerably 2) We will utilize the existing octree infrastructure (i.e., OctContainer and its subclasses) to identify the octs to which particles belong I think this should make it fast enough, as we'll be searching within a chunk, which should already be relatively small. But we can certainly adjust this later.

...

Do the deposition routines already use some organization scheme (oct tree, say) to match particles to grids?

Doug

On Apr 17, 2013, at 3:24 PM, Matthew Turk wrote:

...
Hi all,

Today in IRC, Chris and I went back and forth on the particle deposition stuff in advance of the meeting next week. My task right now is to come up with spatial field chunking for Octs in a consistent way; of course I'll write this up into the extant YTEP, which will allow for feedback particularly from ARTIO, ART and RAMSES folks, but as a quick first pass I was wondering what everyone thought of:

1) Unifying DomainSubsets into a single object with subclasses 2) Making that new base class a subclass, itself, of YTSelectionContainer

This would mean that this new base class would be very similar to the GridPatch object -- which it already is -- and would implement, most crucially, the __getitem__ accessor pattern. This would return a 2x2x2xNoct field.

[+-][01]? Specifically, Doug, Chris, and SamL?

Sidenote: I discovered that we will not have to rewrite spatial fields that currently work for grids; my understanding of the numpy broadcasting system was not complete, and I didn't realize that this script does *exactly what I want it to.

http://paste.yt-project.org/show/3391/

This means that our Oct spatial fields will be (2+NGZ*2, 2+NGZ*2, 2+NGZ*2, NOCT) shaped. _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Douglas Harvey Rudd

8:53 p.m.

...

1) This will happen inside a chunked iteration if at all possible, which reduces the search space for particles considerably

I agree, and was assuming this also, although it's a weak constraint (the ratio of particles to octs in any given region is probably constant for dark matter, but will vary widely for stars).

...

2) We will utilize the existing octree infrastructure (i.e., OctContainer and its subclasses) to identify the octs to which particles belong

Okay, then we should consider how to translate an arbitrary collection of leaf-octs into the OctContainer.

...

I think this should make it fast enough, as we'll be searching within a chunk, which should already be relatively small. But we can certainly adjust this later.

Right, I'm all in favor of getting something that works first, and worry about optimization once it becomes necessary, provided we don't prevent a scheme from running, albeit slowly. As long as we always operate on chunks, that shouldn't be an issue. Doug

...

...
Do the deposition routines already use some organization scheme (oct tree, say) to match particles to grids?

Doug

On Apr 17, 2013, at 3:24 PM, Matthew Turk wrote:

...
Hi all,

Today in IRC, Chris and I went back and forth on the particle deposition stuff in advance of the meeting next week. My task right now is to come up with spatial field chunking for Octs in a consistent way; of course I'll write this up into the extant YTEP, which will allow for feedback particularly from ARTIO, ART and RAMSES folks, but as a quick first pass I was wondering what everyone thought of:

1) Unifying DomainSubsets into a single object with subclasses 2) Making that new base class a subclass, itself, of YTSelectionContainer

This would mean that this new base class would be very similar to the GridPatch object -- which it already is -- and would implement, most crucially, the __getitem__ accessor pattern. This would return a 2x2x2xNoct field.

[+-][01]? Specifically, Doug, Chris, and SamL?

Sidenote: I discovered that we will not have to rewrite spatial fields that currently work for grids; my understanding of the numpy broadcasting system was not complete, and I didn't realize that this script does *exactly what I want it to.

http://paste.yt-project.org/show/3391/

This means that our Oct spatial fields will be (2+NGZ*2, 2+NGZ*2, 2+NGZ*2, NOCT) shaped. _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

4020

Age (days ago)

4020

Last active (days ago)

List overview

Download

5 comments

4 participants

participants (4)

Christopher Moody
Douglas Harvey Rudd
Matthew Turk
Sam Leitner

Make DomainSubsets a subclass of YTSelectionContainer?

Matthew Turk

Douglas Harvey Rudd

Christopher Moody

Sam Leitner

Matthew Turk

Douglas Harvey Rudd

tags

participants (4)