Just a quick reminder -- at 1PM today I'll be holding a G+ hangout about geometry refactoring. I've prepared a bit of a presentation, and I'll also go over some code, but I mainly want to get some feedback and talk about ideas! This will be a great time to brainstorm as well as provide feedback on what is already in place. The hangout will be on my profile, which is here:
That's in just about three hours. No further reminders will be forthcoming.
I also wanted to thank everyone for replying, and I'll respond briefly to comments here. One thing I really want to convey, though, is that this refactor is mostly about stopping thinking of everything in terms of grids. This is the myopia that would prevent yt from really moving forward; to that end, I do not want any data objects in yt to directly access grid objects unless they are specifically tasked with handling grid geometry or handling a
Casey -- returning a chunk object is probably cleaner, but I didn't want to add on a bunch of new overloads and potentially duplicated logic. I don't see reading in a chunked fashion as something that is commonly (or ever) exposed to analysis scripts, and if a script author does choose to use chunking, they should know what they are doing. Each data object (i.e., sphere, region, etc) does not define its own chunking routine, but rather calls the chunking routine from the geometry handler, which is something like a grid-based handler, particle-based, oct-based. These can be overridden by specific codes (for instance, if IO chunking proceeds differently in Nyx versus in Enzo, even though they are both grid based codes.)
Nathan -- I like the syntax you and Casey have come up with. We can also add [:] or "all" for particle typing.
Chris -- yup, data_objects needs rewriting. That's what I have been doing, and I got first light on plotting slices, cutting planes and profiles this weekend. The harder one will be doing data objects that require some degree of construction, like profiles and covering grids. I hope to start work on those soon. As for if this would help with SPH, the answer is emphatically yes! While actually calculating fluid quantities in particles that require neighbor searches will still be challenging, the process of loading and looking at SPH data in the raw should be greatly improved. As an example of what we will be able to do with Octs, we should be able to load ART data, store the index in memory, chunk by whatever is best for IO (sorted by resolution, perhaps) and add to the exported, resorted octree without any regridding. I'd like to work on this with you very soon.
Stephen -- I am not yet sure about implementation of particle IO, but I would see selection of all particles in a given region being available via something like obj.particles[:]["whatever_field"]. I think we can mostly avoid concatenation in this way. As for accessing things via .grid[field], I am -1 on this since I don't really want to be thinking about grids. We could consider .fluid, but in general I think I am okay with continuing to keep fluids first class and particles separate from them. I am willing to be convinced otherwise.
On Sat, Feb 25, 2012 at 7:26 PM, Stephen Skory email@example.com wrote:
One thought comes to mind with regards to selecting particle types as part of the new method: There needs to be a simple and memory efficient way to select multiple types of particles (or all types). For example, for the halo finders, sometimes only dark matter is wanted, and other times stars (or perhaps even only certain types of stars) are wanted as well. In the past we've done things to avoid na.concatenate with particles because it fragments memory. I think Matt has thought of this already, but I'd like to extend this discussion to how to allow users to select particle types without having to do na.concatenate manually (or have it happen behind the scenes).
With regards to obj['particle_position_x'] or obj.particles['position_x'], I think the argument for changing it is not a bad one, but that if we do this, I think it would make the most sense to extend it to other things, like obj.grid['Density'], for continuity's sake.
I'm for a user survey if we think we'll get replies. Have users generally been forthcoming with opinions on technical stuff like this before?
I'll see you in the chat on Monday!
-- Stephen Skory firstname.lastname@example.org http://stephenskory.com/ 510.621.3687 (google voice)
yt-dev mailing list email@example.com http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org