Hi Matt,
There isn't currently an easy way to share boundary conditions. However, I think we could attempt to address this with a few easy-ish changes. I can think of three ways, and there may be more.
1) Provide tools for neighbor location + communication
Here we would use the structure of the tree to quickly identify which processor owns the neighboring region and send across whatever is needed. This is fairly straightforward since the tree is built out to the point where domain decomp is done, so this is super fast to just traverse to find the owner. Once known, request a a boundary layer. This would probably require coming up with some sort of data object that acts as a boundary layer. I guess this could actually be an AMR2DData object that gets filled out, sent over, but that might be overkill?
2) Build the full tree on all procs, but only only own the data for a subset.
I think the AMRKDTree build may be fast enough at this point to just build the entire thing on each processor, then later decide which nodes to populate with data. This could be done by just setting a buffer around the initial parallel decomp.
3) Build the tree once up until the point where you have Nprocs leafs. Then build a second tree that has left and right edges that are buffered around the original tree decomp. This would be much like option 2, but necessary if the tree build time became dominant.
I think option 2 and 3 would probably be the easiest, though I think it would be a neat to set up communication between nodes through option 1.
I'd be happy to sprint on this sometime if you'd like.
Sam