Out of general curiosity, what aspect of the IO would have to change?
Well, there are a couple problems. What I'll present as the most problematic is the copying of arrays. We constructs sphere to analyze halos, which then load-on-demand. If you either ran these as field-cuts (inlineextractedregions) from the tiles or enabled preloading of data and disabled the 'pop' method in the DataQueue object, you could get around this. It's something to think about for later, but I'm not yet it's an issue right now. I think we should try implementing the round robin, then run cProfile on it to see if IO is the limiting factor.
Thank you very much, Matt! While I was certainly not requesting you do my work for me, I had the feeling that what I wanted to do would be related to (or impinge upon) overarching plans you already had for yt. Let me know what I can do for you.
will do. This is in support of a ticket I opened maybe a month or two ago, 192, which is really two-in-one. -Matt