One possibility you could consider is -- if the field matches 1-to-1
with an existing grid or particle structure -- is using a derived
field, setting the validators to NeedsGridType, and then having it
access the data.id to figure out which grid it is and returning the
appropriately shaped field itself.
(Incidentally your use case is what the Stream handler was originally
designed for -- and then the usage for loading arbitrary datasets took
over its use patterns!)
On Fri, Apr 30, 2021 at 4:51 PM Matthew Abruzzo
> Hi Britton,
> Thanks for the response. This is really helpful! I haven't actually written the code to generate this external field data yet, but I could definitely package it in the same shape as the native data.
> I looked through the source code and it seems like this requires 2 main changes:
> 1. Tweak the field_list held in the index attribute of a Dataset instance. (Out of curiosity, will things break if the field_list changes after the Dataset instance has been fully initialized? Or does the _detect_output_fields method of the frontend's Index subclass need to be directly modified?)
> 2. Tweak the implementation of the frontend's subclass of BaseIOHandler such that the _read_fluid_selection method reads the field data from the sidecar files alongside the native data.
> Am I missing any other obvious steps? Thanks again for your help.
> On Wed, Apr 28, 2021 at 4:42 AM Britton Smith <email@example.com> wrote:
>> Hi Matthew,
>> There is no frontend-agnostic support for this, but I believe it has been implemented in a few cases. If the data in the external files has the same layout as the native data, this should be quite straightforward to implement. One would have to add machinery to search for these side-car files and add their contents to the field list, and then to know which files to open to get the associated fields. The difficulty would come in if the field data had a different shape than the native data, as this would confuse the selector functions. If what you have meets these criteria, then I'm happy to discuss in more detail.
>> On Mon, Apr 26, 2021 at 7:53 PM Matthew Abruzzo <firstname.lastname@example.org> wrote:
>>> I wanted to ask if yt supports the registration of external data as a field of an existing dataset.
>>> In case it helps clarify my question, I just wanted to briefly elaborate on what I'm looking to do. I'm trying to characterize the turbulent velocities in some high resolution unigrid simulations. Because this involves convolution, I need to compute the turbulent velocity outside of the yt infrastructure (this seems beyond the scope of what a user-defined derived field can support). It would be convenient to be able to load the results back into yt to use the existing analysis machinery (e.g. for phase plots and slices). I have thought about using the "Generic Unigrid Data" frontend, but the simulation's size unfortunately makes that approach somewhat intractable.
>>> I'm pretty confident that the answer to my question is "no," but I wanted to be sure before I rolled my own solution. Thank you for your help.
>>> Matthew Abruzzo
>>> yt-users mailing list -- email@example.com
>>> To unsubscribe send an email to firstname.lastname@example.org
>>> Member address: email@example.com
>> yt-users mailing list -- firstname.lastname@example.org
>> To unsubscribe send an email to email@example.com
>> Member address: firstname.lastname@example.org
> yt-users mailing list -- email@example.com
> To unsubscribe send an email to firstname.lastname@example.org
> Member address: email@example.com
yt-users mailing list -- firstname.lastname@example.org
To unsubscribe send an email to email@example.com
Member address: firstname.lastname@example.org