Hi Britton,

I've thought about this a bit, and I think this can be addressed.  In
the ytdata frontend, we should be able to make it recognize the index
fields; if we can't, we are overdue for having coordinate handlers
check things out.  There should be logic that prevents derived fields
overriding on-disk fields, so if there is an on-disk field that
presents "index" it should work.

On Tue, Oct 27, 2015 at 9:35 AM, Britton Smith <brittonsmith@gmail.com> wrote:
> Hi all,
>
> I have spent a while looking at the issue involving the derived quantities
> and I have come up with a solution for some of them, but not all of them.  I
> have pushed a couple changes to the PR that add a cell_volume and cell_mass
> field specific to YTDataContainerDatasets that makes use of their "dx",
> "dy", and "dz" fields.  As of now, the following derived quantities work:
> - min location
> - max location
> - extrema
> - bulk velocity
> - total mass
> - weighted average quantity
> - weighted variance
> - total quantity
>
> The ones that don't work are:
> - spin parameter
> - center of mass
> - angular momentum vector
>
> The final three all don't work for the same reason, which is that they
> require access to the "index" position fields.  Unfortunately, these cannot
> be overridden for reasons that have to do with the _determine_fields
> function.  I have made a point during this development process to alter as
> little as possible outside of the ytdata frontend.  However, at this point,
> there is no more room to maneuver.  The only solutions I can come up with
> are ugly and would make it very difficult to smoothly swap in a proper AMR
> hierarchy when we get around to creating one and doing this properly.
>
> The last thing I can think of would be to swap in special derived quantity
> functions for this frontend only that use position fields appropriate for
> the frontend.  The functions themselves would be simple to implement, but
> because the derived quantity collection is added onto data container objects
> and not dataset objects, I have no idea how one would do the swap.  For a
> similar reason, I don't know how to simply not allow these problem cases to
> be called, or to fail in a clean way for them.
>
> If anyone has any ideas, I would love to hear them.
>
> Britton
>
> On Mon, Oct 26, 2015 at 10:12 PM, John ZuHone <jzuhone@gmail.com> wrote:
>>
>> No worries—the main issues now are the tests (which don’t run under Python
>> 2) and Python 3.
>>
>> Tomorrow I’ll point you at the head that I’ve been making changes on to
>> address these.
>>
>> On Oct 26, 2015, at 6:09 PM, Britton Smith <brittonsmith@gmail.com> wrote:
>>
>> Hi John,
>>
>> At the PR hangout, we discussed the possibility of eventually creating a
>> method to extract subsets of AMR hierarchies that would include only grid
>> patches that are at least partially contained within the data container.
>> With them, we would also include the mask arrays to be applied to each grid
>> patch such that you would only get the cells within the container.  However,
>> this was deemed something that should not hold up this PR, but that could
>> eventually subbed in to fill in any gaps in functionality created by
>> treating the grid data like particles.  We decided that patching the
>> existing machinery with a frontend-specific cell_volume field was the best
>> thing as it would get this on front of users faster, especially since it
>> contains a lot of other things that are fully working.  I may have missed
>> some important parts of the discussion, but my memory faded a lot faster
>> than I thought it would.
>>
>> I probably won't have much time to work on this until Friday, but I'll try
>> to play around with the derived quantities when I can.
>>
>> Britton
>>
>> On Mon, Oct 26, 2015 at 9:45 PM, John ZuHone <jzuhone@gmail.com> wrote:
>>>
>>> Hi all,
>>>
>>> Matt asked someone to send a quick note about the current issues
>>> regarding the ytdata frontend PR:
>>>
>>>
>>> https://bitbucket.org/yt_analysis/yt/pull-requests/1788/adding-ytdata-frontend
>>>
>>> I’m sending it because I’m working on it currently, but I was not able to
>>> attend today’s PR hangout so anyone else who was more to add or to correct
>>> should chime in.
>>>
>>> Currently, there are 3 issues, from most important to least important:
>>>
>>> 1. Tests don’t work
>>> 2. Doesn’t support Python 3
>>> 3. Derived quantities don’t work
>>>
>>> I’m currently working on 1 and 2, and there have been some ideas about 3.
>>> The main issue with the latter is that for non-gridded data containers (like
>>> sphere, disk, etc.) created from grid datasets, we treat the new datasets as
>>> “particle data”. For that reason there is an issue with calculating the
>>> cell_volume properly. Without that, we cannot use the total_mass() quantity.
>>> Given that we have the cell deltas in the dataset, we could have a special
>>> derived field for the cell_volume.
>>>
>>> There are other issues with some of the other derived quantities (e.g.,
>>> weighted_average_quantity returning the wrong units), but I haven’t been
>>> able to diagnose them yet.
>>>
>>> I’m working on some changes that will need to be merged into Britton’s
>>> fork eventually to address at least 1 and 2.
>>>
>>> Best,
>>>
>>> John
>>>
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev@lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev@lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev@lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev@lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
_______________________________________________
yt-dev mailing list
yt-dev@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org