Hello all,
I think this may be my first post on this list - I'm Erik Tollerud, one of the members of the Astropy coordinating committee.
Today I learned there are plans afoot to do some refactoring of the yt units system. I was wondering if any of you here have spent much time looking at the Astropy units framework (https://astropy.readthedocs.org/en/stable/units/index.html)?
I couldn't find much in a quick glance over yt's docs about how the unit system actually works, but I'm sure there's a lot of overlap there. So I can't help but think that we should at least try to get some sort of interoperability in place. Of course, you're welcome to directly use any part of Astropy's unit or Quantity frameworks that right now suit your needs. But given that one of Astropy's main goals is to improve compatibility and interoperability in python astronomy, tools, we are happy to do some leg work on our end if it will help you.
Hi Erik,
We did look a bit at astropy.units, but decided not to rely on it for a couple of reasons. Some of this might be mistaken on my part - I’m not very familiar with astropy’s unit system.
1) Since many yt fields are complicated functions of other yt fields, we wanted to use a unit system that nicely and automatically handles unit conversions with fractional units or powers of units. We ended up going with Casey Stark’s dimensional [1] library, which uses sympy to do symbolic manipulations on unit symbols. We’re not using dimensionful proper, but a bundled version that we’ve modified somewhat to suit our needs.
2) We have domain-specific constraints: specifically we need a concept of ‘code units’ as well as simulation parameter dependent units (i.e. cosmological units, magnetic field units, and other domain-specific units). For example, the Enzo code has an internal unit system such that the domain runs from 0.0 to 1.0, yet the physical system the simulation represents can have arbitrary length units. We want to present the user quantities in physical units rather than internal ‘code units', but we still want to make the internal unit data available (for simulation debugging purposes, for example). This means that the code units need to be baked in to the unit system.
3) Avoiding a new hard dependency. This one is a bit more debateable, but we want to expand beyond astronomy to become a general tool for analysis of volumetric data - having a hard dependency on astropy might hamstring those efforts.
Right now there isn’t any documentation for the new unit system, although it is described in a YTEP [2].
That said, I think there’s definitely room for a conversion layer to astropy.units. I’ll admit, I haven’t looked very closely at this, but I think it will be doable. Would love to chat more about this once the unit work has settled down a bit.
[1] https://github.com/caseywstark/dimensionful [2] http://ytep.readthedocs.org/en/latest/YTEPs/YTEP-0011.html
On December 3, 2013 at 1:26:17 PM, Erik Tollerud (erik.tollerud@gmail.com) wrote:
Hello all,
I think this may be my first post on this list - I'm Erik Tollerud, one of the members of the Astropy coordinating committee.
Today I learned there are plans afoot to do some refactoring of the yt units system. I was wondering if any of you here have spent much time looking at the Astropy units framework (https://astropy.readthedocs.org/en/stable/units/index.html)?
I couldn't find much in a quick glance over yt's docs about how the unit system actually works, but I'm sure there's a lot of overlap there. So I can't help but think that we should at least try to get some sort of interoperability in place. Of course, you're welcome to directly use any part of Astropy's unit or Quantity frameworks that right now suit your needs. But given that one of Astropy's main goals is to improve compatibility and interoperability in python astronomy, tools, we are happy to do some leg work on our end if it will help you.
-- Erik T _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Nathan et al.,
See below for the specific items - I think Astropy's units do support this, but of course I understand that you've already started developing and don't necessarily want to change horses in mid-stream.
- Since many yt fields are complicated functions of other yt fields, we
wanted to use a unit system that nicely and automatically handles unit conversions with fractional units or powers of units. We ended up going with Casey Stark’s dimensional [1] library, which uses sympy to do symbolic manipulations on unit symbols. We’re not using dimensionful proper, but a bundled version that we’ve modified somewhat to suit our needs.
This may not have been true when you last looked, but Astropy's unit system does this now - at one point it did not, but it now supports fractional units. E.g., you can do:
from astropy import units as u sqm = u.m ** 0.5 assert not sqm == u.m m2 = sqm ** 2 assert m2 == i.m
- We have domain-specific constraints: specifically we need a concept of
‘code units’ as well as simulation parameter dependent units (i.e. cosmological units, magnetic field units, and other domain-specific units). For example, the Enzo code has an internal unit system such that the domain runs from 0.0 to 1.0, yet the physical system the simulation represents can have arbitrary length units. We want to present the user quantities in physical units rather than internal ‘code units', but we still want to make the internal unit data available (for simulation debugging purposes, for example). This means that the code units need to be baked in to the unit system.
In principal I think this *could* be done with the Astropy units system pretty easily: you just define a new unit type, e.g. "tipsy_length", and give a scale between that and, say m. Then converting, say, an array from simulation units to meters is as simple as:
metersarray = (simarray * tipsy_length).to(u.m)
Or am I missing some subtlety?
- Avoiding a new hard dependency. This one is a bit more debateable, but
we want to expand beyond astronomy to become a general tool for analysis of volumetric data - having a hard dependency on astropy might hamstring those efforts.
That's totally understandable. We've considered splitting out our unit stuff to be a separate package, but I think the feeling is that it's more administrative overhead for no clear payoff. But if someone really wanted it, we wouldn't necessarily be against doing that.
That said, I think there’s definitely room for a conversion layer to astropy.units. I’ll admit, I haven’t looked very closely at this, but I think it will be doable. Would love to chat more about this once the unit work has settled down a bit.
Sounds good - from what I see on the YTEP, I suspect it should be pretty easy to do a mapping in both directions, but of course there could easily be devils in the details.
On December 3, 2013 at 1:26:17 PM, Erik Tollerud (erik.tollerud@gmail.com) wrote:
Hello all,
I think this may be my first post on this list - I'm Erik Tollerud, one of the members of the Astropy coordinating committee.
Today I learned there are plans afoot to do some refactoring of the yt units system. I was wondering if any of you here have spent much time looking at the Astropy units framework (https://astropy.readthedocs.org/en/stable/units/index.html)?
I couldn't find much in a quick glance over yt's docs about how the unit system actually works, but I'm sure there's a lot of overlap there. So I can't help but think that we should at least try to get some sort of interoperability in place. Of course, you're welcome to directly use any part of Astropy's unit or Quantity frameworks that right now suit your needs. But given that one of Astropy's main goals is to improve compatibility and interoperability in python astronomy, tools, we are happy to do some leg work on our end if it will help you.
-- Erik T _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Erik,
On Thu, Dec 5, 2013 at 2:15 PM, Erik Tollerud erik.tollerud@gmail.com wrote:
Hi Nathan et al.,
See below for the specific items - I think Astropy's units do support this, but of course I understand that you've already started developing and don't necessarily want to change horses in mid-stream.
Well, one issue is that we're actually reaching production level at this point. While the PR was only issued this week, Casey and Nathan have spent a good amount of time working on this off and on, including at sprints, for over a year. At the time we started, the astropy units system was not as well-defined as it is now, and we spent a good amount of time bikeshedding on our own. :)
- Since many yt fields are complicated functions of other yt fields, we
wanted to use a unit system that nicely and automatically handles unit conversions with fractional units or powers of units. We ended up going with Casey Stark’s dimensional [1] library, which uses sympy to do symbolic manipulations on unit symbols. We’re not using dimensionful proper, but a bundled version that we’ve modified somewhat to suit our needs.
This may not have been true when you last looked, but Astropy's unit system does this now - at one point it did not, but it now supports fractional units. E.g., you can do:
from astropy import units as u sqm = u.m ** 0.5 assert not sqm == u.m m2 = sqm ** 2 assert m2 == i.m
- We have domain-specific constraints: specifically we need a concept of
‘code units’ as well as simulation parameter dependent units (i.e. cosmological units, magnetic field units, and other domain-specific units). For example, the Enzo code has an internal unit system such that the domain runs from 0.0 to 1.0, yet the physical system the simulation represents can have arbitrary length units. We want to present the user quantities in physical units rather than internal ‘code units', but we still want to make the internal unit data available (for simulation debugging purposes, for example). This means that the code units need to be baked in to the unit system.
In principal I think this *could* be done with the Astropy units system pretty easily: you just define a new unit type, e.g. "tipsy_length", and give a scale between that and, say m. Then converting, say, an array from simulation units to meters is as simple as:
metersarray = (simarray * tipsy_length).to(u.m)
Or am I missing some subtlety?
There are a few reasons we need to tie individual units to a particulr lookup table and/or registry. The first is that we often have multiple timesteps in memory at the same time; the "code_length" for different timesteps will thus be different, and so we can't have a global code_length meaning. There's also the issue of different frontends being compared directly -- resolving to intermediate units, for instance, before comparing the two.
- Avoiding a new hard dependency. This one is a bit more debateable, but
we want to expand beyond astronomy to become a general tool for analysis of volumetric data - having a hard dependency on astropy might hamstring those efforts.
That's totally understandable. We've considered splitting out our unit stuff to be a separate package, but I think the feeling is that it's more administrative overhead for no clear payoff. But if someone really wanted it, we wouldn't necessarily be against doing that.
That said, I think there’s definitely room for a conversion layer to astropy.units. I’ll admit, I haven’t looked very closely at this, but I think it will be doable. Would love to chat more about this once the unit work has settled down a bit.
Sounds good - from what I see on the YTEP, I suspect it should be pretty easy to do a mapping in both directions, but of course there could easily be devils in the details.
On December 3, 2013 at 1:26:17 PM, Erik Tollerud (erik.tollerud@gmail.com) wrote:
Hello all,
I think this may be my first post on this list - I'm Erik Tollerud, one of the members of the Astropy coordinating committee.
Today I learned there are plans afoot to do some refactoring of the yt units system. I was wondering if any of you here have spent much time looking at the Astropy units framework (https://astropy.readthedocs.org/en/stable/units/index.html)?
I couldn't find much in a quick glance over yt's docs about how the unit system actually works, but I'm sure there's a lot of overlap there. So I can't help but think that we should at least try to get some sort of interoperability in place. Of course, you're welcome to directly use any part of Astropy's unit or Quantity frameworks that right now suit your needs. But given that one of Astropy's main goals is to improve compatibility and interoperability in python astronomy, tools, we are happy to do some leg work on our end if it will help you.
-- Erik T _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Erik _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (3)
-
Erik Tollerud -
Matthew Turk -
Nathan Goldbaum