Hello all,
I think this may be my first post on this list - I'm Erik Tollerud,
one of the members of the Astropy coordinating committee.
Today I learned there are plans afoot to do some refactoring of the yt
units system. I was wondering if any of you here have spent much time
looking at the Astropy units
framework (https://astropy.readthedocs.org/en/stable/units/index.html)?
I couldn't find much in a quick glance over yt's docs about how the
unit system actually works, but I'm sure there's a lot of overlap
there. So I can't help but think that we should at least try to get
some sort of interoperability in place. Of course, you're welcome to
directly use any part of Astropy's unit or Quantity frameworks that
right now suit your needs. But given that one of Astropy's main goals
is to improve compatibility and interoperability in python astronomy,
tools, we are happy to do some leg work on our end if it will help
you.
--
Erik T
Hi all,
On Thursday, I'll be hosting a hangout at 2PM Eastern. This will be
mostly or exclusively about the units refactor, which Nathan and I
will be issuing a PR for later today or tomorrow.
Agenda:
* Go over the units refactor from a high level
* Describe the new field plugin system
* Talk about where things have changed
* Get feedback as people pull, update to, and try out the units
system on their data
The final one, where we're all going to pull down the changes, update
to them and try them out, is going to be the most important part.
So, if you can attend, that'd be really great. I've created an event here:
https://plus.google.com/b/107728486871834552760/events/cnsc16nn6v8trc8n0ev1…
I've enabled people who are attending to invite additional people, so
if you aren't already in yt's circles (but I think most people are)
you can just pop in to IRC on the day of and someone will invite you.
-Matt
New issue 743: Derived quantities are difficult to make unit aware
https://bitbucket.org/yt_analysis/yt/issue/743/derived-quantities-are-diffi…
Matthew Turk:
Derived quantities at present do not know anything about the units they are to return. This presents something of a problem, as we perform several operations on them that are challenging to do in a neutral fashion; they can return scalars as well as arrays, and either or both might be unit-ful.
Here is the logic for how they are defined:
```
#!python
retvals = [ [] for i in range(self.n_ret)]
chunks = self._data_source.chunks([], chunking_style="io")
for ds in parallel_objects(chunks, -1):
rv = self.func(ds, *args, **kwargs)
if not iterable(rv): rv = (rv,)
for i in range(self.n_ret): retvals[i].append(rv[i])
# Note that we do some fancy footwork here.
# _par_combine_object and its affiliated alltoall function
# assume that the *long* axis is the last one. However,
# our long axis is the first one!
rv = []
for my_list in retvals:
data = YTArray(my_list).transpose()
rv.append(self.comm.par_combine_object(data,
datatype="array", op="cat").transpose())
retvals = rv
return self.c_func(self._data_source, *retvals)
```
Now, there are only a relatively small number of derived quantities, and some likely have to be rewritten to take advantage of the particle type and fluid type systems. Additionally, we wanted to add them "on demand" similar to fluid and particle fields. So I see a few different options.
1. Rewrite them to be unit-aware. This could also include fixes for how reduction operations are set up and an explicit noting of how many return values are to be expected.
2. Sanitize all arrays, record the units they are transformed into (they must be in the same units when viewed as ndarrays, or else we lose unitful protections), and then re-cast after passing between processors.
3. Hack around it somehow.
I think that the first option is likely the best. We can redefine the derived quantity system to behave more like the new field system, and we can also take the opportunity to make it more explicit. Many problems we've had in the past have been related to `n_ret` being unclear and whatnot.
I will first attempt to do 2, then 3, then 1. I think 1 is where we will end up, however.
The derived quantities currently in the code:
* TotalMass
* CenterOfMass
* WeightedAverageQuantity
* WeightedVariance
* BulkVelocity
* AngularMomentumVector
* StarAngularMomentumVector
* ParticleAngularMomentumVector
* BaryonSpinParameter
* ParticleSpinParameter
* IsBound
* Extrema
* Action
* MaxLocation
* MinLocation
* TotalQuantity
* ParticleDensityCenter
* HalfMass
I think several of these should be removed or rewritten to merge them. For instance, we can join the various spin parameter computations, potentially the variance and averaging, and possibly simplify the TotalMass computation as opposed to the TotalQuantity. We may be able to alias WeightedAverage and CenterOfMass as well.
Responsible: MatthewTurk
Nautilus is back up. I just logged in. I was the only person on the node. I
have no idea if this means it's back or if they just stopped booting people
off...
j
---------- Forwarded message ----------
From: Christine Slocum <cslocum(a)stsci.edu>
Date: Wed, Dec 4, 2013 at 1:13 PM
Subject: [AstroPy] Ureka 1.0 is now available
To: "AstroPy(a)scipy.org" <AstroPy(a)scipy.org>
STScI and Gemini are announcing the release of Ureka 1.0.
Improvements (with respect to beta 6):
- test for completeness of OS library dependencies
- many installation improvements
- expanded documentation
- support for Ubuntu 13.10, Fedora 19, and OS X Mavericks (10.9)
- menu shortcuts on Linux
- optional support for MySQL and PostgreSQL
- bundled Fortran run-time library is always used to avoid conflict with
OS version(s)
Updated packages:
- Gemini IRAF 1.12
- virtualenv (1.10.1)
- SciPy (0.13.0)
- wheel (0.22.0)
- scikit-image (0.9.3)
- Pillow (2.2.1)
- CFITSIO (3350)
- scikit-learn (0.14.1)
- Tornado (3.1.1)
- setuptools (1.3)
- NumPy (1.8.0)
- stsci.distutils (0.3.2)
- astropy (0.3)
- d2to1 (0.2.11)
- PyParsing (2.0.1)
New packages:
- PyKE (2.4.0)
- Six (1.4.1)
- PyTZ (2013.8)
- kapteyn (2.2.1b17)
- BeautifulSoup (3.2.1)
- mechanize (0.2.5)
- jsonschema (2.1.0)
- numexpr (2.2.2)
- Dateutil (2.2)
- mock (1.0.1)
Ureka is a binary packaging installer for common astronomical software
(primarily for the UV/Optical/IR community). The goals of the Ureka
installer are to:
1) minimize the number of actions needed to install all the different
software components. We are seeking a "one button install" (it's not
quite one button, but not far from it).
2) permit installation without requiring system privileges.
3) make installs as problem-free as possible for the great majority of
users.
4) allow users to install their own software (particularly Python-based)
within this framework, or update versions of software within the framework.
5) permit different Ureka installations to coexist and to easily switch
between them.
6) enable installing different versions of the same software package under
a particular Ureka installation.
7) support Macs and most popular Linux variants.
Ureka does not use LD_LIBRARY_PATH (or its Mac equivalent), nor require
PYTHONPATH, minimizing the possibility of affecting existing software
after installation or use. Should conflicts nevertheless arise, Ureka can
easily be disabled temporarily or enabled only in specific terminal
windows.
Keep in mind that no installation system is completely foolproof (that's
very nearly impossible to achieve). In particular, when users update or
add software to the Ureka framework, they increase the risk of breaking
something, but we feel that is an option that users should have as long as
they understand the possible risks.
This release includes IRAF 2.16 and associated packages for IRAF, DS9, and
a fairly full suite of Python scientific software packages (e.g., numpy,
scipy, and matplotlib). The full listing of included software can be
found at this link: http://ssb.stsci.edu/ureka/1.0/docs/components.html
Ureka can be downloaded from: http://ssb.stsci.edu/ureka/ (choose "1.0").
Installation and usage instructions can be found on the same web page.
Please send questions or feedback to help(a)stsci.edu.
_______________________________________________
AstroPy mailing list
AstroPy(a)scipy.org
http://mail.scipy.org/mailman/listinfo/astropy
Hi all,
I'm working with particle filters for the first time and I'm unclear on the
role of the filtered_type keyword. For example, in the following script:
http://paste.yt-project.org/show/4099/
I define filters that should catch every single particle in an Enzo-3.0
dataset. My understanding of the filtered_type keyword is that is
preselected which type of particles to pass through the filter.
In this dataset, I have the following particles:
Type Number
DarkMatter 32768
CenOstriker 10543
Total 43311
So, my understanding is that
print ad['finest_all', 'particle_mass'].size
should print 43311 and
print ad['finest_all', 'particle_mass'].size
should print 32768.
However, they both print 43311. Is this a bug or am I not understanding
what filtered_type is supposed to do?
Britton
Hi all,
Nathan and I sprinted on a few remaining things for the unit refactor,
to the point that I would like to see it get tried out and tested in
production. Casey and Nathan have done some really cool stuff with
the units work, and I think it will be awesome.
I've issued a PR against the tip of the yt-3.0 branch. Nathan and I
will likely be updating this throughout the day.
https://bitbucket.org/yt_analysis/yt/pull-request/662/unit-refactor
What works:
* All of the non-VR *unit* tests
* Plotting of things
* Units on arrays!
Each frontend needs to be manually converted, as fields are now
completely redone -- and much nicer. We have redone:
* Enzo
* FLASH
* RAMSES
* Boxlib (mostly)
* Stream
However, before we got too far down the road, I wanted to verify that
things are heading in a direction the developers are happy with.
Specifically, things like the way fields are created (check out
yt/fields/*.py and the frontends/*/fields.py for converted frontends)
and the new UI for dealing with things.
This *needs* to land in some form or another before the 3.0 release,
and I'd like to start getting testing done now so that we can land the
other big item, the rebranding.
Unit refactor can't be accepted until:
* We get some people to use it, say what's wrong.
* The other frontends get converted.
* The VR interface is fixed.
* Plots have a unit-ful interface, which is mostly done.
So, I'd like to ask that y'all try it out by pulling the bookmark
unitrefactor from my fork ( MatthewTurk/yt ). On Thursday during the
dev hangout at 2PM EST, this will be the biggest part of the agenda --
actually trying it out.
-Matt
Hi all,
Because of a regression in phase plots in yt 2.6, we've just rolled
out a point release of yt 2.6.1. If you have been using phase plots,
or want to use them in the future, we recommend upgrading. If you're
using the "stable" branch already from source, just run "yt update".
-Matt
Hi all,
Nathan and I are working on some unit things. One that came up is
that it looked like the conversion to mpc was wrong.
Turns out, it's not. But what it's set up as is "mpc" for
"milliparsecs" and "Mpc" for "Megaparsecs." We'd like to get rid of
any usage of milliparsecs, since I think the number of people who want
to use that (globally) are probably between 0 and 0, and just make Mpc
and mpc the same thing.
If anybody objects, write back. Silence will be taken as tacit agreement!
-Matt
