yt Survey Responses Summary
by Cameron Hummels
Fellow yt users:
A few weeks ago, we asked yt users and developers to fill out a brief
survey (http://goo.gl/forms/hRNryOWTPO) to provide us with feedback on how
well yt is meeting the needs of the community and how we can improve.
Thank you to all 39 people who responded, as it has given us a great deal
to consider as we move forward with the code. We summarize the results of
the survey below, but I start with the basic takeaway from the survey:
Overall Survey Takeaway:
The survey respondents are generally pleased with yt. It meets their
needs, has a wonderful community, is relatively easy to install, and has
fair documentation. Major short-term requests were for improvements in
documentation, particularly in API docs and source code commenting, as well
as more cross-linking in the existing documentation and making sure docs
were up to date. Furthermore, people wanted more attention to making sure
existing code in 3.0 works and for resurrecting all 2.x functionality in
3.0.
The single biggest takeaway from the survey is that the transition to yt
3.0 has been fraught with difficulties. Many submitters expressed
satisfaction with the new functionality in 3.0, but the overall process of
transition through documentation, analysis modules and community response
has been found to be lacking.
Background:
There were 39 unique responses to our survey. 75% of the respondents were
grads and postdocs with a smattering of faculty, undergrads, and
researchers. Nearly everyone is at 4-year universities. 50% of the
respondents consider themselves intermediate users, 20% novice, 20%
advanced, and 10% gurus.
Installation:
90% of the respondents use the standalone install script, with a several
users employing other methods (potentially in addition to the standalone
script). 95% of the respondents rated installation as a 3 or better (out
of 5) with most people settling on a 4 out of 5. Installation comments
were aimed at having better means of installing on remote supercomputing
systems and/or making pip installs work more frequently.
Community Responsiveness:
72% of respondents gave yt 5 out of 5 and 97% were 3 or greater for
community responsiveness. Clearly this is our strong point. There was a
very wide distribution of ways in which people contacted the community for
help with the most popular means being the mailing lists, the irc channel,
mailing developers directly, and searching google. Comments in this
section were mostly positive, but one user wished for more concrete action
to be taken after bugs were reported.
Documentation:
77% of respondents gave 4 or 5 out of 5 for the overall rating of the
documentation. Individual docs components were more of a mix. Cookbooks
were ranked very highly, and quickstart notebooks and narrative docs were
generally ranked well. The two documentation components that seemed be
ranked lower (although still fair) were API docs and comments in the source
code with 15% of respondents noting that they were “mostly not useful” (ie
2 / 5). There were a lot of comments regarding ways to improve the docs,
which I bullet point here:
-
Organization of docs is difficult to parse; Hard to find what you’re
looking for.
-
Hard to know what to search for, so make command list (ie API docs) more
prominent
-
Docs not always up to date (even between 3.0 and dev)
-
Discrepancies between API docs and narrative docs
-
Examples are either too simple or too advanced--need more intermediate
examples
-
Units docs need more explanation
-
Not enough source code commenting or API docs
-
Not enough cross-linking between docs
-
More FAQ / Gotchas for common mistakes
-
API docs should include more examples and also note how to use all of
the options, not just the most common.
Functionality:
88% of respondents found yt to meet their research needs (4 or 5 out of
5). Respondents are generally using yt on a variety of datasets including
grid data, octree data, particle data, and MHD with only a handful of users
dealing with spherical or cylindrical data at present. Nearly all of the
frontends are being used by respondents, with a few exceptions: Chombo,
Moab, Nyx, Pluto, and non-astro data. Visualization remains the main use
of yt with 97% of respondents, but simple analysis received 82% and
advanced analysis received 62%. Interestingly, 31% of respondents use halo
analysis tools, with only 15% using synthetic observation analysis.
Big Picture:
51% of respondents gave yt 5 out of 5 for general satisfaction, with 28% 4
out of 5 and 15% 3 out of 5. Overall, this is pretty good but probably
biased by the fact that people filled out this survey. Comments on the
greatest strengths of yt include:
-
visualization capabilities
-
community support
-
flexibility
Comments on the biggest shortcomings of yt include:
-
documentation (see above)
-
learning to “think in yt”
-
making new functionality when there is existing broken functionality (or
missing documentation)
-
making sure 3.0 matches all functionality from 2.x
-
keeping the documentation up to date
-
making the transition from 2.x to 3.0 easier (how to update scripts)
Things to focus on in the next year:
-
documentation (almost unanimously)
-
making sure 3.0 can do all functionality from 2.x
Thank you for all of the valuable feedback. We sincerely appreciate the
constructive criticism in making for a better code and community! We will
put together a blueprint of how to address these shortcomings soon. Look
for it after the holiday break. Have a wonderful holiday!
On behalf of the yt development team,
Cameron
--
Cameron Hummels
Postdoctoral Researcher
Steward Observatory
University of Arizona
http://chummels.org
_______________________________________________
yt-dev mailing list
yt-dev(a)lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
4 years, 1 month
Issue #963: RAMSES particle depositition onto mesh does
not work with filtered datasets (yt_analysis/yt)
by Benjamin Thompson
New issue 963: RAMSES particle depositition onto mesh does not work with filtered datasets
https://bitbucket.org/yt_analysis/yt/issue/963/ramses-particle-deposititi...
Benjamin Thompson:
Hello.
With a RAMSES dataset. The deposition of particles onto a mesh works fine for unfilltered cases.
For example, should I call
```
#!python
print data["deposit,"io_density"]
```
I get answers which are sensible.
However, I wish to filter my particle data for stars/dark matter and treat the particles seperatly. If for example I do this,
```
#!python
def star_filter(pfilter,data):
filter = np.logical_and(data.particles.source["particle_age"] != 0, data.particles.source["particle_age"] != None)
return filter
add_particle_filter("stars", function=star_filter, filtered_type="all", requires=["particle_age"])
data.ds.add_particle_filter("stars")
```
Which filters the particles as expected, which also creates the fields
('deposit', 'io_cic')
('deposit', 'io_cic_velocity_x')
('deposit', 'io_cic_velocity_y')
('deposit', 'io_cic_velocity_z')
('deposit', 'io_count')
('deposit', 'io_density')
('deposit', 'io_mass')
('deposit', 'io_nn_velocity_x')
('deposit', 'io_nn_velocity_y')
('deposit', 'io_nn_velocity_z')
('deposit', 'stars_cic')
('deposit', 'stars_cic_velocity_x')
('deposit', 'stars_cic_velocity_y')
('deposit', 'stars_cic_velocity_z')
('deposit', 'stars_count')
('deposit', 'stars_density')
('deposit', 'stars_mass')
('deposit', 'stars_nn_velocity_x')
('deposit', 'stars_nn_velocity_y')
('deposit', 'stars_nn_velocity_z')
As you would expect
However, upon trying to call
print data["deposit,"stars_density"]
I get the error below
```
#!shell
Traceback (most recent call last):
File "selene_gradients.py", line 46, in <module>
print cylinder[("deposit", "stars_density")]
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 240, in __getitem__
self.get_data(f)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 673, in get_data
self._generate_fields(fields_to_generate)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 693, in _generate_fields
fd = self._generate_field(field)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 277, in _generate_field
tr = self._generate_fluid_field(field)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 295, in _generate_fluid_field
rv = self._generate_spatial_fluid(field, ngt_exception.ghost_zones)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 311, in _generate_spatial_fluid
ind += o.select(self.selector, self[field], rv, ind)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 240, in __getitem__
self.get_data(f)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 673, in get_data
self._generate_fields(fields_to_generate)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 693, in _generate_fields
fd = self._generate_field(field)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 277, in _generate_field
tr = self._generate_fluid_field(field)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 297, in _generate_fluid_field
rv = finfo(gen_obj)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/fields/derived_field.py", line 184, in __call__
dd = self._function(self, data)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/fields/particle_fields.py", line 106, in particle_density
pos = data[ptype, coord_name]
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/octree_subset.py", line 76, in __getitem__
tr = super(OctreeSubset, self).__getitem__(key)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 240, in __getitem__
self.get_data(f)
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 615, in get_data
with f.apply(self):
File "/home/bthompson1/python/yt/yt-x86_64/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/gpfs/home/bthompson1/python/yt/yt-x86_64/src/yt-hg/yt/data_objects/particle_filters.py", line 46, in apply
filter = self.function(self, dobj)
File "/gpfs/home/bthompson1/ramses_pp/ramses_pp/ben/real_selene_analysis/ramses_pp/modules/yt/YT.py", line 59, in star_filter
filter = np.logical_and(data.particles.source["particle_age"] != 0, data.particles.source["particle_age"] != None)
AttributeError: 'RAMSESDomainSubset' object has no attribute 'particles'
```
_______________________________________________
yt-dev mailing list
yt-dev(a)lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
4 years, 1 month
Issue #961: total mass quantity called on a clump breaks
subsequent data selection (yt_analysis/yt)
by Nathan Goldbaum
New issue 961: total mass quantity called on a clump breaks subsequent data selection
https://bitbucket.org/yt_analysis/yt/issue/961/total-mass-quantity-called...
Nathan Goldbaum:
The following script:
```
#!python
import numpy as np
import yt
from yt.analysis_modules.level_sets.api import *
ds = yt.load("IsolatedGalaxy/galaxy0030/galaxy0030")
data_source = ds.disk([0.5, 0.5, 0.5], [0., 0., 1.],
(8, 'kpc'), (1, 'kpc'))
field = ("gas", "density")
step = 2.0
c_min = 10**np.floor(np.log10(data_source[field]).min() )
c_max = 10**np.floor(np.log10(data_source[field]).max()+1)
master_clump = Clump(data_source, field)
master_clump.add_validator("min_cells", 20)
find_clumps(master_clump, c_min, c_max, step)
leaf_clumps = get_lowest_clumps(master_clump)
clump = leaf_clumps[3]
print clump['density']
try:
clump.quantities.total_mass()
except Exception as e:
print e
print clump['density']
```
will print out
```
#!python
[ 7.07273796e-24 6.71093874e-24 6.95491269e-24 6.82937561e-24
6.80951625e-24 6.57924229e-24 7.29300493e-24 7.07554671e-24
7.30696998e-24 7.55819187e-24 7.65949253e-24 6.95166906e-24
7.52178995e-24 7.07804069e-24 6.73055650e-24 7.13322789e-24
7.65117396e-24 7.31980022e-24 7.29300148e-24 7.06273581e-24
7.00048981e-24 7.11120382e-24 6.79811352e-24 6.67183093e-24
7.16742091e-24 7.36740952e-24 7.14456988e-24 7.34582239e-24
6.71862846e-24 7.32380039e-24 7.49710554e-24 6.95885833e-24
6.82522495e-24 6.91253709e-24 7.12972196e-24 6.76674443e-24
6.97601938e-24 7.56780126e-24 7.45219661e-24 7.33906110e-24
7.06027220e-24 6.85427194e-24 6.74850585e-24 7.30664210e-24
6.73289034e-24 7.46033156e-24 7.15491925e-24 7.32940616e-24
7.13456220e-24 6.72901856e-24] g/cm**3
Can't mix particle/discrete and fluid/mesh conditions or
quantities. Field: ('all', 'particle_mass') and Conditions specified:
obj['contours_3ed9f4d5c4c8464d85b336044c891112'] == 18
[] g/cm**3
```
h/t to @dcollins4096 for reporting this on the mailing list.
_______________________________________________
yt-dev mailing list
yt-dev(a)lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
4 years, 1 month
3.1?
by John ZuHone
Today was the day we were nominally supposed to release 3.1. I know we have Cameron's docs PR.
Anything else?
John ZuHone
Kavli Center for Astrophysics and Space Research
Massachusetts Institute of Technology
77 Massachusetts Ave., 37-582G
Cambridge, MA 02139
(w) 617-253-2354
(m) 781-708-5004
jzuhone(a)space.mit.edu
jzuhone(a)gmail.com
http://www.jzuhone.com
_______________________________________________
yt-dev mailing list
yt-dev(a)lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
4 years, 1 month
Issue #956: enable_parallelism dies due to missing rootLogger (yt_analysis/yt)
by Nathan Goldbaum
New issue 956: enable_parallelism dies due to missing rootLogger
https://bitbucket.org/yt_analysis/yt/issue/956/enable_parallelism-dies-du...
Nathan Goldbaum:
This script:
```
#!python
import yt
yt.enable_parallelism()
```
Will fail with the following (somewhat garbled) traceback when run in parallel:
```
#!python
Traceback (most recent call last):
File "test.py", line 2, in <module>
yt.enable_parallelism()
File "/Users/goldbaum/Documents/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 116, in enable_parallelism
Traceback (most recent call last):
File "test.py", line 2, in <module>
if len(yt.utilities.logger.rootLogger.handlers) > 0:
yt.enable_parallelism()
AttributeError: 'module' object has no attribute 'rootLogger'
File "/Users/goldbaum/Documents/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 116, in enable_parallelism
if len(yt.utilities.logger.rootLogger.handlers) > 0:
AttributeError: 'module' object has no attribute 'rootLogger'
Traceback (most recent call last):
File "test.py", line 2, in <module>
yt.enable_parallelism()
File "/Users/goldbaum/Documents/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 116, in enable_parallelism
if len(yt.utilities.logger.rootLogger.handlers) > 0:
AttributeError: 'module' object has no attribute 'rootLogger'
yt : [INFO ] 2014-12-13 12:04:00,092 Global parallel computation enabled: 0 / 4
Traceback (most recent call last):
File "test.py", line 2, in <module>
yt.enable_parallelism()
File "/Users/goldbaum/Documents/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py", line 116, in enable_parallelism
if len(yt.utilities.logger.rootLogger.handlers) > 0:
AttributeError: 'module' object has no attribute 'rootLogger'
```
This was caused by PR 1355. I've provisionally assigned this issue to John ZuHone because he's been thinking about loggers a lot lately. I'm also not sure what the parallel analysis interface is doing with the loggers here, so I can't fix this myself immediately. If anyone else knows how to handle this, feel free to jump in with a patch :)
Responsible: jzuhone
4 years, 1 month
){kind=link}
){kind=link}