yt-dev August 2018

yt-dev@python.org

12 participants
9 discussions

Nominate Ricarda Beckmann as a yt project member
by Nathan Goldbaum 30 Aug '18

30 Aug '18

Hi all, Ricarda has been sending in patches and bug reports off and on for a few years now. This summer she's sent in some substantial improvements for the RAMSES frontend and for the derived quantitiy system. To recognize her contributions and encourage more in the future I'd like to nominate Ricarda Bechmann as a yt project member. To that end, per YTEP-1776 I'm going to need three existing yt project members to +1 the nomination. When that happens I'll push commits to the website repo and the YTEP repo adding Ricarda to the list of members and give her a commit bit in github. -Nathan

4 4

yt 3.5 sprint?
by Nathan Goldbaum 30 Aug '18

30 Aug '18

Hi all, I think I'd like to organize a 2-day sprint to work on fixing bugs that are currently marked as blockers for yt 3.5. I think having a dedicated time to do this will help us be more organized and also help get more eyes on the list of open bugs. I was thinking Thursday and Friday September 13th and 14th. Would anyone who would otherwise be interested in participating in such a sprint already have plans those two days? I can reserve a room here at NCSA if anyone wants to come down to Champaign-Urbana and work in person, but I think we should plan on people participating remotely and just have a google hangout session running or slack channel open during the sprint. If you're interested and can make it, just reply to this e-mail with a +1. -Nathan

7 7

Advice on geo projection PR to master vs. yt-4.0
by Madicken Munk 24 Aug '18

24 Aug '18

Hi all, I've been working on building in support for cartopy projections for geographic data in yt. My PR for this work is here: https://github.com/yt-project/yt/pull/1966 I've submitted the PR to be merged into master, but I wanted to get some advice from you all if this should be changed to yt-4.0. It adds in some newer functionality that's not directly targeted for the yt 3.5 release. Another option could be to submit it to master but merge after the 3.5 release. Do you all have any advice or preference on where this should go or what the process should be here? Best, Madicken

3 2

python3 testing on the yt-4.0 branch
by Nathan Goldbaum 22 Aug '18

22 Aug '18

Hi all, I just issued a PR that disables testing on python2 on the yt-4.0 branch: https://github.com/yt-project/yt/pull/1982 This is motivated by a couple incidences of people bending over backwards to support python2. We already agreed that the yt 4.0 release won't support python2, I just haven't gotten around to disabling testing yet. Originally I was thinking of keeping it running to help make backporting easier, but I think that's overkill. As a compromise in my PR I left the linting check on python2.7 running, that will make it so that we don't merge PRs that include python2-specific syntax (e.g. the @ operator or f-strings). I'll eventually remove that once we no longer maintain the 3.x branch in maintenance mode. At that point we can also remove six and all the future imports. If anyone has an issue with this, now is the time to raise it. I think this will be uncontroversial but I wanted to bring it up here just in case anyone has problems with the idea. -Nathan

1 0

Per-field configuration
by Corentin CADIOU 22 Aug '18

22 Aug '18

Hello, As of today, there is no way to configure yt plots per-field. One is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github (https://github.com/yt-project/yt/pull/1931) and opened an issue (https://github.com/yt-project/yt/issues/1888) to discuss the feature. Nathan and I discussed a bit about how this should be implemented in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ... Another suggestion is to add each fields' configuration as a new section in the ini file. For example [config.field_type.field_name] cmap = viridis log = False This is what is implemented in the PR, but maybe some of you have more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format. We should also discuss what can be configured here. There is already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user. So here are the questions. Do you think per-field configuration should be part of yt? If so what format should it use? And what are your thoughts on what should and shouldn't be configurable? Corentin -- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France phone: +33.6.43.18.66.83

7 12

Thank you everyone!
by Abhishek Singh 15 Aug '18

15 Aug '18

Hi all, I would like to thank the entire yt community as the program Google Summer of Code'18 is approaching its official end. During this summer, I have been working with yt to improve its testing infrastructure, increasing code coverage and optimizing the test suite. You can find more details on this in the following links: - Summary of my work <https://gist.github.com/git-abhishek/3bcccb4de59afcf53f499785ffa981d5> - Blog posts <https://git-abhishek.github.io/blog/> I would like to thank my mentors: Nathan <https://github.com/ngoldbaum>, Kacper <https://github.com/Xarthisius> and Colin <https://github.com/colinmarc> who helped me in my endeavor. Nathan has as always been quick to respond to my queries and clearing my doubts on yt internals. Whatever I say is less about him! Colin has been toiling with me to achieve robust and elegant programming solutions. Kacper helped me with valuable advise on Jenkins and answer testing infrastructure. I am glad that I have learned a lot from all of them and will be carrying these valuable lessons in my future ventures. The support and guidance shown by yt developers have been commendable. I not only got the feedback from the "official" mentors but from a lot of people. I thank Matthew <https://github.com/MatthewTurk>, Andrew <https://github.com/atmyers>, Corentin <https://github.com/cphyc>, Britton <https://github.com/brittonsmith>, and others for their valuable feedback on my work. Thank you everyone for making this a memorable summer for me! Best, Abhishek Singh Email: 1. abhisheksing(a)umass.edu 2. in.abhisheksingh(a)gmail.com Github: https://github.com/git-abhishek LinkedIn: https://www.linkedin.com/in/asingh690/

3 2

Art frontend fixes
by sflarkin＠ucsc.edu 08 Aug '18

08 Aug '18

Hello All, I am the one who was making a fuss about the yt ART frontend a few weeks ago. I think I have found the root of the problems I was having. I have a fork here with my changes, outlined below. https://github.com/sflarkin/yt The first change is a simple change from / to // to make a num_pages int check not fail. https://github.com/yt-project/yt/commit/f233658b153e0ee7b31b31f121488a4257c… The second is the more important one. It appears that in the ART data structures file _find_files function, there were two check to find matching datafiles. One compared both the extension number and data file type, and found the correct file. The other second check just compared the data file type, meaning it found all files with the correct extension, like .DAT. This second check would then just dump the last one, which was incorrect. By removing the check, it appears to have fixed the problem, as shown by my run log below. P001 yt : [INFO ] 2018-08-06 14:44:40,241 discovered particle_header:/nobackupp2/sflarkin/VELA07test/PMcrda0.051.DAT P001 yt : [INFO ] 2018-08-06 14:44:40,241 discovered particle_data:/nobackupp2/sflarkin/VELA07test/PMcrs0a0.051.DAT P001 yt : [INFO ] 2018-08-06 14:44:40,242 discovered particle_stars:/nobackupp2/sflarkin/VELA07test/stars_a0.051.dat P001 yt : [INFO ] 2018-08-06 14:44:40,263 Using root level of 14 P001 yt : [INFO ] 2018-08-06 14:44:40,265 Discovered 7 species of particles P001 yt : [INFO ] 2018-08-06 14:44:40,265 Particle populations: 34930688 4943872 855040 139392 21776 2088309 2 P001 yt : [INFO ] 2018-08-06 14:44:40,483 Max level is 08 P001 yt : [INFO ] 2018-08-06 14:44:40,495 Parameters: current_time = 0.2068722930617056 Gyr P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: domain_dimensions = [128 128 128] P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: domain_left_edge = [0. 0. 0.] P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: domain_right_edge = [1. 1. 1.] P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: cosmological_simulation = True P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: current_redshift = 18.578137813635497 P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: omega_lambda = 0.7300000190734863 P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: omega_matter = 0.27000001072883606 P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: hubble_constant = 0.699999988079071 [ 141s] Sending projection requests... [ 143s] Transferring particles to writers... [ 146s] Analyzing for FoF groups... [ 148s] Transferring boundary particles between writers... [ 148s] Linking boundary particles... [ 149s] Analyzing for halos / subhalos... [ 150s] Loading merger tree information... [ 151s] Constructing merger tree... [ 152s] [Success] Done with snapshot 3. [ 152s] Reading 1 blocks for snapshot 4... P001 yt : [INFO ] 2018-08-06 14:45:21,705 discovered particle_header:/nobackupp2/sflarkin/VELA07test/PMcrda0.060.DAT P001 yt : [INFO ] 2018-08-06 14:45:21,705 discovered particle_data:/nobackupp2/sflarkin/VELA07test/PMcrs0a0.060.DAT P001 yt : [INFO ] 2018-08-06 14:45:21,705 discovered particle_stars:/nobackupp2/sflarkin/VELA07test/stars_a0.060.dat P001 yt : [INFO ] 2018-08-06 14:45:21,728 Using root level of 14 P001 yt : [INFO ] 2018-08-06 14:45:21,755 Discovered 7 species of particles P001 yt : [INFO ] 2018-08-06 14:45:21,755 Particle populations: 34930688 4943872 855040 139392 21776 2088309 149 P001 yt : [INFO ] 2018-08-06 14:45:21,948 Max level is 09 P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: current_time = 0.2648085269122017 Gyr P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: domain_dimensions = [128 128 128] P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: domain_left_edge = [0. 0. 0.] P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: domain_right_edge = [1. 1. 1.] P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: cosmological_simulation = True P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: current_redshift = 15.606548426007368 P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: omega_lambda = 0.7300000190734863 P001 yt : [INFO ] 2018-08-06 14:45:21,962 Parameters: omega_matter = 0.27000001072883606 P001 yt : [INFO ] 2018-08-06 14:45:21,962 Parameters: hubble_constant = 0.699999988079071 As can be seen, this shows that two grids were loaded with the correct files. I am not sure if there is some edge case that the second check was meant to catch, as I cannot think of any. Are there any I am not aware of?

1 1

Art frontend fixes
by sflarkin＠ucsc.edu 08 Aug '18

08 Aug '18

Hello All, I am the one who was asking all of the questions about he ART frontend in the yt-users chat a few weeks ago. I have been probing around the front end files and believe I fixed the file loading problem I outlined here. https://mail.python.org/mm3/archives/list/yt-users@python.org/thread/RV5HY2… I have a forked version here. https://github.com/sflarkin/yt There are two changes. One is a simple change of a / to a // to fix a num_pages call. https://github.com/yt-project/yt/commit/f233658b153e0ee7b31b31f121488a4257c… The second is the import one. It appears that in the _find_files function of the art data structures folder had two searches for matching files. The first one works as intended, finding the files with the correct number and data extension. The second just looks for the same data extension, and appears it was just finding all of the files, and then returning the last one found, which from my test was always the file number 170 for the PMcrs files. By removing this check, all appears to work, as shown here on this run log. P001 yt : [INFO ] 2018-08-06 14:44:40,241 discovered particle_header:/nobackupp2/sflarkin/VELA07test/PMcrda0.051.DAT P001 yt : [INFO ] 2018-08-06 14:44:40,241 discovered particle_data:/nobackupp2/sflarkin/VELA07test/PMcrs0a0.051.DAT P001 yt : [INFO ] 2018-08-06 14:44:40,242 discovered particle_stars:/nobackupp2/sflarkin/VELA07test/stars_a0.051.dat P001 yt : [INFO ] 2018-08-06 14:44:40,263 Using root level of 14 P001 yt : [INFO ] 2018-08-06 14:44:40,265 Discovered 7 species of particles P001 yt : [INFO ] 2018-08-06 14:44:40,265 Particle populations: 34930688 4943872 855040 139392 21776 2088309 2 P001 yt : [INFO ] 2018-08-06 14:44:40,483 Max level is 08 P001 yt : [INFO ] 2018-08-06 14:44:40,495 Parameters: current_time = 0.2068722930617056 Gyr P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: domain_dimensions = [128 128 128] P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: domain_left_edge = [0. 0. 0.] P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: domain_right_edge = [1. 1. 1.] P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: cosmological_simulation = True P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: current_redshift = 18.578137813635497 P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: omega_lambda = 0.7300000190734863 P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: omega_matter = 0.27000001072883606 P001 yt : [INFO ] 2018-08-06 14:44:40,496 Parameters: hubble_constant = 0.699999988079071 [ 141s] Sending projection requests... [ 143s] Transferring particles to writers... [ 146s] Analyzing for FoF groups... [ 148s] Transferring boundary particles between writers... [ 148s] Linking boundary particles... [ 149s] Analyzing for halos / subhalos... [ 150s] Loading merger tree information... [ 151s] Constructing merger tree... [ 152s] [Success] Done with snapshot 3. [ 152s] Reading 1 blocks for snapshot 4... P001 yt : [INFO ] 2018-08-06 14:45:21,705 discovered particle_header:/nobackupp2/sflarkin/VELA07test/PMcrda0.060.DAT P001 yt : [INFO ] 2018-08-06 14:45:21,705 discovered particle_data:/nobackupp2/sflarkin/VELA07test/PMcrs0a0.060.DAT P001 yt : [INFO ] 2018-08-06 14:45:21,705 discovered particle_stars:/nobackupp2/sflarkin/VELA07test/stars_a0.060.dat P001 yt : [INFO ] 2018-08-06 14:45:21,728 Using root level of 14 P001 yt : [INFO ] 2018-08-06 14:45:21,755 Discovered 7 species of particles P001 yt : [INFO ] 2018-08-06 14:45:21,755 Particle populations: 34930688 4943872 855040 139392 21776 2088309 149 P001 yt : [INFO ] 2018-08-06 14:45:21,948 Max level is 09 P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: current_time = 0.2648085269122017 Gyr P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: domain_dimensions = [128 128 128] P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: domain_left_edge = [0. 0. 0.] P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: domain_right_edge = [1. 1. 1.] P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: cosmological_simulation = True P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: current_redshift = 15.606548426007368 P001 yt : [INFO ] 2018-08-06 14:45:21,961 Parameters: omega_lambda = 0.7300000190734863 P001 yt : [INFO ] 2018-08-06 14:45:21,962 Parameters: omega_matter = 0.27000001072883606 P001 yt : [INFO ] 2018-08-06 14:45:21,962 Parameters: hubble_constant = 0.699999988079071 This shows two snapshots being loaded in parallel, both finding all of the correct files for the stars and PM files. While this appears to fix the problem, I am not sure if that catch is there for some edge case I cannot think of. Is there one?

2 1

yt_astro_analysis and a new home for level_sets
by Britton Smith 06 Aug '18

06 Aug '18

Hi everyone, Today we accepted a pull request backporting all changes from yt's analysis_modules to their future home in the yt_astro_analysis <https://github.com/yt-project/yt_astro_analysis> package. This makes yt_astro_analysis ready for a stable release from a codebase perspective, although there are still a few more things to do. The current plan (unless their are other opinions) is to announce a stable release of yt_astro_analysis at the same time as the next release of yt. This also makes sense since yt_astro_analysis currently requires the development version of yt. This leaves the level_sets module (countouring and clump finding) as the last non-deprecated package in the analysis_modules directory. A discussion from a long time ago concluded with the idea that level_sets be moved to somewhere in yt/data_objects. What was not decided is where in data_objects, whether it would take on a new name, and if the API should change at all. On Slack, Nathan suggested having something like ds.clump_tree. Does anyone else have any thoughts? Britton

2 2

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

yt-dev August 2018