Hello All,

I am the one who was asking all of the questions about he ART frontend in the yt-users chat a few weeks ago. I have been probing around the front end files and believe I fixed the file loading problem I outlined here.

https://mail.python.org/mm3/archives/list/yt-users@python.org/thread/RV5HY2OUEDOBP64H75GY2Z6VDDJUK62Z/

I have a forked version here.

https://github.com/sflarkin/yt

There are two changes. One is a simple change of a / to a // to fix a num_pages call.

https://github.com/yt-project/yt/commit/f233658b153e0ee7b31b31f121488a4257cf13c7

The second is the import one. It appears that in the _find_files function of the art data structures folder had two searches for matching files. The first one works as intended, finding the files with the correct number and data extension. The second just looks for the same data extension, and appears it was just finding all of the files, and then returning the last one found, which from my test was always the file number 170 for the PMcrs files. By removing this check, all appears to work, as shown here on this run log.

P001 yt : [INFO     ] 2018-08-06 14:44:40,241 discovered particle_header:/nobackupp2/sflarkin/VELA07test/PMcrda0.051.DAT
P001 yt : [INFO     ] 2018-08-06 14:44:40,241 discovered particle_data:/nobackupp2/sflarkin/VELA07test/PMcrs0a0.051.DAT
P001 yt : [INFO     ] 2018-08-06 14:44:40,242 discovered particle_stars:/nobackupp2/sflarkin/VELA07test/stars_a0.051.dat
P001 yt : [INFO     ] 2018-08-06 14:44:40,263 Using root level of 14
P001 yt : [INFO     ] 2018-08-06 14:44:40,265 Discovered 7 species of particles
P001 yt : [INFO     ] 2018-08-06 14:44:40,265 Particle populations:  34930688   4943872    855040    139392     21776   2088309         2
P001 yt : [INFO     ] 2018-08-06 14:44:40,483 Max level is 08
P001 yt : [INFO     ] 2018-08-06 14:44:40,495 Parameters: current_time              = 0.2068722930617056 Gyr
P001 yt : [INFO     ] 2018-08-06 14:44:40,496 Parameters: domain_dimensions         = [128 128 128]
P001 yt : [INFO     ] 2018-08-06 14:44:40,496 Parameters: domain_left_edge          = [0. 0. 0.]
P001 yt : [INFO     ] 2018-08-06 14:44:40,496 Parameters: domain_right_edge         = [1. 1. 1.]
P001 yt : [INFO     ] 2018-08-06 14:44:40,496 Parameters: cosmological_simulation   = True
P001 yt : [INFO     ] 2018-08-06 14:44:40,496 Parameters: current_redshift          = 18.578137813635497
P001 yt : [INFO     ] 2018-08-06 14:44:40,496 Parameters: omega_lambda              = 0.7300000190734863
P001 yt : [INFO     ] 2018-08-06 14:44:40,496 Parameters: omega_matter              = 0.27000001072883606
P001 yt : [INFO     ] 2018-08-06 14:44:40,496 Parameters: hubble_constant           = 0.699999988079071
[   141s] Sending projection requests...
[   143s] Transferring particles to writers...
[   146s] Analyzing for FoF groups...
[   148s] Transferring boundary particles between writers...
[   148s] Linking boundary particles...
[   149s] Analyzing for halos / subhalos...
[   150s] Loading merger tree information...
[   151s] Constructing merger tree...
[   152s] [Success] Done with snapshot 3.
[   152s] Reading 1 blocks for snapshot 4...
P001 yt : [INFO     ] 2018-08-06 14:45:21,705 discovered particle_header:/nobackupp2/sflarkin/VELA07test/PMcrda0.060.DAT
P001 yt : [INFO     ] 2018-08-06 14:45:21,705 discovered particle_data:/nobackupp2/sflarkin/VELA07test/PMcrs0a0.060.DAT
P001 yt : [INFO     ] 2018-08-06 14:45:21,705 discovered particle_stars:/nobackupp2/sflarkin/VELA07test/stars_a0.060.dat
P001 yt : [INFO     ] 2018-08-06 14:45:21,728 Using root level of 14
P001 yt : [INFO     ] 2018-08-06 14:45:21,755 Discovered 7 species of particles
P001 yt : [INFO     ] 2018-08-06 14:45:21,755 Particle populations:  34930688   4943872    855040    139392     21776   2088309       149
P001 yt : [INFO     ] 2018-08-06 14:45:21,948 Max level is 09
P001 yt : [INFO     ] 2018-08-06 14:45:21,961 Parameters: current_time              = 0.2648085269122017 Gyr
P001 yt : [INFO     ] 2018-08-06 14:45:21,961 Parameters: domain_dimensions         = [128 128 128]
P001 yt : [INFO     ] 2018-08-06 14:45:21,961 Parameters: domain_left_edge          = [0. 0. 0.]
P001 yt : [INFO     ] 2018-08-06 14:45:21,961 Parameters: domain_right_edge         = [1. 1. 1.]
P001 yt : [INFO     ] 2018-08-06 14:45:21,961 Parameters: cosmological_simulation   = True
P001 yt : [INFO     ] 2018-08-06 14:45:21,961 Parameters: current_redshift          = 15.606548426007368
P001 yt : [INFO     ] 2018-08-06 14:45:21,961 Parameters: omega_lambda              = 0.7300000190734863
P001 yt : [INFO     ] 2018-08-06 14:45:21,962 Parameters: omega_matter              = 0.27000001072883606
P001 yt : [INFO     ] 2018-08-06 14:45:21,962 Parameters: hubble_constant           = 0.699999988079071

This shows two snapshots being loaded in parallel, both finding all of the correct files for the stars and PM files.

While this appears to fix the problem, I am not sure if that catch is there for some edge case I cannot think of. Is there one?
_______________________________________________
yt-dev mailing list -- yt-dev@python.org
To unsubscribe send an email to yt-dev-leave@python.org