Are you sure that the files are the same on your laptop and on Pleiades? Are you sure that you're running the same version of yt on your laptop and pleiades?A way to check the former would be to make a tarball containing the dataset and then find the tarball's SHA hash on your laptop and on Pleiades, e.g.:# on linux$ sha256sum my_data.tar# on MacOS$ shasum -a 256 my_data.tarIf those commands don't print the same answer on your laptop and on Pleiades, that indicates that the files are not identical and perhaps there has been some corruption. That might be the source of the issue.You can check to make sure the yt versions are the same on your laptop and on Pleiades by running "yt version" on the bash command line.What's happening in the error you're running that yt is trying to parse the octree hierarchy in your data by reading some bytes from the data file. It's dying because it's getting back a piece of data that's not the shape it's expecting based on what we know about the ART data format. It's very odd that this works on your laptop using the exact same data file, because yt is making the same assumptions about the structure of the data file on Pleiades and your laptop.Finally, if none of the diagnostic suggestions I made above help, it would help to make further progress on this issue if you can share the data file that's triggering the problem so that one of us can reproduce the issue and debug locally, although I totally understand if there are possible concerns about sharing data.-NathanOn Tue, Jul 25, 2017 at 3:05 PM, Clayton Strawn <cjstrawn@ucsc.edu> wrote:______________________________Hello,
I have a request for help.
I am trying to run some analysis code remotely through ssh in yt on NASA Pleiades. I have been able to install yt and trident and am running in a virtualenv, but something is having a very deeply buried error that I don't understand and fatally kills any task I send to Pleiades.The same code works fine on my home laptop with the same input files, so I assume the problem is something to do with the yt distribution on Pleiades, but I used "pip install -U yt" so it should be up-to-date. The error code comes from the fortran_utils file, and it says "unpack requires a string argument of length 4"
(myenv) cstrawn@pfe27:~/quasarlines/quasarlines> python quasar_scan.py VELA_v2_08 10MpcBox_csf512_a0.330.d "[H I,H II,O IV,O V,O VI,Si II,Si III,Si IV,C I,C II,C III,C IV,Mg II,Mg III,Mg IV]" yt : [INFO ] 2017-07-25 12:11:26,454 discovered particle_data:/home1/cstrawn/quasarlines/quasarlines/PMcrs0a yt : [INFO ] 2017-07-25 12:11:26,454 discovered particle_header:/home1/cstrawn0.330.DAT /quasarlines/quasarlines/ yt : [INFO ] 2017-07-25 12:11:26,462 Using root level of 14PMcrda0.330.DAT
yt : [INFO ] 2017-07-25 12:11:26,463 Discovered 7 species of particles
yt : [INFO ] 2017-07-25 12:11:26,463 Particle populations: 28016640 4263936 752640 123904 19552 2089406 85309
yt : [INFO ] 2017-07-25 12:11:26,658 Max level is 12
yt : [INFO ] 2017-07-25 12:11:26,682 Parameters: current_time = 3.350407555 Gyr
yt : [INFO ] 2017-07-25 12:11:26,682 Parameters: domain_dimensions = [128 128 128]
yt : [INFO ] 2017-07-25 12:11:26,683 Parameters: domain_left_edge = [ 0. 0. 0.]
yt : [INFO ] 2017-07-25 12:11:26,683 Parameters: domain_right_edge = [ 1. 1. 1.]
yt : [INFO ] 2017-07-25 12:11:26,683 Parameters: cosmological_simulation = True
yt : [INFO ] 2017-07-25 12:11:26,683 Parameters: current_redshift = 2.02694081565
yt : [INFO ] 2017-07-25 12:11:26,683 Parameters: omega_lambda = 0.730000019073
yt : [INFO ] 2017-07-25 12:11:26,684 Parameters: omega_matter = 0.270000010729
yt : [INFO ] 2017-07-25 12:11:26,684 Parameters: hubble_constant = 0.699999988079
18426 LOSs to scan.
Is this an appropriate number? (0 for no)
1
1/18426
Traceback (most recent call last):
File "quasar_scan.py", line 375, in <module>
q.get_coldens(save = 10)
File "quasar_scan.py", line 148, in get_coldens
ftype='gas')
File "/u/cstrawn/myenv/lib/python2.7/site-packages/trident/ray_ge particle_type = _determine_particle_type_from_nerator.py", line 239, in make_simple_ray ftype(ds, ftype) File "/u/cstrawn/myenv/lib/python2.7/site-packages/trident/ray_ge field_list_arr = np.asarray(ds.derived_field_linerator.py", line 590, in _determine_particle_type_from_ ftype st) File "/u/cstrawn/myenv/lib/python2.7/site-packages/yt/data_object self.indexs/static_output.py", line 164, in ireq
File "/u/cstrawn/myenv/lib/python2.7/site-packages/yt/data_object self, dataset_type=self.dataset_types/static_output.py", line 419, in index ) File "/u/cstrawn/myenv/lib/python2.7/site-packages/yt/frontends/a super(ARTIndex, self).__init__(ds, dataset_type)rt/data_structures.py", line 65, in __init__
File "/u/cstrawn/myenv/lib/python2.7/site-packages/yt/geometry/ge self._setup_geometry()ometry_handler.py", line 50, in __init__
File "/u/cstrawn/myenv/lib/python2.7/site-packages/yt/geometry/oc self._initialize_oct_handler()t_geometry_handler.py", line 25, in _setup_geometry
File "/u/cstrawn/myenv/lib/python2.7/site-packages/yt/frontends/a self.domains]rt/data_structures.py", line 91, in _initialize_oct_handler
File "/u/cstrawn/myenv/lib/python2.7/site-packages/yt/frontends/a self.level_offsetsrt/data_structures.py", line 761, in level_count
File "/u/cstrawn/myenv/lib/python2.7/site-packages/yt/frontends/a self.ds.min_level, self.ds.max_level)rt/data_structures.py", line 782, in level_offsets
File "/u/cstrawn/myenv/lib/python2.7/site-packages/yt/frontends/a ns = fpu.peek_record_size(f, endian='>')rt/data_structures.py", line 820, in _count_art_octs
File "/u/cstrawn/myenv/lib/python2.7/site-packages/yt/utilities/f s = struct.unpack('>i', f.read(struct.calcsize('>i')))ortran_utils.py", line 250, in peek_record_size
struct.error: unpack requires a string argument of length 4Please let me know if this is a known problem or if you have any idea what I should do to avoid running into it. I don't really understand fortran enough to know what is even supposed to be happening in fortran_utils.Thanks so much,Clayton Strawn_________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope. org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users- spacepope.org