Re: Rockstar on Blue Waters
by Nick Gnedin
Britton,
Thank you very much, this seems to work, although not without a quirk -
there may be a bug in the yt_astro_analysis installer.
I am using the version of yt that you so kindly created for me a while
ago - the one that fixes the HOP parallel scaling bug; it also includes
some of my modifications to the artio frontend. When I installed
yt_astro_analysis per online instructions, the installer kicked out my
version of yt and activated another one (not sure if it was the system
wide one or it installed its own). I had to juggle with installing and
uninstalling yt and yt_astro_analysis until I figured out the order that
worked: I installed yt_astro_analysis first and then reinstalled my
custom yt on top of it. Then the rockstar error disappeared.
n
On 1/25/19 1:14 PM, Britton Smith wrote:
> Hi Nick,
>
> I think you're running into a bug related to running rockstar with
> Python3. This has been fixed as of the last major yt release and the
> release of the yt_astro_analysis package. I just tried a small rockstar
> test on Blue Waters using yt_astro_analysis and it worked for me. If you
> want to go that route, I recommend following the install instructions here:
> https://yt-astro-analysis.readthedocs.io/en/latest/
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__yt-2Dastro-2Danalysi...>
> and using the --user flag when you install with pip.
>
> Britton
>
> On Wed, Jan 23, 2019 at 8:50 AM Nick Gnedin <ngnedin(a)gmail.com
> <mailto:ngnedin@gmail.com>> wrote:
>
>
> Folks,
>
> I am trying to run Rickstar on Blue Waters, and I am getting an error
> even for a very simple setup: 1 writer and 1 reader running on a single
> node on a small test run. The same job runs just fine on a workstation.
>
> The script is simple:
> ---------------
> import yt, yt.funcs
> import glob, sys, os.path
>
> yt.enable_parallelism()
> from yt.analysis_modules.halo_finding.rockstar.api import
> RockstarHaloFinder
>
>
> root = "../B20"
> n = 1
>
> dirs = ( "D" )
>
> for dir in dirs:
> sets = glob.glob(root+"/"+dir+"/rei20c1_a0.*/*.art")
> sets.sort()
> ts = yt.DatasetSeries(sets)
> rh = RockstarHaloFinder(ts, num_readers=n, num_writers=n,
> outbase=root+"/"+dir+"/rs", dm_only=True, particle_type='N-BODY')
> rh.run()
> ---------------
>
> and the batch script is trivial:
> --------
> #!/bin/csh
> #PBS -l nodes=1:ppn=32:xe
> #PBS -l walltime=30:00,vmem=64gb
> #PBS -q debug
> #PBS -N rshfc.1
> #PBS -e $PBS_JOBID.err
> #PBS -o $PBS_JOBID.out
> #PBS -m be
>
> cd $PBS_O_WORKDIR
>
> module load bwpy
> module load bwpy-mpi
>
> setenv APRUN_XFER_LIMITS 1
> limit stacksize unlimited
>
> aprun -n 3 -N 3 python3 rs-hfc.py >& std.rshfc1
> --------
>
> The error I get is not very informative:
>
> [Error] Couldn't open ^A:0! (Err: Name or service not known)
>
> and the ^A string varies in separate runs, c.f.
>
> [Error] Couldn't open 344ei20:0! (Err: Name or service not known)
>
> which may happen if a null or corrupted pointer is printed with a %s
> specification, but that's all I can guess. Is there a way to obtain
> more
> information and localize the offending code segment?
>
>
>
>
>
>
>
>
> _______________________________________________
> yt-users mailing list -- yt-users(a)python.org
> <mailto:yt-users@python.org>
> To unsubscribe send an email to yt-users-leave(a)python.org
> <mailto:yt-users-leave@python.org>
>
2 weeks, 3 days
ANN: yt dev workshop, March 4-6
by Matthew Turk
Hi everyone!
We are delighted to invite you to a yt development workshop at the
University of Illinois Urbana-Champaign, March 4-6 of 2019. Please
feel encouraged to pass this information on to others that may find it
interesting!
Information on logistics and registration may be found at the workshop website:
http://yt-project.org/workshops/spring2019/
This workshop will be focused on extending existing functionality,
building collaborations with other software projects, and ingestion of
new types and classes of data into yt.
The workshop will take place at the National Center for Supercomputing
Applications building on the north end of the UIUC campus. The NCSA
building is about a block away from the conference hotel and is next
door to a parking structure that offers metered all-day parking. There
are a number of food trucks nearby, a university-run cafeteria about 2
blocks away, and a university business district with many good lunch
restaurants about a half a mile away.
We are planning to offer funding for hotel and travel for those
requesting support. If you request funding, you will be notified of
available funds by February 16. Travel awards will be made in the
form of arranged lodging and airfare, with reservations being made by
the conference organizers.
Best regards,
The Organizing Committee
MT Campbell
Nathan Goldbaum
Madicken Munk
Matt Turk
Samantha Walkow
2 weeks, 3 days
Trying to install yt on XSEDE Comet
by Danielle Skinner
Hi all,
I have been having many issues running yt on Comet recently. I initially installed yt using conda, and quickly ran into problems with yt.enable_plugins(). I found that only some of my fields were being added, and I ended up having to manually load in the fields that were not added (although this also proved to be a struggle).
I was able to run the same script, using the same plugins file, on a different computer, but with yt version 3.4.0. So today I decided to revert to yt version 3.4.0 and try to run my script. Now, when I "import yt", I get the following error:
Traceback (most recent call last):
File "trace_halos_full.py", line 3, in <module>
import yt
File "/home/drenniks/software/anaconda3/lib/python3.6/site-packages/yt/__init__.py", line 102, in <module>
from yt.visualization.api import \
File "/home/drenniks/software/anaconda3/lib/python3.6/site-packages/yt/visualization/api.py", line 22, in <module>
from .particle_plots import \
File "/home/drenniks/software/anaconda3/lib/python3.6/site-packages/yt/visualization/particle_plots.py", line 18, in <module>
from yt.visualization.fixed_resolution import \
File "/home/drenniks/software/anaconda3/lib/python3.6/site-packages/yt/visualization/fixed_resolution.py", line 23, in <module>
from .volume_rendering.api import off_axis_projection
File "/home/drenniks/software/anaconda3/lib/python3.6/site-packages/yt/visualization/volume_rendering/api.py", line 23, in <module>
from .camera import Camera
File "/home/drenniks/software/anaconda3/lib/python3.6/site-packages/yt/visualization/volume_rendering/camera.py", line 21, in <module>
from .utils import data_source_or_all
File "/home/drenniks/software/anaconda3/lib/python3.6/site-packages/yt/visualization/volume_rendering/utils.py", line 11, in <module>
from yt.utilities.lib import mesh_traversal
File "rtcore_scene.pxd", line 61, in init yt.utilities.lib.mesh_traversal
ValueError: pyembree.rtcore_scene.EmbreeScene has the wrong size, try recompiling. Expected 32, got 24
It would be great to hear from other yt users on comet to hear how they installed yt, or if someone can help me out with this error, that would be much appreciated! Please let me know what other info you may need from me. I'll do my best to quickly respond.
3 weeks, 1 day
Rockstar on Blue Waters
by Nick Gnedin
Folks,
I am trying to run Rickstar on Blue Waters, and I am getting an error
even for a very simple setup: 1 writer and 1 reader running on a single
node on a small test run. The same job runs just fine on a workstation.
The script is simple:
---------------
import yt, yt.funcs
import glob, sys, os.path
yt.enable_parallelism()
from yt.analysis_modules.halo_finding.rockstar.api import RockstarHaloFinder
root = "../B20"
n = 1
dirs = ( "D" )
for dir in dirs:
sets = glob.glob(root+"/"+dir+"/rei20c1_a0.*/*.art")
sets.sort()
ts = yt.DatasetSeries(sets)
rh = RockstarHaloFinder(ts, num_readers=n, num_writers=n,
outbase=root+"/"+dir+"/rs", dm_only=True, particle_type='N-BODY')
rh.run()
---------------
and the batch script is trivial:
--------
#!/bin/csh
#PBS -l nodes=1:ppn=32:xe
#PBS -l walltime=30:00,vmem=64gb
#PBS -q debug
#PBS -N rshfc.1
#PBS -e $PBS_JOBID.err
#PBS -o $PBS_JOBID.out
#PBS -m be
cd $PBS_O_WORKDIR
module load bwpy
module load bwpy-mpi
setenv APRUN_XFER_LIMITS 1
limit stacksize unlimited
aprun -n 3 -N 3 python3 rs-hfc.py >& std.rshfc1
--------
The error I get is not very informative:
[Error] Couldn't open ^A:0! (Err: Name or service not known)
and the ^A string varies in separate runs, c.f.
[Error] Couldn't open 344ei20:0! (Err: Name or service not known)
which may happen if a null or corrupted pointer is printed with a %s
specification, but that's all I can guess. Is there a way to obtain more
information and localize the offending code segment?
3 weeks, 1 day
Trying to change axis on phaseplot
by Stephanie Tonnesen
Hi all,
I am making a phaseplot where the x-axis is the distance along the z-axis
and the y-axis is the velocity in the z direction. I am really, however,
plotting distance above the disk, which is at [0.5,0.5,0.5] or [80,80,80]
in kpc. Instead of my x-axis going from 70 to 120 where 80 is the disk
position, I would like it to go from -10 to 40 where 0 is the disk
position.
So, I thought I would define a new variable, zdisk, and use that, but have
run into some problems. Since this plot works beautifully when I am not
messing with the z-values, I am guessing that I have done something wrong
defining my variable. Any help is much appreciated!!
Thanks,
Stephanie
yt version:
---
Version = 3.5.dev0
Changeset = f7ac18f5bf99
---
Okay, so the code:
import sys
import numpy as np
import yt
from yt.units import kpc
def _zdisk(field,data):
return data['index','z'].in_units('kpc')-(80.,'kpc')
ds =
yt.load("/mnt/ceph/users/stonnesen/data1/BPr16vs30arcfo/DD"+loop[i]+"/BPr16vs30arcfo"+loop[i])
yt.add_field(("gas","zdisk"), function=_zdisk,units="kpc")
ad = ds.all_data()
diska = ad.cut_region(["obj['Metal_Density']/obj['density'] > 0.25"])
profile = yt.create_profile(diska, ['zdisk','velocity_z'],
'cell_mass',n_bins=80,weight_field=None)
histogram = profile.field_data['gas','cell_mass']
histogram[histogram < 1e37] = 0
plot2 = yt.PhasePlot.from_profile(profile)
plot2.set_unit('zdisk','kpc')
plot2.set_log('zdisk',False)
plot2.set_unit('velocity_z','km/s')
plot2.set_log('velocity_z',False)
plot2.set_xlim(-10,40)
plot2.set_ylim(-6e2,1.5e3)
plot2.set_zlim('cell_mass',1e37,1e42)
plot2.save('BPr16vs30arcfo'+loop[i]+'_tp25_alls_mcut_nbin80_zdisk')
______________________________
______________________________
and now the error:
yt : [INFO ] 2019-01-19 15:55:25,744 Parameters: current_time
= 3119.9999022826
yt : [INFO ] 2019-01-19 15:55:25,744 Parameters: domain_dimensions
= [256 256 256]
yt : [INFO ] 2019-01-19 15:55:25,744 Parameters: domain_left_edge
= [ 0. 0. 0.]
yt : [INFO ] 2019-01-19 15:55:25,745 Parameters: domain_right_edge
= [ 1. 1. 1.]
yt : [INFO ] 2019-01-19 15:55:25,746 Parameters:
cosmological_simulation = 0.0
/mnt/home/stonnesen/yt-conda/yt/yt/fields/local_fields.py:46: UserWarning:
Because 'sampling_type' not specified, yt will assume a cell 'sampling_type'
warnings.warn("Because 'sampling_type' not specified, yt will "
Parsing Hierarchy : 0%| | 0/4284 [00:00<?, ?it/s]
Parsing Hierarchy : 23%|██▎ | 968/4284 [00:00<00:00, 9672.70it/s]
Parsing Hierarchy : 40%|███▉ | 1702/4284 [00:00<00:00, 8971.55it/s]
Parsing Hierarchy : 62%|██████▏ | 2643/4284 [00:00<00:00, 9101.75it/s]
Parsing Hierarchy : 77%|███████▋ | 3298/4284 [00:00<00:00, 8334.29it/s]
Parsing Hierarchy : 99%|█████████▉| 4241/4284 [00:00<00:00, 8660.50it/s]
Parsing Hierarchy : 100%|██████████| 4284/4284 [00:00<00:00, 8466.80it/s]
yt : [INFO ] 2019-01-19 15:55:26,407 Gathering a field list (this may
take a moment.)
Traceback (most recent call last):
File "yt_allphaseplot_newz.py", line 40, in <module>
profile = yt.create_profile(diska, ['zdisk','velocity_z'],
'cell_mass',n_bins=80,weight_field=None)
File "/mnt/home/stonnesen/yt-conda/yt/yt/data_objects/profiles.py", line
1042, in create_profile
bin_fields = data_source._determine_fields(bin_fields)
File
"/mnt/home/stonnesen/yt-conda/yt/yt/data_objects/data_containers.py", line
1165, in _determine_fields
finfo = self.ds._get_field_info("unknown", fname)
File "/mnt/home/stonnesen/yt-conda/yt/yt/data_objects/static_output.py",
line 756, in _get_field_info
self.index
File "/mnt/home/stonnesen/yt-conda/yt/yt/data_objects/static_output.py",
line 514, in index
self.create_field_info()
File "/mnt/home/stonnesen/yt-conda/yt/yt/data_objects/static_output.py",
line 571, in create_field_info
self.field_info.load_all_plugins(self.default_fluid_type)
File "/mnt/home/stonnesen/yt-conda/yt/yt/fields/field_info_container.py",
line 276, in load_all_plugins
self.find_dependencies(loaded)
File "/mnt/home/stonnesen/yt-conda/yt/yt/fields/field_info_container.py",
line 289, in find_dependencies
deps, unavailable = self.check_derived_fields(loaded)
File "/mnt/home/stonnesen/yt-conda/yt/yt/fields/field_info_container.py",
line 362, in check_derived_fields
fd = fi.get_dependencies(ds = self.ds)
File "/mnt/home/stonnesen/yt-conda/yt/yt/fields/derived_field.py", line
216, in get_dependencies
e[self.name]
File "/mnt/home/stonnesen/yt-conda/yt/yt/fields/field_detector.py", line
108, in __missing__
vv = finfo(self)
File "/mnt/home/stonnesen/yt-conda/yt/yt/fields/derived_field.py", line
256, in __call__
dd = self._function(self, data)
File "yt_allphaseplot_newz.py", line 21, in _zdisk
return data['index','z'].in_units('kpc')-(80.,'kpc')
File "/mnt/home/stonnesen/yt-conda/yt/yt/units/yt_array.py", line 1391,
in __array_ufunc__
inps, units, ufunc, ret_class)
File "/mnt/home/stonnesen/yt-conda/yt/yt/units/yt_array.py", line 161, in
handle_preserve_units
any_nonzero = [np.any(inps[0]), np.any(inps[1])]
File
"/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/numpy/core/fromnumeric.py",
line 1970, in any
return arr.any(axis=axis, out=out, **kwargs)
File
"/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/numpy/core/_methods.py",
line 38, in _any
return umr_any(a, axis, dtype, out, keepdims)
TypeError: cannot perform reduce with flexible type
--
Dr. Stephanie Tonnesen
Associate Research Scientist
CCA, Flatiron Institute
New York, NY
stonnes(a)gmail.com
3 weeks, 6 days
Re: [Enzo 2.5] issue with running rockstar for Dark Matter only simulation
by Nathan Goldbaum
In that case you need to be running Python with 7 MPI processes.
On Fri, Jan 18, 2019 at 7:14 PM Andrew Cook <Andrew.Cook1(a)myport.ac.uk>
wrote:
> Hello,
>
> Latif, I'm not exactly sure where to find this particle_type parameter
> would it be in the parameter file? Also I have tried running it on multi
> particle sims and it still doesn't work.
>
> Nathan, thank you for asking the question. Currently the num_readers is 2
> and num_writers is 4, not sure what this does if i'm honest but if this is
> wrong what should I change it to?
>
> Cheers
> Andy
>
> On Fri, 18 Jan 2019 at 15:13, Nathan Goldbaum <nathan12343(a)gmail.com>
> wrote:
>
>> Hi Andy,
>>
>> That error gets raised when the number of MPI processes running rockstar
>> is not correct (usually when it is less than three), since rockstar needs
>> at least one reader, one writer, and one halo finding process. An error
>> message is getting logged in yt's logging messages, but that information
>> isn't getting printed in the RuntimeError itself.
>>
>> I've issued a PR to yt_astro_analysis to make the error message clearer:
>>
>> https://github.com/yt-project/yt_astro_analysis/pull/42
>>
>> By the way, the analysis modules in yt itself are now deprecated (i.e.
>> they are not being actively maintained and will be removed eventually) and
>> we encourage people to use yt_astro_analysis from now on.
>>
>> -Nathan
>>
>> On Fri, Jan 18, 2019 at 7:19 AM Andrew Cook <Andrew.Cook1(a)myport.ac.uk>
>> wrote:
>>
>>> Hello,
>>>
>>> I'm trying to run rockstar on some Dark Matter data to get merger trees
>>> but I keep getting this error:
>>>
>>> Traceback (most recent call last):
>>> File "halos_rockstar.py", line 53, in <module>
>>> rh = RockstarHaloFinder(es, num_readers= 2, num_writers= 4,
>>> particle_type= "dark_matter_nested")
>>> File
>>> "/opt/apps/yt/3.3.5b/yt-conda/src/yt-git/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
>>> line 203, in __init__
>>> self.runner = StandardRunner(num_readers, num_writers)
>>> File
>>> "/opt/apps/yt/3.3.5b/yt-conda/src/yt-git/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
>>> line 85, in __init__
>>> raise RuntimeError
>>> RuntimeError
>>>
>>> Not entirely sure what this means I've been able to run rockstar before
>>> on the same halo with baryons on. But now rockstar isn't running on
>>> anything. Hope someone can shed some light on this situation.
>>>
>>> Thank you very much
>>> Andy
>>> _______________________________________________
>>> yt-users mailing list -- yt-users(a)python.org
>>> To unsubscribe send an email to yt-users-leave(a)python.org
>>>
>>
4 weeks
[Enzo 2.5] issue with running rockstar for Dark Matter only simulation
by Andrew Cook
Hello,
I'm trying to run rockstar on some Dark Matter data to get merger trees but
I keep getting this error:
Traceback (most recent call last):
File "halos_rockstar.py", line 53, in <module>
rh = RockstarHaloFinder(es, num_readers= 2, num_writers= 4,
particle_type= "dark_matter_nested")
File
"/opt/apps/yt/3.3.5b/yt-conda/src/yt-git/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
line 203, in __init__
self.runner = StandardRunner(num_readers, num_writers)
File
"/opt/apps/yt/3.3.5b/yt-conda/src/yt-git/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
line 85, in __init__
raise RuntimeError
RuntimeError
Not entirely sure what this means I've been able to run rockstar before on
the same halo with baryons on. But now rockstar isn't running on anything.
Hope someone can shed some light on this situation.
Thank you very much
Andy
4 weeks, 1 day
overplot ~100,000 points onto a figure
by Huanqing
Hi,
I want to overlie >100,000 points onto a plot. Using annotate_marker I have no issue with a small number of points, however if I want to plot more than 10,0000 points to overlie, it takes almost an hour to produce that plot. Here is the code I use:
for ipar in range(len(xpMpc)):
slc.annotate_marker(ds.arr([xpMpc[ipar],ypMpc[ipar],zpMpc[ipar]],'Mpc').in_units('code_length'),coord_system='data',marker='.',plot_args={'color':'cyan','s':1,'alpha':0.3})
I wonder is there any way I can speed this up?
Thank you very much,
Huanqing
1 month
2D Profile Plots for User Defined Fields
by piyush.suneet＠gmail.com
Hello,
I have an hdf5 data file from FLASH which I analyze through yt. I use the yt_createprofile function to make a 2D profile plot of density and temperature as mentioned on the website:
import yt
ds = yt.load("some_hdf5_data")
dd = ds.all_data()
center = ds.arr([64.0, 64.0, 64.0], 'code_length')
rvir = ds.quan(1, "pc")
sph = ds.sphere(center, rvir)
profile = yt.create_profile(sph, ['density', 'temperature'], n_bins=[128, 128], fields=['cell_mass'], weight_field=None)
plot = yt.PhasePlot.from_profile(profile)
plot.save()
However, I would also like to make a plot of the number density. Say, I define
numdens = dd['density']/1.67e-24
How can I use 'numdens' to plot the same profile plot?
1 month
RuntimeError when Accessing Exodus II Data
by andfranklin3＠gmail.com
Hi all,
I'm trying to access Exodus II Data from a [MOOSE](https://mooseframework.inl.gov/) application. I'm following the instructions provided [here](http://yt-project.org/doc/examining/loading_data.html#exodus-ii-data). My particular script is follows:
```python
import yt
file_name = "results.e"
ds = yt.load(file_name, step=-1)
time = float(ds.current_time)
ad = ds.all_data()
ad["connect1", "rho"]
```
where ("connect1", "rho") is a valid field. This script fails on the last line with a traceback and the following error message:
```
RuntimeError: Region right edge[0] < left edge: width = 0.0
```
The particular MOOSE simulation consists of two, 1D meshes. Would anyone know why I'm getting this error and how to overcome it?
Thanks in advance,
Andrew
1 month
