Hi Christine, I'm not sure you can use the only_on_root like that. Someone can correct me if I'm wrong. However, you can do this: @only_on_root def do_something: do_stuff more_stuff return stuff do_something() Britton On Tue, Nov 22, 2011 at 11:49 AM, Christine Simpson < csimpson@astro.columbia.edu> wrote:

Hi,

I'm trying to run a script with parallel yt that does some non-yt io. I'm a little confused about how to use only_on_root.

Here is a test script I've tried:

from yt.mods import * from yt.visualization.api import PlotCollection

import matplotlib.colorbar as cb

path =

"/scratch/01112/tg803911/halo88_therm_feed_3.7e-6_LW_RadiationShield_lmax12/DD0010/" fn = "output_0010" pf = load(path+fn) pc = PlotCollection(pf) pc.add_projection("Density",0) pc.set_width(20,'kpc') pc.save(fn)

filename = 'hello_world'

file = only_on_root(open(filename,'w')) only_on_root(file.write("hello_world \n")) only_on_root(file.close())

Pasted below is the error. I'm uncertain about the syntax of only_on_root. Do I just wrap it around normal commands? Also, it seems that the write command is trying to execute on all the processors. That shouldn't be happening, right?

P007 yt : [INFO ] 2011-11-22 09:50:20,558 Saved output_0010_Projection_x_Density.png P008 yt : [INFO ] 2011-11-22 09:50:20,558 Saved output_0010_Projection_x_Density.png P009 yt : [INFO ] 2011-11-22 09:50:20,560 Saved output_0010_Projection_x_Density.png Traceback (most recent call last): Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> File "test_parallel_yt.py", line 19, in <module> Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' only_on_root(file.write("hello_world \n")) AttributeError : only_on_root(file.write("hello_world \n")) 'NoneType' object has no attribute 'write' P010 yt : [INFO ] 2011-11-22 09:50:20,561 Saved output_0010_Projection_x_Density.png AttributeErrorP011 yt : [INFO ] 2011-11-22 09:50:20,561 Saved output_0010_Projection_x_Density.png : 'NoneType' object has no attribute 'write' Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) Traceback (most recent call last): AttributeError: File "test_parallel_yt.py", line 19, in <module> 'NoneType' object has no attribute 'write' P005 yt : [INFO ] 2011-11-22 09:50:20,562 Saved output_0010_Projection_x_Density.png P004 yt : [INFO ] 2011-11-22 09:50:20,562 Saved output_0010_Projection_x_Density.png only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P002 yt : [INFO ] 2011-11-22 09:50:20,583 Saved output_0010_Projection_x_Density.png Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P001 yt : [INFO ] 2011-11-22 09:50:20,589 Saved output_0010_Projection_x_Density.png P003 yt : [INFO ] 2011-11-22 09:50:20,589 Saved output_0010_Projection_x_Density.png Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P013 yt : [INFO ] 2011-11-22 09:50:20,593 Saved output_0010_Projection_x_Density.png P014 yt : [INFO ] 2011-11-22 09:50:20,593 Saved output_0010_Projection_x_Density.png Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P012 yt : [INFO ] 2011-11-22 09:50:20,596 Saved output_0010_Projection_x_Density.png Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P015 yt : [INFO ] 2011-11-22 09:50:20,599 Saved output_0010_Projection_x_Density.png Traceback (most recent call last): File "test_parallel_yt.py", line 19, in <module> only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P000 yt : [INFO ] 2011-11-22 09:50:21,410 Saved output_0010_Projection_x_Density.png Traceback (most recent call last): File "test_parallel_yt.py", line 18, in <module> file = only_on_root(open(filename,'w')) File "/share/home/01112/tg803911/yt_17May2011/yt-x86_64/src/yt-hg/yt/funcs.py", line 357, in only_on_root return func(*args, **kwargs) TypeError: 'file' object is not callable mpispawn.c:303 Unexpected exit status

Child exited abnormally! Killing remote processes...DONE

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Christine, only_on_root looks like this: only_on_root(func, *args, **kwargs) So the idea here is that it accepts the function handle, any positional arguments, and any keyword arguments. Your commands look like this: file = only_on_root(open(filename,'w')) only_on_root(file.write("hello_world \n")) only_on_root(file.close()) The first translation would look like: file = only_on_root(open, filename, 'w') but then you'll run into problems with the second two, because they will require the attributes 'close' and 'write' on the file object, which is going to be None on the non-root procs. So your best bet is a variant of what Britton suggested: def write_file(): file = open(filename,'w') file.write("hello_world \n")) file.close() only_on_root(write_file) -Matt On Tue, Nov 22, 2011 at 11:49 AM, Christine Simpson wrote:

Hi,

I'm trying to run a script with parallel yt that does some non-yt io. I'm a little confused about how to use only_on_root.

Here is a test script I've tried:

from yt.mods import * from yt.visualization.api import PlotCollection

import matplotlib.colorbar as cb

path = "/scratch/01112/tg803911/halo88_therm_feed_3.7e-6_LW_RadiationShield_lmax12/DD0010/" fn = "output_0010" pf = load(path+fn) pc = PlotCollection(pf) pc.add_projection("Density",0) pc.set_width(20,'kpc') pc.save(fn)

filename = 'hello_world'

file = only_on_root(open(filename,'w')) only_on_root(file.write("hello_world \n")) only_on_root(file.close())

Pasted below is the error.  I'm uncertain about the syntax of only_on_root.  Do I just wrap it around normal commands?  Also, it seems that the write command is trying to execute on all the processors.  That shouldn't be happening, right?

P007 yt : [INFO     ] 2011-11-22 09:50:20,558 Saved output_0010_Projection_x_Density.png P008 yt : [INFO     ] 2011-11-22 09:50:20,558 Saved output_0010_Projection_x_Density.png P009 yt : [INFO     ] 2011-11-22 09:50:20,560 Saved output_0010_Projection_x_Density.png Traceback (most recent call last): Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>  File "test_parallel_yt.py", line 19, in <module> Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write'    only_on_root(file.write("hello_world \n")) AttributeError    : only_on_root(file.write("hello_world \n")) 'NoneType' object has no attribute 'write' P010 yt : [INFO     ] 2011-11-22 09:50:20,561 Saved output_0010_Projection_x_Density.png AttributeErrorP011 yt : [INFO     ] 2011-11-22 09:50:20,561 Saved output_0010_Projection_x_Density.png : 'NoneType' object has no attribute 'write' Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) Traceback (most recent call last): AttributeError:   File "test_parallel_yt.py", line 19, in <module> 'NoneType' object has no attribute 'write' P005 yt : [INFO     ] 2011-11-22 09:50:20,562 Saved output_0010_Projection_x_Density.png P004 yt : [INFO     ] 2011-11-22 09:50:20,562 Saved output_0010_Projection_x_Density.png    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P002 yt : [INFO     ] 2011-11-22 09:50:20,583 Saved output_0010_Projection_x_Density.png Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P001 yt : [INFO     ] 2011-11-22 09:50:20,589 Saved output_0010_Projection_x_Density.png P003 yt : [INFO     ] 2011-11-22 09:50:20,589 Saved output_0010_Projection_x_Density.png Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P013 yt : [INFO     ] 2011-11-22 09:50:20,593 Saved output_0010_Projection_x_Density.png P014 yt : [INFO     ] 2011-11-22 09:50:20,593 Saved output_0010_Projection_x_Density.png Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P012 yt : [INFO     ] 2011-11-22 09:50:20,596 Saved output_0010_Projection_x_Density.png Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P015 yt : [INFO     ] 2011-11-22 09:50:20,599 Saved output_0010_Projection_x_Density.png Traceback (most recent call last):  File "test_parallel_yt.py", line 19, in <module>    only_on_root(file.write("hello_world \n")) AttributeError: 'NoneType' object has no attribute 'write' P000 yt : [INFO     ] 2011-11-22 09:50:21,410 Saved output_0010_Projection_x_Density.png Traceback (most recent call last):  File "test_parallel_yt.py", line 18, in <module>    file = only_on_root(open(filename,'w'))  File "/share/home/01112/tg803911/yt_17May2011/yt-x86_64/src/yt-hg/yt/funcs.py", line 357, in only_on_root    return func(*args, **kwargs) TypeError: 'file' object is not callable mpispawn.c:303 Unexpected exit status

Child exited abnormally! Killing remote processes...DONE

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org