
Hi yt-users, I'm compiling from the tip of yt-3.0 where plot seems to be broken for Ramses... yt plot tests/output_00001/info_00001.txttests/output_00001/amr_00001.out00001 yt : [WARNING ] 2012-12-18 10:27:10,365 No current mechanism of distinguishing cosmological simulations in RAMSES! yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: current_time = 0.0 yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_dimensions = [64 64 64] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_left_edge = [ 0. 0. 0.] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_right_edge = [ 1. 1. 1.] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: cosmological_simulation = 1 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: current_redshift = 0.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: omega_lambda = 0.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: omega_matter = 1.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: hubble_constant = 1.0 yt : [INFO ] 2012-12-18 10:27:10,367 Adding plot for axis 0 yt : [INFO ] 2012-12-18 10:27:11,597 xlim = 0.000000 1.000000 yt : [INFO ] 2012-12-18 10:27:11,597 ylim = 0.000000 1.000000 yt : [INFO ] 2012-12-18 10:27:11,597 Making a fixed resolution buffer of (('gas', 'Density')) 800 by 800 python2.7(2499,0x7fff710f6180) malloc: *** error for object 0x102f20608: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug Abort trap: 6 and for projection: yt plot -m tests/output_00001/info_00001.txt tests/output_00001/amr_00001.out00001 yt : [WARNING ] 2012-12-18 10:48:46,412 No current mechanism of distinguishing cosmological simulations in RAMSES! yt : [INFO ] 2012-12-18 10:48:46,412 Parameters: current_time = 0.0 yt : [INFO ] 2012-12-18 10:48:46,412 Parameters: domain_dimensions = [64 64 64] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: domain_left_edge = [ 0. 0. 0.] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: domain_right_edge = [ 1. 1. 1.] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: cosmological_simulation = 1 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: current_redshift = 0.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: omega_lambda = 0.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: omega_matter = 1.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: hubble_constant = 1.0 Traceback (most recent call last): File "/Users/sleitner/repos/yt-3.0-run/yt-x86_64/bin/yt", line 9, in <module> load_entry_point('yt==3.0dev', 'console_scripts', 'yt')() File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 1578, in run_main args.func(args) File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 97, in run self(args) File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 1250, in __call__ v, center = pf.h.find_max("Density") AttributeError: 'RAMSESGeometryHandler' object has no attribute 'find_max' Thanks for your help! Sam Leitner

Hi Sam, Could you have a try at making the plot and specifying the center by hand? Matt On Dec 18, 2012 9:57 AM, "Sam Leitner" <sam.leitner@gmail.com> wrote:
Hi yt-users,
I'm compiling from the tip of yt-3.0 where plot seems to be broken for Ramses...
yt plot tests/output_00001/info_00001.txttests/output_00001/amr_00001.out00001 yt : [WARNING ] 2012-12-18 10:27:10,365 No current mechanism of distinguishing cosmological simulations in RAMSES! yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: current_time = 0.0 yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_dimensions = [64 64 64] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_left_edge = [ 0. 0. 0.] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_right_edge = [ 1. 1. 1.] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: cosmological_simulation = 1 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: current_redshift = 0.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: omega_lambda = 0.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: omega_matter = 1.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: hubble_constant = 1.0 yt : [INFO ] 2012-12-18 10:27:10,367 Adding plot for axis 0 yt : [INFO ] 2012-12-18 10:27:11,597 xlim = 0.000000 1.000000 yt : [INFO ] 2012-12-18 10:27:11,597 ylim = 0.000000 1.000000 yt : [INFO ] 2012-12-18 10:27:11,597 Making a fixed resolution buffer of (('gas', 'Density')) 800 by 800 python2.7(2499,0x7fff710f6180) malloc: *** error for object 0x102f20608: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug Abort trap: 6
and for projection:
yt plot -m tests/output_00001/info_00001.txt tests/output_00001/amr_00001.out00001 yt : [WARNING ] 2012-12-18 10:48:46,412 No current mechanism of distinguishing cosmological simulations in RAMSES! yt : [INFO ] 2012-12-18 10:48:46,412 Parameters: current_time = 0.0 yt : [INFO ] 2012-12-18 10:48:46,412 Parameters: domain_dimensions = [64 64 64] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: domain_left_edge = [ 0. 0. 0.] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: domain_right_edge = [ 1. 1. 1.] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: cosmological_simulation = 1 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: current_redshift = 0.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: omega_lambda = 0.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: omega_matter = 1.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: hubble_constant = 1.0 Traceback (most recent call last): File "/Users/sleitner/repos/yt-3.0-run/yt-x86_64/bin/yt", line 9, in <module> load_entry_point('yt==3.0dev', 'console_scripts', 'yt')() File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 1578, in run_main args.func(args) File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 97, in run self(args) File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 1250, in __call__ v, center = pf.h.find_max("Density") AttributeError: 'RAMSESGeometryHandler' object has no attribute 'find_max'
Thanks for your help! Sam Leitner
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Matt, Sure, by making the plot "by hand" do you mean something other than: slc = SlicePlot(pf, 'x', "Density", center=[0.5,0.5,0.5]) slc.save('ramses') ? In that case, I get the same error (sorry -- the "useful" knowledge of yt I've accumulated is still very limited). Sam On Tue, Dec 18, 2012 at 11:45 AM, Matthew Turk <matthewturk@gmail.com>wrote:
Hi Sam,
Could you have a try at making the plot and specifying the center by hand?
Matt On Dec 18, 2012 9:57 AM, "Sam Leitner" <sam.leitner@gmail.com> wrote:
Hi yt-users,
I'm compiling from the tip of yt-3.0 where plot seems to be broken for Ramses...
yt plot tests/output_00001/info_00001.txttests/output_00001/amr_00001.out00001 yt : [WARNING ] 2012-12-18 10:27:10,365 No current mechanism of distinguishing cosmological simulations in RAMSES! yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: current_time = 0.0 yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_dimensions = [64 64 64] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_left_edge = [ 0. 0. 0.] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_right_edge = [ 1. 1. 1.] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: cosmological_simulation = 1 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: current_redshift = 0.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: omega_lambda = 0.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: omega_matter = 1.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: hubble_constant = 1.0 yt : [INFO ] 2012-12-18 10:27:10,367 Adding plot for axis 0 yt : [INFO ] 2012-12-18 10:27:11,597 xlim = 0.000000 1.000000 yt : [INFO ] 2012-12-18 10:27:11,597 ylim = 0.000000 1.000000 yt : [INFO ] 2012-12-18 10:27:11,597 Making a fixed resolution buffer of (('gas', 'Density')) 800 by 800 python2.7(2499,0x7fff710f6180) malloc: *** error for object 0x102f20608: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug Abort trap: 6
and for projection:
yt plot -m tests/output_00001/info_00001.txt tests/output_00001/amr_00001.out00001 yt : [WARNING ] 2012-12-18 10:48:46,412 No current mechanism of distinguishing cosmological simulations in RAMSES! yt : [INFO ] 2012-12-18 10:48:46,412 Parameters: current_time = 0.0 yt : [INFO ] 2012-12-18 10:48:46,412 Parameters: domain_dimensions = [64 64 64] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: domain_left_edge = [ 0. 0. 0.] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: domain_right_edge = [ 1. 1. 1.] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: cosmological_simulation = 1 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: current_redshift = 0.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: omega_lambda = 0.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: omega_matter = 1.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: hubble_constant = 1.0 Traceback (most recent call last): File "/Users/sleitner/repos/yt-3.0-run/yt-x86_64/bin/yt", line 9, in <module> load_entry_point('yt==3.0dev', 'console_scripts', 'yt')() File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 1578, in run_main args.func(args) File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 97, in run self(args) File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 1250, in __call__ v, center = pf.h.find_max("Density") AttributeError: 'RAMSESGeometryHandler' object has no attribute 'find_max'
Thanks for your help! Sam Leitner
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Sam, Okay, that's interesting. I am without Internet for a bit, but I have just run on 3.0 with all of the ramses data I have on my laptop without any trouble. My script is similar to yours, except with "c" instead of [. 5,.5,.5]. I am running on a minor modification of 6fb278bef80f. Which changeset hash are you on? I'm somewhat worried this will be tricky to debug. Any chance you can also check which fields are in the data file? One sticking point with ramses that I do not yet have a solution for is that the fields must be specified, not guessed, and this is not cleanly implemented yet. Matt On Dec 18, 2012 11:30 AM, "Sam Leitner" <sam.leitner@gmail.com> wrote:
Hi Matt, Sure, by making the plot "by hand" do you mean something other than: slc = SlicePlot(pf, 'x', "Density", center=[0.5,0.5,0.5]) slc.save('ramses') ? In that case, I get the same error (sorry -- the "useful" knowledge of yt I've accumulated is still very limited). Sam
On Tue, Dec 18, 2012 at 11:45 AM, Matthew Turk <matthewturk@gmail.com>wrote:
Hi Sam,
Could you have a try at making the plot and specifying the center by hand?
Matt On Dec 18, 2012 9:57 AM, "Sam Leitner" <sam.leitner@gmail.com> wrote:
Hi yt-users,
I'm compiling from the tip of yt-3.0 where plot seems to be broken for Ramses...
yt plot tests/output_00001/info_00001.txttests/output_00001/amr_00001.out00001 yt : [WARNING ] 2012-12-18 10:27:10,365 No current mechanism of distinguishing cosmological simulations in RAMSES! yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: current_time = 0.0 yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_dimensions = [64 64 64] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_left_edge = [ 0. 0. 0.] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: domain_right_edge = [ 1. 1. 1.] yt : [INFO ] 2012-12-18 10:27:10,366 Parameters: cosmological_simulation = 1 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: current_redshift = 0.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: omega_lambda = 0.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: omega_matter = 1.0 yt : [INFO ] 2012-12-18 10:27:10,367 Parameters: hubble_constant = 1.0 yt : [INFO ] 2012-12-18 10:27:10,367 Adding plot for axis 0 yt : [INFO ] 2012-12-18 10:27:11,597 xlim = 0.000000 1.000000 yt : [INFO ] 2012-12-18 10:27:11,597 ylim = 0.000000 1.000000 yt : [INFO ] 2012-12-18 10:27:11,597 Making a fixed resolution buffer of (('gas', 'Density')) 800 by 800 python2.7(2499,0x7fff710f6180) malloc: *** error for object 0x102f20608: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug Abort trap: 6
and for projection:
yt plot -m tests/output_00001/info_00001.txt tests/output_00001/amr_00001.out00001 yt : [WARNING ] 2012-12-18 10:48:46,412 No current mechanism of distinguishing cosmological simulations in RAMSES! yt : [INFO ] 2012-12-18 10:48:46,412 Parameters: current_time = 0.0 yt : [INFO ] 2012-12-18 10:48:46,412 Parameters: domain_dimensions = [64 64 64] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: domain_left_edge = [ 0. 0. 0.] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: domain_right_edge = [ 1. 1. 1.] yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: cosmological_simulation = 1 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: current_redshift = 0.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: omega_lambda = 0.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: omega_matter = 1.0 yt : [INFO ] 2012-12-18 10:48:46,413 Parameters: hubble_constant = 1.0 Traceback (most recent call last): File "/Users/sleitner/repos/yt-3.0-run/yt-x86_64/bin/yt", line 9, in <module> load_entry_point('yt==3.0dev', 'console_scripts', 'yt')() File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 1578, in run_main args.func(args) File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 97, in run self(args) File "/Users/sleitner/repos/yt-3.0-artio/yt/utilities/command_line.py", line 1250, in __call__ v, center = pf.h.find_max("Density") AttributeError: 'RAMSESGeometryHandler' object has no attribute 'find_max'
Thanks for your help! Sam Leitner
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Matt, Hmm, we are indeed both on the same change set (6fb278bef80f). The Ramses files I'm running on are here (if you get a chance to give them a try): http://kicp.uchicago.edu/~agertz/sam/ Thanks for your help! Sam On Tue, Dec 18, 2012 at 12:52 PM, Matthew Turk <matthewturk@gmail.com>wrote:
6fb278bef80f

Hi Sam and Matt, I'm able to reproduce Sam's crash on my laptop (OS X 10.7, yt-3.0 6fb278bef80f). I was able to run python inside gdb but the backtrace isn't very helpful (at least to my eyes): http://paste.yt-project.org/show/3002/ One possibly useful hint is that it appears to be crashing inside CPython proper rather than _ramses_reader.so. I also tried in yt-2.5 dev ( 4174ccdee99d) and don't see crashes, although looking at the plots I get back, I see some corruption: http://i.imgur.com/PrQyO.png It's not clear if this is an issue on the yt side or if the data file Sam supplied is somehow corrupted. I'm guessing that Sam can independently verify that the data is free or errors. -Nathan On 12/18/12 10:13 AM, Sam Leitner wrote:
Hi Matt, Hmm, we are indeed both on the same change set (6fb278bef80f). The Ramses files I'm running on are here (if you get a chance to give them a try): http://kicp.uchicago.edu/~agertz/sam/ <http://kicp.uchicago.edu/%7Eagertz/sam/> Thanks for your help! Sam
On Tue, Dec 18, 2012 at 12:52 PM, Matthew Turk <matthewturk@gmail.com <mailto:matthewturk@gmail.com>> wrote:
6fb278bef80f
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

The output is not mine, it's an isolated disk run by Oscar Agertz. I can double check with him, but I very much doubt the corruption is on the data side. On Tue, Dec 18, 2012 at 2:03 PM, Nathan Goldbaum <nathan12343@gmail.com<javascript:_e({}, 'cvml', 'nathan12343@gmail.com');>
wrote:
Hi Sam and Matt,
I'm able to reproduce Sam's crash on my laptop (OS X 10.7, yt-3.0 6fb278bef80f).
I was able to run python inside gdb but the backtrace isn't very helpful (at least to my eyes): http://paste.yt-project.org/**show/3002/<http://paste.yt-project.org/show/3002/>
One possibly useful hint is that it appears to be crashing inside CPython proper rather than _ramses_reader.so.
I also tried in yt-2.5 dev ( 4174ccdee99d) and don't see crashes, although looking at the plots I get back, I see some corruption: http://i.imgur.com/PrQyO.png
It's not clear if this is an issue on the yt side or if the data file Sam supplied is somehow corrupted. I'm guessing that Sam can independently verify that the data is free or errors.
-Nathan
On 12/18/12 10:13 AM, Sam Leitner wrote:
Hi Matt, Hmm, we are indeed both on the same change set (6fb278bef80f). The Ramses files I'm running on are here (if you get a chance to give them a try): http://kicp.uchicago.edu/~**agertz/sam/<http://kicp.uchicago.edu/~agertz/sam/>< http://kicp.uchicago.edu/%**7Eagertz/sam/<http://kicp.uchicago.edu/%7Eagertz/sam/>
Thanks for your help! Sam
On Tue, Dec 18, 2012 at 12:52 PM, Matthew Turk <matthewturk@gmail.com<javascript:_e({}, 'cvml', 'matthewturk@gmail.com');><mailto: matthewturk@gmail.com <javascript:_e({}, 'cvml', 'matthewturk@gmail.com');>>**> wrote:
6fb278bef80f
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org <javascript:_e({}, 'cvml', 'yt-users@lists.spacepope.org');> http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org <javascript:_e({}, 'cvml', 'yt-users@lists.spacepope.org');> http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
-- Sent from a mobile device.

Hi Sam and Nathan, Nathan, thanks for verifying! What a weird backtrack. Speaks to funny business with array memory. When I say the fields are in a particular order, I mean that the actual fields in the file are likely different. To my understanding ramses doesn't provide a way to say, "this output has density, energy, metallicity, magnetic field" etc right in the output file, but instead requires the person reading the file in know this in advance. We can try applying heuristics, but I am uncertain how well these would work. As it stands, the field list is currently hard coded although intended to be changed. It's in a tuple inside the ramses code. Additionally, it's also possible that this data file includes precision that I had not expected - modifying the float/int bit width may be necessary for this particular datatype. My recommendation is to check the fields in the file. They may differ from what yt expects, causing a memory overrun. What we should do is accept them as a constructor argument. Unfortunately I won't be able to provide feedback on this immediately but it should be straightforward to implement. Matt On Dec 18, 2012 1:13 PM, "Sam Leitner" <sam.leitner@gmail.com> wrote:
The output is not mine, it's an isolated disk run by Oscar Agertz. I can double check with him, but I very much doubt the corruption is on the data side.
On Tue, Dec 18, 2012 at 2:03 PM, Nathan Goldbaum <nathan12343@gmail.com>wrote:
Hi Sam and Matt,
I'm able to reproduce Sam's crash on my laptop (OS X 10.7, yt-3.0 6fb278bef80f).
I was able to run python inside gdb but the backtrace isn't very helpful (at least to my eyes): http://paste.yt-project.org/**show/3002/<http://paste.yt-project.org/show/3002/>
One possibly useful hint is that it appears to be crashing inside CPython proper rather than _ramses_reader.so.
I also tried in yt-2.5 dev ( 4174ccdee99d) and don't see crashes, although looking at the plots I get back, I see some corruption: http://i.imgur.com/PrQyO.png
It's not clear if this is an issue on the yt side or if the data file Sam supplied is somehow corrupted. I'm guessing that Sam can independently verify that the data is free or errors.
-Nathan
On 12/18/12 10:13 AM, Sam Leitner wrote:
Hi Matt, Hmm, we are indeed both on the same change set (6fb278bef80f). The Ramses files I'm running on are here (if you get a chance to give them a try): http://kicp.uchicago.edu/~**agertz/sam/<http://kicp.uchicago.edu/~agertz/sam/>< http://kicp.uchicago.edu/%**7Eagertz/sam/<http://kicp.uchicago.edu/%7Eagertz/sam/>
Thanks for your help! Sam
On Tue, Dec 18, 2012 at 12:52 PM, Matthew Turk <matthewturk@gmail.com<mailto: matthewturk@gmail.com>**> wrote:
6fb278bef80f
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
______________________________**_________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/**listinfo.cgi/yt-users-**spacepope.org<http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
-- Sent from a mobile device.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Sam, I was able to reproduce the error. I can isolate the first memory corruption to during the octree parsing. I suspect it is related to an off-by-one error somewhere, as I am finding four more Octs than yt expects to find. Following that memory error, arrays become corrupted. My guess is that there is an additional case for IO in RAMSES I was not correctly supporting. I'll write back when I have narrowed it down for a fix. -Matt On Tue, Dec 18, 2012 at 2:43 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Hi Sam and Nathan,
Nathan, thanks for verifying! What a weird backtrack. Speaks to funny business with array memory.
When I say the fields are in a particular order, I mean that the actual fields in the file are likely different. To my understanding ramses doesn't provide a way to say, "this output has density, energy, metallicity, magnetic field" etc right in the output file, but instead requires the person reading the file in know this in advance. We can try applying heuristics, but I am uncertain how well these would work. As it stands, the field list is currently hard coded although intended to be changed. It's in a tuple inside the ramses code.
Additionally, it's also possible that this data file includes precision that I had not expected - modifying the float/int bit width may be necessary for this particular datatype.
My recommendation is to check the fields in the file. They may differ from what yt expects, causing a memory overrun.
What we should do is accept them as a constructor argument. Unfortunately I won't be able to provide feedback on this immediately but it should be straightforward to implement.
Matt
On Dec 18, 2012 1:13 PM, "Sam Leitner" <sam.leitner@gmail.com> wrote:
The output is not mine, it's an isolated disk run by Oscar Agertz. I can double check with him, but I very much doubt the corruption is on the data side.
On Tue, Dec 18, 2012 at 2:03 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi Sam and Matt,
I'm able to reproduce Sam's crash on my laptop (OS X 10.7, yt-3.0 6fb278bef80f).
I was able to run python inside gdb but the backtrace isn't very helpful (at least to my eyes): http://paste.yt-project.org/show/3002/
One possibly useful hint is that it appears to be crashing inside CPython proper rather than _ramses_reader.so.
I also tried in yt-2.5 dev ( 4174ccdee99d) and don't see crashes, although looking at the plots I get back, I see some corruption: http://i.imgur.com/PrQyO.png
It's not clear if this is an issue on the yt side or if the data file Sam supplied is somehow corrupted. I'm guessing that Sam can independently verify that the data is free or errors.
-Nathan
On 12/18/12 10:13 AM, Sam Leitner wrote:
Hi Matt, Hmm, we are indeed both on the same change set (6fb278bef80f). The Ramses files I'm running on are here (if you get a chance to give them a try): http://kicp.uchicago.edu/~agertz/sam/ <http://kicp.uchicago.edu/%7Eagertz/sam/>
Thanks for your help! Sam
On Tue, Dec 18, 2012 at 12:52 PM, Matthew Turk <matthewturk@gmail.com <mailto:matthewturk@gmail.com>> wrote:
6fb278bef80f
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Sent from a mobile device.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Hi Sam, I think I figured it out, and in befae3c21101 I pushed a change to fix it. I believe that calculating the index of the root mesh into which an oct should be inserted was somewhat off, resulting in memory overruns. Let me know if that doesn't fix it for you! Best, Matt On Tue, Dec 18, 2012 at 6:11 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Hi Sam,
I was able to reproduce the error. I can isolate the first memory corruption to during the octree parsing. I suspect it is related to an off-by-one error somewhere, as I am finding four more Octs than yt expects to find. Following that memory error, arrays become corrupted. My guess is that there is an additional case for IO in RAMSES I was not correctly supporting. I'll write back when I have narrowed it down for a fix.
-Matt
On Tue, Dec 18, 2012 at 2:43 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Hi Sam and Nathan,
Nathan, thanks for verifying! What a weird backtrack. Speaks to funny business with array memory.
When I say the fields are in a particular order, I mean that the actual fields in the file are likely different. To my understanding ramses doesn't provide a way to say, "this output has density, energy, metallicity, magnetic field" etc right in the output file, but instead requires the person reading the file in know this in advance. We can try applying heuristics, but I am uncertain how well these would work. As it stands, the field list is currently hard coded although intended to be changed. It's in a tuple inside the ramses code.
Additionally, it's also possible that this data file includes precision that I had not expected - modifying the float/int bit width may be necessary for this particular datatype.
My recommendation is to check the fields in the file. They may differ from what yt expects, causing a memory overrun.
What we should do is accept them as a constructor argument. Unfortunately I won't be able to provide feedback on this immediately but it should be straightforward to implement.
Matt
On Dec 18, 2012 1:13 PM, "Sam Leitner" <sam.leitner@gmail.com> wrote:
The output is not mine, it's an isolated disk run by Oscar Agertz. I can double check with him, but I very much doubt the corruption is on the data side.
On Tue, Dec 18, 2012 at 2:03 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi Sam and Matt,
I'm able to reproduce Sam's crash on my laptop (OS X 10.7, yt-3.0 6fb278bef80f).
I was able to run python inside gdb but the backtrace isn't very helpful (at least to my eyes): http://paste.yt-project.org/show/3002/
One possibly useful hint is that it appears to be crashing inside CPython proper rather than _ramses_reader.so.
I also tried in yt-2.5 dev ( 4174ccdee99d) and don't see crashes, although looking at the plots I get back, I see some corruption: http://i.imgur.com/PrQyO.png
It's not clear if this is an issue on the yt side or if the data file Sam supplied is somehow corrupted. I'm guessing that Sam can independently verify that the data is free or errors.
-Nathan
On 12/18/12 10:13 AM, Sam Leitner wrote:
Hi Matt, Hmm, we are indeed both on the same change set (6fb278bef80f). The Ramses files I'm running on are here (if you get a chance to give them a try): http://kicp.uchicago.edu/~agertz/sam/ <http://kicp.uchicago.edu/%7Eagertz/sam/>
Thanks for your help! Sam
On Tue, Dec 18, 2012 at 12:52 PM, Matthew Turk <matthewturk@gmail.com <mailto:matthewturk@gmail.com>> wrote:
6fb278bef80f
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Sent from a mobile device.
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

I didn't carefully look into how fields are filled in Ramses, do they have to live in a particular order? That could well be the problem. I can take a closer look (especially if you confirm the bug for the outputs I'm looking at, so I know that I'm not making another dumb build mistake). On Tue, Dec 18, 2012 at 12:52 PM, Matthew Turk <matthewturk@gmail.com>wrote:
I'm somewhat worried this will be tricky to debug. Any chance you can also check which fields are in the data file? One sticking point with ramses that I do not yet have a solution for is that the fields must be specified, not guessed, and this is not cleanly implemented yet.
participants (3)
-
Matthew Turk
-
Nathan Goldbaum
-
Sam Leitner