Hi all, I'm getting seg faults on a simple projection: pc = PlotCollection(pf) pc.add_projection("Density", 0) <seg fault> I used 'hg bisect' and tracked it down to this changeset: https://bitbucket.org/yt_analysis/yt/changeset/306232c75c58 If I replace yt/data_objects/static_output.py with the parent version bbb5aeca876f, the seg fault goes away. I see the seg fault, and the fix, on both Mac and Linux. Does anyone else see this issue? Thanks! -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Bah humbug. I did a fresh install on my mac, and the problem went away. I wonder what cruft I had left over was getting in the way? I deleted the .yt file etc... Anyway, sorry for the noise, move along. -- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice)
Hi Stephen, A bunch of people have been reporting segfaults. I am starting to think it may be an issue of an older h5py combined with this change, which may build up multiple copies of the hierarchy and keep multiple copies of a file (in this case the data file) open. I think we should consider backing this change out, and providing it in a more subtle way -- for instance, making field info just always create the hierarchy. Here it actually almost always creates it twice. Remove the deletion, just leave the hierarchy in existence. -Matt On Wed, Mar 14, 2012 at 10:46 PM, Stephen Skory <s@skory.us> wrote:
Bah humbug.
I did a fresh install on my mac, and the problem went away. I wonder what cruft I had left over was getting in the way? I deleted the .yt file etc...
Anyway, sorry for the noise, move along.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Matt and Stephen, I can confirm that the seg fault disappears on Mac OS X when updating to h5py-2.0.1, though a simple projection is still giving an out of bounds error on a simple unigrid data set. Not sure why that occurs. But I think Matt is definitely right about h5py: p = pc.add_projection('Density',0) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/visualization/plot_collection.py", line 758, in add_projection **field_parameters) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 1930, in __init__ self._refresh_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 309, in _refresh_data self.get_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 2127, in get_data pdxs = na.concatenate(pdxs, axis=1) IndexError: axis 1 out of bounds [0, 1) On Mar 15, 2012, at 4:55 AM, Matthew Turk wrote:
Hi Stephen,
A bunch of people have been reporting segfaults. I am starting to think it may be an issue of an older h5py combined with this change, which may build up multiple copies of the hierarchy and keep multiple copies of a file (in this case the data file) open.
I think we should consider backing this change out, and providing it in a more subtle way -- for instance, making field info just always create the hierarchy. Here it actually almost always creates it twice. Remove the deletion, just leave the hierarchy in existence.
-Matt
On Wed, Mar 14, 2012 at 10:46 PM, Stephen Skory <s@skory.us> wrote:
Bah humbug.
I did a fresh install on my mac, and the problem went away. I wonder what cruft I had left over was getting in the way? I deleted the .yt file etc...
Anyway, sorry for the noise, move along.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Okay, let's back out that changeset. I think it is okay to mandate an instantiated hierarchy if you want to have detailed, output-specific field information. On Mar 15, 2012 10:02 AM, "Eric Hallman" <e.hallman@me.com> wrote:
Matt and Stephen, I can confirm that the seg fault disappears on Mac OS X when updating to h5py-2.0.1, though a simple projection is still giving an out of bounds error on a simple unigrid data set. Not sure why that occurs. But I think Matt is definitely right about h5py:
p = pc.add_projection('Density',0) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/visualization/plot_collection.py", line 758, in add_projection **field_parameters) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 1930, in __init__ self._refresh_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 309, in _refresh_data self.get_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 2127, in get_data pdxs = na.concatenate(pdxs, axis=1) IndexError: axis 1 out of bounds [0, 1)
On Mar 15, 2012, at 4:55 AM, Matthew Turk wrote:
Hi Stephen,
A bunch of people have been reporting segfaults. I am starting to think it may be an issue of an older h5py combined with this change, which may build up multiple copies of the hierarchy and keep multiple copies of a file (in this case the data file) open.
I think we should consider backing this change out, and providing it in a more subtle way -- for instance, making field info just always create the hierarchy. Here it actually almost always creates it twice. Remove the deletion, just leave the hierarchy in existence.
-Matt
On Wed, Mar 14, 2012 at 10:46 PM, Stephen Skory <s@skory.us> wrote:
Bah humbug.
I did a fresh install on my mac, and the problem went away. I wonder what cruft I had left over was getting in the way? I deleted the .yt file etc...
Anyway, sorry for the noise, move along.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
I would like to see this addressed in the future (since it makes little sense from a user's standpoint that they need to call pf.h before pf.field_info), but for now I agree it is causing too much of a headache. Sam On Thu, Mar 15, 2012 at 8:10 AM, Matthew Turk <matthewturk@gmail.com> wrote:
Okay, let's back out that changeset. I think it is okay to mandate an instantiated hierarchy if you want to have detailed, output-specific field information. On Mar 15, 2012 10:02 AM, "Eric Hallman" <e.hallman@me.com> wrote:
Matt and Stephen, I can confirm that the seg fault disappears on Mac OS X when updating to h5py-2.0.1, though a simple projection is still giving an out of bounds error on a simple unigrid data set. Not sure why that occurs. But I think Matt is definitely right about h5py:
p = pc.add_projection('Density',0) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/visualization/plot_collection.py", line 758, in add_projection **field_parameters) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 1930, in __init__ self._refresh_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 309, in _refresh_data self.get_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 2127, in get_data pdxs = na.concatenate(pdxs, axis=1) IndexError: axis 1 out of bounds [0, 1)
On Mar 15, 2012, at 4:55 AM, Matthew Turk wrote:
Hi Stephen,
A bunch of people have been reporting segfaults. I am starting to think it may be an issue of an older h5py combined with this change, which may build up multiple copies of the hierarchy and keep multiple copies of a file (in this case the data file) open.
I think we should consider backing this change out, and providing it in a more subtle way -- for instance, making field info just always create the hierarchy. Here it actually almost always creates it twice. Remove the deletion, just leave the hierarchy in existence.
-Matt
On Wed, Mar 14, 2012 at 10:46 PM, Stephen Skory <s@skory.us> wrote:
Bah humbug.
I did a fresh install on my mac, and the problem went away. I wonder what cruft I had left over was getting in the way? I deleted the .yt file etc...
Anyway, sorry for the noise, move along.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Matt, my prior error below is not related to the previous problem. But it does bring up a weird issue I thought you'd be interested in. I did not define any refinement criteria, but Enzo is writing the parameter "RefineBy" as =4 by default in a unigrid run. This is causing yt to use something called "overlap_proj"? Which is crashing the projection. If I go back and set RefineBy=2, the problem vanishes. This is maybe more of a Enzo issue than yt, but if one is using non-factor-of-two refinement, it could cause problems. Could this be related to Dave Collins' problem with RefineBy=4? Eric On Mar 15, 2012, at 10:10 AM, Matthew Turk wrote:
Okay, let's back out that changeset. I think it is okay to mandate an instantiated hierarchy if you want to have detailed, output-specific field information.
On Mar 15, 2012 10:02 AM, "Eric Hallman" <e.hallman@me.com> wrote: Matt and Stephen, I can confirm that the seg fault disappears on Mac OS X when updating to h5py-2.0.1, though a simple projection is still giving an out of bounds error on a simple unigrid data set. Not sure why that occurs. But I think Matt is definitely right about h5py:
p = pc.add_projection('Density',0) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/visualization/plot_collection.py", line 758, in add_projection **field_parameters) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 1930, in __init__ self._refresh_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 309, in _refresh_data self.get_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 2127, in get_data pdxs = na.concatenate(pdxs, axis=1) IndexError: axis 1 out of bounds [0, 1)
On Mar 15, 2012, at 4:55 AM, Matthew Turk wrote:
Hi Stephen,
A bunch of people have been reporting segfaults. I am starting to think it may be an issue of an older h5py combined with this change, which may build up multiple copies of the hierarchy and keep multiple copies of a file (in this case the data file) open.
I think we should consider backing this change out, and providing it in a more subtle way -- for instance, making field info just always create the hierarchy. Here it actually almost always creates it twice. Remove the deletion, just leave the hierarchy in existence.
-Matt
On Wed, Mar 14, 2012 at 10:46 PM, Stephen Skory <s@skory.us> wrote:
Bah humbug.
I did a fresh install on my mac, and the problem went away. I wonder what cruft I had left over was getting in the way? I deleted the .yt file etc...
Anyway, sorry for the noise, move along.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Eric Hallman Tech-X Corporation hallman@txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 254-5833 Boulder, CO 80303 Fax: (303) 448-7756 --
Hi Eric, This is intentional, but I think Dave's were something else. RefineBy=2 allows you to use the (much, much faster) quadtree-based projection; Enzo's default is RefineBy=4, so if you just run a unigrid you're likely not going to override that. The overlap proj is what we used up until about a year ago by default. The specific bug you saw is puzzling to me, and I'm not sure without more detailed traceback how to proceed. Could you run with --paste-detailed ? On Thu, Mar 15, 2012 at 10:29 AM, Eric Hallman <hallman@txcorp.com> wrote:
Matt, my prior error below is not related to the previous problem. But it does bring up a weird issue I thought you'd be interested in. I did not define any refinement criteria, but Enzo is writing the parameter "RefineBy" as =4 by default in a unigrid run. This is causing yt to use something called "overlap_proj"? Which is crashing the projection. If I go back and set RefineBy=2, the problem vanishes. This is maybe more of a Enzo issue than yt, but if one is using non-factor-of-two refinement, it could cause problems. Could this be related to Dave Collins' problem with RefineBy=4?
Eric
On Mar 15, 2012, at 10:10 AM, Matthew Turk wrote:
Okay, let's back out that changeset. I think it is okay to mandate an instantiated hierarchy if you want to have detailed, output-specific field information.
On Mar 15, 2012 10:02 AM, "Eric Hallman" <e.hallman@me.com> wrote:
Matt and Stephen, I can confirm that the seg fault disappears on Mac OS X when updating to h5py-2.0.1, though a simple projection is still giving an out of bounds error on a simple unigrid data set. Not sure why that occurs. But I think Matt is definitely right about h5py:
p = pc.add_projection('Density',0) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/visualization/plot_collection.py", line 758, in add_projection **field_parameters) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 1930, in __init__ self._refresh_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 309, in _refresh_data self.get_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 2127, in get_data pdxs = na.concatenate(pdxs, axis=1) IndexError: axis 1 out of bounds [0, 1)
On Mar 15, 2012, at 4:55 AM, Matthew Turk wrote:
Hi Stephen,
A bunch of people have been reporting segfaults. I am starting to think it may be an issue of an older h5py combined with this change, which may build up multiple copies of the hierarchy and keep multiple copies of a file (in this case the data file) open.
I think we should consider backing this change out, and providing it in a more subtle way -- for instance, making field info just always create the hierarchy. Here it actually almost always creates it twice. Remove the deletion, just leave the hierarchy in existence.
-Matt
On Wed, Mar 14, 2012 at 10:46 PM, Stephen Skory <s@skory.us> wrote:
Bah humbug.
I did a fresh install on my mac, and the problem went away. I wonder what cruft I had left over was getting in the way? I deleted the .yt file etc...
Anyway, sorry for the noise, move along.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Eric Hallman Tech-X Corporation hallman@txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 254-5833 Boulder, CO 80303 Fax: (303) 448-7756 --
Matt, I ran with paste-detailed, but I thought that would give a paste bin link? Anyway, with --detailed, here is the output manually put in the paste bin: http://paste.yt-project.org/show/2232/ Eric On Mar 15, 2012, at 10:32 AM, Matthew Turk wrote:
Hi Eric,
This is intentional, but I think Dave's were something else. RefineBy=2 allows you to use the (much, much faster) quadtree-based projection; Enzo's default is RefineBy=4, so if you just run a unigrid you're likely not going to override that. The overlap proj is what we used up until about a year ago by default. The specific bug you saw is puzzling to me, and I'm not sure without more detailed traceback how to proceed. Could you run with --paste-detailed ?
On Thu, Mar 15, 2012 at 10:29 AM, Eric Hallman <hallman@txcorp.com> wrote:
Matt, my prior error below is not related to the previous problem. But it does bring up a weird issue I thought you'd be interested in. I did not define any refinement criteria, but Enzo is writing the parameter "RefineBy" as =4 by default in a unigrid run. This is causing yt to use something called "overlap_proj"? Which is crashing the projection. If I go back and set RefineBy=2, the problem vanishes. This is maybe more of a Enzo issue than yt, but if one is using non-factor-of-two refinement, it could cause problems. Could this be related to Dave Collins' problem with RefineBy=4?
Eric
On Mar 15, 2012, at 10:10 AM, Matthew Turk wrote:
Okay, let's back out that changeset. I think it is okay to mandate an instantiated hierarchy if you want to have detailed, output-specific field information.
On Mar 15, 2012 10:02 AM, "Eric Hallman" <e.hallman@me.com> wrote:
Matt and Stephen, I can confirm that the seg fault disappears on Mac OS X when updating to h5py-2.0.1, though a simple projection is still giving an out of bounds error on a simple unigrid data set. Not sure why that occurs. But I think Matt is definitely right about h5py:
p = pc.add_projection('Density',0) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/visualization/plot_collection.py", line 758, in add_projection **field_parameters) File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 1930, in __init__ self._refresh_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 309, in _refresh_data self.get_data() File "/Users/hallman/work/yt-i386/lib/python2.7/site-packages/yt-2.4dev-py2.7-macosx-10.4-x86_64.egg/yt/data_objects/data_containers.py", line 2127, in get_data pdxs = na.concatenate(pdxs, axis=1) IndexError: axis 1 out of bounds [0, 1)
On Mar 15, 2012, at 4:55 AM, Matthew Turk wrote:
Hi Stephen,
A bunch of people have been reporting segfaults. I am starting to think it may be an issue of an older h5py combined with this change, which may build up multiple copies of the hierarchy and keep multiple copies of a file (in this case the data file) open.
I think we should consider backing this change out, and providing it in a more subtle way -- for instance, making field info just always create the hierarchy. Here it actually almost always creates it twice. Remove the deletion, just leave the hierarchy in existence.
-Matt
On Wed, Mar 14, 2012 at 10:46 PM, Stephen Skory <s@skory.us> wrote:
Bah humbug.
I did a fresh install on my mac, and the problem went away. I wonder what cruft I had left over was getting in the way? I deleted the .yt file etc...
Anyway, sorry for the noise, move along.
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Eric Hallman Tech-X Corporation hallman@txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 254-5833 Boulder, CO 80303 Fax: (303) 448-7756 --
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Eric Hallman Tech-X Corporation hallman@txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 254-5833 Boulder, CO 80303 Fax: (303) 448-7756 --
participants (5)
-
Eric Hallman -
Eric Hallman -
Matthew Turk -
Sam Skillman -
Stephen Skory