Nathan, usings pmods in the import leads to an error explosion in the matplotlib imports. It's even worse than the original test. Eric On Sep 5, 2012, at 12:42 PM, Nathan Goldbaum wrote:
Hi Eric,
Exactly right. This is a drop-in replacement for yt.mods on high-latency parallel filesystems (like Kraken, unfortunately).
There's some discussion on the dev mailing list: http://lists.spacepope.org/htdig.cgi/yt-dev-spacepope.org/2012-January/00176...
Unfortunately this isn't covered in the docs (except for a note in the changelog) but it should be in there.
Cheers,
Nathan
On Sep 5, 2012, at 9:40 AM, Eric Hallman wrote:
Nathan, I've been off yt for a while, I'm unaware of pmods. It's specific to parallel I'm guessing?
Eric On Sep 5, 2012, at 12:38 PM, Nathan Goldbaum wrote:
Hi Eric,
Have you tried from yt.pmods import * instead of the normal yt.mods?
Cheers,
Nathan
On Sep 5, 2012, at 9:37 AM, Eric Hallman wrote:
Hey everyone, so this issue seems like one I've had before, but I searched the lists and don't find this exact issue.
On batch jobs on kraken, attempting to do halo finding, I get an almost immediate crash (with an eternal hang until the time limit is reached) due to matplotlib. I've been unable to reproduce it in the interactive queue on kraken, which is frustrating. I'm hoping someone has seen it and can comment.
this is with yt/dev on kraken, and I set env variables to MPLCONFIGDIR like so:
export MPLCONFIGDIR=${PBS_O_WORKDIR}/.matplotlib/ [ ! -d ${MPLCONFIGDIR} ] && mkdir ${MPLCONFIGDIR}
because if you don't, it fails immediately with perm issues.
Anyway, it's a simple script and call
aprun -n 12 python halo_finding.py --parallel
but the details of the script are not too important, since the job fails when yt is imported, as so:
Traceback (most recent call last): File "halo_finding.py", line 1, in <module> from yt.mods import * File "/lustre/scratch/proj/sw/yt/dev/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/mods.py", line 115, in <module> from yt.visualization.api import \ File "/lustre/scratch/proj/sw/yt/dev/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/visualization/api.py", line 34, in <module> from plot_collection import \ File "/lustre/scratch/proj/sw/yt/dev/lib/python2.7/site-packages/yt-2.4dev-py2.7-linux-x86_64.egg/yt/visualization/plot_collection.py", line 26, in <module> from matplotlib import figure File "/lustre/scratch/proj/sw/yt/dev/lib/python2.7/site-packages/matplotlib/figure.py", line 18, in <module> from axes import Axes, SubplotBase, subplot_class_factory File "/lustre/scratch/proj/sw/yt/dev/lib/python2.7/site-packages/matplotlib/axes.py", line 18, in <module> import matplotlib.contour as mcontour File "/lustre/scratch/proj/sw/yt/dev/lib/python2.7/site-packages/matplotlib/contour.py", line 21, in <module> import matplotlib.texmanager as texmanager File "/lustre/scratch/proj/sw/yt/dev/lib/python2.7/site-packages/matplotlib/texmanager.py", line 72, in <module> class TexManager: File "/lustre/scratch/proj/sw/yt/dev/lib/python2.7/site-packages/matplotlib/texmanager.py", line 92, in TexManager os.mkdir(texcache) OSError: [Errno 17] File exists: '/lustre/scratch/hallman/gigaCubes/run1024/.matplotlib/tex.cache'
In each case, I have deleted tex.cache before I restart, thinking an old version persisted there, but the same error happens. The most irritating thing is that the job does not kick out of the batch system, so the time continues to run on however many processors you have until the limit is reached (eternal hang!).
I hope this is something obvious and I'm just dumb. Let me know.
Eric -- Eric Hallman Tech-X Corporation hallman@txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 254-5833 Boulder, CO 80303 Fax: (303) 448-7756 --
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Eric Hallman Tech-X Corporation hallman@txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 254-5833 Boulder, CO 80303 Fax: (303) 448-7756 --
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-- Eric Hallman Tech-X Corporation hallman@txcorp.com 5621 Arapahoe Ave, Suite A Phone: (720) 254-5833 Boulder, CO 80303 Fax: (303) 448-7756 --