[Yt-dev] Fw: X1024 jobs on Ranger

I'm breaking Ranger... before I reply, do any of you have something intelligent to add that I can say? Because the .cpu files aren't spatially restricted, each thread of parallel HOP may have to access many of the .cpu files, which is how the MDS server gets pummeled. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ ----- Forwarded Message ----

From: Tommy Minyard To: "sskory@physics.ucsd.edu" Sent: Thursday, April 30, 2009 12:54:49 PM Subject: X1024 jobs on Ranger

Hello Stephen,

In our monitoring of Ranger, we've noticed that some of your recent jobs named x1024 seem to be causing an abnormally high load on the /scratch filesystem meta-data server (MDS). From our monitoring, it appears that the MDS load goes way up when your job initially begins to run for up to the first 30 minutes to hour, but then it drops back down to a more reasonable load after the job has been running for a while.

Do you have any idea what may be triggering such a high load from the application you are running? It does not seem to cause any major problems or generate errors, however, the filesystem access becomes much more sluggish when the MDS load is so high. If you could give us a few more details that might help explain the high load or point us to the source code for your application, we want to check and confirm that the MDS is acting as it should.

Thanks, Tommy

____________________________________________________________________ Tommy Minyard, Ph.D. - Assoc. Director (512) 232-6578 Advanced Computing Systems Group (512) 475-9445 (fax) Texas Advanced Computing Center http://www.tacc.utexas.edu The University of Texas at Austin minyard@tacc.utexas.edu