Hi guys, I had some trouble running HOP in a batch job on Kraken. Specifically, it was crashing at line 443 in hop_regroup.c where it's trying to open a temporary file (although looking at tempnam(NULL) at line 413 it should be in memory, but that wasn't working). I've made a fix for it that works for me, but I believe other people have run yt-HOP on Kraken (Britton?), and have you had this kind of trouble? I don't want to commit this fix if it's my problem only. Thanks! _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Hi Stephen, oof! I thought I got rid of all the opening of temp files in yt-hop. They should all go away. If you are wary of committing the change, maybe you could submit the patch to codereview? http://permalink.gmane.org/gmane.comp.python.numeric.general/28816 this gives a short overview of the *extremely* simple process of doing so. What do you think? -Matt On Tue, Mar 17, 2009 at 2:21 PM, Stephen Skory <stephenskory@yahoo.com> wrote:
Hi guys,
I had some trouble running HOP in a batch job on Kraken. Specifically, it was crashing at line 443 in hop_regroup.c where it's trying to open a temporary file (although looking at tempnam(NULL) at line 413 it should be in memory, but that wasn't working). I've made a fix for it that works for me, but I believe other people have run yt-HOP on Kraken (Britton?), and have you had this kind of trouble? I don't want to commit this fix if it's my problem only.
Thanks!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Matt,
oof! I thought I got rid of all the opening of temp files in yt-hop. They should all go away. If you are wary of committing the change, maybe you could submit the patch to codereview?
http://permalink.gmane.org/gmane.comp.python.numeric.general/28816
I'm not sure the patch is what you want, Matt, I just wanted it to work to make sure that that was the problem. http://codereview.appspot.com/28055
this gives a short overview of the *extremely* simple process of doing so. What do you think?
It's interesting. I haven't made up my mind yet. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
I'm not sure the patch is what you want, Matt, I just wanted it to work to make sure that that was the problem.
This looks okay; so you construct a random int, and use that as the filename? Does it get removed later? I honestly haven't seen these issues before, but if we have multiple processors writing to the same file we *should* have seen them. Darn it. Any chance you can see a way to get rid of the scratch file completely? -Matt
Ah, of course -- the tmpname call should have handled that. But it wasn't, and perhaps it wasn't because CNL is not set up for temporary file names? It looks to me like tmpnam should also respect the environment variable TMPDIR; perhaps without this patch, we could try setting the TMPDIR variable to the current working directory? On Tue, Mar 17, 2009 at 2:46 PM, Matthew Turk <matthewturk@gmail.com> wrote:
I'm not sure the patch is what you want, Matt, I just wanted it to work to make sure that that was the problem.
This looks okay; so you construct a random int, and use that as the filename? Does it get removed later?
I honestly haven't seen these issues before, but if we have multiple processors writing to the same file we *should* have seen them. Darn it. Any chance you can see a way to get rid of the scratch file completely?
-Matt
Matt,
Ah, of course -- the tmpname call should have handled that. But it wasn't, and perhaps it wasn't because CNL is not set up for temporary file names? It looks to me like tmpnam should also respect the environment variable TMPDIR; perhaps without this patch, we could try setting the TMPDIR variable to the current working directory?
I can give it a shot.
This looks okay; so you construct a random int, and use that as the filename? Does it get removed later?
Yup, that temp file is removed, no problem.
I honestly haven't seen these issues before, but if we have multiple processors writing to the same file we *should* have seen them. Darn it. Any chance you can see a way to get rid of the scratch file completely?
Sure, one can store the stuff in an array. It appears that the file is created, written to, read from and removed all in the same function. But then we should ask ourselves why the file is being used in the first place, then? There may be a reason. Memory concerns? But from the looks of it the file shouldn't have very much in it. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
setting the TMPDIR variable to the current working directory?
I can give it a shot.
That's a negative ghostrider, the pattern is full. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Nuts. What precisely is the error? I'm leaning closer to saying go for it until we get the file out of there. On Tue, Mar 17, 2009 at 3:02 PM, Stephen Skory <stephenskory@yahoo.com> wrote:
setting the TMPDIR variable to the current working directory?
I can give it a shot.
That's a negative ghostrider, the pattern is full. _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Nuts. What precisely is the error? I'm leaning closer to saying go for it until we get the file out of there.
The error message here: http://yt.enzotools.org/browser/trunk/yt/lagos/hop/hop_regroup.c#L443 _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Okay, you've convinced me that the pragmatic solution here is to put something in now that helps out with Kraken; fire away. On Tue, Mar 17, 2009 at 3:10 PM, Stephen Skory <stephenskory@yahoo.com> wrote:
Nuts. What precisely is the error? I'm leaning closer to saying go for it until we get the file out of there.
The error message here: http://yt.enzotools.org/browser/trunk/yt/lagos/hop/hop_regroup.c#L443
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Stephen, I haven't actually run yt-hop on Kraken yet, but I should be doing so rather soon. Anybody else run it on there? Britton On Tue, Mar 17, 2009 at 3:21 PM, Stephen Skory <stephenskory@yahoo.com>wrote:
Hi guys,
I had some trouble running HOP in a batch job on Kraken. Specifically, it was crashing at line 443 in hop_regroup.c where it's trying to open a temporary file (although looking at tempnam(NULL) at line 413 it should be in memory, but that wasn't working). I've made a fix for it that works for me, but I believe other people have run yt-HOP on Kraken (Britton?), and have you had this kind of trouble? I don't want to commit this fix if it's my problem only.
Thanks!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ <http://physics.ucsd.edu/%7Esskory/> _.>/ _Graduate Student ________________________________(_)_\(_)_______________
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (3)
-
Britton Smith -
Matthew Turk -
Stephen Skory