Hacking guidelines
by Matthew Turk
Hi guys,
I put up a wiki page today with some stuff I wrote up on the bus this morning:
http://yt.enzotools.org/wiki/HackingGuidelines
It's the first iteration, but I thought it would be a good way to
start codifying what we want in terms of contributions. I have been
following some discussions on other lists, where people are talking
about contributions, and the relative impact of guidelines on the
quality and quantity of contributions; with a release coming up (1.5,
maybe May? maybe in conjunction with the Barn?) that I think will
turn yt from a plotting tool into a full-fledged analysis and
visualization solution, I thought we should start soliciting and
encouraging contributions and bug fixes. These would also be great
guidelines for us to follow as well; I am guilty of not realizing
these ideals quite as well as I should, so now I can be publicly
shamed. :)
I think you all have wiki accounts, so if I've left something out you
should feel free to fix it up.
-Matt
10 years
parallel profiler problem
by Stephen Skory
Hi Britton & Matt,
first of all, thanks very much for working on the parallel halo profiler! I appreciate it very much.
I am having some trouble running it on a L7 dataset. Specifically, I am getting the error below at all processor counts (this is a -np 1 run, rev 1226):
http://paste.enzotools.org/show/83/
It's getting late so I'll take another whack at it tomorrow morning. I'm sending it out on the chance that one of you might have some insight.
Thanks!
_______________________________________________________
sskory(a)physics.ucsd.edu o__ Stephen Skory
http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student
________________________________(_)_\(_)_______________
10 years
debugging decorator
by Matthew Turk
Hi guys,
I just added a new decorator in r1229. This is the 'run_pdb'
decorator, which will drop you to an interactive pdb shell before
calling the function that it is applied to. For instance, if I
applied it to, say, _is_fully_enclosed inside BaseDataTypes, whenever
that function is called I get a pdb prompt. (
http://docs.python.org/library/pdb.html ) This enables FAR easier
debugging. (You can do the same thing by inserting pdb.set_trace() at
any point in the code, but this way yt will handle the imports for
you.)
-Matt
10 years
yt-hop configuration improvements?
by Stephen Skory
Hi guys,
I had an email exchange with Brian a few days ago about trying to reproduce some data he made on L7. In particular, he gave me some settings for HOP hop.c and regroup.c that cannot be changed using the current setup. Happily, I was able to reproduce the list of haloes he found using his three step serial process with parallel yt-hop once I made some hard-coded changes to HOP.
If we want to have as many options as the serial version Brian used, there would have to be about 10 optional inputs to yt-hop, which I know will kind of go against the aesthetic that Matt has for the current yt-hop. Unless there are any questions or protests, I'll try to add these options tomorrow.
_______________________________________________________
sskory(a)physics.ucsd.edu o__ Stephen Skory
http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student
________________________________(_)_\(_)_______________
10 years
minimum halo mass
by Matthew Turk
Hi guys,
We never came to an agreement. I think we need to decide on some
padding-calculator, and then call that good and wrap SS_HopOutput.py
back into the main source file. I really, really think somebody
besides Matt should come up with this. (Since my halos are all 10^6
M_sun and I only care about the biggest one!)
Can we come up with something? Until we do so, the parallel halo
finder is just going to be sitting on the sidelines, and I'd really
like it to be moved in as the primary mode of halo finding.
-Matt
10 years
Auto-registration of callbacks and change in Callback API
by Matthew Turk
Hi guys,
I've been helping some people learn to use yt lately, and it is
becoming apparent to me that the plotting API might need some work.
I'm mostly okay with the way the PlotCollection stuff is handled (but
see http://yt.enzotools.org/ticket/158 for more on that) but the
callbacks specifically are not terribly intuitive. The current API
is:
p = pc.add_projection(...)
p.add_callback(ContourCallback("Temperature"))
Currently the Callback metaclass auto-registers each callback
(http://yt.enzotools.org/browser/trunk/yt/raven/Callbacks.py#L38) so
that they get imported automatically in yt.mods
(http://yt.enzotools.org/browser/trunk/yt/mods.py#L55) but this can be
done in a substantially more straightforward fashion. (However, the
AMRData subclasses are also nearly auto-registering, which is
something I intend to explore a bit more in the future.)
I'd like to propose that each callback be given a static attribute
_type_name -- i.e., "contour", "quiver" etc, and then this be used to
access that particular callback type. I'm torn, however, on how this
should be handled. There are three main ways I've considered for
implementing this:
==
1. Through a .modify(type_name, *args, **kwargs) command. You would then do:
p.modify("contour", "Temperature")
for instance, and it would add that callback.
This has the downside that, while we could provide docstrings for all
the possible callbacks, we would have to implement some introspection
to do so. Not a huge deal, but somewhat annoying.
2. Through dictionary-style lookup, similar to DerivedQuantities:
p.modify["contour"]("Temperature")
this would allow us to have docstrings without any introspection.
3. Through the current style used for generating data objects from the
hierarchy:
p.contour("Temperature")
This would also give us docstrings for free.
==
What do you think? The first appeals to me the strongest for
aesthetic reasons, but the third appeals to me on the grounds of ease
of use and understanding. Does anyone have any strong feelings? This
would all be handled via an auto-registry situation, so the means of
adding new callbacks would change, but only in that they would all
require _type_name . Additionally, I see no reason to get rid of the
current p.add_callback(...) method, which is more general that we're
looking at here.
-Matt
10 years
Ghetto Parallel Halo Profiler Report
by Stephen Skory
Hi guys,
You may find this interesting.
Yesterday I did two runs of the halo profiler on L7. I took the full HopAnalysis.out file and split it up into 16 files in a round-robin fashion, and also in a spatial decomposition. I then ran with 16 separate parameter files for these 16 separate HOP files like this (for each setup):
from mpi4py import MPI
if MPI.COMM_WORLD.rank == 0:
q = HP.HaloProfiler(blah, 'round-robin01.par')
q.makeProfiles()
if MPI.COMM_WORLD.rank == 1:
...'round-robin02.par'
q.makeProfiles()
...
...
MPI.COMM_WORLD.Barrier()
The round-robin took a little under ten hours to finish, while the spatial decomposition took closer to eleven hours. This is because the numbers of haloes per task wasn't as balanced in the spatial case. The haloes per minute per process was more or less identical for both methods.
_______________________________________________________
sskory(a)physics.ucsd.edu o__ Stephen Skory
http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student
________________________________(_)_\(_)_______________
10 years
HOP on Kraken XT5
by Stephen Skory
Hi guys,
I had some trouble running HOP in a batch job on Kraken. Specifically, it was crashing at line 443 in hop_regroup.c where it's trying to open a temporary file (although looking at tempnam(NULL) at line 413 it should be in memory, but that wasn't working). I've made a fix for it that works for me, but I believe other people have run yt-HOP on Kraken (Britton?), and have you had this kind of trouble? I don't want to commit this fix if it's my problem only.
Thanks!
_______________________________________________________
sskory(a)physics.ucsd.edu o__ Stephen Skory
http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student
________________________________(_)_\(_)_______________
10 years
smoothed plots
by david collins
Hi, All--
Is it possible to smooth the image I get from plot_collection.add_projection?
Thanks,
d.
10 years
Matplot lib glitches: long underscore, linestyle
by david collins
Hi--
I've been making a bunch of plots these days, some of them using matplotlib.
I've noticed two strange glitches when making eps files with it.
In both cases, the plots are made just fine, the eps files look good
when you open them with gv/preview. But when you try to make them
into a latex document, the document builds fine, but dvipdf hangs
indefinitely.
First, if you make a histogram with fill=False and linestyle='dashed'
The second is if you try to include long text in a subscript in a
caption. So I had
plt.scatter(stuff, label='\rho_{truelove}')
and it caused problems.
I haven't sorted it out, but I figure most of y'all will be using
matplot lib to put things into tex at some point.
d.
10 years
