Stumped on off-axis projections
by Matthew Turk
Hi all,
I'm writing a helper function for off-axis projections. I'm stuck on
deciding what to do for weighting. For a weighted projection, we
integrate field * weight across the whole domain. For on-axis, this
is well-defined. For off-axis, we have two choices of what to do,
because we interpolate between vertices:
* Construct a field that is everywhere field * weight, and
interpolate inside that to get our v's
* Interpolate field, interpolate weight, and multiply the two
interpolated values to get our v's
These are different operations, and can give very different results.
(You can test this very easily in 1D; I was surprised at the magnitude
of the differences.) I am not sure which one is more 'correct' for
our particular goal. I can actually see arguments for both sides.
The arguments in favor of #1 are that we are constructing a valid
field everywhere in advance, and it will be more stable. The
arguments for #2 are that it may more accurately reflect the local
value you would get compared to, say, a column density taken off axis
(which is what we ultimately divide by at the end of the calculation.)
Does anybody here have an opinion?
1D example code: http://paste.yt-project.org/show/1837/
-Matt
7 years, 2 months
Grid/yt data format meeting
by Matthew Turk
Hi all,
Casey, Cameron, Sam and I are going to be meeting -- likely utilizing
"Google Hangouts" -- to discuss the grid data format (Jeff has
suggested we just call it the 'yt data format,' which I am also down
for.) The meeting will be on Wednesday at 1:15PM Eastern. I'll
announce it again on Wednesday (with URL), but only in IRC, not the
mailing list.
We'll be discussing what improvements could be made to the existing yt
data format:
http://yt-project.org/gdf.txt
as well as Sam's (awesome) progress with importers/exporters, and
integration with the IRATE-format. I think we're all hoping this
could be a serialization format to take to the bank, so if you have
thoughts on interop or ground-up design of a data format, we'd
appreciate your participation.
-Matt
7 years, 2 months
Screencasts
by Cameron Hummels
Hey peeps,
So it turns out that a bunch of people have viewed the "how to use
reason" screencast that we put out last week. It seems like screencasts
may be a good way of introducing certain concepts in the yt universe,
because it really shows our audience exactly *how* to get certain
functionality to work (along with a rewind button!). With that in mind,
I wanted to see if anyone else wanted to create a screencast on some
topic in yt. I tried to create a "How to" on making a yt screencast on
the following page to help you out if you decide to make one.
Furthermore, Matt and I outlined several possibilities for topics on the
following page:
https://bitbucket.org/yt_analysis/yt/wiki/Screencasts
Topics include getting up and running, parallelization in yt, halo
finding/usage, volume rendering, time-series/movie-making, defining your
own field, etc. If you'd like to make one, feel free to volunteer and
put your name next to one of the topics on the wiki (it should be
editable by all), then just post one when you get a chance. We can put
it in our blog and update the site with news of it. This may be a good
way to supplement the documentation. Let me know if you have any comments!
Cameron
7 years, 2 months
Reason crash
by John Wise
Hi everyone,
I'm trying to use Reason, and it's crashing when I try to create a
projection or slice. Viewing the grid data works perfectly fine.
I'm using a version that's installed without the install script. I'm
using a similar set of dependencies as the install script (Python 2.7.1,
h5py 1.3.1, the ext* and PhiloGL files from enzotools.org) Everything
else has been working fine with yt.
Here's the traceback:
-----
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
self.run()
File
"/usr/local/lib/python2.7/dist-packages/yt-2.3dev-py2.7-linux-x86_64.egg/yt/gui/reason/extdirect_repl.py",
line 123, in run
task['name'], task['widget_data_name'])
File
"/usr/local/lib/python2.7/dist-packages/yt-2.3dev-py2.7-linux-x86_64.egg/yt/gui/reason/extdirect_repl.py",
line 451, in _add_widget
payload['data'] = self.locals[widget_data_name]
KeyError: '_twidget_data'
-----
When I inspect the task variable, it has these fields.
(Pdb) p task
{'widget_data_name': '_twidget_data', 'type': 'add_widget', 'name': '_tpw'}
Here is my python session with me briefly looking around with pdb, but
I'm not familiar enough with reason to properly debug this. I also
tried reducing max_threads to 1, and it didn't have any effect.
http://paste.enzotools.org/show/1828/
If you want to try to replicate this error, I've uploaded the dataset
(50MB), but I have a feeling that it's not dataset's fault.
http://www.physics.gatech.edu/~jw254/scpics/DD0036.tgz
Has anyone had this error before?
Thanks!
John
--
John Wise
Assistant Professor of Physics
Center for Relativistic Astrophysics, Georgia Tech
7 years, 2 months
Re: [Yt-dev] Nautilus: ParallelHaloProfiler I/O issue
by Geoffrey So
Hi Stephen,
I was running parallelHF on Nautilus when I got the following email from the
sys admins, I'll try to answer the questions but let me know if I'm wrong or
there's something else to be said.
(1) How much data are read/written by program ?
- After all the particles (3200^3 of them) are read in they are linked with
a fortran KDtree if they satisfy some conditions.
(2) How many parallel readers/writers are used by program ?
- It is reading using 512 cores from my submission script. The amount of
write to disk depends on the distribution of the particle haloes across
processors, if they exist across processors then there will be more files
written out by write_particle_lists.
(3) Do you use MPI_IO ? or something else ?
- Yes, the program uses mpi4py-1.2.2 installed in my home directory
The details of the code can be found at:
http://yt-project.org/doc/analysis_modules/running_halofinder.html#halo-f...
under the section "Parallel HOP"
Currently I am using 512 cores with 4GB per core for a total of 2TB of ram
for this 3200 cube unigrid simulation, should I decrease the amount of
processors but keeping the same amount of ram? Or is there other ways to
optimize as to not affect other users?
Sorry for the inconvenience.
From
G.S.
On Fri, Sep 23, 2011 at 9:12 AM, Patel, Pragneshkumar B <pragnesh(a)utk.edu>wrote:
> Hello,
>
> We have noticed some I/O issue on Nautilus. We suspect that
> "ParallelHaloProfiler" program is doing some very I/O-intensive operations
> periodically (or may be checkpoint). We like to throttle these back a bit ,
> so that other users are not affected by it.
>
> I like to get some more information about your job #60891. E.g.
>
> (1) How much data are read/written by program ?
> (2) How many parallel readers/writers are used by program ?
> (3) Do you use MPI_IO ? or something else ?
>
> Please give me detail and we will work on your I/O issue.
>
> Thanks
> Pragnesh
>
>
7 years, 2 months
possible halo mass function bug
by Geoffrey So
I was running an old script to refresh my memory of how the halo mass
function is used in YT, and it just siphoned all the memory out of my laptop
on a tiny dataset.
As I recalled I was able to test the halo mass function extensively on my
laptop with this 64 cube unigrid run. Thinking memory is the issue (mine or
my laptop's), I then tried to run the script on triton with the TCC node, it
still hangs and runs out of memory, I then tried the PDAF node and got the
same results even after updating to the latest dev (22ac529a1dab (yt) tip).
link to the script:
http://paste.enzotools.org/show/1815/
Or is there something obviously wrong with the script?
From
G.S.
7 years, 2 months
Workshop IRC Meeting Summary
by Matthew Turk
Hi all,
Today we had a meeting in IRC to discuss a potential 2012 yt workshop.
I think it was very successful, and rather than having long,
high-turnover discussions over email it seems like IRC would be a
better venue. I've summarized the important points below, and the
full logs can be found online at:
http://paste.yt-project.org/show/1807/
The important points:
1) The FLASH center has *very* graciously offered to host us in
Chicago, thanks to John ZuHone's efforts.
2) We are currently exploring the following dates, in order: Jan
23-26, Jan 31-Feb 3, or Feb 6-Feb 9. JohnZ will be looking into
availability of hotel blocks.
3) A draft agenda has been posted here:
https://docs.google.com/document/d/1gNGifqqLnu3x4DYVzM-EpYjriqIWraV-ZJHdw...
which is available for commenting. Please leave comments if you have
them, using the 'commenting' feature.
4) If anyone has any ideas about potential funding sources, please
contact Stephen Skory or me.
5) We'll have presentations from some local (UC/FLASH/ANL) HPC experts as well.
The current action items:
1) Figure out hotel situation with candidate dates (JZ)
2) Continue iterating on agenda (Britton, everyone else)
3) Investigate funding (SS, anyone else that has leads)
Please feel encouraged to reply to this list with any thoughts you
have. As new items develop we'll continue updating this list, and
hopefully we will be able to begin advertising it and shoring up
participants/support soon.
Best wishes, and thanks to everyone who participated! This workshop
should be fun, and I'm really excited about the energy I was getting
in the chat room -- about topics, development, learning, sharing code,
everything.
-Matt
7 years, 2 months
Meeting for yt Workshop Discussion
by Matthew Turk
Hi all,
There are a number of exciting things to share and discuss about the
possibility of having a yt workshop in early 2012. The discussion
here and on yt-users has been fast and furious. To avoid that
high-throughput for discussing several items that need to be discussed
with some rapidity, Britton and I would like to call an IRC meeting.
If you are interested in participating in organizing (and we hope that
we can count on participation from everyone who has participated in
the discussion thus far) please help us pick a time by filling out
this doodle *by the end of the day Saturday*.
http://www.doodle.com/frwnhzis5na8g5xi
One of us will announce by Sunday morning through this medium when it
will occur. The meeting will be held in #yt on irc.freenode.org. You
can access this either through web-chat:
http://yt-project.org/irc.html
or through your IRC/IM client. We will keep a log of the discussion,
which one of us will post to this list when concluded. The following
items will be discussed:
* Venue (and there is some exciting news on that front!)
* Dates
* Overall structure
* Agenda (curriculum?)
* Reaching out to other interested parties
* Other items
Thanks very much,
Matt
7 years, 2 months
ellipsoid part 2
by Geoffrey So
Hello Stephen,
This is part 2 of the ellipsoid parameter finding. The method is outlined
mainly in the Dubinski and Carlberg 1991 paper. But since we are doing this
in YT and we already know the particle list to begin with, I put in the
option that the user can specify if they want all the particles in the
region to be taken into account or only the particles found by the halo
finder algorithm. This is an option that I think should be amended to the
first ellipsoid parameter finder.
The script form is linked here:
http://paste.enzotools.org/show/1782/
Once I have written out the methodology for myself, I'll work on the YT
documentation, will probably need some help in this area when it comes to
it.
Doron Lemez has used this method in his paper and helped me tested the
accuracy of this script. I was wondering do I need to ask him if he wants
his name in the credits, or just mention his name as a special thanks? One
thing he did in the paper that was especially useful is defined the starting
sphere to have 2x the virial radius. Without that specification, I run into
non-convergence sometimes, where after many iterations I end up with no
particles. And I want to mention again that there's no guarantee that any
or all the haloes will converge with this method.
I tried to adhere to the PEP8 formatting, but old habits die hard so I'm
sure I made mistakes in the formatting, let me know if you see them.
From
G.S.
7 years, 2 months
AAS splinter meeting
by david collins
I emailed the contact person for setting up a splinter meeting. It
sounds like this is an avenue open to us, and has some potential
benefits. I kind of like the walk-in demo space idea, but that's
probably outside of the scope we were looking for right now. They
charge a a few hundred for the room, but that's likely going to be the
same if we rent a conference room at a hotel. I'm definitely not
married to the idea, so if y'all would rather do somewhere or somewhen
else that's also grand.
d.
David,
It sounds like a splinter meeting is exactly what you would need.
Splinter meetings can be private (invitation only), or public, and
public splinter meetings will be listed in the online meeting
materials as well as in the printed meeting program. Anyone that is
registered for the meeting may attend your splinter meeting.
There is no set format for splinter meetings, it is kind of a "catch
all" term we use for anything that is not specifically science session
related. We can schedule invitation-only splinter meetings in any time
slot you would like, but we try not to schedule public splinter
meetings against our Plenary Invited talks (8:30am, 11:40am, 3:40pm,
and 4:30pm each day). An exception to this would be if you wanted to
have a "open-house" kind of set up all day, where people could come in
and out at their leisure to test your software--we could schedule that
for a long block of hours with no problem. It also sounds like you
might need more than one format--maybe a walk-in space for demos, and
a smaller meeting space for developers and private meetings.
Either way, we can accommodate you. You can fill out the form
available at http://aas.org/meetings/splinter_proposal_form.php, or
you can send the details to me and we can figure out what will work
best. The splinter form also has most of our AV pricing, for
reference. I can take AV orders and catering orders (if applicable) as
well.
Thanks,
Lisa Idem
Meetings Manager
American Astronomical Society
202-328-2010 ext. 126
On Thu, Sep 8, 2011 at 4:50 PM, David C. Collins <dccollins(a)lanl.gov> wrote:
Hi there--
I have a question about Splinter Meetings, specifically for the
January 2011 meeting in Austin. I'm trying to organize a meeting for
a software package yt, which designed to analyze and visualize
astrophysical simulation data. The meeting we're planning is a
combination of new user instruction and developer meeting to discuss
recent work that's been done from our various contributors, as well as
future development ideas (with the goal of blurring the line between
new users and developers). The AAS meeting is an ideal venue, as so
many of us will be going already, and it's a good way to attract new
users. The workshop application deadline has passed, but I was
wondering if a splinter meeting might be an appropriate session type
for us. What is the general setup for this kind of session, and do
you think it would be adaptable for our needs?
Thanks!
David Collins.
--
Sent from my computer.
7 years, 3 months