Hi all,
Last August we announced the creation of the yt Hub, at
hub.yt-project.org, to replace the ailing Barn project. It was
somewhat successful; it received about 30 submissions, which is 30
more than we had before it was announced. Rick Wagner emailed me
Sunday morning to let me know he'd been able to find a considerable
amount of Spam on the site; while we had put in restrictions keeping
it from going to the front page, or being "published" and sent through
the RSS Feed / email subscription, it was still possible to find them
while searching the site. Britton spent some time clearing it out,
and it turns out that in total we had 10,000 spam submissions on the
entire site. I attribute most of the traffic to clever google
searches on the part of spammers -- the strings used on the hub come
from the original software, pligg, which was vulnerable to spam.
Back in March, I received a grant from Amazon in Education to launch a
data pastebin, which has been happily up and running -- although
underused, as it hasn't been announced properly. Hooks into it have
been inserted into the development branch, will undergo a tad bit of
polish before 2.4, and will be played up in the 2.4 release cycle.
(For instance, PlotCollection objects are already uploadable, along
with projections, and slices. We have a vertex-display method, but
uploads are not nice yet.) It's kind of neat, but only time will tell
if this fits a niche for people. I've slowly been adding on new
stuff, and the next item is going to be "named pastes" -- something
that we've kind of wanted for a while, but never implemented.
Anyway, the convergence of these things has led us to try to implement
project listing in the data hub. This will be much less accessible to
spammers, as the only upload mechanism will be via API, rather than
web form. We'll also have substantially more control over the
presentation, methods of interaction, and the entire software stack.
This brings with it some challenges, but I believe we can overcome
them.
As a first step, I've imported all the projects from the old hub, as
well as the users for those projects. (You can regain access by
resetting your password; your passwords are all set to random values.)
This first step is here:
https://data.yt-project.org/
but I would like to migrate it to hub.yt-project.org and unify both
offerings. Project URLs will remain functional, although through
redirects. Note that the getting started instructions require a
change I implemented in development tip this afternoon.
I'd like to solicit feedback on this:
1) What suggestions do you have for views, style, appearance, etc?
2) What problems do you see as is?
3) What needs to be added?
For what it's worth, I'm working to add editing of submitted projects,
to refine the yt mechanism for submitting new projects (right now it's
non-functional), and to add more documentation about how to use the
hub. The repo is hosted (publicly) on BitBucket under my account as
yt.hub, so if you want to help out feel free to dive in.
As a word of warning, as I fiddle with things and add new things,
there may be interruptions in service. Let me know off-list if you
think you see one that's long-lived or broken.
Thanks,
Matt
This might be of interest to people here. Congratulations to Patrik
and the other developers! This is a very impressive accomplishment.
---------- Forwarded message ----------
From: Patrik Jonsson <sunrise(a)familjenjonsson.org>
Date: Wed, May 9, 2012 at 1:56 PM
Subject: [Sunrise] Sunrise v4.0 release!
To: sunrisemcrx(a)googlegroups.com
Hi Sunrise users,
I'm pleased to announce the release of the long-in-the-making Sunrise v4.0!
This new version includes many updates, and a paper describing them is
in the works. (However, it will likely be a while until this paper is
done, for reasons that will become obvious below.) Special thanks
should go to Chris Moody and Greg Snyder, who have done a lot of work
testing the new versions and finding bugs. You can get the version now
by using the tag "v4_00" in the Mercurial repository.
One of the most exciting ones is the support for distributed-memory
machines using MPI. This means that you no longer will have to go
hunting for machines with extremely large memory to run Sunrise on
large simulations. (However, the overhead for running MPI is quite
high, so it is not useful for speeding up small calculations.) See the
"Using MPI" Wiki page (http://code.google.com/p/sunrise/wiki/UsingMPI)
for more information. The MPI functionality works but has not been
extensively tested, so you are likely to run into bugs. Please report
any issues that arise.
A prerequisite for getting the MPI version to work was to redo the way
the adaptive mesh works. The grid is now a pure octree, meaning it
always starts with one root cell that is then recursively subdivided
into octants. Before, the root grid was a cartesian grid of arbitrary
size, but that is no longer supported. (This was necessary to get the
Hilbert ordering of the cells used for domain decomposition working
efficiently.) This new grid is also more efficient than the old
version.
The calculation of infrared emission has also been redone. When
finding the equilibrium dust temperatures/radiation fields, it now
uses the "immediate reemission" method described in Fleck & Cummings
(1971) and Lucy (1999). This exactly preserves the energy balance when
grains absorb/reemit photons, and discretizes only the temperature
field (not the radiation intensity) on the grid when iterating, which
should give better accuracy at a given grid resolution.
Finally, instead of MC sampling the emission distribution in the cells
when making images of dust emission, the direct emission is now
calculated by explicitly integrating the RTE along the line of sight.
This results in nose-free images. The scattered light is still done
with MC, but at the wavelengths where the dust emission dominates, the
albedo is essentially zero anyway so this doesn't make much
difference. The integration is adaptive in the sense that the size of
the cells encountered are monitored, and if the cells are smaller than
the solid angle subtended by the ray, the ray is subdivided (similar
to what Abel & Wandelt 99 did, but for incoming rays as opposed to
outgoing ones).
Furthermore, an explicit monitoring of Monte Carlo error in the
iteration, which adjusts the number of rays to achieve the specified
accuracy, has been included. The previous method of testing
convergence did not guarantee an accurate temperature field, only that
the overall energy balance was within a certain limit. The new method
monitors the error in determining the temperature of each individual
cell. The flipside is that if you ask for a high accuracy, you'd
better be prepared to wait a *long* time. The full force of the
sqrt(N) Monte Carlo convergence rate will hit you.
The polychromatic ray tracing algorithm has been upgraded with an
automatic determination of reference wavelength and forced scattering
positions to avoid large bias factors in situations with high optical
depths. Instead of splitting rays that have been boosted to high
intensities after the fact, it avoids large bias factors in the first
place by forcing several scatterings along the line of propagation.
This lowers the noise caused by the large bias factors and avoids the
guesswork in setting the reference wavelength and forced scattering
parameters.
There are also many efficiency tweaks that have sped up the
calculation. (However, the most important factor affecting runtime, by
far, is the dust equilibrium accuracy required.) The code is capable
of processing snapshots from Gadget, Gasoline, Arepo, and, using YT to
export files, from Enzo and ART. If you are interested in the YT
functionality, contact Matt Turk and he can point you in the right
direction. I don't know anything about how the YT export works.
Finally, Sunrise now handles kinematics, meaning that it can take into
account the effect of velocity and Doppler shift of sources and
scatterers. However, for this to work you need a SED library with
logarithmic wavelength spacing (and for it to be useful, quite high
resolution). A high-resolution set of Starburst99 and MAPPINGS files
have been uploaded to the Downloads section of the Sunrise web site.
Note that with such high spectral resolution, maintaining high spatial
resolution comes at the cost of *very* large memory use and output
file size. It's best to cut out a small wavelength range around the
features you are interested in. (And don't even think of calculating
dust emission with that wavelength resolution...)
For those of you who are interested in knowing more about what's
happening inside the galaxy, there are also now options to save the
dust emission SEDs and the dust temperatures to the output file.
All these changes naturally means that parameter keywords have
changed. I'll post another message containing example configurations,
but I encourage you to carefully read through the Wiki pages
describing the keywords to get an idea of the changes. As usual, you
have to take responsibility for verifying that the results make sense,
so don't just take the example files without actually verifying that
things work correctly in your particular situation. The library
requirements have also changed, so review the compilation instructions
and be sure to update your copy of libPJutil from the repo at
https://bitbucket.org/lutorm/libpjutil.
The other big update is that I will no longer be working primarily on
Sunrise. Next week, I'm moving to a position as software engineer at
SpaceX in LA, working on simulation software for rockets! :-) I will
still have my research grant for the next few years, but I'm unlikely
to have much time to work on astrophysics. Chris Hayward, who has
worked a lot with Sunrise, has graciously accepted taking over as the
primary maintainer of the code. However, everyone will have to help
Chris by pitching in with tracking down bugs you find, and with
providing improvements. You are still welcome to email me with
questions, but I should probably not be the first choice as my ping
time is likely to be very long.
I hope that all the effort I've put into Sunrise over the years will
continue to be built on so Sunrise will remain the premier tool in
galaxy SED prediction!
Happy Monte-Carloing,
/Patrik
--
You received this message because you are subscribed to the Google
Groups "Sunrise" group.
To post to this group, send email to sunrisemcrx(a)googlegroups.com.
To unsubscribe from this group, send email to
sunrisemcrx+unsubscribe(a)googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sunrisemcrx?hl=en.
Chris submitted a pretty sizable pull request -- congrats, Chris! I
am going to review this, but I'd like to invite others to do so as
well.
---------- Forwarded message ----------
From: Christopher Moody <pullrequests-noreply(a)bitbucket.org>
Date: Thu, May 3, 2012 at 1:01 AM
Subject: [yt_analysis/yt] New ART frontend and new Sunrise export,
expanded Rockstar (pull request #150)
To: matthewturk(a)gmail.com
A new pull request has been opened by Christopher Moody.
juxtaposicion/yt has changes to be pulled into yt_analysis/yt.
https://bitbucket.org/yt_analysis/yt/pull-request/150/new-art-frontend-an...
Title: New ART frontend and new Sunrise export, expanded Rockstar
ART Frontend now uses Hilbert-patch coalescing a la the RAMSES
frontend. Particle support included.
Sunrise export is completely revamped, works with the new MPI Sunrise
and it's Hilbert-ordering convention.
Expanded some Rockstar options and expanded the HaloList class so that
we can plot 'HOP' circles for halos found via yt.
Changes to be pulled:
096be56abb5c by Christopher Moo…: "added back Msun to data_structures"
60d0eae312ac by Christopher Moo…: "reverted a lot of the last commit"
33e1017fc58a by Christopher Moo…: "something is wrong with the ART
frontend units"
1503e0ae9825 by Christopher Moo…: "hilbert frameworks works; wrong
ordering though"
db3a9b59d37c by Christopher Moo…: "merge"
42485223eca7 by Christopher Moo…: "fixes in rockstar halo classes.
allowing ART frontend to be dm only"
41bf558abeaf by Christopher Moo…: "merged"
629ca6b1128f by Christopher Moo…: "implemented rockstar halo and halo
list classes"
8512e594a837 by Christopher Moo…: "updated to tip"
6cc606a417a7 by Christopher Moo…: "more rockstar options. fixed ART
particle masses"
fecd8f0251c2 by Christopher Moo…: "removing pdb trace"
26ad4e98e6f7 by Christopher Moo…: "fixed particle positions to unitary units"
05d7dd36866a by Christopher Moo…: "fixes to ART particles"
eb9e749e7253 by Christopher Moo…: "added the ability to pass DM
particle mass directly"
c309f591b93f by Christopher Moo…: "added Rockstar halo class"
91a9d9d9e7d4 by Christopher Moo…: " merged with tip"
a3bc652df01e by Christopher Moo…: "changed le to le-1"
f42fa77f5077 by Christopher Moo…: "changed LE for ART. works!"
053ae2cf5a3a by Christopher Moo…: "fixed temp field. commented out child mask."
0834402f6cde by Christopher Moo…: "reorganized the hydro fields"
549c408579ef by Christopher Moo…: " fixed a bug with particle arrays
being the wrong length. affected stars."
4433494b6bba by Christopher Moo…: "implemented child masks. Broken!
causes a segfault"
761a9e79cf93 by Matthew Turk: "Making regridding more strict and
closer to what we want for the actual
selectio…"
d03f9f1da6f3 by Matthew Turk: "Adding a progressbar and a better
parentage calculator to the ART frontend."
12223d9fef86 by Christopher Moo…: "particle field definitions"
69fee55a453e by Christopher Moo…: "added a few more options to patch splitting"
a97c1ad2ec24 by Christopher Moo…: "fixed so that all fields are fully
loaded in on preload. previously, only up to …"
210c6d907441 by Christopher Moo…: "particle fields can be accessed"
895621adae33 by Christopher Moo…: "added time conversion to data_structure"
858542457124 by Christopher Moo…: "particle fields still not working."
9ee69e4f02cd by Christopher Moo…: "added standard le, active dim
attributes to art grids"
b33d607ce6ec by Christopher Moo…: "particles read in and indices
gridded. fields not setup yet."
e3a1f76ae1c5 by Christopher Moo…: "forced 32 bit numpy arrays for
IO-cython compatibility"
0fcc360c4d58 by Christopher Moo…: "first pass at particle inclusion.
change recursiv splitting to start from the mi…"
d202aaa72971 by Christopher Moo…: "including particle headers"
26eb767bac7f by Christopher Moo…: "bug fixes to io. added new
parameters and units to data structures."
98534589dd52 by Christopher Moo…: "new io is less memory intensive.
seems to work too."
a1e85e8f0816 by Christopher Moo…: " reduced memory"
dd43dbad7df5 by Christopher Moo…: " working version of art reader.
faster than the first pass, uses matt's hilbert …"
4c7fcdbca996 by Christopher Moo…: "bug fixes in the info messages"
b1cb776fed6f by Christopher Moo…: "adding log info. starting on memory
optimization; loading left edges by level."
d4d82426e66c by Christopher Moo…: "fixed a bug in the hilbert grad
patching a bit. added lots of comments."
ca8b7105e354 by Christopher Moo…: "implemented matt's hilbert-ordered
patch-splitting"
8ae85483aad1 by Christopher Moo…: "fixing root grid size. projection
of Ones field works."
41c966d5bcdc by Christopher Moo…: "record arrays overflow silently.
changed the numgrid calculation to 64bit."
45c690fab939 by Christopher Moo…: "special exception for the root grid"
--
This is an issue notification from bitbucket.org.
You are receiving this either because you are the participating
in a pull request, or you are following it.
Hi all,
I've updated hg to 2.2 in the install script. This brings with it "hg
commit --amend", moves "hg glog" into core and comes with tons of
performance improvements. More details:
http://mercurial.selenic.com/wiki/WhatsNew#Mercurial_2.2_.282012-05-01.29
-Matt
---------- Forwarded message ----------
From: Bitbucket <commits-noreply(a)bitbucket.org>
Date: Wed, May 2, 2012 at 8:17 AM
Subject: [yt-svn] commit/yt: MatthewTurk: Updating mercurial to 2.2. See:
To: yt-svn(a)lists.spacepope.org
1 new commit in yt:
https://bitbucket.org/yt_analysis/yt/changeset/6eed6c70ed21/
changeset: 6eed6c70ed21
branch: yt
user: MatthewTurk
date: 2012-05-02 14:17:30
summary: Updating mercurial to 2.2. See:
http://mercurial.selenic.com/wiki/WhatsNew#Mercurial_2.2_.282012-05-01.29
for more details.
affected #: 1 file
diff -r 3e1f218938d83de6a169837549eee92ec0e763a5 -r
6eed6c70ed21b25fed963d934705e159d9d4e054 doc/install_script.sh
--- a/doc/install_script.sh
+++ b/doc/install_script.sh
@@ -366,7 +366,7 @@
echo 'ffc5c9e0c8c8ea66479abd467e442419bd1c867e6dbd180be6a032869467955dc570cfdf1388452871303a440738f302d3227ab7728878c4a114cfc45d29d23c
ipython-0.12.tar.gz' > ipython-0.12.tar.gz.sha512
echo 'e748b66a379ee1e7963b045c3737670acf6aeeff1ebed679f427e74b642faa77404c2d5bbddb922339f009c229d0af1ae77cc43eab290e50af6157a6406d833f
libpng-1.2.43.tar.gz' > libpng-1.2.43.tar.gz.sha512
echo 'f5ab95c29ef6958096970265a6079f0eb8c43a500924346c4a6c6eb89d9110eeeb6c34a53715e71240e82ded2b76a7b8d5a9b05a07baa000b2926718264ad8ff
matplotlib-1.1.0.tar.gz' > matplotlib-1.1.0.tar.gz.sha512
-echo '78715bb2bd7ed3291089948530a59d5eff146a64179eae87904a2c328716f26749abb0c5417d6001cadfeebabb4e24985d5a59ceaae4d98c4762163970f83975
mercurial-2.0.tar.gz' > mercurial-2.0.tar.gz.sha512
+echo '702f67c48e4dbe191dbe5ca0df6b5a84fa4f5c424cf1fae60b5053dfe6532531330738c7aa3012d900d49efdd743cd1ebc238bb15f354f67228e2a2c95b98a89
mercurial-2.2.tar.gz' > mercurial-2.2.tar.gz.sha512
echo 'de3dd37f753614055dcfed910e9886e03688b8078492df3da94b1ec37be796030be93291cba09e8212fffd3e0a63b086902c3c25a996cf1439e15c5b16e014d9
numpy-1.6.1.tar.gz' > numpy-1.6.1.tar.gz.sha512
echo '5ad681f99e75849a5ca6f439c7a19bb51abc73d121b50f4f8e4c0da42891950f30407f761a53f0fe51b370b1dbd4c4f5a480557cb2444c8c7c7d5412b328a474
sqlite-autoconf-3070500.tar.gz' >
sqlite-autoconf-3070500.tar.gz.sha512
echo 'edae735960279d92acf58e1f4095c6392a7c2059b8f1d2c46648fc608a0fb06b392db2d073f4973f5762c034ea66596e769b95b3d26ad963a086b9b2d09825f2
zlib-1.2.3.tar.bz2' > zlib-1.2.3.tar.bz2.sha512
@@ -388,7 +388,7 @@
get_enzotools Python-2.7.2.tgz
get_enzotools numpy-1.6.1.tar.gz
get_enzotools matplotlib-1.1.0.tar.gz
-get_enzotools mercurial-2.0.tar.gz
+get_enzotools mercurial-2.2.tar.gz
get_enzotools ipython-0.12.tar.gz
get_enzotools h5py-2.0.1.tar.gz
get_enzotools Cython-0.15.1.tar.gz
@@ -531,7 +531,7 @@
if [ $INST_HG -eq 1 ]
then
echo "Installing Mercurial."
- do_setup_py mercurial-2.0
+ do_setup_py mercurial-2.2
export HG_EXEC=${DEST_DIR}/bin/hg
else
# We assume that hg can be found in the path.
Repository URL: https://bitbucket.org/yt_analysis/yt/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
_______________________________________________
yt-svn mailing list
yt-svn(a)lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-svn-spacepope.org
This has been discussed a lot in the past.
---------- Forwarded message ----------
From: Matthew Turk <pullrequests-noreply(a)bitbucket.org>
Date: Tue, May 1, 2012 at 3:03 PM
Subject: [yt_analysis/yt] Raise an exception instead of returning None
when load fails. (pull request #149)
To: yt(a)enzotools.org
A new pull request has been opened by Matthew Turk.
MatthewTurk/yt has changes to be pulled into yt_analysis/yt.
https://bitbucket.org/yt_analysis/yt/pull-request/149/raise-an-exception-...
Title: Raise an exception instead of returning None when load fails.
Changes to be pulled:
--
This is an issue notification from bitbucket.org.
You are receiving this either because you are the participating
in a pull request, or you are following it.
