Mailman 3 January 2013 - yt-dev

Re: [yt-dev] [enzo-dev] testing: current gold standard on Amazon
by Michael Kuhlen Jan. 9, 2013

Jan. 9, 2013

Hi guys [yt-dev cc'd] Thanks for looking into this. I've determined that the errors occur because of a change in yt. Changeset 6d71f5517955 changes how the final simulation time/redshift is set, and (I guess) this results in a mismatch between the gold standard on Amazon and the locally generated final output. It's unclear to me whether the yt change should be reverted, or if the Enzo gold standard needs to be regenerated. Probably the latter. Mike On Wed, Jan 9, 2013 at 4:14 AM, Nathan … [View More]Goldbaum <nathan12343(a)gmail.com>wrote: > Hey Mike and Cameron, > > This is confusing. I just ran the quick suite on my desktop in Santa Cruz > using the latest changeset (a3149b6944ef). Enzo was compiled using gcc on > OS X 10.6. I see no failures due to missing answers and all tests > sucessfully run (although a number of tests fail, I guess since I was > running with compiler optimizations). The test_results.txt file for my > test is pasted here: http://pastebin.com/n4nZvEt1 > > I invoked the test runner using the following command: ./test_runner.py > --suite=quick -o ~/Desktop/enzo_tests > > It looks like we're running the same number of tests - the only difference > is that you were unable to find the answer tests on the amazon cloud. > > Have you tried rerunning the test suite? My concern is that amazon may > have been experiencing a hiccup when you tried to run the tests, preventing > you from downloading some of the test results files. I guess that > Cameron's replication of your test failures means that amazon may not be > the culprit. > > Cheers, > > Nathan > > > On 1/9/13 11:58 AM, Cameron Hummels wrote: > >> Hi Mike, >> >> I'm not sure what is going on here. Normally, YTNoOldAnswer arises when >> your simulation runs to a different output than the one which generated the >> gold standard. Unfortunately, when I look at AdiabaticExpansion with a new >> and an old run, they both seem to go to DD0016 (consistent with your runs). >> Unlike, Nathan, I *do* get the errors that you list, when I pull and run >> on a3149b6944ef on OS X using gcc. Will continue to talk to Nathan on this >> (we're at the same conference) and we'll hopefully have something on this >> soon. >> >> Cameron >> >> On Tue, Jan 8, 2013 at 5:19 PM, Michael Kuhlen <mqk(a)astro.berkeley.edu<mailto: >> mqk(a)astro.berkeley.edu**>> wrote: >> >> Hi all >> >> Is something wrong with the current Enzo gold standard on Amazon? >> I'm getting YTNoOldAnswer errors for AdiabaticExpansion, >> ExtremeAdvectionTest, MHDZeldovichPancake, and ZeldovichPancake. >> >> $ cat version.txt >> Enzo: 301413545861 (week-of-code) >> yt: 1ff31ebd14c4 >> >> test_runner.log (stdout and stderr): http://pastebin.com/GtYzEQbT >> >> test_results.txt: http://pastebin.com/DbD6Ke77 >> >> >> Mike >> >> -- ************************************************************** >> *********** >> * * >> * Dr. Michael Kuhlen Theoretical Astrophysics Center * >> * email: mqk(a)astro.berkeley.edu <mailto:mqk@astro.berkeley.edu**> >> UC Berkeley * >> * cell phone: (831) 588-1468 <tel:%28831%29%20588-1468> >> B-116 Hearst Field Annex # 3411 * >> >> * skype username: mikekuhlen Berkeley, CA 94720 * >> * * >> **************************************************************** >> ********* >> -- You received this message because you are subscribed to the >> Google >> Groups "enzo-dev" group. >> To post to this group, send email to enzo-dev(a)googlegroups.com >> <mailto:enzo-dev@googlegroups.**com <enzo-dev(a)googlegroups.com>>. >> >> To unsubscribe from this group, send email to >> enzo-dev+unsubscribe(a)**googlegroups.com<enzo-dev%2Bunsubscribe(a)googlegroups.com> >> <mailto:enzo-dev%**2Bunsubscribe@googlegroups.com<enzo-dev%252Bunsubscribe(a)googlegroups.com> >> **>. >> >> For more options, visit this group at >> http://groups.google.com/**group/enzo-dev?hl=en<http://groups.google.com/group/enzo-dev?hl=en> >> . >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "enzo-dev" group. >> To post to this group, send email to enzo-dev(a)googlegroups.com. >> To unsubscribe from this group, send email to enzo-dev+unsubscribe@** >> googlegroups.com <enzo-dev%2Bunsubscribe(a)googlegroups.com>. >> For more options, visit this group at http://groups.google.com/** >> group/enzo-dev?hl=en <http://groups.google.com/group/enzo-dev?hl=en>. >> > > -- > You received this message because you are subscribed to the Google Groups > "enzo-dev" group. > To post to this group, send email to enzo-dev(a)googlegroups.com. > To unsubscribe from this group, send email to enzo-dev+unsubscribe@** > googlegroups.com <enzo-dev%2Bunsubscribe(a)googlegroups.com>. > For more options, visit this group at http://groups.google.com/** > group/enzo-dev?hl=en <http://groups.google.com/group/enzo-dev?hl=en>. > > -- ********************************************************************* * * * Dr. Michael Kuhlen Theoretical Astrophysics Center * * email: mqk(a)astro.berkeley.edu UC Berkeley * * cell phone: (831) 588-1468 B-116 Hearst Field Annex # 3411 * * skype username: mikekuhlen Berkeley, CA 94720 * * * ********************************************************************* [View Less]

3 3

summary of GDF discussion
by j s oishi Jan. 8, 2013

Jan. 8, 2013

Hi all, On Friday, 4 Jan 2013, we had a hangout to discuss what the next steps to be taken for the implementation of a C language library for GDF. We call this library implementation gdfio. For reference, the GDF standard can be found at https://bitbucket.org/yt_analysis/yt/src/554d144d9d248c6f70d8c665a5963aa39b… One of the biggest issues we discussed was whether or not to rely on libraries other than HDF5 when writing gdfio. The main argument for doing so is that none of us are experienced … [View More]C programmers; thus implementing things like hashes, linked lists, and so forth might be a barrier to making progress. The main argument against doing so is that gdfio should be a very low-level library that can be deployed on many different systems, and dependencies make this difficult. In order to best assess what to do, we attempted to identify *what* non-stdlib C features we would actually need in our implementation. Because GDF itself does not make any links between grids (only recording their parents in an optional metadata step), we came to the conclusion that the only thing we need is a hash table. Even this hash table is optional and only for reading. For example, you could do something like gdfio_read_grid(grid_id, "density") and get back an object that includes both the density data and their associated metadata. We thus decided to proceed without *any* dependencies aside from HDF5. Kacper pointed out that the most important issue for efficiency is how to convert native memory structures into GDF's /data/grid_%010i/ structures without expensive copies. Native data could be 5D (block, quantity, x, y, z), 4D, or 3D depending on the code. On this point, Sam noted that the easiest thing might be to require a "buffer" type interface that gives (pointer, size) and allows gdfio to grab the requisite number of floats or doubles. We decided to try this buffer approach first. Essentially, this is a question of how much gdfio provides to users who will be wrapping it to write their code's data. For now, we decided to keep gdfio's offerings minimal. This allows us to see how it will work and what might be the best additional features to add later. Casey and Sam both brought up issues of how to parallelize. One issue is that parallel HDF5 can be quite tricky to deal with, so we decided to forgo using parallel HDF5 for now. We agreed that the simplest path forward is to use file links, so that each non-root-IO processor writes its data to a separate data-only file that is linked back to the main HDF5 file. This means we need to add an API for creating, writing, and reading from data-only files. Finally, we came up with the next step: I (Jeff) will draft an API for gdfio this week. I will submit it to yt-dev for discussion and iteration. Once we have an API that looks good, we'll begin coding it up. If I misrepresented anything from the meeting or GDF, please let me know. Thanks to all who participated! j [View Less]

2 1

[yt_analysis/yt] Cookbook recipes for streamlines and surfaces (issue #485)
by Matthew Turk Jan. 4, 2013

Jan. 4, 2013

--- you can reply above this line --- New issue 485: Cookbook recipes for streamlines and surfaces https://bitbucket.org/yt_analysis/yt/issue/485/cookbook-recipes-for-streaml… Matthew Turk: We should have cookbook recipes for: 1. Calculating streamlines and querying data along them 2. Identifying isosurfaces and querying data on them 3. Plotting isosurfaces in MPL 4. Plotting streamlines in MPL 5. Plotting both simultaneously in MPL. (Possibly; MPL may not be able to do this due to needing … [View More]

1 0

hangout to discuss GDF
by j s oishi Jan. 3, 2013

Jan. 3, 2013

Hi everyone, My yt New Year's Resolution is to get GDF really going. The main thing we need right now is a portable, easy to use and install C library to read and write GDF files. Right now, we need to identify what kinds of low-level object that are crucial to GDF but are *not* in the C standard library. This likely includes things like linked lists and hashes. What we want to get at the end of this meeting is whether or not we need to use a library like libHX http://freecode.com/projects/… [View More]

2 2