Mailman 3 December 2010 - yt-dev

Volume rendering with current (3628:da3894bf42a5) change set
by Chris Malone 13 Jan '11

13 Jan '11

Hi all, As previously mentioned, I'm working in my spare time to add support for the Maestro code to yt. I am able to read in the data, make slices of various variables, and save the resulting plots. I tried, however, to follow the cookbook recipe for volume rendering and I keep getting an error while attempting to ray_cast. Here is the script I use: http://paste.enzotools.org/show/1470/ and here is the traceback: http://paste.enzotools.org/show/Jn7X2pBPkUD0yhhLwrVv/ I'm not sure if this is an issue with the way I have implemented Maestro support or if there is a bug in the visualization. Could someone please try my script with their own parameter file to at least verify there is no bug in ray_cast? Thanks, Chris

3 9

Clarification of Enzo Parameters, TopGridRank and TopGridDimensions
by Chris Malone 23 Dec '10

23 Dec '10

Hi, In attempting to develop yt support for an additional computer code, MAESTRO, I'm finding a little confusion in relating the Enzo parameters to some parameters used in MAESTRO. I have checked http://lca.ucsd.edu/software/enzo/v1.0.1/amr_guide/parameters-sorted.htmland compared, for example, TopGridRank and TopGridDimensions to what is used in the yt code. My understanding is as follows: (1) TopGridRank = < int > essentially the spatial dimensionality of the problem; As far as I can tell, yt supports only 3-d data sets, so this /should/ be set to 3 for current use? (2) TopGridDimensions = TopGridRank-vector of < int >'s essentially the number of zones (not counting ghosts/guards) in each direction of the problem; Again, as far as I can tell this should be something like: TopGridDimensions = number_zones_x number_zones_y number_zones_z Is this correct? If it is, then I'm confused about how the Orion frontend loads these parameters. For that frontend, there is a dictionary which translates between the Orion (i.e. BoxLib) parameters and those of Enzo: orion2enzoDict. It is defined in frontends/orion/definitions.py as: orion2enzoDict = {"amr.n_cell": "TopGridRank", "materials.gamma": "Gamma", "amr.ref_ratio": "RefineBy" } I don't have a sample Orion parameter file, but I do have one for Castro, which also uses the BoxLib data format. For Castro, the amr.n_cell line of a parameter file looks like: amr.n_cell = 32 32 32 If I make the assumption (and maybe this is where I'm going wrong...) that the grid parameters used in a Castro parameter file are the same as those in an Orion parameter file, then the OrionStaticOutput._parse_parameter_file method is mapping TopGridRank to a list of < int >'s, which is what I thought TopGridDimensions was given (2) above. In other words, if (1) and (2) are correct, shouldn't TopGridRank = len(amr.n_cell.split()) TopGridDimensions = map(int, amr.n_cell.split()) ? Any clarification would be helpful! Chris

4 5

Re: [Yt-dev] HDF5 Format for Merger Trees
by Stephen Skory 18 Dec '10

18 Dec '10

> I just saw a presentation about a semi-analytic code called > Galacticus, by Andrew Benson of Caltech. He presented on an HDF5 > format for merger trees which is quite interesting, and certainly > something I think we should pursue implementing. > > http://www.ctcp.caltech.edu/galacticus/MergerTreeFileFormat.pdf I didn't see the presentation, so I'm wondering how exactly this is better than a SQL database for storing *relational* data? I agree that storing the particle data in the SQL database, for example, is not good, but it seems to me that HDF5 is not ideal for what SQL is designed for. Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

2 1

HDF5 Format for Merger Trees
by Matthew Turk 18 Dec '10

18 Dec '10

Hi all, I just saw a presentation about a semi-analytic code called Galacticus, by Andrew Benson of Caltech. He presented on an HDF5 format for merger trees which is quite interesting, and certainly something I think we should pursue implementing. http://www.ctcp.caltech.edu/galacticus/MergerTreeFileFormat.pdf -Matt

1 0

Particle IO Disabled for now
by Matthew Turk 18 Dec '10

18 Dec '10

Hi all, I have disabled fast ParticleIO for now. There's a bug in this code, which I am struggling to figure out the right way to fix. This only applies to Enzo data, and the bug only showed up when you had a derived field depending on only one field, and only if it's more than a simple conversion factor. The only field I found that this applied to was ParticleAge. During the reading, we pass in conversion factors for fields, so that when returned they will be correctly scaled. However, I was incorrectly setting dependency fields as the main field; this means that, for instance, the ParticleAge conversion was being applied to the creation_time field, and then creation_time was set as ParticleAge. This is inside yt/data_objects/particle_io.py. The problem here is getting all three things playing nicely: * Particle IO * Conversions * Derived fields based on particles I have to sit back and think about ParticleIO. Really I have always kind of hated my particle IO implementation, but I think the solution is to simplify it and make it behave more straightforwardly. I'll notify the list when I've fixed the bug and re-enabled ParticleIO. Best, Matt

1 0

Fwd: [Cython] ANN: Cython 0.14 released
by Matthew Turk 15 Dec '10

15 Dec '10

Cython is now part of the install script, but I won't be upgrading that installed Cython to 0.14 immediately. These features will mostly not help out yt's usage of Cython. I think we should also consider requiring Cython during installation/setup, so that we can stop checking in humongous .c files. ---------- Forwarded message ---------- From: Robert Bradshaw <robertwb(a)math.washington.edu> Date: Tue, Dec 14, 2010 at 1:40 PM Subject: [Cython] ANN: Cython 0.14 released To: python-announce-list(a)python.org, cython-users(a)googlegroups.com, cython-dev(a)codespeak.net The Cython project is happy to announce the release of Cython 0.14. This release fixes several bugs in the previous 0.13 release, improved debugging and build support, and heavily improves Python compatibility in terms of supported features. It can be downloaded from http://cython.org/ or http://pypi.python.org/pypi/Cython/0.14 Release notes: http://wiki.cython.org/ReleaseNotes-0.14 == New Features == * Python classes can now be nested and receive a proper closure at definition time. * Redefinition is supported for Python functions, even within the same scope. * Lambda expressions are supported in class bodies and at the module level. * Metaclasses are supported for Python classes, both in Python 2 and Python 3 syntax. The Python 3 syntax (using a keyword argument in the type declaration) is preferred and optimised at compile time. * "final" extension classes prevent inheritance in Python space. This feature is available through the new "cython.final" decorator. In the future, these classes may receive further optimisations. * "internal" extension classes do not show up in the module dictionary. This feature is available through the new "cython.internal" decorator. * Extension type inheritance from builtin types, such as "cdef class MyUnicode(unicode)", now works without further external type redeclarations (which are also strongly discouraged now and continue to issue a warning). * GDB support. http://docs.cython.org/src/userguide/debugging.html * A new build system with support for inline distutils directives, correct dependency tracking, and parallel compilation. http://wiki.cython.org/enhancements/distutils_preprocessing * Support for dynamic compilation at runtime via the new cython.inline function and cython.compile decorator. http://wiki.cython.org/enhancements/inline == General improvements and bug fixes == * In parallel assignments, the right side was evaluated in reverse order in 0.13. This could result in errors if it had side effects (e.g. function calls). * In some cases, methods of builtin types would raise a SystemError instead of an AttributeError when called on None. * Constant tuples are now cached over the lifetime of an extension module, just like CPython does. Constant argument tuples of Python function calls are also cached. * Closures have tightened to include exactly the names used in the inner functions and classes. Previously, they held the complete locals of the defining function. * "nogil" blocks are supported when compiling pure Python code by writing "with cython.nogil". * The builtin "next()" function in Python 2.6 and later is now implemented internally and therefore available in all Python versions. This makes it the preferred and portable way of manually advancing an iterator. * In addition to the previously supported inlined generator expressions in 0.13, "sorted(genexpr)" can now be used as well. Typing issues were fixed in "sum(genexpr)" that could lead to invalid C code being generated. Other known issues with inlined generator expressions were also fixed that make upgrading to 0.14 a strong recommendation for code that uses them. Note that general generators and generator expressions continue to be not supported. * Iterating over arbitrary pointer types is now supported, as is an optimized version of the in operator, e.g. x in ptr[a:b]. * Inplace arithmetic operators now respect the cdivision directive and are supported for complex types. == Incompatible changes == * Typing a variable as type "complex" previously gave it the Python object type. It now uses the appropriate C/C++ {{{double complex}}} type. A side-effect is that assignments and typed function parameters now accept anything that Python can coerce to a complex, including integers and floats, and not only complex instances. * Large integer literals pass through the compiler in a safer way. To prevent truncation in C code, non 32-bit literals are turned into Python objects if not used in a C context. This context can either be given by a clear C literal suffix such as "UL" or "LL" (or "L" in Python 3 code), or it can be an assignment to a typed variable or a typed function argument, in which case it is up to the user to take care of a sufficiently large value space of the target. * Python functions are declared in the order they appear in the file, rather than all being created at module creation time. This is consistent with Python and needed to support, for example, conditional or repeated declarations of functions. In the face of circular imports this may cause code to break, so a new {{{--disable-function-redefinition}}} flag was added to revert to the old behavior. This flag will be removed in a future release, so should only be used as a stopgap until old code can be fixed. == Contributors == Many people contributed to this release, including: * Haoyu Bai * Stefan Behnel * Robert Bradshaw * Ondrej Certik * Lisandro Dalcin * Mark Florisson * Eric Huss * Vitja Makarov * Corbin Simpson * Kurt Smith _______________________________________________ Cython-dev mailing list Cython-dev(a)codespeak.net http://codespeak.net/mailman/listinfo/cython-dev

1 0

Call for testing: Freetype module build system
by Matthew Turk 15 Dec '10

15 Dec '10

Hi all, I wrote up a very simple wrapper to do textual annotations directly using Freetype. This allows, for instance, adding annotations to image buffers created by fixed resolution buffers, volume rendering, etc. It should, eventually, also allow us to make a subset of plots with colorbars without using Matplotlib. Anyway, I've already added freetype to the yt build system. Additionally, matplotlib requires it. So it *should* already exist on the system, although I've had to add some guesswork to get the yt build system (not the install script) to link against the freetype wrappers. I've bundled up the changes and attached them to this email. It would be amazing if I could have a couple people try unbundling these changes, updating to hash 2f9479b4f1dd, re-cythonizing and attempting a build. This process should look something like: hg unbundle freetype.hg hg up -C 2f9479b4f1dd cython yt/utilities/amr_utils.pyx python setup.py build_ext -i Testing this will help to make sure that we can push the new freetype module out into the wild, and use it more generally for annotating volume rendering, etc etc. You can see the (documented!) function as "annotate_image" which should be in mods, and which accepts a bitmap (as, for instance, output by write_image) as input. Any reports of success/failure would be great. Thanks! -Matt

5 11

simple quantities
by Stephen Skory 13 Dec '10

13 Dec '10

Hi all, after my conversation on -users with Matt, and the recent investigations of inline yt, a thought occurs to me. The whole point of the conversation was that I would like to do some inline analysis of a simulation with star(s). As a zero-order test, the analysis shouldn't run if there are no stars, so I wanted to find out how many stars there are. But as the conversation revealed, it isn't simple/blindingly obvious to do that kind of calculation in a way that works in serial, parallel, and inline, e.g. with derived quantities. How do we feel about my writing a set of convenience functions that make these simple calculations easy? This shouldn't be comprehensive, just things that might come up often. Thoughts? Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

3 2

Re: [Yt-dev] Inline P-Hop
by Stephen Skory 10 Dec '10

10 Dec '10

Hi all, I just confirmed that inline PHOP works. Which is nice. Stephen Skory stephenskory(a)yahoo.com http://stephenskory.com/ 510.621.3687 (google voice)

2 1

Verbosity Increase
by Matthew Turk 10 Dec '10

10 Dec '10

Hi all, I would like to propose that we increase the verbosity of two operations: 1) When a parameter file is parsed, I would like several pieces of information output via the "INFO" level of the logging handler. This would be the current simulation time, the dimensions of the domain, and any cosmological parameters. 2) When print_stats is called in the hierarchy, I would like to include all of the cosmological parameters. The reason I'm bringing this up rather than just changing it is that in the past yt has (rightly) been accused of being too verbose (in fact, it was one of the first things Jeff said to me the first time we met to talk about yt!) and I don't want to increase that unnecessarily. But, I feel strongly that as we move into more codes we need to be more upfront about what yt thinks of a given simulation output. I think that this will also help ensure that errors are avoided. +1/-1? -Matt

5 10