Mailman 3 October 2015 - yt-dev

PR Triage Hangout
by Hilary Egan 23 Nov '15

23 Nov '15

Hi all, Here's the link for next week's PR triage hangout and a reminder that they will now be Mondays at 2pm EST! https://plus.google.com/b/107728486871834552760/events/car4cot30l8gu65a3b6e… -- Hilary Egan Astrophysical and Planetary Science Department University of Colorado Boulder

3 8

Issue #1140: Cannot run quickstart examples (yt_analysis/yt)
by Erik Schnetter 31 Oct '15

31 Oct '15

New issue 1140: Cannot run quickstart examples https://bitbucket.org/yt_analysis/yt/issues/1140/cannot-run-quickstart-exam… Erik Schnetter: The quickstart examples in `yt/doc/source/quickstart` try to download `*.tar` files that don't exist on the respective server. I believe that using the extension `*.tar.gz` instead solves this issue.

1 0

Issue #1139: Cannot run nose tests (yt_analysis/yt)
by Erik Schnetter 31 Oct '15

31 Oct '15

New issue 1139: Cannot run nose tests https://bitbucket.org/yt_analysis/yt/issues/1139/cannot-run-nose-tests Erik Schnetter: I have cloned and installed yt, but running the self-tests fails (see below). I am using Python 2.7 installed via Macports on OS X El Capitan. Can you give me a pointer to remedy this? ``` $ python setup.py develop --user --prefix= /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools/dist.py:285: UserWarning: Normalizing '3.1dev' to '3.1.dev0' normalized_version, non-existing path in 'yt/utilities/spatial': 'yt/utilities/spatial/tests' running develop running build_scripts running egg_info running build_src build_src building py_modules sources building extension "yt.analysis_modules.halo_finding.fof.EnzoFOF" sources building extension "yt.analysis_modules.halo_finding.hop.EnzoHop" sources building extension "yt.analysis_modules.ppv_cube.ppv_utils" sources building extension "yt.frontends.artio._artio_caller" sources building extension "yt.frontends.sph.smoothing_kernel" sources building extension "yt.geometry.oct_container" sources building extension "yt.geometry.oct_visitors" sources building extension "yt.geometry.particle_oct_container" sources building extension "yt.geometry.selection_routines" sources building extension "yt.geometry.particle_deposit" sources building extension "yt.geometry.particle_smooth" sources building extension "yt.geometry.fake_octree" sources building extension "yt.utilities.spatial.ckdtree" sources building extension "yt.utilities.spatial._distance_wrap" sources building extension "yt.utilities.lib.CICDeposit" sources building extension "yt.utilities.lib.ContourFinding" sources building extension "yt.utilities.lib.DepthFirstOctree" sources building extension "yt.utilities.lib.fortran_reader" sources building extension "yt.utilities.lib.geometry_utils" sources building extension "yt.utilities.lib.Interpolators" sources building extension "yt.utilities.lib.alt_ray_tracers" sources building extension "yt.utilities.lib.marching_cubes" sources building extension "yt.utilities.lib.misc_utilities" sources building extension "yt.utilities.lib.Octree" sources building extension "yt.utilities.lib.origami" sources building extension "yt.utilities.lib.image_utilities" sources building extension "yt.utilities.lib.PointsInVolume" sources building extension "yt.utilities.lib.QuadTree" sources building extension "yt.utilities.lib.RayIntegrators" sources building extension "yt.utilities.lib.mesh_utilities" sources building extension "yt.utilities.lib.grid_traversal" sources building extension "yt.utilities.lib.write_array" sources building extension "yt.utilities.lib.ragged_arrays" sources building extension "yt.utilities.lib.GridTree" sources building extension "yt.utilities.lib.amr_kdtools" sources building extension "yt.utilities.data_point_utilities" sources building extension "yt.visualization._MPL" sources building data_files sources build_src: building npy-pkg config files writing yt.egg-info/PKG-INFO writing top-level names to yt.egg-info/top_level.txt writing dependency_links to yt.egg-info/dependency_links.txt writing entry points to yt.egg-info/entry_points.txt reading manifest template 'MANIFEST.in' no previously-included directories found matching 'doc/source/reference/api/generated' no previously-included directories found matching 'doc/build/' writing manifest file 'yt.egg-info/SOURCES.txt' running build_ext customize UnixCCompiler customize UnixCCompiler using build_ext Creating /Users/eschnett/Library/Python/2.7/lib/python/site-packages/yt.egg-link (link to .) yt 3.1.dev0 is already the active version in easy-install.pth Installing yt script to /Users/eschnett/Library/Python/2.7/bin Installing iyt script to /Users/eschnett/Library/Python/2.7/bin Installing pyro_queue.py script to /Users/eschnett/Library/Python/2.7/bin Installing yt_lodgeit.py script to /Users/eschnett/Library/Python/2.7/bin Installed /Users/eschnett/src/yt Processing dependencies for yt==3.1.dev0 Finished processing dependencies for yt==3.1.dev0 ``` ``` $ python Python 2.7.10 (default, Oct 1 2015, 12:52:17) [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import yt >>> yt.run_nose() /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin answer-testing = yt.utilities.answer_testing.framework:AnswerTesting: No module named answer_testing.framework RuntimeWarning) E ====================================================================== ERROR: Failure: OSError ([Errno 2] No such file or directory: '') ---------------------------------------------------------------------- Traceback (most recent call last): File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py", line 86, in importFromDir if (self.sameModule(old, filename) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py", line 131, in sameModule if _samefile(mod_path, new_path): File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 155, in samefile s1 = os.stat(f1) OSError: [Errno 2] No such file or directory: '' ---------------------------------------------------------------------- Ran 1 test in 0.001s FAILED (errors=1) ```

1 0

Re: [yt-dev] Contributing to the YT project - 3D illustration - get_vertex_centered_data
by Tomer Nussbaum 30 Oct '15

30 Oct '15

Hi Matthew, Thank you for your answering. My main problem is that I know generally what code to write, but I'm not sure I get all the YT code design in this area.. So it will be great to talk to someone who knows this part of the code for 10-15 min, so I could get the basic understanding and I won't waste time on it in vain. Thanx, Tomer Date: Thu, 22 Oct 2015 14:49:12 -0700 From: Matthew Turk <matthewturk(a)gmail.com> To: "yt-dev(a)lists.spacepope.org" <yt-dev(a)lists.spacepope.org> Subject: Re: [yt-dev] Contributing to the YT project - 3D illustration - get_vertex_centered_data Message-ID: <CALO3=5GdL8zv8CK_i8hAOHRgZJk7ti+_NU4N33wjtqq6Te85Cw(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 Hi Tomer, Sorry for the long delay in replying to your email. I wanted to make sure I could collect my thoughts, and I hope you don't interpret it as not being interested in reaching out to you! I think this is a great project. It's probably kind of hard, but certainly do-able. The changes will have to be made in the octree_subset.py section of the code, and I think they might eventually need to touch the octree handler objects inside yt/geometry/oct_container.pyx, particularly the base container. To get started, I'd suggest just trying to do an "extrapolation" method, and then filling in from IO on disk. The technical way of doing this would be to fork the main yt repository, make a bookmark with "hg bookmark" and then push the bookmark to your repo with "hg push -B mybookmarkname" after changing the default destination in .hg/hgrc to be your fork (and leaving the main repo as something like ytanalysis). You can then commit, etc etc, and issue a PR which will get reviewed. Feel free also to issue a PR before you're done, if you'd like feedback or help during WIP. Thanks for writing, and stop by Slack or IRC if you want to talk in more detail! -MAtt On Tue, Oct 20, 2015 at 7:21 AM, Tomer Nussbaum <tomer.nussbaum(a)mail.huji.ac.il> wrote: > Hey guys, > > I wish to write the get_vertex_centered_data() function in octree_subset. > and make the YT 3D illustration work also to ART and RAMSES. > > > So.. what is the protocol? > > Thnx, > Tomer

2 1

ytdata frontend work
by John ZuHone 29 Oct '15

29 Oct '15

Hi all, Matt asked someone to send a quick note about the current issues regarding the ytdata frontend PR: https://bitbucket.org/yt_analysis/yt/pull-requests/1788/adding-ytdata-front… <https://bitbucket.org/yt_analysis/yt/pull-requests/1788/adding-ytdata-front…> I’m sending it because I’m working on it currently, but I was not able to attend today’s PR hangout so anyone else who was more to add or to correct should chime in. Currently, there are 3 issues, from most important to least important: 1. Tests don’t work 2. Doesn’t support Python 3 3. Derived quantities don’t work I’m currently working on 1 and 2, and there have been some ideas about 3. The main issue with the latter is that for non-gridded data containers (like sphere, disk, etc.) created from grid datasets, we treat the new datasets as “particle data”. For that reason there is an issue with calculating the cell_volume properly. Without that, we cannot use the total_mass() quantity. Given that we have the cell deltas in the dataset, we could have a special derived field for the cell_volume. There are other issues with some of the other derived quantities (e.g., weighted_average_quantity returning the wrong units), but I haven’t been able to diagnose them yet. I’m working on some changes that will need to be merged into Britton’s fork eventually to address at least 1 and 2. Best, John

3 10

Updating get_yt.sh, a question about defaults
by Nathan Goldbaum 28 Oct '15

28 Oct '15

Hi all, I've just issued a pull request to update the get_yt.sh script. https://bitbucket.org/yt_analysis/yt/pull-requests/1834/updating-miniconda-… This uses miniconda to install yt into a user-writable prefix. The main advantage over the install script is that we don't need to bootstrap python. This is a big help on OS X 10.11, since the openssl headers are no longer available there. In the version of the script I just pull requested, I've set it to no longer build yt from source. Instead it does "conda install yt". I did this because it significantly simplifies the user-visible setup. In particular, users no longer need to have compilers installed to install everything, and we don't need to tell people to install XCode or a bunch of packages. It's also the case that the Conda versions of libraries are less useful for local development due to the way conda makes packages relocatable. My plan with this is to still keep the install script available, but offer the get_yt.sh script as an alternate install script more suitable for "quick" installations for users who aren't likely going to hack on yt. The install script will still be available, and that will produce a more hackable installation of yt. It also comes with more headaches, for example on 10.11 or newer users will be responsible for installing the OpenSSL headers using e.g. homebrew or macports. I know that in the past we've gone to great trouble to make sure people get hackable installations of yt out of the box. The cost of that is substantially increased pain of installation and also increased support workload for developers who respond to mailing list and IRC questions. I can easily set INST_YT_SOURCE=1 to be the default if the consensus is that people should always get hackable yt installations (e.g. source install with "setup.py develop" by default. What do you all think? -Nathan

3 4

duplicate obtain_rvec and obtain_rv_vec functions in source
by Britton Smith 27 Oct '15

27 Oct '15

Hi all, It seems that there are two definitions in the yt source of the following functions: - obtain_rvec - obtain_rv_vec For each of these, one is in utilities/lib/misc_utilities.pyx and the other is in utilities/lib/geometry_utils.pyx. Also in each case, the source code for each is extremely similar, with the versions in misc_utilities containing additional keywords. At least in the case of obtain_rv_vec, both versions are called in different places in the code. Does anyone know of a reason for this? If there is one, this looks like a mess that should probably be cleaned up. Britton

3 3

OS X 10.11 and OpenSSL
by Nathan Goldbaum 25 Oct '15

25 Oct '15

Hi all, So the issue with the install script on 10.11 is that Apple no longer includes OpenSSL in the 10.11 SDK. The way package managers like macports and homebrew deal with this is by installing their own version of OpenSSL. The binary Python.org installers bundle OpenSSL as well. It may be possible to link against a version of OpenSSL included in the SDK for 10.10 (which can be obtained on 10.11), but I suspect that's not a very forward-looking idea and I have no idea under what circumstances the 10.10 SDKs are installed. One way to fix this would be to ship and build our own versions of OpenSSL along with the install script. I'm very wary about doing this, since I'd prefer not to even think about managing crypto libraries, but it's looking increasingly like we might need to do this here. One way to minimize risk would be to only install OpenSSL on OSX 10.11 and newer. Anyone have any thoughts here? Going to try to work on this tomorrow as well and it would be great to hear any ideas from the peanut gallery on this one. -Nathan

3 5

Issue #1138: Cannot install yt (yt_analysis/yt)
by Erik Schnetter 24 Oct '15

24 Oct '15

New issue 1138: Cannot install yt https://bitbucket.org/yt_analysis/yt/issues/1138/cannot-install-yt Erik Schnetter: I am trying to install yt onto OS X El Capitan. First I tried MacPorts, but this didn't work because of a problem with py27-h5py (I filed a MacPorts bug report for this). I then tried installing yt via pip; this apparently worked, but when I try to use it, there is an error: ``` $ python Python 2.7.10 (default, Oct 1 2015, 12:52:17) [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import yt Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/eschnett/src/yt/yt/__init__.py", line 116, in <module> from yt.data_objects.api import \ File "/Users/eschnett/src/yt/yt/data_objects/api.py", line 22, in <module> from .static_output import \ File "/Users/eschnett/src/yt/yt/data_objects/static_output.py", line 54, in <module> from yt.geometry.spherical_coordinates import \ File "/Users/eschnett/src/yt/yt/geometry/spherical_coordinates.py", line 23, in <module> from yt.utilities.lib.misc_utilities import \ ImportError: cannot import name pixelize_aitoff ``` Next I tried installing it via `install_script.sh`: ``` env HDF5_DIR=/opt/local bash install_script.sh ======================================================================== Hi there! This is the yt installation script. We're going to download some stuff and install it to create a self-contained, isolated environment for yt to run within. Inside the installation script you can set a few variables. Here's what they're currently set to -- you can hit Ctrl-C and edit the values in the script if you aren't such a fan. INST_ZLIB = 1 so I will be installing zlib INST_BZLIB = 1 so I will be installing bzlib INST_PNG = 1 so I will be installing libpng INST_FTYPE = 1 so I will be installing freetype2 INST_SQLITE3 = 1 so I will be installing SQLite3 INST_PY3 = 0 so I won't be installing Python 3 INST_HG = 1 so I will be installing Mercurial INST_PYX = 0 so I won't be installing PyX INST_SCIPY = 0 so I won't be installing scipy INST_0MQ = 1 so I will be installing ZeroMQ INST_ROCKSTAR = 0 so I won't be installing Rockstar HDF5_DIR=/opt/local , so I will not be installing HDF5 Installation will be to /Users/eschnett/yt-x86_64 and I'll be logging the installation in /Users/eschnett/yt-x86_64/yt_install.log I think that about wraps it up. If you want to continue, hit enter. If you'd rather stop, maybe think things over, even grab a sandwich, hit Ctrl-C. Looks like you're running on Mac OSX. NOTE: you must have the Xcode command line tools installed. The instructions for obtaining these tools varies according to your exact OS version. On older versions of OS X, you must register for an account on the apple developer tools website: https://developer.apple.com/downloads to obtain the download link. We have gathered some additional instructions for each version of OS X below. If you have trouble installing yt after following these instructions, don't hesitate to contact the yt user's e-mail list. You can see which version of OSX you are running by clicking 'About This Mac' in the apple menu on the left hand side of menu bar. We're assuming that you've installed all operating system updates; if you have an older version, we suggest running software update and installing all available updates. OS X 10.5.8: search for and download Xcode 3.1.4 from the Apple developer tools website. OS X 10.6.8: search for and download Xcode 3.2 from the Apple developer tools website. You can either download the Xcode 3.2.2 Developer Tools package (744 MB) and then use Software Update to update to XCode 3.2.6 or alternatively, you can download the Xcode 3.2.6/iOS SDK bundle (4.1 GB). OS X 10.7.5: download Xcode 4.2 from the mac app store (search for Xcode). Alternatively, download the Xcode command line tools from the Apple developer tools website. OS X 10.8.4, 10.9, and 10.10: download the appropriate version of Xcode from the mac app store (search for Xcode). Additionally, you will have to manually install the Xcode command line tools. For OS X 10.8, see: http://stackoverflow.com/questions/9353444 For OS X 10.9 and 10.10, the command line tools can be installed with the following command: xcode-select --install ======================================================================== Awesome! Here we go. Using wget Using shasum -a 512 Installing PNG Installing FreeType2 Installing SQLite3 Installing Python 2. This may take a while, but don't worry. yt loves you. ******************************************** FAILURE REPORT: ******************************************** from /System/Library/Frameworks/CoreGraphics.framework/Headers/CGBitmapContext.h:9, from /System/Library/Frameworks/CoreGraphics.framework/Headers/CoreGraphics.h:11, from /System/Library/Frameworks/ApplicationServices.framework/Headers/ApplicationServices.h:35, from /System/Library/Frameworks/Carbon.framework/Headers/Carbon.h:24, from Include/pymactoolbox.h:10, from Python/mactoolboxglue.c:27: /System/Library/Frameworks/CoreGraphics.framework/Headers/CGFont.h:53:40: error: initializer element is not constant static const CGFontIndex kCGGlyphMax = kCGFontIndexMax; ^ make: *** [Python/mactoolboxglue.o] Error 1 ******************************************** ******************************************** Failure. Check /Users/eschnett/yt-x86_64/yt_install.log. The last 10 lines are above. ``` I'd be grateful for any pointers.

1 0

Issue #1137: [experimental] need Scene.show() method to display rendered image inline in ipython notebooks (yt_analysis/yt)
by Cameron Hummels 24 Oct '15

24 Oct '15

New issue 1137: [experimental] need Scene.show() method to display rendered image inline in ipython notebooks https://bitbucket.org/yt_analysis/yt/issues/1137/experimental-need-scenesho… Cameron Hummels: Just like PlotWindow's show() command

1 0