Issue #1139: Cannot run nose tests (yt_analysis/yt)
by Erik Schnetter
New issue 1139: Cannot run nose tests
https://bitbucket.org/yt_analysis/yt/issues/1139/cannot-run-nose-tests
Erik Schnetter:
I have cloned and installed yt, but running the self-tests fails (see below). I am using Python 2.7 installed via Macports on OS X El Capitan.
Can you give me a pointer to remedy this?
```
$ python setup.py develop --user --prefix=
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools/dist.py:285: UserWarning: Normalizing '3.1dev' to '3.1.dev0'
normalized_version,
non-existing path in 'yt/utilities/spatial': 'yt/utilities/spatial/tests'
running develop
running build_scripts
running egg_info
running build_src
build_src
building py_modules sources
building extension "yt.analysis_modules.halo_finding.fof.EnzoFOF" sources
building extension "yt.analysis_modules.halo_finding.hop.EnzoHop" sources
building extension "yt.analysis_modules.ppv_cube.ppv_utils" sources
building extension "yt.frontends.artio._artio_caller" sources
building extension "yt.frontends.sph.smoothing_kernel" sources
building extension "yt.geometry.oct_container" sources
building extension "yt.geometry.oct_visitors" sources
building extension "yt.geometry.particle_oct_container" sources
building extension "yt.geometry.selection_routines" sources
building extension "yt.geometry.particle_deposit" sources
building extension "yt.geometry.particle_smooth" sources
building extension "yt.geometry.fake_octree" sources
building extension "yt.utilities.spatial.ckdtree" sources
building extension "yt.utilities.spatial._distance_wrap" sources
building extension "yt.utilities.lib.CICDeposit" sources
building extension "yt.utilities.lib.ContourFinding" sources
building extension "yt.utilities.lib.DepthFirstOctree" sources
building extension "yt.utilities.lib.fortran_reader" sources
building extension "yt.utilities.lib.geometry_utils" sources
building extension "yt.utilities.lib.Interpolators" sources
building extension "yt.utilities.lib.alt_ray_tracers" sources
building extension "yt.utilities.lib.marching_cubes" sources
building extension "yt.utilities.lib.misc_utilities" sources
building extension "yt.utilities.lib.Octree" sources
building extension "yt.utilities.lib.origami" sources
building extension "yt.utilities.lib.image_utilities" sources
building extension "yt.utilities.lib.PointsInVolume" sources
building extension "yt.utilities.lib.QuadTree" sources
building extension "yt.utilities.lib.RayIntegrators" sources
building extension "yt.utilities.lib.mesh_utilities" sources
building extension "yt.utilities.lib.grid_traversal" sources
building extension "yt.utilities.lib.write_array" sources
building extension "yt.utilities.lib.ragged_arrays" sources
building extension "yt.utilities.lib.GridTree" sources
building extension "yt.utilities.lib.amr_kdtools" sources
building extension "yt.utilities.data_point_utilities" sources
building extension "yt.visualization._MPL" sources
building data_files sources
build_src: building npy-pkg config files
writing yt.egg-info/PKG-INFO
writing top-level names to yt.egg-info/top_level.txt
writing dependency_links to yt.egg-info/dependency_links.txt
writing entry points to yt.egg-info/entry_points.txt
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'doc/source/reference/api/generated'
no previously-included directories found matching 'doc/build/'
writing manifest file 'yt.egg-info/SOURCES.txt'
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
Creating /Users/eschnett/Library/Python/2.7/lib/python/site-packages/yt.egg-link (link to .)
yt 3.1.dev0 is already the active version in easy-install.pth
Installing yt script to /Users/eschnett/Library/Python/2.7/bin
Installing iyt script to /Users/eschnett/Library/Python/2.7/bin
Installing pyro_queue.py script to /Users/eschnett/Library/Python/2.7/bin
Installing yt_lodgeit.py script to /Users/eschnett/Library/Python/2.7/bin
Installed /Users/eschnett/src/yt
Processing dependencies for yt==3.1.dev0
Finished processing dependencies for yt==3.1.dev0
```
```
$ python
Python 2.7.10 (default, Oct 1 2015, 12:52:17)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import yt
>>> yt.run_nose()
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin answer-testing = yt.utilities.answer_testing.framework:AnswerTesting: No module named answer_testing.framework
RuntimeWarning)
E
======================================================================
ERROR: Failure: OSError ([Errno 2] No such file or directory: '')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/loader.py", line 411, in loadTestsFromName
addr.filename, addr.module)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py", line 86, in importFromDir
if (self.sameModule(old, filename)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py", line 131, in sameModule
if _samefile(mod_path, new_path):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 155, in samefile
s1 = os.stat(f1)
OSError: [Errno 2] No such file or directory: ''
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (errors=1)
```
3 years, 4 months
Re: [yt-dev] Contributing to the YT project - 3D illustration - get_vertex_centered_data
by Tomer Nussbaum
Hi Matthew,
Thank you for your answering.
My main problem is that I know generally what code to write, but I'm not
sure I get all the YT code design in this area..
So it will be great to talk to someone who knows this part of the code for
10-15 min, so I could get the basic understanding and I won't waste time on
it in vain.
Thanx,
Tomer
Date: Thu, 22 Oct 2015 14:49:12 -0700
From: Matthew Turk <matthewturk(a)gmail.com>
To: "yt-dev(a)lists.spacepope.org" <yt-dev(a)lists.spacepope.org>
Subject: Re: [yt-dev] Contributing to the YT project - 3D illustration
- get_vertex_centered_data
Message-ID:
<CALO3=5GdL8zv8CK_i8hAOHRgZJk7ti+_NU4N33wjtqq6Te85Cw(a)mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Hi Tomer,
Sorry for the long delay in replying to your email. I wanted to make
sure I could collect my thoughts, and I hope you don't interpret it as
not being interested in reaching out to you!
I think this is a great project. It's probably kind of hard, but
certainly do-able. The changes will have to be made in the
octree_subset.py section of the code, and I think they might
eventually need to touch the octree handler objects inside
yt/geometry/oct_container.pyx, particularly the base container.
To get started, I'd suggest just trying to do an "extrapolation"
method, and then filling in from IO on disk. The technical way of
doing this would be to fork the main yt repository, make a bookmark
with "hg bookmark" and then push the bookmark to your repo with "hg
push -B mybookmarkname" after changing the default destination in
.hg/hgrc to be your fork (and leaving the main repo as something like
ytanalysis). You can then commit, etc etc, and issue a PR which will
get reviewed. Feel free also to issue a PR before you're done, if
you'd like feedback or help during WIP.
Thanks for writing, and stop by Slack or IRC if you want to talk in more
detail!
-MAtt
On Tue, Oct 20, 2015 at 7:21 AM, Tomer Nussbaum
<tomer.nussbaum(a)mail.huji.ac.il> wrote:
> Hey guys,
>
> I wish to write the get_vertex_centered_data() function in octree_subset.
> and make the YT 3D illustration work also to ART and RAMSES.
>
>
> So.. what is the protocol?
>
> Thnx,
> Tomer
3 years, 4 months
ytdata frontend work
by John ZuHone
Hi all,
Matt asked someone to send a quick note about the current issues regarding the ytdata frontend PR:
https://bitbucket.org/yt_analysis/yt/pull-requests/1788/adding-ytdata-fro... <https://bitbucket.org/yt_analysis/yt/pull-requests/1788/adding-ytdata-fro...>
I’m sending it because I’m working on it currently, but I was not able to attend today’s PR hangout so anyone else who was more to add or to correct should chime in.
Currently, there are 3 issues, from most important to least important:
1. Tests don’t work
2. Doesn’t support Python 3
3. Derived quantities don’t work
I’m currently working on 1 and 2, and there have been some ideas about 3. The main issue with the latter is that for non-gridded data containers (like sphere, disk, etc.) created from grid datasets, we treat the new datasets as “particle data”. For that reason there is an issue with calculating the cell_volume properly. Without that, we cannot use the total_mass() quantity. Given that we have the cell deltas in the dataset, we could have a special derived field for the cell_volume.
There are other issues with some of the other derived quantities (e.g., weighted_average_quantity returning the wrong units), but I haven’t been able to diagnose them yet.
I’m working on some changes that will need to be merged into Britton’s fork eventually to address at least 1 and 2.
Best,
John
3 years, 4 months
Updating get_yt.sh, a question about defaults
by Nathan Goldbaum
Hi all,
I've just issued a pull request to update the get_yt.sh script.
https://bitbucket.org/yt_analysis/yt/pull-requests/1834/updating-minicond...
This uses miniconda to install yt into a user-writable prefix. The main
advantage over the install script is that we don't need to bootstrap
python. This is a big help on OS X 10.11, since the openssl headers are no
longer available there.
In the version of the script I just pull requested, I've set it to no
longer build yt from source. Instead it does "conda install yt". I did this
because it significantly simplifies the user-visible setup. In particular,
users no longer need to have compilers installed to install everything, and
we don't need to tell people to install XCode or a bunch of packages. It's
also the case that the Conda versions of libraries are less useful for
local development due to the way conda makes packages relocatable.
My plan with this is to still keep the install script available, but offer
the get_yt.sh script as an alternate install script more suitable for
"quick" installations for users who aren't likely going to hack on yt. The
install script will still be available, and that will produce a more
hackable installation of yt. It also comes with more headaches, for example
on 10.11 or newer users will be responsible for installing the OpenSSL
headers using e.g. homebrew or macports.
I know that in the past we've gone to great trouble to make sure people get
hackable installations of yt out of the box. The cost of that is
substantially increased pain of installation and also increased support
workload for developers who respond to mailing list and IRC questions. I
can easily set INST_YT_SOURCE=1 to be the default if the consensus is that
people should always get hackable yt installations (e.g. source install
with "setup.py develop" by default.
What do you all think?
-Nathan
3 years, 4 months
duplicate obtain_rvec and obtain_rv_vec functions in source
by Britton Smith
Hi all,
It seems that there are two definitions in the yt source of the following
functions:
- obtain_rvec
- obtain_rv_vec
For each of these, one is in utilities/lib/misc_utilities.pyx and the other
is in utilities/lib/geometry_utils.pyx. Also in each case, the source code
for each is extremely similar, with the versions in misc_utilities
containing additional keywords. At least in the case of obtain_rv_vec,
both versions are called in different places in the code. Does anyone know
of a reason for this? If there is one, this looks like a mess that should
probably be cleaned up.
Britton
3 years, 4 months
OS X 10.11 and OpenSSL
by Nathan Goldbaum
Hi all,
So the issue with the install script on 10.11 is that Apple no longer
includes OpenSSL in the 10.11 SDK.
The way package managers like macports and homebrew deal with this is by
installing their own version of OpenSSL. The binary Python.org installers
bundle OpenSSL as well. It may be possible to link against a version of
OpenSSL included in the SDK for 10.10 (which can be obtained on 10.11), but
I suspect that's not a very forward-looking idea and I have no idea under
what circumstances the 10.10 SDKs are installed.
One way to fix this would be to ship and build our own versions of OpenSSL
along with the install script. I'm very wary about doing this, since I'd
prefer not to even think about managing crypto libraries, but it's looking
increasingly like we might need to do this here. One way to minimize risk
would be to only install OpenSSL on OSX 10.11 and newer.
Anyone have any thoughts here? Going to try to work on this tomorrow as
well and it would be great to hear any ideas from the peanut gallery on
this one.
-Nathan
3 years, 4 months
Issue #1138: Cannot install yt (yt_analysis/yt)
by Erik Schnetter
New issue 1138: Cannot install yt
https://bitbucket.org/yt_analysis/yt/issues/1138/cannot-install-yt
Erik Schnetter:
I am trying to install yt onto OS X El Capitan. First I tried MacPorts, but this didn't work because of a problem with py27-h5py (I filed a MacPorts bug report for this).
I then tried installing yt via pip; this apparently worked, but when I try to use it, there is an error:
```
$ python
Python 2.7.10 (default, Oct 1 2015, 12:52:17)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import yt
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/eschnett/src/yt/yt/__init__.py", line 116, in <module>
from yt.data_objects.api import \
File "/Users/eschnett/src/yt/yt/data_objects/api.py", line 22, in <module>
from .static_output import \
File "/Users/eschnett/src/yt/yt/data_objects/static_output.py", line 54, in <module>
from yt.geometry.spherical_coordinates import \
File "/Users/eschnett/src/yt/yt/geometry/spherical_coordinates.py", line 23, in <module>
from yt.utilities.lib.misc_utilities import \
ImportError: cannot import name pixelize_aitoff
```
Next I tried installing it via `install_script.sh`:
```
env HDF5_DIR=/opt/local bash install_script.sh
========================================================================
Hi there! This is the yt installation script. We're going to download
some stuff and install it to create a self-contained, isolated
environment for yt to run within.
Inside the installation script you can set a few variables. Here's what
they're currently set to -- you can hit Ctrl-C and edit the values in
the script if you aren't such a fan.
INST_ZLIB = 1 so I will be installing zlib
INST_BZLIB = 1 so I will be installing bzlib
INST_PNG = 1 so I will be installing libpng
INST_FTYPE = 1 so I will be installing freetype2
INST_SQLITE3 = 1 so I will be installing SQLite3
INST_PY3 = 0 so I won't be installing Python 3
INST_HG = 1 so I will be installing Mercurial
INST_PYX = 0 so I won't be installing PyX
INST_SCIPY = 0 so I won't be installing scipy
INST_0MQ = 1 so I will be installing ZeroMQ
INST_ROCKSTAR = 0 so I won't be installing Rockstar
HDF5_DIR=/opt/local , so I will not be installing HDF5
Installation will be to
/Users/eschnett/yt-x86_64
and I'll be logging the installation in
/Users/eschnett/yt-x86_64/yt_install.log
I think that about wraps it up. If you want to continue, hit enter.
If you'd rather stop, maybe think things over, even grab a sandwich,
hit Ctrl-C.
Looks like you're running on Mac OSX.
NOTE: you must have the Xcode command line tools installed.
The instructions for obtaining these tools varies according
to your exact OS version. On older versions of OS X, you
must register for an account on the apple developer tools
website: https://developer.apple.com/downloads to obtain the
download link.
We have gathered some additional instructions for each
version of OS X below. If you have trouble installing yt
after following these instructions, don't hesitate to contact
the yt user's e-mail list.
You can see which version of OSX you are running by clicking
'About This Mac' in the apple menu on the left hand side of
menu bar. We're assuming that you've installed all operating
system updates; if you have an older version, we suggest
running software update and installing all available updates.
OS X 10.5.8: search for and download Xcode 3.1.4 from the
Apple developer tools website.
OS X 10.6.8: search for and download Xcode 3.2 from the Apple
developer tools website. You can either download the
Xcode 3.2.2 Developer Tools package (744 MB) and then use
Software Update to update to XCode 3.2.6 or
alternatively, you can download the Xcode 3.2.6/iOS SDK
bundle (4.1 GB).
OS X 10.7.5: download Xcode 4.2 from the mac app store
(search for Xcode).
Alternatively, download the Xcode command line tools from
the Apple developer tools website.
OS X 10.8.4, 10.9, and 10.10: download the appropriate version of
Xcode from the mac app store (search for Xcode).
Additionally, you will have to manually install the Xcode
command line tools.
For OS X 10.8, see:
http://stackoverflow.com/questions/9353444
For OS X 10.9 and 10.10, the command line tools can be installed
with the following command:
xcode-select --install
========================================================================
Awesome! Here we go.
Using wget
Using shasum -a 512
Installing PNG
Installing FreeType2
Installing SQLite3
Installing Python 2. This may take a while, but don't worry. yt loves you.
********************************************
FAILURE REPORT:
********************************************
from /System/Library/Frameworks/CoreGraphics.framework/Headers/CGBitmapContext.h:9,
from /System/Library/Frameworks/CoreGraphics.framework/Headers/CoreGraphics.h:11,
from /System/Library/Frameworks/ApplicationServices.framework/Headers/ApplicationServices.h:35,
from /System/Library/Frameworks/Carbon.framework/Headers/Carbon.h:24,
from Include/pymactoolbox.h:10,
from Python/mactoolboxglue.c:27:
/System/Library/Frameworks/CoreGraphics.framework/Headers/CGFont.h:53:40: error: initializer element is not constant
static const CGFontIndex kCGGlyphMax = kCGFontIndexMax;
^
make: *** [Python/mactoolboxglue.o] Error 1
********************************************
********************************************
Failure. Check /Users/eschnett/yt-x86_64/yt_install.log. The last 10 lines are above.
```
I'd be grateful for any pointers.
3 years, 5 months
