You were right that I didn’t have python from miniconda in my PATH and so I was picking up the (old) system version of python. I prepended PATH variable and “which python” now points to the right version of python from miniconda. Nevertheless, the problem remains as illustrated below. I am puzzled why this is happening ... 

% pwd
/nobackupnfs2/mruszkow
% ls
Miniconda3-latest-Linux-x86_64.sh
% bash Miniconda3-latest-Linux-x86_64.sh 
% setenv PATH /nobackupnfs2/mruszkow/miniconda3/bin:${PATH}
% echo $PATH
% which python
% conda install yt
% module load pkgsrc/2015Q2
% hg clone https://bitbucket.org/yt_analysis/yt
% ls
Miniconda3-latest-Linux-x86_64.sh  miniconda3  yt
% pwd
/nobackupnfs2/mruszkow
% cd yt/
% hg up yt
% which python
/nobackupnfs2/mruszkow/miniconda3/bin/python
% python setup.py develop
Cython is a build-time requirement for the source tree of yt.
Please either install yt from a provided, release tarball,
or install Cython (version 0.22 or higher).
You may be able to accomplish this by typing:
     pip install -U Cython



On Oct 19, 2015, at 7:53 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:

You don't have the python from miniconda in your PATH and are picking up a version of python 2.6 or older from the operating system on Pleiades.

What's the output of "which python"? If the path returned by that command doesn't have "miniconda" in it, it's the wrong python executable. You might need to manually add miniconda's bin directory to your PATH environment variable.

On Mon, Oct 19, 2015 at 4:51 PM, Mateusz Ruszkowski <mateuszr@umich.edu> wrote:



The reason I originally loaded python_2.7.2 was to avoid the error message below. Now that python_2.7.2 is unloaded this message is back ...

% module unload python_2.7.2
% python setup.py develop
yt currently requires Python version 2.7
certain features may fail unexpectedly and silently with older versions.
%

As the next step I will try to repeat the whole process from scratch w/o using module load pkgsrc/2015Q2. Instead, I will just use standalone mercurial installation to see if this helps.






On Oct 19, 2015, at 7:31 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:



On Mon, Oct 19, 2015 at 4:11 PM, Mateusz Ruszkowski <mateuszr@umich.edu> wrote:


Hi Nathan/Desika,

Thanks! Following your suggestions, I replaced "conda build conda-recipes/yt"  with "conda install yt.” I am also able to do "conda install pip" w/o problem. But I nevertheless get the same error message when I execute the last command in this series:

bash Miniconda3-latest-Linux-x86_64.sh
miniconda3/bin/conda install yt                      [instead of:  miniconda3/bin/conda build conda-recipes/yt]
miniconda3/bin/conda install pip
hg clone https://bitbucket.org/yt_analysis/yt
cd yt/
hg up yt
python setup.py develop

Cython is a build-time requirement for the source tree of yt.
Please either install yt from a provided, release tarball,
or install Cython (version 0.22 or higher).
You may be able to accomplish this by typing:
     pip install -U Cython

Is there anything obvious in the above command sequence that is wrong? I am wondering if this problem has anything to do with how you setup various modules on Pleiades. For example, in order to use mercuria,l I do "module load pkgsrc/2015Q2”  I also do “module load python_2.7.2”  The pkgsrc/2015Q2 package loads a number of obsolete modules and perhaps they are messing things up?

Yes, that is possible.

You want to make sure that you are running the miniconda python and not the python loaded by the module. You can check by doing "which python". If you're using miniconda, you want to unload the python_2.7.2 module.
 

   Thanks,
     Mateusz


   
  


On Oct 19, 2015, at 5:05 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:



On Mon, Oct 19, 2015 at 1:42 PM, Mateusz Ruszkowski <mateuszr@umich.edu> wrote:


   Hi Nathan,

Thanks for your help. I think also this almost works ...  I downloaded the latest miniconda and followed this procedure:


bash Miniconda3-latest-Linux-x86_64.sh
miniconda3/bin/conda build conda-recipes/yt

Here you want to do "conda install yt" to get all of yt's dependencies.  From there you can install the development version of yt. You can "conda install" must python packages.

As desika mentioned, you can also install pip with "conda install pip".

I'd caution against using the full anaconda distribution since anaconda's MPI will shadow the system MPI installation unless you explicitly remove it.
 
cd yt/
hy up yt
python setup.py develop

This resulted in:
 
Cython is a build-time requirement for the source tree of yt.
Please either install yt from a provided, release tarball,
or install Cython (version 0.22 or higher).
You may be able to accomplish this by typing:
     pip install -U Cython

which I am not able to install via "pip install -U Cython." So this is essentially the same error message I got when I attempted to install yt using the method described in my previous e-mail. Any idea what may still be wrong?


   Thanks,
      Mateusz




On Oct 19, 2015, at 2:26 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:

Hi Mateusz

Rather than using the python module on pleiades, I'd suggest setting up your python stack on your own in your home folder.

By far the quickest way to do this is to install miniconda and then install yt into the miniconda environment using "conda install yt". You could also use the yt install script, although you might have issues with that if Pleiades has an older (insecure) version of OpenSSL installed, which is sadly quite common on supercomputers these days...

Personally, I set up my python environment on Pleiades using miniconda.

-Nathan

On Mon, Oct 19, 2015 at 11:23 AM, Mateusz Ruszkowski <mateuszr@umich.edu> wrote:

  Hi,

I am trying to install yt on Pleiades. I was able to complete a number installation steps but I eventually got an error message related to Cython (Cython seems to be installed though). The error message is attached below. Does anybody have an idea what needs to be done to complete this installation? 

    Thanks,
       Mateusz

------------------------------------------------
Dr. Mateusz Ruszkowski
Associate Professor

University of Michigan 
Department of Astronomy
301C West Hall
1085 South University Ave. 
Ann Arbor, MI 48109-1107, USA



———————————————————————— 

% module load pkgsrc/2014Q3
% module load python/2.7.9
% hg clone https://bitbucket.org/yt_analysis/yt
warning: bitbucket.org certificate with fingerprint 46:de:34:e7:9b:18:cd:7f:ae:fd:8b:e3:bc:f4:1a:5e:38:d7:ac:24 not verified (check hostfingerprints or web.cacerts config setting)
destination directory: yt
requesting all changes
adding changesets
adding manifests
adding file changes
added 17568 changesets with 36510 changes to 3053 files (+3 heads)
updating to bookmark @ on branch stable
1209 files updated, 0 files merged, 0 files removed, 0 files unresolved
% cd yt
% hg update yt
317 files updated, 0 files merged, 148 files removed, 0 files unresolved
(leaving bookmark @)
% python setup.py install --user --prefix=/nobackupnfs2/mruszkow
Cython is a build-time requirement for the source tree of yt.
Please either install yt from a provided, release tarball,
or install Cython (version 0.22 or higher).
You may be able to accomplish this by typing:
     pip install -U Cython
% pip install -U Cython
Downloading/unpacking Cython from https://pypi.python.org/packages/source/C/Cython/Cython-0.23.4.tar.gz#md5=157df1f69bcec6b56fd97e0f2e057f6e
  Downloading Cython-0.23.4.tar.gz (1.6MB): 1.6MB downloaded
  Running setup.py (path:/tmp/pip_build_mruszkow/Cython/setup.py) egg_info for package Cython
    Unable to find pgen, not compiling formal grammar.
   
    warning: no files found matching '*.pyx' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.pxd' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.h' under directory 'Cython/Debugger/Tests'
    warning: no files found matching '*.pxd' under directory 'Cython/Utility'
Installing collected packages: Cython
  Found existing installation: Cython 0.21.1
    Uninstalling Cython:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/req.py", line 598, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/req.py", line 1836, in remove
    renames(path, new_path)
  File "/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/pip/util.py", line 295, in renames
    shutil.move(old, new)
  File "/nasa/pkgsrc/2014Q4/lib/python2.7/shutil.py", line 300, in move
    rmtree(src)
  File "/nasa/pkgsrc/2014Q4/lib/python2.7/shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/nasa/pkgsrc/2014Q4/lib/python2.7/shutil.py", line 252, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "/nasa/pkgsrc/2014Q4/lib/python2.7/shutil.py", line 250, in rmtree
    os.remove(fullname)
OSError: [Errno 30] Read-only file system: '/nasa/pkgsrc/2014Q4/lib/python2.7/site-packages/Cython/Build/BuildExecutable.py'

Storing debug log for failure in /u/mruszkow/.pip/pip.log
%


_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org


_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org


_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org


_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org


_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org


_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org


_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org


_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org