[IPython-dev] new doc for parallel sessions on clusters

Johann Cohen-Tanugi johann.cohentanugi at gmail.com
Tue Jun 21 12:31:18 EDT 2011


hmmm ok, after the merge is this the right way to get the code :
  git clone git://github.com/ipython/ipython.git ipython
I removed the whole source directory, and the local install stuff 
related to python, and I started from scratch using the git command above.
then :
python setup.py install 
--prefix=/afs/slac/g/glast/users/cohen/IPYDEV/local/which python

perhaps the problem is that there is an ipython installed with the 
native python that I use above, and I am wrong to assume that having 
/afs/slac/g/glast/users/cohen/IPYDEV/local/.../site-packages at the top 
of PYTHONPATH is enough to ensure that it overrides the one installed 
with python?

Anyway, I will continue investigating....
sorry for the noise,
JOhann

On 06/21/2011 05:27 PM, MinRK wrote:
> On Tue, Jun 21, 2011 at 07:27, Johann Cohen-Tanugi
> <johann.cohen-tanugi at lupm.univ-montp2.fr>  wrote:
>> hi there, I tried to build the sphinx doc on my machine after a pull of the
>> head (after Min announced the merge of newapp into master), and I see the
>> following in docs/source/parallel/parallel_process.txt :
>> 1/ A trivial typo at :
>> :command:`ipcluster` has a notion of Launchers that can start controllers
>> and engines with various remote execution schemes.  Currently supported
>> models include :command:`ssh`, :command`mpiexec`, PBS-style (Torque, SGE),
>> and Windows HPC Server.
>>                                                                     |_____>
>> should be :command:`mpiexec`
>> 2/ I can read the following lines :
>>     $ ipython profile create --parallel profile=mpi
>> or
>>     $ ipython profile create --parallel profile=pbs
>>
>> while ipython profile create does *not* accept a --parallel option AFAICT,
>> but rather --cluster or --no-cluster.
> Then you do not have current master, because the flag is indeed '--parallel'.
>
>> best,
>> Johann
>>
>> On 06/21/2011 07:37 AM, Min RK wrote:
>>> Yes, it is going to be standard for the cluster profile and interactive
>>> IPython to be different - the profile for the Client is principally for
>>> connection info, and the shell profile is for configuring your interactive
>>> environment.  There's no reason to change your interactive config just to
>>> connect to a different cluster.
>>>
>>> That said, the default profile of the Client should probably be that of
>>> the current application, not just 'default'.
>>>
>>> -MinRK
>>>
>>> On Jun 20, 2011, at 22:11, Johann
>>> Cohen-Tanugi<johann.cohen-tanugi at lupm.univ-montp2.fr>    wrote:
>>>
>>>> hi- I don't think we should print the profile name in the default case,
>>>>> it's just noise.  I realize we now have a more consistent structure
>>>>> for profiles and even the default case is now a profile, but we should
>>>>> keep the amount of printed stuff to a minimum in the default cases.
>>>>>
>>>> Actually I have a question here : I was trying newapp, following Min's
>>>> advice, to try to add the LSF support in parallel.apps. From what I could
>>>> gather
>>>> I did
>>>> ipcluster start -p lsf -n 2
>>>> which created profile_lsf in my $HOME/.ipython directory, but then when I
>>>> started another terminal window for the ipython session, I typed
>>>> ipython profile=lsf
>>>> and this loaded the default profile, so that I had to type :
>>>>
>>>> from IPython.parallel import Client
>>>> c = Client(profile='lsf')
>>>>
>>>> so that unless this is a bug or an operator mistake, there seems to be 2
>>>> 'profiles' in such a use case : the ipython global one, and the parallel lsf
>>>> one. I find that a bit confusing, and maybe there is a way to merge the 2?
>>>>
>>>> best,
>>>> Johann



More information about the IPython-dev mailing list