[IPython-dev] ipcluster with ssh

[Harald Schilly]

Hi, I'm currently playing around with the ipcluster using ssh. I had a
problem and I couldn't track it down, until I did some changes to the
source. In file ipcluster.py in line 477 of version 10.1 (in the
SSHEngineSet class/method start()) I change the parameter
"consumeErrors" in "return gatherBoth(...)" from True to False. It
revealed, that a warning of ssh triggered an error and the cluster
just didn't do anything (no engine was started).

The problem/warning is triggered using this ~/.ssh/config file:
UserKnownHostsFile=/dev/null
StrictHostKeyChecking=no
NoHostAuthenticationForLocalhost=no

So, each time a new ssh connection is set up, the hostname is unknown
and this is reported. To get things running, I have to comment the
first line, so that the usual known hostname file is used and after
starting the cluster 2x all hosts are added and it works. The problem
is that those hostnames change sometimes and then it's once again an
error.

My first proposal would be to enable reporting those errors by default!

Is there a recommended/better configuration for ssh? If yes, this
should be part of the documentation. Also, I think the SSHEngineSet
class could be more intelligent and ignore this specific warning which
is indeed no problem.

I'm using Ubuntu linux/openssh.

The warning is:
2011-01-24 16:57:46+0100 [-] Unhandled error in Deferred:
2011-01-24 16:57:46+0100 [-] Unhandled Error
        Traceback (most recent call last):
        Failure: twisted.internet.utils._UnexpectedErrorOutput: got
stderr: "Warning: Permanently added '...,...' (RSA) to the list of
known hosts.\r\n"


Another idea is to change the exec_engine command in _ssh_engine() to
make it more intelligent. For some other project i added additional
options directly to ssh so that it is more reliable. It looks like
this:

   p = Popen(['ssh','-x','-q',\
              '-o','UserKnownHostsFile=/dev/null',\
              '-o','ServerAliveInterval=60',\
              '-o','StrictHostKeyChecking=no',\
              '-o','ConnectTimeout=%s' % SSH_CONN_TIMEOUT,\
              '-o','BatchMode=yes',\
              '-n','%s@%s'%(user, machine),\
              'ionice','-c','3','nice', cmd],\
              stdout=PIPE, stdin=PIPE, stderr=PIPE)


H