[Twisted-Python] TDDing an ssh client - mocking the server
Firstly a belated thanks to everyone who replied to my previous question - I had to put my project on hold for a while, but now I am back with more questions. My objective is to use twisted to do automated logging in to multiple ssh servers to carry out various tasks. Currently this is done by spawning a separate ssh executable for each session and controlling it through pexpect, and I would like to replace this with a twisted/conch system. So far I can create conch "exec" sessions to run a single command/script and return the result, but I would like to be able to create shell sessions and do more interactive work. Does anyone have any examples of doing this? The biggest issue I have at the moment is how to do Test Driven Development with conch. I would like to be able to mock out the server that the ssh client talks to, so that I can keep the unit tests self-contained and keep tight control of both sides of the interaction, e.g. for testing failure scenarios. I can think of three ways of doing this: 1) write a separate ssh server with conch that provides the desired behaviour, and have that spawned as a separate process by the unit tests. 2) as above, but run the ssh server in the same process as the tests, so they are both firing from the same reactor event loop. IMHO this is better than (1) since it is more self-contained. 3) stub out the low-level classes of the client so that they are not really talking to a server at all, but simulating the correct behaviour. The problem with both 1 and 2 is that they require writing test code that is as complex as the code I am testing. The problem with 3 is that it assumes that I know what the correct low-level behaviour is so that I can mock it out. At the moment I do not grok twisted in sufficient detail to do that. How have other people tackled this? Are there any other options that I am missing, or code I can reuse, or other suggestions or advice? TIA -- Dave Kirby Senior Software Engineer T: +44 20 7368 7330 | F: +44 20 7352 4922 E: d.kirby@tideway.com | W: www.tideway.com This email and any attachment may contain confidential, privileged information for the sole use of the intended recipient. If you are not the intended recipient, do not disclose, reproduce, disseminate or otherwise use this communication. If you received this communication in error, please immediately notify the sender via email and delete the communication from your system. Company information : Tideway Systems Ltd, Registered Office: Anchor House, 15-19 Britten Street, London, SW3 3TY. Registered in England & Wales Reg. Reg. No: 4598072 VAT No: 805 5153 50
On Mon, 6 Aug 2007 12:11:29 +0100, Dave Kirby <d.kirby@tideway.com> wrote:
Firstly a belated thanks to everyone who replied to my previous question - I had to put my project on hold for a while, but now I am back with more questions.
My objective is to use twisted to do automated logging in to multiple ssh servers to carry out various tasks. Currently this is done by spawning a separate ssh executable for each session and controlling it through pexpect, and I would like to replace this with a twisted/conch system.
So far I can create conch "exec" sessions to run a single command/script and return the result, but I would like to be able to create shell sessions and do more interactive work. Does anyone have any examples of doing this?
The biggest issue I have at the moment is how to do Test Driven Development with conch. I would like to be able to mock out the server that the ssh client talks to, so that I can keep the unit tests self-contained and keep tight control of both sides of the interaction, e.g. for testing failure scenarios. I can think of three ways of doing this:
1) write a separate ssh server with conch that provides the desired behaviour, and have that spawned as a separate process by the unit tests.
2) as above, but run the ssh server in the same process as the tests, so they are both firing from the same reactor event loop. IMHO this is better than (1) since it is more self-contained.
3) stub out the low-level classes of the client so that they are not really talking to a server at all, but simulating the correct behaviour.
The problem with both 1 and 2 is that they require writing test code that is as complex as the code I am testing. The problem with 3 is that it assumes that I know what the correct low-level behaviour is so that I can mock it out. At the moment I do not grok twisted in sufficient detail to do that.
I suspect a lot of the code that you write will be for handling the output from and generating input to send to processes run via ssh. You should be able to use #3 to test this kind of code, since the API for interacting with conch from a protocol implementation is quite confined and well-defined. I don't have much to suggest for avoiding #2 (but you probably don't want #1 in any case), but if you at least isolate the majority of your application logic from most of conch, then you can test it without having any kind of SSH server or mock around. The code that's left over that does have to interact with conch can be tested by itself, perhaps by setting up real SSH connections, which may not be ideal, but you should at least be able to do it, get it over with, and not worry about it anymore afterwards. Most of the rest of the code you write will be testable in isolation. Hope this helps, Jean-Paul
On Mon, 2007-08-06 at 12:11 +0100, Dave Kirby wrote:
Firstly a belated thanks to everyone who replied to my previous question – I had to put my project on hold for a while, but now I am back with more questions.
My objective is to use twisted to do automated logging in to multiple ssh servers to carry out various tasks. Currently this is done by spawning a separate ssh executable for each session and controlling it through pexpect, and I would like to replace this with a twisted/conch system.
Hi Dave, The framework I created to do something very similar to this is available for free at http://modipy.seafelt.com. ModiPY is written using twisted and currently does pretty much what you've described, spawning a program in a sub-process to do work. This 'spawn and do work' component could be replaced by a conch SSH client instead. In fact, this would be desirable for some things.
So far I can create conch “exec” sessions to run a single command/script and return the result, but I would like to be able to create shell sessions and do more interactive work. Does anyone have any examples of doing this?
I'm guessing you want examples of how to use conch to do interactive work, so ModiPY probably isn't useful here, but you might be able to use some of the ideas in the ConnectingProvisioner and ExpectProtocol. It connects to a remote device and runs a series of commands, handling their output. -- Justin Warren <daedalus@eigenmagic.com>
participants (3)
-
Dave Kirby
-
Jean-Paul Calderone
-
Justin Warren