Here's a description of my general game plan and my questions are below:

THE MISSION
Create a GUI-based, dashboard-like application that monitors the overall health of our resources. It would be a tool that helps pinpoint trouble spots when performance is less than desirable. Ideally, we would get such a good handle on it that it would warn us of impending trouble before it causes a failure.

SAMPLE FUNCTIONS FOR THE APPLICATION
Monitor web traffic by distilling down Apache logs
Monitor server hardware through built in firmware http interface
Monitor server health via command line commands:
        disk space
        processor load
        etc.
Monitor database health via command line commands
Monitor backup success through log output
Monitor misc. server messages
Monitor switch traffic
Monitor processes
...And more.

POSSIBLE CONSTRAINTS
Our main servers are on a WAN whose security is the job of the greater org.  We have access through SSL connections now, but we may not be able to get permissions to open any additional ports for our monitoring purposes.

THE QUESTIONS
Very basic question: am I better off leaving most of the heavy lifting (as in the programming logic) on the client side? This would mean more data would have to pass from server to client app (e.g. a whole web log or part thereof) which is the downside. The upside is that in a failure emergency situation if my server is still up enough to have a SSH connection my dashboard app may still be useful to me. Thoughts?

If we are not constrained to only get our info via SSH what's the best kind of connection to have? My thought is to use the Prospective Broker since that seems to be the most Pythonistic option. Is it possible to encrypt the PB server connection?

Are there any Twisted methods of tunneling through an existing SSH connection (just a wild thought)?

Thanks,

Paul