[Twisted-Python] Help with twisted.conch.manhole
![](https://secure.gravatar.com/avatar/d91852c2bac952ac200a8c3e0390048a.jpg?s=120&d=mm&r=g)
I am real need to debug my python application for memory issues. I found that using twisted.conch.manhole we can get a telnet session to python interpreter so that we can do some debugging sessions there. I am using twisted 16.6 in my server. Could you some one please give me some sample program for achieving this. Thanks ~S PS: I have posted this message in stack overflow. Sorry for cross posting. I have joined this group only today.
![](https://secure.gravatar.com/avatar/63ef3e1c4ce734bfc654bfd31a5ff041.jpg?s=120&d=mm&r=g)
Don't know if this is still state of the art, but I've been using this off and on and please don't use checkers.InMemoryUsernamePasswordDatabaseDontUse HTH, Werner #### controlling terminal command line server # #on port 9999. Connect with # #ssh admin@localhost -p 9999 # #accept new keys and start work # from twisted.conch import manhole, manhole_ssh from twisted.conch.ssh import keys, userauth, factory, connection from twisted.conch.ssh.transport import SSHServerTransport from twisted.cred import portal, checkers """ Generate these keys with $ ckeygen -t rsa -f ssh-keys/ssh_host_rsa_key $ ckeygen -t rsa -f ssh-keys/client_rsa ssh_host_rsa_key.pub is the public key file ssh_host_rsa_key is the private key file """ publicKey = """ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQCoJSZjqVJdh36RoNvwsTMNlqk8heFV6DxNhhkw3uZ58uyrsnz+7JHGpu56tJAO39WMJPLQfdtyWxCy1P2RRKh65hGznvyt2UaAHdZDMqbNcMlupFPg8P4pCgR+/2M14QSopqysfjgXUJRfR/pD93Ofy0L2N5uEGipTjVAjKwxbzQ== wthie@qubik.local""" privateKey = """-----BEGIN RSA PRIVATE KEY----- MIICXgIBAAKBgQCoJSZjqVJdh36RoNvwsTMNlqk8heFV6DxNhhkw3uZ58uyrsnz+ 7JHGpu56tJAO39WMJPLQfdtyWxCy1P2RRKh65hGznvyt2UaAHdZDMqbNcMlupFPg 8P4pCgR+/2M14QSopqysfjgXUJRfR/pD93Ofy0L2N5uEGipTjVAjKwxbzQIDAQAB AoGBAKD/8WI8HsGvm2GVxQMSqO+58xIgn0LCu/r/C9/fjo/2Kue8YDM4Ed1EudSE T+vuJ2updVZtLFKzH6ochy8xrp9wbEPhhQqAl8OEvjFa932mSNqg69403qU57eu1 6j0gqE7Z8pQAyM685RwAv4UujRrT1Vi3EmHu8Rvl7/4MRn+BAkEAwBVzxD7XbEoi lGCoKSvs2JczVIpOOp8s1FCAvsvbRBKzuqLI9N8J9ljM+SalLecalFqsDMJdx4b5 nfPPoK2wswJBAOAYfrih+3pKznkiW56oVlsTBw20baJpD7D9mRBUkURpu/6zaSLh 7mPP5nxnw+92jOmwXW/9/fqSmOT/wNKLAX8CQHxLZWYP5AZG0qmBAk/iBJkC/kwu UwDMD44zqZvM/k3GbFbRD4ik6yVlwP1YbHqPmTt2kO6Qt25tgQkcw7YNrtECQQDG xLNR7Utz03qEc3c0qTe/nIb7HvjHv8yNPsNPetNHDUoUxuoQaLncioF0A04Fzu96 MFcQiQeIBBPduwQ7O9TdAkEAvsQWuNmvVFTg0KQLP2Ity5wHSwoIRWjUq19ciHZu acFIvir+6kDMWEooqY0uXfKx7UUWJAMCjVJztFoSp/jobw== -----END RSA PRIVATE KEY-----""" class ConchSSHFactory(factory.SSHFactory): """ This is the entry point of our SSH server implementation. The SSH transport layer is implemented by L{SSHTransport} and is the protocol of this factory. Here we configure the server's identity (host keys) and handlers for the SSH services: * L{connection.SSHConnection} handles requests for the channel multiplexing service. * L{userauth.SSHUserAuthServer} handlers requests for the user authentication service. """ protocol = SSHServerTransport services = { 'ssh-userauth': userauth.SSHUserAuthServer, 'ssh-connection': connection.SSHConnection } def __init__(self, portal): self.portal = portal self.publicKeys = { 'ssh-rsa': keys.Key.fromString(data=publicKey) } self.privateKeys = { 'ssh-rsa': keys.Key.fromString(data=privateKey) } def get_manhole_factory(namespace, **passwords): realm = manhole_ssh.TerminalRealm() def get_manhole(_): return manhole.Manhole(namespace) realm.chainedProtocolFactory.protocolFactory = get_manhole p = portal.Portal(realm) checker = checkers.InMemoryUsernamePasswordDatabaseDontUse(**passwords) p.registerChecker(checker) return ConchSSHFactory(p) console = get_manhole_factory(pbRoot.initCommands(), admin='admin') internet.TCPServer(9999, console, interface='localhost').setServiceParent(application) For memory debugging I found these few lines helpful exc = [ "function", "type", "list", "dict", "tuple", "wrapper_descriptor", "module", "method_descriptor", "member_descriptor", "instancemethod", "builtin_function_or_method", "frame", "classmethod", "classmethod_descriptor", "_Environ", "MemoryError", "_Printer", "_Helper", "getset_descriptor", "weakreaf" ] inc = [ ] prev = {} def dumpObjects(delta=True, limit=0, include=inc, exclude=[]): global prev if include != [] and exclude != []: print 'cannot use include and exclude at the same time' return print 'working with:' print ' delta: ', delta print ' limit: ', limit print ' include: ', include print ' exclude: ', exclude objects = {} gc.collect() oo = gc.get_objects() for o in oo: if getattr(o, "__class__", None): name = o.__class__.__name__ if ((exclude == [] and include == []) or \ (exclude != [] and name not in exclude) or \ (include != [] and name in include)): objects[name] = objects.get(name, 0) + 1 ## if more: ## print o pk = prev.keys() pk.sort() names = objects.keys() names.sort() for name in names: if limit == 0 or objects[name] > limit: if not prev.has_key(name): prev[name] = objects[name] dt = objects[name] - prev[name] if delta or dt != 0: print '%0.6d -- %0.6d -- ' % (dt, objects[name]), name prev[name] = objects[name] def getObjects(oname): """ gets an object list with all the named objects out of the sea of gc'ed objects """ olist = [] objects = {} gc.collect() oo = gc.get_objects() for o in oo: if getattr(o, "__class__", None): name = o.__class__.__name__ if (name == oname): olist.append(o) return olist On 4/19/20 00:03, salil GK wrote:
![](https://secure.gravatar.com/avatar/eee0b758e45f8555f03352f74f723409.jpg?s=120&d=mm&r=g)
On dg., abr. 19 2020, salil GK wrote:
As glyph mentioned, you really should be running newer Twisted; it is entirely possible that many recommendations would not work because of the old version. That being said, you can peek in this repository, which has a manhole over SSH and public key-based auth: https://github.com/evilham/isalive.me/blob/master/isaliveme/conch_helpers.py (At least parts of the code in that file should end up in Twisted some day) Or peek in matrix-synapse's code, which has a telnet-based one. Or these, even more minimal examples: https://twistedmatrix.com/documents/current/conch/examples/index.html -- Evilham
![](https://secure.gravatar.com/avatar/63ef3e1c4ce734bfc654bfd31a5ff041.jpg?s=120&d=mm&r=g)
Don't know if this is still state of the art, but I've been using this off and on and please don't use checkers.InMemoryUsernamePasswordDatabaseDontUse HTH, Werner #### controlling terminal command line server # #on port 9999. Connect with # #ssh admin@localhost -p 9999 # #accept new keys and start work # from twisted.conch import manhole, manhole_ssh from twisted.conch.ssh import keys, userauth, factory, connection from twisted.conch.ssh.transport import SSHServerTransport from twisted.cred import portal, checkers """ Generate these keys with $ ckeygen -t rsa -f ssh-keys/ssh_host_rsa_key $ ckeygen -t rsa -f ssh-keys/client_rsa ssh_host_rsa_key.pub is the public key file ssh_host_rsa_key is the private key file """ publicKey = """ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQCoJSZjqVJdh36RoNvwsTMNlqk8heFV6DxNhhkw3uZ58uyrsnz+7JHGpu56tJAO39WMJPLQfdtyWxCy1P2RRKh65hGznvyt2UaAHdZDMqbNcMlupFPg8P4pCgR+/2M14QSopqysfjgXUJRfR/pD93Ofy0L2N5uEGipTjVAjKwxbzQ== wthie@qubik.local""" privateKey = """-----BEGIN RSA PRIVATE KEY----- MIICXgIBAAKBgQCoJSZjqVJdh36RoNvwsTMNlqk8heFV6DxNhhkw3uZ58uyrsnz+ 7JHGpu56tJAO39WMJPLQfdtyWxCy1P2RRKh65hGznvyt2UaAHdZDMqbNcMlupFPg 8P4pCgR+/2M14QSopqysfjgXUJRfR/pD93Ofy0L2N5uEGipTjVAjKwxbzQIDAQAB AoGBAKD/8WI8HsGvm2GVxQMSqO+58xIgn0LCu/r/C9/fjo/2Kue8YDM4Ed1EudSE T+vuJ2updVZtLFKzH6ochy8xrp9wbEPhhQqAl8OEvjFa932mSNqg69403qU57eu1 6j0gqE7Z8pQAyM685RwAv4UujRrT1Vi3EmHu8Rvl7/4MRn+BAkEAwBVzxD7XbEoi lGCoKSvs2JczVIpOOp8s1FCAvsvbRBKzuqLI9N8J9ljM+SalLecalFqsDMJdx4b5 nfPPoK2wswJBAOAYfrih+3pKznkiW56oVlsTBw20baJpD7D9mRBUkURpu/6zaSLh 7mPP5nxnw+92jOmwXW/9/fqSmOT/wNKLAX8CQHxLZWYP5AZG0qmBAk/iBJkC/kwu UwDMD44zqZvM/k3GbFbRD4ik6yVlwP1YbHqPmTt2kO6Qt25tgQkcw7YNrtECQQDG xLNR7Utz03qEc3c0qTe/nIb7HvjHv8yNPsNPetNHDUoUxuoQaLncioF0A04Fzu96 MFcQiQeIBBPduwQ7O9TdAkEAvsQWuNmvVFTg0KQLP2Ity5wHSwoIRWjUq19ciHZu acFIvir+6kDMWEooqY0uXfKx7UUWJAMCjVJztFoSp/jobw== -----END RSA PRIVATE KEY-----""" class ConchSSHFactory(factory.SSHFactory): """ This is the entry point of our SSH server implementation. The SSH transport layer is implemented by L{SSHTransport} and is the protocol of this factory. Here we configure the server's identity (host keys) and handlers for the SSH services: * L{connection.SSHConnection} handles requests for the channel multiplexing service. * L{userauth.SSHUserAuthServer} handlers requests for the user authentication service. """ protocol = SSHServerTransport services = { 'ssh-userauth': userauth.SSHUserAuthServer, 'ssh-connection': connection.SSHConnection } def __init__(self, portal): self.portal = portal self.publicKeys = { 'ssh-rsa': keys.Key.fromString(data=publicKey) } self.privateKeys = { 'ssh-rsa': keys.Key.fromString(data=privateKey) } def get_manhole_factory(namespace, **passwords): realm = manhole_ssh.TerminalRealm() def get_manhole(_): return manhole.Manhole(namespace) realm.chainedProtocolFactory.protocolFactory = get_manhole p = portal.Portal(realm) checker = checkers.InMemoryUsernamePasswordDatabaseDontUse(**passwords) p.registerChecker(checker) return ConchSSHFactory(p) console = get_manhole_factory(pbRoot.initCommands(), admin='admin') internet.TCPServer(9999, console, interface='localhost').setServiceParent(application) For memory debugging I found these few lines helpful exc = [ "function", "type", "list", "dict", "tuple", "wrapper_descriptor", "module", "method_descriptor", "member_descriptor", "instancemethod", "builtin_function_or_method", "frame", "classmethod", "classmethod_descriptor", "_Environ", "MemoryError", "_Printer", "_Helper", "getset_descriptor", "weakreaf" ] inc = [ ] prev = {} def dumpObjects(delta=True, limit=0, include=inc, exclude=[]): global prev if include != [] and exclude != []: print 'cannot use include and exclude at the same time' return print 'working with:' print ' delta: ', delta print ' limit: ', limit print ' include: ', include print ' exclude: ', exclude objects = {} gc.collect() oo = gc.get_objects() for o in oo: if getattr(o, "__class__", None): name = o.__class__.__name__ if ((exclude == [] and include == []) or \ (exclude != [] and name not in exclude) or \ (include != [] and name in include)): objects[name] = objects.get(name, 0) + 1 ## if more: ## print o pk = prev.keys() pk.sort() names = objects.keys() names.sort() for name in names: if limit == 0 or objects[name] > limit: if not prev.has_key(name): prev[name] = objects[name] dt = objects[name] - prev[name] if delta or dt != 0: print '%0.6d -- %0.6d -- ' % (dt, objects[name]), name prev[name] = objects[name] def getObjects(oname): """ gets an object list with all the named objects out of the sea of gc'ed objects """ olist = [] objects = {} gc.collect() oo = gc.get_objects() for o in oo: if getattr(o, "__class__", None): name = o.__class__.__name__ if (name == oname): olist.append(o) return olist On 4/19/20 00:03, salil GK wrote:
![](https://secure.gravatar.com/avatar/eee0b758e45f8555f03352f74f723409.jpg?s=120&d=mm&r=g)
On dg., abr. 19 2020, salil GK wrote:
As glyph mentioned, you really should be running newer Twisted; it is entirely possible that many recommendations would not work because of the old version. That being said, you can peek in this repository, which has a manhole over SSH and public key-based auth: https://github.com/evilham/isalive.me/blob/master/isaliveme/conch_helpers.py (At least parts of the code in that file should end up in Twisted some day) Or peek in matrix-synapse's code, which has a telnet-based one. Or these, even more minimal examples: https://twistedmatrix.com/documents/current/conch/examples/index.html -- Evilham
participants (5)
-
Evilham
-
Glyph
-
Richard van der Hoff
-
salil GK
-
Werner Thie