On 01:05 pm, maris@chown.lv wrote:
Hello! Currently I'm trying to write small xmlrpc server for html data processing. Processing is done by html tidy lib, but the problem is that it has massive memory leak. As processing is blocking operation I'm running it in thread, but after some time and huge html document processing daemon eats all memory. I wondering if its possible to load utidylib in thread, do processing and after this kill thread and release memory? Or maybe something like deferToProcess? Thanks in advance!
#!/usr/bin/env python # -*- coding: utf-8 -*-
import utidylib
from twisted.internet import epollreactor epollreactor.install()
from twisted.internet import protocol, defer, threads, reactor from twisted.web import xmlrpc, server from twisted.python import log, threadpool
import sys reload(sys) sys.setdefaultencoding('utf-8')
log.startLogging(sys.stdout)
import codecs
import gc gc.enable() gc.set_debug(gc.DEBUG_LEAK)
Amongst other things, gc.set_debug(gc.DEBUG_LEAK) means "never free any object that is part of a cycle". That is, it's a good way to introduce lots of new object leaks into your program. Jean-Paul