2013/11/6 Brian Bird <Brian.Bird@securetrading.com>
I’ve tried the same thing using Python 3.2 since that uses newer versions of Lxml and LibXml and the same problem exists. In fact, I think I’ve traced the problem to lxml.etree.XPath when it’s compiling an xpath expression. Even disabling smart_strings has no effect.
I’ve also read http://lxml.de/FAQ.html#id1 but I don’t think any of that applies as I’m not running the threads concurrently, I’m just using a thread (in this example anyway) and it’s keeping something in memory.
I’ve also been looking at the error log that lxml seems to store, but I don’t think that applies since there should be no errors in this case (and it only seems to store ~100 messages but in my example the memory just keeps on growing)
Could anyone confirm if this is really a bug?
Thanks,
There is a memory leak indeed, I think I identified it coming from the initXPathParserDict function (in parser.pxi). The (libxml) reference count of pctxt.dict is a bit difficult to follow, but when I added a xmlparser.xmlDictFree(pctxt.dict) the memory leak disappeared. -- Amaury Forgeot d'Arc