[lxml-dev] [PATCH] 1.3 Fatal Python error: deallocating None
All - I have identified a bug with the new attrib.pop() method in lxml-1.3 There is a piece of code in etree.pyx (line 1483) related to using the provided default if the key lookup fails. The issue is that the returned value (from PyTuple_GET_ITEM) is a borrowed reference. I was using it as such: v = elt.attrib.get(k, None) And eventually (after several iterations of my parsing loop while processing various documents) I would get this error: Fatal Python error: deallocating None I was able to create a trivial test case to reproduce: import lxml.etree as etree xml = '''\ <?xml version="1.0"?> <xml/>''' for i in range(10000): et = etree.fromstring(xml) et.attrib.pop('x', None) And the patched code replaces the pop() method in etree.pyx with: def pop(self, key, *default): if python.PyTuple_GET_SIZE(default) > 1: raise TypeError, "pop expected at most 2 arguments, got %d" % \ (python.PyTuple_GET_SIZE(default)+1) result = _getAttributeValue(self._element, key, None) if result is None: if python.PyTuple_GET_SIZE(default) == 0: raise KeyError, key else: result = python.PyTuple_GET_ITEM(default, 0) python.Py_INCREF(result) else: _delAttribute(self._element, key) return result This solves the issue for me, and should not introduce any memory leaks, as the returned object in the defaulted path should be INCREF'd. Thanks, --Dave
David M. Grimes wrote:
I have identified a bug with the new attrib.pop() method in lxml-1.3 There is a piece of code in etree.pyx (line 1483) related to using the provided default if the key lookup fails. The issue is that the returned value (from PyTuple_GET_ITEM) is a borrowed reference.
Thanks for reporting this. Fixed on trunk and branch. Stefan
participants (2)
-
David M. Grimes -
Stefan Behnel