Bug in Python?
Sven R. Kunze
srkunze at mail.de
Sun Feb 28 05:08:55 EST 2016
On 27.02.2016 00:07, eryk sun wrote:
> On Fri, Feb 26, 2016 at 4:08 PM, Sven R. Kunze <srkunze at mail.de> wrote:
>> Python sometimes seems not to hop back and forth between C and Python code.
>> Can somebody explain this?
> Normally a C extension would call PySequence_SetItem, which would call
> the type's sq_ass_item, which for MyList is slot_sq_ass_item. The
> latter function bridges the CPython and Python sides by binding and
> calling the overridden __setitem__ method. However, the _heapq
> extension module uses `PyList_SET_ITEM(heap, 0, lastelt)`. This macro
> expands to `((PyListObject *)(heap))->ob_item = lastelt`. This
> directly modifies the internal ob_item array of the list, so the
> overridden __setitem__ method is never called. I presume it was
> implemented like this with performance in mind, but I don't know
> whether or not that justifies the loss of generality.
I think this is true and it explains the huge performance penalty of the
current RemovalHeap and XHeap implementation as it basically uses Python
only (results here: http://bit.ly/1KU7CyW).
Shoot! I could have seen this earlier. I thought the performance penalty
was due to calling __setitem__ and dict operations. But having all heap
operations carried out in Python slows things down considerably of course.
Let's see if I can manage to create a more efficient mark-and-sweep
approach which uses the C module.
More information about the Python-list