<div dir="ltr"><br><div class="gmail_quote"><div dir="ltr"><br>Hello, I want to swap the content of two dictionaries, the obvious way to do it is:<br>a = {1:"I'am A"}<br>b = {2:"I'm B"}<br>temp = a<br>
a = b<br>b = temp<br><br>However, consider the case in which the dictionary we are referencing lives in another module:<br>
#external.py<br>#<br>#a = {1:"I'am A}<br>import external<br><br>temp = external.a<br>external.a = b<br>b = temp<br><br>Well, still this would work great and anyone later that would refereence external.a would find the content of b, but consider the case where someone else imported "external.py" before my module run, for example:<br>
#external.py<br>#<br>#a = {1:"I'm A}<br><br>#annoyer.py<br>#from external import a<br>#<br># #Do other stuff with a<br>import external<br><br>temp = external.a<br>external.a = b<br>b = temp<br><br>Now in such case when I overwrite the content of external.a in my module, the change in dictionary is not reflected in annoyer module, that's because annoyer hold a reference to the original a but not the new one. annoyer module still see the old content of a.<br>
<br>The workaround that i did to solve this problem is the following (I'm simplifying it right here):<br><br>tempKeys = a.keys()<br>diffKeys = a.keys() - b.keys() #I do it using set()s<br><br>#item = a<br>temp = {}<br>
for key in a.keys():<br> item[key] = a[key]<br><br>for key in diffKeys:<br> del a[key] #delete stuff that exist in a but not b.<br><br>for key in b.keys():<br> a[key] = b[key]<br><br>b = temp<br><br>This works great as the content referenced by the dictionary is changed rather than changing the reference of dictionary itself so it's reflected by any "scope" that references the dictionary.<br>
<br>My problem is that i need to do this operation a LOT, simply I got a problem that my program go through a very long loop and inside this loop this operation is needed to be done twice and the dictionary size ain't very small.<br>
<br>If anyone has a suggestion one how to do it faster then please feel free and welcome to contribute.<br><br>Anyway, I decided to go and implement it in C to gain performance, And since I got no expereince in writing C extension I'm not really sure if what I'm doing right or wrong, long story short here is what i decided to do:<br>
In real python (I mean not the C code) an object is mainly a reference to a C PyObject (or an inheritance of the PyObject).<br>Python dictionaries are represented as PyDictObject (PyDictObject inherits PyObject), so every Python Dictionary is a reference to PyDictObject, <br>
<br>So what i came up with to swap two Python dictionaries is to swap the content of PyDictObjects of the dictionaries, I know that "illegal" way of doing things in Python but I need the hack to gain performance.<br>
<br>I opened up python source files: <br>object.h (extracted the definition of PyObject)<br>dictObject.g (extctracted the definition of PyDictObject)<br><br>And I've written the following C extension:<br><br><div>
<div style="font-family: monospace;"><ol><li><div><span>#include
<Python.h></span></div></li><li><div><span>#include
<dictobject.h> //I know that it was already imported</span></div></li><li><div> </div></li><li><div><span>static</span> <span>char</span> swap_doc<span>[</span><span>]</span> = </div>
</li><li><div><span>"My try to swap
dicts, isa hopefully it will work good."</span>;</div></li><li><div> </div></li><li><div> </div></li><li><div><span>static</span>
PyObject*</div></li><li><div>swap_swapDict<span>(</span>PyObject
*self, PyObject *args<span>)</span></div></li><li><div><span>{</span></div></li><li><div> PyObject *a, *b;</div></li><li><div>
PyDictObject *x, *y;</div></li><li><div> <span>//Temp
PyDictObject attriubutes</span></div></li><li><div> Py_ssize_t ob_refcnt;</div></li><li><div> PyTypeObject *ob_type;</div></li><li><div>
Py_ssize_t ma_fill; </div></li><li><div> Py_ssize_t ma_used; </div></li><li><div> Py_ssize_t ma_mask;</div></li><li><div> PyDictEntry *ma_table;</div>
</li><li><div> PyDictEntry *<span>(</span>*ma_lookup<span>)</span><span>(</span>PyDictObject *mp,
PyObject *key, <span>long</span> hash<span>)</span>;</div></li><li><div> PyDictEntry *ma_smalltable;<span>//[PyDict_MINSIZE];</span></div></li><li>
<div> </div></li><li><div> </div></li><li><div> <span>if</span> <span>(</span>!PyArg_UnpackTuple<span>(</span>args, <span>"swapDict"</span>, <span>2</span>, <span>2</span>, &a, &b<span>)</span><span>)</span> <span>{</span></div>
</li><li><div> <span>return</span>
<span>NULL</span>;</div></li><li><div> <span>}</span></div></li><li><div> </div></li><li><div> <span>//Make sure
that they are dictionaries</span></div></li><li><div> <span>if</span> <span>(</span>!PyDict_Check<span>(</span>a<span>)</span> && !PyDict_Check<span>(</span>b<span>)</span><span>)</span> <span>{</span></div>
</li><li><div> PyErr_SetString<span>(</span>PyExc_TypeError,</div></li><li><div> <span>"dictionary required, something else was passed"</span><span>)</span>;</div>
</li><li><div> <span>return</span>
<span>NULL</span>;</div></li><li><div> <span>}</span></div></li><li><div> </div></li><li><div> x = <span>(</span>PyDictObject*<span>)</span>a;</div>
</li><li><div> y = <span>(</span>PyDictObject*<span>)</span>b;</div></li><li><div> </div></li><li><div> ob_refcnt = x->ob_refcnt;</div>
</li><li><div> x->ob_refcnt =
y->ob_refcnt;</div></li><li><div> y->ob_refcnt = ob_refcnt;</div></li><li><div> </div></li><li><div> ob_type = x->ob_type;</div>
</li><li><div> x->ob_type = y->ob_type;</div></li><li><div> y->ob_type = ob_type;</div></li><li><div> </div></li><li>
<div> ma_fill = x->ma_fill;</div></li><li><div> x->ma_fill = y->ma_fill;</div></li><li><div> y->ma_fill = ma_fill;</div></li><li>
<div> </div></li><li><div> ma_used = x->ma_used;</div></li><li><div> x->ma_used = y->ma_used;</div></li><li><div> y->ma_used = ma_used;</div>
</li><li><div> </div></li><li><div> ma_mask = x->ma_mask;</div></li><li><div> x->ma_mask = y->ma_mask;</div></li><li>
<div> y->ma_mask = ma_mask;</div></li><li><div> </div></li><li><div> ma_table = x->ma_table;</div></li><li><div> x->ma_table =
y->ma_table;</div></li><li><div> y->ma_table = ma_table;</div></li><li><div> </div></li><li><div> ma_lookup = x-> ma_lookup;</div>
</li><li><div> x->ma_lookup = y->
ma_lookup;</div></li><li><div> y->ma_lookup = ma_lookup;</div></li><li><div> </div></li><li><div> ma_smalltable =
x->ma_smalltable;</div></li><li><div> *<span>(</span>x->ma_smalltable<span>)</span> = *<span>(</span>y->ma_smalltable<span>)</span>;</div>
</li><li><div> *<span>(</span>y->ma_smalltable<span>)</span> = *<span>(</span>ma_smalltable<span>)</span>;</div></li><li><div>
</div></li><li><div> <span>return</span>
Py_None;</div></li><li><div><span>}</span></div></li><li><div> </div></li><li><div><span>//This is extracted
form dictObject.h and object.h</span></div></li><li><div><span>//typedef struct
_dictobject PyDictObject;</span></div></li><li><div><span>//struct _dictobject {</span></div></li><li><div><span>// PyObject_HEAD</span></div></li>
<li><div><span>// Py_ssize_t
ma_fill; /* # Active + # Dummy */</span></div></li><li><div><span>// Py_ssize_t
ma_used; /* # Active */</span></div></li><li><div><span>//</span></div></li><li><div><span>// /* The table
contains ma_mask + 1 slots, and that's a power of 2.</span></div></li><li><div><span>// * We store
the mask instead of the size because the mask is more</span></div></li><li><div><span>// * frequently
needed.</span></div></li><li><div><span>// */</span></div></li><li><div><span>// Py_ssize_t
ma_mask;</span></div></li><li><div><span>//</span></div></li><li><div><span>// /* ma_table
points to ma_smalltable for small tables, else to</span></div></li><li><div><span>// * additional
malloc'ed memory. ma_table is never NULL! This rule</span></div></li><li><div><span>// * saves
repeated runtime null-tests in the workhorse getitem and</span></div></li><li><div><span>// * setitem
calls.</span></div></li><li><div><span>// */</span></div></li><li><div><span>// PyDictEntry
*ma_table;</span></div></li><li><div><span>// PyDictEntry
*(*ma_lookup)(PyDictObject *mp, PyObject *key, long hash);</span></div></li><li><div><span>// PyDictEntry
ma_smalltable[PyDict_MINSIZE];</span></div></li><li><div><span>//};</span></div></li><li><div><span>//PyObject_Head is a
simple two statements equal to:</span></div></li><li><div><span>//Py_ssize_t
ob_refcnt; </span></div></li><li><div><span>//struct _typeobject
*ob_type;</span></div></li><li><div> </div></li><li><div><span>static</span> <span>char</span> swap_swapDict_doc<span>[</span><span>]</span> = </div>
</li><li><div><span>"Replaces two dicts
with each other."</span>;</div></li><li><div> </div></li><li><div><span>static</span>
PyMethodDef swap_methods<span>[</span><span>]</span>
= <span>{</span></div></li><li><div> <span>{</span><span>"swapDict"</span>, swap_swapDict, METH_VARARGS,
swap_swapDict_doc<span>}</span>,</div></li><li><div> <span>{</span><span>NULL</span>, <span>NULL</span><span>}</span></div></li><li>
<div><span>}</span>;</div></li><li><div> </div></li><li><div>PyMODINIT_FUNC</div></li><li><div>initswap<span>(</span><span>void</span><span>)</span></div>
</li><li><div><span>{</span></div></li><li><div> Py_InitModule3<span>(</span><span>"swap"</span>, swap_methods, swap_doc<span>)</span>;</div>
</li><li><div><span>}</span> </div></li></ol></div></div><br><br>What the code mainly do is that it swap the content of the PyDict objects. However I got a bug where one of them swaps correctly while the other points to a displaced memory location.<br>
<br></div>
</div><br></div>