Multiple python interpreter contexts
I'm trying to make two separate contexts for my embedded python application. That is, if I set some the X variable in one, it does not set X in the other. I tried to do this by copying the dictionary like this: object mainModule(handle<>(borrowed(PyImport_AddModule("__main__")))); dict pyNamespace = dict(mainModule.attr("__dict__")); dict pyNamespace2 = dict(pyNamespace); handle<> result(PyRun_String( "test = 1\n", Py_file_input, pyNamespace.ptr(), pyNamespace.ptr()) ); handle<> result2(PyRun_String( "test = 2\n", Py_file_input, pyNamespace2.ptr(), pyNamespace2.ptr()) ); handle<> result3(PyRun_String( "print test\n", Py_file_input, pyNamespace.ptr(), pyNamespace.ptr()) ); handle<> result4(PyRun_String( "print test\n", Py_file_input, pyNamespace2.ptr(), pyNamespace2.ptr()) ); But this always prints out 2, 2 instead of 1, 2 like I want it to. What am I doing wrong here? Thanks, Clay
Clay Culver wrote:
I'm trying to make two separate contexts for my embedded python application. That is, if I set some the X variable in one, it does not set X in the other. I tried to do this by copying the dictionary like this:
object mainModule(handle<>(borrowed(PyImport_AddModule("__main__")))); dict pyNamespace = dict(mainModule.attr("__dict__")); dict pyNamespace2 = dict(pyNamespace);
It turns out that this does a shallow copy, i.e. pyNamespace2.ptr() == pyNamespace.ptr(), and thus modifications to one are immediately visible in the other. I'm a bit puzzled by this, too, as comments in dict.hpp indicate: // dict() -> new empty dictionary. // dict(mapping) -> new dictionary initialized from a mapping object's // (key, value) pairs. // dict(seq) -> new dictionary initialized as if via: As there is no mention of the copy constructor I would assume the one that applies is 'dict(mapping)', but it doesn't appear to. David ? In python, you have to explicitely ask for a deep copy by calling the 'copy' method. That should work from within C++, too, i.e. dict pyNamespace2 = pyNamespace.copy() HTH, Stefan
Stefan Seefeld wrote:
It turns out that this does a shallow copy, i.e. pyNamespace2.ptr() == pyNamespace.ptr(), and thus modifications to one are immediately visible in the other.
I'm a bit puzzled by this, too, as comments in dict.hpp indicate:
// dict() -> new empty dictionary. // dict(mapping) -> new dictionary initialized from a mapping object's // (key, value) pairs. // dict(seq) -> new dictionary initialized as if via:
As there is no mention of the copy constructor I would assume the one that applies is 'dict(mapping)', but it doesn't appear to. David ?
In python, you have to explicitely ask for a deep copy by calling the 'copy' method. That should work from within C++, too, i.e.
dict pyNamespace2 = pyNamespace.copy()
Thanks, that did it. I'm still a bit baffled by that as well. Certainly in Python that's how it's done: calling the constructor explicitly will perform a deep copy, giving you a new object. Clay
--- Clay Culver <clay@idleengineer.net> wrote:
I'm a bit puzzled by this, too, as comments in dict.hpp indicate:
// dict() -> new empty dictionary. // dict(mapping) -> new dictionary initialized from a mapping object's // (key, value) pairs. // dict(seq) -> new dictionary initialized as if via:
... Thanks, that did it. I'm still a bit baffled by that as well. Certainly in Python that's how it's done: calling the constructor explicitly will perform a deep copy, giving you a new object.
http://www.boost.org/libs/python/doc/tutorial/doc/html/python/object.html Look for "Beware" and "astute". __________________________________ Do you Yahoo!? Read only the mail you want - Yahoo! Mail SpamGuard. http://promotions.yahoo.com/new_mail
Ralf W. Grosse-Kunstleve wrote:
http://www.boost.org/libs/python/doc/tutorial/doc/html/python/object.html
Look for "Beware" and "astute".
" Beware the common pitfall of forgetting that the constructors of most of Python's mutable types make copies, just as in Python. Python:
d = dict(x.__dict__) # copies x.__dict__ d['whatever'] # modifies the copy
" This is confusing. May be I'm reading the above not correctly, but in my interpretation 'making a copy' means making a deep copy, and 'modifies the copy' implies the original is unchanged. However:
class A: ... def foo(self): return 42 ... d = A.__dict__ print id(d), id(A.__dict__) -1208268764 -1208268764 d['foo'] = 'bar' print A.__dict__ {'__module__': '__main__', 'foo': 'bar', '__doc__': None}
confirms our previous finding, i.e. 'd = A.__dict__' does *not* create a copy of A's dictionary, but instead creates a new reference ('binding') to it. Can you please clarify the meaning of 'copy' in the documentation ? Thanks a lot, Stefan
Stefan Seefeld wrote:
Ralf W. Grosse-Kunstleve wrote:
http://www.boost.org/libs/python/doc/tutorial/doc/html/python/object.html
Look for "Beware" and "astute".
"
Beware the common pitfall of forgetting that the constructors of most of Python's mutable types make copies, just as in Python.
Python:
d = dict(x.__dict__) # copies x.__dict__ d['whatever'] # modifies the copy
"
Sorry, I realized the explicit use of the 'dict()' constructor making the difference just after I hit the 'send' button. Still, the original question remains, as the original poster did use something like 'dict ns2 = dict(ns1)' and yet both objects had the same id, i.e. pointed to the same object internally. So it appears in that particular case dict(foobar) didn't really create a copy of foobar. Thanks, Stefan
Stefan Seefeld wrote:
Python:
d = dict(x.__dict__) # copies x.__dict__ d['whatever'] # modifies the copy
"
This is confusing. May be I'm reading the above not correctly, but in my interpretation 'making a copy' means making a deep copy, and 'modifies the copy' implies the original is unchanged. However:
class A: ... def foo(self): return 42 ... d = A.__dict__ print id(d), id(A.__dict__) -1208268764 -1208268764
Following the code above that though:
class A: ... def foo(self): ... return 42 ... d = dict( A.__dict__ ) print id(d), id( A.__dict__ ) 10356464 10357184
Which is a deep copy. The C++ doesn't seem to do the same thing when you pass a value into dict( ). Clay
Hi all, Clay Culver wrote:
Stefan Seefeld wrote:
Python:
d = dict(x.__dict__) # copies x.__dict__ d['whatever'] # modifies the copy
"
This is confusing. May be I'm reading the above not correctly, but in my interpretation 'making a copy' means making a deep copy, and 'modifies the copy' implies the original is unchanged. However:
class A: ... def foo(self): return 42 ... d = A.__dict__ print id(d), id(A.__dict__) -1208268764 -1208268764
Following the code above that though:
class A: ... def foo(self): ... return 42 ... d = dict( A.__dict__ ) print id(d), id( A.__dict__ ) 10356464 10357184
Which is a deep copy.
The C++ doesn't seem to do the same thing when you pass a value into dict( ).
In Python we got two types of copies, deep and shallow. A shallow copy of a container creates a new container, but the items are still the same. Changing the container itself doesn't affect the original, but changing one of the items does. Consider:
class C: pass ... c = C() c.x = 1 a = [c] b = list(a) a [<__main__.C instance at 0x009D9C10>] b [<__main__.C instance at 0x009D9C10>] a.append(1) a [<__main__.C instance at 0x009D9C10>, 1] b [<__main__.C instance at 0x009D9C10>] a[0].x = 4 b[0].x 4
Note that "list(seq)" creates a shallow copy of "seq", ie, its a different list, but with the same elements (on a side note, "copy" from the copy module provides a shallow copy operation). That's why appending something to "a" doesn't change "b", but changing C's instance does, because both lists point to the same object. This confusion is common in C++ programmers, because variables in C++ have value semantics, while in Python variables are only bindings to objects. To make it clear what is a binding, think of it as a pointer:
c.x 4 d = c d.x = 10 c.x 10
The line "d = c" doesn't make any copies. Instead, now the *binding* d points to the same object as "c", exactly as if both were pointers. For completeness sake, here is an example of a deep copy, which copies not only the container, but all the elements inside the container recursively:
import copy c.x = 4 a = [c] b = copy.deepcopy(a) a[0].x = 0 b[0].x 4
As you can see, now a deep copy is being made: both the lists and elements are being copied. Hope that clears it up! Regards, Nicodemus.
Hi Nicodemus, Nicodemus wrote:
Hope that clears it up!
Not really, unfortunately. The confusion stems from the C++ code dict pyNamespace2 = dict(pyNamespace); not doing the same as 'dict()' does in python, i.e. not copying the dictionary but instead only the pointer. The python side itself is clear. Regards, Stefan
participants (4)
-
Clay Culver -
Nicodemus -
Ralf W. Grosse-Kunstleve -
Stefan Seefeld