Abstract class instances to-python conversion
Hi, I'm currently embedding and extending Python using Boost.Python. I'm trying to expose some c++ library (OIS) classes to my Python scripts. This library mainly exposes abstract classes since the actual implementation are derived classes specialized for each operating system. So, i have to wrap those abstract classes. Then, eventually i will convert some existing C++ object of this class to its Python equivalent. Ideally, I would think that the HeldType is a pointer to the existing C++ object. There is no pointer managment problem since the lifetime of the object is greater than the execution time of the script. There is probably something i don't understand in the design of Boost.Python. At this point, my problem is that i need a to_python conversion which requires the abscence of noncopyable attribute which then implies to be able to build an instance of the object (which i cannot provide since the class is abstract, and i don't have any concrete derived class). I don't yet understand why holding a pointer in the Python object requires the ability to build instances of the wrapped class. The exact compile-time error i get is: boost_1_44/boost/python/object/pointer_holder.hpp:194:14: error: cannot allocate an object of abstract type 'OIS::Keyboard' I'm using boost 1.44 (required by some other library) Python 3.2 and MinGW/Msys. Module declaration: BOOST_PYTHON_MODULE(OIS) { class_<Keyboard, Keyboard*>("Keyboard", no_init); } Python init: try { PyImport_AppendInittab("OIS", PyInit_OIS); Py_InitializeEx(0); object main(import("__main__")); dict globals(main.attr("__dict__")); globals["keyboard"] = ptr(keyboard); } catch (error_already_set&) { PyErr_Print(); } Did i choose the wrong desing ? Did i do something wrong ? What should i do to solve this problem ? Valentin Perrelle.
On 07/30/2011 01:15 PM, Valentin Perrelle wrote:
Hi,
I'm currently embedding and extending Python using Boost.Python. I'm trying to expose some c++ library (OIS) classes to my Python scripts. This library mainly exposes abstract classes since the actual implementation are derived classes specialized for each operating system.
So, i have to wrap those abstract classes. Then, eventually i will convert some existing C++ object of this class to its Python equivalent. Ideally, I would think that the HeldType is a pointer to the existing C++ object. There is no pointer managment problem since the lifetime of the object is greater than the execution time of the script.
In this case, I don't you need to specify a HeldType, because that only affects what happens when you construct the object in Python, and you aren't every doing that. But you do need to specify "noncopyable" if the class doesn't have a copy constructor...
There is probably something i don't understand in the design of Boost.Python. At this point, my problem is that i need a to_python conversion which requires the abscence of noncopyable attribute which then implies to be able to build an instance of the object (which i cannot provide since the class is abstract, and i don't have any concrete derived class). I don't yet understand why holding a pointer in the Python object requires the ability to build instances of the wrapped class. The exact compile-time error i get is:
boost_1_44/boost/python/object/pointer_holder.hpp:194:14: error: cannot allocate an object of abstract type 'OIS::Keyboard'
Why do you think you can't have noncopyable? You can certainly support some to-python conversions (including the one you've invoked below with "ptr(keyboard)") on a noncopyable class. Are you hoping to support some specific one, or is this the root of your misunderstanding? Anyhow, I'd recommend trying class_<Keyboard, noncopyable>("Keyboard", no_init); with the rest of what you have. Good luck! Jim Bosch
BOOST_PYTHON_MODULE(OIS) { class_<Keyboard, Keyboard*>("Keyboard", no_init); }
Python init:
try { PyImport_AppendInittab("OIS", PyInit_OIS); Py_InitializeEx(0);
object main(import("__main__")); dict globals(main.attr("__dict__")); globals["keyboard"] = ptr(keyboard); } catch (error_already_set&) { PyErr_Print(); }
Thank you for your answer.
Why do you think you can't have noncopyable?
I understood it prevents the registration of converters for the class. I just read the manual again and now i understand it only remove converter which copy instances, i.e. conversion of values.
Anyhow, I'd recommend trying
class_<Keyboard, noncopyable>("Keyboard", no_init);
with the rest of what you have.
That's what I initially tried. It produced runtime error: TypeError: No Python class registered for C++ class OIS::Keyboard Thanks to you, I know understand some points i couldn't catch reading the manual 3-4 times. I'm now able to fix the current problem by importing my module before assigning the variable 'keyboard'. Which leads me to another question : is there a way to register converters without importing the module ?
On 07/30/2011 03:18 PM, Valentin Perrelle wrote:
Thanks to you, I know understand some points i couldn't catch reading the manual 3-4 times. I'm now able to fix the current problem by importing my module before assigning the variable 'keyboard'. Which leads me to another question : is there a way to register converters without importing the module ?
Hmm. That's a little tricky. You can of course put your "class_" definitions in an arbitrary C++ function, and call that somehow before importing the module, but then you really shouldn't call that function again when importing because that would register some bits of the class twice. I guess I don't really understand how your program flow is supposed to work - how did you plan to invoke C++ code from Python before importing your Boost.Python module? Usually the natural place to register converters is during module import after you've registered the classes. Jim
Le 31/07/2011 07:38, Jim Bosch a écrit :
I guess I don't really understand how your program flow is supposed to work - how did you plan to invoke C++ code from Python before importing your Boost.Python module? Usually the natural place to register converters is during module import after you've registered the classes.
I don't plan to invoke c++ code from Python before importing the module: I need to import the module before exposing some instance of some class of the module. Does this mean that the module should be already imported when the script start running ? Can't i expose the object and then let decide the Python script wheter it need to use it or not ? Valentin Perrelle.
On 07/31/2011 12:52 AM, Valentin Perrelle wrote:
Le 31/07/2011 07:38, Jim Bosch a écrit :
I guess I don't really understand how your program flow is supposed to work - how did you plan to invoke C++ code from Python before importing your Boost.Python module? Usually the natural place to register converters is during module import after you've registered the classes.
I don't plan to invoke c++ code from Python before importing the module: I need to import the module before exposing some instance of some class of the module. Does this mean that the module should be already imported when the script start running ? Can't i expose the object and then let decide the Python script wheter it need to use it or not ?
All of that sounds sounds. But at what point were you trying to register a to-python converter? It sounded like you were trying to do that before importing the module, and since a to-python converter is by definition C++, I didn't understand how you could do it in a Python script before importing the module. Jim
All of that sounds sounds. But at what point were you trying to register a to-python converter? It sounded like you were trying to do that before importing the module, and since a to-python converter is by definition C++, I didn't understand how you could do it in a Python script before importing the module.
I'm registering the converter by calling the boost::python::import function in C++ code. I don't know any other way to do that. Now, this give me another error. I'm trying to implement a "reload script" feature. I thought that all i had to do was to call Py_Finalize, then Py_Initialize again, and to remove any references i was holding to wrapping classes. But whenever i'm importing my extension again, i get the runtime error: Assertion failed: slot->m_to_python == 0, file libs\python\src\converter\registry.cpp, line 212 which means my to_python converter have been registered once again. Is there a way to unregister them ? should i find a to not initialize the extension again ?
You could use the reload() function in python 2.7 or imp.reload() in python 3. It takes a module object as argument. // Simon On 1 aug 2011, at 12:00, Valentin Perrelle <valentin.perrelle@orange.fr> wrote:
All of that sounds sounds. But at what point were you trying to register a to-python converter? It sounded like you were trying to do that before importing the module, and since a to-python converter is by definition C++, I didn't understand how you could do it in a Python script before importing the module.
I'm registering the converter by calling the boost::python::import function in C++ code. I don't know any other way to do that.
Now, this give me another error. I'm trying to implement a "reload script" feature. I thought that all i had to do was to call Py_Finalize, then Py_Initialize again, and to remove any references i was holding to wrapping classes. But whenever i'm importing my extension again, i get the runtime error:
Assertion failed: slot->m_to_python == 0, file libs\python\src\converter\registry.cpp, line 212
which means my to_python converter have been registered once again. Is there a way to unregister them ? should i find a to not initialize the extension again ?
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
You could use the reload() function in python 2.7 or imp.reload() in python 3. It takes a module object as argument. Thanks. However it wouldn't reset to the initial state in the general case. All modules needs to be unloaded. I don't know a safe way to it yet. I'm just sure i want to do it in c++ not in Python.
I've just found that the call of Py_Finalize with Boost.Python is a known issue, already discussed on this mailing list and that the manual says not to call it : http://www.boost.org/doc/libs/1_47_0/libs/python/doc/tutorial/doc/html/pytho...
In My program I need to unload modules as well. What I do is remove all references to the particular module and it will be unloaded. Are you using boost python for python 2 or 3? If it's the latter it is safe to use Py_Finalize()! I use it myself! // Simon On 1 aug 2011, at 12:58, Valentin Perrelle <valentin.perrelle@orange.fr> wrote:
You could use the reload() function in python 2.7 or imp.reload() in python 3. It takes a module object as argument. Thanks. However it wouldn't reset to the initial state in the general case. All modules needs to be unloaded. I don't know a safe way to it yet. I'm just sure i want to do it in c++ not in Python.
I've just found that the call of Py_Finalize with Boost.Python is a known issue, already discussed on this mailing list and that the manual says not to call it : http://www.boost.org/doc/libs/1_47_0/libs/python/doc/tutorial/doc/html/pytho...
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Le 01/08/2011 13:19, Simon Warg a écrit :
In My program I need to unload modules as well. What I do is remove all references to the particular module and it will be unloaded. It seems i didn't achieve to do that. There should be some references i can't remove, i don't know why yet.
Are you using boost python for python 2 or 3? If it's the latter it is safe to use Py_Finalize()! I use it myself!
I'm using Python 3. But the problem of unregistered converters is still there. See http://mail.python.org/pipermail/cplusplus-sig/2009-August/014736.html
You can remove one reference from the sys module: Import sys del sys.modules['mymodule'] In cpp it would be like: import('sys').attr('modules')['mymodule'].del() I can give you my code later. Don't have it here! // Simon On 1 aug 2011, at 13:38, Valentin Perrelle <valentin.perrelle@orange.fr> wrote:
Le 01/08/2011 13:19, Simon Warg a écrit :
In My program I need to unload modules as well. What I do is remove all references to the particular module and it will be unloaded. It seems i didn't achieve to do that. There should be some references i can't remove, i don't know why yet.
Are you using boost python for python 2 or 3? If it's the latter it is safe to use Py_Finalize()! I use it myself!
I'm using Python 3. But the problem of unregistered converters is still there. See http://mail.python.org/pipermail/cplusplus-sig/2009-August/014736.html
_______________________________________________ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
In cpp it would be like: import('sys').attr('modules')['mymodule'].del() Thank you, it worked. However, I believe it would only remove one module, not any other module imported. I tried to clear the dictionnary, which worked in Python, i didn't achieve to reproduce this in c++. Anyway, this would free memory occupied by cycles in the python reference graph. But this is only a matter of memory leak, not really relevent in my context.
I tried another solution, which doesn't seem at first very suitable. I created a subinterpreter for each new execution of the script. Since there is never 2 execution at the same time, this ensure a good independance between distinct executions. It worked well. This may be a good solution, at least until the Py_Finalize issue is solved. Thank you again for your help.
participants (3)
-
Jim Bosch -
Simon Warg -
Valentin Perrelle