Has anyone every used or even tried to use Boost.Python to dynamically create python types at runtime (instead of the standard compile time method)? I am working on an very large extensible C++ class library that has a reflective interface that I can extend as much as I need. So instead of trying to create an enormous wrapper at compile time (that would be quickly out of date) I have been investigating the possibility of dynamically creating the python class/type wrappers on demand. So for example if I discovered at run-time that I wanted a wrapper for a class Hello I would: - Ask Hello's reflective interface for information about the class and it's bases - Create an associated boost.python class object - For each method Hello reports - add method to the wrapper class using signature and method info from Hello I would also need to handle registering associated smart pointers, etc. The details are definitely complex, but before I get to far into it I was wondering if anyone had done anything similar or had tried and could tell me how it went or if it is impossible with the current system. Thanks, Allen
Allen Bierbaum <allenb@vrsource.org> writes:
Has anyone every used or even tried to use Boost.Python to dynamically create python types at runtime (instead of the standard compile time method)?
I am working on an very large extensible C++ class library that has a reflective interface that I can extend as much as I need. So instead of trying to create an enormous wrapper at compile time (that would be quickly out of date) I have been investigating the possibility of dynamically creating the python class/type wrappers on demand.
So for example if I discovered at run-time that I wanted a wrapper for a class Hello I would:
- Ask Hello's reflective interface for information about the class and it's bases - Create an associated boost.python class object - For each method Hello reports - add method to the wrapper class using signature and method info from Hello
I would also need to handle registering associated smart pointers, etc. The details are definitely complex, but before I get to far into it I was wondering if anyone had done anything similar or had tried and could tell me how it went or if it is impossible with the current system.
Boost.Python is not exactly designed to do that, and what you're asking isn't possible to do in portable C++ without knowing all of the classes and member function signatures that you might want to wrap at compile-time anyway. However, see http://aspn.activestate.com/ASPN/Mail/Message/cpp-sig/2781474 You can of course generate Boost.Python wrapping code dynamically and compile it into extension modules that are then loaded into your running Python program :) -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
Allen Bierbaum <allenb@vrsource.org> writes:
Has anyone every used or even tried to use Boost.Python to dynamically create python types at runtime (instead of the standard compile time method)?
I am working on an very large extensible C++ class library that has a reflective interface that I can extend as much as I need. So instead of trying to create an enormous wrapper at compile time (that would be quickly out of date) I have been investigating the possibility of dynamically creating the python class/type wrappers on demand.
So for example if I discovered at run-time that I wanted a wrapper for a class Hello I would:
- Ask Hello's reflective interface for information about the class and it's bases - Create an associated boost.python class object - For each method Hello reports - add method to the wrapper class using signature and method info from Hello
I would also need to handle registering associated smart pointers, etc. The details are definitely complex, but before I get to far into it I was wondering if anyone had done anything similar or had tried and could tell me how it went or if it is impossible with the current system.
Boost.Python is not exactly designed to do that,
That is what I have come to suspect after looking into the depths of the code. It seems that behind the scenes everything to setup the class and methods ends up as standard run-time calls but the specific methods called and parameters passed are all based on compile time logic. So it does look like this would be difficult and most likely impossible without extensions to the code base to support it. Is this an area that anyone else has ever asked about or have you ever toyed with the idea of having run-time capabilities?
and what you're asking isn't possible to do in portable C++ without knowing all of the classes and member function signatures that you might want to wrap at compile-time anyway.
I could be wrong, but I think I will have all this anyway (or be able to get it) through a reflective interface in the library. Basically each run-time class can return a type descriptor object that can return as much information as necessary. I don't pretend to know C++ as well as you though, so maybe I am missing something. Is there some information or knowledge that would be needed that could not be stored and then looked up later at run-time when registering a wrapper?
However, see http://aspn.activestate.com/ASPN/Mail/Message/cpp-sig/2781474
I may be missing something here. I see how this is related, but I think what I want to do is quite different. I effectively want to do exactly what I could do at compile time with the current code but instead delay the creation so I do it on demand (to save compile time, library size and make the code work with new C++ object types that are introduced and dynamic extensions to the library).
You can of course generate Boost.Python wrapping code dynamically and compile it into extension modules that are then loaded into your running Python program :)
That would be fun but I think my users would wonder a little why the CPU and disk are churning so much at startup. :) Thanks, Allen
Allen Bierbaum <allenb@vrsource.org> writes:
David Abrahams wrote:
Allen Bierbaum <allenb@vrsource.org> writes:
Is this an area that anyone else has ever asked about or have you ever toyed with the idea of having run-time capabilities?
We have about as many run-time capabilities as are possible in Boost.Python.
and what you're asking isn't possible to do in portable C++ without knowing all of the classes and member function signatures that you might want to wrap at compile-time anyway. ^^^^^^^^^^^^^^^
I could be wrong, but I think I will have all this anyway (or be able to get it) through a reflective interface in the library. Basically each run-time class
Too late. The information is needed at compile-time, as I said.
can return a type descriptor object that can return as much information as necessary. I don't pretend to know C++ as well as you though, so maybe I am missing something. Is there some information or knowledge that would be needed that could not be stored and then looked up later at run-time when registering a wrapper?
It doesn't matter if it can be looked up at run-time, because it needs to be available at compile-time.
However, see http://aspn.activestate.com/ASPN/Mail/Message/cpp-sig/2781474
I may be missing something here. I see how this is related, but I think what I want to do is quite different.
Yes, what you want to do includes generating function call wrappers.
I effectively want to do exactly what I could do at compile time with the current code but instead delay the creation so I do it on demand
I understand that.
You can of course generate Boost.Python wrapping code dynamically and compile it into extension modules that are then loaded into your running Python program :)
That would be fun but I think my users would wonder a little why the CPU and disk are churning so much at startup. :)
You would cache the results, so you only have to pay once. There's a successful system that does it that way: it allows you to embed C++ code directly in your Python and have it compiled on the fly, so I know some people can tolerate it. Of course, if you're going to generate code anyway I'd probably generate traditional Python/C API code rather than Boost.Python stuff, since it will compile faster. -- Dave Abrahams Boost Consulting www.boost-consulting.com
participants (2)
-
Allen Bierbaum -
David Abrahams