Extending Python with not-basic-c++
Hi, I have a working C++ game. To make it easier to extend/control/whatever some kind of scripting support is required. I know Python isn't really a scripting language but it seems like the best choice. I should also note that my knowledge of python is very limited. I heard that embedding python isn't a good way so I'm about to extend python with my functionality. At first I would like to just replace my game loop from C++ to python. It's basically a class with a method that keeps looping until the game is finished. But this class and all other classes inherits from a common 'object' class. And all memory is allocated through a memory manager. Also, the object class is dependent on a couple of other classes. And circular dependencies is not uncommon. It is not possible to loose this coupling. When compiling a class module for python it says undefined symbol 'a class method here'. So I include that class-objectfile when I compile the module. Eventfully it works but I cant do like this. Can someone give me a few directions? I don't really understand how to do this.
David Sveningsson wrote:
Hi, I have a working C++ game. To make it easier to extend/control/whatever some kind of scripting support is required. I know Python isn't really a scripting language but it seems like the best choice. I should also note that my knowledge of python is very limited.
I heard that embedding python isn't a good way so I'm about to extend
Why not ?
python with my functionality. At first I would like to just replace my game loop from C++ to python. It's basically a class with a method that keeps looping until the game is finished.
But this class and all other classes inherits from a common 'object' class. And all memory is allocated through a memory manager. Also, the object class is dependent on a couple of other classes. And circular dependencies is not uncommon. It is not possible to loose this coupling.
OK. So you want to keep control over your C++ object's memory allocation in C++. That probably means that all the object creation, as far as the python bindings are concerned, should happen through a factory, with objects marked 'non-copyable', and return-value-policies generally 'return-existing-object' or somesuch.
When compiling a class module for python it says undefined symbol 'a class method here'. So I include that class-objectfile when I compile the module. Eventfully it works but I cant do like this.
You really ought to provide some more detail here if you expect anybody to be able to help. Some code snippets, together with the actual error message from your compiler. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin...
Stefan Seefeld skrev:
David Sveningsson wrote:
Hi, I have a working C++ game. To make it easier to extend/control/whatever some kind of scripting support is required. I know Python isn't really a scripting language but it seems like the best choice. I should also note that my knowledge of python is very limited.
I heard that embedding python isn't a good way so I'm about to extend
Why not ?
What I heard was that python was not designed for this and thus the python C API isn' the best. Are you suggesting I should embedd instead?
When compiling a class module for python it says undefined symbol 'a class method here'. So I include that class-objectfile when I compile the module. Eventfully it works but I cant do like this.
You really ought to provide some more detail here if you expect anybody to be able to help. Some code snippets, together with the actual error message from your compiler As I said, it works but for each module I have to link against many of my c++ object files. So the same files is linked into many modules. What I mant to know if this is the way it has to be done or if there is a better way. Won't it be problematic if each module has it's own version of everything?
Right now I am trying to put everything in a big module instead of spliting it up.
David Sveningsson wrote:
Stefan Seefeld skrev:
David Sveningsson wrote:
Hi, I have a working C++ game. To make it easier to extend/control/whatever some kind of scripting support is required. I know Python isn't really a scripting language but it seems like the best choice. I should also note that my knowledge of python is very limited.
I heard that embedding python isn't a good way so I'm about to extend
Why not ?
What I heard was that python was not designed for this and thus the python C API isn' the best. Are you suggesting I should embedd instead?
I'm not sure what it originally was designed for, but I can assure you that it works great as an embedded scripting environment. I'm not qualified to tell you the best strategy, as I don't know the context. In any case, it's not 'instead', as in order to script your application 'from within', you need to expose your API to some degree to the script's environment, anyhow.
When compiling a class module for python it says undefined symbol 'a class method here'. So I include that class-objectfile when I compile the module. Eventfully it works but I cant do like this.
You really ought to provide some more detail here if you expect anybody to be able to help. Some code snippets, together with the actual error message from your compiler As I said, it works but for each module I have to link against many of my c++ object files. So the same files is linked into many modules. What I mant to know if this is the way it has to be done or if there is a better way. Won't it be problematic if each module has it's own version of everything?
Don't you use shared libraries ? Sharing doesn't imply duplication. HTH, Stefan -- ...ich hab' noch einen Koffer in Berlin...
Stefan Seefeld skrev:
As I said, it works but for each module I have to link against many of my c++ object files. So the same files is linked into many modules. What I mant to know if this is the way it has to be done or if there is a better way. Won't it be problematic if each module has it's own version of everything?
Don't you use shared libraries ? Sharing doesn't imply duplication. Yes, I compile the libraries like this (straight from the Makefile):
GCC=g++ OPTIONS=-c -g -Wall -march=pentium-m -msse3 -pipe FLAGS=-I../../Source/Include -DLINUX CFLAGS=$(OPTIONS) $(FLAGS) LDFLAGS=-g -lX11 -lGL -lGLU OUTPUT=../../Game/Core.so #Skipped the objects here... PyModules: CFLAGS+=-I/usr/include/python2.4 PyModules: LDFLAGS+=-shared -lboost_python -lpython2.4 all: $(OBJS) PyModules %.o : %.cpp Makefile $(GCC) $(CFLAGS) $< -o PyModules: $(PY_OBJS) $(GCC) $(OBJS) $(LDFLAGS) -o $(OUTPUT) So each source file is compiled like this: g++ -c -g -Wall -march=pentium-m -msse3 -pipe -I../../Source/Include -DLINUX -I/usr/include/python2.4 ../../Source/Implementation/PythonModules/CoreModule.cpp -o CoreModule.o And then linked like this: g++ CoreModule.o -g -lX11 -lGL -lGLU -shared -lboost_python -lpython2.4 -shared -lboost_python -lpython2.4 -o ../../Game/Core.so But when I try to import the module into Python (import Core) i get the following error: Traceback (most recent call last): File "./foo.py", line 3, in ? import Core ImportError: /home/ext/workspace/06proj/Game/Core.so: undefined symbol: _ZN15AutoreleasePool9constructEv So I have to link with AutoreleasePool.o After I have linked with all objects that is required and creates another module and imports it AFTER the core module I still get the same error for the new module. The only way to get it working is to link with those modules again but that creates to separate modules. For instance, I have an event dispatcher in the core module. Another module I try to create also needs the event dispatcher but since the is linked separately into both modules they cannot communicate. I don't want to have everything in a big module, I want to split things up. I can live with a core module that have all code with circular dependencies etc but I don't want everything in it. Doesn't it compile the shared libraries as it should? Since they don't seem to behave like one. I only have experience with creating shared libraries in c/c++ for use with c/c++ but as I understood it was basically the same? -- //*David Sveningsson [eXt]* Freelance coder | Game Development Student http://sidvind.com Thou shalt make thy program's purpose and structure clear to thy fellow man by using the One True Brace Style, even if thou likest it not, for thy creativity is better used in solving problems than in creating beautiful new impediments to understanding.
David Sveningsson wrote:
Stefan Seefeld skrev:
As I said, it works but for each module I have to link against many of my c++ object files. So the same files is linked into many modules. What I mant to know if this is the way it has to be done or if there is a better way. Won't it be problematic if each module has it's own version of everything? Don't you use shared libraries ? Sharing doesn't imply duplication. Yes, I compile the libraries like this (straight from the Makefile):
[...]
And then linked like this: g++ CoreModule.o -g -lX11 -lGL -lGLU -shared -lboost_python -lpython2.4 -shared -lboost_python -lpython2.4 -o ../../Game/Core.so
But when I try to import the module into Python (import Core) i get the following error:
Traceback (most recent call last): File "./foo.py", line 3, in ? import Core ImportError: /home/ext/workspace/06proj/Game/Core.so: undefined symbol: _ZN15AutoreleasePool9constructEv
So I have to link with AutoreleasePool.o
No, you have to link with the (hopefully shared) library that contains AutoreleasePool.o. That will only make your 'Core.so' python module aware of the library, so the dynamic loader (presumably ld.so) can load it when you import the 'Core' module. But this has nothing to do with boost.python, really. HTH, Stefan -- ...ich hab' noch einen Koffer in Berlin...
Stefan Seefeld skrev:
No, you have to link with the (hopefully shared) library that contains AutoreleasePool.o. That will only make your 'Core.so' python module aware of the library, so the dynamic loader (presumably ld.so) can load it when you import the 'Core' module.
Ohh, I understand. That would explain it all. I didn't link the python module against a shared library, only as shared. Doing that instead helped.
But this has nothing to do with boost.python, really.
I think it does since I use boost.python to create the modules. Well, it would be the same even if I used the python C API directly but the problem is the same with boost.python. Maybe this should be mentioned in the boost documentation? Anyway, thanks for the help! I really appreciate it! -- //*David Sveningsson [eXt]* Freelance coder | Game Development Student http://sidvind.com Thou shalt make thy program's purpose and structure clear to thy fellow man by using the One True Brace Style, even if thou likest it not, for thy creativity is better used in solving problems than in creating beautiful new impediments to understanding.
David Sveningsson skrev:
Stefan Seefeld skrev:
No, you have to link with the (hopefully shared) library that contains AutoreleasePool.o. That will only make your 'Core.so' python module aware of the library, so the dynamic loader (presumably ld.so) can load it when you import the 'Core' module.
Ohh, I understand. That would explain it all. I didn't link the python module against a shared library, only as shared. Doing that instead helped.
But this has nothing to do with boost.python, really.
I think it does since I use boost.python to create the modules. Well, it would be the same even if I used the python C API directly but the problem is the same with boost.python. Maybe this should be mentioned in the boost documentation?
Anyway, thanks for the help! I really appreciate it!
I had another problem which isn't really related to python or boost but I think you might be able to answer it anyway. My python module tries to load my shared library but it can't find it unless it is in a directory like /usr/lib. I don't want to put my libraries there but rather in the same directory as the module and the python code. I think LD_LIBRARY_PATH could be set but I don't want to set that each time (or use a script either). Is there some way to fix this? -- //*David Sveningsson [eXt]* Freelance coder | Game Development Student http://sidvind.com Thou shalt make thy program's purpose and structure clear to thy fellow man by using the One True Brace Style, even if thou likest it not, for thy creativity is better used in solving problems than in creating beautiful new impediments to understanding.
David Sveningsson wrote:
I had another problem which isn't really related to python or boost but I think you might be able to answer it anyway. My python module tries to load my shared library but it can't find it unless it is in a directory like /usr/lib. I don't want to put my libraries there but rather in the same directory as the module and the python code. I think LD_LIBRARY_PATH could be set but I don't want to set that each time (or use a script either). Is there some way to fix this?
You might configure the dynamic loader ('ld.so' ?) to look there by default. For example, I have an /etc/ld.so.conf config file that I add default search paths such as /usr/local/lib. That said, I don't think it's such a great idea to store ordinary DSOs together in the same directory as python extension modules. HTH, Stefan -- ...ich hab' noch einen Koffer in Berlin...
participants (3)
-
David Sveningsson -
David Sveningsson
-
Stefan Seefeld