compile errors with embedding example from tutorial
Dear People, When trying to compile the following code (direct from sample code in the turorial) I get these errors. I am using distutils and realise my flags may not be exactly right, but the compiler seems to be saying it cannot find the correct version of handler to use! This does not look like it has anything to do with the correct flags, though I could be wrong. I'm trying to call the interpreter from within a extension module. Is there any reason I can't do this? Thanks in advance. Faheem. building 'embed' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.3 -c embed.cc -o /tmp/temp.linux-i686-2.3/embed.o -Wl,-E embed.cc: In function `void foo()': embed.cc:35: error: no matching function for call to ` boost::python::handle<PyObject>::handle(const char[73], int, PyObject*, PyObject*)' /usr/include/boost/python/handle.hpp:129: error: candidates are: boost::python::handle<T>::handle(boost::python ::detail::borrowed_reference_t*) [with T = PyObject] /usr/include/boost/python/handle.hpp:110: error: boost::python::handle<T>::handle(const boost::python::handle<T>&) [with T = PyObject] /usr/include/boost/python/handle.hpp:205: error: boost::python::handle<T>::handle() [with T = PyObject] error: command 'gcc' failed with exit status 1 make: *** [embed.so] Error 1 make: Target `all' not remade because of errors. Compilation exited abnormally with code 2 at Mon Jun 28 00:02:00 ************************************************************************* embed.cc ************************************************************************* #include <boost/python.hpp> #include <boost/scoped_ptr.hpp> #include <python2.3/Python.h> #include <pythonrun.h> using namespace boost::python; void foo() { Py_Initialize(); handle<> main_module(borrowed( PyImport_AddModule("__main__") )); handle<> main_namespace(borrowed( PyModule_GetDict(main_module.get()) )); handle<>( PyRun_String("hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.get(), main_namespace.get()) ); Py_Finalize(); } BOOST_PYTHON_MODULE(embed) { def("foo",foo); } *************************************************************************
Hi Faheem, I ran into that too. In short, the line with "handle<>( PyRun_String" isn't being understood how you want it too... You can surround that expression with parenthesis, and it should work: (handle<>( PyRun_String("hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.get(), main_namespace.get()) )); Chad On Mon, 28 Jun 2004 04:26:19 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
Dear People,
When trying to compile the following code (direct from sample code in the turorial) I get these errors. I am using distutils and realise my flags may not be exactly right, but the compiler seems to be saying it cannot find the correct version of handler to use! This does not look like it has anything to do with the correct flags, though I could be wrong.
I'm trying to call the interpreter from within a extension module. Is there any reason I can't do this? Thanks in advance.
Faheem.
building 'embed' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.3 -c embed.cc -o /tmp/temp.linux-i686-2.3/embed.o -Wl,-E embed.cc: In function `void foo()': embed.cc:35: error: no matching function for call to ` boost::python::handle<PyObject>::handle(const char[73], int, PyObject*, PyObject*)' /usr/include/boost/python/handle.hpp:129: error: candidates are: boost::python::handle<T>::handle(boost::python ::detail::borrowed_reference_t*) [with T = PyObject] /usr/include/boost/python/handle.hpp:110: error: boost::python::handle<T>::handle(const boost::python::handle<T>&) [with T = PyObject] /usr/include/boost/python/handle.hpp:205: error: boost::python::handle<T>::handle() [with T = PyObject] error: command 'gcc' failed with exit status 1 make: *** [embed.so] Error 1 make: Target `all' not remade because of errors.
Compilation exited abnormally with code 2 at Mon Jun 28 00:02:00
************************************************************************* embed.cc ************************************************************************* #include <boost/python.hpp> #include <boost/scoped_ptr.hpp> #include <python2.3/Python.h> #include <pythonrun.h> using namespace boost::python;
void foo() { Py_Initialize();
handle<> main_module(borrowed( PyImport_AddModule("__main__") ));
handle<> main_namespace(borrowed( PyModule_GetDict(main_module.get()) ));
handle<>( PyRun_String("hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.get(), main_namespace.get()) );
Py_Finalize(); }
BOOST_PYTHON_MODULE(embed) { def("foo",foo); } *************************************************************************
_______________________________________________ C++-sig mailing list C++-sig@python.org http://mail.python.org/mailman/listinfo/c++-sig
On Sun, 27 Jun 2004 23:46:06 -0500, Chad Austin <caustin@gmail.com> wrote:
Hi Faheem,
Hi Chad, Thanks for replying.
I ran into that too. In short, the line with "handle<>( PyRun_String" isn't being understood how you want it too... You can surround that expression with parenthesis, and it should work:
(handle<>( PyRun_String("hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.get(), main_namespace.get()) ));
Thanks. Indeed it does. I hope the tutorial can be changed appropriately, unless it already has been, of course. Any idea why this is necessary? It isn't in the other two places <handle> is used. I can load my module, but when I try to run the function, I get NameError: name 'file' is not defined Any idea about this? If you know of other places the tutorial needs to be changed, I would appreciate being informed of them. I think it might be useful if I said explicitly what it is I am trying to do here, and ask for suggestions. I'm writing an extension module, and want to call a python function, namely array from the numarray.strings module. This is part of the Python Numarray library. The question is how to do this. I have tried doing this with callbacks and it seems to work fine. The trouble with callbacks is that it means I have to have a python wrapper with an argument slot for every function I want to import. This is clunky. Suppose I have three functions, say f1, f2, f3. Then I have f1 calling f2 calling f3. f1 -> f2 -> f3 Suppose f3 calls some python function (say `foo') from an external module or something. Then if I rewrite f3 using Boost.Python, I still need a wrapper, say f1', to pass `foo' as an argument. Then I need f1 -> f2 -> f3' -> f3 I suppose I don't need `foo' passed down from f1 because if I import foo into global namespace into the same module where I define f1, f2, f3', then Python's scoping rules will ensure that if I use `foo' as an argument to f3' then it will be picked up from the global namespace. Furthermore, if I decide to rewrite f2 into Boost.Python, then I would have to create another Python wrapper f2' for f2, and pass f3' as an argument to f2' (ugh). So I'd get the following f1 -> f2' -> f2 -> f3' -> f3 I'd rather have f1 -> f2 -> f3 where f2 and f3 only exist as C++ functions. Much cleaner. So I thought that perhaps I could imbed the interpreter inside (say) f3, and persuade it to make `foo' available to me inside C++. I don't know if this is even possible. I have seen no examples of this. In fact, I have seen no examples of using the python interpeter inside extension modules. In fact, I've been rather unsuccessful so far in getting anything related to embedding to work. Am I missing something? Thanks in advance for any information. Faheem.
On Mon, 28 Jun 2004 17:48:59 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
On Sun, 27 Jun 2004 23:46:06 -0500, Chad Austin <caustin@gmail.com> wrote:
Hi Faheem,
Hi Chad,
Thanks for replying.
I ran into that too. In short, the line with "handle<>( PyRun_String" isn't being understood how you want it too... You can surround that expression with parenthesis, and it should work:
(handle<>( PyRun_String("hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.get(), main_namespace.get()) ));
Thanks. Indeed it does. I hope the tutorial can be changed appropriately, unless it already has been, of course. Any idea why this is necessary? It isn't in the other two places <handle> is used.
I think it's something to do with C++ thinking you're trying to declare a function or function pointer or something. That's the best I could come up with... Any insight from others would be appreciated. :)
I can load my module, but when I try to run the function, I get
NameError: name 'file' is not defined
Any idea about this?
Nope, sorry... I had a problem similar to that in that __builtins__ wasn't getting set in my modules, but I was doing some weird stuff. You may want to at least check that __builtins__ is set in your script.
If you know of other places the tutorial needs to be changed, I would appreciate being informed of them.
I think it might be useful if I said explicitly what it is I am trying to do here, and ask for suggestions.
I'm writing an extension module, and want to call a python function, namely array from the numarray.strings module. This is part of the Python Numarray library. The question is how to do this.
Something like: object m = object(handle<>( PyImport_Import("numarray.strings") )); object result = m.attr("array")(arguments); should work for you.
I have tried doing this with callbacks and it seems to work fine. The trouble with callbacks is that it means I have to have a python wrapper with an argument slot for every function I want to import. This is clunky.
Suppose I have three functions, say f1, f2, f3.
Then I have f1 calling f2 calling f3.
f1 -> f2 -> f3
Suppose f3 calls some python function (say `foo') from an external module or something.
Then if I rewrite f3 using Boost.Python, I still need a wrapper, say f1', to pass `foo' as an argument. Then I need
f1 -> f2 -> f3' -> f3
I suppose I don't need `foo' passed down from f1 because if I import foo into global namespace into the same module where I define f1, f2, f3', then Python's scoping rules will ensure that if I use `foo' as an argument to f3' then it will be picked up from the global namespace.
Furthermore, if I decide to rewrite f2 into Boost.Python, then I would have to create another Python wrapper f2' for f2, and pass f3' as an argument to f2' (ugh). So I'd get the following
f1 -> f2' -> f2 -> f3' -> f3
I'd rather have
f1 -> f2 -> f3
where f2 and f3 only exist as C++ functions. Much cleaner.
So I thought that perhaps I could imbed the interpreter inside (say) f3, and persuade it to make `foo' available to me inside C++. I don't know if this is even possible. I have seen no examples of this. In fact, I have seen no examples of using the python interpeter inside extension modules. In fact, I've been rather unsuccessful so far in getting anything related to embedding to work.
I'm totally confused by everything you just said about callbacks, but you should be able to easily call back into Python from C++ using Boost.Python's object (and related subclasses) or even the native Python/C API. I don't see a reason importing another native module from a native module would cause a problem, as long as you write your C++ code in the same way you'd write the Python.
Am I missing something? Thanks in advance for any information.
Hope that helps, -- Chad Austin http://aegisknight.org/
On Mon, 28 Jun 2004 15:56:13 -0500, Chad Austin <caustin@gmail.com> wrote:
On Mon, 28 Jun 2004 17:48:59 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
On Sun, 27 Jun 2004 23:46:06 -0500, Chad Austin <caustin@gmail.com> wrote:
Nope, sorry... I had a problem similar to that in that __builtins__ wasn't getting set in my modules, but I was doing some weird stuff. You may want to at least check that __builtins__ is set in your script.
How should I do that? I thought that handle<> main_module(borrowed( PyImport_AddModule("__main__") )); handle<> main_namespace(borrowed( PyModule_GetDict(main_module.get()))); should do the trick, though I don't really understand what these lines do. I wonder if the fact that I am calling the interpreter inside an extension module is causing any problems.
If you know of other places the tutorial needs to be changed, I would appreciate being informed of them.
I think it might be useful if I said explicitly what it is I am trying to do here, and ask for suggestions.
I'm writing an extension module, and want to call a python function, namely array from the numarray.strings module. This is part of the Python Numarray library. The question is how to do this.
Something like:
object m = object(handle<>( PyImport_Import("numarray.strings") )); object result = m.attr("array")(arguments);
should work for you.
Ok. Thanks.
I have tried doing this with callbacks and it seems to work fine. The trouble with callbacks is that it means I have to have a python wrapper with an argument slot for every function I want to import. This is clunky.
[stuff deleted...]
I'm totally confused by everything you just said about callbacks, but you should be able to easily call back into Python from C++ using Boost.Python's object (and related subclasses) or even the native Python/C API. I don't see a reason importing another native module from a native module would cause a problem, as long as you write your C++ code in the same way you'd write the Python.
Sorry I was unclear. I'd be willing to try to clarify if that would be useful. I can certainly make callbacks work. The point is I have to pass the relevant functions from Python, which imposes some overhead, which I'd like to get rid of. Faheem.
On Mon, 28 Jun 2004 22:39:15 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
On Mon, 28 Jun 2004 15:56:13 -0500, Chad Austin <caustin@gmail.com> wrote:
On Mon, 28 Jun 2004 17:48:59 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
On Sun, 27 Jun 2004 23:46:06 -0500, Chad Austin <caustin@gmail.com> wrote:
Nope, sorry... I had a problem similar to that in that __builtins__ wasn't getting set in my modules, but I was doing some weird stuff. You may want to at least check that __builtins__ is set in your script.
How should I do that? I thought that
handle<> main_module(borrowed( PyImport_AddModule("__main__") )); handle<> main_namespace(borrowed( PyModule_GetDict(main_module.get())));
should do the trick, though I don't really understand what these lines do. I wonder if the fact that I am calling the interpreter inside an extension module is causing any problems.
Just try evaluating the script "import __builtin__" or "str(10)" and see if it fails or not. ;) There should be no problem with calling interpreter from an extension module. Language bindings are a two-way street, so there necessarily must be communication between the two languages. -- Chad Austin http://aegisknight.org/
Faheem Mitha <faheem@email.unc.edu> writes:
On Mon, 28 Jun 2004 15:56:13 -0500, Chad Austin <caustin@gmail.com> wrote:
On Mon, 28 Jun 2004 17:48:59 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
On Sun, 27 Jun 2004 23:46:06 -0500, Chad Austin <caustin@gmail.com> wrote:
Nope, sorry... I had a problem similar to that in that __builtins__ wasn't getting set in my modules, but I was doing some weird stuff. You may want to at least check that __builtins__ is set in your script.
How should I do that? I thought that
handle<> main_module(borrowed( PyImport_AddModule("__main__") )); handle<> main_namespace(borrowed( PyModule_GetDict(main_module.get())));
should do the trick, though I don't really understand what these lines do.
What specifically are you confused about? Did you check the docs? I don't know why you'd do it that way when you can write: object main_module(handle<>(borrowed( PyImport_AddModule("__main__")))); object main_namespace = main_module.attr("__dict__"); instead.
I wonder if the fact that I am calling the interpreter inside an extension module is causing any problems.
Certainly not. Calling the interpreter inside an extension module is normal. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
On Tue, 29 Jun 2004 11:51:48 -0400, David Abrahams <dave@boost-consulting.com> wrote:
Faheem Mitha <faheem@email.unc.edu> writes:
On Mon, 28 Jun 2004 15:56:13 -0500, Chad Austin <caustin@gmail.com> wrote:
On Mon, 28 Jun 2004 17:48:59 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
On Sun, 27 Jun 2004 23:46:06 -0500, Chad Austin <caustin@gmail.com> wrote:
Nope, sorry... I had a problem similar to that in that __builtins__ wasn't getting set in my modules, but I was doing some weird stuff. You may want to at least check that __builtins__ is set in your script.
How should I do that? I thought that
handle<> main_module(borrowed( PyImport_AddModule("__main__") )); handle<> main_namespace(borrowed( PyModule_GetDict(main_module.get())));
should do the trick, though I don't really understand what these lines do.
What specifically are you confused about? Did you check the docs?
Yes. Lots. I've got multiple windows open in my (tabbed) browser) with various bits of the tutorial, the reference manual, and various googled past messages from the mailing list, and I've been looking at these all weekend. There are various things that are fuzzy, but for the moment I'd just like to concentrate on getting an example compiled and working.
I don't know why you'd do it that way when you can write:
Well, this is direct from the tutorial.
object main_module(handle<>(borrowed( PyImport_AddModule("__main__")))); object main_namespace = main_module.attr("__dict__");
instead.
This does not compile with gcc 3.3.4 on Debian. I get the following errors: embed.cc:29: error: variable declaration is not allowed here embed.cc:30: error: request for member `attr' in `main_module', which is of non-aggregate type `boost::python::api::object ()()' error: command 'gcc' failed with exit status 1 with the code ************************************************************************* #include <boost/python.hpp> using namespace boost::python; void foo() { Py_Initialize(); object main_module(handle<>(borrowed (PyImport_AddModule("__main__")))); object main_namespace = main_module.attr("__dict__"); (handle<>( PyRun_String("hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.ptr(),main_namespace.ptr()))); Py_Finalize(); } BOOST_PYTHON_MODULE(embed) { def("foo",foo); } ************************************************************************* I thought that perhaps I had the wrong compiler flags, so I tried it with bjam and the Boost Build system, and I get the same errors. However, if I replace object main_module(handle<>(borrowed (PyImport_AddModule("__main__")))); by object main_module = object(handle<>(borrowed ( PyImport_AddModule("__main__")))); as suggested in a recent post of yours, it compiles. If this is a gcc bug, as you say, has it been reported? If you can briefly explain to me what the bug is, I'd be happy to send in a report, unless it has already been fixed in gcc 3.4. (Can anyone check this?) However, like the earlier code I was using, it fails to run. Ie. I get the following errors when I try to load the code: *************************************************************************** In [2]: embed.foo() --------------------------------------------------------------------------- NameError Traceback (most recent call last) /home/faheem/wc/corrmodel/boost/<console> /home/faheem/wc/corrmodel/boost/<string> NameError: name 'file' is not defined ***************************************************************************** It cannot find the built-in `file'. Any suggestions about what is wrong here, and how to fix it? I am keen to try and get at least some basic examples of embedding working. Can anyone reproduce this problem on Linux with gcc? Unfortunately, I have other problems with the version I compiled using Boost Build, so I can't currently test that. (Subject for another message.) I think that once I have a few examples working on Debian, I will contribute basic examples along with build instructions to the Debian package, if they will take it. It may be useful for other people.
I wonder if the fact that I am calling the interpreter inside an extension module is causing any problems.
Certainly not. Calling the interpreter inside an extension module is normal.
Good to know. Well, if that is not the problem, I am not sure what it could be. Please help. Faheem.
I've had similar problems, and have slogged my way to figuring them out. The tutorial is wrong, at least on gcc 3.2.3. Here's what you want: After calling Py_Initialize(), you should get the Python main namespace object by doing: handle<> main_module(borrowed( PyImport_AddModule("__main__") )); dict main_namespace = extract<dict>(PyModule_GetDict(main_module.get())); The first line gets a pointer to the '__main__' module object and stores it as a borrowed reference in a handle. The second line gets the dictionary of the '__main__' module and puts it in a dict C++ object. You should now use this dictionary as the globals and locals of PyRun_* functions that you call. For example: handle<> unused(PyRun_String("import yourModuleNameHere", Py_file_input, main_namespace.ptr(), main_namespace.ptr())); handle<> unused2(PyRun_String("f = file('test_output', 'w')", Py_file_input, main_namespace.ptr(), main_namespace.ptr())); You need to name the handle<> objects, even though you don't use them. If not, while you may think you're creating an anonymous temporary that will immediately get destructed, the compiler thinks you're declaring a function and gets very confused. Hope this helps. As usual, I invoke the great spirit of Dave Abrahams to correct/clarify anything I've said that's not quite right or misleading. Good luck -- -- Graeme graeme.lufkin@gmail.com
Graeme Lufkin <graeme.lufkin@gmail.com> writes:
I've had similar problems, and have slogged my way to figuring them out. The tutorial is wrong, at least on gcc 3.2.3.
I think the tutorial is right in CVS. Have you checked that? http://www.boost-consulting.com/boost/libs/python/
Here's what you want: After calling Py_Initialize(), you should get the Python main namespace object by doing:
handle<> main_module(borrowed( PyImport_AddModule("__main__") )); dict main_namespace = extract<dict>(PyModule_GetDict(main_module.get()));
Seems harder than neccessary. object main_module = object(handle<> (borrowed( PyImport_AddModule("__main__") ))); object main_namespace = main_module.attr("__dict__"); -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
On Thu, 01 Jul 2004 12:50:45 -0400, David Abrahams <dave@boost-consulting.com> wrote:
Graeme Lufkin <graeme.lufkin@gmail.com> writes:
I've had similar problems, and have slogged my way to figuring them out. The tutorial is wrong, at least on gcc 3.2.3.
I think the tutorial is right in CVS. Have you checked that?
http://www.boost-consulting.com/boost/libs/python/
Here's what you want: After calling Py_Initialize(), you should get the Python main namespace object by doing:
handle<> main_module(borrowed( PyImport_AddModule("__main__") )); dict main_namespace = extract<dict>(PyModule_GetDict(main_module.get()));
Seems harder than neccessary.
object main_module = object(handle<> (borrowed( PyImport_AddModule("__main__") )));
object main_namespace = main_module.attr("__dict__");
http://www.boost-consulting.com/boost/libs/python/doc/tutorial/doc /using_the_interpreter.html uses object main_module( handle<>(borrowed(PyImport_AddModule("__main__")))); instead, which does not compile with gcc 3.3, at least for me. In any case, as I already said, even when this compiles correctly, it still does not seem to execute the interpreter correctly, ie. I get the error NameError: name 'file' is not defined Can you suggest what might be the problem here? Faheem.
Faheem Mitha <faheem@email.unc.edu> writes:
http://www.boost-consulting.com/boost/libs/python/doc/tutorial/doc /using_the_interpreter.html
uses
object main_module( handle<>(borrowed(PyImport_AddModule("__main__"))));
instead, which does not compile with gcc 3.3, at least for me.
GCC bug, as I've already said.
In any case, as I already said, even when this compiles correctly, it still does not seem to execute the interpreter correctly, ie. I get the error
NameError: name 'file' is not defined
Can you suggest what might be the problem here?
Nope, sorry. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
On Thu, 1 Jul 2004 09:05:32 -0700, Graeme Lufkin <graeme.lufkin@gmail.com> wrote:
I've had similar problems, and have slogged my way to figuring them out. The tutorial is wrong, at least on gcc 3.2.3. Here's what you want: After calling Py_Initialize(), you should get the Python main namespace object by doing:
handle<> main_module(borrowed( PyImport_AddModule("__main__") )); dict main_namespace = extract<dict>(PyModule_GetDict(main_module.get()));
I think this should be main_module.ptr() since main_module is an object.
The first line gets a pointer to the '__main__' module object and stores it as a borrowed reference in a handle. The second line gets the dictionary of the '__main__' module and puts it in a dict C++ object. You should now use this dictionary as the globals and locals of PyRun_* functions that you call. For example: handle<> unused(PyRun_String("import yourModuleNameHere", Py_file_input, main_namespace.ptr(), main_namespace.ptr())); handle<> unused2(PyRun_String("f = file('test_output', 'w')", Py_file_input, main_namespace.ptr(), main_namespace.ptr()));
You need to name the handle<> objects, even though you don't use them. If not, while you may think you're creating an anonymous temporary that will immediately get destructed, the compiler thinks you're declaring a function and gets very confused.
Hope this helps. As usual, I invoke the great spirit of Dave Abrahams to correct/clarify anything I've said that's not quite right or misleading. Good luck
I'm trying with ****************************************************************************** #include <boost/python.hpp> using namespace boost::python; void foo() { Py_Initialize(); object main_module = object(handle<>(borrowed(PyImport_AddModule("__main__")))); dict main_namespace = extract<dict>(PyModule_GetDict(main_module.ptr())); handle<> unused(PyRun_String("hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.ptr(),main_namespace.ptr())); Py_Finalize(); } BOOST_PYTHON_MODULE(embed) { def("foo",foo); } ******************************************************************************* This compiles, but I get the same error as before, namely NameError: name 'file' is not defined Does this work correctly for you? I'm wondering if there is some issue with my compiler flags. I'm trying to get Boost.Build to work but there are some linking problems. Faheem.
On Thu, 1 Jul 2004 20:19:37 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
I've had similar problems, and have slogged my way to figuring them out. The tutorial is wrong, at least on gcc 3.2.3. Here's what you want: After calling Py_Initialize(), you should get the Python main namespace object by doing:
handle<> main_module(borrowed( PyImport_AddModule("__main__") )); dict main_namespace = extract<dict>(PyModule_GetDict(main_module.get()));
I think this should be main_module.ptr() since main_module is an object.
In my example it's a handle<>, so get() is correct. In Dave's (and your) examples, it's an object, so ptr() is correct.
I'm trying with
****************************************************************************** #include <boost/python.hpp> using namespace boost::python;
void foo() { Py_Initialize();
object main_module = object(handle<>(borrowed(PyImport_AddModule("__main__"))));
dict main_namespace = extract<dict>(PyModule_GetDict(main_module.ptr()));
handle<> unused(PyRun_String("hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.ptr(),main_namespace.ptr())); Py_Finalize(); }
BOOST_PYTHON_MODULE(embed) { def("foo",foo); } *******************************************************************************
This compiles, but I get the same error as before, namely
NameError: name 'file' is not defined
Does this work correctly for you? I'm wondering if there is some issue with my compiler flags. I'm trying to get Boost.Build to work but there are some linking problems.
First, your extension module should never call Py_Initialize() or Py_Finalize(), as Python will already be initialized by the time you're running methods in it. I think what you want is foo() to be main(), with no extension module stuff needed. Also, Boost.Python and Py_Finalize() don't play nice, so never call it (check the list archive for details, I don't really understand why myself). Second, yes this compiles and runs correctly for me, after removing the Py_Initialize and Py_Finalize calls. Third, you definitely need to use Boost.Build for a few sample programs before you think about switching back to Makefiles. I share your frustration at Boost.Build sometimes, but it does a particular job admirably. I submitted some patches to the docs and the Jamfile rules a while ago that were aimed at making it easier to use Boost.Build for extensions and embedded apps. Dave hasn't gotten back to me yet. (Hint hint) -- -- Graeme graeme.lufkin@gmail.com
On Thu, 1 Jul 2004 13:52:17 -0700, Graeme Lufkin <graeme.lufkin@gmail.com> wrote:
On Thu, 1 Jul 2004 20:19:37 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
I've had similar problems, and have slogged my way to figuring them out. The tutorial is wrong, at least on gcc 3.2.3. Here's what you want: After calling Py_Initialize(), you should get the Python main namespace object by doing:
handle<> main_module(borrowed( PyImport_AddModule("__main__") )); dict main_namespace = extract<dict>(PyModule_GetDict(main_module.get()));
I think this should be main_module.ptr() since main_module is an object.
In my example it's a handle<>, so get() is correct. In Dave's (and your) examples, it's an object, so ptr() is correct.
Ok. Sorry. I stand corrected.
I'm trying with
****************************************************************************** #include <boost/python.hpp> using namespace boost::python;
void foo() { Py_Initialize();
object main_module = object(handle<>(borrowed(PyImport_AddModule("__main__"))));
dict main_namespace = extract<dict>(PyModule_GetDict(main_module.ptr()));
handle<> unused(PyRun_String("hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.ptr(),main_namespace.ptr())); Py_Finalize(); }
BOOST_PYTHON_MODULE(embed) { def("foo",foo); } *******************************************************************************
This compiles, but I get the same error as before, namely
NameError: name 'file' is not defined
Does this work correctly for you? I'm wondering if there is some issue with my compiler flags. I'm trying to get Boost.Build to work but there are some linking problems.
First, your extension module should never call Py_Initialize() or Py_Finalize(), as Python will already be initialized by the time you're running methods in it. I think what you want is foo() to be main(), with no extension module stuff needed. Also, Boost.Python and Py_Finalize() don't play nice, so never call it (check the list archive for details, I don't really understand why myself).
Second, yes this compiles and runs correctly for me, after removing the Py_Initialize and Py_Finalize calls.
Just to make sure we are on the same page here. I want to run the interpreter inside an extension module. In other words, I want an extension module called embed.so, not an executable called embed.
Third, you definitely need to use Boost.Build for a few sample programs before you think about switching back to Makefiles. I share your frustration at Boost.Build sometimes, but it does a particular job admirably. I submitted some patches to the docs and the Jamfile rules a while ago that were aimed at making it easier to use Boost.Build for extensions and embedded apps. Dave hasn't gotten back to me yet. (Hint hint)
Up to now I had been using distutils for extension modules, which has worked fine for examples which don't embed the interpreter. It doesn't seem to be working for this example, though. I've been trying to use Boost.Build for this example. I added a section to libs/python/example/Jamfile as follows. ******************************************************************* # ----- embed ------- # Declare a Python extension called embed extension embed : # sources embed.cc # requirements and dependencies for Boost.Python extensions <template>@boost/libs/python/build/extension ; ******************************************************************* This builds fine, but when I try to load it, I get In [1]: import embed --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /usr/local/src/libboost-python1.31.0/boost-1.31.0/libs/ python/example/bin/example/embed.so/gcc/debug/shared-linkable-true/<console> ImportError: libboost_python.so.1.31.0: cannot open shared object file: No such file or directory Any idea how to fix this? Debian has /usr/lib/libboost_python.so and libboost_python-gcc-mt-1_31.so, where the former is a symbolic link to the latter. There seems to be some gcc.mak files floating around, and it looks like I could modify the shared libs line appropriately, but I am not sure which file I need to modify. In any case, could you tell me what the compile lines you are using for the example above are? I could try it and see. Faheem.
Okay, so you're embedding in your extension. I don't think you need to call Py_Initialize(), as this should already have been done by the python executable that you start to import your module. I don't think it's wrong to call it multiple times, but should be unnecessary. On Thu, 1 Jul 2004 22:48:24 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
In [1]: import embed --------------------------------------------------------------------------- ImportError Traceback (most recent call last)
/usr/local/src/libboost-python1.31.0/boost-1.31.0/libs/ python/example/bin/example/embed.so/gcc/debug/shared-linkable-true/<console>
ImportError: libboost_python.so.1.31.0: cannot open shared object file: No such file or directory
Any idea how to fix this? Debian has /usr/lib/libboost_python.so and libboost_python-gcc-mt-1_31.so, where the former is a symbolic link to the latter.
This looks like libbost isn't in your LD_LIBRARY_PATH. When you make a Python extension with the 'extension' target in Boost.Build, it will build a copy of libboost and your extension (look in the bin directory). This one is the one that your extension gets linked against, and so is the one it's looking for when you import the module. Since you've got Debian's version of the compiled Boost libraries installed, it might be better to link against them. You can do that with a Jamfile entry similar to: extension getting_started1-external : getting_started1.cpp <find-library>boost_python ; Since you're doing embedding in the extension, you probably also need the embedding flags, so it would look like: extension embed_in_extend_test : embed_in_extend.cpp <find-library>boost_python $(BOOST_PYTHON_V2_PROPERTIES) <find-library>$(PYTHON_EMBEDDED_LIBRARY) ; where $(BOOST_PYTHON_V2_PROPERTIES) and $(PYTHON_EMBEDDED_LIBRARY) are variables that are defined by the Boost.Python library Jamfile. The Jamfile entry for the test I ran (that compiled and ran fine) is: exe simpletest : simpletest.cpp : $(BOOST_PYTHON_V2_PROPERTIES) <find-library>$(PYTHON_EMBEDDED_LIBRARY) <find-library>boost_python-gcc <library-path>$(HOME)/Projects/boost_install/lib ; Note the different name for the shared library to link against, that's because I'm using the libraries built by running bjam in the root of my Boost installation. They got put in a different directory (which is also in my LD_LIBRARY_PATH), so I add that to the list of linker paths. -- -- Graeme graeme.lufkin@gmail.com
On Thu, 1 Jul 2004 16:12:13 -0700, Graeme Lufkin <graeme.lufkin@gmail.com> wrote:
Okay, so you're embedding in your extension. I don't think you need to call Py_Initialize(), as this should already have been done by the python executable that you start to import your module. I don't think it's wrong to call it multiple times, but should be unnecessary.
On Thu, 1 Jul 2004 22:48:24 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
In [1]: import embed --------------------------------------------------------------------------- ImportError Traceback (most recent call last)
/usr/local/src/libboost-python1.31.0/boost-1.31.0/libs/ python/example/bin/example/embed.so/gcc/debug/shared-linkable-true/<console>
ImportError: libboost_python.so.1.31.0: cannot open shared object file: No such file or directory
Any idea how to fix this? Debian has /usr/lib/libboost_python.so and libboost_python-gcc-mt-1_31.so, where the former is a symbolic link to the latter.
This looks like libbost isn't in your LD_LIBRARY_PATH. When you make a Python extension with the 'extension' target in Boost.Build, it will build a copy of libboost and your extension (look in the bin directory). This one is the one that your extension gets linked against, and so is the one it's looking for when you import the module. Since you've got Debian's version of the compiled Boost libraries installed, it might be better to link against them.
Yes, I just realised that this is what was going on.
You can do that with a Jamfile entry similar to:
extension getting_started1-external : getting_started1.cpp <find-library>boost_python ;
Since you're doing embedding in the extension, you probably also need the embedding flags, so it would look like:
extension embed_in_extend_test : embed_in_extend.cpp <find-library>boost_python $(BOOST_PYTHON_V2_PROPERTIES) <find-library>$(PYTHON_EMBEDDED_LIBRARY) ;
where $(BOOST_PYTHON_V2_PROPERTIES) and $(PYTHON_EMBEDDED_LIBRARY) are variables that are defined by the Boost.Python library Jamfile.
The Jamfile entry for the test I ran (that compiled and ran fine) is: exe simpletest : simpletest.cpp : $(BOOST_PYTHON_V2_PROPERTIES) <find-library>$(PYTHON_EMBEDDED_LIBRARY) <find-library>boost_python-gcc <library-path>$(HOME)/Projects/boost_install/lib ; Note the different name for the shared library to link against, that's because I'm using the libraries built by running bjam in the root of my Boost installation. They got put in a different directory (which is also in my LD_LIBRARY_PATH), so I add that to the list of linker paths.
This is very useful information. Thanks. Ok. I am able to successfully build and run an executable, using a recipe similar to the one you provided. It creates a file called hello.txt in the directory. However, I still cannot get a Python extension module with the interpreter imbedded to work, however. I still get the same error as before. Details below. Again, I really appreciate your help. The extension module file follows. When I try to load this I get In [1]: import embed_ext In [2]: embed_ext.foo() --------------------------------------------------------------------------- NameError Traceback (most recent call last) /usr/local/src/boost/boost-1.31.0/libs/python/example/ bin/example/embed_ext.so/gcc/debug/shared-linkable-true/<console> /usr/local/src/boost/boost-1.31.0/libs/python/example/ bin/example/embed_ext.so/gcc/debug/shared-linkable-true/<string> NameError: name 'file' is not defined So there is still some problem. It is not clear from your message whether you actually built and tried to run the extension module with the embedded interpreter. If you have not, then I'd be much obliged if you try out the following code, and see if you can reproduce this problem. Any idea what I am doing wrong at this point? Thanks, Faheem. The extension module file: ******************************************************************** embed_ext.cc ******************************************************************** #include <boost/python.hpp> using namespace boost::python; void foo() { handle<> main_module(borrowed( PyImport_AddModule("__main__") )); dict main_namespace = extract<dict>(PyModule_GetDict(main_module.get())); handle<> unused(PyRun_String("hello = file('hello. txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.ptr(),main_namespace.ptr())); } BOOST_PYTHON_MODULE(embed_ext) { def("foo",foo); } ****************************************************************** The executable file: ****************************************************************** embed.cc ****************************************************************** #include <boost/python.hpp> using namespace boost::python; int main() { Py_Initialize(); handle<> main_module(borrowed( PyImport_AddModule("__main__") )); dict main_namespace = extract<dict>(PyModule_GetDict(main_module.get())); handle<> unused(PyRun_String("hello = file('hello. txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()", Py_file_input, main_namespace.ptr(),main_namespace.ptr())); Py_Finalize(); } ******************************************************************** The Jamfile: ******************************************************************** Jamfile ******************************************************************** # This is the top of our own project tree project-root ; # Include definitions needed for Python modules import python ; # Declare an executable called embed exe embed : embed.cc : $(BOOST_PYTHON_V2_PROPERTIES) <find-library>$(PYTHON_EMBEDDED_LIBRARY) <find-library>boost_python <find-library>python2.3 ; # Declare a Python extension called embed_ext extension embed_ext : embed_ext.cc : $(BOOST_PYTHON_V2_PROPERTIES) <find-library>$(PYTHON_EMBEDDED_LIBRARY) <find-library>boost_python <find-library>python2.3 ; ***********************************************************************
Faheem Mitha wrote:
However, I still cannot get a Python extension module with the interpreter imbedded to work, however. I still get the same error as before.
Hi Faheem, The code in the tutorial can be found in the test/embedding.cpp/py You can find a function called test_tutorial() there: void test_tutorial() { using namespace boost::python; object main_module( handle<>(borrowed(PyImport_AddModule("__main__")))); object main_namespace = main_module.attr("__dict__"); handle<>(PyRun_String( "hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()" , Py_file_input , main_namespace.ptr() , main_namespace.ptr()) ); } AFAIK, the code is passing the regression tests. Perhaps you can use the test example as a staring point? HTH, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net
On Fri, 02 Jul 2004 14:57:27 +0800, Joel de Guzman <joel@boost-consulting.com> wrote:
Faheem Mitha wrote:
However, I still cannot get a Python extension module with the interpreter imbedded to work, however. I still get the same error as before.
Hi Faheem,
The code in the tutorial can be found in the test/embedding.cpp/py You can find a function called test_tutorial() there:
void test_tutorial() { using namespace boost::python;
object main_module( handle<>(borrowed(PyImport_AddModule("__main__"))));
object main_namespace = main_module.attr("__dict__");
handle<>(PyRun_String(
"hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()"
, Py_file_input , main_namespace.ptr() , main_namespace.ptr()) ); }
AFAIK, the code is passing the regression tests. Perhaps you can use the test example as a staring point?
Thanks. However, I am not sure which version of the test/embedding.cpp you are referring to. The version at http://www.boost.org/libs/python/test/embedding.cpp which I assume is the current CVS, is the same as the version in 1.31, and neither of them have the code you refer to. In any case, does the code you refer to define an extension module which embeds the interpreter and can be called from Python? Because that is the case I am interested in. I have been unable to find any examples out there along these lines, despite some effort. The current version of test/embedding.cpp that I have seen is a C++ executable, and the extension module defined there is called within the interpreter that is embedded in this executable. This is not the same as what I am trying to do. Faheem.
On Fri, 2 Jul 2004 15:27:08 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
Thanks. However, I am not sure which version of the test/embedding.cpp you are referring to. The version at http://www.boost.org/libs/python/test/embedding.cpp which I assume is the current CVS, is the same as the version in 1.31, and neither of them have the code you refer to.
In any case, does the code you refer to define an extension module which embeds the interpreter and can be called from Python? Because that is the case I am interested in. I have been unable to find any examples out there along these lines, despite some effort.
The current version of test/embedding.cpp that I have seen is a C++ executable, and the extension module defined there is called within the interpreter that is embedded in this executable. This is not the same as what I am trying to do.
Having found this code on the net (by doing a Google search) I see that it indeed does not satisfy the criteria I described. There is no extension module here. It defines some functions which imbed the interpreter and then executes them in main. Does the case I described even work? I am not interested in an executable. Like I said, I have yet to see any examples out there. Maybe I'm missing something. I don't see what, though. Faheem.
Faheem: I think you're confused about one point. All extension modules "embed the interpreter". Since Python has loaded them, they all have access to the Python/C API and can do what they wish with it. Your real problem is that you don't seem to have the 'file' builtin. You should put more work into trying to decompose and solve that problem. Now that I think about it, perhaps your extension module doesn't have direct access to the builtins, while __main__ does. Try changing your hard-coded script to: "from __builtin__ import *\n" "hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()" Chad On Fri, 2 Jul 2004 15:27:08 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
On Fri, 02 Jul 2004 14:57:27 +0800, Joel de Guzman <joel@boost-consulting.com> wrote:
Faheem Mitha wrote:
However, I still cannot get a Python extension module with the interpreter imbedded to work, however. I still get the same error as before.
Hi Faheem,
The code in the tutorial can be found in the test/embedding.cpp/py You can find a function called test_tutorial() there:
void test_tutorial() { using namespace boost::python;
object main_module( handle<>(borrowed(PyImport_AddModule("__main__"))));
object main_namespace = main_module.attr("__dict__");
handle<>(PyRun_String(
"hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()"
, Py_file_input , main_namespace.ptr() , main_namespace.ptr()) ); }
AFAIK, the code is passing the regression tests. Perhaps you can use the test example as a staring point?
Thanks. However, I am not sure which version of the test/embedding.cpp you are referring to. The version at http://www.boost.org/libs/python/test/embedding.cpp which I assume is the current CVS, is the same as the version in 1.31, and neither of them have the code you refer to.
In any case, does the code you refer to define an extension module which embeds the interpreter and can be called from Python? Because that is the case I am interested in. I have been unable to find any examples out there along these lines, despite some effort.
The current version of test/embedding.cpp that I have seen is a C++ executable, and the extension module defined there is called within the interpreter that is embedded in this executable. This is not the same as what I am trying to do.
Faheem.
_______________________________________________ C++-sig mailing list C++-sig@python.org http://mail.python.org/mailman/listinfo/c++-sig
-- Chad Austin http://aegisknight.org/
On Fri, 2 Jul 2004 10:43:29 -0500, Chad Austin <caustin@gmail.com> wrote:
Faheem: I think you're confused about one point. All extension modules "embed the interpreter". Since Python has loaded them, they all have access to the Python/C API and can do what they wish with it.
If you say so. I'm too ignorant about how these things work to have an opinion.
Your real problem is that you don't seem to have the 'file' builtin. You should put more work into trying to decompose and solve that problem.
I've been trying... Actually, I don't think I have ever put so much effort before into getting a library working.
Now that I think about it, perhaps your extension module doesn't have direct access to the builtins, while __main__ does. Try changing your hard-coded script to:
"from __builtin__ import *\n" "hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()"
I've tried this already. I think someone else suggested this. I get (after building with bjam): ************************************************************************** In [1]: import embed_ext In [2]: embed_ext.foo() --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /usr/local/src/boost/boost-1.31.0/libs/python/example/ bin/example/embed_ext.so/gcc/debug/shared-linkable-true/<console> /usr/local/src/boost/boost-1.31.0/libs/python/example/ bin/example/embed_ext.so/gcc/debug/shared-linkable-true/<string> ImportError: __import__ not found ************************************************************************** I don't know what this means. :-( . Perhaps simply that it does not have the `import' builtin available to import `__builtin__'. :-) Have you been able to reproduce any of these results? Thanks. Faheem.
Heh, I've run into that problem before too. You may have to do the import with PyImport_Import or whatever it is. (The C equivalent of __import__) Oddly enough, I'm not seeing any of the problems you are. See the attached files. Works fine for me. Chad@basilisk ~/tmp/bp $ scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... g++ -DBOOST_PYTHON_STATIC_LIB -DBOOST_PYTHON_NO_TEMPLATE_EXPORT -I/usr/local/inc lude/boost-1_31 -I/usr/include/python2.3 -c -o hello.os hello.cpp g++ -shared -o hello.dll hello.os -L/usr/lib/python2.3/config -L/usr/local/lib - lboost_python-gcc -lpython2.3 scons: done building targets. Chad@basilisk ~/tmp/bp $ python Python 2.3.4 (#1, Jun 13 2004, 11:21:03) [GCC 3.3.1 (cygming special)] on cygwin Type "help", "copyright", "credits" or "license" for more information.
import hello w = hello.World() w.run() import os os.system('cat hello.txt') Hello world!0
On Fri, 2 Jul 2004 16:18:59 +0000 (UTC), Faheem Mitha <faheem@email.unc.edu> wrote:
On Fri, 2 Jul 2004 10:43:29 -0500, Chad Austin <caustin@gmail.com> wrote:
Now that I think about it, perhaps your extension module doesn't have direct access to the builtins, while __main__ does. Try changing your hard-coded script to:
"from __builtin__ import *\n" "hello = file('hello.txt', 'w')\n" "hello.write('Hello world!')\n" "hello.close()"
I've tried this already. I think someone else suggested this. I get (after building with bjam):
************************************************************************** In [1]: import embed_ext
In [2]: embed_ext.foo() --------------------------------------------------------------------------- ImportError Traceback (most recent call last)
/usr/local/src/boost/boost-1.31.0/libs/python/example/ bin/example/embed_ext.so/gcc/debug/shared-linkable-true/<console>
/usr/local/src/boost/boost-1.31.0/libs/python/example/ bin/example/embed_ext.so/gcc/debug/shared-linkable-true/<string>
ImportError: __import__ not found
**************************************************************************
I don't know what this means. :-( . Perhaps simply that it does not have the `import' builtin available to import `__builtin__'. :-) Have you been able to reproduce any of these results? Thanks.
Faheem.
_______________________________________________ C++-sig mailing list C++-sig@python.org http://mail.python.org/mailman/listinfo/c++-sig
-- Chad Austin http://aegisknight.org/
I'm interested to see you're using SCons to build your modules. Would you mind posting a short account of your experience using SCons, both in general and with Boost.Python? I still wrestle with Makefile vs Boost.Build vs tearing-my-hair-out every once in while. A more complicate SConstruct file would be great to look at too. Thanks On Fri, 2 Jul 2004 11:48:43 -0500, Chad Austin <caustin@gmail.com> wrote:
Chad@basilisk ~/tmp/bp $ scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... g++ -DBOOST_PYTHON_STATIC_LIB -DBOOST_PYTHON_NO_TEMPLATE_EXPORT -I/usr/local/inc lude/boost-1_31 -I/usr/include/python2.3 -c -o hello.os hello.cpp g++ -shared -o hello.dll hello.os -L/usr/lib/python2.3/config -L/usr/local/lib - lboost_python-gcc -lpython2.3 scons: done building targets.
-- -- Graeme graeme.lufkin@gmail.com
Hey Graeme, SCons is awesome! (Bias note: I've been one of the developers since shortly after the project went public. But that's because I immediately saw how much better a build tool using Python syntax is than make. Or pretty much anything, really.) I use it to build every project I start, especially the larger ones. The focus on build reliability and one global dependency tree is great. You almost never have to do an scons -c and scons in order to get a correct build. And when you do have to clean in order to verify it's correct, something is buggy. :) Another advantage of SCons is Steven Knight, the guy who originally created it. He's an incredibly awesome manager and a nice guy all around. If you feel good about SCons, he does too. If you don't feel good, he and the rest of the community try to make SCons a little nicer. The SConstruct I attached is a total hackjob so I could get something compiling as fast as possible. A better one might look like: def python_tool(env): pybase = 'python%s' % sys.version[0:3] env.Append(CPPPATH=[os.path.join(sys.prefix, 'include', pybase)], LIBPATH=[os.path.join(sys.prefix, 'lib', pybase, 'config')], LIBS=['lib%s' % pybase]) if env['PLATFORM'] not in ['cygwin', 'win32']: env.Append(LIBS=['util']) def boost_python_tool(env): env.Append(CPPDEFINES=['BOOST_PYTHON_STATIC_LIB', 'BOOST_PYTHON_STATIC_MODULE'], CPPPATH=['$boostIncludes'], # boostIncludes is a PathOption LIBS=['boost_python-gcc']) import os env = Environment(ENV=os.environ, tools=['default', python_tool, boost_python_tool]) env.SharedLibrary('hello', 'hello.cpp') Then you could start to migrate in options (via the Options object) and platform tests (via testing env['PLATFORM'] and such. On this list, I keep hearing this vibe "you should use bjam to make sure you get the voodoo build commands right, otherwise things will fail in unpredictable ways." I've never had a problem (win32, cygwin, linux), though. I just define BOOST_PYTHON_STATIC_LIB, BOOST_PYTHON_STATIC_MODULE (for embedding), and BOOST_PYTHON_NO_TEMPLATE_EXPORT (although this is kinda broken in 1.31). Use SharedLibrary if you're building an extension module, etc. Chad On Fri, 2 Jul 2004 12:21:07 -0700, Graeme Lufkin <graeme.lufkin@gmail.com> wrote:
I'm interested to see you're using SCons to build your modules. Would you mind posting a short account of your experience using SCons, both in general and with Boost.Python? I still wrestle with Makefile vs Boost.Build vs tearing-my-hair-out every once in while. A more complicate SConstruct file would be great to look at too. Thanks
On Fri, 2 Jul 2004 11:48:43 -0500, Chad Austin <caustin@gmail.com> wrote:
Chad@basilisk ~/tmp/bp $ scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... g++ -DBOOST_PYTHON_STATIC_LIB -DBOOST_PYTHON_NO_TEMPLATE_EXPORT -I/usr/local/inc lude/boost-1_31 -I/usr/include/python2.3 -c -o hello.os hello.cpp g++ -shared -o hello.dll hello.os -L/usr/lib/python2.3/config -L/usr/local/lib - lboost_python-gcc -lpython2.3 scons: done building targets.
-- -- Graeme graeme.lufkin@gmail.com
_______________________________________________ C++-sig mailing list C++-sig@python.org http://mail.python.org/mailman/listinfo/c++-sig
-- Chad Austin http://aegisknight.org/
--- Chad Austin <caustin@gmail.com> wrote:
SCons is awesome!
SCons is awesome indeed! I am using it ever since the repository support was added (0.08 or so). However, there is one thing I have to hack around: the implicit dependency scan of the boost tree is horribly slow on some older platforms (minutes before the first compilation command is issued) and even on new platforms I find the waits very annoying when I am in a debug/recompile cycle. 18 months ago or so I submitted a patch that allowed me to turn the implicit dependency scan on/off for selected subdirectories. However, the SCons people didn't even buy into the idea, let alone the patch. Right now I resort to the trick where I add include paths to CPPFLAGS instead of CPPPATH. It is not pretty at all because I have remember to do this in each and every Environment, and it is difficult to pinpoint where I might have forgotten to do so. Since you are a SCons developer that is also using Boost: don't you have the same problem? Is there a chance that SCons will at some point provide a rational scheme for turning the implicit dependency scan on/off in selected subdirectories that are known to be static? Is there an alternative that I missed? Ralf __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - 100MB free storage! http://promotions.yahoo.com/new_mail
On Fri, 2004-07-02 at 18:26, Ralf W. Grosse-Kunstleve wrote:
--- Chad Austin <caustin@gmail.com> wrote:
SCons is awesome!
SCons is awesome indeed! I am using it ever since the repository support was added (0.08 or so). However, there is one thing I have to hack around: the implicit dependency scan of the boost tree is horribly slow on some older platforms (minutes before the first compilation command is issued) and even on new platforms I find the waits very annoying when I am in a debug/recompile cycle.
I've only recently started learning how to use Scons, and not with Boost yet. There is a way to use timestamps rather than md5 hashes to check dependency relationships. SourceSignatures('timestamp') Does that help? This sounds like a major scalability problem to me. -Jonathan
On Fri, 2004-07-02 at 22:07, Jonathan Brandmeyer wrote:
On Fri, 2004-07-02 at 18:26, Ralf W. Grosse-Kunstleve wrote:
--- Chad Austin <caustin@gmail.com> wrote:
SCons is awesome!
SCons is awesome indeed! I am using it ever since the repository support was added (0.08 or so). However, there is one thing I have to hack around: the implicit dependency scan of the boost tree is horribly slow on some older platforms (minutes before the first compilation command is issued) and even on new platforms I find the waits very annoying when I am in a debug/recompile cycle.
I've only recently started learning how to use Scons, and not with Boost yet. There is a way to use timestamps rather than md5 hashes to check dependency relationships. SourceSignatures('timestamp')
Does that help? This sounds like a major scalability problem to me.
Never mind, I see now that you mean the preprocessor's scanning to generate the dependency tree. Ungh. -Jonathan
--- Jonathan Brandmeyer <jbrandmeyer@earthlink.net> wrote:
I've only recently started learning how to use Scons, and not with Boost yet. There is a way to use timestamps rather than md5 hashes to check dependency relationships. SourceSignatures('timestamp')
The last time I tried (about 18 months ago) it didn't help. I believe the problem is that the scanner still has to traverse the files, looking for the #include directives. I didn't try combining timestamps and --implicit-cache, though... but for several reasons I am not keen to switch to time stamps anyway.
Does that help? This sounds like a major scalability problem to me.
It is a little bit of a problem, but you can work around it to some degree using the CPPFLAGS trick. The remaining waiting times are probably unavoidable if you want the high degree of automation that SCons provides. However, I often use another simple trick: I capture the output from scons for the module that I am currently working on, put it in a sh script and execute that script to compile and link just that one module. This reduces the waits to zero. When I am done debugging I am happy to wait the 15 seconds for SCons to look for all the side effects and to take over again. Let me say again the SCons is truly awesome. In the grand scheme of things my complaints are minor issues. Like any system of non-trivial complexity, SCons has a few "features" that one has to get used to and learn how to work around.
Never mind, I see now that you mean the preprocessor's scanning to generate the dependency tree. Ungh.
No, the preprocessor is fortunately not involved. SCons is doing all the work by itself. Under Linux with g++ this doesn't make a big difference time-wise, but under Tru64 Unix "make depend" (using cxx) takes almost as long as compiling everything. SCons is *much* faster in this case. Ralf __________________________________ Do you Yahoo!? Read only the mail you want - Yahoo! Mail SpamGuard. http://promotions.yahoo.com/new_mail
(If this thread is getting off-topic, maybe we should move it to the SCons lists.) Hi Ralf, In my experience, the dependency scan for boost.python hasn't been that bad... but maybe that's because my primary machine is a dual xeon 2.8. I remember your patch, but I don't remember why we disagreed with it... I could see them buying into your patch if you were persistent enough and had a strong enough argument. I don't remember how you implemented it, but maybe a good way would be to use an environment variable [SCAN_CPP_IGNORE_PATH?] that the CScanner uses to filter the directories it scans it. Chad p.s. Have you tried always using the --implicit-cache option? That usually works wonders, and very rarely results in an incorrect build. On Fri, 2 Jul 2004 15:26:43 -0700 (PDT), Ralf W. Grosse-Kunstleve <rwgk@yahoo.com> wrote:
--- Chad Austin <caustin@gmail.com> wrote:
SCons is awesome!
SCons is awesome indeed! I am using it ever since the repository support was added (0.08 or so). However, there is one thing I have to hack around: the implicit dependency scan of the boost tree is horribly slow on some older platforms (minutes before the first compilation command is issued) and even on new platforms I find the waits very annoying when I am in a debug/recompile cycle. 18 months ago or so I submitted a patch that allowed me to turn the implicit dependency scan on/off for selected subdirectories. However, the SCons people didn't even buy into the idea, let alone the patch. Right now I resort to the trick where I add include paths to CPPFLAGS instead of CPPPATH. It is not pretty at all because I have remember to do this in each and every Environment, and it is difficult to pinpoint where I might have forgotten to do so. Since you are a SCons developer that is also using Boost: don't you have the same problem? Is there a chance that SCons will at some point provide a rational scheme for turning the implicit dependency scan on/off in selected subdirectories that are known to be static? Is there an alternative that I missed?
Ralf
__________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - 100MB free storage! http://promotions.yahoo.com/new_mail
_______________________________________________ C++-sig mailing list C++-sig@python.org http://mail.python.org/mailman/listinfo/c++-sig
-- Chad Austin http://aegisknight.org/
--- Chad Austin <caustin@gmail.com> wrote:
In my experience, the dependency scan for boost.python hasn't been that bad...
If you just build libboost_python.so and maybe a couple of extensions it is not bad at all... but see below.
but maybe that's because my primary machine is a dual xeon 2.8.
I have the same machine here, with RH8.
p.s. Have you tried always using the --implicit-cache option? That usually works wonders, and very rarely results in an incorrect build.
Here are the timings if I configure our build system to compile "everything" (maybe 150k lines + Boost) and do a "no-op" build (everything up to date) with the full implicit dependency scan enabled, and all files on a local disk: % time libtbx.scons . scons: Reading SConscript files ... Compiling distributed version of phaser scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets. 57.200u 6.450s 1:03.74 99.8% 0+0k 0+0io 1394pf+0w % time libtbx.scons --implicit-cache . scons: Reading SConscript files ... Compiling distributed version of phaser scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets. 57.600u 6.480s 1:04.21 99.7% 0+0k 0+0io 1394pf+0w And here the same but with the implicit dependency scan disabled for the Boost subdirectory, using the CPPFLAGS trick: % time libtbx.scons . scons: Reading SConscript files ... libtbx.scons: implicit dependency scan disabled for directory /net/worm/scratch1/rwgk/dist/boost Compiling distributed version of phaser scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets. 15.540u 1.340s 0:17.41 96.9% 0+0k 0+0io 1402pf+0w % time libtbx.scons --implicit-cache . scons: Reading SConscript files ... libtbx.scons: implicit dependency scan disabled for directory /net/worm/scratch1/rwgk/dist/boost Compiling distributed version of phaser scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets. 15.350u 1.330s 0:16.88 98.8% 0+0k 0+0io 1402pf+0w Four times faster! Cool? --implicit-cache doesn't seem to do anything for me. Am I using this option incorrectly?
I remember your patch, but I don't remember why we disagreed with it...
My initial patch/suggestion was to introduce a global registry that the scanner uses to decide where to prune the search for implicit dependencies. The registry had switches to turn the scan on/off.
I could see them buying into your patch if you were persistent enough and had a strong enough argument. I don't remember how you implemented it, but maybe a good way would be to use an environment variable [SCAN_CPP_IGNORE_PATH?] that the CScanner uses to filter the directories it scans it.
That sounds like a great idea! Why didn't I think of that? :-) It is not quite as global as I had in mind initially; one still has to be careful to define the paths in all Environment instances. However, our usage pattern has evolved over time is to use env.Copy() anyway to get common settings from a basic Environment instance. I.e. your idea appears to be a perfect fit. I'd just have to add the SCAN_CPP_IGNORE_PATH to our basic environment. Do you have an idea how much is involved in adding the SCAN_CPP_IGNORE_PATH feature? Would it be a local patch in the scanner, or does the general framework need to be patched in several places to provide the necessary support? Ralf __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - Send 10MB messages! http://promotions.yahoo.com/new_mail
On Sat, 3 Jul 2004 23:41:38 -0700 (PDT), Ralf W. Grosse-Kunstleve <rwgk@yahoo.com> wrote:
--- Chad Austin <caustin@gmail.com> wrote:
p.s. Have you tried always using the --implicit-cache option? That usually works wonders, and very rarely results in an incorrect build.
Here are the timings if I configure our build system to compile "everything" (maybe 150k lines + Boost) and do a "no-op" build (everything up to date) with the full implicit dependency scan enabled, and all files on a local disk:
% time libtbx.scons . [snip] 57.200u 6.450s 1:03.74 99.8% 0+0k 0+0io 1394pf+0w % time libtbx.scons --implicit-cache . [snip] 57.600u 6.480s 1:04.21 99.7% 0+0k 0+0io 1394pf+0w
Did you try running with --implicit-cache again? The first time you use it, it caches the dependency information of headers. The second time, it uses that information to speed up the build. In short, you could describe --implicit-cache as adding a test: "if signature of header has not changed, then don't scan it: use the previous headers found." That gets rid of the scanner pass (although MD5 is still run).
And here the same but with the implicit dependency scan disabled for the Boost subdirectory, using the CPPFLAGS trick:
% time libtbx.scons . scons: Reading SConscript files ... libtbx.scons: implicit dependency scan disabled for directory /net/worm/scratch1/rwgk/dist/boost Compiling distributed version of phaser scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets. 15.540u 1.340s 0:17.41 96.9% 0+0k 0+0io 1402pf+0w % time libtbx.scons --implicit-cache . scons: Reading SConscript files ... libtbx.scons: implicit dependency scan disabled for directory /net/worm/scratch1/rwgk/dist/boost Compiling distributed version of phaser scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets. 15.350u 1.330s 0:16.88 98.8% 0+0k 0+0io 1402pf+0w
Four times faster! Cool?
Nice. Try running with --implicit-cache a few times in a row... then we can compare it with your patch. Of course, with both, you might get an even bigger speedup. :)
--implicit-cache doesn't seem to do anything for me. Am I using this option incorrectly?
[snip]
That sounds like a great idea! Why didn't I think of that? :-) It is not quite as global as I had in mind initially; one still has to be careful to define the paths in all Environment instances. However, our usage pattern has evolved over time is to use env.Copy() anyway to get common settings from a basic Environment instance. I.e. your idea appears to be a perfect fit. I'd just have to add the SCAN_CPP_IGNORE_PATH to our basic environment.
That sounds good. I've noticed that nearly every SCons build creates an initial environment and uses env.Copy() to customize it, so it should be okay.
Do you have an idea how much is involved in adding the SCAN_CPP_IGNORE_PATH feature? Would it be a local patch in the scanner, or does the general framework need to be patched in several places to provide the necessary support?
I'm pretty sure you would only have to patch the scanner. Fortunately, the scanner code is pretty straightforward. Take a look at Scanner/__init__.py. Should be easy to implement using an environmental variable. :) Cheers, Chad
--- Chad Austin <caustin@gmail.com> wrote:
% time libtbx.scons . [snip] 57.200u 6.450s 1:03.74 99.8% 0+0k 0+0io 1394pf+0w % time libtbx.scons --implicit-cache . [snip] 57.600u 6.480s 1:04.21 99.7% 0+0k 0+0io 1394pf+0w
Did you try running with --implicit-cache again? The first time you use it, it caches the dependency information of headers. The second time, it uses that information to speed up the build. In short, you could describe --implicit-cache as adding a test: "if signature of header has not changed, then don't scan it: use the previous headers found." That gets rid of the scanner pass (although MD5 is still run).
It doesn't help here running the no-op build four times in a row: 57.900u 6.360s 1:04.30 99.9% 0+0k 0+0io 1394pf+0w 58.290u 6.760s 1:05.34 99.5% 0+0k 0+0io 1394pf+0w 61.580u 6.570s 1:08.22 99.8% 0+0k 0+0io 1394pf+0w 57.450u 6.760s 1:04.27 99.9% 0+0k 0+0io 1394pf+0w
Nice. Try running with --implicit-cache a few times in a row... then we can compare it with your patch. Of course, with both, you might get an even bigger speedup. :)
Similar observation using the CPPFLAGS trick: 15.340u 1.310s 0:16.82 98.9% 0+0k 0+0io 1402pf+0w 15.220u 1.480s 0:16.81 99.3% 0+0k 0+0io 1402pf+0w 15.280u 1.440s 0:16.78 99.6% 0+0k 0+0io 1402pf+0w 15.160u 1.410s 0:16.65 99.5% 0+0k 0+0io 1402pf+0w Do I have to run something else before using --implicit-cache? IIRC I also tried --implicit-deps-unchanged and other options at some point in the past. Nothing made a significant difference.
I'm pretty sure you would only have to patch the scanner. Fortunately, the scanner code is pretty straightforward.
I know...
Take a look at Scanner/__init__.py. Should be easy to implement using an environmental variable. :)
OK, I'll put this on my try-again list. :-) Ralf __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - Send 10MB messages! http://promotions.yahoo.com/new_mail
On Sun, 4 Jul 2004 10:24:18 -0700 (PDT), Ralf W. Grosse-Kunstleve <rwgk@yahoo.com> wrote:
--- Chad Austin <caustin@gmail.com> wrote:
% time libtbx.scons . [snip] 57.200u 6.450s 1:03.74 99.8% 0+0k 0+0io 1394pf+0w % time libtbx.scons --implicit-cache . [snip] 57.600u 6.480s 1:04.21 99.7% 0+0k 0+0io 1394pf+0w
Did you try running with --implicit-cache again? The first time you use it, it caches the dependency information of headers. The second time, it uses that information to speed up the build. In short, you could describe --implicit-cache as adding a test: "if signature of header has not changed, then don't scan it: use the previous headers found." That gets rid of the scanner pass (although MD5 is still run).
It doesn't help here running the no-op build four times in a row:
57.900u 6.360s 1:04.30 99.9% 0+0k 0+0io 1394pf+0w 58.290u 6.760s 1:05.34 99.5% 0+0k 0+0io 1394pf+0w 61.580u 6.570s 1:08.22 99.8% 0+0k 0+0io 1394pf+0w 57.450u 6.760s 1:04.27 99.9% 0+0k 0+0io 1394pf+0w
That's surprising. I'd guess most of the time is spent doing MD5 scans? Have you tried using timestamps?
Nice. Try running with --implicit-cache a few times in a row... then we can compare it with your patch. Of course, with both, you might get an even bigger speedup. :)
Similar observation using the CPPFLAGS trick:
15.340u 1.310s 0:16.82 98.9% 0+0k 0+0io 1402pf+0w 15.220u 1.480s 0:16.81 99.3% 0+0k 0+0io 1402pf+0w 15.280u 1.440s 0:16.78 99.6% 0+0k 0+0io 1402pf+0w 15.160u 1.410s 0:16.65 99.5% 0+0k 0+0io 1402pf+0w
Do I have to run something else before using --implicit-cache? IIRC I also tried --implicit-deps-unchanged and other options at some point in the past. Nothing made a significant difference.
--implicit-cache should work automagically. If it's not working, something else is the bottleneck.
Take a look at Scanner/__init__.py. Should be easy to implement using an environmental variable. :)
OK, I'll put this on my try-again list. :-)
Sounds good. :) Cheers, Chad
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 5 Jul 2004 at 18:58, Chad Austin wrote:
That's surprising. I'd guess most of the time is spent doing MD5 scans? Have you tried using timestamps?
No, a good proportion is spent by scons organising its environment - basically shuffling data around. If you run scons with psyco, you'll get a *major* improvement when bound here though last time I checked there were still issues with psyco. Cheers, Niall -----BEGIN PGP SIGNATURE----- Version: idw's PGP-Frontend 4.9.6.1 / 9-2003 + PGP 8.0.2 iQA/AwUBQOoLQ8EcvDLFGKbPEQIgiACfYHI9zjJo+LJdbcDhdZ5MxpombYgAoPz8 S0nNIwe9eJ3Oc3fCd2XFBUt5 =S8yo -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2 Jul 2004 at 12:21, Graeme Lufkin wrote:
I'm interested to see you're using SCons to build your modules. Would you mind posting a short account of your experience using SCons, both in general and with Boost.Python? I still wrestle with Makefile vs Boost.Build vs tearing-my-hair-out every once in while. A more complicate SConstruct file would be great to look at too. Thanks
I'll also add my ha'pennies worth and say SCons is great. And like everyone, all my SConstruct files feel really hacky and I prefer not showing them to people - but that said, changing them is a cinch and it's SOOO much more flexible than any other build system I've ever used. I'd really like it if Ralf's SConstruct files for Boost came with Boost. Thoughts? Cheers, Niall -----BEGIN PGP SIGNATURE----- Version: idw's PGP-Frontend 4.9.6.1 / 9-2003 + PGP 8.0.2 iQA/AwUBQObn7MEcvDLFGKbPEQIFGACfV1C17ZqPq9xMbsDPgcu5MdPL4vsAn2RU gY2luoKcoBLORPSIcoAH367D =98W2 -----END PGP SIGNATURE-----
On Sat, Jul 03, 2004 at 06:07:56PM +0100, Niall Douglas wrote:
I'll also add my ha'pennies worth and say SCons is great. And like everyone, all my SConstruct files feel really hacky
Yay :) And if you think your SConstruct files are hacky, try looking at a couple of automake or configure scripts sometime ;-) Ingo
--- Niall Douglas <s_sourceforge@nedprod.com> wrote:
I'd really like it if Ralf's SConstruct files for Boost came with Boost. Thoughts?
Hi Niall, it is not just one file. You need the "libtbx" (toolbox for building libraries) and "boost_adaptbx" (adaptor toolbox) from our CVS tree (both modules are tiny in comparison). I could make a web site like this one: http://cci.lbl.gov/cctbx_build/, but only with boost, scons libtbx, boost_adaptbx, and only sources, no binaries, and a simple examples that demonstrates how to add custom modules and how to build your own self-extracting installer. I've been thinking about this for a while but didn't/don't have enough time to do it. :-( Ralf __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - 100MB free storage! http://promotions.yahoo.com/new_mail
On Fri, 2 Jul 2004 11:48:43 -0500, Chad Austin <caustin@gmail.com> wrote:
------=_Part_96_10773503.1088786923701 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline
Heh, I've run into that problem before too. You may have to do the import with PyImport_Import or whatever it is. (The C equivalent of __import__)
You mean use PyImport_Import to import __builtin__? Can you tell me what the correct syntax for this would be?
Oddly enough, I'm not seeing any of the problems you are. See the attached files. Works fine for me.
Chad@basilisk ~/tmp/bp $ scons scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... g++ -DBOOST_PYTHON_STATIC_LIB -DBOOST_PYTHON_NO_TEMPLATE_EXPORT -I/usr/local/inc lude/boost-1_31 -I/usr/include/python2.3 -c -o hello.os hello.cpp g++ -shared -o hello.dll hello.os -L/usr/lib/python2.3/config -L/usr/local/lib - lboost_python-gcc -lpython2.3 scons: done building targets.
Chad@basilisk ~/tmp/bp $ python Python 2.3.4 (#1, Jun 13 2004, 11:21:03) [GCC 3.3.1 (cygming special)] on cygwin Type "help", "copyright", "credits" or "license" for more information.
import hello w = hello.World() w.run() import os os.system('cat hello.txt') Hello world!0
Hi. I tried using the second Scons file you posted. I had to add `import sys' at the beginning. For some reason it builds a file called libhello.so. I renamed it to hello.so. Do you know why it is generating a file called libhello.so and how to get it to output to hello.so? Anyway, I get the same problem with your code as I do with mine, namely In [1]: import hello In [2]: foo = hello.World() In [3]: foo.run() --------------------------------------------------------------------------- NameError Traceback (most recent call last) /home/faheem/wc/corrmodel/boost/<console> /home/faheem/wc/corrmodel/boost/<string> NameError: name 'file' is not defined I also used Boost.Build and got the same problem. I wonder what is going on. Possibly some configuration issue with my system. Maybe I should harass the Debian maintainers of Boost.Python about this. :-) I figure that it is worth while trying to figure out how to import __builtin__ somehow. If anyone has specific suggestions about the syntax, let me know. I'm pretty clueless. Scons looks cool. The last time I looked at it I didn't know any Python. Maybe it is time to take a look again. Thanks for your help. Faheem.
Faheem Mitha <faheem@email.unc.edu> writes:
Thanks. However, I am not sure which version of the test/embedding.cpp you are referring to.
The one in the current CVS.
The version at http://www.boost.org/libs/python/test/embedding.cpp which I assume is the current CVS,
It is not.
is the same as the version in 1.31, and neither of them have the code you refer to.
Get the current CVS. http://www.boost.org/more/getting_started.html#CVS
In any case, does the code you refer to define an extension module which embeds the interpreter
What do you mean by "embeds the interpreter?" Normally "embedding" means a stand-alone application calls the Python interpreter. When an extension module calls the Python interpreter it's just more "extending" (unless you're talking about multiple interpreter instances, in which case you should ask on comp.lang.python -- almost nobody knows how to handle that correctly).
and can be called from Python? Because that is the case I am interested in. I have been unable to find any examples out there along these lines, despite some effort.
The current version of test/embedding.cpp that I have seen is a C++ executable, and the extension module defined there is called within the interpreter that is embedded in this executable. This is not the same as what I am trying to do.
Just make a regular extension module and call the interpreter. Nothing to it. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
Faheem Mitha <faheem@email.unc.edu> writes:
This is very useful information. Thanks.
Ok. I am able to successfully build and run an executable, using a recipe similar to the one you provided. It creates a file called hello.txt in the directory.
However, I still cannot get a Python extension module with the interpreter imbedded to work, however. I still get the same error as before.
Have you looked at libs/python/test/embedding.cpp in your Boost distribution? -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
participants (9)
-
Chad Austin -
David Abrahams -
Faheem Mitha -
Graeme Lufkin -
Ingo Luetkebohle -
Joel de Guzman -
Jonathan Brandmeyer -
Niall Douglas -
Ralf W. Grosse-Kunstleve