Mailman 3 May 2000 - Cplusplus-sig

Re: [C++-SIG] Re: C++ extension module crashes on Solaris
by Steve Harris May 19, 2000

May 19, 2000

Phil Austin <phil(a)geog.ubc.ca> writes: [...] > Did you wind up having to move to the March 2000 g++ snapshot to solve your > earlier problems with example.cxx? No, we recompiled the 2.95.2 release to use Gnu binutils, rather than the default Solaris tools. That seemed to do the trick. -- Steven E. Harris Primus Knowledge Solutions, Inc. http://www.primus.com

1 0

Re: C++ extension module crashes on Solaris
by sharris＠nospam.primus.com May 19, 2000

May 19, 2000

The following message is a courtesy copy of an article that has been posted to comp.lang.python as well. "Daniel Berlin" <dan(a)cgsoftware.com> writes: [...] > No offense Guido, but you are probably completely off base. > I'll put money it's a shared library problem. [...] Yes, well, I took Guido's advice and rebuilt Python without threading support enabled. My goal (and problem, I belive) have nothing to do with threading. I'll re-start from the beginning. Following the instructions for the CXX kit, I set two environment variables prior to configuring Python: LINKCC g++ LDSHARED g++ -shared I'm not entirely sure why this is necessary. If it's not, please advise. I ran the configure script as $ ./configure --prefix=/users/sharris/usr/local Everything configured fine, so I ran make, make test, and make install. Next, I copied the file <pysrc>/Misc/Makefile.pre.in into my module build directory and followed the directions in the Python doc. called "Building C and C++ Extensions on Unix." My Setup file looks like this (continuation lines for clarity here): ================================================== # seh_test setup file *shared* CCC=g++ seh_test -I../Include \ pod.cc \ pypod.cc \ seh_test.cc \ ../Src/cxx_extensions.cxx \ ../Src/cxxextensions.c \ ../Src/cxxsupport.cxx ================================================== A note about the directory structure context - it looks like this: ~/temp /Python-1.5.2 <== Python build tree (now installed) /CXX-4.2 /Src <== CXX source files /Include <== CXX includes /seh_test <== My test module stuff using CXX My module builds okay, as evidenced here (again, line wraps are mine): $ make g++ -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c./pod.cc g++ -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c ./pypod.cc g++ -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c ./seh_test.cc g++ -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c ./../Src/cxx_extensions.cxx gcc -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c ./../Src/cxxextensions.c g++ -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c ./../Src/cxxsupport.cxx g++ -shared \ pod.o pypod.o seh_test.o \ cxx_extensions.o cxxextensions.o cxxsupport.o \ -o seh_test.so Next, I copy the built module into place and try to import it: $ cp seh_test.so ~/usr/local/lib/python1.5/ $ python Python 1.5.2 (#14, May 19 2000, 00:04:52) [GCC 2.95.2 19991024 (release)] on sunos5 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import seh_test Traceback (innermost last): File "<stdin>", line 1, in ? ImportError: ld.so.1: python: fatal: relocation error: file ./seh_test.so: symbol PyExc_IndexError: referenced symbol not found >>> Sigh. Okay, so I looked around on Deja for something relevant to this error until I almost fell asleep, and finally found this suggestion: Try passing the flags '-Wl,-E' to the linker. http://www.deja.com/[ST_rn=ps]/getdoc.xp?AN=481811059&search=thread&threade… So I reset the LDSHARED environment variable to 'g++ -shared -Wl,-E', tore down the previous Python build, reconfigured it, and rebuilt it. Python built and tested fine just like the last build. Looking over the build output, it doesn't look like those linker options every got used, since there weren't any shared objects getting built. Finally, I tried rebuilding my test module, forcing the '-Wl,-E' flag into the link line: $ cp ~/temp/Python-1.5.2/Misc/Makefile.pre.in ./ $ make -f Makefile.pre.in boot LDSHARED='g++ -shared -Wl,-E' $ make g++ -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c./pod.cc g++ -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c ./pypod.cc g++ -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c ./seh_test.cc g++ -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c ./../Src/cxx_extensions.cxx gcc -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c ./../Src/cxxextensions.c g++ -I../Include -g -O2 \ -I/users/sharris/usr/local/include/python1.5 -DHAVE_CONFIG_H \ -c ./../Src/cxxsupport.cxx g++ -shared -Wl,-E \ pod.o pypod.o seh_test.o \ cxx_extensions.o cxxextensions.o cxxsupport.o \ -o seh_test.so Again, I copy the (re)built module into place and try to import it: $ cp seh_test.so ~/usr/local/lib/python1.5/ $ python Python 1.5.2 (#1, May 19 2000, 09:34:45) [GCC 2.95.2 19991024 (release)] on sunos5 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import seh_test Traceback (innermost last): File "<stdin>", line 1, in ? ImportError: ld.so.1: python: fatal: relocation error: file ./seh_test.so: symbol PyExc_IndexError: referenced symbol not found >>> Note that I can compile and use the CXX Demo application, which uses both extension and embedding together. I can't get extending to work alone, though. What additional information can I provide to get some suggestions? -- Steven E. Harris Primus Knowledge Solutions, Inc. http://www.primus.com

3 3

Re: [C++-SIG] Const-correctness with PythonExtensionBase/PythonExtension<T>
by Steve Harris May 17, 2000

May 17, 2000

"Barry Scott" <barry(a)scottb.demon.co.uk> writes: > Should we have it as a goal to make these PyCXX extension objects > easy to use from C++? Today I would say its not a goal. Fair enough. > All the objects I've built with CXX have no need to be called from C++. > > I have needed to get at the C++ object that the PyCXX object is > wrapping from other PyCXX code. But thats a different class of > problem from the one you raise and has a solution. This is what I'm interested in. The examples in the Demo directory present what are essentially Python types implemented in C++, not wrappers around existing C++ types. You say that you know of "a solution" to this class of problem. Can you share some of what you've discovered? At present, I'm trying to learn CXX by wrapping a hypothetical "POD-like" C++ type that doesn't know anything about Python. Some of the problems I'm having are: 1. Should the Python type wrap around a reference to the C++ type, where the wrapped instance exists independently of the Python wrapper instance, or should the Python wrapper contain the wrapped type by value? The former sounds realistic for cases where there's some "context" variable existing in C++ (say, an application's open connection to a database) that it wants to "lend" to user-provided Python script. This reminds me of how the Active Server Pages processor binds the global "objects" to the relevant web server request context and hands them to script code and any invoked COM servers. 2. Should the Python type be constructible from Python? If the wrapper holds the wrapped by reference, this construction becomes difficult, because then you'd have to create two variables. How do you know when you can delete the wrapped C++ variable? 3. If you're wrapping a type, you may not have access to pieces that would be good for methods like str(). Does that just mean that the type isn't printable in the Python sense? 4. I haven't gotten to trying to wrap containers yet. What if your C++ type has a method like const std::list<foo>& get_foos() const; where the type exposes a nested container of some other type. That looks like you could coax some Pythonic behavior out of the pattern, but how would you go about wrapping it? Do you have to copy the whole list for each request for the Python-wrapped sequence? > Going back to const. If adding const meets the goal then PyCXX > should be changed. If it does not then I'd leave as is. I think it meets the higher-level aim of making use of wrapped Python types "correct" in the C++ sense, which is consistent with aims like avoiding bald pointers, casting, and such. -- Steven E. Harris Primus Knowledge Solutions, Inc. http://www.primus.com

2 1

Re: [C++-SIG] Const-correctness with PythonExtensionBase/PythonExtension<T>
by Barry Scott May 17, 2000

May 17, 2000

I think the goal of CXX is to allow Python to use objects that must be implemented in C++. I hope this goal is met and will be be met better over time. Should we have it as a goal to make these PyCXX extension objects easy to use from C++? Today I would say its not a goal. All the objects I've built with CXX have no need to be called from C++. I have needed to get at the C++ object that the PyCXX object is wrapping from other PyCXX code. But thats a different class of problem from the one you raise and has a solution. Going back to const. If adding const meets the goal then PyCXX should be changed. If it does not then I'd leave as is. Adding const to sequence_length will help the goal for the reasons you state. We should propose the adding of const to the appropiate methods. Adding const to add_XXX_method would add complexity without helping the main goal. I think your find that what you need to code for your example: template <class T> std::ostream& print_sequence(std::ostream& os, const Py::SeqBase<T>& seq) { // Iterate over sequence and print each member (str()?) return os; } Is this version that takes a sequence from python and checks that the elements of the sequence are of type T. template <class T> std::ostream& print_sequence(std::ostream& os, const Py::Sequence &seq) { // Iterate over sequence and print each member (str()?) for( int i=0; i<seq.length(); i++ ) { Py::Object o( seq[i] ); if( T::check( o ) ) { T *t = static_cast<T *>o.ptr(); os << t.str(); } else throw TypeError(T::name); } return os; } I'm not sure that endusers are supposed to use or know about Seqbase. Or I'd perfer to write: for element in sequence: file.write( str(element) ) which will call T::str() giver the sequence contains T's. BArry

1 0

Re: [C++-SIG] setattr and CXX-4.2
by Steve Harris May 17, 2000

May 17, 2000

Geoffrey Furnish <furnish(a)actel.com> writes: [...] > Also, it would just force the caller of getattr to run strlen on the > string anyway, so I don't really see a net performance improvement > coming from this. Okay. That was the condition in my suggestion: only if _we're_ writing the thing that calls this gettattr() function could this suggestion be useful. Since it's Python itself, then this is fine. -- Steven E. Harris Primus Knowledge Solutions, Inc. http://www.primus.com

1 0

Re: [C++-SIG] setattr and CXX-4.2
by Steve Harris May 16, 2000

May 16, 2000

"Barry Scott" <barry(a)scottb.demon.co.uk> writes: [...] > Here is an example taken from CXX\Demo\r.cxx > > Py::Object r::getattr( const char *name ) > { > if(std::string(name) == "c") > return Py::Float(300.0); > > if(std::string(name) == "start") > return Py::Int(start); > > return getattr_methods( name ); > } Would it be possible to get a length argument in addition to the const char* argument, if we're not getting a const std::string& as a parameter? That is, if the length of the string is known "upstream," it would be convenient to have it here to both speed comparisons or to construct a std::string as you've shown here. If the string hasn't even been examined yet, then this would not be an option here. -- Steven E. Harris Primus Knowledge Solutions, Inc. http://www.primus.com

2 1

Const-correctness with PythonExtensionBase/PythonExtension<T>
by Steve Harris May 16, 2000

May 16, 2000

Perhaps this topic has been covered before, but I just ran into it while working on my first Python extension module using CXX. Why are none of the virtual methods on PythonExtensionBase const? For a particular example, consider PythonExtensionBase::sequence_length(), defined as virtual int sequence_length(); It seems that the signature should be more like virtual int sequence_length() const; to permit it to be called on a const PythonExtension<T> instance. Additionally, the PythonExtension::add_varargs_method() and friends should be able to receive a pointer to a const member function. I've dug into the add_varargs_method() function far enough to see that we'd need a second map of function pointers to the const ones. Some reworking would be required in there. But it seems like it should be possible to make certain member functions const so that if you're calling your Python-bound types from C++, you could still respect the const semantics we know and love. Yes, Python doesn't understand the distinction between const and non-const variable, but our C++ binding should. If this debate is already over, please point me to that discussion. If it's just a current oversight in want of implementation improvement, then I'd like to help with that. -- Steven E. Harris Primus Knowledge Solutions, Inc. http://www.primus.com

2 1

Re: Problems building and testing CXX demo
by Steve Harris May 9, 2000

May 9, 2000

> Which g++ version ? $ g++ -v Reading specs from /usr/local/sun4/lib/gcc-lib/sparc-sun-solaris2.5.1/2.95.2/specs gcc version 2.95.2 19991024 (release) > Looks like (maybe) your compiler has problem with namespaces > (Py::blabla::blablabla) and that certainly is a lack of > old-non-ISO-compilant compilers. > > The error occurs in r.cxx and example.cxx You can try an nm on the > files and look what the compiler has generated as defined and > expected. The mangled names may lack the Py:: prefix (?). Here's some output: $ nm -V nm: Software Generation Utilities (SGU) Solaris/ELF (3.0) $ nm example.o | egrep 'as_string' [841] | 0| 0|NOTY |GLOB |0 |UNDEF |as_string__CQ22Py6Object $ nm r.o | egrep 'number_remainder' [303] | 0| 0|NOTY |GLOB |0 |UNDEF |number_remainder__Q22Py19PythonExtensionBaseRCQ22Py6Object I can't really understand if the "Py" in there is in fact a namespace qualifier. It's also interesting that if I use the '-C' option to nm to de-mangle the names, I get the same results back out. Does that mean that nm doesn't understand the name mangling? > Then, maybe the link editor has problem with mangled names ? Did > you use Gnu ld or not ? (we are NOT using binutils, ld and co...) I'm not sure. Here's the version dump: $ ld -V ld: Software Generation Utilities - Solaris/ELF (3.0) I was going to try building binutils in repsonse to a Usenet post¹ I found that mentioned an error message I've encountered while compiling. That would contradict your advice here. > As I told you in my previous email, we succeeded with a gcc > snapshot. I mean this is NOT gcc 2.95.2, we tryed it but it fails > with CXX. Okay. I missed that detail. We're obviously using 2.95.2 right now. Footnotes: ¹ http://www.deja.com/[ST_rn=ps]/getdoc.xp?AN=512435895&search=thread&CONTEXT… -- Steven E. Harris Primus Knowledge Solutions, Inc. http://www.primus.com

2 1

Re: Problems building and testing CXX demo
by Steve Harris May 9, 2000

May 9, 2000

> Is this an issue with the order of things on the load line? What if > you move libCXX.a to follow rtest.o? Well, after some rearranging, I did come up with this: > g++ -L ~/usr/local/lib/python1.5/config example.o python.o r.o \ rtest.o libCXX.a -lpython1.5 -lsocket -lnsl -ldl -o rtest ld: warning: relocation error: R_SPARC_32: file example.o: symbol in: external symbolic relocation against non-allocatable section .stab; cannot be processed at runtime: relocation ignored Undefined first referenced symbol in file in example.o ld: fatal: Symbol referencing errors. No output written to rtest collect2: ld returned 1 exit status This is a little better, in the sense that most of the references get resolved. There's still than lingering "in" symbol. > Sorry, we're all grasping at straws here. I'm wondering if you've > really built g++ correctly for your platform? I appreciate your help. At this point, perhaps a clean rebuild of everything is in order. I've pushed on this discussion because I figured that others may download CXX and go through similar steps. CXX looks like a great project. I'd just like to get it built so I can start contributing. -- Steven E. Harris Primus Knowledge Solutions, Inc. http://www.primus.com

1 0

Re: [C++-SIG] Re: Problems building and testing CXX demo
by Steve Harris May 9, 2000

May 9, 2000

Geoffrey Furnish <furnish(a)actel.com> writes: [...] > It looks to me like just a small problem with the invocation of the > loader. The KCC driver script may be able to notice that you've > provided a library without using a library introducer token (-l), and > perhaps g++ just isn't grokiing that in the version you're using. > > Try changing your g++ invocation line to: > > g++ python.o -o mypython -L ~/usr/local/lib/python1.5/config \ > -lpython1.5 -lm -lnsl -lsocket -ldl -L. -lcxx Thanks for the tip. Unfortunately, we're still without joy. I tried that change and got the same error regarding the undefined symbol "in." I'm wondering if this is a problem with some unintended interaction between Gnu and Sun tools. -- Steven E. Harris Primus Knowledge Solutions, Inc. http://www.primus.com

2 2