Mailman 3 March 2000 - Cplusplus-sig

just to introduce me and some comments
by Thomas.Malik＠t-online.de May 22, 2000

May 22, 2000

Hello, since i am new to this group, ok, i'm Thomas. I've taken the CXX package 3 months ago (don't know which version it was) and changed many things to suit my needs. My requirements were: - simple, homogeneous look of my wrapper classes, since i had to write a whole lot of them - use of inheritance in the wrapper classes, since the class library i had to incorporate into python, had it, and i didn't want to do mass copies of source code (every base class method in every subclass) - the whole thing has to run rock solid, since it's going to be used for interconnecting stock trading applications in a financial institute. - I wanted to rely on C++ exceptions as error notification mechanism. What i did: I moved the inheritance of the PythonExtension template from PyObject 'one level up', by introducing an intermediate class CXXPyObject with a virtual destructor: // The purpose of this is only to get a virtual destructor into a PyObject and to // be able to use dynamic_cast on it. PythonExtension inherits virtually from this. class CXXPyObject : public PyObject { protected: CXXPyObject() {} public: virtual ~CXXPyObject() {} }; CXXObject is a virtual base class for PythonExtension. Now, i can use dynamic_cast even in situations, where i inherit from PythonExtension multiple times, see below. Another thing i changed was to eliminate the template argument from PythonType. It's only purpose was to get the Object size, but you can pass this easily as a constructor argument, or even pass the size as a template argument. The downside is, you have to keep PythonExtension as a template class, since the constructor cannot determine the object size at construction time. To do easy, safe casting in the static member functions (Python entry points) i introduced a template function T python_cast<class T>(PyObject *). This does first a static_cast to the CXXObject*, and then a dynamic_cast to the object i'm actually in. Of course, this does NOT lead me magically into the paths of the right member functions of the right class, but i'm using dynamic_cast & the virtual function table for type checking, which gives me a fairly efficient type safe conversion. template<class T> T python_cast(PyObject *p) { T result = dynamic_cast<T>(static_cast< CXXPyObject* >(p)); if (result == NULL) { cerr << "!shit! cast failed!" << endl; throw bad_cast(); } return result; } Lastly, i've added some 'standard' functionality into PythonType & PythonExtension, such as str(), repr(), getattr(). I strictly obey the following rule in my source: - Python entry points always have a name starting with 's_' (such as s_str(PyObject* self, PyObject* a) ) - the first lines look like MyClassName* self=python_cast<MyClassName*>(s); // looks damn familiar, doesn't it ? Tuple args(a); so, here's some chunk out of the PythonExtension template: // since CXXPyObject is a virtual Baseclass of PythonExtension, you can // safely use multiple inheritance with the following Scheme: // class A : public PythonExtension<A> { // }; // class B : public A, public PythonExtension<B> { // } // // in A's an B's methods you may use the python_cast<A*>() thing below // to get a A* or B* out of a PyObject * template<class T> class PythonExtension: public virtual CXXPyObject { private: static void extension_object_deallocator(PyObject* t) { CXXPyObject* o=static_cast<CXXPyObject*>(t); delete o; // delete t; } explicit PythonExtension(const PythonExtension<T>& other); void operator=(const PythonExtension<T>& rhs); protected: explicit PythonExtension() { ob_refcnt = 1; ob_type = type_object(); } public: virtual ~PythonExtension() {} static PythonType& behaviors() { static PythonType* p; if(!p) { p = new PythonType(sizeof(T)); p->dealloc(extension_object_deallocator); p->str(s_str); p->repr(s_repr); } ... } If someone's interested; i can send the whole story(code). BTW, I've also implemented some huge macros for easy (and safe!) use of exceptions within C++ Modules. Note that you CAN'T throw exceptions 'through' the python interpreter. Consider this: PyObject* mainEventLoop() { PyObject* o=someObject; try { while (...) // some crufty event magic going on if (eventOccured()) Py_CallMethod(o,"crashbang",...); } catch (const Exception&) { } } and 'crashbang()' calls a C++ method from within your module, which throws an exception. My experience was that the process will go wreck, with the default execption handler aborting the program. What i did is to write a macro to catch these exceptions and to return Null instead. Ok, that's it for today. Comments are welcome. Thomas Malik

3 3

ANN: wrappy -- a Python C++ interface generator
by Greg Couch March 13, 2000

March 13, 2000

I've finally packaged up wrappy into a form that can be redistributed. Comments, contributions, questions, etc. are all welcome. Wrappy takes a C++ header file and generates a Python module that provides access to the classes and functions in the header file. The goal is to write straight C++ and have wrappy take care of generating the Python C interface. Wrappy doesn't use CXX in the generated code, but it might in the future. If you wish to download wrappy, the url is: http://www.cgl.ucsf.edu/otf/wrappy/ The wrappy talk and associated paper from the 1999 O'Reilly Open Source Conference is available from: http://conferences.oreilly.com/cd/python/presentations/gcouch/ or: http://www.cgl.ucsf.edu/home/gregc/oss99/ --- Greg Couch UCSF Computer Graphics Labortory gregc(a)cgl.ucsf.edu

1 0

CXX at Sourceforge
by Paul F. Dubois March 11, 2000

March 11, 2000

Gentleman, I now have CXX-4.2 up at SourceForge. The web page is cxx.sourceforge.net. This leads you to the project page where you can get at the CVS tree. I am looking for one or more of you to take over as lead on this open source project since I do not use CXX in my work since I changed jobs last year. It is better to have someone who has to live with the consequences of their actions. 4.2 has Barry's extension stuff. I think there isn't a great deal of controversy about the CXX_Objects part, and Daniel has some fresh stuff to contribute. But I would think a good first step is to get what is there out by anonymous cvs or by downloading the release and see what changes we need for compilation on various platforms. Once we have a stable and portable base we can move on to the interesting questions about CXX_Extension. I have sent this to each of you because I know that you use, have used, or have developed variants on, CXX. Perhaps there are more people who could contribute, so I will cc this to the c++ sig. There are no ends of ideas and styles being put forward. Rather than just talking about the way each of you sees it, I would like to focus things by looking at what is there now and seeing if we understand exactly its strengths and weaknesses. Opinions about whether something will or won't compile or bloat or act badly should be backed up by actual data, not just speculation. Maybe that way we can keep down the noise level. Excuse this previous paragraph but several people have complained to me about the signal to noise ratio in our discussions and I think it is important to keep things focused. Best regards, Paul

4 3

An introduction
by Daryl V. McDaniel March 10, 2000

March 10, 2000

Hello, I would like to take this opportunity to introduce myself and briefly describe the project I am currently working on and its relevance to the C++ SIG. A quick summary of my idealized goals is to achieve seamless, bi-directional, integration between the C++ and Python portions of our product allowing Python to be used as the product's scripting language and as a prototyping language during development. And, to do this in such a way that my programmers don't have to write special interface code. Now for some background. I began programming in 1968 in SOL and Fortran. LISP, Cobol, and several flavors of assembler followed in the '70s. Somewhere around 1978 I discovered C and ended up writing a C compiler for the National 16032 processor family sometime around 1983. In the late '80s I found out about C++ and began using it professionally in 1990. About 2 months before Python 1.5.2 was released, I began looking for a scripting language to include in our product. I started the search as a TCL advocate but ended up selecting Python, primarily because of its power and usability by non-programmers. Python's modularity, embedding, and extension features were also prime decision points. The product I am responsible for is actually just a small part of our client's complete product. Its target environment is a distributed system of 1 to 64 nodes acting as a federation of autonomous intelligent agents. Each node consists of an embedded processor running VxWorks with Python and the C++ application. There is no disk, no virtual memory, and only a small amount of RAM (32MB). This is a large, fast track project (which one isn't these days?) that had, in previous generations, been implemented entirely in C++. The new incarnation is a 90% rewrite due to a vastly different operating environment and system requirements. We essentially have to deliver about 800,000 new lines of working and tested code by December of this year. The scripting component of the product must allow non-programmer users (Manufacturing and Service technicians), as well as software-savvy engineers, to write scripts to automate operations, debug systems, or modify operational procedures. In order to do this, the Python components need access to the low-level C++ implementation. [So far, I don't think I have said anything really new to anyone, its just background] I read the section on embedding and extending in the Python documentation as well as the help and hints articles on python.org. This led me to some of the wrapper packages and eventually to CXX, scxx, SWIG, sip, and the Extension Classes in Zope. Of all of these, I like CXX best. SWIG would meet all of my requirements except that the current "stable" version doesn't handle C++ well enough. I haven't had time to reverse engineer the makefiles so that I could build one of the newer versions -- that is forthcoming. So, I am currently hoping that between CXX and some documentation I have yet to write, we will be able to write and wrap our C++ code such that the C++ objects are available in Python, as true Python classes, and Pure Python functions and scripts will be available to the C++ code. This is rapidly becoming more important as we begin prototyping in Python. It now looks like the code we have to deliver in April will be mostly Python. The July release to engineering will still be about 80% Python. So, you can see that Python is taking on a larger role than "just an embedded scripting language". What I currently believe would be "perfect" would be a merge of SWIG and CXX. I am not going to be able to do this and won't suggest it be done. Both SWIG and CXX are fine on their own rights and I don't want to start any wars. CXX has taught me that I don't know C++, especially templates, as well as I should. It is providing me with a very practical example to learn from. It is also allowing my team to "write Python in C++". The synergy is producing much more powerful software with less code. Now we get to my possible contributions to Python and the SIG. At a minimum, I believe the documentation I produce will be useful to anyone using CXX. I may be able to expand upon the current work in CXX to make it easier to produce extension modules and classes. I, of course, would make any CXX extensions I produce available to the community. I hope that what I produce will be able to benefit the Python community. To that end, I will probably be asking lots of questions. Disclaimer: I can't guarantee that the client's legal department agrees with any of my views. They are currently examining the Python, Pyro, CXX, IDLE, JPython, and SWIG licenses to see if they will allow us to use any of them in a shippable product. 8-( Sincerely, -------- Daryl V. McDaniel darylm(a)python.mnet.com Micronetics (503) 621-0234 Aloha Research Group Portland, Oregon USA "Aloha Research Group", "Mnet", and "mnet" are trademarks of Micronetics

1 0

CXX at SourceForge
by Paul F. Dubois March 3, 2000

March 3, 2000

I have CXX under CVS at sourceforge: see http://cxx.sourceforge.net. This is version 4 which contains Barry Scott's rewrite of the extension module/object portion. It is ill tested and less ported. It isn't backward compatible in this area, beware. I will make a downloadable release next week but for now you can get it by CVS.

1 0

Reimplementing Lib/codeop.py in C fails...
by Alex Farber March 1, 2000

March 1, 2000

Dear colleagues, I am trying to create a custom "readline" function in C. I have looked at Modules/readline.c, Python/pythonrun.c and Parser/myreadline.c. I've also read the http://www.python.org/doc/current/api/api.html and many postings in Deja.com... I would very appreciate any hints. Here is a simple C code reimplementing Lib/codeop.py for test purposes: int main(int argc, char* argv[]) { char* line; PyObject *globals, *source; Py_Initialize(); globals = PyDict_New(); PyDict_SetItemString(globals, "__builtins__", PyEval_GetBuiltins()); /* while (1) { line = readline(">>> "); */ printf("\nCompiling <for i in [1,2,3]:>\n"); source = Py_CompileString("for i in [1,2,3]:", "<stdin>", Py_file_input); if (source == NULL) PyErr_Print(); printf("\nCompiling <for i in [1,2,3]:\\n>\n"); source = Py_CompileString("for i in [1,2,3]:\n", "<stdin>", Py_file_input); if (source == NULL) PyErr_Print(); printf("\nCompiling <for i in [1,2,3]:\\n\\n>\n"); source = Py_CompileString("for i in [1,2,3]:\n\n", "<stdin>", Py_file_input); if (source == NULL) PyErr_Print(); /* free(line); } */ Py_DECREF(globals); Py_Finalize(); exit(0); } Which produces the following output: Compiling <for i in [1,2,3]:> File "<string>", line 1 for i in [1,2,3]: ^ SyntaxError: unexpected EOF while parsing Compiling <for i in [1,2,3]:\n> File "<string>", line 1 for i in [1,2,3]: ^ SyntaxError: unexpected EOF while parsing Compiling <for i in [1,2,3]:\n\n> File "<string>", line 2 ^ SyntaxError: invalid syntax However the comment in Lib/codeop.py suggests, that the errors in the first and the second case should be different. How can I see this difference in my C code? Shouldn't the second string compile? Regards Alex PS: The background is that I am programming a 3D-viewer in C++ which uses some finite elements library (written in my university; not thread-safe), Qt (not thread-safe) and which embeds Python. Because of the not-thread-safeness I have to run FE, Qt and Python in the same thread. But then I can not run PyRun_InteractiveLoop() or PyRun_InteractiveOne() - the Qt would "freeze". So I am running a readline in another thread and send received strings to a UNIX-pipe. Those strings should be read by the first thread and executed with PyRun_SimpleString(). Now I get the problem with the strings like "for i in [1,2,3]:" - the Python needs more input. To detect such cases I was hoping to mimic the Lib/codeop.py and then either save the received string (if more input is needed) or run it with PyEval_EvalCode().

1 0