Mailman 3 March 2002 - Cplusplus-sig

Profiling object instantiation... a preliminary report
by Greg Landrum June 26, 2002

June 26, 2002

Re-establishing context: I have an extension class that I've wrapped using both Boost and Cxx. I need to be able to pickle and depickle large numbers (1e5-1e6) of these objects quickly for my target application (storing them in a ZODB and querying the resulting database). I discovered that simple instantiation of 1e5 objects took dramatically longer in Boost than in Cxx (6.7 seconds vs 0.9 seconds). I pulled down a copy of Intel's vtune profiler[1] and profiled my Boost and Cxx wrapped libraries to find the origin of the huge performance difference. After fiddling around for a while figuring out the appropriate magic to make vtune actually work, I found something surprising. I had exposed 4 different constructors for my Mol class. What I failed to realize is that when you overload functions (at least constructors) in BPL, this is handled using exceptions. This was at least partially responsible for the tremendous slow down in the Boost wrapper of my code. When I commented out some constructors, rebuilt, and reprofiled my code I got a run time of 1.9 seconds, only twice the Cxx run time. I'm a lot happier with this, but I'll continue to do a bit of exploration to see if I can bring that down more. I'm rather enjoying vtune now that I've figure out what can be done with it. If anyone wants more detailed information, I'd be happy to provide it. -greg [1] The beta of vtune is available as a free download for linux and windows. These results are under Win2K using MSVC++ v6. ---- greg Landrum (greglandrum(a)earthlink.net) Software Carpenter/Computational Chemist

6 12

Boost.Python V2: direct access to data members
by David Abrahams March 30, 2002

March 30, 2002

Direct access to data members is now implemented in Boost.Python v2, using the same def_readonly()/def_readwrite() interface as in v1. See libs/python/test/data_members.* for examples. -Dave +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams(a)rcn.com +---------------------------------------------------------------+

1 0

Boost.Python v2: implicit conversion support
by David Abrahams March 29, 2002

March 29, 2002

Support for C++ implicit conversions has now been added to the v2 codebase. Please see libs/python/test/implicit.* for examples. The facility looks like: boost::python::implicitly_convertible<SourceType,DestType>(); which declares that SourceType is implicitly convertible to DestType. This declaration allows a Python object which can be converted to SourceType to be passed to a wrapped C++ function in place of a DestType or DestType const& parameter. -Dave +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams(a)rcn.com +---------------------------------------------------------------+

3 3

newbie boost questions
by Dean Goodmanson March 29, 2002

March 29, 2002

Greetings, The boost.org site (http://www.boost.org/libs/python/doc/) leaves me lost on how to download & install boost's Python wrapping tools. Am I missing something, or is this not the site to start with... Can you recommend some references for using Python as a C++ developer's testing ground? I apologize for my idiosyncrosies..python gets my reserve brain power. I wouldn't have asked except that these issues may have some timely relevance. (About me: Visual C++ 6 & 7, and Mac CodeWarrior C++ application developer. Python lurker & hopeful.) -Dean See Dave Matthews Band live or win a signed guitar http://r.lycos.com/r/bmgfly_mail_dmb/http://win.ipromotions.com/lycos_02020…

2 1

Boost.Python V2: Virtual function support/changes to class_<> interface
by David Abrahams March 25, 2002

March 25, 2002

Virtual function support is now implemented, including support for abstract base classes. See libs/python/test/virtual_functions.* in the CVS for examples. Some (mostly-backward-compatible) changes were made to the class_<T,...> interface: * The order of the optional arguments is arbitrary. The meaning of the arguments is deduced by the library * You can't use an arbitrary MPL sequence to specify base classes anymore. Instead, you must use an instantiation of boost::python::bases<B1,B2...>. * To expose a class T with no publicly-available copy constructor (e.g. an abstract base class), pass boost::noncopyable (from <boost/utility.hpp>) as one of the optional arguments. You can also use this option if you intend never to wrap a function returning T objects by-value or by const reference. * To specify how a C++ instance is held by a Python class instance, you no longer pass a generator as a template argument to class_<>. The rules are: 1. if you don't specify, or you pass T as an optional argument, the instance is held by-value 2. if you pass a class U derived from T as an optional argument, the actual type of the C++ object held is U. U's constructor(s) must accept an initial additional PyObject* which stores a back-reference to the owning Python object. This is the old familiar virtual function callback interface. 3. If you pass a smart pointer to T as an optional argument, the T instance is held by one of these smart pointers 3a. The traits class boost::python::detail::pointee<Ptr> is used to extract the element_type of the smart pointer. This works by default with any smart pointer that defines an element_type (boost::shared_ptr, std::auto_ptr, etc), but may need to be specialized for other smart pointers. 4. If you pass a smart pointer to U derived from T as an optional argument, the actual type of the C++ object is U. Similar constraints apply as in case 2. In cases 1 and 3, you can optionally specify that T itself gets initialized with an optional, hidden back-reference argument by specializing the boost::python::has_back_reference<T> traits class: struct MyType { MyType(PyObject* self_, char* name_) : self(self_), name(name_) {} PyObject* self; std::string name; }; namespace boost { namespace python { template <class MyType> struct has_back_reference { BOOST_STATIC_CONSTANT(bool, value = true); }; }} ... .add( class<MyType> .def_init(args<char*>()) ) ... Some people want this information stored in the class instance in order to do special reference-counting tricks, etc. +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams(a)rcn.com +---------------------------------------------------------------+

2 2

Re: runtime error with Intel compiler 6.0 beta under windows
by Peter Bienstman March 17, 2002

March 17, 2002

> From: "David Abrahams" <david.abrahams(a)rcn.com> > To: <c++-sig(a)python.org> > Subject: Re: [C++-sig] runtime error with Intel compiler 6.0 > beta under windows > Date: Sat, 16 Mar 2002 22:22:44 -0500 > Reply-To: c++-sig(a)python.org > > This has nothing to do with C++ exceptions. It's Python > telling you that some C/C++ function it called returned a > NULL PyObject* without first calling PyErr_SetString() or one > of the other related Python error reporting function. Whether > or not this has anything to do with Boost.Python I cannot > say, but if the behavior only happens with the Intel compiler > I'd be inclined to think it's a compiler bug. > > HTH, > Dave For what it's worth, the problem also occurs with some of the Boost.Python testcases (e.g. abstract.cpp). The simple workaround is to compile boost_bython.lib with the Microsoft compiler, and then the extensions themselves with the Intel compiler. Unfortunately, I don't have the knowledge of the inner workings of Boost.Python to distill a small testcase for Intel tech support. Peter

2 1

runtime error with Intel compiler 6.0 beta under windows
by Peter Bienstman March 17, 2002

March 17, 2002

Hi, I recently converted my big project from gcc under windows to the beta of the intel 6.0 compiler. Compiling goes without problems, but now all my python scripts fail with: SystemError: error return without exception set. I did compile boost V1 and my C++ extensions with exception support (/GX). Any ideas?

2 1

Visual C++ 6 error
by Ralf W. Grosse-Kunstleve March 13, 2002

March 13, 2002

Using the boost cvs snapshot from a few hours ago I am experiencing problems with Visual C++ 6: cl.exe /nologo /MD /GR /GX /Zm350 -I"r:\cctbx\include" -I"r:\boost" -I"c:\Python21\include" -c uctbxmodule.cpp r:\boost\boost/python/detail/init_function.hpp(178) : error C2899: typename cannot be used outside a template declaration This is the code from init_function.hpp: template <class A1> static init* create(signature1<A1>) { return new init1<T, typename detail::parameter_traits<A1>::const_reference>; } I think this will work with VC6 (and any other compiler): template <class A1> static init* create(signature1<A1>) { typedef typename detail::parameter_traits<A1>::const_reference cr; return new init1<T, cr>; } Ralf __________________________________________________ Do You Yahoo!? Try FREE Yahoo! Mail - the world's greatest free email! http://mail.yahoo.com/

2 1

V2 Tests have moved
by David Abrahams March 8, 2002

March 8, 2002

FYI: The Jamfile for tests of Boost.Python v2 has moved from libs/python to libs/python/test. -Dave +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams(a)rcn.com +---------------------------------------------------------------+

1 0

Callbacks and virtual functions for Boost.Python v2
by David Abrahams March 5, 2002

March 5, 2002

Here's the plan: I aim to provide an interface similar to that of Boost.Python v1's callback<>::call(...) for dealing with callbacks. The interface will look like: returning<ResultType>::call("method_name", self_object, a1, a2...); or returning<ResultType>::call(callable_object, a1, a2...); ARGUMENT HANDLING There is an issue concerning how to make Python objects from the arguments a1...aN. A new Python object must be created; should the C++ object be copied into that Python object, or should the Python object simply hold a reference/pointer to the C++ object? In general, the latter approach is unsafe, since the called function may store a reference to the Python object somewhere. If the Python object is used after the C++ object is destroyed, we'll crash Python. I plan to make the copying behavior the default, and to allow a non-copying behavior if the user writes boost::ref(a1) instead of a1 directly. At least this way, the user doesn't get dangerous behavior "by accident". It's also worth noting that the non-copying ("by-reference") behavior is in general only available for class types, and will fail at runtime with a Python exception if used otherwise** However, pointer types present a problem: My first thought is to refuse to compile if any aN has pointer type: after all, a user can always pass *aN to pass "by-value" or ref(*aN) to indicate a pass-by-reference behavior. However, this creates a problem for the expected NULL pointer => None conversion: it's illegal to dereference a null pointer value. We could use another construct, say "ptr(aN)", to deal with null pointers, but then what does it mean? We know what it does when aN is NULL, but it might either have by-value or by-reference behavior when aN is non-null. A reasonable compromise might be to give ptr(aN) reference behavior, and to ask people to write an expression such as aN ? to_python_value(*aN) : py::none() if they want the copying behavior. Of course, we'll want copying behavior (into a Python String object) by default when aN is char const*, but this exception should be easy enough to implement. I'm interested in hearing opinions about this compromise approach. RESULT HANDLING As for results, we have a similar problem: if ResultType is allowed to be a pointer or reference type, the lifetime of the object it refers to is probably being managed by a Python object. When that Python object is destroyed, our pointer dangles. The problem is particularly bad when the ResultType is char const* - the corresponding Python String object is typically uniquely-referenced, meaning that the pointer dangles as soon as returning<char const*>::call() returns. Boost.Python v1 deals with this issue by refusing to compile any uses of callback<char const*>::call(), but IMO this goes both too far and not far enough. It goes too far because there are cases where the owning String object survives beyond the call (just for instance when it's the name of a Python class), and it goes not far enough because we might just as well have the same problem with any returned pointer or reference. I propose to address this in Boost.Python v2 by 1. lifting the compile-time restriction on const char* callback returns 2. detecting the case when the reference count on the result Python object is 1 and throwing an exception inside of returning<U>::call() when U is a pointer or reference type. I think this is acceptably safe because users have to explicitly specify a pointer/reference for U in returning<U>, and they will be protected against dangles at runtime, at least long enough to get out of the returning<U>::call() invocation. -Dave could reference an object which is going to be destroyed **It would be possible to make it fail at compile-time for non-class types such as int and char, but I'm not sure it's a good idea to impose this restriction yet. +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams(a)rcn.com +---------------------------------------------------------------+

1 0