Mailman 3 September 2006 - Cplusplus-sig

[Py++] First imressions
by Kirill Lapshin Sept. 12, 2006

Sept. 12, 2006

Hello, Recently I was charged with exposing our project to Python, and naturally went to Boost.Python documentation. After digging a bit, learning, etc, I decided to give some code generator a try -- so the choice was Pyste or Py++. I've decided to try the latter one first, since it seems to have quite powerful query language, and was offering a promise of doing certain operations in bulk. Another reason was that Py++ is actively maintained. Long story short, I dived in, and I haven't look back. I am quite happy with progress, albeit the initial development is not finished yet, so it is not a success story yet, but I am quite confident it will become one quite soon. I would like to thank Roman for his hard work on Py++ and for being extremely helpful in our private e-mail discussion. Just for the record I would like to post some of the problems I had and the way they were resolved. 1. The way overrides are generated does not compile on MSVC8 (2005). Looks like the problem is in conversion of return type if return type is bool or enum. For example virtual bool is_representative( ) const { bp::override func_is_representative = this->get_override( "is_representative" ); return func_is_representative( ); // errors out } error C2440: 'return' : cannot convert from 'boost::python::detail::method_result' to 'bool' No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called Don't know why is this happening, but the good news, there is a workaround. One can use explicit conversion of return type, i.e. return func_is_representative()..as(&boost::type<T>()); If someone could verify this problem, I guess the solution would be to patch code creator. 2. How to disable function overriding. I don't really need ability to override virtual functions from Python anyway, and moreover using wrapper classes does not seem to work for me, since I am trying to expose abstract interfaces to python and hide concrete implementations (hence wrapper is abstract as well and can't be instantiated). Maybe there is still a way to expose it with wrapper, but I've decided not bother for now. Roman suggested following way to suppress overriding: from pygccxml import declarations members = cls.decls( declarations.virtuality_type_matcher( declarations.VIRTUALITY_TYPES.VIRTUAL ) , decl_type=pd.member_calldef_t , allow_empty=True) members.set_virtuality( declarations.VIRTUALITY_TYPES.NOT_VIRTUAL ) Note that Py++ differentiates between VIRTUAL and PURE_VIRTUAL, so above listed code does not cover pure virtual functions. 3. Observation: Py++ generates wrappers even if there is no virtual functions, but there is nested class/struct. Nested enums are Ok. 4. Gccxml is buggy, especially with respect to default arguments. See http://language-binding.net/pygccxml/design.html#patchers. In my case I had problems with default std::vector<T> arguments. Those can be worked around manually with mb = module_builder_t( ... ) Foo = mb.class_( 'Foo' ) Foo_constructor = Foo.constructor( select write constructor ) #help: http://language-binding.net/pygccxml/query_interface.html Foo_constructor.arguments[0].default_value = fixed default value 5. Precompiled header support is not implemented even though documentation claims it is. Roman says it is in works. For a meantime one can use following workaround: mb.code_creator.adopt_creator( code_creators.include_t(header='_pch.hpp'), 0) md.split_module(...) 6. shared_ptr<T> works out of the box, while shared_ptr<const T> does not. Roman gave me following pointers: http://boost.org/libs/python/doc/v2/register_ptr_to_python.html Also read this: http://language-binding.net/pygccxml/apidocs/pygccxml.declarations.type_tra… Now you can implement some general solution similar to http://python-ogre.python-hosting.com/file/trunk/python-ogre/shared_ptr.py I did not have time to try it yet, but planning to do it soon. Cheers, Kirill

2 1

Possible to inspect type of a boost::python::object
by anders langlands Sept. 12, 2006

Sept. 12, 2006

Currently I'm using extract<T>::check() over a bunch of different types to decide what to do with python objects. Is there any way to inspect the type directly? A

2 1

Patch for py++
by Ganssauge, Gottfried Sept. 11, 2006

Sept. 11, 2006

I had to make the following patch to pyplusplus\code_repository\array_1.py in order to make it compile using Visual-C++ 6.5: Index: array_1.py =================================================================== --- array_1.py (Revision 494) +++ array_1.py (Arbeitskopie) @@ -28,13 +28,14 @@ inline void raise_on_out_of_range( long unsigned int size, long unsigned int index ){ - if( index < 0 || size <= index ){ + if( size <= index ){ throw std::out_of_range("index out of range"); } } template< class TItemType, long unsigned int size > struct const_array_1_t{ + typedef boost::call_traits<const TItemType>::param_type ref; const_array_1_t( TItemType const * const data ) : m_data( data ){ @@ -47,8 +48,7 @@ return size; } - typename boost::call_traits<const TItemType>::param_type - item_ref( long unsigned int index ) const{ + ref item_ref( long unsigned int index ) const{ raise_on_out_of_range( size, index ); return m_data[index]; } @@ -61,6 +61,7 @@ template< class TItemType, long unsigned int size > struct array_1_t{ + typedef boost::call_traits<const TItemType>::param_type ref; array_1_t( TItemType * data ) : m_data( data ){ @@ -73,15 +74,14 @@ return size; } - typename boost::call_traits<const TItemType>::param_type - item_ref( long unsigned int index ) const{ + ref item_ref( long unsigned int index ) const{ raise_on_out_of_range( size, index ); return m_data[index]; } void set_item( long unsigned int index - , typename boost::call_traits<const TItemType>::param_type new_value ){ + , ref new_value ){ raise_on_out_of_range( size, index ); m_data[index] = new_value; } Cheers, Gottfried

2 2

Re: [C++-sig] array of strings in boost python
by David Abrahams Sept. 11, 2006

Sept. 11, 2006

The following message is a courtesy copy of an article that has been posted to gmane.comp.lib.boost.user as well. <ctrembla(a)pcigeomatics.com> writes: > Hi, > > newbie in python and boost::python Please post Boost.Python questions to the C++-sig: http://www.boost.org/more/mailing_lists.htm#cplussig > I have a method in c++ that returns a vector of strings, how can i > expose this method in python, and how should i use it in python Sorry to turn the question around on you, but how do you /want/ to be able to use it in Python? That will determine how you should expose it. You might look into http://boost.org/libs/python/doc/v2/indexing.html#vector_indexing_suite_cla… to see if it meets your needs. -- Dave Abrahams Boost Consulting www.boost-consulting.com

1 0

Bug in opaque_pointer_converter and patch
by Ganssauge, Gottfried Sept. 8, 2006

Sept. 8, 2006

While debugging a python extension I found out that a segmentation fault was produced by the python statement print type(some_opaque_pointer) After some further debugging I found out that all instances created from opaque_pointer_converter had a type object which itself didn't have a type, and this finally lead to the crash. Find attached a patch which corrects this behaviour by initialising the type object using PyType_Ready(). Cheers, Gottfried

1 0

[boost.python] Indexing suite problems with convertable element type
by Kirill Lapshin Sept. 7, 2006

Sept. 7, 2006

Hi, I am just making my first step in boost.python world, so please forgive me for potential FAQ, but I can't find an answer in docs and by googling. I have a custom date class, that I registered for conversion to Python's datetime.date object. Individual dates convert fine, but when I registered vector<date> with indexing suite, I have a problem that I can insert python date objects into vector, but can't extract them. E.g. >>> v = vector_day() >>> v.append( datetime.date(2006,1,1) ) >>> len(v) 1 >>> v[0] Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: No Python class registered for C++ class class day vector_day is exposed like this: bp::class_< std::vector<day, std::allocator<day> > >( "vector_day" ) .def( bp::vector_indexing_suite< ::std::vector<day, std::allocator<day> > >() ); As I said, individual date objects convert just fine back and forth, i.e. function day test_day( day x ) { return x+1; } exposed as typedef day ( *function_ptr_t )( day ); bp::def( "test_day", function_ptr_t( &test_day ), ( bp::arg("x") ), bp::default_call_policies() ); can be used from Python without problems: >>> test_day( datetime.date(2006,1,1) ) datetime.date(2006, 1, 2) What am I doing wrong? Regards, Kirill Lapshin

2 2

[Py++] Nested enum as default arg
by Kirill Lapshin Sept. 7, 2006

Sept. 7, 2006

Hi, Suppose I have struct Foo { enum Bar {BAR1, BAR2}; explicit Foo(Bar arg = BAR1); }; Py++ generates code that tries to add constructor (and default argument) prior to enum being registered. Result is exception in module import. Any ideas how to fix / work around? Other then moving enum outside the class? Cheers, Kirill

2 7

boost::python tutorial on overloading not up-to-date?
by Neal Becker Sept. 7, 2006

Sept. 7, 2006

In boost::python tutorial there is a whole section on manual overloading. AFAICT, overloading "just works" without my doing anything. Is this information out-of-date?

1 0

Re: [C++-sig] Py++ wrapper generation and docstrings
by Ganssauge, Gottfried Sept. 4, 2006

Sept. 4, 2006

> -----Original Message----- > From: Roman Yakovenko [mailto:roman.yakovenko@gmail.com] > Sent: Sunday, September 03, 2006 2:49 PM > To: Ganssauge, Gottfried > Subject: Re: Py++ wrapper generation and docstrings > > On 9/3/06, Ganssauge, Gottfried <Gottfried.Ganssauge(a)haufe.de> wrote: > > Hi Roman, > > > > I'm still working on my API with Py++ and there are a few bits left: > > Is this because of Py++ or you have pretty big project :-)? Neither nor I guess. The project is quite old, it's got a C-API and the API is highly irregular :-( > > > 1.) my API contains several functions expecting pointers to callback > > functions. > > Those callbacks get an instance pointer as one of their > arguments > > which has to be supplied as well. > > An example: > > typedef int (*callback_t) (void *instance); > > > > int search (void *instance, callback_t callback); > > > > I have written a generator which generates a wrapper like this: > > > > int wrap_callback_t (void *instance) { > > You can use boost::python::object No I can't :-( Gccxml botches on the bpl headers: $ cat x.hpp # include <boost/python.hpp> boost::python::object test (); $ gccxml -I ../../../../../../FremdProdukte/boost2 -I /usr/include/python2.4 x.hpp -fxml=test.xml ../../../../../../FremdProdukte/boost2/boost/python/object_core.hpp:285: sorry, unimplemented: call_expr cannot be mangled due to a defect in the C++ ABI This problem has been reported on the gccxml list already ... > > > try { > > return (boost::python::call<int> > > (reinterpret_cast<PyObject *> (instance))); > > } > > > > catch (...) { > > return (-1); // python > exceptions must not > > disrupt program flow > > } > > } > > > > int search_wrapper (PyObject *p) { > > return (search (p, &wrap_callback_t)); > > } > > > > This works quite well, but ... > > How do I make this generator interact with generators already > > supplied by Py++ > > For example: > > you're planning to support automatic conversion of > pointers to > > C++ array arguments from python sequences > > May be the next article will provide you with the answer. > https://realityforge.vrsource.org/view/PyppApi/CodeInserter > > I and Matthias work on this feature this days. Comments and test cases > are welcome :-) I'll have a look at it. > > > Now suppose the following: > > type char array_t [7]; > > > > int search2 (void *instance, callback_t cb, > array_t *result); > > If you already create a wrapper, you can create "better" wrapper: > > boost::python::tuple search2( void* instance, callback_t cb ){ > ... > return boost::python::make_tuple( .... ) > } That's what I'd like to achieve - but automatically! > > > How would both our generators interact with each other? > > I don't see the whole picture, so I can not answer your question. > I assume that you have some functions that take pointer to > callback, right? > You can not expose these functions as is, so you have to > write a wrapper for > them. Basically you should exclude the function from being exported, > and then you > should add your declaration and registration code. Next link should > help you with > your task: > > http://www.language-binding.net/pyplusplus/documentation/inserting_code. html > > > > Finally I'm not sure about when to run my generator. > > In your case you may consider to run it twice: > > 1. first time you can generate your own header file. This file will > contain all wrappers. > 2. second time you weel feed just generated header fileand original > sources to Py++. > > > Currently I'm running it on every declaration > expecting callback- > > and instance pointers. > > The declarations are then modified like this (f is a > declaration > > for "search"): > > pyobject_type = pointer_t (dummy_type_t ("PyObject"))): > > wrapperargs = [ argument_t (name = "instance", type = > > pyobject_type) ] > > function = f.name > > f.name = "%s_wrapper" % function > > f.arguments = wrapperargs > > f.exportable = None # redo the check for exportability > > # rename f if this is the first wrapper > > if not f.alias or f.name == f.alias: > > f.rename (function) > > > > Does this look reasonable? (at least it works :-0) > > No. Too much work and too much details. Consider the way I described. That was my first attempt, but it became difficult to chain several generators together. Eventually I should really consider doing several generator passes to a temp file until the temp file doesn't change anymore. This could take quite some time because gccxml must parse the intermediate files... Furthermore I'm not yet sure how to forward the original function names ... > > > I got another generator which generates wrappers for functions > > expecting "const unsigned char *" arguments for strings ... > > > > 2.) docstring extraction is not really usable at the moment > > consider the attached test case > > [test1.hpp, pypp_test1.py] > > > > When using the "latin1_doc_string" doc_extractor some unicode > > exception is thrown. > > I will take a look on this. I did not thought about unicodes :-(. It > will take some time to > convert Py++ to support it. I think, that doc_extractor will change, > because you will > have to provide encoding. Otherwise, I will not be able to > write code to files. This could be a property on the doc_extractor. > Also quick work around exists: > > my_module > class_< X >( "X" ) > .def( "do_smth", ... ) > ; > > import my_module > my_module.X.__doc__ = .... > my_module.X.do_smth.__doc__ = ... > > This will work and it is simple to generate it. Ok. > > > When using the "ascii_doc_string" doc_extractor invalid code is > > generated > > Only when the doc extractor returns a string which is > conforming to > > C++ syntax everything runs smooth. > > This is how it was created. See explanation later. > > > Pragmatically I would request that doc extractors must return > > docstrings conforming to C++ syntax. > > This means you wouldn't need to change Py++ - just the > > documentation. > > You could supply some function which makes C++-syntax > from a normal > > string like > > def make_cstring (s): > > return ('"%s"' % (repr (s) [1:-1])) > > It is not as simple as you think :-). You have to escape " in the > string and you have to do > it in a smart way. That is why I leaved this to user. If you want we > can work together > to improve the situation. I will make Py++ to work with unicode and > you will contribute > better make_cstring function. Currently I'm using string_escapes = re.compile (r'((\\x[a-f0-9][a-f0-9])|(\\*"))', re.I) def make_cstring (s): def replace_escape(m): g = m.group(1) if g.startswith ('\\x'): return g + '""' elif g == '"': return '\\"' else: return g return '"%s"' % hex_escape.sub (replace_escape, repr (s) [1:-1]) This copes with the problem of hex escapes followed by valid hex characters and with embedded quotes. > > > Can I ask you something? > > Could we switch to one of the mailing lists: pygccxml of boost.python? > I'd like my answers > to be available to other users too. at your request :-) > > It seems that you found work around to "pointer to function" problem. > Can you create > small example with explanation how it should be done. May be I will > able to introduce > this functionality to Py++. Well not for any case. Function pointers without instance pointers still can't be exposed for the reasons given in the reference provided by Py++. Function pointers _with_ instance pointers are a different kind of beast. Generally function declarations using this type of callback have exactly one function pointer argument and exactly one void *argument which is forwarded to the callback function. typedef callback_return_type (*callback_t) (callback_args, void *instance, more callback_args); return_type function (args ..., callback_t, more args, void *instance, more args) Functions having this kind of signature may be wrapped like this: callback_return_type wrap_callback_t (callback_args, void *instance, more callback_args) { PyObject *py_instance = reinterpret_cast<PyObject *> (instance); if (py_instance) { try { return (boost::python::call<callback_return_type> (py_instance, callback_args, more callback_args)) } except (...) { } } return some error value; } return_type function_wrapper (PyObject *instance, args, more_args, ...) { return (function (args ..., &wrap_callback_t, more args, instance, more args)); } I'm attaching the generators module I'm currently using ... The generators are called like this: def wrap_functions (self, decls): """ Generate wrappers for functions otherwise not exportable in their current_form @param decls Function declarations to be wrapped """ wrapper_generators = [ callback_wrapper_generator (self), const_byte_wrapper_generator (self) ] finished = False while not finished: finished = True for d in decls: for g in wrapper_generators: cookie = g.applies (d) if cookie: g (cookie) finished = False break > > Thanks. > > -- > Roman Yakovenko > C++ Python language binding > http://www.language-binding.net/ > Cheers, Gottfried

3 4

problem with tkinter gui running in different interpreters
by Andreas Sept. 3, 2006

Sept. 3, 2006

hello, I'm not sure if this is the right place for my question. I have a problem with tkinter running in different python interpreters. I need to separate different python scripts as much as possible. Therefor each script runs in its own interpreter. If only one of the scripts uses tkinter gui and runs the tkinter's mainloop in a created thread, everything is fine. But if more scripts uses this, the whole application with the created dialogs hangs. To solve this only the first script must call tkinter's mainloop(). But thats not an option, because the scripts are independent and must not have dialogs. Thus a script does not know if it is the first which uses tkinter. So the question is: Is it possible to detect from different interpreters whether the tkinter's mainloop is already running or not? Below is the running example code. In main() i want to start multiple dialogs calling startDialog(true), wich does not work. (I'm using boost 1.33.1, MSVC7.1 and Python 2.4.3) Any help would be appreciated, andreas rose void startDialog(bool bCallMainloop) { PyEval_AcquireLock(); PyThreadState* pThreadState = Py_NewInterpreter(); PyThreadState_Swap(pThreadState); char pBuff[] = "Test.exe"; char* Argv[1]; Argv[0] = pBuff; PySys_SetArgv(1, Argv); PyObject* pDict = PyDict_New(); PyDict_SetItemString(pDict, "__builtins__", PyEval_GetBuiltins()); std::string strExec; if (bCallMainloop) { strExec = "\ from threading import * \n\ from Tkinter import * \n\ \n\ class DialogThread(Thread): \n\ def run(self): \n\ label = Label(None, text='hello') \n\ label.pack() \n\ label.mainloop() \n\ \n\ DialogThread().start() \n\ "; } else { strExec = "\ from threading import * \n\ from Tkinter import * \n\ \n\ class DialogThread(Thread): \n\ def run(self): \n\ label = Label(None, text='hello') \n\ label.pack() \n\ \n\ DialogThread().start() \n\ "; } boost::python::handle<> result(PyRun_String(strExec.c_str(), Py_file_input, pDict, pDict)); result.reset(); PyEval_ReleaseThread(pThreadState); } int main(int argc, char** argv) { Py_Initialize(); PyEval_InitThreads(); PyThreadState* pyMainState = PyThreadState_Swap(NULL); PyEval_ReleaseLock(); startDialog(true); startDialog(false); //wait little time to enable user interaction with the dialogs Sleep(10000); PyEval_AcquireLock(); PyThreadState_Swap(pyMainState); Py_Finalize(); return 0; }

1 0