[C++-sig] Py++ wrapper generation and docstrings
Ganssauge, Gottfried
Gottfried.Ganssauge at haufe.de
Sun Sep 3 16:07:14 CEST 2006
> -----Original Message-----
> From: Roman Yakovenko [mailto:roman.yakovenko at gmail.com]
> Sent: Sunday, September 03, 2006 2:49 PM
> To: Ganssauge, Gottfried
> Subject: Re: Py++ wrapper generation and docstrings
>
> On 9/3/06, Ganssauge, Gottfried <Gottfried.Ganssauge at haufe.de> wrote:
> > Hi Roman,
> >
> > I'm still working on my API with Py++ and there are a few bits left:
>
> Is this because of Py++ or you have pretty big project :-)?
Neither nor I guess.
The project is quite old, it's got a C-API and the API is highly
irregular :-(
>
> > 1.) my API contains several functions expecting pointers to callback
> > functions.
> > Those callbacks get an instance pointer as one of their
> arguments
> > which has to be supplied as well.
> > An example:
> > typedef int (*callback_t) (void *instance);
> >
> > int search (void *instance, callback_t callback);
> >
> > I have written a generator which generates a wrapper like this:
> >
> > int wrap_callback_t (void *instance) {
>
> You can use boost::python::object
No I can't :-(
Gccxml botches on the bpl headers:
$ cat x.hpp
# include <boost/python.hpp>
boost::python::object test ();
$ gccxml -I ../../../../../../FremdProdukte/boost2 -I
/usr/include/python2.4 x.hpp -fxml=test.xml
../../../../../../FremdProdukte/boost2/boost/python/object_core.hpp:285:
sorry, unimplemented: call_expr
cannot be mangled due to a defect in the C++ ABI
This problem has been reported on the gccxml list already ...
>
> > try {
> > return (boost::python::call<int>
> > (reinterpret_cast<PyObject *> (instance)));
> > }
> >
> > catch (...) {
> > return (-1); // python
> exceptions must not
> > disrupt program flow
> > }
> > }
> >
> > int search_wrapper (PyObject *p) {
> > return (search (p, &wrap_callback_t));
> > }
> >
> > This works quite well, but ...
> > How do I make this generator interact with generators already
> > supplied by Py++
> > For example:
> > you're planning to support automatic conversion of
> pointers to
> > C++ array arguments from python sequences
>
> May be the next article will provide you with the answer.
> https://realityforge.vrsource.org/view/PyppApi/CodeInserter
>
> I and Matthias work on this feature this days. Comments and test cases
> are welcome :-)
I'll have a look at it.
>
> > Now suppose the following:
> > type char array_t [7];
> >
> > int search2 (void *instance, callback_t cb,
> array_t *result);
>
> If you already create a wrapper, you can create "better" wrapper:
>
> boost::python::tuple search2( void* instance, callback_t cb ){
> ...
> return boost::python::make_tuple( .... )
> }
That's what I'd like to achieve - but automatically!
>
> > How would both our generators interact with each other?
>
> I don't see the whole picture, so I can not answer your question.
> I assume that you have some functions that take pointer to
> callback, right?
> You can not expose these functions as is, so you have to
> write a wrapper for
> them. Basically you should exclude the function from being exported,
> and then you
> should add your declaration and registration code. Next link should
> help you with
> your task:
>
>
http://www.language-binding.net/pyplusplus/documentation/inserting_code.
html
>
>
> > Finally I'm not sure about when to run my generator.
>
> In your case you may consider to run it twice:
>
> 1. first time you can generate your own header file. This file will
> contain all wrappers.
> 2. second time you weel feed just generated header fileand original
> sources to Py++.
>
> > Currently I'm running it on every declaration
> expecting callback-
> > and instance pointers.
> > The declarations are then modified like this (f is a
> declaration
> > for "search"):
> > pyobject_type = pointer_t (dummy_type_t ("PyObject"))):
> > wrapperargs = [ argument_t (name = "instance", type =
> > pyobject_type) ]
> > function = f.name
> > f.name = "%s_wrapper" % function
> > f.arguments = wrapperargs
> > f.exportable = None # redo the check for exportability
> > # rename f if this is the first wrapper
> > if not f.alias or f.name == f.alias:
> > f.rename (function)
> >
> > Does this look reasonable? (at least it works :-0)
>
> No. Too much work and too much details. Consider the way I described.
That was my first attempt, but it became difficult to chain several
generators together.
Eventually I should really consider doing several generator passes to a
temp file until the temp file doesn't change anymore.
This could take quite some time because gccxml must parse the
intermediate files...
Furthermore I'm not yet sure how to forward the original function names
...
>
> > I got another generator which generates wrappers for functions
> > expecting "const unsigned char *" arguments for strings ...
> >
> > 2.) docstring extraction is not really usable at the moment
> > consider the attached test case
> > [test1.hpp, pypp_test1.py]
> >
> > When using the "latin1_doc_string" doc_extractor some unicode
> > exception is thrown.
>
> I will take a look on this. I did not thought about unicodes :-(. It
> will take some time to
> convert Py++ to support it. I think, that doc_extractor will change,
> because you will
> have to provide encoding. Otherwise, I will not be able to
> write code to files.
This could be a property on the doc_extractor.
> Also quick work around exists:
>
> my_module
> class_< X >( "X" )
> .def( "do_smth", ... )
> ;
>
> import my_module
> my_module.X.__doc__ = ....
> my_module.X.do_smth.__doc__ = ...
>
> This will work and it is simple to generate it.
Ok.
>
> > When using the "ascii_doc_string" doc_extractor invalid code is
> > generated
> > Only when the doc extractor returns a string which is
> conforming to
> > C++ syntax everything runs smooth.
>
> This is how it was created. See explanation later.
>
> > Pragmatically I would request that doc extractors must return
> > docstrings conforming to C++ syntax.
> > This means you wouldn't need to change Py++ - just the
> > documentation.
> > You could supply some function which makes C++-syntax
> from a normal
> > string like
> > def make_cstring (s):
> > return ('"%s"' % (repr (s) [1:-1]))
>
> It is not as simple as you think :-). You have to escape " in the
> string and you have to do
> it in a smart way. That is why I leaved this to user. If you want we
> can work together
> to improve the situation. I will make Py++ to work with unicode and
> you will contribute
> better make_cstring function.
Currently I'm using
string_escapes = re.compile (r'((\\x[a-f0-9][a-f0-9])|(\\*"))',
re.I)
def make_cstring (s):
def replace_escape(m):
g = m.group(1)
if g.startswith ('\\x'):
return g + '""'
elif g == '"':
return '\\"'
else:
return g
return '"%s"' % hex_escape.sub (replace_escape, repr (s) [1:-1])
This copes with the problem of hex escapes followed by valid hex
characters and with embedded quotes.
>
>
> Can I ask you something?
>
> Could we switch to one of the mailing lists: pygccxml of boost.python?
> I'd like my answers
> to be available to other users too.
at your request :-)
>
> It seems that you found work around to "pointer to function" problem.
> Can you create
> small example with explanation how it should be done. May be I will
> able to introduce
> this functionality to Py++.
Well not for any case.
Function pointers without instance pointers still can't be exposed for
the reasons given in the reference provided by Py++.
Function pointers _with_ instance pointers are a different kind of
beast.
Generally function declarations using this type of callback have exactly
one function pointer argument and exactly one void *argument which is
forwarded to the callback function.
typedef callback_return_type (*callback_t) (callback_args, void
*instance, more callback_args);
return_type function (args ..., callback_t, more args, void
*instance, more args)
Functions having this kind of signature may be wrapped like this:
callback_return_type wrap_callback_t (callback_args, void
*instance, more callback_args) {
PyObject *py_instance = reinterpret_cast<PyObject *>
(instance);
if (py_instance) {
try {
return
(boost::python::call<callback_return_type> (py_instance, callback_args,
more callback_args))
}
except (...) {
}
}
return some error value;
}
return_type function_wrapper (PyObject *instance, args,
more_args, ...) {
return (function (args ..., &wrap_callback_t, more args,
instance, more args));
}
I'm attaching the generators module I'm currently using ...
The generators are called like this:
def wrap_functions (self, decls):
"""
Generate wrappers for functions otherwise not exportable in
their
current_form
@param decls
Function declarations to be wrapped
"""
wrapper_generators = [
callback_wrapper_generator (self),
const_byte_wrapper_generator (self)
]
finished = False
while not finished:
finished = True
for d in decls:
for g in wrapper_generators:
cookie = g.applies (d)
if cookie:
g (cookie)
finished = False
break
>
> Thanks.
>
> --
> Roman Yakovenko
> C++ Python language binding
> http://www.language-binding.net/
>
Cheers,
Gottfried
-------------- next part --------------
A non-text attachment was scrubbed...
Name: generators.py
Type: application/octet-stream
Size: 9682 bytes
Desc: generators.py
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20060903/d49c6c54/attachment.obj>
More information about the Cplusplus-sig
mailing list