C functions that modify their arguments.
I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide some public good at the same time :) The library is proprietary and I can't change it. Many of its functions modify a pointer provided via an address arg. Can python deal with functions that modify their arguments? If so (generally) how?
If there is no formalised way I'm may have to consider registering some sort of callback to alter the argument from within C... kind of awkward but since I'm code generating all the C so long as I have a workable approach it doesn't matter how much code it takes :)
Thanks... wcn
Wendell Nichols <wcn00@shaw.ca> writes:
I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide some public good at the same time :) The library is proprietary and I can't change it. Many of its functions modify a pointer provided via an address arg. Can python deal with functions that modify their arguments? If so (generally) how?
I assume you're referring to the use of pointers to simulate pass-by-reference:
void update_sample(Type1 **arg1, Type2 **arg2) { *arg1 = new_type1(); *arg2 = new_type2(); ... }
The standard way of implementing such out parameters in Python is to have the function return a tuple of values. The function is then called like this:
val1, val2 = sample.update(val1, val2)
Thanks, that looks like it will work just fine! The simples answers are the best :) wcn
On 12-02-28 03:30 AM, Hrvoje Niksic wrote:
Wendell Nichols<wcn00@shaw.ca> writes:
I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide some public good at the same time :) The library is proprietary and I can't change it. Many of its functions modify a pointer provided via an address arg. Can python deal with functions that modify their arguments? If so (generally) how? I assume you're referring to the use of pointers to simulate pass-by-reference:
void update_sample(Type1 **arg1, Type2 **arg2) { *arg1 = new_type1(); *arg2 = new_type2(); ... }
The standard way of implementing such out parameters in Python is to have the function return a tuple of values. The function is then called like this:
val1, val2 = sample.update(val1, val2)
On Feb 27, 2012, at 2:48 PM, Wendell Nichols wrote:
I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide some public good at the same time :) The library is proprietary and I can't change it. Many of its functions modify a pointer provided via an address arg. Can python deal with functions that modify their arguments? If so (generally) how?
If there is no formalised way I'm may have to consider registering some sort of callback to alter the argument from within C... kind of awkward but since I'm code generating all the C so long as I have a workable approach it doesn't matter how much code it takes :)
Hi Wendell, IMO you have to be more specific about what the library does. If the library isn't Python-aware (which I guess it isn't, otherwise you wouldn't be wrapping it), then any pointers it returns won't point to Python objects. You can return those pointers to Python, but Python will just see a block of raw memory, not Python objects. This is usually not so useful.
Personally, I'm confused about what you're trying to do.
Cheers Philip
Wendell Nichols wrote:
I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide some public good at the same time :) The library is proprietary and I can't change it. Many of its functions modify a pointer provided via an address arg. Can python deal with functions that modify their arguments? If so (generally) how?
If there is no formalised way I'm may have to consider registering some sort of callback to alter the argument from within C... kind of awkward but since I'm code generating all the C so long as I have a workable approach it doesn't matter how much code it takes :)
You mean: The C functions write data into buffers your Python wrapper has to provide ?
That's possible, but you'll have to manage memory allocation in your wrapper. Alternatively, you could copy the data into Python objects to have Python take over memory management.
PS: You might want to consider using a wrapper generator such as SWIG, Boost or sip to help you with the wrapping of the library. They typically take care of all this.
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Feb 28 2012)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2012-02-13: Released eGenix pyOpenSSL 0.13 http://egenix.com/go26 2012-02-09: Released mxODBC.Zope.DA 2.0.2 http://egenix.com/go25 2012-02-06: Released eGenix mx Base 3.2.3 http://egenix.com/go24
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
I'm code generating c code based on the library's headers using python itself. half the point of this exercise was to learn python :) Turns out that I can detect arguments that are to be modified and return them as the second item in a tuple. so: int = func(&tobechanged,char*,char*)
Can become a python signature like
(int,int) = func(str,str)
If this works as well for all the functions as it has for the first few then I'm off to the races. The argument that gets modified internally by the lib all have a particular naming convention, and are always the first arg, and are always pointers to void*. That makes detecting them easy and changing the generated code equally as easy.
Thanks for all your interest and assistance Wendell Nichols
On 12-02-28 07:59 AM, M.-A. Lemburg wrote:
Wendell Nichols wrote:
I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide some public good at the same time :) The library is proprietary and I can't change it. Many of its functions modify a pointer provided via an address arg. Can python deal with functions that modify their arguments? If so (generally) how?
If there is no formalised way I'm may have to consider registering some sort of callback to alter the argument from within C... kind of awkward but since I'm code generating all the C so long as I have a workable approach it doesn't matter how much code it takes :) You mean: The C functions write data into buffers your Python wrapper has to provide ?
That's possible, but you'll have to manage memory allocation in your wrapper. Alternatively, you could copy the data into Python objects to have Python take over memory management.
PS: You might want to consider using a wrapper generator such as SWIG, Boost or sip to help you with the wrapping of the library. They typically take care of all this.
Wendell Nichols wrote:
I'm code generating c code based on the library's headers using python itself. half the point of this exercise was to learn python :) Turns out that I can detect arguments that are to be modified and return them as the second item in a tuple. so: int = func(&tobechanged,char*,char*)
Can become a python signature like
(int,int) = func(str,str)
Right, that's the standard Python approach to output variables. If you have in/out variables, you'd use:
(int,int) = func(int,str,str)
If this works as well for all the functions as it has for the first few then I'm off to the races. The argument that gets modified internally by the lib all have a particular naming convention, and are always the first arg, and are always pointers to void*. That makes detecting them easy and changing the generated code equally as easy.
Thanks for all your interest and assistance Wendell Nichols
On 12-02-28 07:59 AM, M.-A. Lemburg wrote:
Wendell Nichols wrote:
I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide some public good at the same time :) The library is proprietary and I can't change it. Many of its functions modify a pointer provided via an address arg. Can python deal with functions that modify their arguments? If so (generally) how?
If there is no formalised way I'm may have to consider registering some sort of callback to alter the argument from within C... kind of awkward but since I'm code generating all the C so long as I have a workable approach it doesn't matter how much code it takes :) You mean: The C functions write data into buffers your Python wrapper has to provide ?
That's possible, but you'll have to manage memory allocation in your wrapper. Alternatively, you could copy the data into Python objects to have Python take over memory management.
PS: You might want to consider using a wrapper generator such as SWIG, Boost or sip to help you with the wrapping of the library. They typically take care of all this.
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Feb 28 2012)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2012-02-13: Released eGenix pyOpenSSL 0.13 http://egenix.com/go26 2012-02-09: Released mxODBC.Zope.DA 2.0.2 http://egenix.com/go25 2012-02-06: Released eGenix mx Base 3.2.3 http://egenix.com/go24
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
Wendell Nichols, 27.02.2012 20:48:
I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide some public good at the same time :) The library is proprietary and I can't change it. Many of its functions modify a pointer provided via an address arg. Can python deal with functions that modify their arguments? If so (generally) how?
If there is no formalised way I'm may have to consider registering some sort of callback to alter the argument from within C... kind of awkward but since I'm code generating all the C so long as I have a workable approach it doesn't matter how much code it takes :)
Marc-Andre already pointed you to a couple of tools that do this for you. It's best to use an existing tool, because writing both a wrapping tool and then a wrapper with it means that you will end up having to maintain both the wrapping tool and the wrapper, whereas you'd normally only want to care about the wrapper in the long term.
Another (very) common tool to wrap C(++) libraries for Python is Cython. Specifically, it will allow you to not only write a thin 1:1 wrapper (as I guess you are doing here), but to actually write an efficient thicker wrapper that provides a more pythonic interface to the library you are wrapping. In most cases, this is largely preferable to a straight C-ish interface, especially when it comes to things like memory management for C data types or object oriented APIs.
Stefan
participants (5)
-
Hrvoje Niksic -
M.-A. Lemburg -
Philip Semanchuk -
Stefan Behnel -
Wendell Nichols