From wcn00 at shaw.ca  Mon Feb 27 20:48:06 2012
From: wcn00 at shaw.ca (Wendell Nichols)
Date: Mon, 27 Feb 2012 12:48:06 -0700
Subject: [capi-sig] C functions that modify their arguments.
Message-ID: <4F4BDDF6.7020602@shaw.ca>

I have a library that I want to wrap for Python (as part of an exercise 
to learn Python, and provide some public good at the same time :)  The 
library is proprietary and I can't change it.  Many of its functions 
modify a pointer provided via an address arg.  Can python deal with 
functions that modify their arguments?  If so (generally) how?

If there is no formalised way I'm may have to consider registering some 
sort of callback to alter the argument from within C... kind of awkward 
but since I'm code generating all the C so long as I have a workable 
approach it doesn't matter how much code it takes :)

Thanks...
wcn


From hniksic at xemacs.org  Tue Feb 28 11:30:04 2012
From: hniksic at xemacs.org (Hrvoje Niksic)
Date: Tue, 28 Feb 2012 11:30:04 +0100
Subject: [capi-sig] C functions that modify their arguments.
In-Reply-To: <4F4BDDF6.7020602@shaw.ca> (Wendell Nichols's message of "Mon, 27
	Feb 2012 12:48:06 -0700")
References: <4F4BDDF6.7020602@shaw.ca>
Message-ID: <87fwdv9ro3.fsf@xemacs.org>

Wendell Nichols <wcn00 at shaw.ca> writes:

> I have a library that I want to wrap for Python (as part of an
> exercise to learn Python, and provide some public good at the same
> time :)  The library is proprietary and I can't change it.  Many of
> its functions modify a pointer provided via an address arg.  Can
> python deal with functions that modify their arguments?  If so
> (generally) how?

I assume you're referring to the use of pointers to simulate
pass-by-reference:

void update_sample(Type1 **arg1, Type2 **arg2)
{
  *arg1 = new_type1();
  *arg2 = new_type2();
  ...
}

The standard way of implementing such out parameters in Python is to
have the function return a tuple of values.  The function is then called
like this:

val1, val2 = sample.update(val1, val2)

From wcn00 at shaw.ca  Tue Feb 28 15:18:47 2012
From: wcn00 at shaw.ca (Wendell Nichols)
Date: Tue, 28 Feb 2012 07:18:47 -0700
Subject: [capi-sig] C functions that modify their arguments.
In-Reply-To: <87fwdv9ro3.fsf@xemacs.org>
References: <4F4BDDF6.7020602@shaw.ca> <87fwdv9ro3.fsf@xemacs.org>
Message-ID: <4F4CE247.9050202@shaw.ca>

Thanks, that looks like it will work just fine!  The simples answers are 
the best :)
wcn

On 12-02-28 03:30 AM, Hrvoje Niksic wrote:
> Wendell Nichols<wcn00 at shaw.ca>  writes:
>
>> I have a library that I want to wrap for Python (as part of an
>> exercise to learn Python, and provide some public good at the same
>> time :)  The library is proprietary and I can't change it.  Many of
>> its functions modify a pointer provided via an address arg.  Can
>> python deal with functions that modify their arguments?  If so
>> (generally) how?
> I assume you're referring to the use of pointers to simulate
> pass-by-reference:
>
> void update_sample(Type1 **arg1, Type2 **arg2)
> {
>    *arg1 = new_type1();
>    *arg2 = new_type2();
>    ...
> }
>
> The standard way of implementing such out parameters in Python is to
> have the function return a tuple of values.  The function is then called
> like this:
>
> val1, val2 = sample.update(val1, val2)
>

From mal at egenix.com  Tue Feb 28 15:59:19 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 28 Feb 2012 15:59:19 +0100
Subject: [capi-sig] C functions that modify their arguments.
In-Reply-To: <4F4BDDF6.7020602@shaw.ca>
References: <4F4BDDF6.7020602@shaw.ca>
Message-ID: <4F4CEBC7.3070905@egenix.com>

Wendell Nichols wrote:
> I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide
> some public good at the same time :)  The library is proprietary and I can't change it.  Many of its
> functions modify a pointer provided via an address arg.  Can python deal with functions that modify
> their arguments?  If so (generally) how?
> 
> If there is no formalised way I'm may have to consider registering some sort of callback to alter
> the argument from within C... kind of awkward but since I'm code generating all the C so long as I
> have a workable approach it doesn't matter how much code it takes :)

You mean: The C functions write data into buffers your Python wrapper
has to provide ?

That's possible, but you'll have to manage memory allocation in your
wrapper. Alternatively, you could copy the data into Python objects
to have Python take over memory management.

PS: You might want to consider using a wrapper generator such as SWIG,
Boost or sip to help you with the wrapping of the library. They typically
take care of all this.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 28 2012)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-02-13: Released eGenix pyOpenSSL 0.13        http://egenix.com/go26
2012-02-09: Released mxODBC.Zope.DA 2.0.2         http://egenix.com/go25
2012-02-06: Released eGenix mx Base 3.2.3         http://egenix.com/go24

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From wcn00 at shaw.ca  Tue Feb 28 16:10:01 2012
From: wcn00 at shaw.ca (Wendell Nichols)
Date: Tue, 28 Feb 2012 08:10:01 -0700
Subject: [capi-sig] C functions that modify their arguments.
In-Reply-To: <4F4CEBC7.3070905@egenix.com>
References: <4F4BDDF6.7020602@shaw.ca> <4F4CEBC7.3070905@egenix.com>
Message-ID: <4F4CEE49.8070607@shaw.ca>

I'm code generating c code based on the library's headers using python 
itself.  half the point of this exercise was to learn python :)
Turns out that I can detect arguments that are to be modified and return 
them as the second item in a tuple.
so:
int = func(&tobechanged,char*,char*)

Can become a python signature like

(int,int) = func(str,str)

If this works as well for all the functions as it has for the first few 
then I'm off to the races.  The argument that gets modified internally 
by the lib all have a particular naming convention, and are always the 
first arg, and are always pointers to void*.  That makes detecting them 
easy and changing the generated code equally as easy.

Thanks for all your interest and assistance
Wendell Nichols


On 12-02-28 07:59 AM, M.-A. Lemburg wrote:
> Wendell Nichols wrote:
>> I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide
>> some public good at the same time :)  The library is proprietary and I can't change it.  Many of its
>> functions modify a pointer provided via an address arg.  Can python deal with functions that modify
>> their arguments?  If so (generally) how?
>>
>> If there is no formalised way I'm may have to consider registering some sort of callback to alter
>> the argument from within C... kind of awkward but since I'm code generating all the C so long as I
>> have a workable approach it doesn't matter how much code it takes :)
> You mean: The C functions write data into buffers your Python wrapper
> has to provide ?
>
> That's possible, but you'll have to manage memory allocation in your
> wrapper. Alternatively, you could copy the data into Python objects
> to have Python take over memory management.
>
> PS: You might want to consider using a wrapper generator such as SWIG,
> Boost or sip to help you with the wrapping of the library. They typically
> take care of all this.
>

From philip at semanchuk.com  Tue Feb 28 14:58:10 2012
From: philip at semanchuk.com (Philip Semanchuk)
Date: Tue, 28 Feb 2012 08:58:10 -0500
Subject: [capi-sig] C functions that modify their arguments.
In-Reply-To: <4F4BDDF6.7020602@shaw.ca>
References: <4F4BDDF6.7020602@shaw.ca>
Message-ID: <F5F76B0C-776A-4A74-B717-A333552E320A@semanchuk.com>


On Feb 27, 2012, at 2:48 PM, Wendell Nichols wrote:

> I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide some public good at the same time :)  The library is proprietary and I can't change it.  Many of its functions modify a pointer provided via an address arg.  Can python deal with functions that modify their arguments?  If so (generally) how?
> 
> If there is no formalised way I'm may have to consider registering some sort of callback to alter the argument from within C... kind of awkward but since I'm code generating all the C so long as I have a workable approach it doesn't matter how much code it takes :)

Hi Wendell,
IMO you have to be more specific about what the library does. If the library isn't Python-aware (which I guess it isn't, otherwise you wouldn't be wrapping it), then any pointers it returns won't point to Python objects. You can return those pointers to Python, but Python will just see a block of raw memory, not Python objects. This is usually not so useful. 

Personally, I'm confused about what you're trying to do.

Cheers
Philip


From mal at egenix.com  Tue Feb 28 16:34:41 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 28 Feb 2012 16:34:41 +0100
Subject: [capi-sig] C functions that modify their arguments.
In-Reply-To: <4F4CEE49.8070607@shaw.ca>
References: <4F4BDDF6.7020602@shaw.ca> <4F4CEBC7.3070905@egenix.com>
	<4F4CEE49.8070607@shaw.ca>
Message-ID: <4F4CF411.1020704@egenix.com>

Wendell Nichols wrote:
> I'm code generating c code based on the library's headers using python itself.  half the point of
> this exercise was to learn python :)
> Turns out that I can detect arguments that are to be modified and return them as the second item in
> a tuple.
> so:
> int = func(&tobechanged,char*,char*)
> 
> Can become a python signature like
> 
> (int,int) = func(str,str)

Right, that's the standard Python approach to output variables. If you
have in/out variables, you'd use:

(int,int) = func(int,str,str)

> If this works as well for all the functions as it has for the first few then I'm off to the races. 
> The argument that gets modified internally by the lib all have a particular naming convention, and
> are always the first arg, and are always pointers to void*.  That makes detecting them easy and
> changing the generated code equally as easy.
> 
> Thanks for all your interest and assistance
> Wendell Nichols
> 
> 
> On 12-02-28 07:59 AM, M.-A. Lemburg wrote:
>> Wendell Nichols wrote:
>>> I have a library that I want to wrap for Python (as part of an exercise to learn Python, and provide
>>> some public good at the same time :)  The library is proprietary and I can't change it.  Many of its
>>> functions modify a pointer provided via an address arg.  Can python deal with functions that modify
>>> their arguments?  If so (generally) how?
>>>
>>> If there is no formalised way I'm may have to consider registering some sort of callback to alter
>>> the argument from within C... kind of awkward but since I'm code generating all the C so long as I
>>> have a workable approach it doesn't matter how much code it takes :)
>> You mean: The C functions write data into buffers your Python wrapper
>> has to provide ?
>>
>> That's possible, but you'll have to manage memory allocation in your
>> wrapper. Alternatively, you could copy the data into Python objects
>> to have Python take over memory management.
>>
>> PS: You might want to consider using a wrapper generator such as SWIG,
>> Boost or sip to help you with the wrapping of the library. They typically
>> take care of all this.
>>

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 28 2012)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-02-13: Released eGenix pyOpenSSL 0.13        http://egenix.com/go26
2012-02-09: Released mxODBC.Zope.DA 2.0.2         http://egenix.com/go25
2012-02-06: Released eGenix mx Base 3.2.3         http://egenix.com/go24

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From python_capi at behnel.de  Tue Feb 28 20:27:54 2012
From: python_capi at behnel.de (Stefan Behnel)
Date: Tue, 28 Feb 2012 20:27:54 +0100
Subject: [capi-sig] C functions that modify their arguments.
In-Reply-To: <4F4BDDF6.7020602@shaw.ca>
References: <4F4BDDF6.7020602@shaw.ca>
Message-ID: <4F4D2ABA.5020001@behnel.de>

Wendell Nichols, 27.02.2012 20:48:
> I have a library that I want to wrap for Python (as part of an exercise to
> learn Python, and provide some public good at the same time :)  The library
> is proprietary and I can't change it.  Many of its functions modify a
> pointer provided via an address arg.  Can python deal with functions that
> modify their arguments?  If so (generally) how?
> 
> If there is no formalised way I'm may have to consider registering some
> sort of callback to alter the argument from within C... kind of awkward but
> since I'm code generating all the C so long as I have a workable approach
> it doesn't matter how much code it takes :)

Marc-Andre already pointed you to a couple of tools that do this for you.
It's best to use an existing tool, because writing both a wrapping tool and
then a wrapper with it means that you will end up having to maintain both
the wrapping tool and the wrapper, whereas you'd normally only want to care
about the wrapper in the long term.

Another (very) common tool to wrap C(++) libraries for Python is Cython.
Specifically, it will allow you to not only write a thin 1:1 wrapper (as I
guess you are doing here), but to actually write an efficient thicker
wrapper that provides a more pythonic interface to the library you are
wrapping. In most cases, this is largely preferable to a straight C-ish
interface, especially when it comes to things like memory management for C
data types or object oriented APIs.

Stefan