Python bindings tutorial

[Alf P. Steinbach]

* Dave Angel:
> Alf P. Steinbach wrote:
>> * Dave Angel:
>>> Stefan Behnel wrote:
>>>> <div class="moz-text-flowed" style="font-family: 
>>>> -moz-fixed">MikeLisanke at gmail.com, 17.03.2010 10:08:
>>>>> Its interesting you've mentioned the hard work involved in this
>>>>> interface (binding to an EXE instead of a DLL). A year or more ago I
>>>>> was looking at interfacing IPMITOOL to python. Do to the problems
>>>>> incurred with swig/python I switched to a running the process through
>>>>> its command-line interface. I always felt the problems in interfacing
>>>>> python to an EXE should be worked on (to minimize them), making the
>>>>> direct use of an EXE API's a routine task. I understand some of the
>>>>> problems using an EXE (not running all of its startup code   but
>>>>> enough for its proper operation). Have you found this a recurring
>>>>> question? Thanks.
>>>>
>>>> I think the point here is that executable binaries are not supposed 
>>>> to be used as libraries. Libraries are. That's the difference 
>>>> between a DLL and an executable in the first place. To run an 
>>>> executable, execute it. The subprocess module is the tool of choice 
>>>> here. To use a DLL, link against it.
>>>>
>>>> Stefan
>>>>
>>> There's no real reason parts of an exe cannot be exported, same as a 
>>> dll.  They are in fact the same structure.  And in fact many other 
>>> files in the Windows environment are also the same structure, from 
>>> fonts to ocx's
>>>
>>> Saying they're "not supposed to be used" is like saying that a python 
>>> module should not have an
>>>
>>> if __name__ == "__main__":
>>>
>>> section.  After all, who could want to both run a file, and import 
>>> the same file??
>>
>> A Windows DLL has defined initialization and cleanup per process and 
>> per thread.
>>
>> This means that e.g. static variables can be properly initialized when 
>> you load the DLL in order to use its functions (I'm skipping 
>> discussion of subtle problems, but that's the essence).
>>
>> A Windows EXE has (only) a single entry point which is for process 
>> startup. It invokes the EXE's behavior-as-a-program. There is no way 
>> to use it to e.g. initialize static variables in order to use exported 
>> functions.
>>
>> Hence Mike Lisanke's idea of "not running all of its startup code but 
>> enough for its proper operation" is generally not possible.
>>
>> An EXE can be used as a kind of server, /if/ it is designed for that. 
>> In particular it can be a COM server, allowing access of its 
>> functionality from any COM-enabled binding, which for Python would 
>> mean OLE Automation (COM, OLE, Automation: this is Microsoft 
>> technology, we're talking Windows EXEs here). But a Python binding to 
>> EXEs in general can't, as far as I can see, make assumptions about any 
>> particular kind of server being implemented by the EXE.
>>
> I'm not talking about COM servers, which run in a separate process.  
> Only about calling functionality that happens to be encoded in an .EXE 
> file.
> 
> You're trying to generalize what I said.  I never said you could call 
> into any .EXE file, just that calling into one is not infeasible.

Well, true. And I'm sorry if my reply sounded as a misrepresentation. I haven't 
tried this calling-a-function-in-an-exe (since 16-bit Windows, that is!, but 
that was very different), but I can imagine two such cases:

   (1) a function f is exported by the EXE, and f doesn't depend on anything
       but its arguments and does not use any static storage, or

   (2) a DLL loaded by the EXE calls back into the EXE, which could work
       because then everything's initialized (however, there are better ways).

But these are very special cases where one really has to know what one is doing.

As Gabriel Genellina wrote up-thread, "you *could* do that if you work hard 
enough, but that's not how things are usually done".

That said, in Windows the least uncommon reason to load an EXE as a DLL is, IME, 
to access resource data in the EXE. Tip for that: the module handle, casted to 
appropriate pointer type, points to the start of the loaded image (great for 
accessing e.g. version info resource). I think this is still undocumented...


> You're describing using an .EXE written using the Microsoft C compiler 
> and/or libraries, not a Windows .EXE in general.  In any case, whenever 
> you link to a module written in a different environment, you'll have 
> restrictions.  Chief among them is the use of a compatible memory model, 
> the stack conventions, the allocators, and so on..  Solving the thread 
> issue is probably the easiest one to fix.
> 
> I'm not recommending it, just refuting the "not supposed to" quote above.

Again, I'm sorry if my reply sounded as a misrepresentation.

I just tried to make the solution-constraining technical facts available, 
addressing Mike's "making the direct use of an EXE API's a routine task".

I think we're all in (violent?) agreement on this. :-)


Cheers,

- Alf