[python-win32] C extension class wrapping and sequence questions

[Mark English]

Hi,
I'm a bit stumped about how to develop this problem, so I'll overburden
you with questions and information, and appreciate any answers you may
have.

We use a networking "Stream" to send and receive "Messages".
I want to wrap streams and messages in Python, so that ultimately I can
do the following at a prompt:

> msgMine = PyMessages.MsgFoo(args_msg)
> streamMine = PyStream.Stream(args_stream)
> streamMine.Send(msgMine)

There are a lot of messages and associated structures, currently
implemented in C++. The C++ is actually auto-generated from a
proprietary format. There has been an earlier attempt to wrap the
messages in Python which works to some extent, although it requires a
fairly high-level interface that runs in an executable with Python
embedded in it, and is ultimately pretty difficult to use. It's also
hand-coded, and therefore error-prone.

I'd like to auto-generate my code, and have some experience of Python,
and plenty of experience with C++, mostly under Windows. I have very
little experience with writing Python C extensions.

So here are the questions:
1) How should I aim to construct my code ? Should the bulk of the
intelligence lie in the Python code, ultimately resulting in something I
can pass to a C-extension "Stream" type which will know how to create
the appropriate C++ "Message" and send it off ? This means no access to
the underlying C++ implementation of messages which may not be a great
loss seeing as how it is autogenerated anyway. On the other hand, it
also means no access to all the associated C++ message handling code
which IS useful, and not really easily wrapped in its own set of Python
classes.

Alternatively, I could make the Python "Message" classes thin-wrappers
around the C++ code. Each "Message" class would have its own Python
type, and the C module would export constructors for the types. This
seems reasonably attractive. I could then override "__setattr__()" so
that programmers could either write:
> msgMine.SetFoo("Bar")
OR
> msgMine.strFoo = "Bar"
and the underlying implementation would call the same code in C++.
Because this is autogenerated, I don't mind going this route.

2) How do I handle "Message" instances which contain sequences
(especially sequences of auto-generated structures) ? For example, if
the "Message" contains an array of "Record" instances, then this is
fine:
> msgMine.arrayRec[0].strFoo = "Bar"
> strSomething = msgMine.arrayRec[0].strFoo
because I can return a sequence of thin wrapper objects for
"msgMine.arrayRec" from my C module.

Similarly this also is fine:
> arrayRec = PyRecords.Record(args_record) msgMine.SetArrayRec(index,
> arrayRec)
since I am accessing the array through the owner "Message".

However, I can't see an easy way around this problem:
> arrayRec = PyRecords.Record(args_record) #Construct a new record 
> object msgMine.arrayRec[0] = arrayRec #Put it in the sequence directly

If I simply build a sequence of thin wrappers for "msgMine.arrayRec",
then setting one of its members is effectively changing a temporary
object. I could write my own sequence wrapping object (like a Python
class inheriting from "UserList"), perhaps by providing implementations
for the sequence type, so that setting an attribute actually calls a
method in the owner C++ "Message" object. This seems like hard work for
something that would be easy to do in Python.

Alternatively, if I go with a Python-centric implementation then code
like that above isn't a problem. The message doesn't get converted to
its equivalent C++ object until the stream needs to do so.

Trying to frame the questions a bit more generally then, when wrapping
an existing C++ class hierarchy, how do you choose between thin Python
wrappers (which may not leverage all that Python has to offer), and
"more Python less C++" (which may leave out some of the C++
functionality). Also, how do C extension types present instances which
contain sequences of other instances ?

I suspect if you're any good you do it either way you prefer and don't
mess anything up, or leave anything out, but for a first-timer this
isn't really an option, and I'd like to get some minimal functionality
up and about.

I'm looking at following the approach used in win32ui where types are
factories, and contain enough mapping intelligence to go back and forth
between C and python. I should be able to autogenerate the types. This
doesn't solve my owner-of-a-sequence problem though, and I'm worried
I'll hit other as yet unforeseen problems down the road. Similarly the
SWIG implementation seems to make arrays read-only, and you have to
write your own code with typemaps if you want to expand on that
implementation.

Any thoughts, or pointers to modules which solve similar problems ?

Thanks for your time and patience,
Cheers,
mE


-----------------------------------------------------------------------
The information contained in this e-mail is confidential and solely 
for the intended addressee(s). Unauthorised reproduction, disclosure, 
modification, and/or distribution of this email may be unlawful. If you 
have received this email in error, please notify the sender immediately 
and delete it from your system. The views expressed in this message 
do not necessarily reflect those of LIFFE Holdings Plc or any of its subsidiary companies.
-----------------------------------------------------------------------