[PYTHON C++-SIG] Re: LLNL Python/C++ integration: current status

[Geoffrey Furnish]

David Ascher writes:
 > Hi all.
 > 
 > Geoff said:
 > > Anyway, the result of this is that autoconf now allows building a
 > > python, either with or without C++ support.  Without it, you get
 > > exactly the python you would've gotten in the first place.  With it,
 > > you get a better python, one that can support C++ extension modules.
 > 
 > Just a question: how do you deal with frozenmain.c?  Do you treat it
 > as a C++ file?  

Uhhh.  Never heard of it.  This is an excellent example of why we need
to do this in a public forum.  Thanks for calling it to my attention.
Suggestions welcome.

 > I assume you've also modified the .c files to do the appropriate 
 > extern "C" {...} stuff...

I just copied main.c to cxxmain.cc, and compile that with C++ and link
it in.  The makefiles know which one to use, based on whether or not
it was configured --with-cxx.

 > > [description of argument tuplification to callbacks which I didn't
 > > mean to delete]
 > 
 > Is it possible to have the argument detuplification and parsing
 > automated when so desired?  So that if I specify a callback as expecting
 > 2 ints, I can define those in the method definition and the 'system'
 > will deal with the ArgTuple handling?  It seems it would be a nice
 > feature, if it's possible [this is getting into areas of C++ that I
 > don't know well].

Ohhhhhhh yes.  Yes yes yes.  

I am working on a tool I call the "pythalyzer" (nomenclature would be
obvious only to those I've worked with over the years, who have been
subjected to an endless stream of xxxalyzers for various automatic
code generation purposes).

It is not really ready for public review yet, but the basic idea is
that it can generate these shadow classes for you, by constructing a
Python class (by which I mean, a C++ class for the python extension
module) which directly shadows a particular C++ class (the compiled
asset you are trying to propagate to the Python script level).  This
wrapper class will have one method for each method in the real class.
The wrapper will have the characteristic signature
	PyObject *<python_wrapper_class>::<method>( PyArgs *args );
(Recall from yesterday's post that "self" has been expunged, since it
is trivially identified as "this" inside C++ classes called directly
from Python).  The action of this generated method will be to suck the
arguments out of the argument list tuple, and then call the method on
the real C++ class.  This is all done with dufus cut and paste, so it
is up to the compiler to figure out the types of everything.  (This is
good, b/c the compiler is never wrong, so this plan is not subject to
human error).  This is another area where our C++ interface will
de-C-ize the Python API.  Instead of calling ParseTuple with a format
string and gobs of pointers, you'll just call it directly with the
variables you want filled in.  The template system, using the C++
"traits" technique, will construct the format string for you, and
populate the arguments.  This should make ParseTuple foolproof, and
imune to the wide variety of catistrophic errors which acrue to all
systems which use printf style format string semantics.

 > > #include "PythonX.hh"
 > 
 > This maybe too late, but 'PythonX' makes me think of Python/ActiveX, which
 > is a completely different system...  

This is under discussion here.  Suggestions welcome.  Python++.hh is
another posibility, but I'm not sure how portable that is to systems
with pathetic filesystems (biting tounge).

Ultimately, this stuff could coneivably be made available directly
through the usual Python.h, but I think that is way down the road.

Anyway, "nothing is too late".  The reason we are discussing this here
is specifically so that we can find out what people think, and we are
prepared to act on good suggestions (timescale unspecified of course).

 > > This is the C++ language as we know it, and we are not buckling under
 > > on making a quality interface on account of compiler vendors who don't
 > > support the language as the C++ committee defines it.  This may well
 > > mean that certain C++ compilers in the market place will not compile
 > > this code.  If you have such a compiler, you will need to wait until
 > > your compiler is upgraded to a level conformant with the language
 > > spec, before you will be able to use this.
 > 
 > This is not at all a criticism, but I'm curious: does the latest g++
 > qualify?  I think this will have at least some impact on the public
 > reaction to your system.

2.7.2 is certainly nowhere near adequate.  I don't think 2.8.0 is out
yet, but early indications I have received make me fear it will also
be inadequate, despite being a dramatic improvement over 2.7.2.

We are aware this is a short term liability.  Be we do believe it is a
/short/ term liability, and in our group here at LLNL, we simply do
not work with compilers that do not conform to a recent edition of the
draft.  It will be a great day indeed, when g++ is ANSI C++
compatible, and we look forward to that day, but we are not prepared
to wait idly for that day.

 > > The code described in this note will be available to others.  We have
 > > not made clear release plans/schedules yet.  It would be useful to
 > > get a reading on how much interest there is in this.  Certainly it is
 > > an ongoing development effort, with all the cautionary statements
 > > about instability that go along with that.  Nevertheless, we are
 > > highly motivated to get it into a stable form for our own purposes,
 > > and are prepared to cut occasional patch sets for external use.
 > 
 > What are your thoughts regarding eventual integration in the main source
 > distribution?  It strikes me that such an integration would be a very
 > worthwhile effort, but I realize it is not one without compromises or
 > headaches.

We have in fact already paid some of this "cost of compromise".  My
original design for the C++ trampoline was a direct modification to
the python core.  In my opinion, this is where it fits best.  But
there was significant concern in our group that such an approach was
too invasive for the realistic expectation of being accepted into the
Python official release, so I backed it out to extension status.

We would be very interested in having this work accepted into the
Python standard distribution, in the full course of time.  In order
for that to be realistic, we realize the code needs to be reviewed by
a significantly wider audience than just our little research group,
which is why I lobbied for this SIG.  If it becomes clear through this
forum that our work is of interest to others, we will cut releases of
our patch set for public use.  We will consider suggestions made by
others.  And if/when this system achieves a reasonable degree of
stability and maturity, we would be happy to see it absorbed into the
Python official distribution.

-- 
Geoffrey Furnish		email: furnish at llnl.gov
LLNL X/ICF			phone: 510-424-4227	fax: 510-423-0925

_______________
C++-SIG - SIG for Development of a C++ Binding to Python

send messages to: c++-sig at python.org
administrivia to: c++-sig-request at python.org
_______________