From furnish at acl.lanl.gov Sat May 3 00:36:05 1997 From: furnish at acl.lanl.gov (Geoffrey Furnish) Date: Fri, 2 May 1997 16:36:05 -0600 (MDT) Subject: [PYTHON C++-SIG] Re: LLNL Python/C++ integration: current status In-Reply-To: <9705021756.ZM20529@styx.hks.com> References: <9705021756.ZM20529@styx.hks.com> Message-ID: <199705022236.QAA09518@gray.acl.lanl.gov> Uttam M. Narsu writes: > On Wed, 12 Feb 1997 Geoffrey Furnish wrote: > > > Availability of the LLNL C++ patch set: > > The code described in this note will be available to others. We have > > not made clear release plans/schedules yet. It would be useful to > > get a reading on how much interest there is in this. Certainly it is > > an ongoing development effort, with all the cautionary statements > > about instability that go along with that. Nevertheless, we are > > highly motivated to get it into a stable form for our own purposes, > > and are prepared to cut occasional patch sets for external use. > > Has any progress been mad on external availability? I would like to try > generating the interface code for our C++ classes with the Open C++ MOP > and targeting the LLNL C++ patch set. Sorry for the dearth of info of late. Since I wrote that, I have left Livermore and now work at Los Alamos. The C++/Python integration effort continues however, so in principle this should not be a problem. I do know that the Livermore staff has recently performed some of the work which would have to be done before a patch set could be productively cut. So I know it is "closer" to a distributable condition, but I do not know the ETA. Perhaps Paul or Lee can provide some prognostication to the list. On the actual development of the interface front, Paul and I were able to make major headway over the last couple of months, so the system is rapidly approaching something we both regard as being a "good first cut". Sorry to tease, but please just hold tight a while longer... -- Geoffrey Furnish email: furnish at lanl.gov LANL CIC-19 POOMA/RadTran phone: 505-667-0393 _______________ C++-SIG - SIG for Development of a C++ Binding to Python send messages to: c++-sig at python.org administrivia to: c++-sig-request at python.org _______________ From mjl at dino.ph.utexas.edu Tue May 6 12:51:52 1997 From: mjl at dino.ph.utexas.edu (Maurice J. LeBrun) Date: Tue, 6 May 1997 05:51:52 -0500 (CDT) Subject: [PYTHON C++-SIG] Re: LLNL Python/C++ integration: current status In-Reply-To: <199705022236.QAA09518@gray.acl.lanl.gov> from "Geoffrey Furnish" at May 2, 97 04:36:05 pm Message-ID: <199705061106.HAA09230@python.org> A non-text attachment was scrubbed... Name: not available Type: text Size: 1922 bytes Desc: not available URL: From phil at geog.ubc.ca Thu May 29 01:35:21 1997 From: phil at geog.ubc.ca (Phil Austin) Date: Wed, 28 May 1997 16:35:21 -0700 Subject: [PYTHON C++-SIG] SWIG, NumPy and C++ templated arrays Message-ID: <199705282335.QAA29653@curlew.geog.ubc.ca> Someone on the SWIG mailing list asked about SWIG and numeric Python. I've put together an example of our initial attempt using MV++ arrays (see ftp://ftp.geog.ubc.ca/pub/swig/example.tar.gz (11 Kbytes)). I'm including below the README and interface files from the example. Any comments/questions appreciated. SWIG-python example for templated C++ arrays and the numerical python extension. Michael Cherkassoff (mcherk at geog.ubc.ca) and Phil Austin (phil at geog.ubc.ca). This directory contains example code which glues Python NumPy arrays to C++ templated arrays (all necessary headers from Roldan Poza's MV++ library are included). This approach currently works for us, and we will soon be using it wholesale as we move to either MV++'s replacement (http://math.nist.gov/tnt/lib/) or one of the new expression-template based array classes (like Blitz++ http://monet.uwaterloo.ca/blitz/oon.html, or A++ http://www.c3.lanl.gov/~dquinlan/A++P++.html ). Below I describe how the example works, but for the savants, we'd appreciate help with three unresolved issues with the interface file (plus, of course, any other comments on either the python or c++). 1) Because SWIG doesn't handle templates, we fool it with a second set of typedefs (see cutmod.i and cutmodfake.h)--is there a more elegant approach? 2) We were unable to 'do the right thing' in the typemap to go from $source to a PyArrayObject*. That is, in cutmod.i (see comments therein): aa=(PyArrayObject *)PyArray_ContiguousFromObject($source, PyArray_Float,1,1); gives a zero pointer, and so we resorted to: aa = (PyArrayObject *) $source; 3) We're still confused about reference counting. Should we be calling PyDECREF(aa) in cutmod.i? The code: Cutmod.cpp defines a C++ function "cut", which cuts data into a series of histogram bins. Cutmod.i gives the SWIG interface, and test.py contains a simple test. The cut function (modeled on cut in Splus) takes either two python lists or two arrays (data and bins), and returns a list of lists containing the indexes of the data elements that fit into the specified bin intervals. i.e. in Python, typing _______________ from Numeric import * import cutmod data=array( [10, 50, 24, 45, 70, 59, 21, 33, 44, 65, 13, 11, 99, 15, 14, 78, 93, 45, 18, 32, 37, 16, 18, 84, 75, 48, 33, 17, 61, 90]) bins =array( [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]) output=cutmod.cut(data,bins) _________________ produces: output [[0, 11, 10, 14, 13, 21, 27, 18, 22], [6, 2], [19, 7, 26, 20], [8, 3, 17, 25], [1, 5], [28, 9], [4, 24, 15], [23], [29, 16, 12]] to install, just untar the file and make with make cutmode.so invoke python and import test the output should be test.output tested only on Linux 2.0.27 using gcc 2.7.2.1 Appendix: For those unfamiliar with MV++: Here's the way I would implement cut in Python: def cut(data,bins): #input: data (numarray), bins (numarray) to cut data. #output: binlist (list of lists), indexes in each bin binlength=bins.shape[0] datalength=data.shape[0] #populate an empty list of lists binlist=[0]*(binlength-1) for i in range(len(binlist)): binlist[i]=[] index=range(bins.shape[0]) sortedIndex=argsort(data) i=0 #move through the bins, dropping the sorted indices in the appropriate bin for leftbinmarker in range(binlength-1): if i > (datalength-1): return binlist while data[sortedIndex[i]] < bins[leftbinmarker+1]: binlist[leftbinmarker].append(sortedIndex[i]) i=i+1 if i > (datalength-1): return binlist cutmod.cc is a straight translation of this code using MV++ vectors instead of lists, with a little extra error checking. ==> cutmod.i <== // Code for SWIG to generate wrapper for C++ function that // is to be called from Python. // On the Python's end the function takes two arguments, each of // them either 1-D NumPy array or a Python list of floats. It returns a // list of lists of integers. // On the C++ end the function takes two MV++ vectors of floats. // It returns a MV++ vector of vectors of integers. /* Here are the contents of cutmod.h: #include "mvmtp.h" typedef MV_Vector VectorInt; typedef MV_Vector VectorFloat; typedef MV_Vector VectorList; ******************************************************/ %module cutmod %{ #include "cutmod.h" // this is going to be included without #include "arrayobject.h" // changes into wrapper code %} %include "cutmodfake.h" // this is for SWIG to look at. // cutmod.h cannot be put here since // SWIG doesn't like templates. And without // something here it generates // PyArg_ParseTuple(args,"ss",&_obj0,&_obj1) // instead of desired // PyArg_ParseTuple(args,"OO",&_obj0,&_obj1) %typemap(python,in) VectorFloat { int i, size; enum TYPE {List, Array}; TYPE type; PyArrayObject * aa; float * bb; if (PyList_Check($source)) { size = PyList_Size($source); type = List; } else if (PyArray_Check($source)) { size = PyArray_Size($source); type = Array; aa = (PyArrayObject *) $source; // This is a hack, but it bb = (float *) aa->data; // works. The proper way: // with PyArray_ContiguousFromObject() does not. 8-( /* The proper code would have been: aa=(PyArrayObject *)PyArray_ContiguousFromObject($source, PyArray_Float,1,1); but this gives zero pointer for some reason obscure for me */ } else { PyErr_SetString(PyExc_TypeError, "neither List nor Array"); return NULL; } VectorFloat a(size); // Cannot declare $target since it is // already being done by SWIG. for (i=0; i Someone on the SWIG mailing list asked about SWIG and numeric Python. I've put together an example of our initial attempt using MV++ arrays (see ftp://ftp.geog.ubc.ca/pub/swig/example.tar.gz (11 Kbytes)). I'm including below the README and interface files from the example. Any comments/questions appreciated. SWIG-python example for templated C++ arrays and the numerical python extension. Michael Cherkassoff (mcherk at geog.ubc.ca) and Phil Austin (phil at geog.ubc.ca). This directory contains example code which glues Python NumPy arrays to C++ templated arrays (all necessary headers from Roldan Poza's MV++ library are included). This approach currently works for us, and we will soon be using it wholesale as we move to either MV++'s replacement (http://math.nist.gov/tnt/lib/) or one of the new expression-template based array classes (like Blitz++ http://monet.uwaterloo.ca/blitz/oon.html, or A++ http://www.c3.lanl.gov/~dquinlan/A++P++.html ). Below I describe how the example works, but for the savants, we'd appreciate help with three unresolved issues with the interface file (plus, of course, any other comments on either the python or c++). 1) Because SWIG doesn't handle templates, we fool it with a second set of typedefs (see cutmod.i and cutmodfake.h)--is there a more elegant approach? 2) We were unable to 'do the right thing' in the typemap to go from $source to a PyArrayObject*. That is, in cutmod.i (see comments therein): aa=(PyArrayObject *)PyArray_ContiguousFromObject($source, PyArray_Float,1,1); gives a zero pointer, and so we resorted to: aa = (PyArrayObject *) $source; 3) We're still confused about reference counting. Should we be calling PyDECREF(aa) in cutmod.i? The code: Cutmod.cpp defines a C++ function "cut", which cuts data into a series of histogram bins. Cutmod.i gives the SWIG interface, and test.py contains a simple test. The cut function (modeled on cut in Splus) takes either two python lists or two arrays (data and bins), and returns a list of lists containing the indexes of the data elements that fit into the specified bin intervals. i.e. in Python, typing _______________ from Numeric import * import cutmod data=array( [10, 50, 24, 45, 70, 59, 21, 33, 44, 65, 13, 11, 99, 15, 14, 78, 93, 45, 18, 32, 37, 16, 18, 84, 75, 48, 33, 17, 61, 90]) bins =array( [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]) output=cutmod.cut(data,bins) _________________ produces: output [[0, 11, 10, 14, 13, 21, 27, 18, 22], [6, 2], [19, 7, 26, 20], [8, 3, 17, 25], [1, 5], [28, 9], [4, 24, 15], [23], [29, 16, 12]] to install, just untar the file and make with make cutmode.so invoke python and import test the output should be test.output tested only on Linux 2.0.27 using gcc 2.7.2.1 Appendix: For those unfamiliar with MV++: Here's the way I would implement cut in Python: def cut(data,bins): #input: data (numarray), bins (numarray) to cut data. #output: binlist (list of lists), indexes in each bin binlength=bins.shape[0] datalength=data.shape[0] #populate an empty list of lists binlist=[0]*(binlength-1) for i in range(len(binlist)): binlist[i]=[] index=range(bins.shape[0]) sortedIndex=argsort(data) i=0 #move through the bins, dropping the sorted indices in the appropriate bin for leftbinmarker in range(binlength-1): if i > (datalength-1): return binlist while data[sortedIndex[i]] < bins[leftbinmarker+1]: binlist[leftbinmarker].append(sortedIndex[i]) i=i+1 if i > (datalength-1): return binlist cutmod.cc is a straight translation of this code using MV++ vectors instead of lists, with a little extra error checking. ==> cutmod.i <== // Code for SWIG to generate wrapper for C++ function that // is to be called from Python. // On the Python's end the function takes two arguments, each of // them either 1-D NumPy array or a Python list of floats. It returns a // list of lists of integers. // On the C++ end the function takes two MV++ vectors of floats. // It returns a MV++ vector of vectors of integers. /* Here are the contents of cutmod.h: #include "mvmtp.h" typedef MV_Vector VectorInt; typedef MV_Vector VectorFloat; typedef MV_Vector VectorList; ******************************************************/ %module cutmod %{ #include "cutmod.h" // this is going to be included without #include "arrayobject.h" // changes into wrapper code %} %include "cutmodfake.h" // this is for SWIG to look at. // cutmod.h cannot be put here since // SWIG doesn't like templates. And without // something here it generates // PyArg_ParseTuple(args,"ss",&_obj0,&_obj1) // instead of desired // PyArg_ParseTuple(args,"OO",&_obj0,&_obj1) %typemap(python,in) VectorFloat { int i, size; enum TYPE {List, Array}; TYPE type; PyArrayObject * aa; float * bb; if (PyList_Check($source)) { size = PyList_Size($source); type = List; } else if (PyArray_Check($source)) { size = PyArray_Size($source); type = Array; aa = (PyArrayObject *) $source; // This is a hack, but it bb = (float *) aa->data; // works. The proper way: // with PyArray_ContiguousFromObject() does not. 8-( /* The proper code would have been: aa=(PyArrayObject *)PyArray_ContiguousFromObject($source, PyArray_Float,1,1); but this gives zero pointer for some reason obscure for me */ } else { PyErr_SetString(PyExc_TypeError, "neither List nor Array"); return NULL; } VectorFloat a(size); // Cannot declare $target since it is // already being done by SWIG. for (i=0; i