Pyste: wrapper function for overloaded function problems
I am trying to expose a C++ member function with a bunch of default arguments, but I also want to provide a C++ wrapper for it. The problem I've had is that Pyste still generates the overload machinery even though it's not wanted since my wrapper function takes different arguments. Any hints? I'm going digging in the Pyste source. Paul Bridger
Hi all, We have 2 versions of the same software. One on linux RH9, and one Win XP. Both of them compile and work fine. They use several C++ files compiled with boost to interact with Python. On windows, a dll with a size of a few hundreds of ko. On linux, the size for the equivalent .so is a few Mo. Have we missed something ? Any hints to reduce this size, if possible, are welcome ! Thanks in advance ... C. Grimault
--- Christophe Grimault <christophe.grimault@novagrid.com> wrote:
We have 2 versions of the same software. One on linux RH9, and one Win XP. Both of them compile and work fine. They use several C++ files compiled with boost to interact with Python. On windows, a dll with a size of a few hundreds of ko. On linux, the size for the equivalent .so is a few Mo.
Have we missed something ? Any hints to reduce this size, if possible, are welcome !
You seem to imply a size difference of three orders of magnitude (kB vs. MB). In my experience it is more a factor of roughly 2-4. I never looked in detail, but you can compare the sizes of the downloads posted here: http://cci.lbl.gov/cctbx_build/ E.g. the Redhat 9 download is 6.8MB, the Windows download is 5.0MB. This comparison is not a really good size-benchmark (gunzip vs. info-zip, Python sources are the same in both downloads), but it gives you a ballpark estimate. Could it be that you are comparing debug builds? Last time I did a Linux debug build (ages ago...) I noticed that the object files are enormous. I didn't pay attention to the sizes of the final .so files, though. Ralf __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - Send 10MB messages! http://promotions.yahoo.com/new_mail
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 15 Jun 2004 at 10:28, Christophe Grimault wrote:
We have 2 versions of the same software. One on linux RH9, and one Win XP. Both of them compile and work fine. They use several C++ files compiled with boost to interact with Python. On windows, a dll with a size of a few hundreds of ko. On linux, the size for the equivalent .so is a few Mo.
Have we missed something ? Any hints to reduce this size, if possible, are welcome !
1. Use GCC 3.4. This should knock 50% off your binary. 2. Patch GCC with my patch at http://www.nedprod.com/programs/gccvisibility.html. With - fvisibility=hidden this should knock a further few hundred Kb off your binary and produce better quality code too. No matter what you do, the Linux binary will be at least 20% larger than your MSVC binary. Cheers, Niall -----BEGIN PGP SIGNATURE----- Version: idw's PGP-Frontend 4.9.6.1 / 9-2003 + PGP 8.0.2 iQA/AwUBQM+cocEcvDLFGKbPEQLS8wCbBUA+oewaC4wT2JlPiE3ZpB47xBUAoK6o y6Gdwodyc/ngkqG3aWpsOLPC =sbJi -----END PGP SIGNATURE-----
Niall Douglas wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 15 Jun 2004 at 10:28, Christophe Grimault wrote:
We have 2 versions of the same software. One on linux RH9, and one Win XP. Both of them compile and work fine. They use several C++ files compiled with boost to interact with Python. On windows, a dll with a size of a few hundreds of ko. On linux, the size for the equivalent .so is a few Mo.
Have we missed something ? Any hints to reduce this size, if possible, are welcome !
1. Use GCC 3.4. This should knock 50% off your binary.
I have tried the gcc-3.4. Compared to gcc-3.2 used before, a .so typically goes from 5.4 Mo to 4.0 Mo.
2. Patch GCC with my patch at http://www.nedprod.com/programs/gccvisibility.html. With - fvisibility=hidden this should knock a further few hundred Kb off your binary and produce better quality code too.
Thanks for the hint. I did not have the time to try yet !
No matter what you do, the Linux binary will be at least 20% larger than your MSVC binary.
Last question. I do not compile my .so with bjam but with a separate Makefile, outside of the boost architecture. May be I'm also missing something in the Makefile that the Jamfile does !? I think i remember that, if I try to compile the same .so, inside the boost architecture, I get a big improvement. Uhm ? Thanks a lot !!
Cheers, Niall
-----BEGIN PGP SIGNATURE----- Version: idw's PGP-Frontend 4.9.6.1 / 9-2003 + PGP 8.0.2
iQA/AwUBQM+cocEcvDLFGKbPEQLS8wCbBUA+oewaC4wT2JlPiE3ZpB47xBUAoK6o y6Gdwodyc/ngkqG3aWpsOLPC =sbJi -----END PGP SIGNATURE-----
_______________________________________________ C++-sig mailing list C++-sig@python.org http://mail.python.org/mailman/listinfo/c++-sig
Hi Paul, sorry about the delay, paul.bridger wrote:
I am trying to expose a C++ member function with a bunch of default arguments, but I also want to provide a C++ wrapper for it. The problem I've had is that Pyste still generates the overload machinery even though it's not wanted since my wrapper function takes different arguments.
Any hints? I'm going digging in the Pyste source.
You want to take a look at the method ExportMethods in ClassExporter.py. ;) Patches are welcome, as always! Regards, Nicodemus.
Hi, I've just done a diff of my Pyste vs. CVS. There are a few small improvements I've made. Two in ClassExporter.py: The problem described below was easy to fix: ClassExporter.py: 375 if method.minArgs != method.maxArgs: # here pyste adds overload declarations and creates extra argument to def() becomes if method.minArgs != method.maxArgs and not method_info.wrapper: # so we nolonger do this if the method has a wrapper This worked fine for me, but I am nolonger using (and hence testing) this change every day. I was writing function/method wrappers for functions that took a customised string implementation, but I stopped doing this when I found I couldn't write constructor wrappers (AFAIK). Now everything is consistently unwrapped. A diff snippet for that: 375c381 < if method.minArgs != method.maxArgs: ---
if method.minArgs != method.maxArgs and not method_info.wrapper:
I also noticed that for inherited virtual methods, Pyste would not: a) warn about lack of defined policies, and b) register the default policy when it could Here is the fix: 778,779c787,792 < # Add policy to overloaded methods also < policy = self.info[method.name].policy or '' ---
# warn the user if this method needs a policy and doesn't have one method_info = self.info[method.name] method_info.policy = exporterutils.HandlePolicy(method,
method_info.policy)
# Add policy to overloaded methods also policy = method_info.policy or ''
There was also a change in GCCXMLParser.py. I posted to C++-Sig about this a little while ago: http://aspn.activestate.com/ASPN/Mail/Message/c++-sig/2088336 GCCXMLParser.py 280c280 < if type(decl) in Class.ValidMemberTypes(): ---
if type(decl) in Class.ValidMemberTypes() and decl.name != '':
Thanks. Paul Bridger Nicodemus wrote:
Hi Paul, sorry about the delay,
paul.bridger wrote:
I am trying to expose a C++ member function with a bunch of default arguments, but I also want to provide a C++ wrapper for it. The problem I've had is that Pyste still generates the overload machinery even though it's not wanted since my wrapper function takes different arguments.
Any hints? I'm going digging in the Pyste source.
You want to take a look at the method ExportMethods in ClassExporter.py. ;) Patches are welcome, as always!
Regards, Nicodemus.
_______________________________________________ C++-sig mailing list C++-sig@python.org http://mail.python.org/mailman/listinfo/c++-sig
Hello all, I've managed to get myself stuck again, and could use some pointers in how to debug my way out of this situation. I have to kinds of objects that are wrapped in a boost::shared_ptr and returned to python. They are both registered with python like this: boost::python::register_ptr_to_python< boost::shared_ptr<ClassOne> >(); boost::python::register_ptr_to_python< boost::shared_ptr<ClassTwo> >(); the strange thing is that the release function of ClassOne is called when python loses all references to it, but ClassTwo's is never called. From within python I have managed to ensure that ClassTwo is actually being released, by using weakref as follows: # win is my ClassTwo object wr = weakref.ref(win) assert wr() del win sys.exc_clear() gc.collect() assert not wr() So now I am trying to debug this situation. Any help or suggestions are greatly appreciated. Thank you, Eric
participants (6)
-
Christophe Grimault -
Eric -
Niall Douglas -
Nicodemus -
paul.bridger -
Ralf W. Grosse-Kunstleve