[Python-Dev] Accelerating extension module compilation [distutils]

Alex Leach albl500 at york.ac.uk
Tue Apr 9 12:37:09 CEST 2013


Hi,

Apologies if this is the wrong place to ask, but thought this question  
would be relevant to Python core and extension module devs.. This the  
right place?

I've been using distutils to compile C++ extensions / bindings written  
with Boost.Python, and have been implementing some (often frowned-upon)  
monkey-patching magic to speed up the compilation process. I was wondering  
if other Python devs would appreciate the benefit from a  
distutils-integrated patch of the same functionality, using less-frowned  
upon programming techniques.

More specifically, I have felt it useful during development to incorporate  
the following functionality into a setupext.py file:-

   1. Parallel compilation, by monkey-patching  
distutils.ccompiler.CCompiler.compile. (A basic solution was provided on  
StackOverflow[1].)

   2. Create a "unity-build"[2].

   3. Only re-compile objects whose source-code / included files have  
changed.

I'll happily share my code (hacks), but before getting too technical with  
the discussion, I was just wondering whether any of these would be  
considered useful enough / easy enough to implement without breaking  
backwards-compatibility, to incorporate into core distutils. Any thoughts?

Thanks for your time.
Kind regards,
Alex


[1] -  
http://stackoverflow.com/questions/11013851/speeding-up-build-process-with-distutils
[2] -  
http://stackoverflow.com/questions/543697/include-all-cpp-files-into-a-single-compilation-unit

-- 
Using Opera's mail client: http://www.opera.com/mail/


More information about the Python-Dev mailing list