On 20 July 2017 at 23:32, Stefan Behnel <stefan_ml@behnel.de> wrote:
So, before considering to write an accelerator module in C that replaces some existing Python module, and thus duplicating its entire source code with highly increased complexity, I'd like to remind you that simply compiling the Python module itself to C should give at least reasonable speed-ups *without* adding to the maintenance burden, and can be done optionally as part of the build process. We do that for Cython itself during its installation, for example.
And if folks are concerned about the potential bootstrapping issues with this approach, the gist is that it would have to look something like this: Phase 0: freeze importlib - build a CPython with only builtin and frozen module support - use it to freeze importlib Phase 1: traditional CPython - build the traditional Python interpreter with no Cython accelerated modules Phase 2: accelerated CPython - if not otherwise available, use the traditional Python interpreter to download & install Cython in a virtual environment - run Cython to selectively precompile key modules (such as those implicitly imported at startup) Technically, phase 2 doesn't actually *change* CPython itself, since the import system is already setup such that if an extension module and a source module are side-by-side in the same directory, then the extension module will take precedence. As a result, precompiling with Cython is similar in many ways to precompiling to bytecode, its just that the result is native machine code with Python C API calls, rather than CPython bytecode. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia