On 25/06/15 16:08, Nick Coghlan wrote:
It generally isn't scientific programmers I personally hit problems with (although we have to allow for the fact many of the scientists I know I met *because* they're Pythonistas). For that use case, there's not only HPC to point to, but a number of papers that talking about Cython and Numba in the same breath as C, C++ and FORTRAN, which is pretty spectacular company to be in when it comes to numerical computation.
Cython can sometimes give the same performance as C or Fortran, but as soon as you start to use classes in the Cython code you run into GIL issues. It is not that the GIL is a problem per se, but because Cython compiles to C, the GIL is not released until the Cython function returns. That is, unless you manually release it inside Cython. This e.g. means that the interpreter might be locked for longer durations, and if you have a GUI it becomes unresponsive. The GIL is more painful in Cython than in Python. Personally I often end up writing a mix of Cython and C or C++. Numba is impressive but still a bit immature. It is an LLVM based JIT compiler for CPython that for simple computational tasks can give performance similar to C. It can also run Python code on Nvidia GPUs. Numba is becoming what the dead swallow should have been.
Instead, the folks that I think have a more valid complaint are the games developers, and the folks trying to use games development as an educational tool.
I have not developed games myself, but for computer graphics with OpenGL there is certainly no reason to complain. NumPy arrays are great for storing vertex and texture data. OpenGL with NumPy is just as fast as OpenGL with C arrays. GLSL shaders are just plain text, Python is great for that. Cython and Numba are both great if you call glVertex* functions the old way, doing this as fast as C. Display lists are also equally fast from Python and C. But if you start to call glVertex* multiple times from a Python loop, then you're screwed.
That does suggest to me a possible "archetypal problem" for the work Eric is looking to do here: a 2D canvas with multiple interacting circles bouncing around. We'd like each circle to have its own computational thread, but still be able to deal with the collision physics when they run into each other.
There are people doing Monte Carlo simulations with thousands or millions of particles, but not with one thread per particle. :-) Sturla