On Mon, Jul 26, 2010 at 3:00 AM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
At Resolver Systems we created a "calculation system" that does large calculations on background threads using IronPython. Doing them on a background thread allows the ui to remain responsive. Several calculations could run simultaneously using multiple cores.
As the calculation operates on a large object graph (which the ui then needs access to in order to display it) using multiprocessing would have imposed a very big overhead due to serialization / deserialization (the program runs on windows).
Using CPython would have made the program a lot slower due to the GIL.
Sure. Note that using threads with the GIL, it is not a problem to keep the UI responsive even if background calculations are going on (at worst it requires some tweaking of sys.setcheckinterval() or its new-GIL equivalent). However with the GIL multiple calculations would be limited to a single core. According to CSP advicates, this approach will break down when you need more than 8-16 cores since cache coherence breaks down at 16 cores. Then you would have to figure out a message-passing approach (but the messages would have to be very fast). -- --Guido van Rossum (python.org/~guido)