[Twisted-Python] Twisted, Threading, Extension Modules and the GIL

Hi everyone,
I have a threading question regarding Twisted and the explanation is a bit long-winded but please bear with me.
The scenario:
I have created an automated build system in Python for PowerBuilder (awful language) that allows a scheduler to assign build jobs to dispatcher programs across a network. These dispatchers have a GUI component coded in wxPython. The initial implementation involved also creating a separate builder program that was spawned by the dispatchers when a build job was received. The builder code called an extension module to interface with the low-level PowerBuilder compiler library. This was done due to the fact the calls to rebuild the application and deploy it are exceptionally CPU-intensive and don't return for many minutes (approx. 30 or so for a large application) and all attempts to include this in the dispatcher resulted in the user interface freezing.
Recently I came across twisted documentation with regards to programming using twisted threads. As an experiment on this concept I amalgamated the dispatcher and builder programs and called the builder code using the following:
d = threads.deferToThread(self.builder.newbuild(buildJob)) d.addCallback(self.builderFinished)
It should be noted that the extension module (a pyrex extension I have created myself) implements the releasing and re-acquiring of the Global Interpreter Lock for all blocking calls. From my understanding this should work but the user interface freezes. Can someone please point me in the direction of a solution?
Thanks,
Grant McDonald
participants (1)
-
Grant McDonald