-----Original Message----- From: email@example.com [mailto:firstname.lastname@example.org] On Behalf Of Guido van Rossum Sent: 31. október 2012 15:01 To: Kristján Valur Jónsson Cc: email@example.com Subject: Re: [Python-ideas] non-blocking buffered I/O
Modern CPUs are black boxes full of magic. I'm not too surprised that running Python code on multiple threads incurs some kind of overhead that keeping the Python interpreter in one thread avoids.
Ah, but I forgot to mention one weird thing: If we used a pool of threads for the callbacks, and pre-initalized those threads with python states, and then acquired the GIL using PyEval_RestoreThread(), then this overhead went away. It was only the dynamic tread state acquired using PyGilState_Ensure() that caused cpu overhead. Using the fixed pool was not acceptable in the long run, in particular we din't want to complicate things to another level by adding a thread pool manger to the whole thing when the OS is fully capable of providing an external callback thread.
I regret not spending more time on this and to be able to provide an actual performance analysis and fix. Instead I have to be that weird old man in the tavern uttering inscrutable warnings that no young adventurer pays any attention to :)