C replacement for Queue module

Peter Hansen peter at engcorp.com
Sun Oct 23 14:34:41 CEST 2005


Jason Lai wrote:
> As far as Queues go, the adding/popping is apparently done with deque
> which are implemented in C. The Queue class pretty much just provides
> blocking operations and is otherwise a very thin layer around deque. As
> far as primitives go, only threading.Lock is written in C and the
> others are pure Python, so they're not that fast, which might be a
> reason for Queue's slowness.

BTW, I didn't mean (or expect) that one could improve significantly on 
Queue's performance in general just by re-writing it.

I meant that if the OP took into account _his own unique situation_, he 
might see opportunities for improvement which Queue couldn't possibly 
provide, given that it's a general purpose tool.  If his own situation 
simply involves a massive number of discrete put/get operations which 
cannot be bundled in any fashion, rewriting the entire Queue in C just 
to avoid the Python method calls (which have high setup overhead) might 
be the only real option.

-Peter



More information about the Python-list mailing list