Cost of Queue.put
gagsl-py2 at yahoo.com.ar
Tue Mar 18 07:00:42 CET 2008
On 18 mar, 01:13, James <rent.lupin.r... at gmail.com> wrote:
> Basically, what I need is a multi-process safe persistent quick queue.
> The arrangement I had was a simple XML-RPC service running on the web
> server which the various web server threads POST the relevant search
> engine updates to. These updates were added to a persistent queue
> built on top of Queue.Queue and cPickle, and there was a separate
> thread in the XML-RPC server actually adding the updates to the search
> However, the CPU consumption of the XML-RPC server seems to grow
> pretty much linearly with the length of the persistent queue. Isn't
> Queue.put O(1)?
Maybe, but pickle.dumps is very likely O(n) and dominates. Why don't
you use a database? Just insert and delete records with each get and
More information about the Python-list