Non blocking socket server and storage engine

Chris Rebert clp at
Wed Nov 19 11:57:36 CET 2008

On Wed, Nov 19, 2008 at 2:36 AM, kdeveloper
<Krzysztof.Retel at> wrote:
> Hello Pythonists,
> I am building a non blocking socket server for incomming UDP packets.
> The server needs to run at least three threads:
> 1. getting data and pushing to "some" storage (at the moment I use
> queue),
> 2. acknowledge the package received
> 3. retrieve the information from the storage and insert it in DB.
> The problem I have is that when I use queue it stores more packets in
> the queue than it actually receives. An example: sent 99 UDP packets
> and queue stored 600-750 entries (?) Why? I have no idea. I have
> impression that I still do not understand completely how does the
> queue work in python.

No, I believe rather you don't completely understand UDP.

Quoting from Wikipedia (
"UDP does not guarantee reliability or ordering in the way that TCP
does. Datagrams may arrive out of order, ***appear duplicated***, or
go missing without notice."

I think that might at least partially account for your duplicate entries.

Follow the path of the Iguana...

> Another issue is that I want the server to run very rupidly. The
> server needs to get at least 100 packets per second, and the same
> amount of acknowledges has to be sent back.
> The questions are:
> 1. What is the best storage engine for Python and multithreading
> server?
> 2. Can I use queue for such problem?
> 3. How does actually queue work in Python? (I know how it should work
> generally, but somehow it doesn't work as I expect)
> Any hints & helps? Would be very grateful
> Cheers
> K
> --

More information about the Python-list mailing list