Non blocking socket server and storage engine

kdeveloper Krzysztof.Retel at googlemail.com
Wed Nov 19 11:36:45 CET 2008


Hello Pythonists,

I am building a non blocking socket server for incomming UDP packets.
The server needs to run at least three threads:
1. getting data and pushing to "some" storage (at the moment I use
queue),
2. acknowledge the package received
3. retrieve the information from the storage and insert it in DB.

The problem I have is that when I use queue it stores more packets in
the queue than it actually receives. An example: sent 99 UDP packets
and queue stored 600-750 entries (?) Why? I have no idea. I have
impression that I still do not understand completely how does the
queue work in python.

Another issue is that I want the server to run very rupidly. The
server needs to get at least 100 packets per second, and the same
amount of acknowledges has to be sent back.

The questions are:
1. What is the best storage engine for Python and multithreading
server?
2. Can I use queue for such problem?
3. How does actually queue work in Python? (I know how it should work
generally, but somehow it doesn't work as I expect)

Any hints & helps? Would be very grateful

Cheers
K



More information about the Python-list mailing list