python CPU usage 99% on ubuntu aws instance using eventlet

Teddy Toyama teddy.toyama at gmail.com
Fri Feb 3 05:13:25 CET 2012


Okay, I am crossposting this from the eventlet dev mailing list since I am
in urgent need of some help.

I am running eventlet 0.9.16 on a Small (not micro) reserved ubuntu
11.10 aws instance.

I have a socketserver that is similar to the echo server from the examples
in the eventlet documentation. When I first start running the code,
everything seems fine, but I have been noticing that after 10 or 15 hours
the cpu usage goes from about 1% to 99+%. At that point I am unable to make
further connections to the socketserver.

This is the important (hopefully) parts of the code that I'm running:

<code>
   # the part of the code that listens for incoming connections
    def socket_listener(self, port, socket_type):
        L.LOGG(self._CONN, 0, H.func(), 'Action:Starting|SocketType:%s' %
socket_type)
        listener = eventlet.listen((self._host, port))
        listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        pool = eventlet.GreenPool(20000)
        while True:
            connection, address = listener.accept()
            connection.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
            """ I want this loop to run as fast as possible.

                I previously grabbed the first message that a plug/device
sent here
                and used that information to add a new object to the
socket_hash.
                Instead of doing that here I've relocated that logic to the
spawned
                object so that this loop is doing as little work as
possible.
            """
            L.LOGG(self._CONN, 0, H.func(),
'IPAddress:%s|GreenthreadsFree:%s|GreenthreadsRunning:%s' %
(str(address[0]), str(pool.free()),str(pool.running())))
            pool.spawn_n(self.spawn_socketobject, connection, address,
socket_type)
        listener.shutdown(socket.SHUT_RDWR)
        listener.close()
</code>
The L.LOGG method simply logs the supplied parameters to a mysql table.

I am running the socket_listener in a thread like so:

<code>
    def listen_phones(self):
        self.socket_listener(self._port_phone, 'phone')

    t_phones = Thread(target = self.listen_phones)
    t_phones.start()
</code>

>From my initial google searches I thought the issue might be similar to the
bug reported at
https://lists.secondlife.com/pipermail/eventletdev/2008-October/000140.html but
I am using a new version of eventlet so surely that cannot be it?

Is there any additional information I can provide to help further
troubleshoot the issue?

Teddy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20120202/62da8ed3/attachment.html>


More information about the Python-list mailing list