[Twisted-Python] Memcache client issues
Hello! I'm trying to write small tcp server, which doing some stuff with memcache. Everything was fine until I start it on production, after few minutes I got to rollback to old one. My script got max open files limit :( After liitle inspection, I saw that, new connection to my daemon opens new connection to memcache and after disconnect its not closed and on new connections is not reused. I could not find how to disconnect form memcache after processing is done, or how to reuse open connections. Maybe someone has some good example of this? Thanks!
Hi, You might want to take a look at the txmemcache project: https://launchpad.net/tx.memcache Keep in mind that it might be more useful to use the server hash function used in python-memcache/cmemcache: http://www.tummy.com/Community/software/python-memcached/ Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun
On Thu, Dec 17, 2009 at 5:17 AM, Reza Lotun <rlotun@gmail.com> wrote:
Hi,
You might want to take a look at the txmemcache project: https://launchpad.net/tx.memcache
Please proceed with utmost caution if you choose to use tx.memcache - I wrote this for a small utility script for a project that got axed, and I'm not using memcache for any production apps now. So, due to it not being fully complete, tested in production, or even unit tested, expect the same level of grandness in terms of stability. The hashing function is pluggable, and it worked fine for my purposes.
Hi Drew,
Please proceed with utmost caution if you choose to use tx.memcache - I wrote this for a small utility script for a project that got axed, and I'm not using memcache for any production apps now. So, due to it not being fully complete, tested in production, or even unit tested, expect the same level of grandness in terms of stability.
Thanks for writing it, by the way. When I first got around to playing with memcached in my Twisted app, your project was really the only thing out there that I could find that spoke Twisted and memcached. It was a great place to *start* in terms of customizing something for my own needs.
The hashing function is pluggable, and it worked fine for my purposes.
Well, the default hashing function is quite slow. If I was dealing with a Twisted-only memcached integration, I would probably use the built-in python hash function. If I were integrating with other apps sharing the same memcached pool - like a Django web app - I would use the python-memcache server hash for sure. Cheers, Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun
On Wed, Dec 16, 2009 at 2:34 PM, MārisR <maris@chown.lv> wrote:
Hello! I'm trying to write small tcp server, which doing some stuff with memcache. Everything was fine until I start it on production, after few minutes I got to rollback to old one. My script got max open files limit :( After liitle inspection, I saw that, new connection to my daemon opens new connection to memcache and after disconnect its not closed and on new connections is not reused. I could not find how to disconnect form memcache after processing is done, or how to reuse open connections. Maybe someone has some good example of this? Thanks!
Sorry to post twice on this, but setting aside tx.memcache or other abstractions over the memcache client, I never had any problems with leaking connections; that is tx.memcache wasn't written to address such a problem, so I don't think using it would necessarily solve your issues. Can you post some code for a short program that reproduces the problem you're describing? -Drew
participants (3)
-
Drew Smathers -
MārisR -
Reza Lotun