Py2.7/FreeBSD: maximum number of open files

[Christian Heimes]

Am 14.11.2011 17:36, schrieb Tobias Oberstein:
> This is a dedicated machine doing nothing else .. I'm monitoring global FD usage
> 
> sysctl kern.openfiles
> 
> and it's way beyond the configured limit
> 
> $ ulimit -n
> 200000

Apparently you did everything right here. Well, it was worth the try. ;)


> It's not for files:
> 
> This is a network service .. I tested it with up to 50k TCP connections .. however
> at this point, when the service tries to open a file, it'll bail out.
> 
> Sockets+Files both contribute to open FDs.
> 
> I need 50k sockets + 100 files.
> 
> Thus, this is even more strange: the Python (a Twisted service) will happily
> accept 50k sockets, but as soon as you do open() a file, it'll bail out.

A limit of 32k smells like a overflow in a signed int. Perhaps your
system is able and configured to handle more than 32k FDs but you hit an
artificial limit because some C code or API has a overflow. This seems
to be a known bug in FreeBSD
http://lists.freebsd.org/pipermail/freebsd-bugs/2010-July/040689.html


Christian