[Python-Dev] FreeBSD 7 amd64 and large memory tests

[Andrew MacIntyre]

Andrew MacIntyre wrote:

> I'll take this up with FreeBSD folk, but I'm open to ideas as to how
> best to deal with the problem in the context of the test suite pending
> resolution by FreeBSD.

The response I got from Jason Evans (author of the new malloc()
implementation), along with that of another respondent, indicates that
the behaviour on FreeBSD 7.1 and later will (mostly) be restored to that
similar to 6.x and earlier through the default use of sbrk() and
consequent obedience to the data segment size limit (ulimit -d) - which
defaults to 512MB in a standard FreeBSD install in recent times.

The residual problem (as of 7.1) is that malloc() defaults to falling
back to the mmap() strategy when it can't get more address space via
sbrk().  As noted in the tracker item for issue 3862, the only way to
control this is the virtual memory size limit (ulimit -v), which
unfortunately defaults to "unlimited"...

FreeBSD's malloc() can be tuned in several ways, so it is possible to
force use of the sbrk() only strategy (as of 7.1) which would exactly
match behaviour of the old malloc().

It seems to me that the most practical way forward is to just institute a
policy that tests that want to try and test out of memory behaviour must
ensure that appropriate resource limits are in place; if they can't (such
as because the platform running the tests doesn't support
getrlimit()/setrlimit()) the test should be skipped.

As Mark Dickinson has suggested a patch for issue 3862 which should worm
around the issue with test_array on 64 bit platforms, I think we can move
forward for the time being.

Cheers,
Andrew.

-- 
-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac at pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia