[Numpy-discussion] allocated memory cache for numpy

[Julian Taylor]

On 17.02.2014 22:27, Sturla Molden wrote:
> Nathaniel Smith <njs at pobox.com> wrote:
>> Also, I'd be pretty wary of caching large chunks of unused memory. People
>> already have a lot of trouble understanding their program's memory usage,
>> and getting rid of 'greedy free' will make this even worse.
> 
> A cache would only be needed when there is a lot of computing going on, so
> one could set an "early expire date" on chached chunks. Anything older than
> a second or two could be returned. A cache would thus require a garbage
> collector thread.
> 


I was thinking of something much simpler, just a layer of pointer stacks
for different allocations sizes, the larger the size the smaller the
cache with pessimistic defaults.
e.g. the largest default cache layer is 128MB and with one or two
entries so we can cache temporal close operations like a + (c * b).
Maybe an age counter can be included that clears the largest old entry
if it hasn't been used for X allocations.

We can also monitor the allocations and make the maximum cache layer a
fraction of the biggest allocation that occurred during the runtime.

Devising a good scheme is probably tricky but we can start out with
something simple like limiting the cache to 512MB which will already
profit many cases while still being a acceptable amount of memory to
waste on a modern machine.