[Image-SIG] Workarounds for MemoryErrors
ben at ekran.org
Wed Jun 24 01:50:28 CEST 2009
I have tried the code, the Image.new() function just bails due to
I've now enabled some extra swap, giving me about 20GB of free swap+mem.
Still python bails on Image.new(). Looking at memory usage it seems
python does not even try to allocate, just bails before any memory is used.
Does PIL have some intelligence to bail when some pre-test shows there
is not enough available memory? If so is there a way to override it?
I just tried allocating a 32000x32000 RGBA image, which leaves this:
total used free shared buffers cached
Mem: 4065824 4037288 28536 0 3376 6960
-/+ buffers/cache: 4026952 38872
Swap: 15623164 112048 15511116
So very little swap is actually being used, and I'm guessing not by PIL.
creating an image much larger than 32000x32000 bails due to memoryerror.
Seems the memory usage check does not include swap.
Kevin Cazabon wrote:
> Quite a number of years back I processed a 1.5GB image on a machine with
> 256MB of RAM (the rest virtual memory - 48x96" poster at 304
> pixels/inch). It took days, but it worked. The software actually was
> pretty inefficient too, so I was probably using well over 2GB of memory
> space at a time.
> I think your image is about 19GB (at 24bits/pixel)... not that different
> of a ratio than what I did. Have you tried it? Besides ensuring you
> have lots of VM available, it might just work as-is. Memory is cheap
> these days too... probably cheaper than the hours of programming a
> On 22 Jun 2009, at 12:52, B. Bogart wrote:
>> Hello all,
>> I want to create a very large RGBA image (96000x72000 pixels).
>> I have 4GB of RAM.
>> Is there an easy way of getting around this error by having PIL only
>> allocate one section of the image at a time?
>> If PIL does not have any internal trick to work with large images then
>> I'll have to make 4+ smaller images one at a time, but then I'm not sure
>> how I could combine them without needing to allocate a memory chunk for
>> the whole image.
>> Otherwise I suppose I'll have to try with some other language, perhaps
>> C/SDL, though a quick calculation seems to show that such a large RGBA
>> image is just unworkable. Is there some way of using disk space rather
>> than memory? Does not matter if it is slow, just that it is possible!
>> Any advice?
>> Image-SIG maillist - Image-SIG at python.org
More information about the Image-SIG