[Tutor] Tutor Digest, Vol 53, Issue 99

[kinuthiA muchanE]

On Tue, 2008-07-29 at 01:12 +0200, tutor-request at python.org wrote:
> Message: 3
> Date: Mon, 28 Jul 2008 13:26:13 -0500
> From: "Daniel Sarmiento" <dsarmientos at gmail.com>
> Subject: Re: [Tutor] Memory error - how to manage large data sets?
> To: tutor at python.org
> Message-ID:
>         <d356a5240807281126t6bd27dc5y6ef411305a9f5f5a at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> Hi
> 
> I tried to run your code and checked (with top) the memory ussage and
> it uses more than 2 Gb of memory.
> 
> I tried to modify the code a little bit to use less memory and came up
> with this:
> 
> fib = {0:0,1:1}
> even = []
> 
> def fibonacci(x,y):
>    return x+y
> 
> for j in xrange (2,1000000):
>     i = fib[j-1] + fib[j-2]
>     if i % 2 == 0:
>         even.append(i)
>     fib = {j-1:fib[j-1], j:i}
> 
> total = reduce(fibonacci,even)
> print total
> 
> First, I replaced range with xrange.
> I figured that you only need the last two values in the fibonnaci
> series to calculate the next value, so I replaced the fib list with a
> dictionary to only store the last two values instead of the whole
> series.
> 
> It looks like the progam still hangs and I did not notice any memory
> imrovements when running it with 1 000 000
> 
> Am I wrong thinking that the modifications I made help use less
> memory?
> 
I have realised that when you need to work with large numbers, lists are
slow and tend to gobble up bytes. Another thing try to be as simple as
possible :). This is how I would approach the problem, without using
lists. It produces the result instantly, even for large numbers like a
100, 000, 000.

def fibonacci()
    a = 0
    b = 1
    evenTotal = 0
    while a < 100000000:
        a,b = b,a+b
        if a%2 == 0:
    	    evenTotal += a
    print evenTotal

Does this help?
Kinuthia...