List comprehension vs generator expression memory allocation

[Dave Angel]

Jon Clements wrote:
> On 20 Sep, 14:35, candide <cand... at free.invalid> wrote:
>   
>> Let's code a function allowing access to the multiples of a given
>> integer (say m) in the range from a to b where a and b are two given
>> integers. For instance, with data input
>>
>> a,b,m, 42, 5
>>
>> the function allows access to :
>>
>> 20 25 30 35 40
>>
>> Each of the following two functions mult1() and mult2() solves the
>> question :
>>
>> # -----------------------------------------
>> def mult1(a,b,m):
>>     return (x for x in range(a,b)[(m-a%m)%m:b:m])
>>
>> def mult2(a,b,m):
>>     return range(a,b)[(m-a%m)%m:b:m]
>> # -----------------------------------------
>>
>> mult2() function returns a list and obviously mult2() needs Python to
>> allocate memory for this list. What I was wondering is if the same might
>> be said about mult1(). More precisely, does Python allocate memory for
>> the whole target list range(a,b)[m-a%m:b:m]?
>>     
>
> Yes - range always creates a list (in the 2.x series) -- mult1 creates
> a list, then returns a generator, so list(mult1(a,b,m)) is identical
> to mult2(a,b,m).
>
> Jon.
>
>   
First, you're clearly using python 2.x, since in version 3, you'd get an 
error in each of the definitions, doing the slice on a generator.

Both of those functions create the identical list (after first creating 
a bigger one!).  Then mult1() creates a generator object that owns the 
list, and passes back the generator object.  The list isn't freed till 
the calling code releases the generator object.

The code in mult2() just passes the list back directly, and it'll get 
freed when the calling code stops using it (eg. reuses the attribute 
which references it).

So mult1() adds extra complexity without saving any memory.  If you want 
to save memory, try (not thoroughly tested):

def mult3(a, b, m):
    start =  m*((a-1)/m + 1)
    end = m*((b-1)/m + 1)
    return xrange(start, end, m)