[Numpy-discussion] iterate over multiple arrays

[Robert Kern]

On Tue, Sep 13, 2011 at 01:53, David Froger <david.froger at gmail.com> wrote:
>
> Thank you Olivier and Robert for your replies!
>
> Some remarks about the dictionnary solution:
>
> from numpy import *
>
> def f(arr):
>     return arr + 100.
>
> arrs = {}
> arrs['a'] = array( [1,1,1] )
> arrs['b'] = array( [2,2,2] )
> arrs['c'] = array( [3,3,3] )
> arrs['d'] = array( [4,4,4] )
>
> for key,value in arrs.iteritems():
>    arrs[key] = f(value)
>
> 1. about the memory
> Memory is first allocated with the array functions:
>    arrs['a'] = array( [1,1,1] )
>    arrs['b'] = array( [2,2,2] )
>    arrs['c'] = array( [3,3,3] )
>    arrs['d'] = array( [4,4,4] )
>
> Are there others memory allocations with this assignemnt:
>    arrs[key] = f(value)
> or is the already allocated memory used to store the result of f(value)?
>
> In other words, if I have N arrays of the same shape, each of them costing
> nbytes of memory, does it use N*nbytes memory, or 2*N*bytes?

Temporarily, yes, for all of the variations mentioned. When the
expression "f(value)" is evaluated, both the result array and the
input array will exist simultaneously in memory. Once the assignment
happens, the original input array will be destroyed and free up the
memory. There is no difference memory-wise between assigning into a
dictionary or assigning to a variable name.

Sometimes, you can write your f() such that you just need to do

  f(value)

and have the value object modified in-place. In that case, there is no
need to reassign the result to a variable or dictionary key.

> I think this is well documented on the web and I can find it....
>
> 2. about individual array
> The problem is that now, if one want to use a individual array, one have now to
> use:
>    arrs['a']
> instead of just:
>    a
> So I'm sometime tempted to use locals() instead of arrs...

Seriously, don't. It makes your code worse, not better. It's also
unreliable. The locals() dictionary is meant to be read-only (and even
then for debugger tooling and the like, not regular code), and this is
sometimes enforced. If you want to use variable names instead of
dictionaries, use them, but write out each assignment statement.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco