[Python-ideas] Temporary variables in comprehensions

Thu Feb 15 18:34:28 EST 2018

On Thu, Feb 15, 2018 at 2:13 AM, Jamie Willis <jw14896.2014 at my.bristol.ac.uk
> wrote:

>  These laws define behaviour that is expected equivalent by users;
>
> [x for x in xs] = xs
>

OK -- that's the definition...

> [f(x) for x in [x]] = f(x)
>

well, not quite:

[f(x) for x in [x]] = [f(x)]

Using x in two places where they mean different things makes this odd, but
yes, again the definition (of a list comp, and a length-1 sequence)

[g(y) for y in [f(x) for x in xs]] = [g(y) for x in xs for y in f(x)]
>

well, no. using two for expressions yields the outer product -- all
combinations:

In [*14*]: xs = range(3)

In [*15*]: [(x,y) *for* x *in* xs *for* y *in* xs]

Out[*15*]: [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1),
(2, 2)]

so the result is a length len(seq1)*len(seq(2)) list. Or in this case, a
len(xs)**2.

But nesting the comps applies one expression, and then the other, yielding
a length len(xs) list.

but you wrote:

[g(y) for x in xs for y in f(x)]

which I'm not sure what you were expecting, as f(x) is not a sequence
(probably)...

To play with your examples:

Define some functions that make it clear what's been applied:

In [*16*]: *def* f(x):

    ...:     *return* "f(*{}*)".format(x)

    ...:

In [*17*]: *def* g(x):

    ...:     *return* "g(*{}*)".format(x)

and a simple sequence to use:

In [*18*]: xs = range(3)

Now your examples:

In [*19*]: [x *for* x *in* xs]

Out[*19*]: [0, 1, 2]

In [*20*]: [f(x) *for* x *in* [x]]

Out[*20*]: ['f(5)']

In [*21*]: [g(y) *for* y *in* [f(x) *for* x *in* xs]]

Out[*21*]: ['g(f(0))', 'g(f(1))', 'g(f(2))']

OK -- all good f applied, then g, but then the last one:

In [*27*]: [g(y) *for* x *in* xs *for* y *in* f(x)]

Out[*27*]:

['g(f)',

 'g(()',

 'g(0)',

 'g())',

 'g(f)',

 'g(()',

 'g(1)',

 'g())',

 'g(f)',

 'g(()',

 'g(2)',

 'g())']

in this case, f(x) is returning a string, which is a sequence, so you get
that kind odd result. But what if f(x) was a simple scalr function:

In [*29*]: *def* f(x):

    ...:     *return* 2*x

Then you just get an error:

In [*30*]: [g(y) *for* x *in* xs *for* y *in* f(x)]

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-30-82ba3864654b> in <module>()

----> 1 [g(y) for x in xs for y in f(x)]

<ipython-input-30-82ba3864654b> in <listcomp>(.0)

----> 1 [g(y) for x in xs for y in f(x)]

TypeError: 'int' object is not iterable

The the nested comp is what is desired here:

In [*31*]: [g(y) *for* y *in* [f(x) *for* x *in* xs]]

Out[*31*]: ['g(0)', 'g(2)', 'g(4)']

Except you probably want a generator expression in the inner loop to avoid
bulding an extra list:

In [*33*]:  [g(y) *for* y *in* (f(x) *for* x *in* xs)]

Out[*33*]: ['g(f(0))', 'g(f(1))', 'g(f(2))']

So back to the OP's example:

In [*34*]: [f(x) + g(f(x)) *for* x *in* range(10)]

Out[*34*]:

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']

that is best done with comps as:

In [*36*]: [fx + g(fx) *for* fx *in* (f(x) *for* x *in* range(10))]

Out[*36*]:

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']

which really doesn't seem bad to me. And if the function names are longer
-- which they should be, you might want to use a temp as suggested earlier:

In [*41*]: fx = (f(x) *for* x *in* range(10))

In [*42*]: [x + g(x) *for* x *in* fx]

Out[*42*]:

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']

The truth is, comprehensions really are a bit wordy, if you are doing a lot
of this kind of thing (at least with numbers), you might be happier with an
array-oriented language or library, such as numpy:

In [*46*]: *import* *numpy* *as* *np*

In [*47*]: *def* f(x):

    ...:     *return* x * 2

    ...:

In [*48*]: *def* g(x):

    ...:     *return* x * 3

    ...:

    ...:

In [*49*]: xs = np.arange(3)

In [*50*]: f(xs) + g(f(xs))

Out[*50*]: array([ 0,  8, 16])

is pretty compact, and can be "optimized with a temp:

In [*51*]: fx = f(xs)

    ...: fx + g(fx)

    ...:

Out[*51*]: array([ 0,  8, 16])

pretty simple  isn't it?

So this gets back to -- does anyone have a suggestion for a syntax for
comprehensions that would make this substantially clearer, more readable,
or more compact?

I'm guessing not :-)

(the compact bit comes from having to type the "for x in" part twice -- it
does *feel* a bit unnecessary. which is why I like numpy -- no "for" at all
:-)

(I'm still trying to figure out why folks think map() or filter() help
here...)

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180215/ae90cb8e/attachment-0001.html>