On Thu, Feb 15, 2018 at 2:13 AM, Jamie Willis <jw14896.2014@my.bristol.ac.uk> wrote:
 These laws define behaviour that is expected equivalent by users; 

[x for x in xs] = xs

OK -- that's the definition...
 
[f(x) for x in [x]] = f(x)

well, not quite:

[f(x) for x in [x]] = [f(x)]

Using x in two places where they mean different things makes this odd, but yes, again the definition (of a list comp, and a length-1 sequence)

[g(y) for y in [f(x) for x in xs]] = [g(y) for x in xs for y in f(x)]

well, no. using two for expressions yields the outer product -- all combinations:

In [14]: xs = range(3)


In [15]: [(x,y) for x in xs for y in xs]

Out[15]: [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2)]


so the result is a length len(seq1)*len(seq(2)) list. Or in this case, a len(xs)**2.

But nesting the comps applies one expression, and then the other, yielding  a length len(xs) list.

but you wrote:

[g(y) for x in xs for y in f(x)]

which I'm not sure what you were expecting, as f(x) is not a sequence (probably)...

To play with your examples:

Define some functions that make it clear what's been applied:

In [16]: def f(x):

    ...:     return "f({})".format(x)

    ...: 


In [17]: def g(x):

    ...:     return "g({})".format(x)


and a simple sequence to use:

In [18]: xs = range(3)


Now your examples:

In [19]: [x for x in xs]

Out[19]: [0, 1, 2]


In [20]: [f(x) for x in [x]]

Out[20]: ['f(5)']


In [21]: [g(y) for y in [f(x) for x in xs]]

Out[21]: ['g(f(0))', 'g(f(1))', 'g(f(2))']


OK -- all good f applied, then g, but then the last one:

In [27]: [g(y) for x in xs for y in f(x)]

Out[27]: 

['g(f)',

 'g(()',

 'g(0)',

 'g())',

 'g(f)',

 'g(()',

 'g(1)',

 'g())',

 'g(f)',

 'g(()',

 'g(2)',

 'g())']


in this case, f(x) is returning a string, which is a sequence, so you get that kind odd result. But what if f(x) was a simple scalr function:

In [29]: def f(x):

    ...:     return 2*x


Then you just get an error:

In [30]: [g(y) for x in xs for y in f(x)]

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-30-82ba3864654b> in <module>()

----> 1 [g(y) for x in xs for y in f(x)]


<ipython-input-30-82ba3864654b> in <listcomp>(.0)

----> 1 [g(y) for x in xs for y in f(x)]


TypeError: 'int' object is not iterable


The the nested comp is what is desired here:

In [31]: [g(y) for y in [f(x) for x in xs]]

Out[31]: ['g(0)', 'g(2)', 'g(4)']


Except you probably want a generator expression in the inner loop to avoid bulding an extra list:


In [33]:  [g(y) for y in (f(x) for x in xs)]

Out[33]: ['g(f(0))', 'g(f(1))', 'g(f(2))']


So back to the OP's example:

In [34]: [f(x) + g(f(x)) for x in range(10)]

Out[34]: 

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']


that is best done with comps as:

In [36]: [fx + g(fx) for fx in (f(x) for x in range(10))]

Out[36]: 

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']


which really doesn't seem bad to me. And if the function names are longer -- which they should be, you might want to use a temp as suggested earlier:

In [41]: fx = (f(x) for x in range(10))


In [42]: [x + g(x) for x in fx]

Out[42]: 

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']


The truth is, comprehensions really are a bit wordy, if you are doing a lot of this kind of thing (at least with numbers), you might be happier with an array-oriented language or library, such as numpy:

In [46]: import numpy as np


In [47]: def f(x):

    ...:     return x * 2

    ...: 


In [48]: def g(x):

    ...:     return x * 3

    ...: 

    ...: 


In [49]: xs = np.arange(3)


In [50]: f(xs) + g(f(xs))

Out[50]: array([ 0,  8, 16])


is pretty compact, and can be "optimized with a temp:

In [51]: fx = f(xs)

    ...: fx + g(fx)

    ...: 

Out[51]: array([ 0,  8, 16])


pretty simple  isn't it?

So this gets back to -- does anyone have a suggestion for a syntax for comprehensions that would make this substantially clearer, more readable, or more compact?

I'm guessing not :-)

(the compact bit comes from having to type the "for x in" part twice -- it does *feel* a bit unnecessary. which is why I like numpy -- no "for" at all :-)

(I'm still trying to figure out why folks think map() or filter() help here...)

-CHB


--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov