[Python-ideas] Temporary variables in comprehensions

Thu Feb 15 20:37:33 EST 2018

I’d like to clarify that f(x) was indeed meant to be a sequence. As per
monad law:

*do* { y *<-* *do* { x *<-* m;

               f x

             }

     g y

   }

===

*do* { x *<-* m;

     y *<-* f x;

     g y

   }

I think you might have misunderstood the types of things; f: Function[a,
List[b]] and g: Function[b, List[c]]. m: List[a]

I’m still confused — f and g take a scale and return a list? Or take two
parameters, a scalar and a list???

Either way, doesn’t really fit with python comprehensions, which are more
or less expecting functions that except and return a single element (which
could be a triple, but I digress)

And the key point is that in python:

for x in seq1 for y in seq2

Does all combinations, like a nested for loop would.

But I DO think my old the fly write up went wrong, converting do-notation
to list comprehensions isn’t completely straightforward

Well, I don’t think I get the do notation at all....

The above is equivalent to

[g y | x <- m, y <- f x] in Haskell and the top is [g y | y <- [z |x <- m,
z <- f x]]

These have analogous structures in python;

[g(y) for x in m for y in f(x)] and [g(y) for y in [z for x in m for z in
f(x)]] (I think?)

Is this current Python or a new proposed syntax — ‘cause I can’t make any
sense of it in python.

Maybe define some f,g,m that behave as you expect, and see what python
does....

If I’ve not made another mistake, that **should* *now work?

Now on a phone, so can’t test, but I also can’t tell what you are expecting
the result to be.

Back to one of your examples:

[f(x) for x in [x]]

What does that mean???

for x in seq

Means iterate through seq, and assign each item to the name x.

If that seq has x in it — I’m not sure that is even legal python — the
scope in a comprehension confuses me.

But that is the equivalent is something like:

it= iter(seq)
while True:
    Try:
        x = next(it)
    Except StopIteration:
        Break

(Excuse the caps — hard to write code on a phone)

So not sure how x gets into that sequence before the loop starts.

-CHB

*From:* Chris Barker [mailto:chris.barker at noaa.gov <chris.barker at noaa.gov>]
*Sent:* 15 February 2018 23:34
*To:* jw14896.2014 at my.bristol.ac.uk
*Cc:* Evpok Padding <evpok.padding at gmail.com>; Python-Ideas <
python-ideas at python.org>
*Subject:* Re: [Python-ideas] Temporary variables in comprehensions

On Thu, Feb 15, 2018 at 2:13 AM, Jamie Willis <jw14896.2014 at my.bristol.ac.uk>
wrote:

 These laws define behaviour that is expected equivalent by users;

[x for x in xs] = xs

OK -- that's the definition...

[f(x) for x in [x]] = f(x)

well, not quite:

[f(x) for x in [x]] = [f(x)]

Using x in two places where they mean different things makes this odd, but
yes, again the definition (of a list comp, and a length-1 sequence)

[g(y) for y in [f(x) for x in xs]] = [g(y) for x in xs for y in f(x)]

well, no. using two for expressions yields the outer product -- all
combinations:

In [*14*]: xs = range(3)

In [*15*]: [(x,y) *for* x *in* xs *for* y *in* xs]

Out[*15*]: [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1),
(2, 2)]

so the result is a length len(seq1)*len(seq(2)) list. Or in this case, a
len(xs)**2.

But nesting the comps applies one expression, and then the other, yielding
a length len(xs) list.

but you wrote:

[g(y) for x in xs for y in f(x)]

which I'm not sure what you were expecting, as f(x) is not a sequence
(probably)...

To play with your examples:

Define some functions that make it clear what's been applied:

In [*16*]: *def* f(x):

    ...:     *return* "f(*{}*)".format(x)

    ...:

In [*17*]: *def* g(x):

    ...:     *return* "g(*{}*)".format(x)

and a simple sequence to use:

In [*18*]: xs = range(3)

Now your examples:

In [*19*]: [x *for* x *in* xs]

Out[*19*]: [0, 1, 2]

In [*20*]: [f(x) *for* x *in* [x]]

Out[*20*]: ['f(5)']

In [*21*]: [g(y) *for* y *in* [f(x) *for* x *in* xs]]

Out[*21*]: ['g(f(0))', 'g(f(1))', 'g(f(2))']

OK -- all good f applied, then g, but then the last one:

In [*27*]: [g(y) *for* x *in* xs *for* y *in* f(x)]

Out[*27*]:

['g(f)',

 'g(()',

 'g(0)',

 'g())',

 'g(f)',

 'g(()',

 'g(1)',

 'g())',

 'g(f)',

 'g(()',

 'g(2)',

 'g())']

in this case, f(x) is returning a string, which is a sequence, so you get
that kind odd result. But what if f(x) was a simple scalr function:

In [*29*]: *def* f(x):

    ...:     *return* 2*x

Then you just get an error:

In [*30*]: [g(y) *for* x *in* xs *for* y *in* f(x)]

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-30-82ba3864654b> in <module>()

----> 1 [g(y) for x in xs for y in f(x)]

<ipython-input-30-82ba3864654b> in <listcomp>(.0)

----> 1 [g(y) for x in xs for y in f(x)]

TypeError: 'int' object is not iterable

The the nested comp is what is desired here:

In [*31*]: [g(y) *for* y *in* [f(x) *for* x *in* xs]]

Out[*31*]: ['g(0)', 'g(2)', 'g(4)']

Except you probably want a generator expression in the inner loop to avoid
bulding an extra list:

In [*33*]:  [g(y) *for* y *in* (f(x) *for* x *in* xs)]

Out[*33*]: ['g(f(0))', 'g(f(1))', 'g(f(2))']

So back to the OP's example:

In [*34*]: [f(x) + g(f(x)) *for* x *in* range(10)]

Out[*34*]:

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']

that is best done with comps as:

In [*36*]: [fx + g(fx) *for* fx *in* (f(x) *for* x *in* range(10))]

Out[*36*]:

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']

which really doesn't seem bad to me. And if the function names are longer
-- which they should be, you might want to use a temp as suggested earlier:

In [*41*]: fx = (f(x) *for* x *in* range(10))

In [*42*]: [x + g(x) *for* x *in* fx]

Out[*42*]:

['f(0)g(f(0))',

 'f(1)g(f(1))',

 'f(2)g(f(2))',

 'f(3)g(f(3))',

 'f(4)g(f(4))',

 'f(5)g(f(5))',

 'f(6)g(f(6))',

 'f(7)g(f(7))',

 'f(8)g(f(8))',

 'f(9)g(f(9))']

The truth is, comprehensions really are a bit wordy, if you are doing a lot
of this kind of thing (at least with numbers), you might be happier with an
array-oriented language or library, such as numpy:

In [*46*]: *import* *numpy* *as* *np*

In [*47*]: *def* f(x):

    ...:     *return* x * 2

    ...:

In [*48*]: *def* g(x):

    ...:     *return* x * 3

    ...:

    ...:

In [*49*]: xs = np.arange(3)

In [*50*]: f(xs) + g(f(xs))

Out[*50*]: array([ 0,  8, 16])

is pretty compact, and can be "optimized with a temp:

In [*51*]: fx = f(xs)

    ...: fx + g(fx)

    ...:

Out[*51*]: array([ 0,  8, 16])

pretty simple  isn't it?

So this gets back to -- does anyone have a suggestion for a syntax for
comprehensions that would make this substantially clearer, more readable,
or more compact?

I'm guessing not :-)

(the compact bit comes from having to type the "for x in" part twice -- it
does *feel* a bit unnecessary. which is why I like numpy -- no "for" at all
:-)

(I'm still trying to figure out why folks think map() or filter() help
here...)

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180215/2df26da2/attachment-0001.html>