Question regarding Higher-Order-Programming in Python

[Peter Otten]

Mark Fink wrote:

> I am about to learn Higher-Order-Programming with Lisp, Haskell, and
> Python. Some people who have gone completely out of their mind call
> this FP.
> 
> In Haskell I learned that when I use map on a list it starts nesting
> as soon as I start adding elements. If I do not like the nesting I use
> ConcatMap.
> 
> In Python I have a similar scenario. I have a generator which creates
> some combinatorics of a input dictionary. The list starts nesting. Now
> I could use reduce(concat, ...) which would be the correct thing from
> a functional perspective. But from a resource utilization perspective
> it is not a good idea since the reduce is executing the generator.
> 
> I tried to illustrate this using a small example (the often in
> combinatorics the real thing would be much bigger that is why I need
> to use a generator):
> 
>>>> from operator import itemgetter, concat
>>>> import itertools as it
>>>> from functools import partial
>>>>
>>>> dims = {'number': [1,2,3], 'letter': ['a', 'b'], 'special': ['+', '-']}
>>>> dims
> {'special': ['+', '-'], 'number': [1, 2, 3], 'letter': ['a', 'b']}
>>>> def get_products(keys):
> ...     # helper to get products from keys in the following form:
> ...     # [('bold', True), ('color', 'black')]
> ...     values  = itemgetter(*keys)(dims)
> ...     product = it.product(*values)
> ...     return map(partial(zip, keys), product)
> ...
>>>> comb = it.combinations(dims, 2)
>>>> comb_l = list(comb)
>>>> comb_l
> [('special', 'number'), ('special', 'letter'), ('number', 'letter')]
>>>> res = map(get_products, comb_l)
>>>> res
> [[[('special', '+'), ('number', 1)], [('special', '+'), ('number',
> 2)], [('special', '+'), ('number', 3)], [('special', '-'), ('number',
> 1)], [('special', '-'), ('number', 2)], [('special', '-'), ('number',
> 3)]], [[('special', '+'), ('letter', 'a')], [('special', '+'),
> ('letter', 'b')], [('special', '-'), ('letter', 'a')], [('special',
> '-'), ('letter', 'b')]], [[('number', 1), ('letter', 'a')],
> [('number', 1), ('letter', 'b')], [('number', 2), ('letter', 'a')],
> [('number', 2), ('letter', 'b')], [('number', 3), ('letter', 'a')],
> [('number', 3), ('letter', 'b')]]]  # the resulting list is nested one
> level to deep caused by the map(get_products, ..
>>>>
> 
> My problem is that I want to get single elements from the generator
> like [('special', '+'), ('number', 1)]. But this does not work because
> the list is now nested to deep.

Like this?

>>> [(k, v) for k, vv in dims.iteritems() for v in vv]
[('special', '+'), ('special', '-'), ('number', 1), ('number', 2), 
('number', 3), ('letter', 'a'), ('letter', 'b')]

But these are just glorified for-loops, so:

>>> list(chain.from_iterable(starmap(product, izip(izip(dims.iterkeys()), 
dims.itervalues()))))
[('special', '+'), ('special', '-'), ('number', 1), ('number', 2), 
('number', 3), ('letter', 'a'), ('letter', 'b')]

Peter