[Python-ideas] Iterable function calls and unpacking [was: PEP for issue2292, "Missing *-unpacking generalizations"]

[Joshua Landau]

On 15 July 2013 21:43, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
> On 15 July 2013 21:32, Joshua Landau <joshua at landau.ws> wrote:
>> I've moved this to a different thread, as I agree with Guido that it's
>> a different PEP.
>>
>> On 15 July 2013 21:06, Guido van Rossum <guido at python.org> wrote:
>>> On Mon, Jul 15, 2013 at 1:01 PM, Oscar Benjamin
>>> <oscar.j.benjamin at gmail.com> wrote:
>>>> On 15 July 2013 12:08, Joshua Landau <joshua.landau.ws at gmail.com> wrote:
>>>>> On 15 July 2013 11:40, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>>>>>
>>>>> In fact, I'd much like it if there was an iterable "unpacking" method
>>>>> for functions, too, so "chain.from_iterable()" could use the same
>>>>> interface as "chain" (and str.format with str.format_map, etc.). I
>>>>> feel we already have a good deal of redundancy due to this.
>>>>
>>>> I've also considered this before. I don't know what a good spelling
>>>> would be but lets say that it uses *args* so that you have a function
>>>> signature like:
>>>>
>>>>     def chain(*iterables*):
>>>>        for iterable in iterables:
>>>>             yield from iterable
>>
>> Personally some form of decorator would be simpler:
>>
>> @lazy_unpack()
>> def chain(*iterables):
>>     ...
>
> How would the above decorator work? It would need to exploit some new
> capability since this requires unpacking everything:

Yeah, it would just set an attribute on the function that tells Python
to special-case it. It's new functionality, just without new syntax.

My way also makes it so you can change old-style unpackers into
new-style iter-packers by doing "lazy_version =
lazy_unpack(original)".

>> (and it could theoretically work for mappings too, by just "bundling"
>> them; useful for circumstances where the mapping is left untouched and
>> just passed to the next function in line.)
>
> I don't understand. Do you mean to use it somehow for **kwargs?

Yup. A "lazy_kwargs" version that lets you do nothing more than pass
it along or convert to dict. In fact, for str.format you'd want a
"frozen non-copy" that lets you access elements of the original dicts
and kwargs without changing them too.

Say you have:

    def foo(*args, **kwargs):
        return bar(*modified_args, **kwargs, possibly=more_keywords)

there's little point in copying kwargs twice, is there?

Same idea with:

    "{foo}".format(**very_many_things)

>>>> And then if the function is called with
>>>>
>>>>     for line in chain(first_line, *inputfile):
>>>>         # do stuff
>>>>
>>> But how could you do this without generating different code depending
>>> on how the function you are calling is declared? Python's compiler
>>> doesn't have access to that information.
>>
>> You could simply make the code such that if has an unpack inside the
>> call it does a run-time check. Whilst this will be slower for the
>> false-positives, the number of times *args is pass-through (and thus
>> you save a redundant copy of the argument tuple) and *args is a simple
>> loop-once construct makes it plausible that those losses would be
>> outweighed.
>
> It probably would be better to have a specific syntax at the calling
> site since you probably want to know when you look at
> f(*infinite_iterator) whether or not infinite_iterator is going to be
> expanded.

True. "*args*" and "**kwargs**" are actually quite reasonable; *args*
for chain.from_iterable and **kwargs** for collections.ChainMap.

Some of the silly things you could do:

    (item for item in *iter_a*, *iter_b*) == itertools.chain(iter_a, iter_b)

    {default: foo, **mydict**}[default] === mydict.get({default: foo,
**mydict**}) === mydict.get(default, foo)

    "{SPARKLE}{HOUSE}{TANABATA TREE}".format(**unicode**) ===
"{SPARKLE}{HOUSE}{TANABATA TREE}".format_map(unicode)

    print("Hey, look!", *lines*, sep="\n") === print("Hey, look!",
"\n".join(lines), sep="\n")

    next((*iterable*, default)) == next(iterable, default)¹

Next PEP anyone? :D


¹ What's the point? Well, we wouldn't have *needed* the default
argument if it was that easy in the first place. Same with dict.get.


I also have changed my mind where I said:

> You'd also need to add a call to exhaust the iterator at the end of
> every function utilising this (transparently, probably) to make this
> have no obvious externally-visible effects. There would still be a
> call-order change, but that's much more minor.

because I obviously wasn't thinking when I said it.