[Python-ideas] iterable.__unpack__ method

[Eric Snow]

On Sat, Feb 23, 2013 at 4:58 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> However, Alex's original idea of an "unpack protocol" (distinct from,
> but falling back to, the ordinary iterable protocol) definitely has at
> least a few points in its favour.
>
> 1. Iterating over heterogeneous types doesn't really make sense, but
> unpacking them does. A separate protocol lets a type support unpacking
> assignment without supporting normal iteration.

This is the big one for me.  And as masklinn pointed, it works the
other way too (keep an iterable from unpacking).

> 2. The protocol could potentially be designed to allow an *iterable*
> to be assigned to the star target rather than requiring it to be
> unpacked into a tuple. This could be used not only to make unpacking
> assignment safe to use with infinite iterators, but also to make it
> cheaper with large sequences (through appropriate use of
> itertools.islice in a custom container's __unpack__ implementation)
> and with virtual sequences like range() objects.

Cool, though it would be complicated when the star target isn't the last target.

> 3. As Alex noted, if a function that previously returned a 2-tuple
> wants to start returning a 3-tuple, that's currently a backwards
> incompatible change because it will break unpacking assignment. With
> an unpack protocol, such a function can return an object that behaves
> like a 3-tuple most of the time, but also transparently supports
> unpacking assignment to two targets rather than only supporting three.
>
> I would suggest a different signature for "__unpack__", though, built
> around the fact the star target can be used at most once, but in an
> arbitrary location:
>
>     def __unpack__(target_len, star_index=None):
>         """Unpack values into an iterable of the specified length.
>
>         If star_index is not None, the value at that index will be
>         an iterable representing the remaining contents that were not
>         unpacked.
>         """
>         ...

+1

>
> While I'm somewhere between +0 and +1 on the idea myself, there are
> some open questions/potential downsides to the idea:
>
> - this change increases the complexity of the language, by explicitly
> separating the concept of heterogeneous unpacking from homogeneous
> iteration. Those *are* two distinct concepts though, so the increased
> complexity may be justifiable.

Agreed.  Also, I expect that __unpack__ would be an opt-in API that
falls back to iteration.

> - for function parameter unpacking, would we allow "*args" to be an
> arbitrary iterable rather than promising that it is always a tuple? We
> would probably want to do so for consistency with unpacking assignment
> under this scheme, which may lead to some confusing error messages
> thrown by existing functions which expect it to always be a tuple.

This should be handled/proposed separately, though it is dependent on
an unpacking protocol.

> - we would want to document very clearly that it remains possible to
> bypass custom unpacking by explicitly calling iter(obj) rather than
> using obj directly.

An unpacking protocol would add power to the language.  We really
don't have a good way to separate unpacking from iteration.
Structured record types like namedtuples aren't exactly sequences, and
could make use of the new protocol.  (I'm not advocating for the
change to namedtuple itself.)

-eric