[Python-ideas] Adding __getter__ to compliment __iter__.

[Ron Adam]

[May be a duplicate... access to news server acting up.]

On 07/16/2013 08:01 AM, Nick Coghlan wrote:
 > On 16 July 2013 22:28, Joshua Landau 
<joshua at landau.ws> wrote:
 >> On 16 July 2013 11:21, Oscar Benjamin 
<oscar.j.benjamin at gmail.com> wrote:
 >>> On 16 July 2013 07:50, Nick Coghlan 
<ncoghlan at gmail.com> wrote:
 >>>
 >>> If people are using sum() to concatenate lists then this should be
 >>> taken not as evidence that a new solution needs to be found but as
 >>> evidence that chain is not sufficiently well-known. The obvious
 >>> solution to that is not to implement a new protocol but to make the
 >>> existing solution more well known i.e. move chain.from_iterable to
 >>> builtins and rename it (the obvious choice being concat).
 >>
 >> You could wait for PEP 448, which will let you use [*sublist for
 >> sublist in list_to_be_flattened].
 >
 > Ah, true, I forgot about that. Too many interesting things going on
 > for me to keep track of everything :)
 >
 > In effect, PEP 448 goes further than making chain a builtin: it gives
 > it syntax! With PEP 448, the generator expression:
 >
 >      (*itr for itr in iterables)
 >
 > would be equivalent to either of the current:
 >
 >      itertools.chain(*iterables)
 >      itertools.chain.from_iterable(iterables)
 >
 > That's pretty cool. It also means I can go back to happily ignoring
 > the sum threads :)
 >
 > Cheers,
 > Nick.
 >
 > P.S. Something about this should probably be added to the rationale
 > section of PEP 448

I played around with trying to find something that would work like the 
example Nick put up and found out that the different python types are not 
similar enough in how they do things to make a function that takes a method 
or other operator work well.

What happens is you either end up with widely varying results depending on 
how the methods are implemented on each type, or an error because only a 
few methods are very common on all types.  Mostly introspection methods.

I believe this to be stronger underlying reason why functions like reduce 
and map were removed. And it's also a good reason not to recommend 
functions like sum() for things other than numbers.

To use functions similar to that, you really have to think about what will 
happen in each case because the gears of the functions and methods are not 
visible in the same way a comprehension or generator expression is.

It's too late to change how a lot of those methods work and I'm not sure it 
will still work very well.

One of the most consistent protocols python has is the iterator and 
generator protocols.  The reason they work so well is that they need to 
interface with for-loops and nearly all containers support that.

examples...

 >>> a = [1,2,3]
 >>> iter(a)
<list_iterator object at 0x7f3bc9306e90>

 >>> b = (1,2,3)
 >>> iter(b)
<tuple_iterator object at 0x7f3bc9306f10>

 >>> c = {1:2, 3:4}
 >>> iter(c)
<dict_keyiterator object at 0x7f3bc93cf6d8>

 >>> d = {1, 2, 3}
 >>> iter(d)
<set_iterator object at 0x7f3bc9380500>

 >>> e = "123"
 >>> iter(e)
<str_iterator object at 0x7f3bc9306e90>

And is why chain is the recommended method of joining multiple containers. 
  This really only addresses getting stuff OUT of containers.

PEP 448's * unpacking in comprehensions helps with the problem of putting 
things into containers.  But that isn't the PEP's main point.



What I'm thinking of is the inverse operation of an iter.  Lets call it a 
"getter".

You would get a getter the same way you get an iter.

     g = getter(obj)

But instead of __next__ or send() methods, it would have an iter_send(), or 
isend() method.  The isend method would takes an iter object, or an object 
that iter() can be called on.

The getter would return either the object it came from, or a new object 
depending on weather or not it was created from a mutable or immutable obj.


Mutable objects...

     g = getter(A)     # A needs a __getter__ method.
     A = g.isend(B)

     A += B            # extend


Mutable objects...

     g = getter(A)
     C = g.isend(B)

     C = A + B         # join

The point, is to have something that works on many types and is as 
consistent in how it's defined as the iter protocol.  Having a strict and 
clear definition is a very important!

The internal implementation of a getter could do a direct copy to make it 
faster, like slicing does, but that would be a private implementation detail.

They don't replace generator expressions or comprehensions.  Those 
generally will do something with each item.

Functions like extend() and concat() could be implemented with 
*getter-iters*, and work with a larger variety of objects with much less 
work and special handling.

      def extend(A, B):
         return getter(A).isend(B)

      def concat(A, B):
         """ Extend A with multiple containers from B. """
         g = getter(A)
         if g.isend() is not A:
             raise TypeError("can't concat immutable arg, use merge()")
         for x in B:
            g.isend(x)
         return A

      def merge(A, B):
         """ Combine A with containers in B, return new container. """
         a = list(A)
         g = getter(a)
         for x in B:
            g.isend(x)
         return type(A)(a)


Expecting many holes to be punched in this idea ...
    But hope not too many.  ;-)

       Ron