[Python-ideas] Adding "Typed" collections/iterators to Python

[Nathan Rice]

> Not everything, no. The real criteria is to have solid use cases where
> a proposal clearly improves the language. Sometimes that's just
> obvious (e.g. supporting a popular new compression protocol),
> sometimes a PEP is enough to explain, other times real world
> experience on PyPI is the best option. You don't have *any* of those
> at this point (just some vague hand-waving), so publishing a PyPI
> package is the most obvious way to start acquiring more data (and to
> prove that there's even a version of the idea that can be taken beyond
> the hand-waving stage).

Couple things.

1. The "broadcasting" that people seemed to have latched on to is only
part of what I put forward, and I agree it is something that would
have to be done *correctly* to be beneficial.  I would have no issues
with providing a userspace lib to do this if type decorations were
included in homogeneous collections/iterables, as long as the
implementation of the decoration didn't suffer from some form of
"string failure" (string subclasses are basically worthless as methods
return strings, not an instance of the class).

2. A "type decorator" on homogeneous collections and iterables has a
lot of nice little benefits throughout the toolchain.

3. Being able to place methods on a "type decorator" is useful, it
solves issues like "foo".join() which really wants to be a method on
string collections.

4. I wanted to gauge people's feelings before I went through the steps
involved in writing a PEP.  I believe that is the right thing to do,
so I don't feel the "hand waving" comment is warranted.  I've already
learned people view collections that provide child object methods in
vector form as a very big change even if it is backwards compatible;
that is fine, I'm willing to shelve that if consensus is that people
aren't comfortable with it.

> What you seem to be asking for is a general purpose typed container
> factory along the following lines:

>    def typed_container(container_type, data_type):
>        class TypedContainer(container_type):
>            def __getattr__(self, attr):
>                data_type_attr = getattribute(data_type, attr)
>                if callable(data_type_attr):
>                    _result_type = type(self)
>                    def _broadcast(*args, **kwds):
>                        _result_type(data_type_attr(x, *args, **kwds)
> for x in self)
>                    return _broadcast
>                return data_type_attr
>        return TypedContainer

Something along those lines. Again I feel people have latched on to
one element of what I proposed here, to the detriment of the proposal
as a whole.

> I think it will have a lot of problems in practice (note that NumPy
> doesn't try to solve the broadcasting problem in general, just for a
> single specific data type), but, if the concept has any merit at all,
> that's certainly something that can be demonstrated quite adequately
> on PyPI.

This may be the case, thus my request for input.  I agree that a
thoughtful approach is prudent, however as I have stated, as it
currently stands the language would not be very supportive of an
add-on module that does this.  With PyPy, I could probably get my
hooks in deeply enough that I could make something really useful,
however given CPython is the current flavor du jour, I doubt it would
get much traction.  Kind of hard to sell someone on syntactic sugar
when they have to constantly wrap return values.

> To get a better idea of the level of evidence you're trying to reach
> if your suggestion is ever going to get anywhere, try taking a look at
> http://www.boredomandlaziness.org/2011/02/justifying-python-language-changes.html
> and http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html.

I agree that changes to syntax and commonly used modules that impact
how people interface with them should be carefully vetted.  Type
decorations on homogeneous collections/iterators are effectively
invisible in that perspective though; the main problem with them as I
see it is that it involves touching a lot of code to implement, even
if the actual implementation would be simple.

I appreciate your feedback,

Nathan