[Python-3000] my take on "typeclasses"

[Talin]

Guido van Rossum wrote:
> [Talin]
> 
>> Ordinarily I would agree with you on this point, and in fact I
>> wouldn't even bring it up at all, except for two data points:
>>
>> 1) The proposal to include generic function dispatch
>>    in Python 3000, based on explicit typing.
>>
>> 2) The proposal to remove all forms of type testing
>>    other than those based on isinstance() (in other words,
>>    removing callable, isSequence, isMapping, etc.)
>>    (I'm not including attribute testing here because its
>>    neither standardized nor systematic.)
> 
> 
> Note that these are *proposals*. IMO they are minority proposals and
> I'm doubtful that they will fly as you state them here. (For example,
> see my other post where I mention that generic/overloading functions
> will at best be used to augment traditional class-based dispatch; they
> will play a minor role at best.)

OK. I admit that much of the motivation for those statements is not 
about what you've stated, but a reaction to some of the statements that 
have been made on this list.

> My proposal of removing callable() wasn't intended to increase the
> number of isinstance() calls. Rather, it was intended to face paranoia
> head-on, by saying "just call it!" And (re.sub notwithstanding) APIs
> that depend on being able to distinguish between a callable and a
> non-callable are usually poor API design, and should be replaced by
> other approaches (like keyword arguments).

While it may be true that these are "usually" poor API design, I have 
provided several examples of popular, well-known and well-respected 
Python libraries that use exactly this feature, and for which simply 
calling and catching the exception is not an effective substitute.

I also feel that one can only judge design quality in context, 
specifically designs that would be considered "poor" for Java or C++ may 
be perfectly reasonable for Python. There are a lot of Python APIs that 
have a kind of "do what I mean" quality to them, and I don't see that as 
bad style per se; What *is* bad style is unpredictability and surprise.
A DWIM API can be well designed if the internal behavior exactly matches 
the average user's preconcieved expectations.

> You're way ahead of me here. You talk like this is a done deal. It
> isn't. And in fact I am strongly *against* any kind of "accidental
> paradigm shift" like this -- I'm doing my best to prevent those from
> happening. (There are some *intentional* paradigm shifts planned, but
> they are more about the behavior of some specific built-in types and
> operations such as files, strings and mappings, than about the very
> foundation of the type system.)

OK. I think where Phillip and I are headed is that we see the current 
paradigm as incomplete. I would like to see a system in which explicit 
typing and duck typing are both fully realized, and neither is "second 
class" as compared to the other.

While I think generic functions are cool, the discussion of how to 
design criteria for testing vs. duck types is something that ought to be 
dealt with regardless.

>> I think that I agree with the gist of Tomer's point.
> 
> Well, to most of us it's far from clear what he's trying to say...

Well, I won't claim to speak for him directly. But its well known that 
hierarchical type systems have limitations.

I wrote an article on Advogato a long time ago called "The Platypus 
Effect", which talks about how you have all your nice taxonomies set up, 
you know, mammals are the ones with fur and live birth, reptiles and 
birds lay eggs, and so on - and then along comes a platypus, a 
duck-billed, egg-laying, fur-bearing annoyance, which just *doesn't fit* 
anywhere in your scheme. And you can't just "patch" your taxonomy 
either, you end up having to refactor the whole thing.

> Maybe. Although generic/overloaded functions can easily use types
> Of course there's nothing to stop us from establishing a convention
> that allows one to use a single hasattr() call to test for such a
> thing -- or any number of other approaches whose surface API is a
> single call: isinstance(), hasattr(), implements(), all are at some
> level equivalent. So there's not that much new.
 >
> I think I would encourage proposals here to come up with a lightweight
> and reliable convention. It's okay to require that all sequences types
> must do something specific before they will be considered to be
> sequence by the new testing method -- because if we want to be able to
> reliably distinguish between sequences and mappings, we're *somehow*
> going to have to cut the tie. The convention should address existing
> de-facto categories such as iterable, sequence, file-like, etc., and
> also let users (and especially frameworks and libraries) define their
> own catagories.

I've been thinking about this quite a bit. What about a hierarchy of 
"concepts" that looks like this:

iterable:
    -- An item that can be iterated
    -- Test: hasattr( __iter__ )

bounded_iterable( iterable ):
    -- an iterable whose end is predictable in advance
    -- Test: iterable() + hasattr( __len__ )

indexable( bounded_iterable ):
    -- an iterable that allows random access to its elements
    -- Test: bounded_iterable() + hasattr( __getitem__ )

sequence( indexable ):
    -- an indexable in which the indexes are successive integers
    -- Test: isinstance( index_type, int )

mapping( indexable ):
    -- an indexable in which the indices are hashable values:
    -- Test: hashable( index_type )
    -- (Also test for immutable if its feasible)

So if all you want to do is iterate over an item, you don't test to see 
if its a sequence, you test to see if its iterable. And because you want 
to define your API in such a way as to eliminate surprise, you can use 
the term "iterable" so that everyone knows what you mean. In other 
words, if my function does one thing when passed an iterable, and 
something else when passed a non-iterable, the programmer can then know 
exactly what will happen when they pass their own data type in.

> Bill Janssen is proposing that the specific thing a type must do is
> inherit from some abstract base class.
> 
> Phillip Eby is countering that that isn't sufficient because he wants
> to be able to make up his own categories and apply these to existing
> types defined by 3rd party libraries.
> 
> It would be great if we had a solution that allowed either approach!

Yes indeed.

-- Talin