[Python-3000] callable()

[Talin]

Greg Ewing wrote:
> Guido van Rossum wrote:
> 
> 
>>But I'm not proposing to use hasattr(obj, '__call__'). I'm basically
>>saying that the callable test has very little to do with whether the
>>later call will succeed, because we have no way to test the signature.
> 
> 
> I don't think that's needed for the sort of things people
> want callable() for. I think they want it for the purpose
> of implementing type-dependent APIs -- similar in spirit
> to doing different things depending on whether you're
> passed a sequence or not, etc.

I think this is exactly correct. Those who want to retain callable() 
[including myself] aren't looking for an ironclad guarantee that calling 
a given object will succeed - they merely want to check to see if the 
object is "of callable type", in other words, is it the roughly sort of 
object that one would expect to be able to call.

I think part of the reason that this debate has gone on so long is that 
it's really a debate about the nature of Python as a scripting language.

Every so-called 'scripting' language maintains a balance between 
strictness and forgiveness. On the one extreme, there are languages such 
as Lua, where accessing an undefined variable is silently ignored, or 
Javascript, where exceptions that would otherwise be fatal errors are 
merely logged as warnings. On the other hand, there is a long tail of 
increasingly bondage-and-dominance-oriented languages with ever more 
strict checking of the rules.

As I am sure that all of us have discovered, the 'looser' language rules 
can greatly increase the speed of rapid prototyping. My own experience 
is that I can code Java about twice as fast as C++, and Python about 
twice as fast as Java, for equivalent functionality.

However, these looser rules have their own costs, which I won't go into 
detail about. Certainly the strict languages create lots more 
opportunities for compile-time optimization. As to whether the strict 
languages are more reliable overall, that's an open question - while its 
likely that the the programs written in such languages are more provably 
correct, the very strictness of the language's own self-checking can 
itself become a source of, um, programs that don't work when they should.

There's a common architectural pattern seen in both the Python standard 
library as well as many user-written applications, which I will call the 
"rough protocol" pattern (you might also think of it as "duck 
protocols"). The pattern is simple: Objects which conform to a given 
interface need not conform to it in theoretical exactitude, as would be 
required in a language such as C# or C++. Instead, objects only need 
conform to an interface so far as it is practical and convenient to do so.

As an example, I can create a "file-like" object, without having to 
support all of the various little methods and subtle behaviors of "file".

The reason these rough protocols are of such benefit is that you save a 
great deal of time - you can avoid having to track down and implement 
all the little subtle fiddly bits of the given interface that would 
otherwise be needed to make the compiler happy, but which you don't 
actually plan to use.

One of the types of rough protocols that we often see is one in which an 
object is passed as an argument to a function, and the function attempts 
to classify the object according to some simple criteria into one of 
several recognized types, and then take action based on that categorization.

You see this all the time with things like "if X is a sequence, then do 
this with it, otherwise do that".

Now, this violates all kinds of theoretical considerations such as 
"well, what if X only vaguely resembles a sequence', or 'what if X looks 
like a sequence, but really isn't'. One can take the formal definition 
of 'sequence' and generate an endless series of mutations and 
permutations of that definition which may or may not work for a given 
function (or even make sense for that matter).

The practical answer to all of this is that such contrivances hardly 
ever occur in real code. As programmers, we have common-sense 
expectations of what is a sequence, what is a map, what is a function, 
and so on. Those are the primitive concepts with which we assemble our 
mental designs before putting them down on disk.  As soon as a class 
starts to stray too far outside the envelope of those expectations, we 
tend to call it something besides 'sequence' or 'map' or whatever.

Rough protocols don't need to know exactly the type of object or what 
its exact method signatures are. It only needs to know enough about the 
object to decide what common-sense category to place it in.

Rough protocols *aren't* guaranteed to be able to inspect the object 
thoroughly enough to know whether the category test is valid - it 
assumes that the user is smart enough to only pass in objects for which 
the classification test makes sense. If you happen to pass in an object 
that doesn't fit in any category, or is categorized wrongly, then that's 
a bug in your code and you should go fix it.

So for example, when the documentation for 'sort' says that the 'key' 
argument can be a function that returns the key value, that doesn't mean 
that you can pass in an arbitrary function such as 'file.open' and 
expect it to magically work.

So, the point of all this is that while callable() isn't at all useful 
for the kind of formal proof-that-it-will-not-blow-up-horribly kind of 
inspection, it is very useful as a rough category test. In fact that's 
pretty much all I have ever used it for, and I am sure that many others 
would agree.

In fact, I would go so far as to say that the kind of "exact parameter 
signature" test being advocated would be *less* useful, simply because 
it would take more time to code to get the same effect.

-- Talin