[Python-ideas] The case against static type checking, in detail (long)

[Talin]

(This is a fragment of an email that I sent to Guido earlier, I mention 
this here so that Guido can skip reading it. Of course, I recognize that 
most people here already know all this - but since it relates to the 
recent discussion about the value of type checking, I'd like to post it 
here as a kind of "manifesto" of why Python is the way it is.)

Strongly-typed languages such as C++ and Java require up-front 
declaration of everything. It is the nature of such languages that there 
is a lot of cross-checking in the compiler between the declaration of a 
thing and its use. The idea is to prevent programmer errors by insuring 
internal consistency.

However, we find in practice that much of the programmer's effort is 
spent in maintaining this cross-checking structure. To use a building 
analogy, a statically-typed program is like a truss structure, where 
there's a complex web of cross-braces, and a force applied at any given 
point is spread out over the whole structure. Each time such a program 
is modified, the programmer must partially dismantle and then 
re-assemble the existing structure. This takes time.

It also clutters the code. Reading the source code of a program written 
in a statically typed language reveals that a substantial part of the 
text serves only to support the compile-time checking of definitions, or 
provides visual redundancy to aid the programmer in connecting two 
aspects of the program which are defined far apart.

An example of what I mean is the use of variable type declarations - 
even in a statically typed language, it would be fairly easy for the 
compiler to automatically infer most variable types if the language were 
designed that way; The fact that the programmer is required to manually 
specify these types serves as an additional consistency check on the code.

However, time spend serving the needs of these consistency checks is 
time away from actually serving the functional purpose of the code. 
Programmers in Python, on the other hand, not only need not worry about 
type declarations, they also spend much less time worrying about 
converting from one type to another in order to meet the constraints of 
a particular API.

This is one of the reasons why I can generally write Python code about 4 
times as fast as the C++ equivalent.

(Understand that this is coming from someone who loves working in C++ 
and Java and has used them daily for the last 15 years. At the same 
time, however, I also enjoy programming in Python and I recognize that 
each language has their strengths.)

There is also the question of how much static typing helps improve 
program reliability.

In statically typed languages, there are two kinds of ways that types 
are used. In languages such as C and Pascal, the type declarations serve 
primarily as a consistency check. However, in C++ template 
metaprogramming, and in languages like Haskell, there is a second use 
for types, which is to provide a kind of type calculus or type 
inferencing, which gives additional expressive power to the language. 
C++ templates can act as powerful code generators, allowing the 
programmer to program in ever higher levels of abstraction and express 
the basic ideas even more succinctly and clearly than before.

In a rapid-prototyping environment, the second use of types can be a 
major productivity win; However I would argue that the first use of 
types, consistency checking, is less beneficial, and is often more of a 
distraction to the programmer than a help. Yes, static type checking 
does detect some errors; But it also causes errors by making the code 
larger and more wordy, because that the programmer cannot hold large 
portions of the program in their mind all at once, which can lead to 
errors in overall design. It means the programmer spends more time 
thinking about the behavior of individual variables and less about the 
algorithm as a whole.

At this point, I want to talk about a related matter, another 
fundamental design aspect of Python which I call "decriminalization of 
minor errors".

An example of this is illustrated by the recent discussion over string 
slicing. As you know, when you attempt to index a string with a slice 
object that extends outside of the bounds of the string, the range is 
silently truncated.

Some argued that Python should be more strict, and report an error when 
this occurs - but instead, it was reaffirmed that the current behavior 
is correct. I would agree that this current behavior is the more 
Pythonic, and is part of a general pattern, which I shall attempt to 
describe:

To "decriminalize" an error means to find some alternative 
interpretation of the programmer's intent that produces a non-error 
result. That is, given a syntactical construct, and a choice of several 
interpretations of what that construct should mean, attempt to pick an 
interpretation that, when executed, does not produce an error.

In the design of the Python language, it is a regular practice to 
decriminalize minor errors, provided that the alternative interpretation 
can meet some fairly strict criteria: That it is useful, intuitive, 
reasonable, and pedagogically sound.

Note that this is a much more conservative rule than that used by 
languages such as Rexx, Javascript, and Perl, languages which make 
"heroic efforts" to bend the interpretation of an operation to a 
non-error result. Python does not do this.

Nor is decriminalizing errors isn't the same as ignoring errors. Errors 
are still, and should be, enforced vigorously and reported. The 
distinction is that decriminalizing an error results in code that 
produces a useful, logical result, whereas ignoring errors results in 
code that produces garbage or nothing. Decriminalization comes about 
when we broaden our definitions of what is the correct result of a given 
operation.

A couple of other examples of decriminalization:

1) there are languages in which the only valid argument for an 'if' 
statement is a boolean. Attempts to say "if 0" are errors. In Python we 
relax that rule, allowing any type to be used as the argument to an 
if-statement. We do this by having a broader interpretation of what it 
means to test an object for 'trueness', and allow 'trueness' to be 
implied by 'non-emptiness'.

2) Duck-typing is a decriminalization of the error that polymorphic 
types are required to inherit from a common interface. It also 
decriminalizes "missing methods", as long as those methods are never 
called. Again, this is due to having a broader interpretation of 
'polymorphism'.

(In fact, this aspect of Python is so fundamental, that I think that it 
deserves its own acronym alongside TOOWTDI and others, but I can't think 
of a short, pithy description of it. Maybe IOANEIR - "Interpret 
operations as non-errors if reasonable.")

Both static typing and decriminalization serve the same end - telling 
the programmer "don't sweat the small stuff". Both are very helpful and 
powerful, because they allow programmers to spend much less time 
worrying about minor error cases, things that would have to be checked 
for in C++. Python code is simply more *concise* than the C++ 
equivalent, yet it achieves this without being terse and cryptic, 
because the text of a Python program more closely embodies the "essence" 
of an algorithm, uncluttered by other agendas.

The price we pay for this, of course, is that sometimes errors show up 
much later (like, after ship) than they would have otherwise. But unit 
testing can catch a lot of the same errors.

And in many cases, the seriousness of such errors depends on what we 
mean by "ship". It's one thing to discover a fatal error after you've 
pressed thousands of CDs and shipped them all over the world; It's a 
much different matter if the program has the ability to automatically 
update itself, or is downloaded from some kind of subscription model 
such as a package manager.

In many environments, it is far more important to get something done 
quickly and validate the general concept, than it is to insure that the 
code is 100% correct. In other words, if it would take you 6 months to 
write it in a statically typed language, but only 2 months to write it 
in a dynamic language - well, that's 4 extra months you have to write 
unit tests and make sure it's right! And in the mean time, you can have 
real users banging on the code and making sure of something that is far 
more important, which is whether what you wrote is the right thing at all.

-- Talin