[Types-sig] Static typing considered HARD

[Martijn Faassen]

Uche Ogbuji wrote:
> Paul Prescod wrote:
[snip]
> > But in Python, type objects only come about *through* the execution of
> > code. This makes Python incredibly dynamic but it also means that the
> > question of what exactly static type checking means is confused. Simple
> > example:
> > 
> > import sys
> > 
> > if sys.argv[0]=="weirdness":
> >         from foo_mod import foo_class
> > else:
> >         from foo_mod2 import foo_class
> 
> This is the sort of thing that gives Python its power, and it is the
> sort of thing without which I'm not sure I wouldn't be considering
> another language.
> 
> > One could imagine that in some Python 2, import statements and class
> > definitions could be limited to being at the top, before "code". There
> > might be some special syntax (e.g. __import__, __define_class__ ) for
> > doing module-loading and type definition at runtime. Still, I don't
> > consider that something for the types-sig to work out. My personal
> > opinion is that it would be a Good Thing for Python to become a tad less
> > dynamic in the "core syntax" in exchange for compile-time checking of
> > names.
> 
> This is exactly the sort of idea that terrifies me about Python 2, as
> I've done a poor job of expressing before.  My hope is that Python 2
> remains Python, and such artificial constraints as "imports only at the
> top" and all that in order to satisfy IMHO mis-placed notions of type
> safety are dropped in the nearest dustbin.

It's good that someone expressed this. While I myself would argue for
some form of static typing being added to (part of) Python, I do think
Python's dynamicism should be kept in mind very strongly.

[snip more arguments against any curtailing of Python's dynamicism]

> > Even if we ignore static type checking Python 2 really has to do
> > something about the "misspelling problem." One extra character on a
> > method name can crash a server that has been running for weeks. Once
> > this problem is fixed, the term "static type checking" will become
> > meaningful. In the current environment, it is probably not and thus
> > should not be the first focus of a new types-sig.
> 
> I keep hearing this sort of thing, and I keep saying that it's a red
> herring.  Lack of static typing does _not_ prevent Python from being
> scalable to large-scale and production environments.  Our experience at
> FourThought, where many of our projects are small-enterprise systems
> built with Python and sometimes CORBA, will make it very hard for anyone
> to convince me so.  I think the experience of users such as eGroups
> supports my feeling.

Likewise the experiences of the Zope user base. I've been debugging my
own Zope products, which had syntax errors and misspellings all over the place.
Zope itself however keeps running happily, as it'll catch the exceptions.
As you say in a part of your post that I snipped, good exception handling
facilities and testing procedures alleviate a lot of the problems with
misspellings and the like.

That said, I am interested in attempts that make Python even more robust.
I do occasionally worry about code that may contain bugs but that is not
exercised enough doing debugging. Of course this happens with statically
typed languages as well, but at least the compiler catches some problems.

>  If anything, it is Java that I think is
> tremendously over-rated for large-scale projects and I predict its
> failure in that space will soon be an industry scandal.

Interesting.

Anyway, it's good that your view is present on the types-SIG.

My take on static types in Python has been the Swallow proposal. The idea
is that we want some early-result points in the project to add static
types to Python. With quite a few others I deem the possible speed payoff
of adding static types as least as important as the possible code-quality
payoff. 

Adding static types to Python proper is hard, and undesirable if it entails
giving up too much of Python's dynamicism, as has been observed.

The assumption of Swallow is that many parts of a typical Python program
do not profit a lot from Python's dynamic typing, though of course other
parts do. Traditionally the only way to gain speed with Python programs
has been to move parts that can be static anyway to C. This is however a 
rather big step. It would be nicer if our extension modules could be more
like Python itself. This way there is a gradual transition between 
dynamic Python to static Python code.

The Swallow proposal is to find a subset of Python (Swallow) that is 
horrible in all the ways Uche so empathically dislikes. :) Get rid of
whatever is necessary in Python to make Swallow amenable to static types;
restrict imports to the top, restrict what magic one can do with classes,
etc. The important point is that Swallow is a strict subset of Python,
not adding any facilities or different semantics of its own, as much as
possible.

Then, provide a facility to describe the type signature of any class,
function or variable in the Swallow code. No fancy type inference, just
the programmer describing everything. After that the Swallow code could be
compiled (or translated to C).

Of course I'm skimming over lots and lots of problems here; Swallow code
can't for instance use any non-Swallow module. Writing a C translator
is hard. Writing a static type checker for Swallow is hard. Identifying
the Swallow subset is hard, and preserving Python semantics in it is hard.

Still, it seems to be me it's less hard than adding optional static types
to Python itself, while still keeping lots of the payoff. It'd be great if
Python 2 had a Swallow subsystem.

A possible activity for the types-SIG could in fact be to identify the
proper Swallow subset of Python; that subset of Python amenable to 
static types and fairly straightforward translation to C code. 

Of course I keep pushing Swallow without writing any actual code, so you
all may be bored of it by now. :)

Regards,

Martijn