some random reflections of a "Python newbie": (2) language issues

[skaller]

Preston Landers wrote:
> This is one of the few obvious inconsistencies in
> Python's implementation.  If you could subclass builtins, then what you
> want to do would be completely trivial.

[skaller}
> >   I don't agree: python is NOT inconsistent here.

> I think we are talking about two different kinds of inconsistency.  

> I'm speaking from the point of view of a developer who doesn't care
> (much) about types vs. classes.  he only wants to say, "Okay, here is a
> thing that is in every respect like a dictionary, only it can be locked
> and unlocked."  

	Yes, but there are ways to do this _other_ than by using inheritance.

> In that sense, almost all OO languages suffer from this
> 'inconsistency.'  I'm not really complaining, just noting it.  Why
> should a developer care about a distinction between types and classes?

	Simple: they're different concepts. It is unfortunate
that 'OO' languages typically make the mistake of identifying them.
It turns out this is completely wrong, and  is the reason why
OO doesn't work.

	I should be more precise: in most OO languages,
a class is a construction that is used to declare a type.
In static languages, class declarations specify types,
which are _compile time_ concepts (only).

	This is not the case in dynamic languages,
where the abstract notion of 'type' is distinct from,
but represented by, actual type objects at run time:
for example, in Python, there are TWO kinds of
objects representing the 'type' abstraction:
actual type objects, and classes. The latter can be 
constructed by the python programmer, whereas the former
can only be constructed by the extension writer (or Guido :-)

	The 'inconsistency' I believe you are refering to is
that there are two such 'type' representations at run time,
and they have different properties: you can modify classes
dynamically, and you can derived new classes, so that
the class is 'really' the representation of the 'type'
abstraction of the application programmer .. but when it
comes to dictionaries, the abstraction has to be switched
to type objects.

	This seems inconsistent to the programmer, right?

> That is an implementation detail that interferes with the "purity" of OO
> and tends to confuse people.

	Seems we agree so far...

	First, please excuse a digression: "purity" of OO is a 
bad thing, because OO doesn't work. It is good to support a familiar
paradigm like OO, but it is also useful to support others as well.
Python, for example, support procedural programming, and, to some
extent, functional programming, as well as event driven
programming (in some extensions like Tkinter), and it also
supports generic programming (via both dynamism and also
the 'protocols' API for sequences, etc). From my point of view,
it is the 'protocols' feature which is Python's strongest
point, not the OO (although I use that too).

	Now, to continue, lets examine the 'typeness'
abstraction in more detail. Consider a sequence.
The notion of sequence is a type abstraction of some
kind. But it is NOT represented by a class, or by a type object.
Why? Because OO doesn't work in general. What _does_ work
is the notion of protocols: a set of functions operating
on a set of types with particular behaviours, for example
a sequence is characterised by having a length, and being
able to get the i'th object of the sequence.

	So here's what happens in Python: the protocols
are fixed by the language, and implemented by type objects..
ONE such protocol is the notion of 'object orientation':
and it is represented by 'getattr' 'setattr' protocols,
because 'objects in the OO sense' are characterised by
having states which can be modified, usually
via methods.

	Classes then are not 'types' in the fully abstract
sense that protocols allow, but a special case.

	In Viper, type objects can be any object.
What this means is that, unlike CPython, the
'virtual table' which supports protocols like
'sequence' and 'mapping' and 'class instance/object orientation'
is not limited. [The cost, in the interpreter, is performance
and robustness]

	In Viper, all the standard types ARE represented
by type objects which are classes. But the instances
of the objects are NOT instances of these classes.
An 'integer' has a class for a type object, but that
is a convenience to allow 'methods' to be defined
for integers -- IN PYTHON (instead of C).
[In other words, the 'type object' is a kind of 
attribute/method server for the instance]

	If I can put the model another way,
there are TWO levels of indirection here:
a class instance has a class which has a type,
and the instance itself has a type: and the type
of an instance is "PyInstance" and that of a class
is "PyClass" and _neither_ represents the typeness
of the instance from the application programmers point
of view, instead, the class itself represents that.
In CPython you cannot manipulate the type objects
from python, in Viper you can, which opens up a whole
lot of new, interesting things.

	For example, I have previously posted an example
Viper is going to support (it doesn't yet). It is possible
to define a class representing 

	ListOfSomething

where the 'Something' is bound when an instance is constructed:

	PyListOfInt = ListOfSomething(PyInt)

and now, objects of type 'PyListOfInt' can be defined.
These will be ordinary lists, except that even element
is checked when it is put in the list, to make sure
it is has type PyInt.

	This kind of flexibility is sometimes
called 'metaprogramming'. And there is a vital
theorem of metaprogramming: TWO levels of indirection
are vital: you need the possibility that the
type of something be an instance of another meta-type.

	Looked at another way: Pythons existing
'type' objects cannot be manipulated, whereas
Viper's can, but in both cases, the fact that
class objects themselves have a type, but are
not themselves general types, is consequence
of the fact that classes only implement ONE
meta-programming protocol.

	I think the 'big' question is:
'can we uses classes for all type objects'?
And I think the answer is NO, not in general.
But, Viper shows, the converse is indeed possible:
we _should_ be able to use classes for SOME
type objects. So i don't think there is an inconsistency
in CPython, so much as a lack of flexibility: there's no
reason why type objects should themselves have a fixed type
(namely TypeType).

	Summary: unifiying the notion of "type object"
and "class" is a bad idea, but generalising the notion
of type object so you can use a class AS a type object,
seems to be an excellent idea: a class is a KIND of
type object, but not the only possible one.

	If you want to have a look, you can download
Viper from

	ftp://ftp.cs.usyd.edu.au/jskaller

note that the 'ListOfSomething' cannot yet be made
to work, because there is not yet a builtin function
which is the equivalent to the 'new object' function
in the new module. (that is, there is no way yet
to create a raw object of a given type).

-- 
John Skaller, mailto:skaller at maxtal.com.au
10/1 Toxteth Rd Glebe NSW 2037 Australia
homepage: http://www.maxtal.com.au/~skaller
voice: 61-2-9660-0850