RFC: Viper: yet another python implementation

[Martijn Faassen]

John (Max) Skaller <skaller at maxtal.com.au> wrote:
> On 11 Aug 1999 08:21:14 GMT, m.faassen at vet.uu.nl (Martijn Faassen) wrote:

[type inference, me saying it might be hard, John saying it is doable]

I'm sure type inference can be done (at least in a limited fashion), but
it would be nice to have an intermediate dropoff point; a viperi or
viperc that would work with type annotations. This is useable already,
especially if it could be used to create fast Python extension modules.
(something you aren't aiming for right now, I gather)

Once you have that working, and people bash on it a lot, you will end up
with a system that then is reasonably well tested. Then you tack on
the type inference engine, reusing large parts of what you already
have.

If the project has usuable results at many stages, you have a bigger chance
to get the 'open source' effect going (if you're aiming for that), too.

>>Does viperc compile Python to C code? 

> 	I don't know yet. In the first instance,
> a run time is still needed that works like existing Python,
> where type inference fails. Therefore, I still need an
> interpreter. So Viperi, the interpreter, is the first step.
[snip]
> 	I will try to optimise it first, before considering
> how to build the compiler, and what back ends to implement.

Ah, so the reason you don't use Python itself as the interpreter is
learning, and also as an intermediate dropoff point.

> It is possible the first back end will actually general Ocaml
> rather than C (since Viper is currently implemented in ocaml).

Hm, why Ocaml? I don't know this language, though I heard some
interesting things about it. But it begs the question why you're 
not using Python itself. One reason may be because Python is too
slow to build an interpreter, but as you are aiming for a Python compiler
this can't last. :) And I'd definitely consider at least a C 
backend for the compiler, because C is so common. Though Ocaml may be
okay that way too, I don't know.

[snip]
>>So you could do with Swallow's "type system" (big word for nothing there),
>>right? 

> 	Yes. It is useful to have optional typing, to 
> help make the type inference work.

Cool. :)

> I was considering:

> 	def f(x:int): ...

> as the syntax ( because this is fairly conventional,
> and a pure extension). However, it isn't
> backward compatible, whereas your idea is,
> so I will look at it. [The method resembles the
> 'traits' templates in C++ STL]

I did write a bit of Python that could check these Swallow type definitions
and compare if they happened to be of the same (for instance, two
functions may have the same argument types and return value type). It also
supports a kind of typedefs. This sounds too impressive for what it actually
is. :) If anyone wants a look I can dredge it up for them, just ask.
 
>>> 	It is difficult to optimise individual
>>> modules. 

>>Hm, but the possibility to optimise individual modules is what attracts
>>many people to this idea. 

[snip]
> 	I agree. However, it turns out adding extensions is often
> trivial, whereas providing compatibility is often difficult. Similarly,
> it may be more difficult to generate a function than a whole
> module, and more difficult to do that than build a whole
> program.

But Python's C interface is well known and I believe also quite stable;
or you could use something like Swig too. So if you generated a C
function it wouldn't be that hard to automatically wrap it so it'd
be useable from Python.

> 	In particular, building a whole program is easiest,
> since it doesn't require interfacing to python. That interface
> is non-trivial: binding to the C API, and then the bytecode
> interpreter, and then again, the result is different if you
> were using JPython.

Okay, I didn't include JPython in my argument.

> 	So it may be easier to just provide
> a binding at a single level, namely 
> 'pure python' source code, than try to build something
> that interfaces to existing Python.

I can see where you're coming from, but I'd still seriously consider
adding at least a CPython interface early on -- the more stages in your
project that are usable, the higher the chance is your project will
bear fruit.

> 	In addition, Python 1.6 may have different
> internals, and so may Python 2.x. This creates
> a maintenance nightmare which may make the project
> less useful.

Why would you need access to Python internals to build a Python 
extension module? You'd just need to wrap up your functions with some C
that does type checking on the arguments -- and Python's C interface
is quite stable.

I can see where full program analysis may help with type inference, 
of course. But if you go by the intermediate route of manual adding of
types, it doesn't sound that hard to support some interface with Python.

Of course you need to restrict the modules that can be called by Viper
code to the set of modules supported by Viper; I'm not proposing Viper
code can call random Python modules, if you're thinking about that. That'd
make the whole project far more difficult.

> 	OTOH, many python 'built in C modules'
> have corresponding Pure Python sources, for example
> the string module. In that case, if the compiler works,
> there is no need for the C module.

Hm, reading this it seems you're talking about the other thing; supporting
C Python modules in Viper? I was talking about allowing people to Viper-compile
their Python modules (as long as they are Viper compliant), and use them
from Python. But as you say, you can use Pure Python modules in Viper
as well; if they restrict themselves to the Viper subset. 

Regards,

Martijn