Q: Python 2.0 preliminary features?

[skaller]

Jeremy Hylton wrote:
> 
> >>>>> "MS" == skaller  <skaller at maxtal.com.au> writes:
> 
>   MS> Michael Hudson wrote:
>   >>  skaller <skaller at maxtal.com.au> writes:
> 
>   >> Hold it there a second; you are proposing that knowing that x has
>   >> a certain type in the expression
>   >>
>   >> x = x + "hello"
>   >>
>   >> has an impact on inference?
> 
>   MS>   Yes. Rather, the type of x is _deduced_ from the fact that
>   MS> operator + must be string concatenation in this context, since
>   MS> the second argument is a string, and thus x must be a string.
> 
> I assume you're handling some of the special cases, and I'd be
> interested to hear how.

	Please note: the inference system is not written yet.
I have done some tests to see if it can work (my results agree with
Jim Hunguins: yes, it can work).

	The method is simple enough: each function (such as + )
has a set of possible signatures, such as:

	int + int -> int
	float + float -> float
	string + string -> string

Now, when you see a +, there is a constraint on the arguments.
If one of the arguments, for example, is a string, then
given the three signatures above, the other has to be a string
too, and, the result is also a string.

Now, the list above is not complete for Python. For all
core language types, there is a fixed list of all the
functions and all the signatures: these are for operators
plus builtin functions.

In addition, user defined classes can do interesting things.
So we could allow, in most cases, the possibility that
the type is 'class instance'. We could go further and say 
_which_ classes, by adding to these lists for each
class defining, say '__add__' or '__radd__'.

However, the really big speed improvements from
the type inference will come with core language
operations, where the overhead of lookup and
function call and allocations can be avoided
by using say, machine registers to hold
an integer.

This is already the case in some parts of Viper.
For example, Viper executes the loop:

	for i in xrange(limit):

MUCH faster than Python. The time for 100000
iterations on my 166Mhz Pentium I machine
is ZERO second. Python takes 3 seconds.
That is because this case is hard coded in the interpreter
as a native for loop, which is compiled to machine
fast machine code.

> It is easy enough to write a class that will do something when added
> to a string.  Arbitrary example:
> 
> class Foo:
>     def __init__(self):
>             self.strings = []
>     def __add__(self, other):
>             new = Foo()
>             new.strings = self.strings[:]
>             new.strings.append(other)
>             return new
> 
> x = Foo()
> x = x + "hello"
> 
> I assume you can do some analysis to discover that x *might* be a
> class that implements __add__.  How specific is your analysis?  

	It is vague: I haven't written the code yet.

>If you
> see a class like Foo, are all bets off about the meaning of x +
> "hello"?

	No. There are some rules for inference.
For example, consider:

	if something: x = 1
	else: x= "string"

Here, Viper should be able to tell that x is either an integer or
a string (and nothing else).

What Viper will do is glean all the available 'static' information.
What it will not do, is be able to do dynamic flow analysis.
{except in special idiomatic cases like 'if 1: .. '}

However, there is a trick. Viper compiles code by FIRST running
the interpreter Viperi to load the modules: in this stage,
stuff is imported. After this stage is finished,
THERE IS NO MODULE LEVEL CODE LEFT. All the code is in functions,
and the dictionaries of the modules are built.
So cases like:

	# module level code ..
	try: import x
	except: x = None

will work, because they're actually interpreted: this code
is not compiled.

 
> The second variation is a library that accepts some object as an
> argument and adds "hello" to it.  Do you do whole-program analysis to
> determine whether any client of the library might pass something other
> than a string? 

	Yes.

> What are the implications of that for writing a
> generic library?

	None. Viperc does not build modules or packages,
it builds whole executable programs. So, a particular
module or function may be used differently in different
programs. [At least, this is the initial goal]

	The reason for this choice, despite the appeal
of compiling modules, packages, or libraries, is simply
that experience of others indicates that the existing
python interpreter is so fast, compiling code which 
is equivalent on a per module/per function kind of basis
is not really worth the effort.

	The only way to gain any leverage is to 
analyse a whole program, in which case many
possible uses of, say, a function like:

	def f (x,y): return x + y

are eliminated because they're not in fact used
in a particular program.


-- 
John Skaller, mailto:skaller at maxtal.com.au
1/10 Toxteth Rd Glebe NSW 2037 Australia
homepage: http://www.maxtal.com.au/~skaller
downloads: http://www.triode.net.au/~skaller