Mailman 3 - pypy-dev

getting rid of compile.c?
by holger krekel 15 Jan '03

15 Jan '03

Hi, i was wondering how/if to get rid of compile.c. ASFAIK it is currently needed for bootstrapping purposes. Wouldn't it be possible to deliver precompiled bytecode for the python compiler package and thus obsolete compile.c? The bytecode isn't platform dependent so this seems sensible to me. regards, holger

3 6

RE: Seeking Minimal Python Project Name
by Tim Jarman 15 Jan '03

15 Jan '03

How about: XPython as in: eXperimental Python or : eXtreme Python (but not ex-Python as in "This is an ex-parrot")? Helping-to-keep-this-thread-immortal-ly yrs, Tim J.

1 0

RE: [pypy-dev] Seeking Minimal Python Project Name
by Delaney, Timothy 14 Jan '03

14 Jan '03

> From: Christian Tismer [mailto:tismer@tismer.com] > > So what this ridiculous name seekig is about > (besides obvious entries into search engines, > which is appreciated), is only a code name > for the current project, provided that we will Yeah - I know. I was being facetious. I therefore submit 'Whython' ;) Short for 'Why are we waiting to call this Python?' ;) Tim Delaney

2 1

RE: [pypy-dev] Seeking Minimal Python Project Name
by Delaney, Timothy 14 Jan '03

14 Jan '03

A radical proposal. Python implemented in Python becomes the reference version. As a result, it is called 'Python'. The C implementation offically becomes 'CPython'. :) Tim Delaney

2 1

Minimal VM
by Paul Boddie 14 Jan '03

14 Jan '03

Christian Tismer <tismer(a)tismer.com> wrote: > >There will be a very tiny, portable little virtual >machine written in C. It is not meant to be efficient, >just enough to implement the bytecode interpreter. >I'm right now tinkering with parts of such a beast. >It is getting very small, just a few kilobytes executable. The most interesting parts, it seems to me, will be those which implement the more complicated bytecode semantics (like LOAD_NAME, for example, which seems to have the potential to be pretty deep). Although you are bound to know much more about this than me - I've never looked at the Python VM source code - there must surely be a huge chunk of C code implementing the name lookup semantics. >This thing will be able to interpret Python bytecode. Without getting carried away by the promise of massive performance increases, an interesting application of a simplified VM should be the increased potential for reimplementations of the platform. It would be most amusing to be able to reduce the scope of VM operations such that they could be more easily implemented on "really small" computing platforms. Of course, a side effect of having simpler VM operations (ie. simpler bytecode semantics) is that Psyco (or its successor) will have much more to play with, as more of the VM "magic" moves out of the VM and into bytecode routines which can then be specialised. Anyway, those are my thoughts. Paul P.S. I wrote a similar analysis on Kendall Clark's Weblog at... http://www.pyzine.com/archives.phtml?a=000108&b=3#comments ...for those of you who might think they've read this before. ____________________________________________________________ Free 20MB Web Site Hosting and Personalized E-mail Service! Get It Now At Doteasy.com http://www.doteasy.com/et/

2 1

Re: [ann] Minimal Python project
by Christian Tismer 14 Jan '03

14 Jan '03

Paul Rubin wrote: > "Edward K. Ream" <edream(a)tds.net> writes: > >>1. The main reason for thinking C is particularly suited to >>producing fast code is that in C the types of all objects are known >>at compile time. ... I would expect that running psyco on Python >>code will at best run half as fast as the equivalent hand-written C >>code. The reasons are clear: psyco has neither the time nor the >>information to do as well as the best hand-written C code compiled >>by a descent compiler. > > > In fact some Lisp systems have been shown to produce compiled code > that runs as fast as compiled C code. I don't see that Python needs > to be different. As an addition: Pysco will have a much easier game with the re-implementation of Python in Python than with general Python code. I'm going to build the basic objects with Python, but based upon an object description which is similar to the new type objects with slots and properties. This concept can be expanded to not only describe derivatives of Python objects, but also for ordinary data structures which we know from C. This leads to high-level descriptions of low-level fields of all structures. These field descriptors will be the objects which are modified from the Python code. Example: An integer field in a structure describing a Python frame, for instance f_lineno, is not a regular integer object, but is accessed via a descriptor which tells that this is a fixed sized integer field which does not overflow. By using this kind of objects in Python expressions, Psyco can absolute easily deduce that it is dealing with primitive types. There is no experimentation necessary, and even without gathering any runtime experience, Psyco *can* emit code of the same quality as a C compiler does, provided that we make it into a smart enough compiler. The code dealing with these objects is ordinary Python code. The objects just happen to know what they are and how they should be rendered into assembly. So this leads to efficient implementations even of the innermost objects. Everything else can be bootsrapped upon this. This simple principle would even work without all of Psyco's sophistication, which makes me so confident that this project will have a great result, even when compiling statically. cheers - chris

2 1

reading suggestion
by Samuele Pedroni 14 Jan '03

14 Jan '03

Mixed-mode Bytecode Execution Ole Agesen and David Detlefs Abstract Modern high-performance virtual machines use dynamic compilation. There is a tension between compilation speed and code quality. We argue that a highly-optimizing compiler is best deployed with both a fast, less-optimizing compiler and an interpreter. We present measurements showing that such a system can achieve the same peak performance as a system with just the optimizing compiler, and startup costs similar to a system with just the interpreter and fast compiler. http://research.sun.com/research/techrep/2000/abstract-87.html [Java HotSpot at the moment uses and interpreter + fast medium less optimizing compiler in the client version, and an interpreter + more higly optimizing compiler in the server version] regards

1 0

Re: [ann] Minimal Python project
by Edward K. Ream 14 Jan '03

14 Jan '03

> We announce a mailinglist dedicated to developing > a "Minimal Python" version. Minimal means that > we want to have a very small C-core and as much > as possible (re)implemented in python itself. This > includes (parts of) the VM-Code. > Building on the expected gains in flexibility > we also hope to use distribution techniques such > as PEP302 to ship only a minimal set of modules > and load needed modules on demand. > As Armin Rigo of PSYCO fame takes part in the effort, > we are confident that MinimalPython will eventually > run faster than today's CPython. > And because Christian Tismer takes part, we are > confident that we will find a radical enough > approach which also fits Stackless :-) It now appears that I have misunderstood what these paragraphs meant. I apologize for any confusion that I may have created. My initial reading of this was that MinimalPython would be faster _because_ parts of the VM code were merely recoded in Python. After studying psyco further it seems more likely that what was meant was that MinimalPython could be faster than the present CPython because the techniques of psyco might be applied to improve the speed of the present main interpreter loop. The former statement seemed (and still seems) most unlikely. The latter statement seems quite plausible. In short, I really don't understand the aims of this project very well. I can imagine that it has one or more of the following goals: 1. To code as much as possible of the Python distribution in Python. 2. To increase the speed of Python by integrating psyco into the interpreter. 3. To provide a test-bed for alternative implementations such as stackless. What happens if these goals conflict? For example, much of psyco is presently written in C. I don't suppose these goals imply that all of psyco will be recast in Python, or do they? If recasting code in Python conflicts with increasing code speed, which is most important? What happens if the goal of supporting stackless Python conflicts with the other goals? Any clarifications would be most welcome. Edward -------------------------------------------------------------------- Edward K. Ream email: edream(a)tds.net Leo: Literate Editor with Outlines Leo: http://personalpages.tds.net/~edream/front.html --------------------------------------------------------------------

3 4

Thought experiments
by Edward K. Ream 14 Jan '03

14 Jan '03

I'd like to throw out some thought experiments, just for fun. As I understand it, the essence of psyco is that it can generate efficient machine code corresponding to cases in the C interpreter based primarily on knowledge of the _runtime types_ of operands (variables of the _interpreter_ corresponding to Python operands and intermediate operands). pyrex generates C code based on the _declared_ types of objects. Is this summary basically correct? Might we not combine these two approaches? Are there any circumstances in which we might generate _optimized_ C/assembly code based on the runtime types of objects in functions? This suggests a multi-layered approach: use psyco to determine the types of objects, then _afterwards_ use something like a real C compiler to generate code. This can only work if the types of object to functions remains constant, which is usually true, but certainly not always true. This would be similar to other optimization techniques that gather statistics about the frequency that basic code blocks are executed. The difference is that here the first-stage optimizer (psyco) would gather statistics about the _types_ of variables. Yes, there are problems here, but they correspond, I think, to similar problems in psyco. Indeed, psyco must continually check (using psyco_compatible, if I am not mistaken) to see that the types presented to the function match the types used to create the assembly-language code. In other words, might we not consider a new kind of "assembled byte code"? My guess is that compiled functions/methods would have to start with a "preamble" that selects the proper compiled code to execute based on the types of the arguments passed to the function. Actually, I wonder whether psyco itself could generate such preambles as an alternative to using psyco_compatble. (Or maybe the two techniques are equivalent). Sure, there could be code explosion in general. But how likely is this in practice? And psyco must limit code bloat as well. It would be interesting to get some statistics. In Leo I suspect the vast majority of functions and methods use only a single type of each argument and return only a single type of result. The null class pattern could be used to deal with None values of arguments. We might limit this kind of technique to routines explicitly selected by the user, or the technique could limit itself to only those functions with non-varying types. Just as psyco does, or will do, we would need some kind of escape if unexpected types of arguments were discovered later. Any comments? Edward -------------------------------------------------------------------- Edward K. Ream email: edream(a)tds.net Leo: Literate Editor with Outlines Leo: http://personalpages.tds.net/~edream/front.html --------------------------------------------------------------------

1 0

Pyrex
by Florian Schulze 14 Jan '03

14 Jan '03

Hi! I just thought that for some core functionality and interfacing to C functions something similar to Pyrex could be used. Just an extension to the python language to easily call C code from it. Not emmiting C code like Pyrex does, but doing it directly somehow. Something like a mix of calldll, ctypes and pyrex. Psyco then could be run over it to improve the code unlike in the current Pyrex implementation which just emmits C blindly and not very efficiently. Florian ps: Pyrex is really a great tool, it's very easy to write interfaces with it.

2 1