Mailman 3 January 2003 - pypy-dev

RE: [pypy-dev] Seeking Minimal Python Project Name
by Delaney, Timothy 15 Jan '03

15 Jan '03

> From: Christian Tismer [mailto:tismer@tismer.com] > > So what this ridiculous name seekig is about > (besides obvious entries into search engines, > which is appreciated), is only a code name > for the current project, provided that we will Yeah - I know. I was being facetious. I therefore submit 'Whython' ;) Short for 'Why are we waiting to call this Python?' ;) Tim Delaney

2 1

RE: [pypy-dev] Seeking Minimal Python Project Name
by Delaney, Timothy 15 Jan '03

15 Jan '03

A radical proposal. Python implemented in Python becomes the reference version. As a result, it is called 'Python'. The C implementation offically becomes 'CPython'. :) Tim Delaney

2 1

Minimal VM
by Paul Boddie 15 Jan '03

15 Jan '03

Christian Tismer <tismer(a)tismer.com> wrote: > >There will be a very tiny, portable little virtual >machine written in C. It is not meant to be efficient, >just enough to implement the bytecode interpreter. >I'm right now tinkering with parts of such a beast. >It is getting very small, just a few kilobytes executable. The most interesting parts, it seems to me, will be those which implement the more complicated bytecode semantics (like LOAD_NAME, for example, which seems to have the potential to be pretty deep). Although you are bound to know much more about this than me - I've never looked at the Python VM source code - there must surely be a huge chunk of C code implementing the name lookup semantics. >This thing will be able to interpret Python bytecode. Without getting carried away by the promise of massive performance increases, an interesting application of a simplified VM should be the increased potential for reimplementations of the platform. It would be most amusing to be able to reduce the scope of VM operations such that they could be more easily implemented on "really small" computing platforms. Of course, a side effect of having simpler VM operations (ie. simpler bytecode semantics) is that Psyco (or its successor) will have much more to play with, as more of the VM "magic" moves out of the VM and into bytecode routines which can then be specialised. Anyway, those are my thoughts. Paul P.S. I wrote a similar analysis on Kendall Clark's Weblog at... http://www.pyzine.com/archives.phtml?a=000108&b=3#comments ...for those of you who might think they've read this before. ____________________________________________________________ Free 20MB Web Site Hosting and Personalized E-mail Service! Get It Now At Doteasy.com http://www.doteasy.com/et/

2 1

Re: [ann] Minimal Python project
by Christian Tismer 15 Jan '03

15 Jan '03

Paul Rubin wrote: > "Edward K. Ream" <edream(a)tds.net> writes: > >>1. The main reason for thinking C is particularly suited to >>producing fast code is that in C the types of all objects are known >>at compile time. ... I would expect that running psyco on Python >>code will at best run half as fast as the equivalent hand-written C >>code. The reasons are clear: psyco has neither the time nor the >>information to do as well as the best hand-written C code compiled >>by a descent compiler. > > > In fact some Lisp systems have been shown to produce compiled code > that runs as fast as compiled C code. I don't see that Python needs > to be different. As an addition: Pysco will have a much easier game with the re-implementation of Python in Python than with general Python code. I'm going to build the basic objects with Python, but based upon an object description which is similar to the new type objects with slots and properties. This concept can be expanded to not only describe derivatives of Python objects, but also for ordinary data structures which we know from C. This leads to high-level descriptions of low-level fields of all structures. These field descriptors will be the objects which are modified from the Python code. Example: An integer field in a structure describing a Python frame, for instance f_lineno, is not a regular integer object, but is accessed via a descriptor which tells that this is a fixed sized integer field which does not overflow. By using this kind of objects in Python expressions, Psyco can absolute easily deduce that it is dealing with primitive types. There is no experimentation necessary, and even without gathering any runtime experience, Psyco *can* emit code of the same quality as a C compiler does, provided that we make it into a smart enough compiler. The code dealing with these objects is ordinary Python code. The objects just happen to know what they are and how they should be rendered into assembly. So this leads to efficient implementations even of the innermost objects. Everything else can be bootsrapped upon this. This simple principle would even work without all of Psyco's sophistication, which makes me so confident that this project will have a great result, even when compiling statically. cheers - chris

2 1

reading suggestion
by Samuele Pedroni 15 Jan '03

15 Jan '03

Mixed-mode Bytecode Execution Ole Agesen and David Detlefs Abstract Modern high-performance virtual machines use dynamic compilation. There is a tension between compilation speed and code quality. We argue that a highly-optimizing compiler is best deployed with both a fast, less-optimizing compiler and an interpreter. We present measurements showing that such a system can achieve the same peak performance as a system with just the optimizing compiler, and startup costs similar to a system with just the interpreter and fast compiler. http://research.sun.com/research/techrep/2000/abstract-87.html [Java HotSpot at the moment uses and interpreter + fast medium less optimizing compiler in the client version, and an interpreter + more higly optimizing compiler in the server version] regards

1 0

Re: [ann] Minimal Python project
by Edward K. Ream 15 Jan '03

15 Jan '03

> We announce a mailinglist dedicated to developing > a "Minimal Python" version. Minimal means that > we want to have a very small C-core and as much > as possible (re)implemented in python itself. This > includes (parts of) the VM-Code. > Building on the expected gains in flexibility > we also hope to use distribution techniques such > as PEP302 to ship only a minimal set of modules > and load needed modules on demand. > As Armin Rigo of PSYCO fame takes part in the effort, > we are confident that MinimalPython will eventually > run faster than today's CPython. > And because Christian Tismer takes part, we are > confident that we will find a radical enough > approach which also fits Stackless :-) It now appears that I have misunderstood what these paragraphs meant. I apologize for any confusion that I may have created. My initial reading of this was that MinimalPython would be faster _because_ parts of the VM code were merely recoded in Python. After studying psyco further it seems more likely that what was meant was that MinimalPython could be faster than the present CPython because the techniques of psyco might be applied to improve the speed of the present main interpreter loop. The former statement seemed (and still seems) most unlikely. The latter statement seems quite plausible. In short, I really don't understand the aims of this project very well. I can imagine that it has one or more of the following goals: 1. To code as much as possible of the Python distribution in Python. 2. To increase the speed of Python by integrating psyco into the interpreter. 3. To provide a test-bed for alternative implementations such as stackless. What happens if these goals conflict? For example, much of psyco is presently written in C. I don't suppose these goals imply that all of psyco will be recast in Python, or do they? If recasting code in Python conflicts with increasing code speed, which is most important? What happens if the goal of supporting stackless Python conflicts with the other goals? Any clarifications would be most welcome. Edward -------------------------------------------------------------------- Edward K. Ream email: edream(a)tds.net Leo: Literate Editor with Outlines Leo: http://personalpages.tds.net/~edream/front.html --------------------------------------------------------------------

3 4

Thought experiments
by Edward K. Ream 15 Jan '03

15 Jan '03

I'd like to throw out some thought experiments, just for fun. As I understand it, the essence of psyco is that it can generate efficient machine code corresponding to cases in the C interpreter based primarily on knowledge of the _runtime types_ of operands (variables of the _interpreter_ corresponding to Python operands and intermediate operands). pyrex generates C code based on the _declared_ types of objects. Is this summary basically correct? Might we not combine these two approaches? Are there any circumstances in which we might generate _optimized_ C/assembly code based on the runtime types of objects in functions? This suggests a multi-layered approach: use psyco to determine the types of objects, then _afterwards_ use something like a real C compiler to generate code. This can only work if the types of object to functions remains constant, which is usually true, but certainly not always true. This would be similar to other optimization techniques that gather statistics about the frequency that basic code blocks are executed. The difference is that here the first-stage optimizer (psyco) would gather statistics about the _types_ of variables. Yes, there are problems here, but they correspond, I think, to similar problems in psyco. Indeed, psyco must continually check (using psyco_compatible, if I am not mistaken) to see that the types presented to the function match the types used to create the assembly-language code. In other words, might we not consider a new kind of "assembled byte code"? My guess is that compiled functions/methods would have to start with a "preamble" that selects the proper compiled code to execute based on the types of the arguments passed to the function. Actually, I wonder whether psyco itself could generate such preambles as an alternative to using psyco_compatble. (Or maybe the two techniques are equivalent). Sure, there could be code explosion in general. But how likely is this in practice? And psyco must limit code bloat as well. It would be interesting to get some statistics. In Leo I suspect the vast majority of functions and methods use only a single type of each argument and return only a single type of result. The null class pattern could be used to deal with None values of arguments. We might limit this kind of technique to routines explicitly selected by the user, or the technique could limit itself to only those functions with non-varying types. Just as psyco does, or will do, we would need some kind of escape if unexpected types of arguments were discovered later. Any comments? Edward -------------------------------------------------------------------- Edward K. Ream email: edream(a)tds.net Leo: Literate Editor with Outlines Leo: http://personalpages.tds.net/~edream/front.html --------------------------------------------------------------------

1 0

Pyrex
by Florian Schulze 14 Jan '03

14 Jan '03

Hi! I just thought that for some core functionality and interfacing to C functions something similar to Pyrex could be used. Just an extension to the python language to easily call C code from it. Not emmiting C code like Pyrex does, but doing it directly somehow. Something like a mix of calldll, ctypes and pyrex. Psyco then could be run over it to improve the code unlike in the current Pyrex implementation which just emmits C blindly and not very efficiently. Florian ps: Pyrex is really a great tool, it's very easy to write interfaces with it.

2 1

Introducing myself, again
by Edward K. Ream 14 Jan '03

14 Jan '03

We didn't get off to the best start on the comp.lang.python list, so I'd like to reintroduce myself to you. In this posting I'll discuss my background and how I might contribute to the psyco project. Background I have been programming for 35+ years, first in assembly and C, then in C++ and Objective-C, and finally and most joyously in Python. I've used the NextStep/Yellow Box/Cocoa, Borland C++ Builder, wxWindows and tcl/tk class frameworks. I am familiar with the Smalltalk and Java class libraries. My primary interests have always been: 1) the programming process itself and 2) getting maximum speed out of programs. One of my earliest projects was improving the speed of a screen driver by about a factor of 10. Doing so changed qualitatively how people interacted with the screen. Sometime in the 80's Leor Zolman gave me the source code of the BDS-C compiler to study. For several years this was the fastest C compiler available on micros (as we called PC's then), and to this day is one of the fastest compilers ever built. Studying this compiler was a revelation for me: it broke all the rules about compilers I learned in graduate school, and broke them very well. I realized then that much of my career would be unlearning what I thought I knew :-) In the 90's I designed and built a commercial optimizing C compiler. This was a technical success: it produced very good m68000 code and did it faster than the Borland C compiler. It was not a commercial success as the company folded before the compiler was released. The compiler was not particularly elegant; I am not a compiler expert, and I _do_ have real experience. I am the BDFL of the Leo project. Leo is a major Open Source project written in Python. The Leo project takes up the bulk of my time, and I would gladly give some time to this project. I am also the creator of the pl68k programming language and compiler, the Sherlock tracing system, and the ancient RED text editor. With the exception of the C compiler (and related linker and file system), everything I have done has been Open Software. How I might help I have lots of experience with making code work fast and I have a number of ideas about generalizing and improving psycho... I would be willing to help improve the documentation of psyco. I think psyco promises a huge win for Python, and I would like to see it explained so many people can understand and improve it. I have created a Leo outline of an old version of psyco, and I would be happy to convert all or parts of this project to Leo. Leo is an excellent way of organizing, studying and presenting complex code. Whether the psyco project chooses to adopt Leo "officially" is not for me to say, and I would recommend this project experiment with using Leo. Oh yes, I have a Python script that helps with conversion of C code to Python. It simply hacks on the syntax, removing braces, semicolons, type declarations, converting argument lists to def statements, that kind of thing. The script is buried in the source code for Leo. Edward -------------------------------------------------------------------- Edward K. Ream email: edream(a)tds.net Leo: Literate Editor with Outlines Leo: http://personalpages.tds.net/~edream/front.html --------------------------------------------------------------------

1 0

Possible logo idea - ouroboros
by Damien Morton 14 Jan '03

14 Jan '03

http://www.millenniumdesktop.co.uk/ouroboros.htm The snake biting its own tail image used in the opening title sequence of Millennium is called an Ouroboros. In the series it is the symbol of the Millennium Group. In mythology, the Ouroboros is any image of a snake, worm, serpent, or dragon biting its own tail. It was first seen as early as 1600 years BC in Egypt. The Greeks called it the Ouroboros, which means "Tail Eater." Generally taking on a circular form, the symbol is representative of many broad concepts. Time, life continuity, completion, the repetition of history, the self-sufficiency of nature and the rebirth of the earth can all be seen within the circular boundaries of the Ouroboros. Societies from throughout history have shaped the Ouroboros to fit their own beliefs and purposes. The image has been seen in Japan, India, utilized in Greek alchemic texts, European woodcuts, Native American Indian tribes and even by the Aztecs. It has, at times, been directly associated to such varying symbols as the Roman god Janus, the Chinese Ying Yang, and the Biblical serpent of the garden of Eden. http://abacus.best.vwh.net/oro/ouroboros2.html This symbol appears principally among the Gnostics and is depicted as a dragon, snake or serpent biting its own tail. In the broadest sense, it is symbolic of time and the continuity of life. It sometimes bears the caption Hen to pan - 'The One, the All', as in the Codex Marcianus, for instance, of the 2nd century A.D. It has also been explained as the union between the chthonian principle as represented by the serpent and the celestial principal as signified by the bird (a synthesis which can also be applied to the dragon). Ruland contends this proves that it is a variant of the symbol for Mercury - the duplex god. In some versions of the Ouroboros, the body is half light and half dark, alluding in this way to the successive counterbalancing of opposing principls as illustrated in the Chinese Yin-Yang symbol for instance. Evola asserts that it represents the dissoluotion of the body, or the universal serpent which (to quote the Gnostic saying) 'passes through all things'. Poison, the viper and the universal solvent are all symbols of the undifferentiated-of the 'unchanging law' which moves through all things, linking them by a common bond. Both the dragon and the bull are symbolic antagonists of the solar hero. The Ouroboros biting its own tail is symbolic of self-fecundation, or the primitive idea of a self-sufficient Nature - a Nature, that is which, à la Nietzsche, continually returns, within a cyclic pattern, to its own beginning. There is a Venetian manuscript on alchemy which depicts the Ouroboros with its body half-black (symbolizing earth and night) and half-white (denoting heaven and light).

9 10