Mailman 3 January 2003 - pypy-dev

Concrete Syntax Tree
by Grant Olson 14 Jan '03

14 Jan '03

I wrote some rough python code a few months ago that built Concrete Syntax Trees from the grammar file in the Python source, and also turned token streams into Abstract Trees based on the Concrete Trees. The parser module successfully compiled the Abstract Trees. Is this something you guys would be interested in or am I off base here? --------------------------------------- Get your free e-mail address @zworg.com

2 1

Questions about the C core
by Florian Schulze 14 Jan '03

14 Jan '03

Hi! I would like to know if there are already some concrete plans what the C core needs to be able to do. Will there be anything like the old builtin module? Will dicts and the other types still be done in C or some mix of C and Python etc. Also will the Psyco part be optional to ease the initial porting to new platforms? Florian

9 22

Intellectual property
by David Ascher 14 Jan '03

14 Jan '03

I'd like to suggest that the issue of intellectual property and license be tackled _very_ early. Anything else is inexcusable IMO =). Proposal: - all of the minipy IP is assigned to the PSF. - the license is either the PSF license or the Academic Free License http://www.opensource.org/licenses/academic.php. (the advangate of the AFL is the patent protection, which I suspect may be relevant in this particular implementation). --da

7 11

Re: [ann] Minimal Python project
by Christian Tismer 14 Jan '03

14 Jan '03

(copying to pypy-dev since this is an incentive) Paul Rubin wrote: > Christian Tismer <tismer(a)tismer.com> writes: > >>You know what my goals are. >>Smaller, more flexible, faster, easier to change, >>easier to maintain, easier to keep backwards >>compatible, more portable due to less C code, >>down-sizeable by features (which is most difficult), >>the full catastrophe... > > > Well ok, it wasn't clear before what the entire set of goals were. > > >>We will try to implement Python as exact and clean >>as possible. The langage should be implemented >>completely. Now Paul is nailing thing down :-) > Are you going to try to keep the C API backwards compatible? I wish to, but I cant guarantee it. This is a matter of experimentation. Fore sure, we will try to build a version that adheres to the C API, at least in the early bootstrap phase, we really need to do so, in order to use "borrowed" internals and extensions. It then depends on experience to be gathered, how much conformance to the C API costs. If it turns out to be much more efficient to use a different API, and maybe a radical change of internal data structures as well, then TiPy (randomly picked name from all the proposals :) is the first chance to try such new ways at all. This can lead to a compatibility layer (maybe hard to implement), and these insights may become proposals to change core Python as well. It is also thinkable to have options in TiPy which make even this configurable at boot time, and people can decide whether they want it compatible or faster. Some of the "internal" API like compiler related functions and stuff from ceval.c will most probably be changed, anyway, but the common API will be as it is now. This is where we start with. > Are you going to include exact implementations of currently > not-really-documented features such as metaclasses? Absolutely. I'm using metaclasses very much, and I also will chime a compatible patch in, which allows to use slots in metaclasses. I need that, and Stackless has it. > Will stuff like frame objects still work as documented? Do you expect > to be able to run the current version of pdb.py without changes? Yes, for the first round, frames should be rather like they are now. They will get more interfaces to be manipulated by Python code. pdb.py should run as it is now. I know there is an issue with Psyco, which doesn't create frames all the time. This is an issue that need thorough discussion and design. > I don't think you should necessarily hold yourself to any of the > above, but that's just me. If the above can be achieved for a reasonable price, then we really should try. If it hurts too much, then it may be easier to change the rest of the world :-) >>It will be a major amount of work to deduce the >>dependencies of features, and how to arrange them >>in a scalable shape. I do believe that the core >>group will help us with that. > > > I'm pretty excited by the project. I think you're going to push the > limits of the Python language harder than they've ever been pushed > against before. And you'll be in a unique position to actually remove > the limits when you hit them, rather than having to work around them. > I hope you'll trust yourself to use such opportunities when it's right > to do so. Thanks for the encouragement. I hope to to as well as possible. Not loosing chances to stay compatible, at the same time not loosing good new opportunities by sticking too much with old principles. This is not easy and a balancing act. Something that I could not do alone, and I'm happy that there are so many supporters and people going to help. I'd-love-to-have-*you*-in-that-group-too -- chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

1 0

Re: [ann] Minimal Python project
by Christian Tismer 13 Jan '03

13 Jan '03

Paul Rubin wrote: > Christian Tismer <tismer(a)tismer.com> writes: Now I'm trying to do my second answer to this. >>>I think that local declarations and type hints are useful language >>>improvements for more reasons than helping generate fast code... [me, arguing against introduction of new features, introducing many fruitless discussions...] > Yes, if you want to implement an extension of this type it's better to > just pick out a way to do it and code it up, than to spend weeks > posting netnews about different possible methods. There's also a nice > characteristic of experimental implementations, that if you don't like > how a feature works out, you can change it, unlike if it gets into a > real Python release and people start depending on it. However, I can > understand your approach of wanting to leave it out completely at > first and possibly add it later if it's needed. I guess that something will be needed sooner or later, also that we will implement some extensions raher soon, but not publishing them as a valid langugage extension. It is great to keep the flexibility as you mentioned, and this project will need several iterations of many constructs. I think I should have mentioned Extreme Programming, earlier. One necessary is to do without lots of fixed design decisions. It is urgent to be flexible and open to new ideas. In the sprint, we will probably go many different ways at the same time, and drop most of them, soon. Our way of examinining new ways of programming will be as extreme as the principles of extreme programming. There is for sure no other way. [conservative about changing language] > I think you should feel willing to take some liberties with the > language if it makes your implementation cleaner. A lot of the weird > corners of Python seem to me to be implementation hacks based on > CPython internals anyway. Plus, I've mentioned that coding in Python > gives me something like the joy that I imagine that the 1960's Lisp > hackers must have felt. The language itself is in similar shape to > 1960's Lisp, with just two implementations (CPython and Jython), both > of them interpreters. If the development of native-code Python > compilers results in some language evolution like it did for Lisp, > that's natural and not a bad thing. However, it all depends on what > your goals are. You know what my goals are. Smaller, more flexible, faster, easier to change, easier to maintain, easier to keep backwards compatible, more portable due to less C code, down-sizeable by features (which is most difficult), the full catastrophe... > I don't personally see a pure, faithful, exact > reimplementation of a static target whose existing implementation is > free and works perfectly well on a wide range of platforms as being > something I'd want to devote precious volunteer energy to. It's much > more interesting to be able to expand the boundaries of what's been > done before (as Stackless expanded boundaries). However, YMMV. We will try to implement Python as exact and clean as possible. The langage should be implemented completely. At the same time, as much as possible should become pluggable. It will be possible to have MiniPy without floats, without longs, without Unicode, without generators, without bool, without enums, it will be possible to have a Python that cannot generate any new types and classes, and so on. Modules which depend on these features will then not work. It will be a major amount of work to deduce the dependencies of features, and how to arrange them in a scalable shape. I do believe that the core group will help us with that. positively yours -- chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

4 4

RE: Seeking Minimal Python Project Name (was: [pypy-dev] Re: [ann ] MinimalPython project)
by Delaney, Timothy 13 Jan '03

13 Jan '03

> From: Christian Tismer [mailto:tismer@tismer.com] > > There is no official name, yet. > For some time, I was thinking of > "Lilipyt". > "Ptn" was the first thought, while > not sounding very sexy just expressing brevity. > "Pippy" is already occupied. > "Minipy"? Well, there are so many "py" projects, > this makes it hard to come up with a good new name. > > While a name is most unimportant for me, > I agree that it *is* important for the > community to be able to spell it. Mython Minython with an aim of course for it to eventually be Python :) Tim Delaney

1 0

PyVM: Python Bytecode interpreter, in Python
by Paul Swartz 13 Jan '03

13 Jan '03

An idea I've been toying around with is a Python bytecode interpreter, written in Python. Currently, it works decently, supporting pretty much everything except generators. Might be useful to look at. The code is available at: http://www.twistedmatrix.com/users/z3p/files/pyvm2.py -p -- Paul Swartz (o_ http://twistedmatrix.com/users/z3p.twistd/ //\ z3p(a)twistedmatrix.com V_/_ AIM: Z3Penguin

2 3

pypy: static vs. dynmic (was: [ann] Minimal Python project)
by Christian Tismer 13 Jan '03

13 Jan '03

Paul Rubin wrote: > Christian Tismer <tismer(a)tismer.com> writes: > >>>>... that can be >>>>achieved with a static system. >> >>With a dynamic system, you can have backwards compatibility as well, >>together with the flexibility to try alternatives on-the-fly rather >>easily. Changing some very internal things to try something out is >>still not easy, but with welded C code, this is almost impossible. > > > Can you explain what you mean by static vs. dynamic? Yes. Please see below. > Is PHP a static system or a dynamic one? I ask this because the PHP > interpreter has been benchmarked as quite a bit faster than Python. > Of course that must be partly because the PHP language doesn't use > abstract objects so extensively, so the interpreter doesn't have to do > as many dictionary lookups all the time. Sorry, I don't have enough insight into PHP to answer that question. And I doubt I will ever have that, since what I saw my son programming in PHP was much too perlish to invoke my interest :-) (although I'm happy to see him programming at all, hoping he will get back to Python in some future. At the moment, he's trying to distinguish form his father, which I can understand...) I will try to explain my current perception of static and dynamic in the context of building a Python compiler. Please excuse that I'm writing out of the top of my head. This is more a feeling than exact information. Somebody will augment this for sure, and I appreciate it. What is STATIC? --------------- Regardles of PHP (someone more enlighted may judge this), by static I mean something like a C compiler, which is fed by a number of C sources, which are fixed at some compile time, with no chance to get changed. There is exactly one build process, which leads to an executable, and that's it. From then on, this executable has to fit everything, for every application and for all time, until another version is compiled. The static executable is also almost always not optimized for the platform, the processor brand, the amount of physical memory available, and the applications to be run. Finally, the static build is also dependant of a C compiler created by somebody, where it has no real control about. This is especially true for closed source platforms like Windows, but also open sourced compilers like the gcc family are huge projects, very difficult to understand for average programmers like me, so they are black holes as well. That makes optimizations into something like a try and error game instead into a scientific project. I admit that the last argument is not specific to the static tag, it is just an observation that comes for free. Implementing one's own system, specifically designed towards the language to implement and not trying to solve the world's needs for a compiler surely creates an easier to understand and optimize tool. What is DYNAMIC? ---------------- Dynamic means to avoid compling lots of your system based upon a fixed set of C sources, with some typical C compiler. You try to avoid C at all. Instead, you try to express as much as possible with a language that is dynamic by nature (for the implicit definition of dynamicness every Python programmer should feel). You rely on the fact that your Python code will be fed into a specializing compiler which is able to do several things which a C compiler normally cannot do: - The executed code is analyzed at runtime. This gives the compiler the opportunity to optimize code driven by the needs of the actual application. This can lead to very different implementation of certain interpreter activities, in a very application dependant manner. - The machine code can be created with knowledge of the actual hardware platform, to any thinkable extent! If there is a machine specification available that matches the current hardware, every speciality of the hardware can be used for optimization. To my knowledge, there is no such extensive optimization available at this time, but it is possible and likely to appear. The compiler can take measures of cache lines, main memory size and access time, special register sets in the CPU, everything is possible. This is out of the scope of a static C compiler. - By avoiding the so-called C runtime system, or minimizing it to the absolutely necessary, the fence between Python code and library implementation of Python objects vanishes. This opens these things to optimization at runtime. There is no longer a Python interpreter, written in C, and a set of Python objects, implemented in C, with the interpreter calling methods of these objects. Instead, the optmization of the interpreter (written in Python) can dynamically decide, wheather it makes sense to call a method of certain objects in a pre-defined manner, or it can try a specialized version of that method (again written in Python), and it can implement this method inlined into the specialized version of the interpreter, that makes sense in just that current context. A C compiler has no chance to do something comparable, simply because it does not have the runtime info, but especially since it has no idea of Python at all! By writing almost everything in Python, we are able to generate every possible optimization, since we still have the full knowledge of the abstractions, of what we wanted to implement, of our objects and of our targets.* This is what I call 'dynamic', and this is what I hope to be the __future__ of Python dynamically y'rs -- chris P.s.: * This is btw. something that would be lost by using macros, unless we invent a macro language which does know what it is doing. Almost all macro language I've seen so far did a dumb textual or tuple replacement. This is what language designers do when they don't know how to continue. Macros are a powerful extension to weak languages. Powerful languages don't need macros by definiton. -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

1 0

multithread support review?
by Rodrigo B. de Oliveira 13 Jan '03

13 Jan '03

Are you considering reviewing the current state of multi thread support in python? I mean something like removing the need for the GIT :-) Regards, Rodrigo

2 1

link documentation
by holger krekel 13 Jan '03

13 Jan '03

hi, it would be nice if people that recommend good links would also care to write a paragraph why a link might be significant. Maybe you know the theory that *everything* is just six links (or less) away :-) thanks, holger

1 0