Mailman 3 January 2003 - pypy-dev

(no subject)
by logistix 22 Jan '03

22 Jan '03

> -----Original Message----- > From: pypy-dev-bounces(a)codespeak.net [mailto:pypy-dev- > bounces(a)codespeak.net] On Behalf Of Rocco Moretti > Sent: Monday, January 20, 2003 11:08 PM > To: pypy-dev(a)codespeak.net > Subject: Re: [pypy-dev] question about core implementation language > > logistix <logistix(a)zworg.com> wrote: > > >I think the point is that Python as a language provides absolutely no > >access to the machine internals. You can't examine memory contents, > >registers, I/O bus or call code directly. > > Can you do this all in C? (without library functions coded in > assembly) > > >As part of the bootstrap > >process the following four functions should probably be added to the > >interperter (either as builtin functions or a module): > > > > GetMemoryFromOS() > > ReturnMemouryToOS() > > Peek() > > Poke() > > > >ANd probably: > > get/putRegister() > > callCode() > > > >Other than that, everything could (eventually) be written in Python. > > If we were writing an Operating System, I would agree with you. (We > can put down a Python based OS as another goal of the project. <while > 1:wink()>) > > Fortunately, we aren't quite so low level. The standard OS functions > tend to be higher level. As I understand it, much of libc.so on Unix > systems is simply a very thin wrapper around OS functions which behave > much like the C functions that call them. > > Since we don't want to reinvent the wheel, it would probably be best > to reuse the standard interface provided. This is where the minimal C > core comes in. We can probably limit it to basic input and output > (PyBIOS?), plus a generic mechanism for discovering and calling dll/so > functions. > > Or am I missing your point entirely? > > -Rocco > Yeah, Armin also seemed to disagree with me. Here's where I'm coming from (and maybe I'm missing a very big part of what psycho does). Okay we've got a theoretical python interpreter written in python. It: Compiles to python bytecode Uses this to generate C code(or other backend) Uses VC++ or gcc to generate machine code This all works well and good until the interpreter calls eval() or exec (like it does via the code module anytime you do something in IDLE or PythonWin, or even importing an updated module) what happens in this case? The way I see things now, you'd have to: Compile to bytecode Generate C (or other backend) Create dll/so Load DLL/SO into python I would want some way to load code directly into memory (and preferably with minimal File I/O). Hence a few low level memory functions (coded in C or assembly) that are callable from the Python framework. Everything else would be written on a higher level. Is the above the intended workflow for interactive compilation or am I just being an idiot here? ;)

2 1

RE: [pypy-dev] How to translate 300000 lines of C
by roccomoretti＠netscape.net 22 Jan '03

22 Jan '03

On another thread Christian Tismer <tismer(a)tismer.com> wrote: >I tried to map [frameobject.c] on a 3.5 hour yourney from >Kiel to Berlin, and I had one fourth done by an hour. >Nevertheless, I got into trouble, just by comparing its >implementation differences between 2.2.2 and 2.3a. >Then I dropped that and decided that this is the wrong way. I'm interested to know what problems you were encountering. One issue I recall from frameobject.c is that it has a lot of optimizations regarding object caching and reuse. As a first approximation, we should probably ignore such optimizations. If we do add it in later, it should probably be as an optimization by psyco for *all* objects. That said, theoretically, I like the idea of of a C->Py converter. (It did get tedious when I was doing it.) However, I'm concerned if the time invested to make one would be worth it. Christian Tismer <tismer(a)tismer.com> wrote: >There are a number of free-ware C compilers around, and also >some C interpreters. But coded in which language? How difficult would it be to get them to "play nice" with Python (which is, I am assuming, where you want to code the conversion logic)? - This question probably boils down to asking what compiler/interpreter you have in mind to co-opt. One caveat with me evaluating the benefits of this idea is that I don't have a feel how difficult tracking changes in CPython would be. We do have the changelog and Unit tests for CPython, so we wouldn't necessarily need to do a line-by-line comparison. We could approach the changes in more of a top-down level. Isn't Python supposed to be easier to maintain than C? How does Jython do it? Just my two atoms of copper-coated zinc. -Rocco __________________________________________________________________ The NEW Netscape 7.0 browser is now available. Upgrade now! http://channels.netscape.com/ns/browsers/download.jsp Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/

5 10

RE: pypy-dev Digest, Vol 13, Issue 2
by Gerald S. Williams 22 Jan '03

22 Jan '03

Rocco Moretti wrote: > How would the interface between the two work if the hand translated sections differ > from the C version? I can give general answers from past experience. > Simplest example is basic types. A Python version would presumably be represented as a > class with member functions. But the C code uses calls to accessor functions. How could > we redirect the automatically generated code to call member functions as opposed to > free floating functions? One answer: provide such functions, which call the member functions as needed. > A more difficult problem lies when we we start to reorganize the object/structure > internals. Good case of this is the frame object. The C version has an array with > multiple pointers to internal objects. Once you reorganize the internals, how do > you tell the C->Python translator that f_stacktop isn't a PyObject pointer anymore, > but an index off of another list? If the class was providing public attributes, you may not want to change their meaning. f_stacktop could provide an accessor for the old representation. The new representation would be in _f_stacktop or something. -Jerry

1 0

Re: [pypy-dev] How to translate 300000 lines of C
by roccomoretti＠netscape.net 22 Jan '03

22 Jan '03

One issue re: the automatic C to Python translater just occured to me. It's been mentioned that only portions of the C code would be automatically translated, certain sections being translated by hand. How would the interface between the two work if the hand translated sections differ from the C version? Simplest example is basic types. A Python version would presumably be represented as a class with member functions. But the C code uses calls to accessor functions. How could we redirect the automatically generated code to call member functions as opposed to free floating functions? A more difficult problem lies when we we start to reorganize the object/structure internals. Good case of this is the frame object. The C version has an array with multiple pointers to internal objects. Once you reorganize the internals, how do you tell the C->Python translator that f_stacktop isn't a PyObject pointer anymore, but an index off of another list? Not to dissuade the effort, but I'm concerned we could get a situation where we are hesitant to rearrange the internals for the better because it would break our tools. Cautiously Optimistic, -Rocco __________________________________________________________________ The NEW Netscape 7.0 browser is now available. Upgrade now! http://channels.netscape.com/ns/browsers/download.jsp Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/

1 0

MinimalPython newbie questions
by Sébastien Pierre 22 Jan '03

22 Jan '03

Hi all, I recently found MinimalPython and immediately subscribed to the list after seeing that it was trying to make a better, smaller CPython with ideas from Psycho and most notably Stackless. I found many interesting ideas in Stackless, most notably the notion of microthread, which seems particularly adapted to the implementation of "multi agent systems". Looking at the <www.tunes.org> site, which is maintained by Armin, I found also many ideas that would ease the life of people who want to write "agent-based applications". My question is: will MinimalPython try to incorporate the microthreads from stackless and will try to incorporate ideas from tunes project (like migration) ? If so MinimalPython would really be an ideal language for developing agent-based applications. Cheers, -- Sébastien -- «Si tous les autres acceptaient le mensonge imposé par le Parti - si tous les rapports racontaient la même chose - le mensonge passait dans l'histoire et devenait la vérité» <http://www.type-z.org> -- George Orwell, 1984

5 5

Sprint dates
by Armin Rigo 22 Jan '03

22 Jan '03

Hello everybody, If we are running for a February sprint, we should fix the dates now. I've no preference myself in February, but cannot free myself during the first week of March. I remember Christian suggesting dates but I seem to have lost them somewhere... A bientôt, Armin.

3 3

Keeping it simple
by Chui Tey 22 Jan '03

22 Jan '03

Hi All, Minimal Python sounds cool. :) I think we should try to stick as closely to the original objectives of the Minimal Python project, namely to aid in the fast prototyping of the Python VM, and reducing the C core to a minimum. Work could be started on two ends: a) making the types in the C interpreter even more Python like. eg. making every attribute in the frameobject writable, and making it subclassable in Python itself. b) Providing a set of patches that hooks the C core into using alternate implementations particular bytecodes. For instance, we might have a Python implementation of BUILD_CLASS instead of the services provided by the C-Core. ... y) Having a python implementation of a bytecode interpreter z) making some of the services of the python implementation eg. frameobjects available to the C core. Eventually this may lead to a version of PyPy where the main interpreter loop lives in Python by using a very small set of core CPython services like lists, and dictionaries (circa Python 1.0). The aim of making PyPy run faster then CPython is noble. However, even if it is not achieved, PyPy will still have great utility. You see, CPython has been finely tuned for many years and there is no reason why people will continue to tune it for many years to come. The tuning has come at the cost of understandability in some cases. A Minimal Python would fill a void in providing a reference implementation which can be understood in a single sitting. Aligning PyPy closely with CPython will aid the longevity of the Python Python project and encourage wider adoption of PyPy from the core python developers. Chui

1 0

RE: [pypy-dev] Objective of minimal python
by Delaney, Timothy 22 Jan '03

22 Jan '03

> From: Armin Rigo [mailto:arigo@tunes.org] > > On Mon, Jan 20, 2003 at 12:51:00PM -0600, Nathan Heagy wrote: > > This might be as good a point as any to jump in with my > question: is > > there planned support for non-x86 platforms? > > Yes, I expect the new Psyco-like code to be much more > flexible and easy to > port to other processors. We are also talking about mixed approaches > involving gcc. I would think that there should be multiple possible targets from psyco depending on the platform. 1. By default the output format is python bytecode. 2. Optionally, C source code could be output and compiled on-the-fly (this would really need a persistent cache to avoid the big hit). 3. For specific platforms, machine code could be output directly. 4. Finally, Python source code which matches the C code that *would* have been produced in (2) as a reference. I would think (1) and (2) would be the first to be implemented. At this point, I don't consider performance to be *any* kind of objective. Getting a working version on one platform, then a working version on all platforms are the first steps. Tim Delaney

1 0

[pypydev] C to Python translation
by Scott David Daniels 22 Jan '03

22 Jan '03

As I read some of the plans here and do the standard "Oooh, Good Idea; Oh No, there's a swamp; Oh, Maybe" dance in my head, I had what might either be a flash of inspiration or maybe a deep lack of insight: Instead of having the C-to-Python compiler derived from lcc (or whatever) in charge of generating python, maybe we can make it track points corresponding to changes in the in C programs. Its job would be simpler: no code generation at all. Find identifed points in a C module which may have changes in it, and indicate the corresponding points "hand-translated" program. This is in part simpler because the "hand-translated" program can have comments (or something) indication what original source range they came from. By original source range I do not mean line, character, or token numbers, but locations that are somehow rediscoverable with a parser on altered source. At the roughest cut, this could be declarations and definitions, but what level of detail you need these points should be much clearer to Christian than to me. For concreteness: A : original CPython source (version 2.33) B : hand-translated version of A Z : possibly unnecessary file describing the A->B mapping and how identification points in A correspond to those in B A': New improved CPython source for A (version 2.34) The tool woud be responsible for taking as input: B Z A A', a diff of A and A' and producing change ranges in B in human readable form (and the corresponding ranges in A and/or A'. It would not automate any code semantics comparisons, but it could focus the maintainer's attention on where changes might have effect. This scheme might reduce the work of hand-tracking changes while avoiding the nightmare of Python-in-name-only code that comes from translating C code to python mechanically. On the other hand, we have five fingers. -Scott David Daniels

1 0

Re: [pypy-dev] Restricted language
by Armin Rigo 21 Jan '03

21 Jan '03

Hello Christian, On Tue, Jan 21, 2003 at 03:58:41AM +0100, Christian Tismer wrote: > Just to make sure I understand what you mean (private mail). > > Having hacked on CPython for a very long time now, I always > had the impression that CPython never does any distinction > withing exception, whether they are caused by a script, > or whether they are caused inside a builtin C object. > I the runtime decides to throw a memory error, it is the > same, as if there is a syntax error, or an error in a user > object. Yes, I think you are right. There is some confusion already in the C core which uses exceptions both for user-program-visible exceptions and internal handling. For example, the fact that a generator can signal it is exhausted by throwing a StopIteration exception (instead of finishing on a "return"): the StopIteration exception is *internally* used to signal the end of iterators and a user-level StopIteration should never have been able to interfere. Of course, given the way CPython's exceptions are coded, it seemed natural at the time to add in the definition of the Python language that you can raise a StopIteration to stop generators... For memory errors it is still a bit more fuzzy, because it is clearly an internal error, but there isn't much else that the core can do about it than turn it into a user-level visible error to cleanly interrupt the user program... In Python the RuntimeError exception is reserved for internal errors. Ideally it should have been more clearly separated. But it is tempting to re-use the existing exception mecanism for RuntimeErrors. Other interpreters do that, BTW (e.g. Java). In Python-in-Python we must internally raise an EPython exception to signal application-level exceptions, and use the normal exception mecanism for all other internal error conditions (e.g. use assert's or raise ValueErrors when internal calls are made with bad values). In a first phase these would just crash the interpreter with a traceback that is useful for debugging. In a second phase (and only then) we can make the interpreter more robust by catching *all* unexpected exceptions in the main loop and throwing them into the application, which would then see them as RuntimeErrors whatever the original internal exception class had been. I know this can be quite confusing; it took me some time to figure out this, but now I am confident that it is the correct point of view. Again, think about implementing a Python interpreter in another different language that also has its own (different) family of exceptions. If the interpreter internally raises an unexpected exception, you cannot show it as is in the Python application because it may have no immediate equivalent. So you let the application see a generic RuntimeError exception. Conversely, if a part of the interpreter wants to raise, say, a ValueError, there may be no directly-corresponding exception existing in the interpreter's language; you have to define a new kind of exception in the interpreter's language, and throw and catch that exception internally. This new kind of exception can be generic ("EPython") and embed information about the Python type and value of the application-level exception. A bientôt, Armin.

1 0