Mailman 3 June 2003 - pypy-dev

Re: [pypy-svn] rev 1064 - in pypy/trunk/src/pypy: interpreterobjspace/stdobjspace/std/test
by Michael Hudson 30 Jun '03

30 Jun '03

hpk(a)codespeak.net writes: > M interpreter/executioncontext.py > fixed exception printing problem (probably a bootstrap issue > regarding string ops) You have the wrong fix here, I'll check in the right one momentarily. It's definitely from the "how the hell did that *ever* work?!" category. Cheers, M. -- You owe The Oracle a TV with an 'intelligence' control - I've tried 'brightness' but that didn't work. -- Internet Oracularity #1192-01

1 0

website update / pypy developments
by holger krekel 30 Jun '03

30 Jun '03

Hi PyPy, just updated the website http://codespeak.net/pypy with some information about what happened at the PyPy-Sprint in Louvain-La-Neuve before EuroPython and at the conference itself. We got quite some attention at the EuroPython 2003 conference where Armin Rigo gave a pygame-based talk and Guido van Rossum was repeatedly seen hacking on PyPy. In his keynote he presented PyPy as a "very ambitious project", adding "It's just fun to work on it". OTOH, everyone was making fun of the fact that PyPy runs 20000 times slower than CPython :-) To bring everybody who is interested up-to-date with recent developments and discussions i am giving a bit of an overview on current developments. - the interpreter is now in pretty good shape. It works nicely with our three object spaces: Trivial dispatches object operations quite directly CPython objects. Standard implements the standard python type system (and methods) including Generators, __new__ methods and metaclasses. Annotation implements "abstract interpretation" which heads for inferencing types for function parameters and return values of a complete python program. this space will form the basis of the soon-to-come code-generator. which doesn't mean that there is not lots of refactoring and improvements on the current code base to do. - we decided to put our evolving documentation into the subversion respository instead of the wiki. This allows for more consistent documents and notifications about changes, versioning etc. Anna has done quite some work in that area. We are using the RestructeredText text format which is rendered into html on the codespeak server whenever documentation changes. - Laura is going forward to present PyPy at the OSCON 2003 and she together with help from other developers improved the paper. IIUC the paper should be transformed into ReST and further improved (i am somewhat starting to do that). - we discussed going for EU-funding. The general oppinion was that the current Call for Proposals fits the possible targets of PyPy pretty well. But going for EU-funding is by far a simple task. We have already talked to some companies and are also looking for research institutions that would want to participate in a big EU-Project. After some discussions especially with Armin i now think that we should promote as a goal to dynamically tie into/adapt arbritrary C-Libraries and operation system calls. The latter would basically mean that you could use Python directly to drive your favourite OS. Think rapidly specializing for whatever embedded device without even a libc and doing that with 99% of the code beeing written in Python. However, we only have three month to fullfill the EU-funding conditions. If we want to involve larger companies then we have to be very quick because large companies often take month to sign contracts. We are grateful for any hint or person you can recommend in order to support us with matching the EU requirements. In return, we can easily help you to envision usages of PyPy for your company or research institution :-) - the EU-funding will probably make it necessary to form some body representing the pypy developers among companies and researchers. The current idea is to setup a "pypy foundation" or even better "py fundation" which brings together three driving forces: Python, Fun & Funding. The latest date to found such an organization will be the next sprint (oh dear) ... - the next PyPy sprint will probably take place in Berlin end of september 2003. It's focus will probably be to emit C-Code in one way or another. After long discussions Samuele, Armin and me think that it's a good idea to target Pyrex as the first "emitted code" language. It has two advantages: - Pyrex handles garbage collection aka hairy details of refcounting and is a nice language to generate :-) - generating runnable CPython modules allows early (and faster!) testing. Basically we want to be able to import pypy as an extension module into CPython *taking over* interpretation of byte codes. - the test framework was improved further but needs more work in various areas. Michael and me even considered rewriting the unittest module alltogether because we need a fine-grained flexibilty regarding object spaces. Something like a test-coverage tool that tells us which code pathes in our programs are not tested. And I'd like a webpage which reports which revision broke which tests and some other nice reports (a nicely rendered commit-log for starters). - we currently have around 10000 lines of code (LOC) and 4 KLOC of tests. it is too early to compare these LOC to CPython's but it's nevertheless an interesting number. so far for now. have fun, holger

2 2

New FP6 Call
by Laura Creighton 21 Jun '03

21 Jun '03

17 June 2003 -- This week (Tuesday). http://fp6.cordis.lu/fp6/call_details.cfm?CALL_ID=74# Click on 'Areas and Instruments Addressed within this Call' You will get the new proposals, all of which could be of interest. But take the time to read IST-2002-2.3.2.3 Open development platforms for software and services If I didn't know any better I would think that somebody in the EU knows all about PyPy and wants to fund us. I think they are describing us to a T. happily astonished, Laura

3 2

Subversion 0.24.2 binaries available
by Jens-Uwe Mager 20 Jun '03

20 Jun '03

I have put the source tar ball and binaries for Windows and Mac OS X for subversion 0.24.2 at: http://codespeak.net/~jum/ Besides fixing some memory leaks and other bug fixes this includes a new command to switch the repository URL, which is quite a useful feature. For example: svn switch --relocate http://codespeak.net:8080/ http://codespeak.net/ will switch an existing repository at the current working directory from our old URL to the new one. Simliarly you can use it to switch between file: and http: URLs and vice versa. This does not work yet on thoth.codespeak.net itself, as I did not upgrade the subversion binaries there. I will do that after the sprint to avoid accidently breaking a working system. -- Jens-Uwe Mager <pgp-mailto:F476EBC2>

1 0

Recent Checkins
by roccomoretti＠netscape.net 20 Jun '03

20 Jun '03

First off, mea culpa. I'm not sure how unpardonable a sin this is, but I now realize that my recent checkins make the test_all.py script fail. This (probably) isn't as bad as it sounds. Firstly, it works fine for the trivial object space - it only has problems with StdObjSpace. Secondly, it's only one file that has problems (test_exec.py), and it seems that the problems were there before the checkin, it was just that the unittest wasn't picking them up. (It was running at interpreter level, not at application level as it should be - and no surprise here; CPython handles the exec statement perfectly :) I'd try to fix the errors, but they seem to be complex, and not directly due to the EXEC_STMT opcode - as I said, TrivialObjSpace passes. You can run test_all for yourself on revision 839+, but here is a synopsis of the errors: * The StdObjSpace dictionary does not have a "has_key" function attribute. * You get a "Type Error: an integer is required" when executing a code object. This error message occurs twice in the standard obj space [objspace.py, concretespace.py], and an unknown number of times in CPython. - We need more specific error messages. * An exception within an exec statement does not bubble up to the caller. - AppTestCase needs more specific messages with self.AssertRaises calls. - Currently it only gives a blank AssertionError (with no message). * Don't pass an integer as the local dictionary to the exec statement. (but an integer for the global dict works fine) That's about as far as I can go for now. I hope nobody minds that I checked in code that gives error messages. The best I can say in my defense is that it was broken before the checkin. Safe travels to all the sprinters, -Rocco __________________________________________________________________ McAfee VirusScan Online from the Netscape Network. Comprehensive protection for your entire computer. Get your free trial today! http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397 Get AOL Instant Messenger 5.1 free of charge. Download Now! http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

2 3

Ideas and Opinions
by Günter Jantzen 20 Jun '03

20 Jun '03

In the moment my pypy understanding is at the Novice Level and all my comments are a bit on the surface "Hey Prof! You forgot a closing paranthese". So I try to browse ideas and opinions: 1)PyPy is not only about Python. Its about compiling and interpreting - translating and executing code. This two approaches are necessary for every programming language. Translating, because a programming language is designed for humans. Executing, because the behaviour of a programm is not decidable. So even compiled languages are eventually executed on a real machine (maybe with some intermediate translation/execution on virtual machines) At compile time we try to collect all informations which are necessary in the target environment/execution context. For example type information in static typed languages, because the target language (assembler) makes a differece in adding ints or floats (as I remember from Armins lecture last year) In Python progamms is not enough type information at compile time, so CPython´s target is not Assembler on a CPU, but Python bytecode on a Python virtual machine. CPython interprets this bytecode at runtime with some opcode machinery PyPy does an abstract interpretation in several ObjectSpaces Usually execution and translation work on different data structures at different times. Translation works at compile time at the static data structure of an abstract syntax tree (given by the parser). Execution works at runtime on the dynamic sequence of instructions (given by the interpration of bytecode) which is the input for abstract interpretation. If we want to do some magic translation/execution entanglement we have to think about doing the right thing at the right time (several times of execution and translation) and about mappings between our dynamic and static data structures. For example in interpretation code we enter a loop which we want to optimize. We can´t do this earlier without receiving a big memory overhead, because earlier we havent got the necessary type informations. So we follow the mapping from our instruction to the the abstract syntax tree, set type labels at some nodes and do some local translation and optimization. As a result we receive (local?) changed bytecode for the body of our loop. Give it to the interpreter and execute it. This hot spot approach will give us the freedom not to calulate all execution pathes of a programm, but only the actual necessary ones. Another approach to filter only important pathes could be in combination with a test framework. Execute all your tests and cache the dynamic generated code in a database. When your programm is still slow, you forgot maybe some tests. 2) At a first glance, Pyrex seems to me a beauty example for a concrete syntax which is not in the way of the programmer. (of course its more than that). Pyrex can be annotated with C-style static informations. But it is not necessary to do this. You can program without annotations. Then its just Python. If you use annotations, Pyrex generates the C-source for extension types ------example--------- def primes(int kmax): cdef int n, k, i cdef int p[1000] result = [] if kmax > 1000: kmax = 1000 k = 0 n = 2 while k < kmax: i = 0 while i < k and n % p[i] <> 0: i = i + 1 if i == k: p[k] = n k = k + 1 result.append(n) n = n + 1 return result --------------------- If we add some concrete syntax in pypy, we will not only optimize the language core. We will give the Application programmer the opportunity to choose her codestyle in a scale from dynamic to static. For example, all modules of the Python Standard Library should be written as static as possible. Günter P.S: comments and clearing of misunderstandings are welcome

2 2

EU funding
by Laura Creighton 20 Jun '03

20 Jun '03

I found another page of great links. Here is the text I added to the wiki, under PyPyFunding: Here is a Guide for how to write a proposal. http://europa.eu.int/comm/research/fp6/index_en.html You want to click on the 'participating in European Research' links. Available in German French Italian and Spanish as well as English. Laura

1 0

More PyFunding Links
by Laura Creighton 20 Jun '03

20 Jun '03

I found another page of great links. Here is the text I added to the wiki, under PyPyFunding: Here is a Guide for how to write a proposal. http://europa.eu.int/comm/research /fp6/index_en.html You want to click on the 'participating in European Research ' links. Available in German French Italian and Spanish as well as English. Laura

1 0

finished apache2 upgrade
by holger krekel 20 Jun '03

20 Jun '03

hello pypy, the upgrade to apache2 is completed. So from now on you can access the subversion repository via http://codespeak.net/svn/pypy/trunk and don't need the ":8080" any more. Neverthelss, port 8080 is served by the same apache2 so that continues to work. We will fade out port 8080 end of next week, so don't hesitate to "recheckout" pypy. There is no way to switch URL's in a checked out directory except if you run a very new svn-client (option "relocate" or so) So you have to check it all out again... sorry for the inconvenience. Also we automatized some stuff like changing passwords with "passwd" on shell-level should change your svn-password. Plus some other stuff which you probably don't want to know right now. cheers, holger & jum

1 0

temporary downtimes today (thursday)
by holger krekel 19 Jun '03

19 Jun '03

hello pypy, the codespeak apache2 setup will be enhanced today. The goal is to have everything (instead of just subversion) running on apache2. If we get this running then the subversion url will eventually be http://codespeak.net/svn ^^^^ note the "missing" :8080 We will also change ownerships and permissions of apache2 around so you might temporarily be unable to access codespeak.net:8080 (current subversion url). We'll report when we are finished. holger & jum

1 0