Mailman 3 May 2005 - pypy-dev

Just so you know...
by Armin Rigo 14 May '05

14 May '05

Hi all, Samuele and me have been working on a better approach for GenC, something less hackish. It seems that we have a nice model by now, but it's still a bit experimental. The description of this approach requires more than I want to write down right now; we're starting a motivation / documentation / TODO, in the following folder: http://codespeak.net/svn/user/pedronis/llrpy/ The prototype is in ll.py, the text in overview.txt. The idea is that we'll send a more useful e-mail to pypy-dev once overview.txt gets completed and expanded. In the meantime you can try to make sense out of that if you feel like it :-) (Note that it's not Yet Another Completely New paradigm; there are a lot of ideas that have already been floating around. But it seems that we are starting to put these ideas into a consistent whole now. Also, the approach should be able to reuse a lot of the infrastructure that we already have now -- flow graph, annotator, etc.: it's unlikely that we will need yet another large subdirectory of pypy :-) A bientot, Samuele & Armin

5 7

Re: [pypy-svn] r12253 - pypy/dist/pypy/tool
by Christian Tismer 13 May '05

13 May '05

ale(a)codespeak.net wrote: > Author: ale > Date: Sat May 14 02:19:17 2005 > New Revision: 12253 > > Modified: > pypy/dist/pypy/tool/_enum_exceptions.py > Log: > added arguments checking. > > There are checks for correct number of arguments and correct types (for the case I understand an exact nuber of arguments > > I haven't run any test - it is too late for that. Will do in the morning Looks well done! After all, looking at this stuff again, I still like the approach, although it is limited in a way that you write a program that actually tries to catch stuff that you know already. Originally, I planned to catch all the __str__ formatting methods by probing them, too. But this seems to be very hard, especially in the only relevant case of unicode. I'd like to keep the code alive. Perhaps we can find an application for other cases. Especially the __init__ tracking seems to be working very well and could be extended. ciao -- chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

1 0

(no subject)
by Ben.Young＠risk.sungard.com 13 May '05

13 May '05

Hi there, I have been following the development of pypy and have a couple of questions about the recent development. I was wondering what the exact reasoning behind the new low level code. Is it just there to replace the code templates that are in genc? or is there a deeper reason? Already the code seems much more complicated than the code templates (and seems to contain its own templates in rlist.p for example). Will the new code be easily transferrable to other backends (e.g llvm or java) or is it tied to the c stuff? Sorry if these are silly questions, I am just trying to get a better understanding of why all this stuff is changing when you seemed so close to getting more goals finished. Keep up the good work! Cheers, Ben

2 1

europython sprints/upcoming activities
by hpk＠trillke.net 12 May '05

12 May '05

Hi, i have been asked by Erik to post some preliminary information on the upcoming EuroPython sprint(s). I guess we'll make some more complete announcement soon but meanwhile here is some brief information on upcoming activities in the next two months: - 15th-20th May IRC-coordinated session ending in a first PyPy release (targeting the goal of having an interpreter that passes 90% of CPython's core language tests but still runs on top of CPython). - 23rd-26th June: Pre-EuroPython sprint for people who already attended sprints previously (IOW this is not the best time for newcomers to join) - 27th-29th June: EuroPython! - 30th June: a day of break (really) - 1st-7th July: Post-EuroPython sprint open for anyone interested in PyPy development. It's likely there will be a strong focus on translation and low level backends. - 8th July: PyPy/EU-Project evaluation workshop The phyiscal meetings will all take place in Goetheborg (Sweden) where this year's EuroPython happens. Sprint locations, meeting points etc.pp. are to be announced soon. cheers, holger

1 0

Magic problems
by Christian Tismer 10 May '05

10 May '05

Hi, during my wrestling with the ovfcheck "macro", I came along a couple of problems, which are unrelated but nevertheless noteworthy. I discovered, that when testall.py ../*dunno*/test_geninterp.py is run, the generated code does not expose any exception branches!!! This is probably due to the fact that some magic module makes all exceptions into AssertionError subclasses. But these are removed by simplify.py ! I was unable to follow the magic logic of this, but the pygame view was at least able to show me that we really have a problem here. You can try this by writing a small function call, which usually should expose at least one simple exception outlet. It won't, but becomes a straight sequence of simple blocks. Nice, anyway, but unfortunately Python world is not that nice :-) This problem seems to be local to geninterp.py, which since a while tries to completely believe into __builtin__ objects as god given. Maybe I should modify this and look into tjhe exceptions module more closely, or is this solvable from the testing environment? ciao - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

2 2

Re: applevel _exceptions.py
by Armin Rigo 09 May '05

09 May '05

Hi Holger, > In addition i wouldn't mind if we allow to modify the __str__ methods > directly in the applevel file at least while staying with a minor python > version, it's also more straightforward to test via lib/test2 then. In both case, i.e. whether we write __str__ directly in lib/ or generate them from elsewhere into there, it's possible (and a good idea) to write a test in lib/test2/. A bientot, Armin.

4 4

Compiler
by Armin Rigo 09 May '05

09 May '05

Hi Ludovic, hi all, I had a look at recparser, and how to integrate it into PyPy. Ideally, it can be exported as the 'parser' module by adding a line to interpreter/baseobjspace.py (see the commented-out line about the other 'parser'). A few comments about the interface file pyparser.py (this should be put in some documentation...): * applevel() requires obscure tweaking about the 'import compiler' statement, the prevent the whole compiler package to be dragged in and compiled by PyPy (which may be what we want later, but for now it just doesn't work, I expect). I checked that in. * the interpleveldef exports a class, 'STType'. I added another hack in lazymodule.py to make that work. Basically, the interp-level exports had to be wrapped objects, or functions -- which get wrapped automatically. Types now also get wrapped automatically. Previously, you'd have needed an interpleveldef like 'STType': 'space.gettypeobject(pyparser.STType.typedef)' which fishes the typedef (i.e. the definition of the app-level type) corresponding to the class STType, and asks the space to build a real app-level type object for it. At the moment, with the above changes, it appears to work rather nicely (at least the few exported methods). But we cannot feed the parse tuples to the pure Python compiler package because the latter expect tuples with line number information, and as far as I see you're always generating tuples without. It seems that you're collecting the information already so it should not be difficult to fix. The next step would be to integrate it so that it is used by the built-ins, like compile(). There is a new abstraction, class Compiler, in pypy.interpreter.compiler. Its purpose is to be subclassed by concrete compilers; currently there is only CPythonCompiler, which cheats and calls compile() at interpreter-level. I guess that it should be possible to create another subclass that uses recparser and the pure Python compiler package to do its job, or even a generic PythonCompiler that uses whatever built-in 'parser' module is available, and then the pure Python compiler package. All of PyPy ends up using the compiler instance is stored in the current execution context whenever it needs to compile source code (including at the interactive prompt). Finally, a quick look over the recparser sources shows a few constructs that are clearly not "RPython", i.e. too dynamic. We need to think a bit and see how to address the issue. About RPython: http://codespeak.net/pypy/index.cgi?doc/coding-style.html#restricted-python Before we actually try to perform type inference on recparser, it's a bit hard to know if there are type problems or not. It is often the case that even when we write code knowing that it should be RPython we oversee some subtle typing problem. I'll give it a try, I guess (this is done by enabling the recparser module in baseobjspace as hinted above, running "dist/goal/translate_pypy.py targetpypy", and trying to make sense out of the obscure assertion errors and enormous flow graphs we get...) For now, a problematic feature that is obvious is the visitor pattern that you use extensively. It's definitely a great pattern, but not one that immediately applies to C- or Java-like languages. I'm not saying that you should rewrite all of recparser; more that we need to find a trick to implement visitor patterns without the getattr() with a computed attribute name. Possibly something along these lines: class MyVisitor: def visit_name1(self, node): ... def visit_name2(self, node): ... # this can be computed by a for loop instead: VISIT_MAP = {'name1': visit_name1, 'name2': visit_name2, } class Node: def visit(self, visitor): visit_meth = visitor.VISIT_MAP[self.name] visit_meth(visitor, self) The difference with the getattr() case is that the operation that replaces it, a getitem on a constant dictionary, has a reasonable C-level equivalent, namely a (precomputed) hash table lookup. That's it for now. Don't hesitate to ask if I'm not making sense, or for more help about integration issues. I am aware that it is some kind of guesswork at the moment. Just feel free to post to pypy-dev. A bientot, Armin.

2 2

do we still need to generate applevel _exceptions.py?
by hpk＠trillke.net 08 May '05

08 May '05

Hi pypy-dev, We have pypy/lib/_exceptions.py which is a generated applevel definition and gets translated to pypy/module/exceptionsinterp.py by Christian fine geninterp mechanism. Now to fix some remaining compliancy bugs it seems easy to fix the applevel exceptions definition unless there is a need to still be able to regenerate that as well. I'd guess the latter is not neccessary anymore so is it fine to just fix pypy/lib/_exceptions.py and regenerate? And what was the incantation again to regenerate the exceptionsinterp.py again (did i miss some piece of documentation somewhere)? cheers, holger

3 2

Re: [pypy-svn] r12044 - in pypy/dist/pypy/objspace/std: . test
by Michael Hudson 08 May '05

08 May '05

pedronis(a)codespeak.net writes: > + assert sorted(d[0].keys()) == ['__dict__','__metaclass__','__module__'] So we require Python 2.4 now? Cheers, mwh -- The above comment may be extremely inflamatory. For your protection, it has been rot13'd twice. -- the signature of "JWhitlock" on slashdot

2 2

Re: applevel _exceptions.py
by Armin Rigo 08 May '05

08 May '05

Hi Holger, Christian should know better than me if we'll want to regenerate _exceptions.py in the future. About your second question, I guess that it's time to remove exceptionsinterp.py from svn and use the more general compile-and-cache mecanisms by now, which has the obvious advantage that we don't have to fish for the incantation (which can be found in the svn log)... Armin

1 0