Mailman 3 March 2007 - pypy-dev

new architecture document
by holger krekel 24 Mar '07

24 Mar '07

Hi all, i have a first draft of a pypy/doc/new-architecture.txt I've used a new file so that one can easier check back with the old one, and also because architecture.txt is an often visited web link. I tried to implement the following intentions: * separate cleanly between mission/goals and architecture description * keep status information mostly out (particularly of the mission statement and goals) * Talk about PyPy as two things, the translation framework and the Python Interpreter. * clarify terminology, particulary trying to use terms so that it becomes more obvious that PyPy can be used for implementing other dynamic languages, i.e. cannot only be used to implement Python Interpreters (although the latter is a major goal and also helps the translation framework with its architecture) If we'd like to provide status information we should do that in a different section or document, i think. Well, and I am sure it still has lots of room for fixing, improvement and refinement, please hack away and/or tell me what you think :) best, holger -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77)

1 0

Google SoC extension
by Jacob Hallén 24 Mar '07

24 Mar '07

Google has extended the application period for SoC projects to the 26th of March, so if there is anyone who dropped sending in an application due to time constraints, you have a second chance. Jacob

2 1

Re: [pypy-dev] Compiling pypy with python2.5
by Armin Rigo 24 Mar '07

24 Mar '07

Hi Amaury, On Fri, Mar 23, 2007 at 10:57:43PM +0100, Amaury Forgeot d'Arc wrote: > the struct module is built-in in CPython2.4; so the hack does work. Ah, I see: the struct module is pure Python in 2.5, so my theory doesn't hold. (Btw by 'built-in' I meant 'not a C extension module in a separate DLL': for the import "semantics" of CPython, built-in modules take precedence over any other module from any kind of external file, whereas C extensions can be hidden by other modules earlier on sys.path.) But that's not the problem here. As you noted the difference seems to be whether the struct module was already imported before or not. I now think that the proper hack is to make the 'import struct' local instead of global, so that geninterp won't try to freeze it in. The current situation is very strange, btw, because our struct module is not geninterp'ed, but the array module is - and it includes a geninterp'ed version of most of the struct module via this global import. Can you tell me if it solves the problem if the 'from struct import xxx' is moved into the fromstring() and tostring() methods? A bientot, Armin.

1 0

Compiling pypy with python2.5
by Amaury Forgeot d'Arc 23 Mar '07

23 Mar '07

Hello, After some work I managed to run the pypy translation using python 2.5. I know that it is not the best moment because of the imminent release, so feel free to put this under your pillow for now. With some changes, I successfully translate pypy-c on Windows (with --gc=framework), and the produced program seems to work. But I did not rerun the entire test suite. I join the patch (too shy to commit it), with some comments: 1/ The struct module was rewritten in python 2.5. Better use the full python implementation provided by pypy. 2/ KeyboardInterrupt is used by RPython; translation of exceptions must now process all classes derived from BaseException. 3/ This one was difficult, and I am not sure of the correction: with 2.5, IMPORT_NAME takes a new parameter to specify the "relative import" level. CPython-2.5 generate pcode with this extra parameter, but pypy-2.4 can ignore it. The hard part is that pyopcode.py must interpret pcode coming from both the pypy ast compiler and CPython .pyc files. Anyway the implementation in ceval.c itself is not clear (it seems to try to be compatible with 2.4 pcode. How can this work?). 4/ 5/ Easy corrections (not specific to 2.5) while trying the --jit option on Windows: platform.machine() is empty on my machine, and codebuf_nt is expected to export a PTR type. Index: pypy/module/array/app_array.py ================================================================ --- pypy/module/array/app_array.py (revision 41035) +++ pypy/module/array/app_array.py (working copy) @@ -24,7 +24,7 @@ array(typecode [, initializer]) -- create a new array """ import sys -from struct import pack, unpack +from pypy.lib.struct import pack, unpack if sys.maxunicode == 65535: UNICODE_SIZE = 2 Index: pypy/annotation/annrpython.py ================================================================ --- pypy/annotation/annrpython.py (revision 41035) +++ pypy/annotation/annrpython.py (working copy) @@ -8,6 +8,7 @@ from pypy.objspace.flow.model import Variable, Constant from pypy.objspace.flow.model import FunctionGraph from pypy.objspace.flow.model import c_last_exception, checkgraph +from py.builtin import BaseException import py log = py.log.Producer("annrpython") py.log.setconsumer("annrpython", ansi_log) @@ -622,7 +623,7 @@ last_exc_value_var = link.last_exc_value # may be None for non-exception link if isinstance(link.exitcase, (types.ClassType, type)) \ - and issubclass(link.exitcase, Exception): + and issubclass(link.exitcase, BaseException): assert last_exception_var and last_exc_value_var last_exc_value_object = self.bookkeeper.valueoftype(link.exitcase) last_exception_object = annmodel.SomeObject() Index: pypy/interpreter/pyopcode.py =================================================================== --- pypy/interpreter/pyopcode.py (revision 41035) +++ pypy/interpreter/pyopcode.py (working copy) @@ -705,6 +705,12 @@ w_modulename = f.getname_w(nameindex) modulename = f.space.str_w(w_modulename) w_fromlist = f.popvalue() + # AFA: Python 2.5 relative import + # we MAY encounter an additional '-1' parameter + if f.valuestackdepth > 0: + w_level = f.peekvalue() + if space.is_true(space.eq(w_level, space.wrap(-1))): + f.popvalue() w_import = f.get_builtin().getdictvalue_w(f.space, '__import__') if w_import is None: raise OperationError(space.w_ImportError, Index: pypy/jit/codegen/detect_cpu.py =================================================================== --- pypy/jit/codegen/detect_cpu.py (revision 41035) +++ pypy/jit/codegen/detect_cpu.py (working copy) @@ -8,10 +8,13 @@ pass def autodetect(): + mach = None try: import platform mach = platform.machine() except ImportError: + pass + if not mach: platform = sys.platform.lower() if platform.startswith('win'): # assume an Intel Windows return 'i386' Index: pypy/jit/codegen/i386/codebuf_nt.py =================================================================== --- pypy/jit/codegen/i386/codebuf_nt.py (revision 41035) +++ pypy/jit/codegen/i386/codebuf_nt.py (working copy) @@ -19,24 +19,24 @@ globals().update(ctypes_platform.configure(CConfig)) # cannot use c_void_p as return value of functions :-( -LPVOID = ctypes.POINTER(ctypes.c_char) +PTR = ctypes.POINTER(ctypes.c_char) VirtualAlloc = ctypes.windll.kernel32.VirtualAlloc -VirtualAlloc.argtypes = [LPVOID, SIZE_T, DWORD, DWORD] -VirtualAlloc.restype = LPVOID +VirtualAlloc.argtypes = [PTR, SIZE_T, DWORD, DWORD] +VirtualAlloc.restype = PTR VirtualProtect = ctypes.windll.kernel32.VirtualProtect -VirtualProtect.argtypes = [LPVOID, SIZE_T, DWORD, ctypes.POINTER(DWORD)] +VirtualProtect.argtypes = [PTR, SIZE_T, DWORD, ctypes.POINTER(DWORD)] VirtualProtect.restype = BOOL VirtualFree = ctypes.windll.kernel32.VirtualFree -VirtualFree.argtypes = [LPVOID, SIZE_T, DWORD] +VirtualFree.argtypes = [PTR, SIZE_T, DWORD] VirtualFree.restype = BOOL # ____________________________________________________________ def alloc(map_size): - res = VirtualAlloc(LPVOID(), map_size, MEM_COMMIT|MEM_RESERVE, + res = VirtualAlloc(PTR(), map_size, MEM_COMMIT|MEM_RESERVE, PAGE_EXECUTE_READWRITE) if not res: raise MemoryError -- Amaury Forgeot d'Arc

3 3

translator.cli.query bug/patch for win32
by Scott Dial 23 Mar '07

23 Mar '07

Under windows, subprocess.Popen will return stdout with newlines of '\r\n' which exec does not like (python code always has '\n' newlines). The solution is to simply turn on universal_newlines. http://scottdial.com/pypy-dev/translator.cli.query.diff -Scott -- Scott Dial scott(a)scottdial.com scodial(a)cs.indiana.edu

1 0

duplicate extregistry entry
by Scott Dial 23 Mar '07

23 Mar '07

Ever since a few revisions ago, rypthon/extregistry.py has been throwing an assertion: http://scottdial.com/pypytest/41143/failed/annotation.test.test_annmm.py.te… I've not located the change which caused this and unfortunately wyvern isn't showing the same problem. However maybe it is obvious to someone what is broke. -Scott -- Scott Dial scott(a)scottdial.com scodial(a)cs.indiana.edu

1 1

Scheme front end - Google Summer of Code
by Jakub Gustak 22 Mar '07

22 Mar '07

I am deeply interested in writing scheme interpreter/front-end as a part of PyPy Google Summer of Code project. As advised on GSoC site i decided to contact to this mailing list to discuss my application. ### begin ### ================================= PyPy Scheme interpreter/front-end ================================= == Intro == PyPy project's main part is not Python interpreter implementation, but its configurable translator. It provides a good way to avoid writing interpreter for every language and platform. More information: http://codespeak.net/pypy/dist/pypy/doc/architecture.html This project aims to write an interpreter for Scheme for the PyPy framework. Scheme as very interesting dynamic language is nice goal to implement in RPython and provide a translator. It's interesting from a academic point of view to see if call/cc can be implemented on top of the primitives the stackless transform provides. This can lead to possibility of implementing first-class continuations (similar to Scheme ones) in Python. == Project Details == Proposed features: I am aiming to implement almost complete Scheme implementation (according to r5rs specification) in RPython More details: * Complete scheme parser (read macro) in RPython including delayed evaluation and quasi-quotations and macros. * Object space for Scheme covering all Scheme data types. * Research on possibility of implementing call/cc continuations using primitives provided by stackless transform. == Project Schedule == I am able to spend 10 hours/week starting from May until the end of July (end of semester in Poland). During the summer I am able to spend 40 hours/week working on a project. By the end of July I would like to complete the parser, without macros, delayed evaluation and continuations. Then focus on Object space to provide semi functional translator/interpreter. Later I would go on with missing features (e.g call/cc on stackless). If all will go fine we will end up with complete Scheme front-end before mid September, although some restrictions are possible. == About Me == I am a third year student at Wroclaw University of Technology (Poland), Computer Science major. I am interested in meta programming and programming languages design. I have experience in both Lisp and Python languages, including some meta programming. I also am currently studying MIT Structure and Implementation of Computer Programs lectures and text book by Harold Abelson, Gerald J. Sussman and Julie Sussman. I wanted to participate in Google Summer of Code coding in Python or Lisp. This subject gives me opportunity to experiment with both of them. I wrote most of my academic projects in Python. I also am currently working in web application based on Django framework. I am participating in algorithmic contest as spoj.pl or opss.safo.biz writing some of my solutions in Common Lisp, discovering functional programming. I also am a active member (actually a chairman) of IT Academic Association (Akademickie Stowarzyszenie Informatyczne) at Wroclaw University of Technology. I also actively promote free and open software at my University, giving lectures about Xen hypervisor and Kernel-based VM, Vim 7 editor, Subversion and Django-project. Some informations about my work (in Polish): http://jamit.pl/wiki/ http://www.linuxacademy.wroc.pl/ http://www.asi.pwr.wroc.pl/ ### end ### I am looking forward for some feedback from mentors. Sincerely, Jakub Łukasz Gustak

6 10

jit.timeshifter.test uses a lot of memory
by Scott Dial 22 Mar '07

22 Mar '07

I'm not sure if there has been a recent change, but I've always noticed that this test suite requires a lot of memory. What has brought my attention to it especially is that I've been awake while my box has been running the test and it brings things to a crawl with swapping :-( I can't say for sure what the memory requirements are but it is clearly more than what a system with 2GiB of RAM can handle. I can say I've seen it using 1.7GiB (right before I cut it off because I needed to get some work done). My first thought is there is something going terribly wrong due to the tests being run under win32, but I don't get any errors indicating that sort of thing is going on. Is this memory usage intended? Is there anything that can be done to reduce it? I'd hate to have to start skipping those tests. -Scott -- Scott Dial scott(a)scottdial.com scodial(a)cs.indiana.edu

2 3

Summer of Code - deadline extended until next Monday
by Armin Rigo 21 Mar '07

21 Mar '07

Hi all, The Summer of Code deadline for student application submission is approaching fast! But you've just been given the whole week-end to ponder over it. The deadline has just been pushed out to: Monday night, the 26th (midnight in UTC). A bientot, Armin.

1 0

Re: [pypy-dev] [pypy-svn] r40710 - pypy/dist/pypy/interpreter/astcompiler
by Alexandre Fayolle 18 Mar '07

18 Mar '07

On Sun, Mar 18, 2007 at 06:09:33PM +0100, arigo(a)codespeak.net wrote: > Node.typedef = TypeDef('ASTNode', > __new__ = interp2app(descr_Node_new, unwrap_spec=[ObjSpace, W_Root, int]), > #__repr__ = interp2app(Node.descr_repr, unwrap_spec=['self', ObjSpace] ), > - __repr__ = interp2app(Node.descr_repr, unwrap_spec=['self', ObjSpace] ), > + #__repr__ = interp2app(Node.descr_repr, unwrap_spec=['self', ObjSpace] ), Sorry about this. I did not mean to check in the uncommented version in subversion. -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations Développement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science Reprise et maintenance de sites CPS: http://www.migration-cms.com/

1 0