Mailman 3 July 2006 - pypy-dev

[emil@dmr.ath.cx: Re: [pypy-svn] r29899 - pypy/extradoc/talk/ep2006]
by Armin Rigo 10 Jul '06

10 Jul '06

Hi all, ----- Forwarded message from Emil Mikulic <emil(a)dmr.ath.cx> ----- Date: Mon, 10 Jul 2006 17:23:05 +1000 From: Emil Mikulic <emil(a)dmr.ath.cx> To: arigo(a)codespeak.net Subject: Re: [pypy-svn] r29899 - pypy/extradoc/talk/ep2006 On Sun, Jul 09, 2006 at 03:35:03PM +0200, arigo(a)codespeak.net wrote: > http://snake.cs.uni-duesseldorf.de/tgtest/ I see: Bad Gateway The proxy server received an invalid response from an upstream server. Apache Server at snake.cs.uni-duesseldorf.de Port 80 =/ ----- End forwarded message -----

1 0

PyPy Post-EuroPython 2006 Sprint Report
by Michael Hudson 09 Jul '06

09 Jul '06

Greetings from an assortment of tired brains at CERN, home to a larger assortment of hopefully less tired physicist brains. We are reaching the end of a succesful and well-attended (24 participants!) sprint. 3.5 days times 24 people is a bit more than two man months of work -- we are going to try to cover all we've achieved, but will undoubtedly forget some stuff! Summer of Code student Lawrence and his mentor Arre spent the majority of the time integrating Lawrence's SoC work -- ctypes implementations of various CPython extension modules -- into PyPy. He has implemented the time, mmap and fcntl modules so far, and the work at this sprint concentrated on putting the time module into the form required for PyPy -- making it a mixed module, conforming to the restrictions of rctypes and fighting the extension compiler. Brian Sutherland and Sad Rejeb also joined in this play, attempting to convert some prewritten performance sensitive caching code used in Zope(!) somewhere to a mixed module suitable for feeding to the extension compiler. Unfortunately the resulting code was no faster than the pure Python version... probably because of the ext-compiler using refcounting for GC. Antonio appeared to spend his entire time writing commit messages but somehow must have written some code in between (with the help of *his* SoC mentor, Armin) because now all of PyPy's Standard Interpreter can be rtyped with the ootypesystem. "All" that remains are the dict implementation and external function support in the CLI backend before IronPython has a competitor in the shape of PyPy.Net :-) Guido (not *that* Guido) being the JavaScript deity he is wrote a JS tool to instrospect the DOM methods and produce useful information for Maciej's SoC (we only had three SoC students here...) project. Unfortunately browsers' JavaScript implementations are famously eccentric and running his tool popped up the Print dialog box twice and then crashed Epiphany. It proved marginally less frustrating to write a scraper for Mozilla's documentation and fix the broken HTML he found there. Aurélien, Ludovic and part of Armin tried to figure out why the example of coroutine cloning written for the "What can PyPy do for you?" talk didn't work. Assorted mind boggling bugs were found and fixed, with the aggravating result that it now works fine when run on the llinterpreter but crashes -- after a while -- when translated to C. More wizardry will be required here... Just before the sprint Maciej attempted to improve the annotator's error messages and he and Samuele spent some time fixing this new code (it's very frustrating when your error printing routines crash on you...). A varying team of people including but not limited to Pieter, Carl Friedrich, Maciej and Alexander worked on alternative, hopefully faster implementations of Python data types, particularly strings and dictionaries. We now have efficient representations of slices of strings and for the result of adding two strings, all completely transparent to application-level. For a sufficiently cooked benchmark, we are now up to seven times faster than CPython (and use 30 times less memory). On the dictionary side, we stole an optimization for CPython: assume on creation that a dictionary will only contain strings keys and switch to an alternative implementation on the insertion of a non-string key. This makes pystone and richards both around 20% faster, some benchmarks up to 40% faster (but also some a few percent slower). Alexandre and David started to think about configuration and option handling, but got distracted when they tried to run a translation on a 64-bit machine (or more specifically, a machine with 64-bit userland), because it broke in obscure ways. A few obscure fixes later and it worked again apart from stackless, and another few fixes fixed that too. Michael and Fabrizio worked on the stackless code, "un-inlining" some of the frame-state saving code with the goal of reducing the size of the generated code. This involved taking Fabrizio on a whirlwind tour of one of the scarier areas of the PyPy codebase, but he survived and the feature was implemented, although the size reduction was not as great as had been hoped -- more investigation required (we also need to build a stacklessgc build to check that). Michael and Fabrizio then worked on AST- and bytecode-level optimizations, which mostly involved building an infrastructure to make modifications of the AST easier. They implemented a simple constant folder at the AST level and at the bytecode level an optimization to remove the temporary tuple in code like "x,y=y,x". Simon and Eric resurrected and experimented more with LLVM's JIT interface. Simon then worked with Ludovic and Arre on implementing RPython support for numarrays and Eric worked on making the output of genc compilable with C++. Guido and Holger implemented the beginning and perhaps the middle but probably not the end of doctest support in py.test. After some discussion with Alexandre, David and Holger, Guido and Carl Friedrich began to work on the rather ugly-looking task of taming pypy's growing menagerie of options. They started to implement a general mechanism for defining a hierarchy of options, handling dependencies and conflicts among them. From this you can generate optparse Options for command line interfaces and will one day be able to generate a web form to choose among the zoo of pypy's options. They used the config mechanisms to de-insane the selection of the various object space variants. This, for example, makes automated testing of the above-mentioned variant string and dictionary implementations possible. Pieter worked on random modules^W^Wthe random module, porting it to RPython and implementing some missing functions. So, after a pre-sprint (for some of us), a conference and a post-sprint, we are well prepared for the usual post-sprint week long sleep. Apart from Carl, he has an exam on Friday. Oops! Met vriendelijke groeten, mwh&cfbolz -- This proposal, if accepted, will probably mean a heck of a lot of work for somebody. But since I don't want it accepted, I don't care. -- Laura Creighton, PEP 666

1 0

Question on pypy exception...
by Jiwon Seo 08 Jul '06

08 Jul '06

I'm getting an exception with pypy-c, and the exception occurs right after pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr function in pypy_g_PyInterpFrame_dispatch_translated. 1. Is "pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr" same as LL_thread_fused_releaseacquirelock ? I can only see "#define LL_thread_fused_releaseacquirelock pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr", but not the other way around. 2. Is that function there for forcing context switching? 3. What could happen in pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr() that can set (&pypy_g_pypy_translator_c_exceptiontransform_ExcData)->ed_inst_exc_type? the error message that I'm seeing is like following propagating implement_6.c:100878 pypy_g_PyInterpFrame_dispatch_translated error propagating implement_5.c:30508 pypy_g_PyFrame_eval error propagating implement_5.c:37782 pypy_g_EvalFrame_resume error propagating implement_39.c:41465 pypy_g_ll_deallocator__Address_68 error propagating implement.c:27685 pypy_g_funccall_star1 error propagating implement_39.c:32105 pypy_g_ll_deallocator__Address_14 error propagating implement_39.c:32158 pypy_g_ll_deallocator__Address_15 error propagating implement_39.c:32294 pypy_g_ll_deallocator__Address_17 error propagating implement.c:1336 pypy_g_entry_point error Thanks! -Jiwon

1 0

Re: [pypy-dev] question on using ctypes
by Amaury Forgeot d'Arc 06 Jul '06

06 Jul '06

Hello, 2006/7/5, Jiwon Seo <seojiwon(a)gmail.com>: > After properly installing ctypes, translate.py now gives this error. > > ################################################################## > > [translation:ERROR] Error: > [translation:ERROR] Traceback (most recent call last): > [translation:ERROR] File "./translate.py", line 332, in main > [translation:ERROR] drv.proceed(goals) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 506, in > proceed > [translation:ERROR] return self._execute(goals, task_skip = > self._maybe_skip()) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/translator/tool/taskengine.py", line > 108, in _execute > [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 195, in _do > [translation:ERROR] res = func() > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 208, in > task_annotate > [translation:ERROR] s = annotator.build_types(self.entry_point, > self.inputtypes) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 100, in > build_types > [translation:ERROR] return self.build_graph_types(flowgraph, > inputcells, complete_now=complete_now) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 143, in > build_graph_types > [translation:ERROR] self.complete() > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 188, in > complete > [translation:ERROR] self.processblock(graph, block) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 434, in > processblock > [translation:ERROR] self.flowin(graph, block) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 490, in > flowin > [translation:ERROR] self.consider_op(block.operations[i]) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 661, in > consider_op > [translation:ERROR] argcells = [self.binding(a) for a in op.args] > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 238, in > binding > [translation:ERROR] return self.bookkeeper.immutableconstant(arg) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 297, in > immutableconstant > [translation:ERROR] return self.immutablevalue(const.value) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 393, in > immutablevalue > [translation:ERROR] result = SomePBC([self.getdesc(x)]) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 524, in > getdesc > [translation:ERROR] return self.descs[pyobj] > [translation:ERROR] TypeError: unhashable type > [translation:ERROR] Processing block: > [translation:ERROR] block@-1 is a <class > 'pypy.objspace.flow.flowcontext.SpamBlock'> > [translation:ERROR] in (pypy.module.tcc.tcc:9)tcc_print0 > [translation:ERROR] containing the following operations: > [translation:ERROR] v107 = simple_call((_FuncPtr TCC_Print), msg_0) > [translation:ERROR] --end-- > > ############# end of error message ################################## > > pypy/module/tcc/__init__.py is like following > # Package initialisation > from pypy.interpreter.mixedmodule import MixedModule > > class Module(MixedModule): > appleveldefs = {} > interpleveldefs = { > 'print0': 'tcc.tcc_print0' > } > > and pypy/module/tcc/tcc.py is like following > > from pypy.interpreter.baseobjspace import ObjSpace > from ctypes import * > > libtcc = CDLL('/u/jiwon/proj/less-bus/libLESS.so') > myprint0 = libtcc.TCC_Print > myprint0.argtypes = [c_char_p] > myprint0.restype = c_int > > def tcc_print0(space, msg): > return myprint0(msg) > tcc_print0.unwrap_spec = [ObjSpace, str] > > > What am I missing here? I think the currently supported way to compile modules using rctypes is the pypy/bin/compilemodule.py script, and indeed your code translates well with it. I suggest (but I did not test) adding this line somewhere in targetstandalone.py: import pypy.rpython.rctypes.implementation This should register the annotation for calling ctypes functions. -- Amaury Forgeot d'Arc

2 1

Writing os.pipe() using ctypes
by Stephen Thorne 06 Jul '06

06 Jul '06

Hi, I've been having fun this week trying to get twisted's unit tests running on top of pypy. This has the dual goals of getting twisted to run on a funky platform like pypy, and being able to give pypy a bit of a run around the yard. I have to report this is going "better than expected". I've already used rhymes's fcntl implementation and had to extend the posix module, I intend to do a diff once I get everything sorted. I have a problem with rpython translation of my implementation of posix.pipe(). I have other ctypes implementations of functions (such as posix.access()) which work okay, but I'm having trouble with this particular case. man 2 pipe says that I use it like this: int pipe(int filedes[2]); My implementation looks like: fd_arr = c_int * 2 def pipe(space): fds = fd_arr() fds[0] = 0 fds[1] = 0 if libc.pipe(fds): raise makeOSError(space) return space.wrap((fds[0], fds[1])) pipe.unwrap_spec = [ObjSpace] And the error I get from time {{{python translate.py --text --batch standalone --thread}}} is: [translation:ERROR] Exception': unexpected prebuilt constant: <slot wrapper '__init__' of '_ctypes.Array' objects> [translation:ERROR] .. v0 = simple_call((ArrayType c_long_Array_2)) [translation:ERROR] Processing block: [translation:ERROR] block@3 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'> [translation:ERROR] in (pypy.module.posix.ctypes_posix:43)pipe [translation:ERROR] containing the following operations: [translation:ERROR] v0 = simple_call((ArrayType c_long_Array_2)) [translation:ERROR] v1 = setitem(v0, (0), (0)) [translation:ERROR] v2 = setitem(v0, (1), (0)) [translation:ERROR] v3 = getattr((<CDLL '/usr/lib/libc...20a89f0>), ('pipe')) [translation:ERROR] v4 = simple_call(v3, v0) [translation:ERROR] v5 = is_true(v4) [translation:ERROR] --end-- I've tried a bunch of ways of initialising the 'fds' object as to not make it a 'prebuilt constant', fds = fd_arr(0,0), creating fd_arr dynamically, fd_arr(zero, zero), etc. And I simply cannot seem to get rid of this exception. So, any ideas as to how I go about making this work? Stephen.

2 1

question on using ctypes
by Jiwon Seo 05 Jul '06

05 Jul '06

Hi, I added a module that uses ctypes under pypy/module/ but I'm getting ImportError while running translate.py for targetpypystandalone.py The error message is like, ..... cut ... [translation:ERROR] w_value = loader(space) [translation:ERROR] File "/u/jiwon/proj/pypy-dist/pypy/interpreter/mixedmodule.py", line 135, in ifileloader [translation:ERROR] d[name] = __import__(name, None, None, [name]) [translation:ERROR] ImportError': No module named tcc [translation:ERROR] .. v0 = getattr(self_0, ('w_dict')) [translation:ERROR] Processing block: [translation:ERROR] block@-1 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'> [translation:ERROR] in (pypy.interpreter.mixedmodule:33)MixedModule.MixedModule.getdictvalue [translation:ERROR] containing the following operations: [translation:ERROR] v1 = getattr(space_0, ('finditem')) [translation:ERROR] v0 = getattr(self_0, ('w_dict')) [translation:ERROR] v2 = simple_call(v1, v0, w_name_0) [translation:ERROR] v3 = getattr(self_0, ('lazy')) [translation:ERROR] v4 = is_true(v3) [translation:ERROR] --end-- and the part I added is pypy/module/tcc/__init__.py and pypy/module/tcc/tcc.py ### __init__.py # Package initialisation from pypy.interpreter.mixedmodule import MixedModule class Module(MixedModule): appleveldefs = {} interpleveldefs = { 'print0': 'tcc.tcc_print0' } ### tcc.py from pypy.interpreter.baseobjspace import ObjSpace from ctypes import * libtcc = CDLL('/path/to/lib.so') print0 = libtcc.TCC_Print print0.argtypes = [c_char_p] print0.restype = c_int def tcc_print0(space, msg): print0(msg) tcc_print0.unwrap_spec = [ObjSpace, str] What have I done wrong? Thaniks, -Jiwon

2 4

[dls2006@hpi.uni-potsdam.de: DLS 2006 -- Author Notifications Delayed]
by Armin Rigo 01 Jul '06

01 Jul '06

Hi all, Looks like we won't know for another week: ----- Forwarded message from dls2006(a)hpi.uni-potsdam.de ----- Date: Sat, 01 Jul 2006 21:16:12 +0200 From: dls2006(a)hpi.uni-potsdam.de To: arigo(a)tunes.org Subject: DLS 2006 -- Author Notifications Delayed Reply-To: dls2006(a)hpi.uni-potsdam.de Dear Authors -- Due to unforeseen circumstances author notifications are delayed, expected to be ready mid of next week. My apologies for the inconvenience, Robert ----- End forwarded message ----- Armin

1 0

Re: [pypy-dev] [pypy-svn] r29534 - pypy/dist/pypy/rpython/ootypesystem/test
by Samuele Pedroni 01 Jul '06

01 Jul '06

antocuni(a)codespeak.net wrote: > Author: antocuni > Date: Fri Jun 30 16:08:16 2006 > New Revision: 29534 > > Modified: > pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py > Log: > Added a failing test. > > Basically, if we modify the Record after its hash has already been > computed, we might obtain two objects that compare equal but hash > differently. > > Disabling the caching in LowLevelType.__hash__ fix the test, but I > think it's not what we want. > > Adding the following __hash__ to ootype.Record also fix the test, but > I'm not sure if it's correct, especially with recursive structures: > > def __hash__(self): > return hash(self._fields) > Records are really like lltypesystem Structs, although because some of their code was copied from Instance, they may give the impression that you can add fields after the fact, but that should not be done, it breaks the fact that they are supposed to compare by structure. If you are needing that you need some other approach or introduce some kind of forward definition. _add_fields should really disappear from Record. > > > > Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py > ============================================================================== > --- pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py (original) > +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py Fri Jun 30 16:08:16 2006 > @@ -38,3 +38,14 @@ > t.a = 1 > t.b = 2 > assert ooidentityhash(t) != ooidentityhash(t2) > + > +import py > +def test_hash(): > + #py.test.skip("LowLevelType.__hash__ bug waiting to be fixed") > + T1 = Record({"item0": Signed, "item1": Signed}) > + T2 = Record({"item0": Signed}) > + > + hash(T2) # compute the hash, it will stored in __cached_hash > + T2._add_fields({"item1": Signed}) # modify the object > + assert T1 == T2 > + assert hash(T1) == hash(T2) > _______________________________________________ > pypy-svn mailing list > pypy-svn(a)codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn

3 3

patch for __getattribute__()
by Richard Emslie 01 Jul '06

01 Jul '06

Hi! The following patch initiated by a chat with Runar Petursson about descriptors - increases pypy-c/pypy-llvm by 25% on richards.py. I couldnt be bothered creating a branch - but if someone wants to sanity check it and fix the method repr and check it in, you are more than welcome! :-) Cheers, Richard

2 1