Mailman 3 July 2006 - pypy-dev

[emil@dmr.ath.cx: Re: [pypy-svn] r29899 - pypy/extradoc/talk/ep2006]
by Armin Rigo July 10, 2006

July 10, 2006

Hi all, ----- Forwarded message from Emil Mikulic <emil(a)dmr.ath.cx> ----- Date: Mon, 10 Jul 2006 17:23:05 +1000 From: Emil Mikulic <emil(a)dmr.ath.cx> To: arigo(a)codespeak.net Subject: Re: [pypy-svn] r29899 - pypy/extradoc/talk/ep2006 On Sun, Jul 09, 2006 at 03:35:03PM +0200, arigo(a)codespeak.net wrote: > http://snake.cs.uni-duesseldorf.de/tgtest/ I see: Bad Gateway The proxy server received an invalid response from an upstream server. Apache Server at snake.cs.uni-… [View More]

1 0

PyPy Post-EuroPython 2006 Sprint Report
by Michael Hudson July 9, 2006

July 9, 2006

Greetings from an assortment of tired brains at CERN, home to a larger assortment of hopefully less tired physicist brains. We are reaching the end of a succesful and well-attended (24 participants!) sprint. 3.5 days times 24 people is a bit more than two man months of work -- we are going to try to cover all we've achieved, but will undoubtedly forget some stuff! Summer of Code student Lawrence and his mentor Arre spent the majority of the time integrating Lawrence's SoC work -- ctypes … [View More]implementations of various CPython extension modules -- into PyPy. He has implemented the time, mmap and fcntl modules so far, and the work at this sprint concentrated on putting the time module into the form required for PyPy -- making it a mixed module, conforming to the restrictions of rctypes and fighting the extension compiler. Brian Sutherland and Sad Rejeb also joined in this play, attempting to convert some prewritten performance sensitive caching code used in Zope(!) somewhere to a mixed module suitable for feeding to the extension compiler. Unfortunately the resulting code was no faster than the pure Python version... probably because of the ext-compiler using refcounting for GC. Antonio appeared to spend his entire time writing commit messages but somehow must have written some code in between (with the help of *his* SoC mentor, Armin) because now all of PyPy's Standard Interpreter can be rtyped with the ootypesystem. "All" that remains are the dict implementation and external function support in the CLI backend before IronPython has a competitor in the shape of PyPy.Net :-) Guido (not *that* Guido) being the JavaScript deity he is wrote a JS tool to instrospect the DOM methods and produce useful information for Maciej's SoC (we only had three SoC students here...) project. Unfortunately browsers' JavaScript implementations are famously eccentric and running his tool popped up the Print dialog box twice and then crashed Epiphany. It proved marginally less frustrating to write a scraper for Mozilla's documentation and fix the broken HTML he found there. Aurélien, Ludovic and part of Armin tried to figure out why the example of coroutine cloning written for the "What can PyPy do for you?" talk didn't work. Assorted mind boggling bugs were found and fixed, with the aggravating result that it now works fine when run on the llinterpreter but crashes -- after a while -- when translated to C. More wizardry will be required here... Just before the sprint Maciej attempted to improve the annotator's error messages and he and Samuele spent some time fixing this new code (it's very frustrating when your error printing routines crash on you...). A varying team of people including but not limited to Pieter, Carl Friedrich, Maciej and Alexander worked on alternative, hopefully faster implementations of Python data types, particularly strings and dictionaries. We now have efficient representations of slices of strings and for the result of adding two strings, all completely transparent to application-level. For a sufficiently cooked benchmark, we are now up to seven times faster than CPython (and use 30 times less memory). On the dictionary side, we stole an optimization for CPython: assume on creation that a dictionary will only contain strings keys and switch to an alternative implementation on the insertion of a non-string key. This makes pystone and richards both around 20% faster, some benchmarks up to 40% faster (but also some a few percent slower). Alexandre and David started to think about configuration and option handling, but got distracted when they tried to run a translation on a 64-bit machine (or more specifically, a machine with 64-bit userland), because it broke in obscure ways. A few obscure fixes later and it worked again apart from stackless, and another few fixes fixed that too. Michael and Fabrizio worked on the stackless code, "un-inlining" some of the frame-state saving code with the goal of reducing the size of the generated code. This involved taking Fabrizio on a whirlwind tour of one of the scarier areas of the PyPy codebase, but he survived and the feature was implemented, although the size reduction was not as great as had been hoped -- more investigation required (we also need to build a stacklessgc build to check that). Michael and Fabrizio then worked on AST- and bytecode-level optimizations, which mostly involved building an infrastructure to make modifications of the AST easier. They implemented a simple constant folder at the AST level and at the bytecode level an optimization to remove the temporary tuple in code like "x,y=y,x". Simon and Eric resurrected and experimented more with LLVM's JIT interface. Simon then worked with Ludovic and Arre on implementing RPython support for numarrays and Eric worked on making the output of genc compilable with C++. Guido and Holger implemented the beginning and perhaps the middle but probably not the end of doctest support in py.test. After some discussion with Alexandre, David and Holger, Guido and Carl Friedrich began to work on the rather ugly-looking task of taming pypy's growing menagerie of options. They started to implement a general mechanism for defining a hierarchy of options, handling dependencies and conflicts among them. From this you can generate optparse Options for command line interfaces and will one day be able to generate a web form to choose among the zoo of pypy's options. They used the config mechanisms to de-insane the selection of the various object space variants. This, for example, makes automated testing of the above-mentioned variant string and dictionary implementations possible. Pieter worked on random modules^W^Wthe random module, porting it to RPython and implementing some missing functions. So, after a pre-sprint (for some of us), a conference and a post-sprint, we are well prepared for the usual post-sprint week long sleep. Apart from Carl, he has an exam on Friday. Oops! Met vriendelijke groeten, mwh&cfbolz -- This proposal, if accepted, will probably mean a heck of a lot of work for somebody. But since I don't want it accepted, I don't care. -- Laura Creighton, PEP 666 [View Less]

1 0

Question on pypy exception...
by Jiwon Seo July 8, 2006

July 8, 2006

I'm getting an exception with pypy-c, and the exception occurs right after pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr function in pypy_g_PyInterpFrame_dispatch_translated. 1. Is "pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr" same as LL_thread_fused_releaseacquirelock ? I can only see "#define LL_thread_fused_releaseacquirelock pypy_g_ll_fused_releaseacquirelock__ThreadLockPtr", but not the other way around. 2. Is that function there for forcing context switching? 3. What could … [View More]

1 0

Re: [pypy-dev] question on using ctypes
by Amaury Forgeot d'Arc July 6, 2006

July 6, 2006

Hello, 2006/7/5, Jiwon Seo <seojiwon(a)gmail.com>: > After properly installing ctypes, translate.py now gives this error. > > ################################################################## > > [translation:ERROR] Error: > [translation:ERROR] Traceback (most recent call last): > [translation:ERROR] File "./translate.py", line 332, in main > [translation:ERROR] drv.proceed(goals) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/… [View More]translator/driver.py", line 506, in > proceed > [translation:ERROR] return self._execute(goals, task_skip = > self._maybe_skip()) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/translator/tool/taskengine.py", line > 108, in _execute > [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 195, in _do > [translation:ERROR] res = func() > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/translator/driver.py", line 208, in > task_annotate > [translation:ERROR] s = annotator.build_types(self.entry_point, > self.inputtypes) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 100, in > build_types > [translation:ERROR] return self.build_graph_types(flowgraph, > inputcells, complete_now=complete_now) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 143, in > build_graph_types > [translation:ERROR] self.complete() > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 188, in > complete > [translation:ERROR] self.processblock(graph, block) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 434, in > processblock > [translation:ERROR] self.flowin(graph, block) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 490, in > flowin > [translation:ERROR] self.consider_op(block.operations[i]) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 661, in > consider_op > [translation:ERROR] argcells = [self.binding(a) for a in op.args] > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/annrpython.py", line 238, in > binding > [translation:ERROR] return self.bookkeeper.immutableconstant(arg) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 297, in > immutableconstant > [translation:ERROR] return self.immutablevalue(const.value) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 393, in > immutablevalue > [translation:ERROR] result = SomePBC([self.getdesc(x)]) > [translation:ERROR] File > "/u/jiwon/proj/pypy-dist/pypy/annotation/bookkeeper.py", line 524, in > getdesc > [translation:ERROR] return self.descs[pyobj] > [translation:ERROR] TypeError: unhashable type > [translation:ERROR] Processing block: > [translation:ERROR] block@-1 is a <class > 'pypy.objspace.flow.flowcontext.SpamBlock'> > [translation:ERROR] in (pypy.module.tcc.tcc:9)tcc_print0 > [translation:ERROR] containing the following operations: > [translation:ERROR] v107 = simple_call((_FuncPtr TCC_Print), msg_0) > [translation:ERROR] --end-- > > ############# end of error message ################################## > > pypy/module/tcc/__init__.py is like following > # Package initialisation > from pypy.interpreter.mixedmodule import MixedModule > > class Module(MixedModule): > appleveldefs = {} > interpleveldefs = { > 'print0': 'tcc.tcc_print0' > } > > and pypy/module/tcc/tcc.py is like following > > from pypy.interpreter.baseobjspace import ObjSpace > from ctypes import * > > libtcc = CDLL('/u/jiwon/proj/less-bus/libLESS.so') > myprint0 = libtcc.TCC_Print > myprint0.argtypes = [c_char_p] > myprint0.restype = c_int > > def tcc_print0(space, msg): > return myprint0(msg) > tcc_print0.unwrap_spec = [ObjSpace, str] > > > What am I missing here? I think the currently supported way to compile modules using rctypes is the pypy/bin/compilemodule.py script, and indeed your code translates well with it. I suggest (but I did not test) adding this line somewhere in targetstandalone.py: import pypy.rpython.rctypes.implementation This should register the annotation for calling ctypes functions. -- Amaury Forgeot d'Arc [View Less]

2 1

Writing os.pipe() using ctypes
by Stephen Thorne July 6, 2006

July 6, 2006

Hi, I've been having fun this week trying to get twisted's unit tests running on top of pypy. This has the dual goals of getting twisted to run on a funky platform like pypy, and being able to give pypy a bit of a run around the yard. I have to report this is going "better than expected". I've already used rhymes's fcntl implementation and had to extend the posix module, I intend to do a diff once I get everything sorted. I have a problem with rpython translation of my implementation of posix.… [View More]

2 1

question on using ctypes
by Jiwon Seo July 5, 2006

July 5, 2006

Hi, I added a module that uses ctypes under pypy/module/ but I'm getting ImportError while running translate.py for targetpypystandalone.py The error message is like, ..... cut ... [translation:ERROR] w_value = loader(space) [translation:ERROR] File "/u/jiwon/proj/pypy-dist/pypy/interpreter/mixedmodule.py", line 135, in ifileloader [translation:ERROR] d[name] = __import__(name, None, None, [name]) [translation:ERROR] ImportError': No module named tcc [translation:ERROR] .. v0 … [View More]

2 4

[dls2006@hpi.uni-potsdam.de: DLS 2006 -- Author Notifications Delayed]
by Armin Rigo July 1, 2006

July 1, 2006

Hi all, Looks like we won't know for another week: ----- Forwarded message from dls2006(a)hpi.uni-potsdam.de ----- Date: Sat, 01 Jul 2006 21:16:12 +0200 From: dls2006(a)hpi.uni-potsdam.de To: arigo(a)tunes.org Subject: DLS 2006 -- Author Notifications Delayed Reply-To: dls2006(a)hpi.uni-potsdam.de Dear Authors -- Due to unforeseen circumstances author notifications are delayed, expected to be ready mid of next week. My apologies for the inconvenience, Robert ----- End forwarded message ----- Armin

1 0

Re: [pypy-dev] [pypy-svn] r29534 - pypy/dist/pypy/rpython/ootypesystem/test
by Samuele Pedroni July 1, 2006

July 1, 2006

antocuni(a)codespeak.net wrote: > Author: antocuni > Date: Fri Jun 30 16:08:16 2006 > New Revision: 29534 > > Modified: > pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py > Log: > Added a failing test. > > Basically, if we modify the Record after its hash has already been > computed, we might obtain two objects that compare equal but hash > differently. > > Disabling the caching in LowLevelType.__hash__ fix the test, but I > think it's … [View More]

3 3

patch for __getattribute__()
by Richard Emslie July 1, 2006

July 1, 2006

Hi! The following patch initiated by a chat with Runar Petursson about descriptors - increases pypy-c/pypy-llvm by 25% on richards.py. I couldnt be bothered creating a branch - but if someone wants to sanity check it and fix the method repr and check it in, you are more than welcome! :-) Cheers, Richard

2 1