Mailman 3 March 2008 - pypy-dev

Summer of Code
by Carl Friedrich Bolz 18 Mar '08

18 Mar '08

Hi all, there was some discussion about possible Summer of Code topics on the #pypy channel today. We had several project ideas: - work on the JIT assembler backends. There are several possible projects here, like writing a much simpler i386 backend and adding float support to that; or writing a x64 backend; or writing a new i386 backend that uses the more modern bits of the processor for arithmetic (like SSE and friends). - support ctypes on more backends. right now ctypes is supported only when compiling PyPy to C. A nice project would be to support it when compiling to .NET. That's not too hard, the only thing needed is to port a small module that does the actual invocation of external libraries (a related project would be to port this module to Jython or IronPython to get support for ctypes there). - support some of the more recent features of CPython. A nice project would be to support all of the remaining CPython 2.5 features and port over the changes to the standard library (which includes re-writing some of the C-coded modules in RPython or pure Python). Another possibility would be to even start looking into the 2.6 features. - Anto proposed to make the javascript backendopt more useful by turning it into something more frameworkish. I am sceptical of the viability of the task, it would need a very clever student and probably a lot of work too. What do you all think of these ideas? Are there other ideas around? I know that Jakub posted something a while ago (haven't looked in detail yet, though). In my opinion we should try to avoid "yet another incomplete interpreter implentation" for this year. The timeplan is to collect some more ideas, then write a better description of them and make a blog post about it (say around Thursday). Sounds like a plan? Cheers, Carl Friedrich

7 8

Summer of Code
by Paul deGrandis 18 Mar '08

18 Mar '08

I agree that new approaches to concurrency would be cool. Possible SoCers might start by looking at PARELY (an actor model impl for python that runs on PyPy as is). http://osl.cs.uiuc.edu/parley/ Regards, Paul On 3/18/08, Jakub Gustak <jgustak(a)gmail.com> wrote: > > What do you all think of these ideas? Are there other ideas around? I > > know that Jakub posted something a while ago (haven't looked in detail > > yet, though). In my opinion we should try to avoid "yet another > > incomplete interpreter implentation" for this year. > > > I fully agree. > > Still, what I proposed was not attempt to create another one, but keep > existing one alive. > It is bothering me, that scheme interpreter is incomplete, but the > problem is, i could not > find a use case for it (except of Thousand Parsecs internal scripting > language - if none > other - faster - scheme implementation available). > > So if PyPy is such a nice platform to experiment, let's do it, and > play with different > approaches to implement parallel paradigms, or whatsoever. There is no > need for yet another > scheme interpreter (even complete) if it doesn't have any interesting features. > That's my 3 cents for now. > > Cheers, > > Jakub Gustak > > _______________________________________________ > pypy-dev(a)codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev >

1 0

summer of code 2008
by Jakub Gustak 14 Mar '08

14 Mar '08

Hello. I am wondering if pypy is going to take part in Google Summer of Code this year. I have an idea, which I would like to hack on (and already talked a little about it with Armin). But I sill am not sure if it makes much sense. I will paste what I send to Armin some time ago: msg << EOM I have this weird idea of creating (yet another) scheme dialect on top of pypy scheme interpreter. Main features of this dialect would be: * strict (in definition) distinction between pure functions, and functions with side-effect. * parallelism support: * e.g. (parallel (f1 ...) (f2 ...)) would compute functions f1 and f2 on different processors (if available) and return list of results. of course, f1 and f2 have to be pure functions in this case. * interpreter would start as many processes as processors available and parallel should schedule functions computation on this "nodes". * communication between nodes should be as much transparent as possible, so we can create nodes on different machines. * there is also another interesting pattern: map-reduce, or scatter-gather, which would be nice to have, if parallel back-end would be capable of this. We could go further, and even avoid using (parallel) construction, and parallelize every pure function. Of course that will make sense, if computation cost is greater than cost of spawning. I am interested, in this, because lot of people are talking, that functional programming is <the way> to control parallelism/concurrency, but i haven't seen any functional language where programming "parallel applications" is like eating cream and butter. Similar idea is implemented in haskell, but haskell is lazy, and it works somehow different: http://www.haskell.org/ghc/docs/latest/html/users_guide/lang-parallel.html http://www.haskell.org/ghc/docs/latest/html/libraries/parallel/Control-Para… EOM Cheers, Jakub Gustak

2 1

module SomeObject setattr.
by Toon Verwaest 10 Mar '08

10 Mar '08

Hi, I'm not yet fully comfortable with the PyPy code so I just lay it out here: I have written the following code in the Squeak VM (should change in the future but its just to show the "problem"): <file objtable.py:> objects = [] <file shadow.py:> objtable.objects += [objofspecifictype] This seems not to be translatable since this is translated to: vX = setattr(objtable-module, 'objects', vY). generally setattr are done on SomeInstance which nicely returns None -> SomeImpossibleValue -> the annotator is happy. However, modules appear to be instances of SomeObject on which setattr returns a SomeObject -> type of vX gets broken. (and the bug received is not very obvious since it says that the original type was (none)...) cheers, Toon ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

2 1

SPy (almost) works in the JVM
by Niko Matsakis 10 Mar '08

10 Mar '08

Just wanted to mention that I fixed several bugs holding SPy [Smalltalk in PyPy] back from working in the JVM. The "mini.image" code now runs, but I'm not sure it does precisely the right thing, as the number of hash marks ("#") that are output differs in the JVM and CLI. The biggest error was the fact that we were converting byte arrays into Java Strings using the default encoding, which I think is UTF-8 (or something). The end result was that we did not read binary files correctly (hence the assertion errors in image decoding). Here is the output of the JVM and the CLI on my machine: ;./targetimageloadingsmalltalk-jvm ../../lang/smalltalk/mini.image ################<pypy.lang.smalltalk.model.W_CompiledMethod object> Going to execute 78 toplevel bytecodes #####################################################################################################################################994035 bytecodes/sec; 70776 sends/sec ;./targetimageloadingsmalltalk-cli ../../lang/smalltalk/mini.image ################<pypy.lang.smalltalk.model.W_CompiledMethod object> Going to execute 78 toplevel bytecodes #################################################################################################################################################################################################2418379 bytecodes/sec; 106874 sends/sec Anyway, getting closer... Niko

2 1

Fwd: module SomeObject setattr.
by Michael Hudson 09 Mar '08

09 Mar '08

This was supposed to go to the list too... ---------- Forwarded message ---------- From: Michael Hudson <micahel(a)gmail.com> Date: 10 Mar 2008 13:02 Subject: Re: [pypy-dev] module SomeObject setattr. To: Toon Verwaest <verwaest(a)iam.unibe.ch> On 10/03/2008, Toon Verwaest <verwaest(a)iam.unibe.ch> wrote: > Hi, > > I'm not yet fully comfortable with the PyPy code so I just lay it out here: > > I have written the following code in the Squeak VM (should change in > the future but its just to show the "problem"): > > <file objtable.py:> > objects = [] > > <file shadow.py:> > objtable.objects += [objofspecifictype] > > This seems not to be translatable since this is translated to: > vX = setattr(objtable-module, 'objects', vY). > > generally setattr are done on SomeInstance which nicely returns None -> > SomeImpossibleValue -> the annotator is happy. > However, modules appear to be instances of SomeObject This is, already, your problem: modules are not supported by RPython (anything that ends up as SomeObject() means translation is not going to work). It's a bit peculiar I guess that the module gets examined here, though I can see why it happens. I think if you write objtable.objects.extend(...) you'll be happier. > on which setattr > returns a SomeObject -> type of vX gets broken. If anything, this should break even more noisily. > (and the bug received is not very obvious since it says that the > original type was (none)...) I can believe this part :/ Cheers, mwh

1 0

Possibly of use in the quest to parse pypy better
by Laura Creighton 09 Mar '08

09 Mar '08

------- Forwarded Message From: Andrew Dalke <dalke(a)dalkescientific.com> Date: Sun, 9 Mar 2008 03:29:29 +0100 To: testing-in-python(a)lists.idyll.org X-Mailer: Apple Mail (2.753) Subject: Re: [TIP] branch coverage In late January I mentioned I was working on a parser for Python using PLY as the parser engine. I wanted to experiment with branch coverage in Python by reading the source .py file, building a parse tree, adding instrumentation to the AST, and saving .pyc files with the instrumentation. Then run the regression tests and see which branches weren't covered. I'm not there yet. What I have done is write a Python parser using PLY. It's called "python4ply" and is available at: http://dalkescientific.com/Python/python4ply.html What took even longer than writing the code was writing the tutorial. It shows examples of how to change the lexer, parser, AST, and bytecode generation. I worked through two examples where I instrumented the code to improve testing. The first of these assumes there are problems like this assert 0 not in results, "problem in: %r" % data where the "data" variable should be "results". Assert statements are rarely fully tested and so prone to failure if they ever occur. My example transforms the code to something like _$assert = Y assert X, _$assert where '$' is something that normal Python code can't generate but AST manipulation code can. Details at: http://dalkescientific.com/Python/python4ply- tutorial.html#assert The second shows a way to add statement coverage without using settrace. Details at: http://dalkescientific.com/Python/python4ply-tutorial.html#coverage I didn't get to the point where I could do branch coverage. I didn't have a good enough use cases so wasn't sure what I should implement. Perhaps you've got an idea of how to do it, or something else involving parsing Python code? (If so, take a look at the _ast module from the standard library. It's probably a better solution.) If so, I hope this grammar and code helps. Download: http://dalkescientific.com/Python/python4ply.html Tutorial: http://dalkescientific.com/Python/python4ply-tutorial.html Andrew dalke(a)dalkescientific.com ------- End of Forwarded Message

1 0

buffer protocol.
by Maciek Fijalkowski 05 Mar '08

05 Mar '08

I feel a bit clueless about buffer protocol. Current implemntation in pypy is purely app-level, which means it's slow, but more importantly, one cannot get raw, C-level address of it. I've tried approaching the problem, but it seems that there are several serious issues. * what's the future of buffer protocol? * do people use that? ctypes tests are relying on it. anybody knows ctypes module implementation that relies on it? * testing suite sucks. we've got two tests, cpython is not any better in this regard. where to proceed? :.

2 1

Re: [pypy-dev] Tim Bray Blog
by Laura Creighton 03 Mar '08

03 Mar '08

In a message of Mon, 03 Mar 2008 16:48:16 +0100, Stephan Diehl writes: >Hi all, > >just found this (at reddit, what a time waster :-) ): >http://www.tbray.org/ongoing/When/200x/2008/03/03/Python-at-Sun >At the end, he's saying: >"Quick Python trivia question: Near as I can tell, Guido works half-time >on >Python over at Google. Is there anyone in the world, aside from Frank and > >Ted, getting paid to work full-time on Python?" >Within pypy, there are several people to work full time on python, aren't > >there? > >Stephan There was as one time, and we hope there will be again. But our EU funding ran out last year. Laura

1 0

Tim Bray Blog
by Stephan Diehl 03 Mar '08

03 Mar '08

Hi all, just found this (at reddit, what a time waster :-) ): http://www.tbray.org/ongoing/When/200x/2008/03/03/Python-at-Sun At the end, he's saying: "Quick Python trivia question: Near as I can tell, Guido works half-time on Python over at Google. Is there anyone in the world, aside from Frank and Ted, getting paid to work full-time on Python?" Within pypy, there are several people to work full time on python, aren't there? Stephan

2 1