Mailman 3 March 2010 - pypy-dev

Python Ireland Conference
by Michael Twomey 08 Apr '10

08 Apr '10

Hi all, I'm one of the folks involved in organising the first python Ireland conference, to be held in Dublin on Saturday 17th of July (with sprints on Sunday). We'd love it if someone from PyPy could make it over to give a talk. The audience is your typical spread of python developers, many of whom haven't been exposed to PyPy and what it can do. An introduction to PyPy and the benefits it offers would go down very well. cheers, Michael

2 2

Re: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
by Jason Creighton 31 Mar '10

31 Mar '10

michaelschneider wrote: > Jason, > > My name is Mike Schneider (bigdog on the irc). > > I think what you have is very good. Take these as suggestions. I found them extremely helpful, thank you very much. I've tried to implement them in my latest draft, and I've also taken fijal's suggestion of storing it in svn. Here it is, if anyone has any further feedback/suggestions: http://codespeak.net/svn/user/jcreigh/documents/pypy_amd64_gsoc_proposal.txt Cheers, Jason

1 0

Rough draft of x86-64 JIT backend GSoC proposal
by Jason Creighton 31 Mar '10

31 Mar '10

Hi Guys, I am a student planning on applying to GSoC to create an x86-64 backend for the JIT. I have a (very) rough draft of my proposal, and I was hoping to get some feedback on it. Specifically, I'm wondering about operating system support. I've written the proposal as if I would support Linux/Mac OS X/Windows. I would be developing on Linux, so I think we can assume that would be fairly well supported, but obviously I would like it to work on Mac OS X and Windows as well. (I'm not sure if I would have the time/motivation to care about obscure BSDs). OTOH, if there are already a lot of outstanding issues on one of those platforms, I don't know that I would be able to get it working. So what do you think would be a reasonable goal here? Secondly, my timeline is pretty vague. The PSF proposal template recommends a week-by-week timeline, but honestly, I'm not sure how the time usage would break down. Any comments on that would be greatly appreciated. Here's the draft: === Proposal === The PyPy JIT, which has shown substantial performance improvements over CPython, often several times faster, does not currently support the x86-64 instruction set, making it impractical to use on 64-bit x86 systems. My proposal is to extend the existing x86 JIT backend to support x86-64 as well. === Deliverables === Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged into PyPy trunk. === Implementation plan === This is not a research proposal. The goal is simply to have a PyPy JIT that works out of the box on 64-bit CPUs, implemented as conservatively as possible. As such, I will attempt to reuse as much of the existing x86 backend that I can. In fact, the architectural similarities between x86 and x86-64 are large enough that I hope to implement a unified x86/x86-64 backend with the majority of the code working for either platform. There is an existing branch that, while very incomplete, has the beginnings of a unified x86/x86-64 instruction encoding module. I intend to use that branch as a starting point. Rough outline: 1. Take the existing "remove-ri386-multimethod-2" branch and use it as a basis for instruction encoding. 2. Port the existing 32-bit backend to use the new instruction encoding scheme. 3. Add 64-bit support to the backend, A) Modify register allocator to use new general purpose and floating point registers. B) Port "ResOperation" operations to 64-bit C) Port guard failure handling to 64-bit 4. Test 64-bit on Mac OS X and Windows and fix inevitable issues. === About Me === I am a first-year Computer Science student at Flathead Valley Community College planning to transfer to Montana State University. I have several years of professional development experience. I am comfortable programming in Python, C and x86 assembly. Starting May 17th, I will be able to commit 40 hours/week to the project until the end of August. I may travel for a few weeks at some point in the summer, but I will have a laptop with me with the expectation of continuing full-time work. === Contact information === Email: jcreigh(a)gmail.com IRC: "jcreigh" on Freenode Phone: (will be given on request, but not preferred)

3 4

GSoC Project/Bachelor's Thesis
by kai.aras 29 Mar '10

29 Mar '10

Hi Everyone, My name is Kai Aras, I'm a Computer-Science student from Germany and I'm very interested in doing my Bachelor's Thesis on PyPy and possibly also combine that with a GSoC project. I became aware of the PyPy-Project by the Chaosradio Express (CRE088) episode about PyPy from 2008. Last year, I had the opportunity to take some time to play around with PyPy in order to present it at my University, ever since that I became more and more attracted to the Python Language and the PyPy-Project itself, so writing my Bachelor's Thesis about something PyPy-related occured to me quite early. Now, as the time has come, combining my Thesis with a GSoC Project would be a great opportunity. During the last year, I've worked on a smarthome project using CPython on embedded-linux platforms, as the project grew bigger, more and more problems arised and the need for some spezialized Python Runtime grew bigger as well. I know that targeting Embedded Platforms was an Issue in the original EU-Project and there was that Maemo thing last year, so I'm guessing this could be a general topic for a GSoC Project as well as a Bachelor's Thesis, what do you guys think ? Is this something you would like to see as well ? Personally, I'd love to use PyPy on openWRT, could this might be a target of general interest ? This is just something I thought about from time to time but never really had the time to experiment with, so I'm open to all suggestions, please let me know if you have any ideas that could be suitable for such a project. Thanks, Kai

3 3

GSoC projects
by Ehsan Amiri 27 Mar '10

27 Mar '10

Hello all I am a graduate student interested in participation in GSoC 2010. I found some PyPy projects closely related to my background. In particular writing a backend for Intel 64, for JIT compiler, sounds interesting. Also I would like to know more about projects on stackless features. I would be happy to hear any comment on these projects and pointers to documents/paper/code that might be helpful. Last year I participated in GSoC as well. I developed a prototype of an architecture independent SIMD library for Python. This library provides an interface for programmers to write SIMD code in Python and then translates it to low level architecture specific SIMD code using CorePy. The prototype that I developed last year supports intel 64 as I did not have enough time to develop code generators for any other architecture. Best Ehsan

3 3

Sandboxing pypy
by Søren Laursen 26 Mar '10

26 Mar '10

Hi, I have been following the pypy project for over a year. And I have been playing around with it for some time. The project I am working with Minimum Intrusion Grid : MiG ( http://sites.google.com/site/minimumintrusiongrid/) are looking into using pypy. I would like to use it for sandboxing user code in MiG, more specific allow the users to develop their own “scheduler” . The MiG, it self is written in Python so we might even be able to run it on pypy. But right now it is the sandboxing that I am working with. For me it is a bit unclear in the documentation/website, but for that I read into “An attacker that tries to escape the sandbox is stuck within a C program that contains no external function calls at all except for writing to stdout and reading from stdin.” means that I have to write functions that emulates file read/write operations. I have tried different things using the pypy_interact.py (just love the –timeout parameter J), looked at code but have not been able to read files or write files using the pypy_interact.py. What have I missed? Regards, Søren Laursen

5 6

Wikipedia pypy article
by Maciej Fijalkowski 26 Mar '10

26 Mar '10

http://en.wikipedia.org/wiki/PyPy It's somewhat out of date, but also extremely confusing with notions of meta-circular beast. Does someone feel like cleaning it up a bit? Especially keeping it up to date would be cool.

1 0

various benchmark directories
by Maciej Fijalkowski 26 Mar '10

26 Mar '10

We have a couple of strange benchmark directories scattered a bit all over the place in PyPy: pypy/rpython/microbench pypy/translator/goal/various pypy/interpreter/callbench pypy/translator/benchmark build/benchmem benchmarks comes to mind Do we need all of them? Can we kill some and factor the rest to be in one place?

1 0

Quick "state of performance" analysis
by Miquel Torres 25 Mar '10

25 Mar '10

Hi all! I have implemented a small new feature for the speed center. In the Overview, you are now able to select to which results you wish to compare the performance of the selected interpreter. That allows a couple of interesting investigations to be made (bear with me if I say obvious things): 1 - http://speed.pypy.org/overview/?interpreter=3) We already knew that there are 5 benchmarks where pypy-c without the jit is more than twice times slower than cpython: slowspitfire, twisted_iteration, twisted_tcp, html5lib and worst of all spambayes. 2 - http://speed.pypy.org/overview/ The jit makes up for some of them, only twisted_tcp, slowspitfire and spambayes remain as cases where cpython is much faster. 3 - http://speed.pypy.org/overview/?baseline=4 We can now compare pypy-c-jit with cpython with psyco acceleration. The first thing that stands out, is that pypy is much slower... exactly in the same cases where pypy-c without the jit is slower than cpython (no surprise). The second one, is that apart from those 5 cases where the jit is held back by bad pypy-c baseline performance, pypy-c-jit is already much better than psyco, about twice as fast! 4 - http://speed.pypy.org/overview/?baseline=3 Another new comparison is pypy-c-jit to pypy-c (revision 72012). Here we can confirm that the jit accelerates *everything* that we currenly measure, and most by a very good factor. The 3 slow benchmarks from comparison 2 are here last, the are accelerated the least by the jit. So to me the conclusion is that the jit has good enough performance for a first version, and that only poor pypy-c performance in some cases is holding the pypy project from becoming a better option than cpython. Wouldn't it be a good aim to make it a priority of the next (1.3?) release to improve on that? ;-) Apart from it making pypy more "ready", there is a problem if that is not addressed while further jit development continues. An application is only so fast as its slowest part, so even if the jit accelerates some cases 100-fold, it won't help many applications. I plan to add a new view with graphs that will show exactly what the problem is. Cheers! Miquel PD: you can also for example compare pypy-c-jit to pypy-c-jit 72012 and see what improvements there have been since then

2 1

Branches
by Armin Rigo 25 Mar '10

25 Mar '10

Hi all, Can we decide if the following branches have been merged, are useful enough to be merged, need more work, "maybe one day", are very outdated, or turned out just to be a plain bad idea? The numbers are the rev number of the last change. Also, what about separating in-development branches that we expect to merge or kill soon, from the branches that are for "some other idea", e.g. "effect-analysis", which may not be merged in a long while but still we may want to keep around? * 71164 abort-no-asm (fijal, cfbolz) * 70752 bridges-experimental (cfbolz) * 55732 build-external (amaury) * 63757 classdeco (benjamin) * 68723 effect-analysis (verte) * 55751 eval-loop-experiments (antocuni) * 68703 gc-arena (arigo) * 70970 gc-huge-list (fijal, arigo) * 70689 jit-profiling (pedronis, fijal) * 55472 judy-trees (fijal) * 71270 kill-can-inline (cfbolz) * 49688 newdotviewer (misto) * 64690 pypycpp (igorto) * 70786 separate-compilation (amaury) * 70024 sepcomp (xoraxax) * 66009 type-celldict (cfbolz) * 64645 unicode_filename (amaury) * 70890 unroll-safe-if-const-arg (fijal, cfbolz, arigo) Interesting to see the wide range of authors :-) A bientot, Armin.

5 4