Mailman 3 April 2015 - pypy-dev

Porting PyPy/rpython to Python 3
by VanL 20 Apr '15

20 Apr '15

Hi everyone, For the last little bit I have been working on porting the rpython toolchain to Python 3. My initial goal is to get either pypy2 or pypy3 to build with either pypy2 or pypy3. I had gotten the impression from some previous statements that these efforts would not be welcome, so I was doing my work in a private fork. After a few conversations at PyCon, though, I was encouraged to package some of these changes up and send them as a series of pull requests. A couple questions/thoughts: 1. I am happy to send the pull requests up using bitbucket. Rather than do a big dump, I will send up chunks that each address a particular issue across the entire codebase. Even if a PR touches a number of files, each PR will implement the same change so that correctness is easy to check. If these PRs are not wanted, let me know, and I will stop sending them up. 2. I am initially doing this work in a way that maintains 2/3 compatibility - my check before each major commit is whether I can still build pypy using pypy2. Would the pypy devs be willing to make building pypy be 2.7+ only? That way I could use __future__ imports to ease some of the porting. 3. I will likely vendor or require six before I am done. Let me know if this would likely be a problem. 4. At some point in the future, I plan on reworking the rpython toolchain in various ways - use python 3 function and type annotations so as to make the flow of types be easier to see, fully split out the rpython and non-rpython bits, etc. Again, I am happy to do this on my own, but will gladly contribute upstream if wanted. Thanks, Van

7 23

Re: [pypy-dev] Porting PyPy/rpython to Python 3
by Laura Creighton 20 Apr '15

20 Apr '15

In a message of Mon, 20 Apr 2015 12:28:19 +0200, Armin Rigo writes: >Ok. Then yes, I think there should be little intrinsic reason for it >to be slower (apart from some bytes/unicodes changes, which should not >be too important in this case), and it would be a good excuse to focus >on the performance of PyPy3. > > >A bientôt, > >Armin. Sounds to me as if you are talking youself into it. ;) Laura

2 1

Re: [pypy-dev] Porting PyPy/rpython to Python 3
by Laura Creighton 20 Apr '15

20 Apr '15

In a message of Mon, 20 Apr 2015 12:04:10 +0200, Armin Rigo writes: >Hi Laura, > >On 20 April 2015 at 11:53, Laura Creighton <lac(a)openend.se> wrote: >> I worry that this will be slow. > >Slow at which level? The final speed of some translated PyPy should >not be influenced, but maybe translation itself can become slower. >But then it would be good motivation to do performance improvements in >PyPy3, which would at that point gain one major user --- our own >translations. > > >A bientôt, > >Armin. I was worried about translation speed. Laura

2 1

Re: [pypy-dev] Porting PyPy/rpython to Python 3
by Laura Creighton 20 Apr '15

20 Apr '15

In a message of Mon, 20 Apr 2015 10:15:52 +0200, Armin Rigo writes: >I think I still prefer the "upgrade everything at once and forget >about Python 2" approach. I worry that this will be slow. Laura

2 1

Can someone explain __extend__?
by VanL 17 Apr '15

17 Apr '15

I am having some trouble wrapping my head around it. Reading through rpython/tools/pairtype.py, it looks like it could be one or more of a number of things: - An implementation of javascript-style prototypes. (The similarity: you don't subclass an object in js - you use the base object as a prototype and extend it with new functionality) - A way to do specialization and automatic dispatching on types so that a+b works (both "a" and "b" know what they are, and whether they are compatible with each other in an __add__/__radd__ sense, and what type should be returned as a result of that call) - Sort of a first draft of ABCs, allowing composition and type buildup without explicit inheritance (roughly, __extend__ is similar to ABC.register) - Other? Thanks, Van

2 1

PyPy translation on Xeon Phi (pka MIC)
by Ajit Dingankar 14 Apr '15

14 Apr '15

I tried the translator for BF example on Xeon Phi: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy… It failed due to a "safety check" related to asserts at the end of the translate module. I use CPython v2.7.2 since that's the latest I could find for the Phi accelerator. I thought I'd try it before going to the step of cross-compiling a more recent version. (Just for reference the example works with CPython v2.7.5 on the Xeon host.) I tried to search for previous experience with Phi (or MIC) but could only find this old post on the mailing list: http://permalink.gmane.org/gmane.comp.python.pypy/11981 which is mainly about STM but mentions MIC at the very end: "Still trying to see whether I can get PyPy to run on the MIC. :)" I'd appreciate any pointers to making PyPy translate work on Phi, with CPython or PyPy binary itself (if needed, since it may be hard to get it working on Phi). At least whether Xeon Phi is or is not a supported platform, and what options there are to support it, in the latter case). Thanks, Ajit ==== PS: For full disclosure, I work for Intel but my day job is related to hardware, hence posting from my personal account.

6 11

Vectorizing pypy traces no3
by Richard Plangger 14 Apr '15

14 Apr '15

Hi, I have recently managed to correctly transform a trace to a vectorized trace that includes a guard. I'm hoping that this might be merged into the code base of pypy (when it is finished), thus it would be nice to get feedback and iron out some problems I currently have. Of course this needs explanation (hope that does not lead to tl;dr): Consider the following trace: short version (pseudo syntax): ``` label(...,i,...) store(c,i) = load(a,i) + load(b,i) j = i+1 guard(j<MAX) jump(...,j,...) ``` long version: http://pastebin.com/e24s1vZg By unrolling this short trace, it is _NOT_ possible to vectorize it. The guard prohibits the store operation to be executed after the guard. I solved this problem by introducing a new guard (called 'early-exit'). It saves the live variables at the beginning of the trace. By finding the index calculations + guards and moving them above the early exit the following is possible: short version (pseudo syntax): ``` label(...,i,...) j = i + 1 guard(j<MAX) k = j + 1 guard(k<MAX) guard_early_exit() # will not be emitted va = vec_load(a,i,2) vb = vec_load(b,i,2) vc = vec_add(va,vb) vec_store(c, i, 2) = vc jump(...,k,...) ``` long version http://pastebin.com/vc3HaZCn My assumptions: Any guard that fails before the early exit must guide blackhole to the original loop at instruction 0. Only pure operations and the guards protecting the index are allowed to move before early-exit. The previous and the use of the live variables of the early exit (at the guard instructions) preserve correctness. I'm not quite sure how to handle the following problems: 1) I had the problem that uneven iterations moved to the blackhole interpreter and executed the loop from the beginning. I fixed it by resetting the blackhole interpreter position to the jitcode index 0. Is this the right way to start from the beginning? 2) Is there a better way to tell the blackhole interpreter to resume from the beginning of the trace, or even do not blackhole and just jump into the normal interpreter? 3) Are there any objections to do it this way (guard early-exit)? Best, Richard

2 1

Which pypy with >=3.3 Python compatibility
by Mike Müller 14 Apr '15

14 Apr '15

I need pypy that is Python 3.3 or, even better, Python 3.4 compatible. I can found the nightly builds at http://buildbot.pypy.org//nightly Which one should I use py3.3 or py3k? There are many more version. Should I use one of them? Thanks, Mike

4 7

Re: [pypy-dev] http://speed.pypy.org/timeline/ -- show the code?
by Laura Creighton 13 Apr '15

13 Apr '15

In a message of Sun, 12 Apr 2015 17:39:17 +0200, Armin Rigo writes: >Hi Laura, > >On 12 April 2015 at 14:46, Laura Creighton <lac(a)openend.se> wrote: >> I thought there was a way to show the code that was actually run to get >> the results. Maybe I am confusing things with the pshootout site. Have >> I just forgotten how to do this? > >Yes, there is a way: from a page that shows one graph, like for example >http://speed.pypy.org/timeline/?exe=3%2C6%2C1%2C5&base=2%2B472&ben=ai&env=1… >, you can see a link called "Code" below the graph itself, next to the >short description of what the benchmark does. > > >A bientôt, > >Armin. Fantastic, sorry I missed that. Thank you. Laura

1 0

http://speed.pypy.org/timeline/ -- show the code?
by Laura Creighton 12 Apr '15

12 Apr '15

I thought there was a way to show the code that was actually run to get the results. Maybe I am confusing things with the pshootout site. Have I just forgotten how to do this? I wanted to show an astronomer what sort of code we run blazingly fast vs what sort we are less speedy at, so he can decide if he needs to write his algorithm in C++ or not. If speed.pypy.org doesn't have such a feature, it would seem to be a good one to add. Laura

2 1