Mailman 3 February 2015 - pypy-dev

Pyrlang Post
by 黄若尘 17 Feb '15

17 Feb '15

1 0

Re: [pypy-dev] report of some experience in Pyrlang and RPython
by Matti Picus 16 Feb '15

16 Feb '15

Here's a quick edit job, again as both PDF and docx (with revision marks). Graphs are still missing axis labels and some kind of captioning Matti On Thu, Feb 12, 2015 at 7:23 AM, 黄若尘<hrc706(a)gmail.com> wrote: > I like the blog post. It's full of technical details and might need > some editing, but otherwise fine. I'm also happy to just send it as it > is (but please add numbers and explanations on axis of graphs > > On Thu, Feb 12, 2015 at 7:23 AM, 黄若尘<hrc706(a)gmail.com> wrote: > Hi, Fijal, > I have modified some contents of my post and added the section about the comparison among Pyrlang, Erlang and HiPE. > I think the last part of the comparison is interesting, which showed a slowing down when the scale of benchmark become quite huge. > The docx format seems has something wrong so I also attached the pdf one. > > Cheers, > Huang

3 2

could it be that cpyext is faster than python + cffi?
by Matti Picus 15 Feb '15

15 Feb '15

I am close to releasing a blog post about numpy.linalg https://gist.github.com/mattip/25680e68fe7e2856fe0c Note the benchmark section, this is not a typo. Here's how I got to the conclusion that cpyext is faster than python+cffi. I installed numpy from the pypy/numpy repo on a nightly after 2.5.0 pip install git+https://bitbucket.org/pypy/numpy.git I then benchmarked the two pypy methods of calling numpy.linalg.inv() via this script which simply calls linalg.inv() for different sized ndarray. https://gist.github.com/mattip/a7b22dc237fb09a46758 $pypy inverse.py 10 UserWarning: npy_clear_floatstatus, npy_set_floatstatus_invalid not found warn('npy_clear_floatstatus, npy_set_floatstatus_invalid not found') after 999999, for n= 10, time=20.64, 20.64 msec/1000 loops and $USE_CPYEXT=yes pypy inverse.py 10 after 999999, for n= 10, time= 8.52, 8.52 msec/1000 loops Any thoughts? Comments on the blog post? Matti

3 5

mix-and-match approach to implementation decisions
by anatoly techtonik 14 Feb '15

14 Feb '15

Hi, Reading http://rpython.readthedocs.org/en/latest/ to see if I can understand the text without a formal CS education. What does "mix-and-match approach to implementation decisions" mean? -- anatoly t.

2 3

report of some experience in Pyrlang and RPython
by 黄若尘 12 Feb '15

12 Feb '15

Hi, everyone, I tried to wrote down the experience I have done recently about my project of Pyrlang, but I’m not sure if there are enough explanation for details of JIT, and if there are some inappropriate expressions. So I think it may be better to show it to you then post it to the PyPy Blog. Please feel free to add or modify it or delete verbose expressions (if any). Thank you in advance. Ruochen Huang

3 5

numpy installation
by Maciej Fijalkowski 12 Feb '15

12 Feb '15

Hey There is STILL no clear installation path for PyPy + numpy, with no instructions anywhere, can you please do something about it? Ideally pip install numpy-pypy should work and we should be done (this + info on pypy.org will do) I asked about this months ago, if noone picks it up, I'll deal with it, but this is by far the biggest problem for PyPy+numpy users (they can't install it). Cheers, fijal

7 8

Leysin Sprint location
by Armin Rigo 09 Feb '15

09 Feb '15

Hi all, If you're coming back to Leysin for the sprint at the end of February: warning, the main entrance to the two chalets Ermina was moved from the upper chalet to the lower chalet. In case you need it, here's a map showing how to go to the lower chalet (red marker) when you're standing in front of the upper chalet: https://www.google.ch/maps/dir/46.3488973,7.0201613/46.3486126,7.0210687/@4… A bientôt, Armin.

1 0

Playing with PyPy and Django
by Tin Tvrtković 09 Feb '15

09 Feb '15

Hello, PyPy folks! While trying to speed up one of my Django sites, I noticed a new version of PyPy had just been released. So I grabbed a fresh download of PyPy 3 (since this is a Python 3 codebase) and tried taking it out for a spin. However, as far as I can see, whatever I try PyPy is consistently slower than CPython for this. Since this is a proprietary site, I've basically ripped out all the code except my settings.py and my requirements; and am benchmarking the Django admin index. The results are about the same. I've set up a small repo that can be used to reproduce the environment: https://github.com/Tinche/PyPy-Django-Playground. There's additional info in the README there. These tests have been carried out on Ubuntu Trusty, 64-bit. CPython 3 is the system Python, 3.4. PyPy has been downloaded from the official site and unzipped. So what I basically do is set up an admin session, and use the Django main admin page. 200 warmup requests, then 100 benchmarked requests, look at the mean request time. Some results: Django's runserver, DEBUG mode: PyPy3 485.389 [ms] CPython 3.4 105.777 [ms] Django's runserver, no debug: PyPy3 44.661 [ms] CPython 3.4 18.697 [ms] Gunicorn, 1 worker, no debug: PyPy3 28.615 [ms] CPython 3.4 13.532 [ms] I don't exactly claim to be an expert on benchmarking, but assuming my site is similar to the Django admin, CPython's gonna be giving me better performance. Also the debug runserver performance is kinda worrying. Nobody's going to be running this in production, but it makes development a little slower and more annoying than it should be. Is there anything to make PyPy more competitive in these kinds of scenarios? Kind regards, Tin

5 16

Another idea for improving warm-up times
by Armin Rigo 07 Feb '15

07 Feb '15

Hi all, Here's an idea that came up today on irc (thanks the_drow_) while discussing again saving the generated assembler into files and reloading it on the next run. As you know we categorize this idea as "can never work", but now I think that there is a variant that could work, unless I'm missing something. What *cannot* work is storing in the files any information about Python-level data, like "this piece of assembler assumes that there is a module X with a class Y with a method Z". I'm not going to repeat again the multiple problems with that (see http://pypy.readthedocs.org/en/latest/faq.html#couldn-t-the-jit-dump-and-re…). However, it might be possible to save enough lower-level information to avoid the problem completely. The idea would be that when we're about the enter tracing mode and there is a saved file, we use a "fast-path": * we find a *likely* candidate from the saved file, based on some explicitly-provided way to hash the Python code objects, for example (it doesn't have to be a guaranteed match) * this likely saved trace comes with a recording of all the *green* guards that we originally did (both promotions and things that just depend on purely green inputs). (This means extra work for making sure we save this information in the first place.) * we run it like in the blackhole interpreter, but checking the result of the green guards against the recorded ones * we also take all green values that we get this way and pass them as constants to the next step (see below, "**"). This means that we generalize (and lower-level-ize) the vague idea "a module X with a class Y with a method Z" to be instead a series of guards that need to give the same results as before. They would automatically check some subset of the new interpreter's state by comparing it against the old's --- but only as much as the actual loop happens to need. For example, if we had in the (old) normal trace a guard_value(p12, <constant pointer>), then of course it makes no sense to record the old interpreter's constant pointer, which will change. But it makes sense to record *what* was really deduced from this constant pointer, i.e. all the green getfields and getarrayitems we did. And for example, if it was a PyCode object, we would record the green switch that we did on the integer value that we got in the old interpreter (which is the next opcode), even though that's all green. That's the real condition: that we would follow the same path by constant-folding the decisions on the green variables. So, to finish the new interpreter's reloading: if the checking done above passes, the next step is a fast-path through the assembler. We "just" need to reload the saved assembler as a sequence of bytes, and fix all constants there. To continue the example above, if a piece of assembler was generated from the instruction guard_value(p12, <constant pointer>), then the saved file must contain enough information so that we know we must replace this old constant pointer's value in the assembler with the new constant pointer's value recorded above (at "**"). Overall, this would result in a much faster warm-up: faster tracing, and no optimization nor regular assembler generation at all --- only a very quick assembler reloading and fixing step. There are of course complications from the fact that we don't simply record loops, but bridges. They might be seen in a different order in the new process, so that when we are in the checking mode, we might run the start of the loop, but then jump into the bridge --- even though the loop was not fully seen so far. This is not impossible to implement by reloading the complete loop+bridge, but making the tail of the loop invalid until we really run into it (with an extra temporary guard). And I'm sure that unrolling will somehow come with its lot of funniness, as usual. However, does it sound reasonable at all, or am I missing something else? A bientôt, Armin.

2 2

Fwd: [pypy-commit] pypy default: Rename the --output option to --really-force-output, and document it
by Armin Rigo 06 Feb '15

06 Feb '15

Hi Tobias, (answering an off-list-by-mistake mail) On 6 February 2015 at 13:50, Tobias Pape <Das.Linux(a)gmx.de> wrote: > So this is changed in rpython for a pypy issue? Yes, someone else made the same point on IRC. I have now reverted this change, and added a check to "targetpypystandalone.py" that you don't try to translate pypy with both "--shared" and "--output", which should be enough to avoid the original issue. A bientôt, Armin.

1 0