Mailman 3 June 2014 - pypy-dev

Cross-post from python-ideas: Compressing the stack on the fly
by Ram Rachum July 28, 2014

July 28, 2014

Hi guys, I made a post on the Python-ideas mailing list that I was told might be relevant to Pypy. I've reproduced the original email below. Here is the thread on Python-ideas with all the discussion.<https://groups.google.com/forum/?fromgroups#!topic/python-ideas/hteGSNTyC_4> -------------- Hi everybody, Here's an idea I had a while ago. Now, I'm an ignoramus when it comes to how programming languages are implemented, so this idea will most likely be either (a) completely impossible or (b) trivial knowledge. I was thinking about the implementation of the factorial in Python. I was comparing in my mind 2 different solutions: The recursive one, and the one that uses a loop. Here are example implementations for them: def factorial_recursive(n): if n == 1: return 1 return n * factorial_recursive(n - 1) def factorial_loop(n): result = 1 for i in range(1, n + 1): result *= i return result I know that the recursive one is problematic, because it's putting a lot of items on the stack. In fact it's using the stack as if it was a loop variable. The stack wasn't meant to be used like that. Then the question came to me, why? Maybe the stack could be built to handle this kind of (ab)use? I read about tail-call optimization on Wikipedia. If I understand correctly, the gist of it is that the interpreter tries to recognize, on a frame-by-frame basis, which frames could be completely eliminated, and then it eliminates those. Then I read Guido's blog post explaining why he doesn't want it in Python. In that post he outlined 4 different reasons why TCO shouldn't be implemented in Python. But then I thought, maybe you could do something smarter than eliminating individual stack frames. Maybe we could create something that is to the current implementation of the stack what `xrange` is to the old-style `range`. A smart object that allows access to any of a long list of items in it, without actually having to store those items. This would solve the first argument that Guido raises in his post, which I found to be the most substantial one. What I'm saying is: Imagine the stack of the interpreter when it runs the factorial example above for n=1000. It has around 1000 items in it and it's just about to explode. But then, if you'd look at the contents of that stack, you'd see it's embarrassingly regular, a compression algorithm's wet dream. It's just the same code location over and over again, with a different value for `n`. So what I'm suggesting is an algorithm to compress that stack on the fly. An algorithm that would detect regularities in the stack and instead of saving each individual frame, save just the pattern. Then, there wouldn't be any problem with showing informative stack trace: Despite not storing every individual frame, each individual frame could still be accessed, similarly to how `xrange` allow access to each individual member without having to store each of them. Then, the stack could store a lot more items, and tasks that currently require recursion (like pickling using the standard library) will be able to handle much deeper recursions. What do you think? Ram.

2 1

Question about PyPy Sandboxing
by Kit Ham July 21, 2014

July 21, 2014

Hi devs, Can multiple PyPy sandboxes be executed and interacted with from C/C++? For context, I'm adding plugin support to my application. I'd like people to be able to write a python module, and call it's functions from the C/C++ in a sandboxed environment. This is for 2 reasons: 1) So that a plugin can't mess with the user's system 2) So that a plugin's access to other plugins can be controlled Cheers, Kit P.S. I'm currently using PyOtherSide in my Qt app, but I'm realising it may not be a good solution for plugins.

2 2

Support for current versions of Pypy on Heroku and Cloud Foundry platforms?
by cclauss July 18, 2014

July 18, 2014

Hi Folks, Currently Heroku only has Pypy 1.9 on an unsupported, experimental basis. https://devcenter.heroku.com/articles/python-runtimes#supported-python-runt… and https://github.com/heroku/heroku-buildpack-python/issues/139 Other Cloud Foundry-based IaaS offering (IBM BlueMix, etc.) seem to use similar buildpacks. https://github.com/cloudfoundry-community/cf-docs-contrib/wiki/Buildpacks Do any of you know of an Infrastructure as a Service provider that supports current versions of Pypy? Thanks for any pointers that you can provide. Chris Clauss

5 11

europython sprints?
by Martin Matusiak July 1, 2014

July 1, 2014

Hi, I'm wondering if there will be pypy sprints at Europython this year (?) And if so, which are the principal topics (I assume stm is one of them..)? Thanks, Martin

4 7

pythonapi and pypy on windows
by Matti Picus June 30, 2014

June 30, 2014

3 7

The correct way to incrementally build list in RPython
by Yichao Yu June 27, 2014

June 27, 2014

Hi, On an attempt to fix the hash function of numpy.dtype, I came up with the patch in the attachment translated from hashdescr.c in the original numpy source. The patch passed the test when running in pyinteractive.py. However, when started to translate pypy, I get an ListChangeUnallowed error. I read in the Coding Guide that list.append should be allowed (which also said there's a lot of special rules) and I have seen usage of append in other RPython modules. So what is the rule of using list.append and what is the best way of doing this? (I guess I can always using application level lists but is that the only/best way?) Other comments on this patch (style/naming etc) are also welcome. Yichao Yu

2 5

PyPy on JVM
by Fabio Zadrozny June 24, 2014

June 24, 2014

Hi PyPy-dev, I remember there was some work on getting PyPy on JVM... is that still possible? Can someone point me on how to do it? Thank you, Fabio

3 3

[pypy] Adding gcc flags to Makefile
by Joe Borġ June 23, 2014

June 23, 2014

Hi all, Because I've had to build zlib away from its normal home, I need to pass -I to gcc (LD_LIBRARY_PATH doesn't work). Setting this -I fixes my build issue: gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused /tmp/usession-release-2.3.1-3/platcheck_57.c -o /tmp/usession-release-2.3.1-3/platcheck_57.o /tmp/usession-release-2.3.1-3/platcheck_57.c:33:19: fatal error: bzlib.h: No such file or director gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -I/scratch/root/include /tmp/usession-release-2.3.1-3/platcheck_57.c -o /tmp/usession-release-2.3.1-3/platcheck_57.o Thanks, Joseph David Borġ josephb.org

5 5

2.3.1 distribution issues?
by Laurence Tratt June 22, 2014

June 22, 2014

I feel I might be going mad but I think there are a few web-page issues with the 2.3.1 release. First the links to pypy-2.3.1-src.tar.bz2 and pypy-2.3.1-src.zip point to release-2.3.1.tar.bz2 and release-2.3.1.zip respectively in the get directory [1]. This confused the life out of me for a while. It also makes OS packagers nervous because there is a history of files being fetched from tags changing their checksums over time. [As an aside, decompressing release-2.3.1.tar.bz2 leads to a pypy-pypy-<somerandomhash> directory, which makes it annoying for OS packagers to upgrade the package. 2.3 has this problem as well, although I'm sure some past source releases had a more standard pypy-V.V directory.] If I go instead to BitBucket's downloads page [2], I can see binaries for various platforms for 2.3.1, but not a source archive? Laurie [1] http://pypy.org/download.html#building-from-source [2] https://bitbucket.org/pypy/pypy/downloads

2 2

how can a class method become frozen in tests only on windows?
by Matti Picus June 21, 2014

June 21, 2014

in looking at the own test_rzipfile failures on windows[0], the annotator is failing in rtyper.rpbc.redispatch_call since the method being called is a MultipleFrozenPBCRepr . The method is flush_buffers from (it seems) streamio.Stream [1] Any hints how this can happen? Matti [0] http://buildbot.pypy.org/summary/longrepr?testname=TestRZipFile.%28%29.test… [1] I added --pdb to the test run, and here is what the debugger had to say: > d:\pypy_stuff\pypy\rpython\rtyper\rpbc.py(825)redispatch_call() -> SmallFunctionSetPBCRepr)) (Pdb) p self <MethodsPBCRepr * GcStruct rpython.rlib.streamio.Stream { super }> (Pdb) p self.methodname 'flush_buffers'

2 1