Mailman 3 December 2017 - pypy-dev

Re: [pypy-dev] a question about translating pypy for raspberry pi
by Antonio Cuni 23 Dec '17

23 Dec '17

Hi Gelin, Please make sure to include pypy-dev in the CC, so that the others can read the answer. I suggest you to try two things: 1. Try to run pypy under strace: this way you can see where it tries to find .so libraries and which one it cannot find (it might be a dependency of libpypy-c itself) 2. Try a nightly build from here: http://buildbot.pypy.org/nightly/trunk/ Il 10 nov 2017 7:51 PM, "Gelin Yan" <dynamicgl(a)gmail.com> ha scritto: > > > On Sat, Nov 11, 2017 at 1:50 AM, Antonio Cuni <anto.cuni(a)gmail.com> wrote: > >> Maybe it's a stupid advice but: are you sure to have copied also >> libpypy-c.so to the raspberry? >> Is it in the same directory as the "pypy" or "pypy-c" executable? >> >> I suggest you to run the script "pypy/tool/release/package.py", which >> builds a tarball containing all the necessary stuff (including the stdlib). >> Then you can simply untar on the raspberry and run it. >> >> You should run it this way: >> python package.py --archive-name=my-pypy-for-rpi >> >> >> >> >> > Hi Antonio > > Yes. I did run the package.py to make a pypy package for ARM. I did > check libpypy-c.so which is placed on the same directory of where pypy is. > > Regards > > gelin yan > >

2 1

Re: [pypy-dev] Mysterious IndexError in service running with PyPy
by Armin Rigo 19 Dec '17

19 Dec '17

Hi Hubo, On 18 December 2017 at 08:58, hubo <hubo(a)jiedaibao.com> wrote: > The services are running with PyPy 5.3.0. I've upgraded a few of them to > 5.9, but it will take a long time to validate whether this still happens. The root cause might be some issue with the JIT: if there is a bug there, then we can get this kind of nonsense-looking result. It is however completely impossible to know what the issue is just by looking at the Python code. Usually it takes a lot of efforts inside gdb to figure these bugs out. The good news is that these issues are extremely rare: much less than one per release, in average. It's possible that we fixed a JIT issue between PyPy 5.3 and PyPy 5.9 and that this issue caused the misbehavior you're observing. All I can suggest is to continue trying with PyPy 5.9. A bientôt, Armin.

2 1

Mysterious IndexError in service running with PyPy
by hubo 18 Dec '17

18 Dec '17

I'm reporting this issue in this mail group, though I don't know if it is related with PyPy, because it is really strange, and is not able to reproduce stably. But I hope someone may know the reason or have some points. I'm running some SDN services written in Python with PyPy 5.3.0. The related code is here: https://github.com/hubo1016/vlcp/blob/8022e3a3c67cf4305af503d507640a730ca39… The full code is also in the repo, but may be too complex to describe. But this related piece is quite simple: def _siftdown(self, pos): temp = self.heap[pos] l = len(self.heap) while pos * 2 + 1 < l: cindex = pos * 2 + 1 pt = self.heap[cindex] if cindex + 1 < l and self.heap[cindex+1][0] < pt[0]: cindex = cindex + 1 pt = self.heap[cindex] if pt[0] < temp[0]: self.heap[pos] = pt self.index[pt[1]] = pos else: break pos = cindex self.heap[pos] = temp self.index[temp[1]] = pos It is a simple heap operation. The service uses a heap to process timers. When the service is not busy, it usually runs this piece of code several times per minute. I have 32 servers running this service. They are quite stable in about three months, but one day one of the services crashes on line 100 reporting IndexError: pt = self.heap[cindex] As you can see, cindex = pos * 2 + 1, which is tested by the while pre-conditon just two lines before. And there is not any multi-threading issues here because this piece of code always runs in the same thread. So it is not possible in theory for this to happen. Only the following facts are known about this issue: It reproduces - through quite rarely. I've met this kind of crashes 4 times, each with different machine, so it should not be related to hardware issuess. Since I've got 32 servers, it might take more than one year to reproduce with a single server. It seems not to be related to pressures. All of the crashes happens at night, when there are little requests. Only some cleanup tasks are running in fixed interval. The services are running with PyPy 5.3.0. I've upgraded a few of them to 5.9, but it will take a long time to validate whether this still happens. And It is not validated on CPython too. I'm also trying to collect more debugging information for this issue, but it is very hard since it rarely reproduces. It is not a serious issue. It could be workarounded with a auto-restart, but I'm searching the cause. 2017-12-18 hubo

1 0

[MoreVMs'18] Call for Papers
by Edd Barrett 15 Dec '17

15 Dec '17

Hi Everyone, This may be of interest to some of the members on this list. Please consider contributing. ============================================================================ Call for Papers: MoreVMs'18 2nd Workshop on Modern Language Runtimes, and Ecosystems Co-located with <Programming>'18 April 9th, 2018, Nice, France https://2018.programming-conference.org/track/MoreVMs-2018 ============================================================================ The MoreVMs'18 workshop aims to bring together industrial and academic programmers to discuss the design, implementation, and usage of modern languages and runtimes. This includes aspects such as reuse of language runtimes, modular implementation, language design and compilation strategies. The workshop aims to enable a diverse discussion on how languages and runtimes are currently being utilized, and where they need to improve further. We welcome presentation proposals in the form of extended abstracts discussing experiences, work-in-progress, as well as future visions, from either an academic or industrial perspective. Relevant topics include, but are definitely not limited to, the following: - Extensible VM design (compiler- or interpreter-based VMs) - Reusable components (e.g. interpreters, garbage collectors, ...) - Static and dynamic compilation techniques - Techniques for targeting high-level languages such as JavaScript - Interoperability between languages - Tooling support (e.g. debugging, profiling, etc.) - Programming language development environments - Case studies of existing language implementation approaches - Language implementation challenges and trade-offs - Surveys and usage reports to understand usage in the wild - Ideas for more predictable performance - Ideas for how VMs could take advantage of new hardware features - Ideas for how we should build languages in the future Workshop Format and Submissions ------------------------------- Submissions should use the ACM Conference acmart Format with the ‘sigconf’ option with a font size of 10 point and the font family Times New Roman. All submissions should be in PDF format. If you use LaTeX or Word, please use the provided ACM acmart templates. Otherwise, please follow the ACM author instructions. https://www.acm.org/publications/proceedings-template If you are formatting your paper using LaTeX, you will need to set the 10pt option in the \documentclass command. If you are formatting your paper using Word, you may wish to use the provided Word template that supports this font size. Please include page numbers in your submission for review using the LaTeX command \settopmatter{printfolios=true} (see examples in template). Please also ensure that your submission is legible when printed on a black and white printer. In particular, please check that colors remain distinct and font sizes are legible. Submission details will be on: https://2018.programming-conference.org/track/MoreVMs-2018 Important Dates --------------- Extended abstract submission: 2018-01-26 Author notification: 2018-02-23 Workshop: 2018-04-09 All deadlines are Anywhere on Earth (AoE), i.e. GMT/UTC-12:00 hour. Program Committee ----------------- Clément Bera, Inria Lille Nord Europe, France Maxime Chevalier-Boisvert, University of Montreal, Canada Sébastien Doeraene, EPFL, Switzerland Johan Fabry, Raincode Labs, Belgium Juliana Franco, Imperial College London, United Kingdom Tobias Grosser, ETH Zurich, Switzerland Ben L. Titzer, Google, Germany Niko Matsakis, Mozilla Research, United States Armin Rigo, PyPy, Switzerland Maoni Stephens, Microsoft, United States Gaël Thomas, Telecom SudParis, France Mario Wolczko, Oracle Labs, United States Workshop Organizers ------------------- Edd Barrett, King's College London, United Kingdom Adam Welc, Uber Technologies, United States Stefan Marr, University of Kent, United Kingdom -- Best Regards Edd Barrett http://www.theunixzoo.co.uk

1 0

Delimited Continuations in PyPy?
by Timothy Baldridge 15 Dec '17

15 Dec '17

I recently came across this paper: https://www.microsoft.com/en-us/research/publication/implementing-algebraic… The gist is that they use setjmp/longjmp to create delimited continuations in compliant C99. It seems like this method is fairly close to what PyPy already does with stacklets. Any reason why this couldn't be ported to PyPy? Of course there'd have to be some sort of stack integration as with stacklets, but it seems that that mostly boils down to making sure the GC has a way to trace the bits of the stack contained in the continuations. I'm considering trying my hand at using their library via rffi, and copying what I need from _stacklet_shadowstack.py. But perhaps there's something I'm missing that makes this a no-go from the start. Thanks, Timothy Baldridge

3 3

Improving the documentation on how we test
by Manuel Jacob 11 Dec '17

11 Dec '17

Hi, I'm currently implementing PEP 526 [1], which requires additions to the interpreter from the parser down to the bytecode interpreter. Since this is a common task, I want to expand the documentation a bit, adding a "how to extend the interpreter" recipe. One thing which I wasn't 100% sure about myself is how to test it. Clearly every addition has to come with a test, preferably written before implementing anything. But a few things are a bit unclear: For every change in CPython there is (hopefully) a test case in lib-python/*/test. So technically there's already a test case when modifying the interpreter. One reason to add app-level tests is to avoid re-translations. But other than this convenience, how much app-level testing to we *require*? Should we test everything up to the last detail or are we fine with testing the basic functionality and leaving the rest to the CPython test suite? A related question is which style of testing — integration-test-style or unit-test-style — we prefer. Different people — also among the core developers — have different preferences, so we have a bit of a mix. So in this concrete example, it is tempting to create a single test like AppTestFunctionAnnotations.test_simple from pypy/interpreter/test/test_syntax.py [2] and then add all the functionality to the interpreter until the test passes (this would be integration-style testing). From the unit-style testing perspective, this test is not very good, because it doesn't test only the syntax, like the file name "test_syntax.py" would suggest. The alternative would be to add a unit test for every affected part of the interpreter (parser, AST builder, AST validator, bytecode generation, symtable, stack effect computation, bytecode interpreter). What do you think? Feel free to braindump your general thoughts (also outside of the syntax addition case), so I can extend other relevant parts of the documentation as well. This will hopefully make it easier for new (but also more advanced) contributors and ensure a consistent test suite quality all over our code base. -Manuel [1] https://www.python.org/dev/peps/pep-0526/ [2] https://bitbucket.org/pypy/pypy/src/e30b20325b579b5a92cba08d326d2e385caba2d…

4 3