Mailman 3 May 2014 - pypy-dev

Deadlock in cpyext callback
by Nick Williams 26 May '14

26 May '14

We have an extension module which works fine with CPython but deadlocks every time in PyPy. The problem is induced when the C++ code in the module (having been invoked from a Python thread) prepares to call back into Python again by calling PyGILState_Ensure. It seems at this point we try to acquire the GIL mutex while already (presumably) holding it - any other threads at this point are also waiting on the mutex so it isn't held elsewhere. This seems to occur from a call to rffi.aroundstate.after() - I assume that the invariant here is that the before() has already executed but that seems not to be the case here. What is supposed to trigger that? Thanks for any assistance in advance. Nick

2 2

Issue #1777: Unable to create proper numpy array from cffi buffer (pypy/pypy)
by Yichao Yu 26 May '14

26 May '14

New issue 1777: Unable to create proper numpy array from cffi buffer https://bitbucket.org/pypy/pypy/issue/1777/unable-to-create-proper-numpy-ar… Yichao Yu: I am writing something that need to convert a c buffer from cffi to a numpy array with arbitrary shape and strides. Besides the fact that [`as_strided`](https://bitbucket.org/pypy/numpy/issue/9/numpylibstride_tricksas_strided-is-not) is not really working, another problem is that the c buffer is maintained by the underlaying c/c++ library (OpenCL in this case). Therefore it is necessary to let the numpy ndarray hold a reference of the object which manage the memory. In CPython, this is done by [setting the base on the ndarray](https://github.com/pyopencl/pyopencl/blob/master/src/wrapper/wrap_…. However, there doesn't seem to be a way to do the same in (num)pypy. In pypy, although I can pass the buffer to `np.frombuffer` and call `reshape` to get an array roughly correct, the array is only holding a reference to the buffer returned by cffi but not the (c++) object (or the python delegate of it) which manages the memory. Furthermore, the base property of the result array cannot be used to access the underlying object (which is currently a PyOpenCL API that I would love not to break.) (The problem can probably be solved along with the [`as_strided` support bug](https://bitbucket.org/pypy/numpy/issue/9/numpylibstride_tricksas_strid… by allowing creating arrays from object with a proper array interface)

1 0

Issue #1776: Creating instances of a namedtuple is inexplicably very slow (pypy/pypy)
by Alex Gaynor 23 May '14

23 May '14

New issue 1776: Creating instances of a namedtuple is inexplicably very slow https://bitbucket.org/pypy/pypy/issue/1776/creating-instances-of-a-namedtup… Alex Gaynor: Compare: ``` $ pypy -mtimeit -s "from collections import namedtuple; pos = namedtuple('pos', ['x', 'y'])" "p = (1, 2)" 1000000000 loops, best of 3: 0.000867 usec per loop $ pypy -mtimeit -s "from collections import namedtuple; pos = namedtuple('pos', ['x', 'y'])" "p = (1, 2)" 1000000000 loops, best of 3: 0.000867 usec per loop $ pypy -mtimeit -s "from collections import namedtuple; pos = namedtuple('pos', ['x', 'y'])" "p = (1, 2)" 1000000000 loops, best of 3: 0.00087 usec per loop $ $ $ pypy -mtimeit -s "from collections import namedtuple; pos = namedtuple('pos', ['x', 'y'])" "p = pos(1, 2)" 100000000 loops, best of 3: 0.026 usec per loop $ pypy -mtimeit -s "from collections import namedtuple; pos = namedtuple('pos', ['x', 'y'])" "p = pos(1, 2)" 100000000 loops, best of 3: 0.0264 usec per loop $ pypy -mtimeit -s "from collections import namedtuple; pos = namedtuple('pos', ['x', 'y'])" "p = pos(1, 2)" 100000000 loops, best of 3: 0.0268 usec per loop ``` Looking at the traces, I don't see anything that explains the difference in the `jit-log-opt-loop` section, however looking at `jit-summary`, I see that the namedtuple version compiles many many more bridges, we should figure out why.

1 0

PyPy current status
by Antonio Cuni 23 May '14

23 May '14

Hi all, I am preparing the usual "PyPy status talk" which I'll give to the upcoming Pycon Italy, which is going to cover what happened in the last two years of PyPy. If you are interested, the draft slides are here: https://bitbucket.org/pypy/extradoc/src/tip/talk/pycon-italy-2014/talk.rst?… In the talk, I will to give an overview of the current status of the various subprojects, so I'd be glad if you could help because you surely know better than me the status of the area of your competence :) David: what is the current status of PyPy on ARM? Should I say "it just works" or there is something more to add? What about performance? Matti, Brian: what about numpy? Since people like numbers, what percentage of numpy we can consider completed? Philip: same question for py3k. Is it still considered beta quality or we can say it's stable? Alex, Maciej: I'll also briefly talk about other frontends, so Topaz and Hippy. How much complete are they? What are the performance? I know that hippy is still actively developed, but what about Topaz? Other than what I asked, I'll also highlight CFFI and STM. If anyone has ideas for other cool things which happened in PyPy since 2012, suggestions are welcome :) thank you very much! Anto

6 12

Issue #1775: imp.lock_held isn't quite right (pypy/pypy)
by eevee 22 May '14

22 May '14

New issue 1775: imp.lock_held isn't quite right https://bitbucket.org/pypy/pypy/issue/1775/implock_held-isnt-quite-right eevee: PyPy's `lock_held()` returns whether the lock is held by the current thread: https://bitbucket.org/pypy/pypy/src/6e9376d22e0ecc83bfcdda81d0e37e695b435dd… But CPython returns whether the lock is held _at all_: http://hg.python.org/cpython/file/3a1db0d2747e/Python/import.c#l348

1 0

Where is pypy.org?
by Laura Creighton 21 May '14

21 May '14

I am getting a ton of spam as 'contracts(a)pypy.org'. Somewhere that is a mail alias for me. I'd like to remove that alias, or have it removed if it isn't something I can do. This brings up the larger question as to whether that whole address is needed at all, or if it should just bounce. Laura

3 2

Issue #1774: Pypy not translating with gcc 4.9 on Arch Linux x86_64 (pypy/pypy)
by Aidan Epstein 17 May '14

17 May '14

New issue 1774: Pypy not translating with gcc 4.9 on Arch Linux x86_64 https://bitbucket.org/pypy/pypy/issue/1774/pypy-not-translating-with-gcc-49… Aidan Epstein: I am trying to translate the latest version of pypy from the mercurial repositories using the copy of pypy included in the Arch Linux repos. The arch linux version fails after a short amount of time with the attached stack trace. The official version from pypy.org (or rather the portable binaries from https://github.com/squeaky-pl/portable-pypy) works without a hiccup, however, when trying to use the resulting binary to translate, it fails with the same error. I reported it to the Arch Linux bug tracker, and they said that it was probably a gcc 4.9 issue and to report it here.

1 0

Issue #1773: py3k: continuation.test_zpickle test_pickle_continulet_real_subclass triggers a CPython crash (pypy/pypy)
by Philip Jenvey 16 May '14

16 May '14

New issue 1773: py3k: continuation.test_zpickle test_pickle_continulet_real_subclass triggers a CPython crash https://bitbucket.org/pypy/pypy/issue/1773/py3k-continuationtest_zpickle Philip Jenvey: test_pickle_continulet_real_subclass has been disabled on the py3k branch for now as it can trigger a seg fault when running under CPython. It'll crash with a "Fatal Python error: GC object already tracked", or at least on OSX: "malloc: *** error for object 0x109620af0: pointer being realloc'd was not allocated" continulets do some potentially 'evil' low level things but AFAICT nothing pypy's doing is at at fault here. The crash doesn't seem to occur on default even though the continulet implementations/tests are basically identical

1 0

Issue #1772: Complex dtype unhashable. (pypy/pypy)
by Yichao Yu 16 May '14

16 May '14

New issue 1772: Complex dtype unhashable. https://bitbucket.org/pypy/pypy/issue/1772/complex-dtype-unhashable Yichao Yu: In pypy, ``` pypy -c 'import numpy as np; t = np.dtype([("s1", np.int8), ("s2", np.int8)]); print(hash(t))' Traceback (most recent call last): File "app_main.py", line 75, in run_toplevel File "app_main.py", line 581, in run_it File "<string>", line 1, in <module> TypeError: 'dict' objects are unhashable ``` In CPython ``` python2 -c 'import numpy as np; t = np.dtype([("s1", np.int8), ("s2", np.int8)]); print(hash(t))' -5634517127405799957 ```

1 0

Issues moved to Bitbucket
by Sebastian Pawluś 16 May '14

16 May '14

Hi Small announcement. The new location for PyPy's issues/bugs from now on is on Bitbucket https://bitbucket.org/pypy/pypy/issues The old location (bugs.pypy.org) should be working, but only as a redirect to Bitbucket issues page. Fijał is working on it :) Special thanks to Marcus Bertrand (Bitbucket guy) for help. regards, sebastian pawluś

2 1