Mailman 3 February 2014 - pypy-dev

Telling the JIT to remove a stack array
by Timothy Baldridge 24 Feb '14

24 Feb '14

I have an interpreter inner loop that looks something like this: jitdriver = JitDriver(greens=['ip', 'tos', 'bytecodes', 'consts'], reds=['locals', 'stack']) def interpret(code, args): assert isinstance(code, Code) bytecodes = code.bytecodes consts = code.consts vars = code.vars locals = array_of_size(code.locals) ip = 0 stack = array_of_size(code.stacksize) tos = -1 while True: jitdriver.jit_merge_point(ip=ip, tos=tos, bytecodes=bytecodes, consts=consts, locals=locals, stack=stack) op = ord(bytecodes[ip]) if op == POP: stack[tos] = None tos -= 1 ip += 1 continue elif op == PUSH_CONST: arg = ord(bytecodes[ip + 1]) tos += 1 stack[tos] = consts[arg] ip += 2 continue The jit traces from this aren't bad, but it insists on maintaining the stack and locals arrays. Since these arrays are reference arrays, my primitive types (Integers for example) are allocated when math is being performed. What can I do to force the JIT to not maintain a stack/locals array. Or should I construct the stack in a different way? I tried digging into the PyPy source to find how how this is done there, but I haven't been able to find it yet. Timothy

3 5

Easy way to get access to cmpxchg?
by Timothy Baldridge 24 Feb '14

24 Feb '14

It'd be nice to be able to do something like this in rpython: compare_and_swap(foo.bar, oldval, newval) Or basically the cmpxchg. How hard would it be to get this in? Timothy -- "One of the main causes of the fall of the Roman Empire was that-lacking zero-they had no way to indicate successful termination of their C programs." (Robert Firth)

4 5

Python C-API on Windows
by Johan Råde 24 Feb '14

24 Feb '14

Is it possible to use the Python C-API with PyPy on Windows? Has it ever been tested? I tried to build and import a simple hello world extension module using the C-API with PyPy on Windows. It fails. You can find the code (and its expected and observed behavior) at the end of this post. One reason it fails is that the definition of PyMODINIT_FUNC in modsupport.h is incorrect. On Windows the module initialization function needs a __declspec(dllexport) declaration or it will not be exported from the DLL. Currently modsupport.h contains the code #ifdef __cplusplus #define PyMODINIT_FUNC extern "C" void #else #define PyMODINIT_FUNC void #endif This should be something like #ifdef __cplusplus # ifdef _WIN32 # define PyMODINIT_FUNC extern "C" __declspec(dllexport) void # else # define PyMODINIT_FUNC extern "C" void # endif #else # ifdef _WIN32 # define PyMODINIT_FUNC __declspec(dllexport) void # else # define PyMODINIT_FUNC void # endif #endif But even after I fixed this issue, I'm still unable to import the extension module using the Python C-API on Windows. (The observed behavior was the same, before and after I fixed this.) ------------------------------------------------------------------------------------------------------------------------- C code: #include <Python.h> static PyObject* bar(void) { return PyString_FromString("Hello world!"); } static PyMethodDef fooMethods[] = { {"bar", (PyCFunction)&bar, METH_NOARGS, ""}, {NULL, NULL, 0, NULL} }; PyMODINIT_FUNC initfoo(void) { Py_InitModule("foo", fooMethods); } Python code: import foo print foo.bar() Expected behavior and observed behavior with CPython: Hello world! Observed behavior with PyPy: Traceback (most recent call last): File "app_main.py", line 72, in run_toplevel File "C:\Test\test\test.py", line 1, in <module> import foo ImportError: No module named foo --------------------------------------------------------------------------------------- --Johan

3 6

numpy plans in the next 6 months?
by Ian Ozsvald 22 Feb '14

22 Feb '14

Hi again. In relation to my other mail, I'm curious about the plans for numpy in the next 6 months. Is there an expectation that better-than-numpy results might be obtained (e.g. using AVX/SSE or being cache friendly)? I'll mention these as they will probably be relevant to some of the audience. Cheers, Ian. -- Ian Ozsvald (A.I. researcher) ian(a)IanOzsvald.com http://IanOzsvald.com http://MorConsulting.com/ http://Annotate.IO http://SocialTiesApp.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald http://ShowMeDo.com

5 8

Debugging Blocked Block exceptions
by Timothy Baldridge 22 Feb '14

22 Feb '14

In an interpreter I'm writing I'm often running into Blocked block exceptions. I've been able to solve several of these errors but the problem is always figuring out what exactly I've done to cause the error. Most of the time it seems to come down to some sort of missing assertions that cause the translator to think that there is no way for an operation to succeed, but I just can't find it. For example, I'm currently getting this error: [translation:ERROR] AnnotatorError: [translation:ERROR] [translation:ERROR] Blocked block -- operation cannot succeed [translation:ERROR] [translation:ERROR] v1 = getattr(v0, ('data')) [translation:ERROR] [translation:ERROR] In <FunctionGraph of (clojure.vm.array:18)Array.aget at 0x1019d52d0>: [translation:ERROR] Happened at file /Users/tim/oss/clojure-vm/clojure/vm/array.py line 20 [translation:ERROR] [translation:ERROR] ==> return self.data[idx] [translation:ERROR] [translation:ERROR] Known variable annotations: [translation:ERROR] v0 = SomeInstance(can_be_None=False, classdef=clojure.vm.array.Array) for this class: from clojure.vm.object import Object, type_ids from clojure.vm.primitives import nil class Array(Object): type_id = type_ids.next() def __init__(self, data, meta = nil): self.data = data self.meta = meta def type(self): return self.type_id def aclone(self): return Array(self.data[:], self.meta) def aget(self, idx): assert isinstance(idx, int) return self.data[idx] def aset(self, idx, val): assert isinstance(idx, int) assert isinstance(val, Object) self.data[idx] = val def alength(self): return len(self.data) Of course, the error is wrong, and it must be related to some other part of the code, but I feel like a blind man in the dark trying to find it. Is there any good way to debug these sorts of errors? Timothy

2 1

GitHub
by Armin Rigo 18 Feb '14

18 Feb '14

Hi all, GitHub has a two-years-old mirror of PyPy: https://github.com/pypy/pypy . Nowadays that looks not only useless, but harmful. Would anyone with the proper priviledges disable, hide, or kill it? I can't do it myself apparently (or didn't find the correct button). A bientôt, Armin.

2 2

matplotlib, and numpy
by matti picus 18 Feb '14

18 Feb '14

I have had less time to contribute to pypy lately, so here is where the things I took responsibility for stand: numpy - there are many failing tests due to the way numpy random.seed works. Upstream uses a mtrand module, and I simply wrapped the existing python random module. There are subtle differences between the seed in the different modules, we get a different series when setting the seed in our mtrand module. - numpy.linalg.inv does not work (see below) matplotlib - linlag support in numpy is not really working. This means any kind of point transform does not work, which precludes use of legends and maybe subplots, I did not check. - backend support for a gui framework is also missing. so what works (I last checked 4 months ago) is basic plotting and saving to a file. The repo, for lack of a better place, lives here, I would be happy to put it somewhere more public. https://github.com/mattip/matplotlib master branch Matti

1 0

Bytearray broken complexity
by Armin Rigo 18 Feb '14

18 Feb '14

Hi all, Yet another mail :-) This is a small note to mention that the str/unicode/bytearray branch that was merged earlier into "default" has some problems of complexity with bytearrays. For example, simply reading "ba[5]" requires a complete copy of the bytearray to be done internally. This means that expected-constant-time operations are not. This mail is thus a reminder that we should do something about it --- we *have* to, before the next release, in my opinion. A bientôt, Armin.

1 0

Conferences for STM paper
by Armin Rigo 18 Feb '14

18 Feb '14

Hi Remi, For reference, here are two workshops where an STM paper would probably be a good idea: * Dyla'14 (Workshop on Dynamic Languages and Applications), June 12th, Edinburgh, UK, co-located with PLDI'14. We have already published about PyPy at a couple of previous Dyla workshops. Fwiw, Carl Friedrich Bolz is a co-organizer of this one. Deadline is March 15th, either for a complete article, or for the title of a 10-minutes lightning talk. See http://www.lifl.fr/dyla14/ * WTM 2014 (Workshop on Transactional Memory), Amsterdam, April 13, co-located with EuroSys 2014. More directly about TM, and less far in time -- but I can't be there myself, as the dates conflict with PyCon US. This workshop doesn't really publish anything. We send an abstract only, deadline Feb 22nd. http://www.eurotm.org/action-meetings/wtm2014 I think we should aim for Dyla in any case. If we're accepted I would like to go there, and hope that you'd come too. It may be a good thing for you to go to WTM as well, or instead, but that's up to you, of course. A bientôt, Armin.

1 0

PyPy Intermediate C Files
by Kirk Liberty 18 Feb '14

18 Feb '14

Hello all, I'm looking to do some experiments with different C compilers and flags for PyPy, but I don't want to have to go through the entire translation process only to change a flag. Are the C files that are written as one of the last steps of translation written in q way that they can be rebuilt many times? Thanks, Kirk

5 5