Mailman 3 April 2007 - pypy-dev

Re: [pypy-dev] Releasing the GIL in CPython extensions?
by Gary Robinson 11 Apr '07

11 Apr '07

>> It's easy enough to call the PyGIL_ functions manually from rpython.. > That might well be, but that does not release PyPy's GIL when the module > is used together with PyPy. OK, but overall it sounds like, if one were to write an CPython extension using RPython programming style so that the critical parts are translated to C, one CAN release the CPython GIL and thereby take advantage of multiprocessors for threading. My understanding from the second comment above is that this extension would not have the GIL-releasing ability when the module is used with PyPy. The docs say that the extension-writing ability is beta and that "bugs and rough edges are likely." Are there any estimates as to when it might be fairly stable? Thanks, Gary -- Gary Robinson CTO Emergent Music, LLC grobinson(a)goombah.com 207-942-3463 Company: http://www.goombah.com Blog: http://www.garyrobinson.net

1 0

translate.py --jit targetpypystandalone: broken?
by Simon Burton 09 Apr '07

09 Apr '07

this is not working for me: [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "./translate.py", line 287, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/driver.py", line 711, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/simon/site-packages/pypy/translator/tool/taskengine.py", line 112, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/driver.py", line 260, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/driver.py", line 365, in task_hintannotate_lltype [translation:ERROR] {OriginFlags(): True}) [translation:ERROR] File "/home/simon/local/pypy/pypy/jit/hintannotator/annotator.py", line 62, in build_types [translation:ERROR] return self.build_graph_types(flowgraph, input_args_hs) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 192, in build_graph_types [translation:ERROR] self.complete() [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 248, in complete [translation:ERROR] self.processblock(graph, block) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 477, in processblock [translation:ERROR] self.flowin(graph, block) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 537, in flowin [translation:ERROR] self.consider_op(block.operations[i]) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 724, in consider_op [translation:ERROR] raise Exception,"unknown op: %r" % op [translation:ERROR] Exception: unknown op: v6855 = cast_ptr_to_int(i_25) [translation:ERROR] Processing block: [translation:ERROR] block@-1 is a Block at some unknown location [translation:ERROR] containing the following operations: [translation:ERROR] v6856 = cast_pointer(i_25) [translation:ERROR] v6855 = cast_ptr_to_int(i_25) [translation:ERROR] v6857 = cast_int_to_uint(v6855) [translation:ERROR] v6858 = getfield(v6856, ('typeptr')) [translation:ERROR] v6859 = getfield(v6858, ('name')) [translation:ERROR] v6860 = getarraysize(v6859) [translation:ERROR] v6861 = int_sub(v6860, (1)) [translation:ERROR] v6862 = direct_call((<* fn mallocstr__Signed>), v6861) [translation:ERROR] v6863 = getfield((<* struct ExcData { exc_type=..., exc_value=... }>), ('exc_type')) [translation:ERROR] v6864 = ptr_iszero(v6863) [translation:ERROR] --end-- [translation] start debugger... > /home/simon/site-packages/pypy/annotation/annrpython.py(724)consider_op() -> raise Exception,"unknown op: %r" % op $ svn info Path: . URL: https://codespeak.net/svn/pypy/dist/pypy Repository Root: https://codespeak.net/svn Repository UUID: fd0d7bf2-dfb6-0310-8d31-b7ecfe96aada Revision: 41953 Node Kind: directory Schedule: normal Last Changed Author: hpk Last Changed Rev: 41916 Last Changed Date: 2007-04-05 09:29:13 -0700 (Thu, 05 Apr 2007) Properties Last Updated: 2006-10-08 12:39:47 -0700 (Sun, 08 Oct 2006) Simon.

2 2

Releasing the GIL in CPython extensions?
by Gary Robinson 05 Apr '07

05 Apr '07

Hello, In PyPy's facilities for writing a CPython extension, would it be conceivable to have an option to release the CPython GIL when the extension is entered and grab it back upon exit? That would enable multiprocessing and could be extremely useful when the programmer knows he's not going to modifying shared items during the execution of the extension. Thanks, Gary -- Gary Robinson CTO Emergent Music, LLC grobinson(a)goombah.com 207-942-3463 Company: http://www.goombah.com Blog: http://www.garyrobinson.net

3 2

Towards pypy-jvm
by Antonio Cuni 04 Apr '07

04 Apr '07

Hi Niko, hi all! I've read in the IRC logs that there has been a bit of discussion about what genjvm still lacks in order to translate pypy. Some weeks ago I also tried to translate pypy-jvm; it seems that the two most important missing features are r_dict and weakrefs. The good news is that with some hacks it's possible to get a pypy version that doesn't make use of r_dict or weakrefs: have a look at this IRC log: http://tismerysoft.de/pypy/irc-logs/pypy/%23pypy.log.20070307 The bad news is that even with those changes, jvm crashed because of a failed assertion, then I gave up. I've no clue what it's going wrong, but maybe it's not something terribly wrong to fix. I hope that this infos can help you in some way :-). ciao Anto

3 9

Re: [pypy-dev] Paul Fernhout wants edit-and-continue in python
by Luke Kenneth Casson Leighton 02 Apr '07

02 Apr '07

integration with leo - http://leo.sf.net - would be a good start. that would give you the GUI part, and context-sensitive execution. you _might_ have to do something similar to what gcc currently do, which is to hook in to eevvvveerryyything via __blahblah__ functions, for memory access eevveryyyything, in the "profiling". but that shouldn't be too hard (but would result in a performance hit - oh well, can't have everything). execution itself (from leo) could be performed via "yield". ok. you can obtain a list of all variables (global etc.) that a particular function modifies, right? so, before you go into any function, right, you record the previous "state". in fact, you might be able to record "state" on a per-line basis (bit extreme i know) and basically have an "undo" stack yeh. it's very simple. look up how "undo" is implemented, in editors, and then treat all variables like that. then, you can keep a record (using Leo) of which lines were modified, and use that to calculate which bits need JIT recompilation and how far back the "undo" stack you need to roll. in _fact_, and this is going to _really_ blow you away, Leo probably already _has_ an "Undo" capability - and so if you mentally turn things round, by treating "execute 1 function" as an "edit" operation under Leo, then you get what you want, for free!!! ooooo :) of course, if any variables are "volatile" - e.g. in an OS, like cleese, then you're shafted - but that's another story. and, of course, if anything calls out to _non_ python code, then you're equally shafted. in both cases, you can simply throw an exception saying "sorry, you need to run this program from scratch, dude". you stand a better chance of doing this in pypy than the python developers do. they consider it "impossible", because it's too radical a rethink, because they have been trapped, by design, in c-code. l. p.s. leo is in python, in case you hadn't guessed.

1 0