Hi,
FYI, translating pypy for the CLI backend using mono-2.11.1 in a 32bit environment results in the following error: [Timer] Timings: [Timer] annotate --- 309.3 s [Timer] rtype_ootype --- 326.7 s [Timer] ========================================== [Timer] Total: --- 636.0 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 308, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 791, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 285, in _do [translation:ERROR] res = func() [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 383, in task_rtype_ootype [translation:ERROR] crash_on_first_typeerror=insist) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 211, in specialize [translation:ERROR] self.specialize_more_blocks() [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 254, in specialize_more_blocks [translation:ERROR] self.specialize_block(block) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 408, in specialize_block [translation:ERROR] self.gottypererror(e, block, hop.spaceop, newops) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 406, in specialize_block [translation:ERROR] self.translate_hl_to_ll(hop, varmapping) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 535, in translate_hl_to_ll [translation:ERROR] resultvar = hop.dispatch() [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 768, in dispatch [translation:ERROR] return translate_meth(self) [translation:ERROR] File "<18375-codegen /tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py:610>", line 4, in translate_op_simple_call [translation:ERROR] return r_arg1.rtype_simple_call(hop) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rbuiltin.py", line 126, in rtype_simple_call [translation:ERROR] return self._call(hop2) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rbuiltin.py", line 117, in _call [translation:ERROR] v_result = bltintyper(hop2, **kwds_i) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rlib/jit.py", line 802, in specialize_call [translation:ERROR] v_value = hop.inputarg(repr, arg=2) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 788, in inputarg [translation:ERROR] return self.llops.convertvar(v, r_binding, converted_to) [translation:ERROR] File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 933, in convertvar [translation:ERROR] (r_from, r_to)) [translation:ERROR] TyperError: don't know how to convert from <StringRepr String> to <StringRepr * GcStruct rpy_string { hash, chars }> [translation:ERROR] .. (pypy.rlib.jit:557)set_param__None_enable_opts [translation:ERROR] .. block@12 with 1 exits [translation:ERROR] .. v552 = simple_call((function _set_param), driver_0, name_0, value_0)
I know the CLI backend is not really supported but I am sharing this in the event someone is interested.
Regards,
David
Hi David,
On Tue, Jun 12, 2012 at 11:58 AM, David Naylor naylor.b.david@gmail.com wrote:
[translation:ERROR] .. (pypy.rlib.jit:557)set_param__None_enable_opts
Nowadays, you cannot use the JIT with the CLI backend.
Armin
On Wed, Jun 13, 2012 at 1:06 PM, Armin Rigo arigo@tunes.org wrote:
Hi David,
On Tue, Jun 12, 2012 at 11:58 AM, David Naylor naylor.b.david@gmail.com wrote:
[translation:ERROR] .. (pypy.rlib.jit:557)set_param__None_enable_opts
Nowadays, you cannot use the JIT with the CLI backend.
Armin _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
That probably does not mean that someone somewhere is not calling set_param, like from command line parsing.
Hi Fijal,
On Wed, Jun 13, 2012 at 1:17 PM, Maciej Fijalkowski fijall@gmail.com wrote:
That probably does not mean that someone somewhere is not calling set_param, like from command line parsing.
It does mean it. At RPython level the only callers of rlib.jit.set_param() are all from module/pypyjit. For the command-line parsing, this is done in app_main.py, which will just fail to import "pypyjit" on a no-jit pypy (and give a nice explanation).
A bientôt,
Armin.
On Wednesday, 13 June 2012 13:06:14 Armin Rigo wrote:
Hi David,
On Tue, Jun 12, 2012 at 11:58 AM, David Naylor naylor.b.david@gmail.com wrote:
[translation:ERROR] .. (pypy.rlib.jit:557)set_param__None_enable_opts
Nowadays, you cannot use the JIT with the CLI backend.
Translating with -O2 I get: [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 308, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 791, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 285, in _do [translation:ERROR] res = func() [translation:ERROR] File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 618, in task_source_cli [translation:ERROR] filename = self.gen.generate_source() [translation:ERROR] File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/cli/gencli.py", line 56, in generate_source [translation:ERROR] GenOO.generate_source(self) [translation:ERROR] File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/genoo.py", line 68, in generate_source [translation:ERROR] self.gen_pendings() [translation:ERROR] File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/genoo.py", line 83, in gen_pendings [translation:ERROR] node.render(self.ilasm) [translation:ERROR] File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/function.py", line 131, in render [translation:ERROR] self.render_normal_block(block) [translation:ERROR] File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/function.py", line 239, in render_normal_block [translation:ERROR] self._render_op(op) [translation:ERROR] File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/function.py", line 369, in _render_op [translation:ERROR] assert instr_list is not None, 'Unknown opcode: %s ' % op [translation:ERROR] AssertionError: Unknown opcode: v2539 = jit_is_virtual(vref_0)
Regards
Hi David,
On Wed, Jun 13, 2012 at 2:49 PM, David Naylor naylor.b.david@gmail.com wrote:
[translation:ERROR] assert instr_list is not None, 'Unknown opcode: %s ' % op [translation:ERROR] AssertionError: Unknown opcode: v2539 = jit_is_virtual(vref_0)
Try again?
Armin
On Friday, 15 June 2012 07:44:59 Armin Rigo wrote:
Hi David,
On Wed, Jun 13, 2012 at 2:49 PM, David Naylor naylor.b.david@gmail.com
wrote:
[translation:ERROR] assert instr_list is not None, 'Unknown opcode: %s ' % op [translation:ERROR] AssertionError: Unknown opcode: v2539 = jit_is_virtual(vref_0)
Try again?
Aren't the builds deterministic? I got a similar result when using -O0 (although different opcode). Are you asking me to try the latest for mercurial? I had been using the released sources.
On Fri, Jun 15, 2012 at 10:55 PM, David Naylor naylor.b.david@gmail.comwrote:
On Friday, 15 June 2012 07:44:59 Armin Rigo wrote:
Hi David,
On Wed, Jun 13, 2012 at 2:49 PM, David Naylor naylor.b.david@gmail.com
wrote:
[translation:ERROR] assert instr_list is not None, 'Unknown opcode: %s ' % op [translation:ERROR] AssertionError: Unknown opcode: v2539 = jit_is_virtual(vref_0)
Try again?
Aren't the builds deterministic? I got a similar result when using -O0 (although different opcode). Are you asking me to try the latest for mercurial? I had been using the released sources.
That typically means armin has checked something in and you have to update and try again (I would think).
For completeness sake (not relevant to this mail) - builds are not entirely deterministic. In edge cases the exact error (or lack thereof) depends on some dictionary order (the order in which it's seen by the translator), which in turn might depend on memory locations of objects.
Cheers, fijal
On Saturday, 16 June 2012 09:18:22 Maciej Fijalkowski wrote:
On Fri, Jun 15, 2012 at 10:55 PM, David Naylor
naylor.b.david@gmail.comwrote:
On Friday, 15 June 2012 07:44:59 Armin Rigo wrote:
Hi David,
On Wed, Jun 13, 2012 at 2:49 PM, David Naylor naylor.b.david@gmail.com
wrote:
[translation:ERROR] assert instr_list is not None, 'Unknown opcode: %s ' % op [translation:ERROR] AssertionError: Unknown opcode: v2539 = jit_is_virtual(vref_0)
Try again?
Aren't the builds deterministic? I got a similar result when using -O0 (although different opcode). Are you asking me to try the latest for mercurial? I had been using the released sources.
That typically means armin has checked something in and you have to update and try again (I would think).
Oh, okay. I don't have access to the latest sources. I normally work of the released sources. But thanks for looking at this.
For completeness sake (not relevant to this mail) - builds are not entirely deterministic. In edge cases the exact error (or lack thereof) depends on some dictionary order (the order in which it's seen by the translator), which in turn might depend on memory locations of objects.
Interesting, thanks.
Regards
Hi David,
On Sat, Jun 16, 2012 at 4:41 PM, David Naylor naylor.b.david@gmail.com wrote:
Oh, okay. I don't have access to the latest sources. I normally work of the released sources. But thanks for looking at this.
It's going to take us a loooooong time to fix CLI translation if the process goes: "you find some minor bug in the release, we attempt to fix it a bit blindly, and then we wait for the next release to know if it worked and what the next minor bug is" :-)
In case you want to help slightly more actively, you need to get the sources via Mercurial's command "hg clone https://bitbucket.org/pypy/pypy" and then keep up-to-date with "hg pull && hg update". Or if you don't mind vasting bandwidth you can download every time the latest source from https://bitbucket.org/pypy/pypy/get/default.tar.bz2 or https://bitbucket.org/pypy/pypy/get/default.zip .
A bientôt,
Armin.
On Sunday, 17 June 2012 09:32:35 Armin Rigo wrote:
Hi David,
On Sat, Jun 16, 2012 at 4:41 PM, David Naylor naylor.b.david@gmail.com wrote:
Oh, okay. I don't have access to the latest sources. I normally work of the released sources. But thanks for looking at this.
It's going to take us a loooooong time to fix CLI translation if the process goes: "you find some minor bug in the release, we attempt to fix it a bit blindly, and then we wait for the next release to know if it worked and what the next minor bug is" :-)
This is true. Would it be possible to do a RC release, or a heads up, so that I may track down these small minor bugs *before* the release is made?
In case you want to help slightly more actively, you need to get the sources via Mercurial's command "hg clone https://bitbucket.org/pypy/pypy" and then keep up-to-date with "hg pull && hg update".
Here are my results: # mono --version Mono JIT compiler version 2.11.1 (tarball Wed Jun 13 11:35:28 UTC 2012) Copyright (C) 2002-2012 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com TLS: normal SIGSEGV: normal Notification: kqueue Architecture: x86 Disabled: none Misc: softdebug LLVM: supported, not enabled. GC: Included Boehm (with typed GC and Parallel Mark)
1) First attempt (fixed with quick patch): # python translate.py --source -O2 --backend=cli <snip/> [translation:ERROR] Skipped: ilasm2 is not on your path.
2) Second attempt (fixed with quick patch): # python translate.py --source -O2 --backend=cli <snip/> [translation:ERROR] CompilationError': CompilationError(err=""" [translation:ERROR] /tmp/usession-unknown-1/platcheck_4.c:38:17: error: pty.h: No such file or directory [translation:ERROR] """) [translation:ERROR] .. v1 = simple_call((builtin_function_or_method write), (2), v0) [translation:ERROR] .. '(targetpypystandalone:19)debug' [translation:ERROR] Processing block: [translation:ERROR] block@15 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'> [translation:ERROR] in (targetpypystandalone:19)debug [translation:ERROR] containing the following operations: [translation:ERROR] v2 = add(('debug: '), msg_0) [translation:ERROR] v0 = add(v2, ('\n')) [translation:ERROR] v1 = simple_call((builtin_function_or_method write), (2), v0) [translation:ERROR] --end--
3) Third attempt: # python translate.py --source -O2 --backend=cli <snip/> [rtyper] -=- specialized 10 more blocks -=- [Timer] Timings: [Timer] annotate --- 280.7 s [Timer] rtype_ootype --- 367.2 s [Timer] backendopt_ootype --- 54.7 s [Timer] source_cli --- 9.8 s [Timer] ========================================== [Timer] Total: --- 712.3 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 308, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/tmp/pypy/pypy/translator/driver.py", line 791, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/tmp/pypy/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/tmp/pypy/pypy/translator/driver.py", line 285, in _do [translation:ERROR] res = func() [translation:ERROR] File "/tmp/pypy/pypy/translator/driver.py", line 618, in task_source_cli [translation:ERROR] filename = self.gen.generate_source() [translation:ERROR] File "/tmp/pypy/pypy/translator/cli/gencli.py", line 56, in generate_source [translation:ERROR] GenOO.generate_source(self) [translation:ERROR] File "/tmp/pypy/pypy/translator/oosupport/genoo.py", line 68, in generate_source [translation:ERROR] self.gen_pendings() [translation:ERROR] File "/tmp/pypy/pypy/translator/oosupport/genoo.py", line 83, in gen_pendings [translation:ERROR] node.render(self.ilasm) [translation:ERROR] File "/tmp/pypy/pypy/translator/oosupport/function.py", line 131, in render [translation:ERROR] self.render_normal_block(block) [translation:ERROR] File "/tmp/pypy/pypy/translator/oosupport/function.py", line 239, in render_normal_block [translation:ERROR] self._render_op(op) [translation:ERROR] File "/tmp/pypy/pypy/translator/oosupport/function.py", line 377, in _render_op [translation:ERROR] instr_list.render(self.generator, op) [translation:ERROR] File "/tmp/pypy/pypy/translator/oosupport/metavm.py", line 246, in render [translation:ERROR] instr.render(generator, op) [translation:ERROR] File "/tmp/pypy/pypy/translator/cli/metavm.py", line 27, in render [translation:ERROR] _OOCall.render(self, generator, op) [translation:ERROR] File "/tmp/pypy/pypy/translator/oosupport/metavm.py", line 456, in render [translation:ERROR] is_primitive = get_primitive_name(callee) [translation:ERROR] File "/tmp/pypy/pypy/translator/oosupport/metavm.py", line 450, in get_primitive_name [translation:ERROR] return sm._name.rsplit('.', 1) [translation:ERROR] File "/tmp/pypy/pypy/rpython/lltypesystem/lltype.py", line 1171, in __getattr__ [translation:ERROR] field_name)) [translation:ERROR] AttributeError: <Func ( ) -> Signed> instance has no field '_name'
To make sure things were working properly on my system I did: # python translate.py --source -O2 with success.
Regards
On Fri, Jun 22, 2012 at 5:31 AM, David Naylor naylor.b.david@gmail.com wrote:
On Sunday, 17 June 2012 09:32:35 Armin Rigo wrote:
It's going to take us a loooooong time to fix CLI translation if the process goes: "you find some minor bug in the release, we attempt to fix it a bit blindly, and then we wait for the next release to know if it worked and what the next minor bug is" :-)
This is true. Would it be possible to do a RC release, or a heads up, so that I may track down these small minor bugs *before* the release is made?
You said that you do not have access to the latest sources, but never explained why. You also did not respond to Armin's question about accessing source from the Bitbucket repository, or downloading source tarballs or the periodic builds from the buildbot (buildbot.pypy.org).
If you want to help fix the CLI backend, why can't you experiment with the development version of PyPy?
Thanks, David
On Friday, 22 June 2012 15:41:04 David Edelsohn wrote:
On Fri, Jun 22, 2012 at 5:31 AM, David Naylor naylor.b.david@gmail.com
wrote:
On Sunday, 17 June 2012 09:32:35 Armin Rigo wrote:
It's going to take us a loooooong time to fix CLI translation if the process goes: "you find some minor bug in the release, we attempt to fix it a bit blindly, and then we wait for the next release to know if it worked and what the next minor bug is" :-)
This is true. Would it be possible to do a RC release, or a heads up, so that I may track down these small minor bugs *before* the release is made?
You said that you do not have access to the latest sources, but never explained why.
It was due to mercurial not being installed on my computer, and only recently have I been able to install more software (hence my 5 day delayed reply to Armin's email).
You also did not respond to Armin's question about accessing source from the Bitbucket repository, or downloading source tarballs or the periodic builds from the buildbot (buildbot.pypy.org).
My original email was based on the downloaded source for pypy-1.9, the follow up email to Armin was based on the bitbucket repository. I don't know how to work mercurial very well but I think `hg log` shows the latest commit was 55751:906bdd8dc432 made "Thu Jun 21 18:22:48 2012 +0200".
If you want to help fix the CLI backend, why can't you experiment with the development version of PyPy?
I have tried the developer version (on the main branch?) and reported back on my results.
I am happy to help pypy, and I am now in a position to do so (on the developer version), however I would like to note that I have no need for the CLI backend and would prefer to not divert energy away from other areas of the project. But, to reiterate, I am happy, willing and able, to help where I can.
Regards
participants (4)
-
Armin Rigo -
David Edelsohn -
David Naylor -
Maciej Fijalkowski