Mailman 3 July 2018 - pypy-dev

Bug report: AST stage crash in PyPy3.5
by hubo 28 Jul '18

28 Jul '18

In PyPy 3.5, the following small piece of code cannot be imported, and causes a SystemError: def test(*a: lambda x: None): pass Stderr: # pypy3 test.py RPython traceback: File "pypy_interpreter.c", line 33969, in BuiltinCode_funcrun_obj File "pypy_module___builtin__.c", line 2705, in compile File "pypy_interpreter.c", line 50963, in PythonAstCompiler__compile_ast File "pypy_interpreter_astcompiler.c", line 1543, in compile_ast File "pypy_interpreter_astcompiler.c", line 5995, in Module_walkabout File "pypy_interpreter_astcompiler.c", line 13362, in PythonCodeGenerator__handle_body File "pypy_interpreter_astcompiler_2.c", line 20473, in PythonCodeGenerator_visit_FunctionDef File "pypy_interpreter_astcompiler_2.c", line 37632, in _visit_function__FunctionCodeGenerator File "pypy_interpreter_astcompiler_2.c", line 61299, in _visit_annotations__pypy_interpreter_astcompiler_1 File "pypy_interpreter_astcompiler_3.c", line 8128, in PythonCodeGenerator__visit_arg_annotation File "pypy_interpreter_astcompiler_2.c", line 22498, in PythonCodeGenerator_visit_Lambda File "pypy_interpreter_astcompiler_2.c", line 35719, in PythonCodeGenerator_sub_scope File "pypy_interpreter_astcompiler.c", line 11371, in PythonCodeGenerator___init__ SystemError: unexpected internal exception (please report a bug): <KeyError object at 0x7efeb2749d20>; internal traceback was dumped to stderr The crash appears when: A varaible argument of a function has an annotation (like *args or **kwargs, normal arguments are not affected), and A lambda expression is used in the annotation Both criterias must be met. This cannot be reproduced on CPython3.5+. Also, Python document says: Parameters may have annotations of the form “: expression” following the parameter name. Any parameter may have an annotation even those of the form *identifier or **identifier. Functions may have “return” annotation of the form “-> expression” after the parameter list. These annotations can be any valid Python expression. The presence of annotations does not change the semantics of a function. The annotation values are available as values of a dictionary keyed by the parameters’ names in the __annotations__ attribute of the function object. Mutiple versions of PyPy 3.5 have this bug, including 5.8, 5.10 and 6.0 2018-07-27 hubo

2 1

Re: [pypy-dev] How to ptrace pypy stack
by Armin Rigo 21 Jul '18

21 Jul '18

Hi, On 21 July 2018 at 17:00, tao he <hetaofirst(a)gmail.com> wrote: > Yes, I have just read the code of https://eng.uber.com/pyflame/ . And I > try to do it with pypy, but not work. So I have to ask for help. This is more complicated to do in PyPy than in CPython. PyPy has got a JIT, and the JIT does not actually create the frame objects that are found not to be needed. This has been done, however, in "vmprof". "vmprof" is similar to what you're trying to do, as far as I can tell, but it is using an in-process timer instead of using "ptrace". It seems to me like it is simpler anyway: I don't understand why you would use ptrace at all, honestly. ptrace can be useful for profiling a random external process, but if the goal is the profile exactly the CPython (or PyPy) interpreter, then you may as well add code directly inside instead of messing around with ptrace introspecting debugging symbols to recontruct the state. A bientôt, Armin.

1 0

How to ptrace pypy stack
by ht 21 Jul '18

21 Jul '18

Dear friends: I'm trying to use ptrace to profile pypy program, but i can't get the stack or virtual memory address more than file no. But python has a global variable named _PyThreadState_Current, that can help to extracting the thread state, so i wander if pypy has some method to do this. thanks!

2 2

[PATCH] FreeBSD build fix
by David CARLIER 18 Jul '18

18 Jul '18

Here a new version of the patch. Kind regards.

2 1

Why does elasticsearch raise an exception on pypy but not cpython?
by Sean Whalen 17 Jul '18

17 Jul '18

I have some code <https://github.com/domainaware/parsedmarc/blob/master/parsedmarc/elastic.py…> that saves data to Elasticsearch. It runs fine in Python 3.5.2 (cpython), but raises an exception when running on pypy 3 6.0.0 (Python 3.5.3). Any ideas why? File "/opt/venvs/parsedmarc/site-packages/parsedmarc/elastic.py", line 366, in save_forensic_report_to_elasticsearch forensic_doc.save()File "/opt/venvs/parsedmarc/site-packages/elasticsearch_dsl/document.py", line 394, in save index=self._get_index(index),File "/opt/venvs/parsedmarc/site-packages/elasticsearch_dsl/document.py", line 138, in _get_index raise ValidationException('You cannot write to a wildcard index.') elasticsearch_dsl.exceptions.ValidationException: You cannot write to a wildcard index I also asked Stack Overflow. You can answer there for internet points. :) https://stackoverflow.com/questions/51368293/why-does-elasticsearch-raise- elasticsearch-dsl-exceptions-validationexception-y

3 3

bug in PyPy3 context __exit__ handling
by Nathaniel Pierce 01 Jul '18

01 Jul '18

Please see attached test. Returning True from __exit__ when type is None can break expected program flow. So far, I've found that each of break/continue/return inside a with block is affected. Both with and without '--jit off' PyPy2 is unaffected. Thanks! -Nate

2 1

Translating of FreeBSD fails for v6.0.0
by David Naylor 01 Jul '18

01 Jul '18

Hi When trying to translate the latest version of PyPy on FreeBSD I get the following error: [translation:info] Error: File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/translator/goal/translate.py", line 318, in main drv.proceed(goals) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/translator/driver.py", line 554, in proceed result = self._execute(goals, task_skip = self._maybe_skip()) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/translator/tool/taskengine.py", line 114, in _execute res = self._do(goal, taskcallable, *args, **kwds) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/translator/driver.py", line 278, in _do res = func() File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/translator/driver.py", line 315, in task_annotate s = annotator.build_types(self.entry_point, self.inputtypes) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/annrpython.py", line 92, in build_types return self.build_graph_types(flowgraph, inputs_s, complete_now=complete_now) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/annrpython.py", line 140, in build_graph_types self.complete() File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/annrpython.py", line 229, in complete self.complete_pending_blocks() File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/annrpython.py", line 224, in complete_pending_blocks self.processblock(graph, block) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/annrpython.py", line 398, in processblock self.flowin(graph, block) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/annrpython.py", line 501, in flowin self.consider_op(op) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/annrpython.py", line 653, in consider_op resultcell = op.consider(self) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/flowspace/operation.py", line 104, in consider return spec(annotator, *self.args) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/unaryop.py", line 118, in simple_call_SomeObject return s_func.call(argspec) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/unaryop.py", line 978, in call return bookkeeper.pbc_call(self, args) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/bookkeeper.py", line 535, in pbc_call results.append(desc.pycall(whence, args, s_previous_result, op)) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/description.py", line 398, in pycall s_result = self.specialize(inputcells, op) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/description.py", line 281, in specialize return self.specializer(self, inputcells) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/specialize.py", line 303, in memo _, _, memotable = memotables.find(firstvalues) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/tool/algo/unionfind.py", line 48, in find info = self.info_factory(obj) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/annotator/specialize.py", line 293, in compute_one_result value = func(*args) File "/home/dbn/ports/ports/lang/pypy/work/pypy2-v6.0.0-src/rpython/rlib/rarithmetic.py", line 154, in _should_widen_type r_class = rffi.platform.numbertype_to_rclass[tp] [translation:ERROR] KeyError: <Void> Processing block: block@28[n_1] is a <class 'rpython.flowspace.flowcontext.SpamBlock'> in (rpython.rlib.rarithmetic:139)widen__NoneType containing the following operations: v630 = simple_call((function typeOf), n_1) v631 = simple_call((function _should_widen_type), v630) v632 = bool(v631) --end-- My best guess is a call to `widen` with a None value is being specialized. But, how does one see more information about the translation back trace (i.e. the context of the specialization of `widen`)? Any, any idea why this is triggering on FreeBSD? Regards

2 1