[pypy-svn] r79386 - in pypy/branch/arm-backend: . dotviewer lib-python lib-python/modified-2.5.2/distutils lib-python/modified-2.5.2/test lib_pypy/_ctypes lib_pypy/ctypes_config_cache lib_pypy/ctypes_config_cache/test lib_pypy/pypy_test pypy pypy/annotation pypy/annotation/test pypy/config pypy/config/test pypy/doc pypy/doc/config pypy/interpreter pypy/interpreter/astcompiler/test pypy/interpreter/pyparser pypy/interpreter/pyparser/test pypy/interpreter/test pypy/jit/backend pypy/jit/backend/arm pypy/jit/backend/llgraph pypy/jit/backend/llsupport pypy/jit/backend/llsupport/test pypy/jit/backend/test pypy/jit/backend/x86 pypy/jit/backend/x86/test pypy/jit/backend/x86/tool pypy/jit/codewriter pypy/jit/codewriter/test pypy/jit/metainterp pypy/jit/metainterp/optimizeopt pypy/jit/metainterp/test pypy/jit/tl pypy/jit/tl/spli pypy/jit/tool pypy/jit/tool/test pypy/module/__builtin__ pypy/module/__builtin__/test pypy/module/__pypy__ pypy/module/__pypy__/test pypy/module/_bisect pypy/module/_bisect/test pypy/module/_ffi pypy/module/_ffi/test pypy/module/_rawffi pypy/module/_rawffi/test pypy/module/_socket/test pypy/module/_sre pypy/module/_stackless/test pypy/module/_weakref pypy/module/_winreg pypy/module/array pypy/module/array/benchmark pypy/module/array/test pypy/module/bz2 pypy/module/cpyext pypy/module/cpyext/test pypy/module/gc pypy/module/gc/test pypy/module/imp pypy/module/imp/test pypy/module/parser/test pypy/module/posix pypy/module/pypyjit pypy/module/pypyjit/test pypy/module/rctime pypy/module/signal pypy/module/signal/test pypy/module/sys pypy/module/sys/test pypy/module/test_lib_pypy/ctypes_tests pypy/module/thread pypy/module/thread/test pypy/objspace pypy/objspace/flow pypy/objspace/flow/test pypy/objspace/std pypy/objspace/std/test pypy/objspace/test pypy/rlib pypy/rlib/rsre pypy/rlib/rsre/test pypy/rlib/test pypy/rpython pypy/rpython/lltypesystem pypy/rpython/lltypesystem/test pypy/rpython/memory pypy/rpython/memory/gc pypy/rpython/memory/gc/test pypy/rpython/memory/gctransform pypy/rpython/memory/test pypy/rpython/module pypy/rpython/test pypy/tool pypy/tool/release pypy/tool/release/test pypy/tool/test pypy/translator pypy/translator/c pypy/translator/c/gcc pypy/translator/c/gcc/test pypy/translator/c/gcc/test/darwin64 pypy/translator/c/gcc/test/elf pypy/translator/c/gcc/test/elf64 pypy/translator/c/src pypy/translator/c/test pypy/translator/cli/src pypy/translator/goal pypy/translator/goal/test2 pypy/translator/jvm/test pypy/translator/oosupport/test_template pypy/translator/platform
david at codespeak.net
david at codespeak.net
Tue Nov 23 10:13:07 CET 2010
Author: david
Date: Tue Nov 23 10:12:47 2010
New Revision: 79386
Added:
pypy/branch/arm-backend/lib-python/modified-2.5.2/test/test_dict.py
- copied unchanged from r79385, pypy/trunk/lib-python/modified-2.5.2/test/test_dict.py
pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/dumpcache.py
- copied unchanged from r79385, pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py
pypy/branch/arm-backend/pypy/doc/config/objspace.soabi.txt
- copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.soabi.txt
pypy/branch/arm-backend/pypy/doc/config/objspace.std.withmapdict.txt
- copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.std.withmapdict.txt
pypy/branch/arm-backend/pypy/doc/config/objspace.std.withstrbuf.txt
- copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.std.withstrbuf.txt
pypy/branch/arm-backend/pypy/doc/config/objspace.translationmodules.txt
- copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.translationmodules.txt
pypy/branch/arm-backend/pypy/doc/config/objspace.usemodules._bisect.txt
- copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.usemodules._bisect.txt
pypy/branch/arm-backend/pypy/doc/config/objspace.usemodules._ffi.txt
- copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.usemodules._ffi.txt
pypy/branch/arm-backend/pypy/doc/release-1.4.0.txt
- copied unchanged from r79385, pypy/trunk/pypy/doc/release-1.4.0.txt
pypy/branch/arm-backend/pypy/doc/release-1.4.0beta.txt
- copied unchanged from r79385, pypy/trunk/pypy/doc/release-1.4.0beta.txt
pypy/branch/arm-backend/pypy/interpreter/pyparser/autopath.py
- copied unchanged from r79385, pypy/trunk/pypy/interpreter/pyparser/autopath.py
pypy/branch/arm-backend/pypy/interpreter/pyparser/genpytokenize.py
- copied unchanged from r79385, pypy/trunk/pypy/interpreter/pyparser/genpytokenize.py
pypy/branch/arm-backend/pypy/interpreter/pyparser/pylexer.py
- copied unchanged from r79385, pypy/trunk/pypy/interpreter/pyparser/pylexer.py
pypy/branch/arm-backend/pypy/jit/backend/conftest.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/backend/conftest.py
pypy/branch/arm-backend/pypy/jit/backend/llsupport/ffisupport.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/backend/llsupport/ffisupport.py
pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_ffisupport.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/backend/llsupport/test/test_ffisupport.py
pypy/branch/arm-backend/pypy/jit/codewriter/test/test_void_list.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/codewriter/test/test_void_list.py
pypy/branch/arm-backend/pypy/jit/metainterp/greenfield.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/metainterp/greenfield.py
pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/fficall.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_fficall.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/metainterp/test/test_fficall.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_greenfield.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/metainterp/test/test_greenfield.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizefficall.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/metainterp/test/test_optimizefficall.py
pypy/branch/arm-backend/pypy/jit/tl/jittest.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/tl/jittest.py
pypy/branch/arm-backend/pypy/jit/tool/log-template.gnumeric
- copied unchanged from r79385, pypy/trunk/pypy/jit/tool/log-template.gnumeric
pypy/branch/arm-backend/pypy/jit/tool/log2gnumeric.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/tool/log2gnumeric.py
pypy/branch/arm-backend/pypy/jit/tool/loopcounter.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/tool/loopcounter.py
pypy/branch/arm-backend/pypy/jit/tool/oparser.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/tool/oparser.py
pypy/branch/arm-backend/pypy/jit/tool/pypytrace.vim
- copied unchanged from r79385, pypy/trunk/pypy/jit/tool/pypytrace.vim
pypy/branch/arm-backend/pypy/jit/tool/test/test_loopcounter.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/tool/test/test_loopcounter.py
pypy/branch/arm-backend/pypy/jit/tool/test/test_oparser.py
- copied unchanged from r79385, pypy/trunk/pypy/jit/tool/test/test_oparser.py
pypy/branch/arm-backend/pypy/module/_bisect/ (props changed)
- copied from r79385, pypy/trunk/pypy/module/_bisect/
pypy/branch/arm-backend/pypy/module/_ffi/ (props changed)
- copied from r79385, pypy/trunk/pypy/module/_ffi/
pypy/branch/arm-backend/pypy/module/_stackless/test/conftest.py
- copied unchanged from r79385, pypy/trunk/pypy/module/_stackless/test/conftest.py
pypy/branch/arm-backend/pypy/objspace/std/mapdict.py
- copied unchanged from r79385, pypy/trunk/pypy/objspace/std/mapdict.py
pypy/branch/arm-backend/pypy/objspace/std/strbufobject.py
- copied unchanged from r79385, pypy/trunk/pypy/objspace/std/strbufobject.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_mapdict.py
- copied unchanged from r79385, pypy/trunk/pypy/objspace/std/test/test_mapdict.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_methodcache.py
- copied unchanged from r79385, pypy/trunk/pypy/objspace/std/test/test_methodcache.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_strbufobject.py
- copied unchanged from r79385, pypy/trunk/pypy/objspace/std/test/test_strbufobject.py
pypy/branch/arm-backend/pypy/rlib/clibffi.py
- copied unchanged from r79385, pypy/trunk/pypy/rlib/clibffi.py
pypy/branch/arm-backend/pypy/rlib/libffi.py
- copied unchanged from r79385, pypy/trunk/pypy/rlib/libffi.py
pypy/branch/arm-backend/pypy/rlib/rerased.py
- copied unchanged from r79385, pypy/trunk/pypy/rlib/rerased.py
pypy/branch/arm-backend/pypy/rlib/rsre/rsre_jit.py
- copied unchanged from r79385, pypy/trunk/pypy/rlib/rsre/rsre_jit.py
pypy/branch/arm-backend/pypy/rlib/rsre/test/conftest.py
- copied unchanged from r79385, pypy/trunk/pypy/rlib/rsre/test/conftest.py
pypy/branch/arm-backend/pypy/rlib/rsre/test/test_zjit.py
- copied unchanged from r79385, pypy/trunk/pypy/rlib/rsre/test/test_zjit.py
pypy/branch/arm-backend/pypy/rlib/test/test_clibffi.py
- copied unchanged from r79385, pypy/trunk/pypy/rlib/test/test_clibffi.py
pypy/branch/arm-backend/pypy/rlib/test/test_libffi.py
- copied unchanged from r79385, pypy/trunk/pypy/rlib/test/test_libffi.py
pypy/branch/arm-backend/pypy/rlib/test/test_rerased.py
- copied unchanged from r79385, pypy/trunk/pypy/rlib/test/test_rerased.py
pypy/branch/arm-backend/pypy/tool/gcdump.py
- copied unchanged from r79385, pypy/trunk/pypy/tool/gcdump.py
pypy/branch/arm-backend/pypy/tool/leakfinder.py
- copied unchanged from r79385, pypy/trunk/pypy/tool/leakfinder.py
pypy/branch/arm-backend/pypy/tool/test/test_leakfinder.py
- copied unchanged from r79385, pypy/trunk/pypy/tool/test/test_leakfinder.py
pypy/branch/arm-backend/pypy/translator/c/gcc/test/darwin64/
- copied from r79385, pypy/trunk/pypy/translator/c/gcc/test/darwin64/
pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track10.s
- copied unchanged from r79385, pypy/trunk/pypy/translator/c/gcc/test/elf/track10.s
pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track11.s
- copied unchanged from r79385, pypy/trunk/pypy/translator/c/gcc/test/elf/track11.s
pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf64/track_loadconst.s
- copied unchanged from r79385, pypy/trunk/pypy/translator/c/gcc/test/elf64/track_loadconst.s
pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf64/track_rpyassertfailed.s
- copied unchanged from r79385, pypy/trunk/pypy/translator/c/gcc/test/elf64/track_rpyassertfailed.s
pypy/branch/arm-backend/pypy/translator/c/src/asm_gcc_x86_64.h
- copied unchanged from r79385, pypy/trunk/pypy/translator/c/src/asm_gcc_x86_64.h
pypy/branch/arm-backend/pypy/translator/c/src/debug_alloc.h
- copied unchanged from r79385, pypy/trunk/pypy/translator/c/src/debug_alloc.h
Removed:
pypy/branch/arm-backend/pypy/doc/config/objspace.std.withinlineddict.txt
pypy/branch/arm-backend/pypy/doc/config/objspace.std.withshadowtracking.txt
pypy/branch/arm-backend/pypy/doc/config/objspace.std.withsharingdict.txt
pypy/branch/arm-backend/pypy/jit/backend/test/conftest.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/oparser.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_oparser.py
pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_find.py
pypy/branch/arm-backend/pypy/objspace/std/inlinedict.py
pypy/branch/arm-backend/pypy/objspace/std/sharingdict.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_inlinedict.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_shadowtracking.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_sharingdict.py
pypy/branch/arm-backend/pypy/rpython/rspecialcase.py
pypy/branch/arm-backend/pypy/rpython/test/test_rspecialcase.py
Modified:
pypy/branch/arm-backend/ (props changed)
pypy/branch/arm-backend/dotviewer/drawgraph.py
pypy/branch/arm-backend/lib-python/conftest.py
pypy/branch/arm-backend/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py
pypy/branch/arm-backend/lib-python/modified-2.5.2/test/mapping_tests.py
pypy/branch/arm-backend/lib-python/modified-2.5.2/test/test_descr.py
pypy/branch/arm-backend/lib_pypy/_ctypes/function.py
pypy/branch/arm-backend/lib_pypy/_ctypes/structure.py
pypy/branch/arm-backend/lib_pypy/_ctypes/union.py
pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/ (props changed)
pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/hashlib.ctc.py
pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/locale.ctc.py
pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/pyexpat.ctc.py
pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/rebuild.py
pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/resource.ctc.py
pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/syslog.ctc.py
pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/test/test_cache.py
pypy/branch/arm-backend/lib_pypy/pypy_test/test_ctypes_support.py
pypy/branch/arm-backend/pypy/ (props changed)
pypy/branch/arm-backend/pypy/annotation/bookkeeper.py
pypy/branch/arm-backend/pypy/annotation/builtin.py
pypy/branch/arm-backend/pypy/annotation/classdef.py
pypy/branch/arm-backend/pypy/annotation/description.py
pypy/branch/arm-backend/pypy/annotation/dictdef.py
pypy/branch/arm-backend/pypy/annotation/listdef.py
pypy/branch/arm-backend/pypy/annotation/model.py
pypy/branch/arm-backend/pypy/annotation/policy.py
pypy/branch/arm-backend/pypy/annotation/specialize.py
pypy/branch/arm-backend/pypy/annotation/test/test_annrpython.py
pypy/branch/arm-backend/pypy/annotation/test/test_model.py
pypy/branch/arm-backend/pypy/annotation/unaryop.py
pypy/branch/arm-backend/pypy/config/pypyoption.py
pypy/branch/arm-backend/pypy/config/test/test_pypyoption.py
pypy/branch/arm-backend/pypy/config/translationoption.py
pypy/branch/arm-backend/pypy/conftest.py
pypy/branch/arm-backend/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt
pypy/branch/arm-backend/pypy/doc/docindex.txt
pypy/branch/arm-backend/pypy/doc/getting-started.txt
pypy/branch/arm-backend/pypy/doc/interpreter-optimizations.txt
pypy/branch/arm-backend/pypy/interpreter/argument.py
pypy/branch/arm-backend/pypy/interpreter/astcompiler/test/test_compiler.py
pypy/branch/arm-backend/pypy/interpreter/baseobjspace.py
pypy/branch/arm-backend/pypy/interpreter/executioncontext.py
pypy/branch/arm-backend/pypy/interpreter/function.py
pypy/branch/arm-backend/pypy/interpreter/gateway.py
pypy/branch/arm-backend/pypy/interpreter/pycode.py
pypy/branch/arm-backend/pypy/interpreter/pyframe.py
pypy/branch/arm-backend/pypy/interpreter/pyopcode.py
pypy/branch/arm-backend/pypy/interpreter/pyparser/pytokenize.py
pypy/branch/arm-backend/pypy/interpreter/pyparser/test/test_pyparse.py
pypy/branch/arm-backend/pypy/interpreter/test/test_argument.py
pypy/branch/arm-backend/pypy/interpreter/test/test_executioncontext.py
pypy/branch/arm-backend/pypy/interpreter/test/test_function.py
pypy/branch/arm-backend/pypy/interpreter/typedef.py
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py
pypy/branch/arm-backend/pypy/jit/backend/llgraph/llimpl.py
pypy/branch/arm-backend/pypy/jit/backend/llgraph/runner.py
pypy/branch/arm-backend/pypy/jit/backend/llsupport/descr.py
pypy/branch/arm-backend/pypy/jit/backend/llsupport/gc.py
pypy/branch/arm-backend/pypy/jit/backend/llsupport/llmodel.py
pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py
pypy/branch/arm-backend/pypy/jit/backend/llsupport/symbolic.py
pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_descr.py
pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_gc.py
pypy/branch/arm-backend/pypy/jit/backend/model.py
pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py
pypy/branch/arm-backend/pypy/jit/backend/test/test_ll_random.py
pypy/branch/arm-backend/pypy/jit/backend/test/test_random.py
pypy/branch/arm-backend/pypy/jit/backend/x86/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py
pypy/branch/arm-backend/pypy/jit/backend/x86/regloc.py
pypy/branch/arm-backend/pypy/jit/backend/x86/runner.py
pypy/branch/arm-backend/pypy/jit/backend/x86/rx86.py
pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_assembler.py
pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_gc_integration.py
pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_regalloc.py
pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_runner.py
pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_string.py
pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_zrpy_gc.py
pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_ztranslation.py
pypy/branch/arm-backend/pypy/jit/backend/x86/tool/viewcode.py
pypy/branch/arm-backend/pypy/jit/codewriter/assembler.py
pypy/branch/arm-backend/pypy/jit/codewriter/call.py
pypy/branch/arm-backend/pypy/jit/codewriter/codewriter.py
pypy/branch/arm-backend/pypy/jit/codewriter/effectinfo.py
pypy/branch/arm-backend/pypy/jit/codewriter/format.py
pypy/branch/arm-backend/pypy/jit/codewriter/jtransform.py
pypy/branch/arm-backend/pypy/jit/codewriter/support.py
pypy/branch/arm-backend/pypy/jit/codewriter/test/test_codewriter.py
pypy/branch/arm-backend/pypy/jit/codewriter/test/test_effectinfo.py
pypy/branch/arm-backend/pypy/jit/codewriter/test/test_jtransform.py
pypy/branch/arm-backend/pypy/jit/codewriter/test/test_list.py
pypy/branch/arm-backend/pypy/jit/metainterp/blackhole.py
pypy/branch/arm-backend/pypy/jit/metainterp/compile.py
pypy/branch/arm-backend/pypy/jit/metainterp/executor.py
pypy/branch/arm-backend/pypy/jit/metainterp/graphpage.py
pypy/branch/arm-backend/pypy/jit/metainterp/history.py
pypy/branch/arm-backend/pypy/jit/metainterp/jitdriver.py
pypy/branch/arm-backend/pypy/jit/metainterp/logger.py
pypy/branch/arm-backend/pypy/jit/metainterp/optimize_nopspec.py (contents, props changed)
pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/intbounds.py
pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/optimizer.py (contents, props changed)
pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/rewrite.py
pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/string.py
pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/branch/arm-backend/pypy/jit/metainterp/pyjitpl.py
pypy/branch/arm-backend/pypy/jit/metainterp/resoperation.py
pypy/branch/arm-backend/pypy/jit/metainterp/resume.py
pypy/branch/arm-backend/pypy/jit/metainterp/simple_optimize.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_basic.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_compile.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_del.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_exception.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_jitdriver.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_logger.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_loop_nopspec.py (props changed)
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizefindnode.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_recursive.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resoperation.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resume.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_string.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_virtualref.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_warmspot.py
pypy/branch/arm-backend/pypy/jit/metainterp/test/test_ztranslation.py
pypy/branch/arm-backend/pypy/jit/metainterp/virtualizable.py
pypy/branch/arm-backend/pypy/jit/metainterp/virtualref.py
pypy/branch/arm-backend/pypy/jit/metainterp/warmspot.py
pypy/branch/arm-backend/pypy/jit/metainterp/warmstate.py
pypy/branch/arm-backend/pypy/jit/tl/pypyjit.py
pypy/branch/arm-backend/pypy/jit/tl/pypyjit_child.py
pypy/branch/arm-backend/pypy/jit/tl/spli/interpreter.py
pypy/branch/arm-backend/pypy/jit/tool/loopviewer.py
pypy/branch/arm-backend/pypy/jit/tool/pypytrace-mode.el
pypy/branch/arm-backend/pypy/jit/tool/showstats.py
pypy/branch/arm-backend/pypy/jit/tool/test/test_traceviewer.py
pypy/branch/arm-backend/pypy/jit/tool/traceviewer.py
pypy/branch/arm-backend/pypy/module/__builtin__/descriptor.py
pypy/branch/arm-backend/pypy/module/__builtin__/interp_classobj.py
pypy/branch/arm-backend/pypy/module/__builtin__/test/test_classobj.py
pypy/branch/arm-backend/pypy/module/__pypy__/__init__.py
pypy/branch/arm-backend/pypy/module/__pypy__/interp_magic.py
pypy/branch/arm-backend/pypy/module/__pypy__/test/test_special.py
pypy/branch/arm-backend/pypy/module/_bisect/test/ (props changed)
pypy/branch/arm-backend/pypy/module/_ffi/test/ (props changed)
pypy/branch/arm-backend/pypy/module/_rawffi/__init__.py
pypy/branch/arm-backend/pypy/module/_rawffi/array.py
pypy/branch/arm-backend/pypy/module/_rawffi/callback.py
pypy/branch/arm-backend/pypy/module/_rawffi/interp_rawffi.py
pypy/branch/arm-backend/pypy/module/_rawffi/structure.py
pypy/branch/arm-backend/pypy/module/_rawffi/test/test__rawffi.py
pypy/branch/arm-backend/pypy/module/_rawffi/test/test_nested.py
pypy/branch/arm-backend/pypy/module/_socket/test/test_sock_app.py
pypy/branch/arm-backend/pypy/module/_sre/interp_sre.py
pypy/branch/arm-backend/pypy/module/_weakref/interp__weakref.py
pypy/branch/arm-backend/pypy/module/_winreg/interp_winreg.py
pypy/branch/arm-backend/pypy/module/array/benchmark/Makefile (props changed)
pypy/branch/arm-backend/pypy/module/array/benchmark/intimg.c (props changed)
pypy/branch/arm-backend/pypy/module/array/benchmark/intimgtst.c (props changed)
pypy/branch/arm-backend/pypy/module/array/benchmark/intimgtst.py (props changed)
pypy/branch/arm-backend/pypy/module/array/benchmark/loop.c (props changed)
pypy/branch/arm-backend/pypy/module/array/benchmark/sum.c (props changed)
pypy/branch/arm-backend/pypy/module/array/benchmark/sumtst.c (props changed)
pypy/branch/arm-backend/pypy/module/array/benchmark/sumtst.py (props changed)
pypy/branch/arm-backend/pypy/module/array/interp_array.py
pypy/branch/arm-backend/pypy/module/array/test/test_array_old.py (props changed)
pypy/branch/arm-backend/pypy/module/bz2/interp_bz2.py
pypy/branch/arm-backend/pypy/module/cpyext/api.py
pypy/branch/arm-backend/pypy/module/cpyext/cdatetime.py
pypy/branch/arm-backend/pypy/module/cpyext/classobject.py
pypy/branch/arm-backend/pypy/module/cpyext/presetup.py
pypy/branch/arm-backend/pypy/module/cpyext/pyobject.py
pypy/branch/arm-backend/pypy/module/cpyext/test/test_borrow.py
pypy/branch/arm-backend/pypy/module/cpyext/test/test_cpyext.py
pypy/branch/arm-backend/pypy/module/cpyext/test/test_unicodeobject.py
pypy/branch/arm-backend/pypy/module/cpyext/typeobject.py
pypy/branch/arm-backend/pypy/module/gc/__init__.py
pypy/branch/arm-backend/pypy/module/gc/app_referents.py
pypy/branch/arm-backend/pypy/module/gc/interp_gc.py
pypy/branch/arm-backend/pypy/module/gc/referents.py
pypy/branch/arm-backend/pypy/module/gc/test/test_gc.py
pypy/branch/arm-backend/pypy/module/imp/importing.py
pypy/branch/arm-backend/pypy/module/imp/interp_imp.py
pypy/branch/arm-backend/pypy/module/imp/test/test_app.py
pypy/branch/arm-backend/pypy/module/imp/test/test_import.py
pypy/branch/arm-backend/pypy/module/parser/test/test_parser.py
pypy/branch/arm-backend/pypy/module/posix/interp_posix.py
pypy/branch/arm-backend/pypy/module/pypyjit/__init__.py
pypy/branch/arm-backend/pypy/module/pypyjit/interp_jit.py
pypy/branch/arm-backend/pypy/module/pypyjit/policy.py
pypy/branch/arm-backend/pypy/module/pypyjit/test/test_policy.py
pypy/branch/arm-backend/pypy/module/pypyjit/test/test_pypy_c.py
pypy/branch/arm-backend/pypy/module/rctime/interp_time.py
pypy/branch/arm-backend/pypy/module/signal/__init__.py
pypy/branch/arm-backend/pypy/module/signal/interp_signal.py
pypy/branch/arm-backend/pypy/module/signal/test/test_signal.py
pypy/branch/arm-backend/pypy/module/sys/__init__.py
pypy/branch/arm-backend/pypy/module/sys/state.py
pypy/branch/arm-backend/pypy/module/sys/test/test_initialpath.py
pypy/branch/arm-backend/pypy/module/sys/version.py
pypy/branch/arm-backend/pypy/module/sys/vm.py
pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py
pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py
pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py
pypy/branch/arm-backend/pypy/module/thread/gil.py
pypy/branch/arm-backend/pypy/module/thread/ll_thread.py
pypy/branch/arm-backend/pypy/module/thread/test/test_gil.py
pypy/branch/arm-backend/pypy/objspace/descroperation.py
pypy/branch/arm-backend/pypy/objspace/flow/flowcontext.py
pypy/branch/arm-backend/pypy/objspace/flow/model.py
pypy/branch/arm-backend/pypy/objspace/flow/objspace.py
pypy/branch/arm-backend/pypy/objspace/flow/test/test_objspace.py
pypy/branch/arm-backend/pypy/objspace/std/callmethod.py
pypy/branch/arm-backend/pypy/objspace/std/celldict.py
pypy/branch/arm-backend/pypy/objspace/std/dictmultiobject.py
pypy/branch/arm-backend/pypy/objspace/std/dicttype.py
pypy/branch/arm-backend/pypy/objspace/std/fake.py
pypy/branch/arm-backend/pypy/objspace/std/frozensettype.py
pypy/branch/arm-backend/pypy/objspace/std/inttype.py
pypy/branch/arm-backend/pypy/objspace/std/listobject.py
pypy/branch/arm-backend/pypy/objspace/std/longtype.py
pypy/branch/arm-backend/pypy/objspace/std/model.py
pypy/branch/arm-backend/pypy/objspace/std/objspace.py
pypy/branch/arm-backend/pypy/objspace/std/proxyobject.py
pypy/branch/arm-backend/pypy/objspace/std/setobject.py
pypy/branch/arm-backend/pypy/objspace/std/settype.py
pypy/branch/arm-backend/pypy/objspace/std/smallintobject.py
pypy/branch/arm-backend/pypy/objspace/std/stringobject.py
pypy/branch/arm-backend/pypy/objspace/std/stringtype.py
pypy/branch/arm-backend/pypy/objspace/std/strutil.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_dictmultiobject.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_longobject.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_setobject.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_strutil.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_typeobject.py
pypy/branch/arm-backend/pypy/objspace/std/test/test_versionedtype.py
pypy/branch/arm-backend/pypy/objspace/std/typeobject.py
pypy/branch/arm-backend/pypy/objspace/test/test_descroperation.py
pypy/branch/arm-backend/pypy/rlib/_rsocket_rffi.py
pypy/branch/arm-backend/pypy/rlib/jit.py
pypy/branch/arm-backend/pypy/rlib/objectmodel.py
pypy/branch/arm-backend/pypy/rlib/rarithmetic.py
pypy/branch/arm-backend/pypy/rlib/rbigint.py
pypy/branch/arm-backend/pypy/rlib/rgc.py
pypy/branch/arm-backend/pypy/rlib/rmmap.py
pypy/branch/arm-backend/pypy/rlib/rsre/rsre_char.py
pypy/branch/arm-backend/pypy/rlib/rsre/rsre_core.py
pypy/branch/arm-backend/pypy/rlib/rsre/test/test_match.py
pypy/branch/arm-backend/pypy/rlib/rstring.py
pypy/branch/arm-backend/pypy/rlib/streamio.py
pypy/branch/arm-backend/pypy/rlib/test/test_jit.py
pypy/branch/arm-backend/pypy/rlib/test/test_rarithmetic.py
pypy/branch/arm-backend/pypy/rlib/test/test_rbigint.py
pypy/branch/arm-backend/pypy/rlib/test/test_rdynload.py
pypy/branch/arm-backend/pypy/rlib/test/test_rsocket.py
pypy/branch/arm-backend/pypy/rlib/test/test_rstring.py
pypy/branch/arm-backend/pypy/rlib/test/test_runicode.py
pypy/branch/arm-backend/pypy/rlib/test/test_rzlib.py
pypy/branch/arm-backend/pypy/rpython/annlowlevel.py
pypy/branch/arm-backend/pypy/rpython/llinterp.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/llarena.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/lloperation.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/lltype.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/rbuilder.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/rclass.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/rdict.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/rffi.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_llmemory.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_lltype.py
pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_rffi.py
pypy/branch/arm-backend/pypy/rpython/memory/gc/base.py
pypy/branch/arm-backend/pypy/rpython/memory/gc/generation.py
pypy/branch/arm-backend/pypy/rpython/memory/gc/inspector.py
pypy/branch/arm-backend/pypy/rpython/memory/gc/markcompact.py
pypy/branch/arm-backend/pypy/rpython/memory/gc/marksweep.py
pypy/branch/arm-backend/pypy/rpython/memory/gc/minimark.py
pypy/branch/arm-backend/pypy/rpython/memory/gc/minimarkpage.py
pypy/branch/arm-backend/pypy/rpython/memory/gc/semispace.py
pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_direct.py
pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_minimark.py
pypy/branch/arm-backend/pypy/rpython/memory/gctransform/asmgcroot.py
pypy/branch/arm-backend/pypy/rpython/memory/gctransform/framework.py
pypy/branch/arm-backend/pypy/rpython/memory/gctransform/transform.py
pypy/branch/arm-backend/pypy/rpython/memory/gcwrapper.py
pypy/branch/arm-backend/pypy/rpython/memory/support.py
pypy/branch/arm-backend/pypy/rpython/memory/test/test_gc.py
pypy/branch/arm-backend/pypy/rpython/memory/test/test_transformed_gc.py
pypy/branch/arm-backend/pypy/rpython/module/ll_time.py
pypy/branch/arm-backend/pypy/rpython/rbuilder.py
pypy/branch/arm-backend/pypy/rpython/rbuiltin.py
pypy/branch/arm-backend/pypy/rpython/rmodel.py
pypy/branch/arm-backend/pypy/rpython/rpbc.py
pypy/branch/arm-backend/pypy/rpython/rtyper.py
pypy/branch/arm-backend/pypy/rpython/test/test_llinterp.py
pypy/branch/arm-backend/pypy/rpython/test/test_nongc.py
pypy/branch/arm-backend/pypy/rpython/test/test_rbuilder.py
pypy/branch/arm-backend/pypy/rpython/test/test_rclass.py
pypy/branch/arm-backend/pypy/rpython/test/test_rdict.py
pypy/branch/arm-backend/pypy/rpython/test/test_rpbc.py
pypy/branch/arm-backend/pypy/rpython/test/test_rptr.py
pypy/branch/arm-backend/pypy/tool/alarm.py
pypy/branch/arm-backend/pypy/tool/readdictinfo.py
pypy/branch/arm-backend/pypy/tool/release/ (props changed)
pypy/branch/arm-backend/pypy/tool/release/__init__.py (props changed)
pypy/branch/arm-backend/pypy/tool/release/force-builds.py (contents, props changed)
pypy/branch/arm-backend/pypy/tool/release/make_release.py (contents, props changed)
pypy/branch/arm-backend/pypy/tool/release/package.py
pypy/branch/arm-backend/pypy/tool/release/test/ (props changed)
pypy/branch/arm-backend/pypy/tool/release/test/__init__.py (props changed)
pypy/branch/arm-backend/pypy/tool/release/test/test_make_release.py (props changed)
pypy/branch/arm-backend/pypy/tool/release/test/test_package.py
pypy/branch/arm-backend/pypy/tool/rundictbenchmarks.py
pypy/branch/arm-backend/pypy/tool/statistic_irc_log.py
pypy/branch/arm-backend/pypy/tool/terminal.py
pypy/branch/arm-backend/pypy/tool/watchdog.py
pypy/branch/arm-backend/pypy/tool/watchdog_nt.py
pypy/branch/arm-backend/pypy/translator/c/funcgen.py
pypy/branch/arm-backend/pypy/translator/c/gcc/instruction.py
pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track5.s
pypy/branch/arm-backend/pypy/translator/c/gcc/test/test_trackgcroot.py
pypy/branch/arm-backend/pypy/translator/c/gcc/trackgcroot.py
pypy/branch/arm-backend/pypy/translator/c/genc.py
pypy/branch/arm-backend/pypy/translator/c/node.py
pypy/branch/arm-backend/pypy/translator/c/src/asm_gcc_x86.h
pypy/branch/arm-backend/pypy/translator/c/src/debug_print.h
pypy/branch/arm-backend/pypy/translator/c/src/g_include.h
pypy/branch/arm-backend/pypy/translator/c/src/int.h
pypy/branch/arm-backend/pypy/translator/c/src/main.h
pypy/branch/arm-backend/pypy/translator/c/src/signals.h
pypy/branch/arm-backend/pypy/translator/c/src/stack.h
pypy/branch/arm-backend/pypy/translator/c/test/test_genc.py
pypy/branch/arm-backend/pypy/translator/c/test/test_lltyped.py
pypy/branch/arm-backend/pypy/translator/c/test/test_newgc.py
pypy/branch/arm-backend/pypy/translator/c/test/test_standalone.py
pypy/branch/arm-backend/pypy/translator/c/test/test_typed.py
pypy/branch/arm-backend/pypy/translator/cli/src/pypylib.cs
pypy/branch/arm-backend/pypy/translator/driver.py
pypy/branch/arm-backend/pypy/translator/goal/ann_override.py
pypy/branch/arm-backend/pypy/translator/goal/app_main.py
pypy/branch/arm-backend/pypy/translator/goal/targetpypystandalone.py
pypy/branch/arm-backend/pypy/translator/goal/test2/test_app_main.py
pypy/branch/arm-backend/pypy/translator/goal/translate.py
pypy/branch/arm-backend/pypy/translator/jvm/test/test_class.py
pypy/branch/arm-backend/pypy/translator/oosupport/test_template/class_.py
pypy/branch/arm-backend/pypy/translator/platform/darwin.py
pypy/branch/arm-backend/pypy/translator/platform/linux.py
pypy/branch/arm-backend/pypy/translator/platform/posix.py
pypy/branch/arm-backend/pypy/translator/transform.py
Log:
merged trunk changes r77502 through r79385 into arm backend branch
Modified: pypy/branch/arm-backend/dotviewer/drawgraph.py
==============================================================================
--- pypy/branch/arm-backend/dotviewer/drawgraph.py (original)
+++ pypy/branch/arm-backend/dotviewer/drawgraph.py Tue Nov 23 10:12:47 2010
@@ -423,20 +423,43 @@
else:
for line in lines:
raw_line = line.replace('\\l','').replace('\r','') or ' '
- img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
- w, h = img.get_size()
- if w>wmax: wmax = w
- if raw_line.strip():
- if line.endswith('\\l'):
- def cmd(img=img, y=hmax):
- img.draw(xleft, ytop+y)
- elif line.endswith('\r'):
- def cmd(img=img, y=hmax, w=w):
- img.draw(xright-w, ytop+y)
- else:
- def cmd(img=img, y=hmax, w=w):
- img.draw(xcenter-w//2, ytop+y)
+ if '\f' in raw_line: # grayed out parts of the line
+ imgs = []
+ graytext = True
+ h = 16
+ w_total = 0
+ for linepart in raw_line.split('\f'):
+ graytext = not graytext
+ if not linepart.strip():
+ continue
+ if graytext:
+ fgcolor = (128, 160, 160)
+ else:
+ fgcolor = (0, 0, 0)
+ img = TextSnippet(self, linepart, fgcolor, bgcolor)
+ imgs.append((w_total, img))
+ w, h = img.get_size()
+ w_total += w
+ if w_total > wmax: wmax = w_total
+ def cmd(imgs=imgs, y=hmax):
+ for x, img in imgs:
+ img.draw(xleft+x, ytop+y)
commands.append(cmd)
+ else:
+ img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
+ w, h = img.get_size()
+ if w>wmax: wmax = w
+ if raw_line.strip():
+ if line.endswith('\\l'):
+ def cmd(img=img, y=hmax):
+ img.draw(xleft, ytop+y)
+ elif line.endswith('\r'):
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xright-w, ytop+y)
+ else:
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xcenter-w//2, ytop+y)
+ commands.append(cmd)
hmax += h
#hmax += 8
Modified: pypy/branch/arm-backend/lib-python/conftest.py
==============================================================================
--- pypy/branch/arm-backend/lib-python/conftest.py (original)
+++ pypy/branch/arm-backend/lib-python/conftest.py Tue Nov 23 10:12:47 2010
@@ -144,7 +144,7 @@
RegrTest('test_binhex.py'),
RegrTest('test_binop.py', core=True),
- RegrTest('test_bisect.py', core=True),
+ RegrTest('test_bisect.py', core=True, usemodules='_bisect'),
RegrTest('test_bool.py', core=True),
RegrTest('test_bsddb.py', skip="unsupported extension module"),
RegrTest('test_bsddb185.py', skip="unsupported extension module"),
@@ -464,7 +464,12 @@
RegrTest('test_coding.py'),
RegrTest('test_complex_args.py'),
RegrTest('test_contextlib.py', usemodules="thread"),
- RegrTest('test_ctypes.py', usemodules="_rawffi"),
+ # we skip test ctypes, since we adapted it massively in order
+ # to test what we want to support. There are real failures,
+ # but it's about missing features that we don't want to support
+ # now
+ RegrTest('test_ctypes.py', usemodules="_rawffi",
+ skip="missing features that we don't want to support now"),
RegrTest('test_defaultdict.py'),
RegrTest('test_email_renamed.py'),
RegrTest('test_exception_variations.py'),
Modified: pypy/branch/arm-backend/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py
==============================================================================
--- pypy/branch/arm-backend/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py (original)
+++ pypy/branch/arm-backend/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py Tue Nov 23 10:12:47 2010
@@ -3,6 +3,7 @@
import sys
import os
+import imp
from distutils.errors import DistutilsPlatformError
@@ -47,11 +48,17 @@
_config_vars = None
+def _get_so_extension():
+ for ext, mod, typ in imp.get_suffixes():
+ if typ == imp.C_EXTENSION:
+ return ext
+
def _init_posix():
"""Initialize the module as appropriate for POSIX systems."""
g = {}
g['EXE'] = ""
- g['SO'] = ".so"
+ g['SO'] = _get_so_extension() or ".so"
+ g['SOABI'] = g['SO'].rsplit('.')[0]
global _config_vars
_config_vars = g
@@ -61,7 +68,8 @@
"""Initialize the module as appropriate for NT"""
g = {}
g['EXE'] = ".exe"
- g['SO'] = ".pyd"
+ g['SO'] = _get_so_extension() or ".pyd"
+ g['SOABI'] = g['SO'].rsplit('.')[0]
global _config_vars
_config_vars = g
Modified: pypy/branch/arm-backend/lib-python/modified-2.5.2/test/mapping_tests.py
==============================================================================
--- pypy/branch/arm-backend/lib-python/modified-2.5.2/test/mapping_tests.py (original)
+++ pypy/branch/arm-backend/lib-python/modified-2.5.2/test/mapping_tests.py Tue Nov 23 10:12:47 2010
@@ -1,6 +1,7 @@
# tests common to dict and UserDict
import unittest
import UserDict
+from test import test_support
class BasicTestMappingProtocol(unittest.TestCase):
@@ -525,7 +526,8 @@
self.assertEqual(va, int(ka))
kb, vb = tb = b.popitem()
self.assertEqual(vb, int(kb))
- self.assert_(not(copymode < 0 and ta != tb))
+ if test_support.check_impl_detail():
+ self.assert_(not(copymode < 0 and ta != tb))
self.assert_(not a)
self.assert_(not b)
Modified: pypy/branch/arm-backend/lib-python/modified-2.5.2/test/test_descr.py
==============================================================================
--- pypy/branch/arm-backend/lib-python/modified-2.5.2/test/test_descr.py (original)
+++ pypy/branch/arm-backend/lib-python/modified-2.5.2/test/test_descr.py Tue Nov 23 10:12:47 2010
@@ -2028,7 +2028,9 @@
except TypeError, msg:
verify(str(msg).find("weak reference") >= 0)
else:
- verify(0, "weakref.ref(no) should be illegal")
+ # in PyPy it is (sometimes) possible to take a weakref here
+ #verify(0, "weakref.ref(no) should be illegal")
+ pass
class Weak(object):
__slots__ = ['foo', '__weakref__']
yes = Weak()
Modified: pypy/branch/arm-backend/lib_pypy/_ctypes/function.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/_ctypes/function.py (original)
+++ pypy/branch/arm-backend/lib_pypy/_ctypes/function.py Tue Nov 23 10:12:47 2010
@@ -171,7 +171,7 @@
return self._build_result(restype, resbuffer, argtypes, argsandobjs)
def _getfuncptr(self, argtypes, restype, thisarg=None):
- if self._ptr is not None:
+ if self._ptr is not None and argtypes is self._argtypes_:
return self._ptr
if restype is None or not isinstance(restype, _CDataMeta):
import ctypes
Modified: pypy/branch/arm-backend/lib_pypy/_ctypes/structure.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/_ctypes/structure.py (original)
+++ pypy/branch/arm-backend/lib_pypy/_ctypes/structure.py Tue Nov 23 10:12:47 2010
@@ -7,7 +7,7 @@
def round_up(size, alignment):
return (size + alignment - 1) & -alignment
-def size_alignment_pos(fields):
+def size_alignment_pos(fields, is_union=False):
import ctypes
size = 0
alignment = 1
@@ -15,14 +15,19 @@
for fieldname, ctype in fields:
fieldsize = ctypes.sizeof(ctype)
fieldalignment = ctypes.alignment(ctype)
- size = round_up(size, fieldalignment)
alignment = max(alignment, fieldalignment)
- pos.append(size)
- size += fieldsize
+ if is_union:
+ pos.append(0)
+ size = max(size, fieldsize)
+ else:
+ size = round_up(size, fieldalignment)
+ pos.append(size)
+ size += fieldsize
size = round_up(size, alignment)
return size, alignment, pos
-def names_and_fields(_fields_, superclass, zero_offset=False, anon=None):
+def names_and_fields(_fields_, superclass, zero_offset=False, anon=None,
+ is_union=False):
if isinstance(_fields_, tuple):
_fields_ = list(_fields_)
for _, tp in _fields_:
@@ -36,7 +41,7 @@
rawfields = [(name, ctype._ffishape)
for name, ctype in all_fields]
if not zero_offset:
- _, _, pos = size_alignment_pos(all_fields)
+ _, _, pos = size_alignment_pos(all_fields, is_union)
else:
pos = [0] * len(all_fields)
fields = {}
@@ -73,8 +78,8 @@
# ________________________________________________________________
-def _set_shape(tp, rawfields):
- tp._ffistruct = _rawffi.Structure(rawfields)
+def _set_shape(tp, rawfields, is_union=False):
+ tp._ffistruct = _rawffi.Structure(rawfields, is_union)
tp._ffiargshape = tp._ffishape = (tp._ffistruct, 1)
tp._fficompositesize = tp._ffistruct.size
@@ -92,13 +97,14 @@
raise AttributeError("Structure or union cannot contain itself")
self._names, rawfields, self._fieldtypes = names_and_fields(
value, self.__bases__[0], False,
- self.__dict__.get('_anonymous_', None))
+ self.__dict__.get('_anonymous_', None), self._is_union)
_CDataMeta.__setattr__(self, '_fields_', value)
- _set_shape(self, rawfields)
+ _set_shape(self, rawfields, self._is_union)
return
_CDataMeta.__setattr__(self, name, value)
-class StructureMeta(_CDataMeta):
+class StructOrUnionMeta(_CDataMeta):
+
def __new__(self, name, cls, typedict):
res = type.__new__(self, name, cls, typedict)
if '_fields_' in typedict:
@@ -109,8 +115,8 @@
raise AttributeError("Anonymous field not found")
res._names, rawfields, res._fieldtypes = names_and_fields(
typedict['_fields_'], cls[0], False,
- typedict.get('_anonymous_', None))
- _set_shape(res, rawfields)
+ typedict.get('_anonymous_', None), self._is_union)
+ _set_shape(res, rawfields, self._is_union)
return res
@@ -150,8 +156,8 @@
res.__dict__['_index'] = -1
return res
-class Structure(_CData):
- __metaclass__ = StructureMeta
+class StructOrUnion(_CData):
+ __metaclass__ = StructOrUnionMeta
def __new__(cls, *args, **kwds):
if not hasattr(cls, '_ffistruct'):
@@ -213,3 +219,10 @@
def _get_buffer_value(self):
return self._buffer.buffer
+
+
+class StructureMeta(StructOrUnionMeta):
+ _is_union = False
+
+class Structure(StructOrUnion):
+ __metaclass__ = StructureMeta
Modified: pypy/branch/arm-backend/lib_pypy/_ctypes/union.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/_ctypes/union.py (original)
+++ pypy/branch/arm-backend/lib_pypy/_ctypes/union.py Tue Nov 23 10:12:47 2010
@@ -1,118 +1,7 @@
+from _ctypes import structure
+class UnionMeta(structure.StructOrUnionMeta):
+ _is_union = True
-import _rawffi
-from _ctypes.basics import _CData, _CDataMeta, store_reference, keepalive_key
-from _ctypes.basics import ensure_objects
-from _ctypes.structure import round_up, names_and_fields, struct_getattr,\
- struct_setattr
-
-
-def _set_shape(tp):
- size = tp._sizeofinstances()
- alignment = tp._alignmentofinstances()
- tp._ffiopaque = _rawffi.Structure((size, alignment)) # opaque
- tp._ffiargshape = tp._ffishape = (tp._ffiopaque, 1)
- tp._fficompositesize = tp._ffiopaque.size
- # we need to create an array of size one for each
- # of our elements
- tp._ffiarrays = {}
- for name, field in tp._fieldtypes.iteritems():
- tp._ffiarrays[name] = _rawffi.Array(field.ctype._ffishape)
-
-class UnionMeta(_CDataMeta):
- def __new__(self, name, cls, typedict):
- res = type.__new__(self, name, cls, typedict)
- if '_fields_' in typedict:
- res._names, rawfields, res._fieldtypes = names_and_fields(
- typedict['_fields_'], cls[0], True,
- typedict.get('_anonymous_', None))
- _set_shape(res)
-
- def __init__(self): # don't allow arguments by now
- if not hasattr(self, '_ffiarrays'):
- raise TypeError("Cannot instantiate union, has no type")
- # malloc size
- size = self.__class__._sizeofinstances()
- self.__dict__['_objects'] = {}
- self.__dict__['_buffer'] = self._ffiopaque(autofree=True)
- res.__init__ = __init__
- return res
-
- def _sizeofinstances(self):
- if not hasattr(self, '_size_'):
- self._size_ = max([field.size for field in
- self._fieldtypes.values()] + [0])
- return self._size_
-
- def _alignmentofinstances(self):
- from ctypes import alignment
- if not hasattr(self, '_alignment_'):
- self._alignment_ = max([alignment(field.ctype) for field in
- self._fieldtypes.values()] + [1])
- return self._alignment_
-
- __getattr__ = struct_getattr
-
- def __setattr__(self, name, value):
- if name == '_fields_':
- if self.__dict__.get('_fields_', None):
- raise AttributeError("_fields_ is final")
- if self in [v for k, v in value]:
- raise AttributeError("Union cannot contain itself")
- self._names, rawfields, self._fieldtypes = names_and_fields(
- value, self.__bases__[0], True,
- self.__dict__.get('_anonymous_', None))
- _CDataMeta.__setattr__(self, '_fields_', value)
- _set_shape(self)
- _CDataMeta.__setattr__(self, name, value)
-
- def _CData_output(self, resarray, base=None, index=-1):
- res = self.__new__(self)
- ffiopaque = self._ffiopaque.fromaddress(resarray.buffer)
- res.__dict__['_buffer'] = ffiopaque
- res.__dict__['_base'] = base
- res.__dict__['_index'] = index
- return res
-
- def _CData_retval(self, resbuffer):
- res = self.__new__(self)
- res.__dict__['_buffer'] = resbuffer
- res.__dict__['_base'] = None
- res.__dict__['_index'] = -1
- return res
-
-
-class Union(_CData):
+class Union(structure.StructOrUnion):
__metaclass__ = UnionMeta
-
- def __getattr__(self, name):
- try:
- field = self._fieldtypes[name]
- except KeyError:
- raise AttributeError(name)
- fieldtype = field.ctype
- val = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1)
- offset = field.num
- return fieldtype._CData_output(val, self, offset)
-
- def __setattr__(self, name, value):
- try:
- field = self._fieldtypes[name]
- except KeyError:
- raise AttributeError(name)
- fieldtype = field.ctype
- cobj = fieldtype.from_param(value)
- if ensure_objects(cobj) is not None:
- key = keepalive_key(field.num)
- store_reference(self, key, cobj._objects)
- arg = cobj._get_buffer_value()
- if fieldtype._fficompositesize is not None:
- from ctypes import memmove
- dest = self._buffer.buffer
- memmove(dest, arg, fieldtype._fficompositesize)
- else:
- buf = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1)
- buf[0] = arg
-
- def _get_buffer_value(self):
- return self._buffer.buffer
Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/hashlib.ctc.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/hashlib.ctc.py (original)
+++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/hashlib.ctc.py Tue Nov 23 10:12:47 2010
@@ -4,7 +4,8 @@
"""
from ctypes import *
-from ctypes_configure import configure, dumpcache
+from ctypes_configure import configure
+import dumpcache
class CConfig:
@@ -17,4 +18,4 @@
[('digest', c_void_p)])
config = configure.configure(CConfig)
-dumpcache.dumpcache(__file__, '_hashlib_cache.py', config)
+dumpcache.dumpcache2('hashlib', config)
Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/locale.ctc.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/locale.ctc.py (original)
+++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/locale.ctc.py Tue Nov 23 10:12:47 2010
@@ -5,7 +5,7 @@
from ctypes_configure.configure import (configure, ExternalCompilationInfo,
ConstantInteger, DefinedConstantInteger, SimpleType, check_eci)
-from ctypes_configure.dumpcache import dumpcache
+import dumpcache
# ____________________________________________________________
@@ -70,4 +70,4 @@
config['ALL_CONSTANTS'] = tuple(_CONSTANTS)
config['HAS_LANGINFO'] = HAS_LANGINFO
-dumpcache(__file__, '_locale_cache.py', config)
+dumpcache.dumpcache2('locale', config)
Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/pyexpat.ctc.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/pyexpat.ctc.py (original)
+++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/pyexpat.ctc.py Tue Nov 23 10:12:47 2010
@@ -5,7 +5,8 @@
import ctypes
from ctypes import c_char_p, c_int, c_void_p, c_char
-from ctypes_configure import configure, dumpcache
+from ctypes_configure import configure
+import dumpcache
class CConfigure:
@@ -41,4 +42,4 @@
config = configure.configure(CConfigure)
-dumpcache.dumpcache(__file__, '_pyexpat_cache.py', config)
+dumpcache.dumpcache2('pyexpat', config)
Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/rebuild.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/rebuild.py (original)
+++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/rebuild.py Tue Nov 23 10:12:47 2010
@@ -31,10 +31,25 @@
sys.path[:] = path
def try_rebuild():
+ from pypy.jit.backend import detect_cpu
+ model = detect_cpu.autodetect_main_model_and_size()
+ # remove the files '_*_model_.py'
+ left = {}
+ for p in os.listdir(_dirpath):
+ if p.startswith('_') and (p.endswith('_%s_.py' % model) or
+ p.endswith('_%s_.pyc' % model)):
+ os.unlink(os.path.join(_dirpath, p))
+ elif p.startswith('_') and (p.endswith('_.py') or
+ p.endswith('_.pyc')):
+ for i in range(2, len(p)-4):
+ left[p[:i]] = True
+ # remove the files '_*_cache.py' if there is no '_*_*_.py' left around
for p in os.listdir(_dirpath):
if p.startswith('_') and (p.endswith('_cache.py') or
p.endswith('_cache.pyc')):
- os.unlink(os.path.join(_dirpath, p))
+ if p[:-9] not in left:
+ os.unlink(os.path.join(_dirpath, p))
+ #
for p in os.listdir(_dirpath):
if p.endswith('.ctc.py'):
try:
Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/resource.ctc.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/resource.ctc.py (original)
+++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/resource.ctc.py Tue Nov 23 10:12:47 2010
@@ -4,7 +4,7 @@
"""
from ctypes import sizeof
-from ctypes_configure.dumpcache import dumpcache
+import dumpcache
from ctypes_configure.configure import (configure,
ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger,
SimpleType)
@@ -58,4 +58,4 @@
del config[key]
config['ALL_CONSTANTS'] = _CONSTANTS + tuple(optional_constants)
-dumpcache(__file__, '_resource_cache.py', config)
+dumpcache.dumpcache2('resource', config)
Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/syslog.ctc.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/syslog.ctc.py (original)
+++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/syslog.ctc.py Tue Nov 23 10:12:47 2010
@@ -5,7 +5,7 @@
from ctypes_configure.configure import (configure,
ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger)
-from ctypes_configure.dumpcache import dumpcache
+import dumpcache
_CONSTANTS = (
@@ -72,4 +72,4 @@
all_constants = config.keys()
all_constants.sort()
config['ALL_CONSTANTS'] = tuple(all_constants)
-dumpcache(__file__, '_syslog_cache.py', config)
+dumpcache.dumpcache2('syslog', config)
Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/test/test_cache.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/test/test_cache.py (original)
+++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/test/test_cache.py Tue Nov 23 10:12:47 2010
@@ -7,19 +7,26 @@
def run(filename, outputname):
filepath = dirpath.join(filename)
- tmpdir = udir.ensure('testcache-' + filename, dir=True)
- outputpath = tmpdir.join(outputname)
- d = {'__file__': str(outputpath)}
+ tmpdir = udir.ensure('testcache-' + os.path.splitext(filename)[0],
+ dir=True)
+ tmpdir.join('dumpcache.py').write(dirpath.join('dumpcache.py').read())
path = sys.path[:]
try:
- sys.path.insert(0, str(dirpath))
- execfile(str(filepath), d)
+ sys.path.insert(0, str(tmpdir))
+ execfile(str(filepath), {})
finally:
sys.path[:] = path
+ sys.modules.pop('dumpcache', None)
#
+ outputpath = tmpdir.join(outputname)
assert outputpath.check(exists=1)
- d = {}
- execfile(str(outputpath), d)
+ modname = os.path.splitext(outputname)[0]
+ try:
+ sys.path.insert(0, str(tmpdir))
+ d = {}
+ execfile(str(outputpath), d)
+ finally:
+ sys.path[:] = path
return d
Modified: pypy/branch/arm-backend/lib_pypy/pypy_test/test_ctypes_support.py
==============================================================================
--- pypy/branch/arm-backend/lib_pypy/pypy_test/test_ctypes_support.py (original)
+++ pypy/branch/arm-backend/lib_pypy/pypy_test/test_ctypes_support.py Tue Nov 23 10:12:47 2010
@@ -22,12 +22,11 @@
assert get_errno() == 0
def test_argument_conversion_and_checks():
- import ctypes
- libc = ctypes.cdll.LoadLibrary("libc.so.6")
- libc.strlen.argtypes = ctypes.c_char_p,
- libc.strlen.restype = ctypes.c_size_t
- assert libc.strlen("eggs") == 4
-
+ strlen = standard_c_lib.strlen
+ strlen.argtypes = [c_char_p]
+ strlen.restype = c_size_t
+ assert strlen("eggs") == 4
+
# Should raise ArgumentError, not segfault
- py.test.raises(ctypes.ArgumentError, libc.strlen, False)
+ py.test.raises(ArgumentError, strlen, False)
Modified: pypy/branch/arm-backend/pypy/annotation/bookkeeper.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/bookkeeper.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/bookkeeper.py Tue Nov 23 10:12:47 2010
@@ -24,7 +24,7 @@
from pypy.rpython import extregistry
from pypy.tool.identity_dict import identity_dict
-class Stats:
+class Stats(object):
def __init__(self, bookkeeper):
self.bookkeeper = bookkeeper
@@ -87,13 +87,6 @@
else:
return obj.knowntype.__name__
- def consider_tuple_iter(self, tup):
- ctxt = "[%s]" % sys._getframe(4).f_code.co_name
- if tup.is_constant():
- return ctxt, tup.const
- else:
- return ctxt, tuple([self.typerepr(x) for x in tup.items])
-
def consider_tuple_random_getitem(self, tup):
return tuple([self.typerepr(x) for x in tup.items])
@@ -137,7 +130,7 @@
def consider_dict_delitem(self, dic):
return dic
-class Bookkeeper:
+class Bookkeeper(object):
"""The log of choices that have been made while analysing the operations.
It ensures that the same 'choice objects' will be returned if we ask
again during reflowing. Like ExecutionContext, there is an implicit
@@ -736,7 +729,7 @@
return True
# for parsing call arguments
-class RPythonCallsSpace:
+class RPythonCallsSpace(object):
"""Pseudo Object Space providing almost no real operation.
For the Arguments class: if it really needs other operations, it means
that the call pattern is too complex for R-Python.
Modified: pypy/branch/arm-backend/pypy/annotation/builtin.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/builtin.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/builtin.py Tue Nov 23 10:12:47 2010
@@ -423,7 +423,7 @@
from pypy.annotation.model import SomePtr
from pypy.rpython.lltypesystem import lltype
-def malloc(s_T, s_n=None, s_flavor=None, s_zero=None):
+def malloc(s_T, s_n=None, s_flavor=None, s_zero=None, s_track_allocation=None):
assert (s_n is None or s_n.knowntype == int
or issubclass(s_n.knowntype, pypy.rlib.rarithmetic.base_int))
assert s_T.is_constant()
@@ -438,19 +438,24 @@
r = SomePtr(lltype.typeOf(p))
else:
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# not sure how to call malloc() for the example 'p' in the
# presence of s_extraargs
r = SomePtr(lltype.Ptr(s_T.const))
return r
-def free(s_p, s_flavor):
+def free(s_p, s_flavor, s_track_allocation=None):
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# same problem as in malloc(): some flavors are not easy to
# malloc-by-example
#T = s_p.ll_ptrtype.TO
#p = lltype.malloc(T, flavor=s_flavor.const)
#lltype.free(p, flavor=s_flavor.const)
+def render_immortal(s_p, s_track_allocation=None):
+ assert s_track_allocation is None or s_track_allocation.is_constant()
+
def typeOf(s_val):
lltype = annotation_to_lltype(s_val, info="in typeOf(): ")
return immutablevalue(lltype)
@@ -518,6 +523,7 @@
BUILTIN_ANALYZERS[lltype.malloc] = malloc
BUILTIN_ANALYZERS[lltype.free] = free
+BUILTIN_ANALYZERS[lltype.render_immortal] = render_immortal
BUILTIN_ANALYZERS[lltype.typeOf] = typeOf
BUILTIN_ANALYZERS[lltype.cast_primitive] = cast_primitive
BUILTIN_ANALYZERS[lltype.nullptr] = nullptr
Modified: pypy/branch/arm-backend/pypy/annotation/classdef.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/classdef.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/classdef.py Tue Nov 23 10:12:47 2010
@@ -58,7 +58,7 @@
# same name in all subclasses of A, if any. (Parent class attributes can
# be visible in reads from instances of subclasses.)
-class Attribute:
+class Attribute(object):
# readonly-ness
# SomeThing-ness
# NB. an attribute is readonly if it is a constant class attribute.
@@ -402,7 +402,7 @@
# ____________________________________________________________
-class InstanceSource:
+class InstanceSource(object):
instance_level = True
def __init__(self, bookkeeper, obj):
Modified: pypy/branch/arm-backend/pypy/annotation/description.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/description.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/description.py Tue Nov 23 10:12:47 2010
@@ -6,7 +6,7 @@
from pypy.tool.sourcetools import valid_identifier
from pypy.tool.pairtype import extendabletype
-class CallFamily:
+class CallFamily(object):
"""A family of Desc objects that could be called from common call sites.
The call families are conceptually a partition of all (callable) Desc
objects, where the equivalence relation is the transitive closure of
@@ -51,7 +51,7 @@
self.total_calltable_size += 1
-class FrozenAttrFamily:
+class FrozenAttrFamily(object):
"""A family of FrozenDesc objects that have any common 'getattr' sites.
The attr families are conceptually a partition of FrozenDesc objects,
where the equivalence relation is the transitive closure of:
@@ -80,7 +80,7 @@
self.attrs[attrname] = s_value
-class ClassAttrFamily:
+class ClassAttrFamily(object):
"""A family of ClassDesc objects that have common 'getattr' sites for a
given attribute name. The attr families are conceptually a partition
of ClassDesc objects, where the equivalence relation is the transitive
Modified: pypy/branch/arm-backend/pypy/annotation/dictdef.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/dictdef.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/dictdef.py Tue Nov 23 10:12:47 2010
@@ -77,7 +77,7 @@
dictdef.dictvalue = self
-class DictDef:
+class DictDef(object):
"""A dict definition remembers how general the keys and values in that
particular dict have to be. Every dict creation makes a new DictDef,
and the union of two dicts merges the DictKeys and DictValues that each
Modified: pypy/branch/arm-backend/pypy/annotation/listdef.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/listdef.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/listdef.py Tue Nov 23 10:12:47 2010
@@ -6,7 +6,7 @@
class TooLateForChange(Exception):
pass
-class ListItem:
+class ListItem(object):
mutated = False # True for lists mutated after creation
resized = False # True for lists resized after creation
range_step = None # the step -- only for lists only created by a range()
@@ -117,7 +117,7 @@
return updated
-class ListDef:
+class ListDef(object):
"""A list definition remembers how general the items in that particular
list have to be. Every list creation makes a new ListDef, and the union
of two lists merges the ListItems that each ListDef stores."""
Modified: pypy/branch/arm-backend/pypy/annotation/model.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/model.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/model.py Tue Nov 23 10:12:47 2010
@@ -574,11 +574,11 @@
NUMBER = object()
annotation_to_ll_map = [
+ (SomeSingleFloat(), lltype.SingleFloat),
(s_None, lltype.Void), # also matches SomeImpossibleValue()
(s_Bool, lltype.Bool),
(SomeInteger(knowntype=r_ulonglong), NUMBER),
(SomeFloat(), lltype.Float),
- (SomeSingleFloat(), lltype.SingleFloat),
(SomeChar(), lltype.Char),
(SomeUnicodeCodePoint(), lltype.UniChar),
(SomeAddress(), llmemory.Address),
Modified: pypy/branch/arm-backend/pypy/annotation/policy.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/policy.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/policy.py Tue Nov 23 10:12:47 2010
@@ -1,4 +1,4 @@
-# base annotation policy for overrides and specialization
+# base annotation policy for specialization
from pypy.annotation.specialize import default_specialize as default
from pypy.annotation.specialize import specialize_argvalue, specialize_argtype, specialize_arglistitemtype
from pypy.annotation.specialize import memo
@@ -41,7 +41,7 @@
if directive is None:
return pol.default_specialize
- # specialize|override:name[(args)]
+ # specialize[(args)]
directive_parts = directive.split('(', 1)
if len(directive_parts) == 1:
[name] = directive_parts
@@ -60,14 +60,6 @@
except AttributeError:
raise AttributeError("%r specialize tag not defined in annotation"
"policy %s" % (name, pol))
- if directive.startswith('override:'):
- # different signature: override__xyz(*args_s)
- if parms:
- raise Exception, "override:* specialisations don't support parameters"
- def specialize_override(funcdesc, args_s):
- funcdesc.overridden = True
- return specializer(*args_s)
- return specialize_override
else:
if not parms:
return specializer
@@ -92,9 +84,5 @@
from pypy.rpython.annlowlevel import LowLevelAnnotatorPolicy
return LowLevelAnnotatorPolicy.specialize__ll_and_arg(*args)
- def override__ignore(pol, *args):
- bk = getbookkeeper()
- return bk.immutablevalue(None)
-
class StrictAnnotatorPolicy(AnnotatorPolicy):
allow_someobjects = False
Modified: pypy/branch/arm-backend/pypy/annotation/specialize.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/specialize.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/specialize.py Tue Nov 23 10:12:47 2010
@@ -100,7 +100,7 @@
# ____________________________________________________________________________
# specializations
-class MemoTable:
+class MemoTable(object):
def __init__(self, funcdesc, args, value):
self.funcdesc = funcdesc
self.table = {args: value}
Modified: pypy/branch/arm-backend/pypy/annotation/test/test_annrpython.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/test/test_annrpython.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/test/test_annrpython.py Tue Nov 23 10:12:47 2010
@@ -1,4 +1,4 @@
-
+from __future__ import with_statement
import autopath
import py.test
import sys
@@ -766,28 +766,6 @@
s = a.build_types(f, [list])
assert s.classdef is a.bookkeeper.getuniqueclassdef(IndexError) # KeyError ignored because l is a list
- def test_overrides(self):
- excs = []
- def record_exc(e):
- """NOT_RPYTHON"""
- excs.append(sys.exc_info)
- record_exc._annspecialcase_ = "override:record_exc"
- def g():
- pass
- def f():
- try:
- g()
- except Exception, e:
- record_exc(e)
- class MyAnnotatorPolicy(policy.AnnotatorPolicy):
-
- def override__record_exc(pol, s_e):
- return a.bookkeeper.immutablevalue(None)
-
- a = self.RPythonAnnotator(policy=MyAnnotatorPolicy())
- s = a.build_types(f, [])
- assert s.const is None
-
def test_freeze_protocol(self):
class Stuff:
def __init__(self, flag):
@@ -3359,6 +3337,26 @@
s = a.build_types(f, [])
assert isinstance(s, annmodel.SomeChar)
+ def test_context_manager(self):
+ class C:
+ def __init__(self):
+ pass
+ def __enter__(self):
+ self.x = 1
+ def __exit__(self, *args):
+ self.x = 3
+ def f():
+ c = C()
+ with c:
+ pass
+ return c.x
+
+ a = self.RPythonAnnotator()
+ s = a.build_types(f, [])
+ assert isinstance(s, annmodel.SomeInteger)
+ # not a constant: both __enter__ and __exit__ have been annotated
+ assert not s.is_constant()
+
def g(n):
return [0,1,2,n]
Modified: pypy/branch/arm-backend/pypy/annotation/test/test_model.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/test/test_model.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/test/test_model.py Tue Nov 23 10:12:47 2010
@@ -128,7 +128,7 @@
assert isinstance(s_p, SomeOOInstance) and s_p.ootype == C
def test_annotation_to_lltype():
- from pypy.rlib.rarithmetic import r_uint
+ from pypy.rlib.rarithmetic import r_uint, r_singlefloat
s_i = SomeInteger()
s_pos = SomeInteger(nonneg=True)
s_1 = SomeInteger(nonneg=True); s_1.const = 1
@@ -151,6 +151,9 @@
C = ootype.Instance('C', ROOT, {})
ref = SomeOOInstance(C)
assert annotation_to_lltype(ref) == C
+ s_singlefloat = SomeSingleFloat()
+ s_singlefloat.const = r_singlefloat(0.0)
+ assert annotation_to_lltype(s_singlefloat) == lltype.SingleFloat
def test_ll_union():
PS1 = lltype.Ptr(lltype.GcStruct('s'))
Modified: pypy/branch/arm-backend/pypy/annotation/unaryop.py
==============================================================================
--- pypy/branch/arm-backend/pypy/annotation/unaryop.py (original)
+++ pypy/branch/arm-backend/pypy/annotation/unaryop.py Tue Nov 23 10:12:47 2010
@@ -434,6 +434,9 @@
def method_clear(dct):
pass
+ def method_popitem(dct):
+ return dct.getanyitem('items')
+
def _can_only_throw(dic, *ignore):
if dic1.dictdef.dictkey.custom_eq_hash:
return None # r_dict: can throw anything
Modified: pypy/branch/arm-backend/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/arm-backend/pypy/config/pypyoption.py (original)
+++ pypy/branch/arm-backend/pypy/config/pypyoption.py Tue Nov 23 10:12:47 2010
@@ -30,9 +30,15 @@
"rctime" , "select", "zipimport", "_lsprof",
"crypt", "signal", "_rawffi", "termios", "zlib",
"struct", "md5", "sha", "bz2", "_minimal_curses", "cStringIO",
- "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array"]
+ "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array",
+ "_bisect"]
))
+translation_modules = default_modules.copy()
+translation_modules.update(dict.fromkeys(
+ ["fcntl", "rctime", "select", "signal", "_rawffi", "zlib",
+ "struct", "md5", "cStringIO", "array"]))
+
working_oo_modules = default_modules.copy()
working_oo_modules.update(dict.fromkeys(
["md5", "sha", "cStringIO", "itertools"]
@@ -73,9 +79,10 @@
}
module_import_dependencies = {
- # no _rawffi if importing pypy.rlib.libffi raises ImportError
+ # no _rawffi if importing pypy.rlib.clibffi raises ImportError
# or CompilationError
- "_rawffi" : ["pypy.rlib.libffi"],
+ "_rawffi" : ["pypy.rlib.clibffi"],
+ "_ffi" : ["pypy.rlib.clibffi"],
"zlib" : ["pypy.rlib.rzlib"],
"bz2" : ["pypy.module.bz2.interp_bz2"],
@@ -147,6 +154,12 @@
cmdline="--allworkingmodules",
negation=True),
+ BoolOption("translationmodules",
+ "use only those modules that are needed to run translate.py on pypy",
+ default=False,
+ cmdline="--translationmodules",
+ suggests=[("objspace.allworkingmodules", False)]),
+
BoolOption("geninterp", "specify whether geninterp should be used",
cmdline=None,
default=True),
@@ -162,6 +175,11 @@
default=False,
requires=[("objspace.usepycfiles", True)]),
+ StrOption("soabi",
+ "Tag to differentiate extension modules built for different Python interpreters",
+ cmdline="--soabi",
+ default=None),
+
BoolOption("honor__builtins__",
"Honor the __builtins__ key of a module dictionary",
default=False),
@@ -198,6 +216,9 @@
BoolOption("withstrslice", "use strings optimized for slicing",
default=False),
+ BoolOption("withstrbuf", "use strings optimized for addition (ver 2)",
+ default=False),
+
BoolOption("withprebuiltchar",
"use prebuilt single-character string objects",
default=False),
@@ -210,7 +231,8 @@
BoolOption("withrope", "use ropes as the string implementation",
default=False,
requires=[("objspace.std.withstrslice", False),
- ("objspace.std.withstrjoin", False)],
+ ("objspace.std.withstrjoin", False),
+ ("objspace.std.withstrbuf", False)],
suggests=[("objspace.std.withprebuiltchar", True),
("objspace.std.sharesmallstr", True)]),
@@ -224,19 +246,17 @@
requires=[("objspace.opcodes.CALL_LIKELY_BUILTIN", False),
("objspace.honor__builtins__", False)]),
- BoolOption("withsharingdict",
- "use dictionaries that share the keys part",
- default=False),
-
BoolOption("withdictmeasurement",
"create huge files with masses of information "
"about dictionaries",
default=False),
- BoolOption("withinlineddict",
- "make instances more compact by revoming a level of indirection",
+ BoolOption("withmapdict",
+ "make instances really small but slow without the JIT",
default=False,
- requires=[("objspace.std.withshadowtracking", False)]),
+ requires=[("objspace.std.getattributeshortcut", True),
+ ("objspace.std.withtypeversion", True),
+ ]),
BoolOption("withrangelist",
"enable special range list implementation that does not "
@@ -251,12 +271,6 @@
# weakrefs needed, because of get_subclasses()
requires=[("translation.rweakref", True)]),
- BoolOption("withshadowtracking",
- "track whether an instance attribute shadows a type"
- " attribute",
- default=False,
- requires=[("objspace.std.withtypeversion", True),
- ("translation.rweakref", True)]),
BoolOption("withmethodcache",
"try to cache method lookups",
default=False,
@@ -328,9 +342,6 @@
config.objspace.std.suggest(optimized_list_getitem=True)
config.objspace.std.suggest(getattributeshortcut=True)
config.objspace.std.suggest(newshortcut=True)
- if type_system != 'ootype':
- config.objspace.std.suggest(withsharingdict=True)
- config.objspace.std.suggest(withinlineddict=True)
# extra costly optimizations only go in level 3
if level == '3':
@@ -343,7 +354,7 @@
config.objspace.std.suggest(withprebuiltint=True)
config.objspace.std.suggest(withrangelist=True)
config.objspace.std.suggest(withprebuiltchar=True)
- config.objspace.std.suggest(withinlineddict=True)
+ config.objspace.std.suggest(withmapdict=True)
config.objspace.std.suggest(withstrslice=True)
config.objspace.std.suggest(withstrjoin=True)
# xxx other options? ropes maybe?
@@ -359,6 +370,7 @@
# extra optimizations with the JIT
if level == 'jit':
config.objspace.std.suggest(withcelldict=True)
+ config.objspace.std.suggest(withmapdict=True)
def enable_allworkingmodules(config):
@@ -373,6 +385,11 @@
modules = [name for name in modules if name not in essential_modules]
config.objspace.usemodules.suggest(**dict.fromkeys(modules, True))
+def enable_translationmodules(config):
+ modules = translation_modules
+ modules = [name for name in modules if name not in essential_modules]
+ config.objspace.usemodules.suggest(**dict.fromkeys(modules, True))
+
if __name__ == '__main__':
config = get_pypy_config()
Modified: pypy/branch/arm-backend/pypy/config/test/test_pypyoption.py
==============================================================================
--- pypy/branch/arm-backend/pypy/config/test/test_pypyoption.py (original)
+++ pypy/branch/arm-backend/pypy/config/test/test_pypyoption.py Tue Nov 23 10:12:47 2010
@@ -47,7 +47,7 @@
def test_set_pypy_opt_level():
conf = get_pypy_config()
set_pypy_opt_level(conf, '2')
- assert conf.objspace.std.withsharingdict
+ assert conf.objspace.std.newshortcut
conf = get_pypy_config()
set_pypy_opt_level(conf, '0')
assert not conf.objspace.std.newshortcut
@@ -59,7 +59,6 @@
assert not conf.objspace.std.withtypeversion
assert not conf.objspace.std.withmethodcache
- assert not conf.objspace.std.withshadowtracking
def test_check_documentation():
def check_file_exists(fn):
Modified: pypy/branch/arm-backend/pypy/config/translationoption.py
==============================================================================
--- pypy/branch/arm-backend/pypy/config/translationoption.py (original)
+++ pypy/branch/arm-backend/pypy/config/translationoption.py Tue Nov 23 10:12:47 2010
@@ -115,7 +115,7 @@
default="auto", cmdline="--jit-backend"),
ChoiceOption("jit_debug", "the amount of debugging dumps made by the JIT",
["off", "profile", "steps", "detailed"],
- default="profile", # XXX for now
+ default="off",
cmdline="--jit-debug"),
ChoiceOption("jit_profiler", "integrate profiler support into the JIT",
["off", "oprofile"],
Modified: pypy/branch/arm-backend/pypy/conftest.py
==============================================================================
--- pypy/branch/arm-backend/pypy/conftest.py (original)
+++ pypy/branch/arm-backend/pypy/conftest.py Tue Nov 23 10:12:47 2010
@@ -7,6 +7,7 @@
from inspect import isclass, getmro
from pypy.tool.udir import udir
from pypy.tool.autopath import pypydir
+from pypy.tool import leakfinder
# pytest settings
pytest_plugins = "resultlog",
@@ -326,6 +327,31 @@
class PyPyTestFunction(py.test.collect.Function):
# All PyPy test items catch and display OperationErrors specially.
#
+ def runtest(self):
+ self.runtest_open()
+ try:
+ self.runtest_perform()
+ finally:
+ self.runtest_close()
+ self.runtest_finish()
+
+ def runtest_open(self):
+ leakfinder.start_tracking_allocations()
+
+ def runtest_perform(self):
+ super(PyPyTestFunction, self).runtest()
+
+ def runtest_close(self):
+ if leakfinder.TRACK_ALLOCATIONS:
+ self._pypytest_leaks = leakfinder.stop_tracking_allocations(False)
+ else: # stop_tracking_allocations() already called
+ self._pypytest_leaks = None
+
+ def runtest_finish(self):
+ # check for leaks, but only if the test passed so far
+ if self._pypytest_leaks:
+ raise leakfinder.MallocMismatch(self._pypytest_leaks)
+
def execute_appex(self, space, target, *args):
try:
target(*args)
@@ -352,9 +378,9 @@
def _keywords(self):
return super(IntTestFunction, self)._keywords() + ['interplevel']
- def runtest(self):
+ def runtest_perform(self):
try:
- super(IntTestFunction, self).runtest()
+ super(IntTestFunction, self).runtest_perform()
except OperationError, e:
check_keyboard_interrupt(e)
raise
@@ -367,12 +393,15 @@
py.test.skip('%s: %s' % (e.__class__.__name__, e))
cls = cls.__bases__[0]
raise
+
+ def runtest_finish(self):
if 'pygame' in sys.modules:
global _pygame_imported
if not _pygame_imported:
_pygame_imported = True
assert option.view, ("should not invoke Pygame "
"if conftest.option.view is False")
+ super(IntTestFunction, self).runtest_finish()
class AppTestFunction(PyPyTestFunction):
def _prunetraceback(self, traceback):
@@ -385,7 +414,7 @@
def _keywords(self):
return ['applevel'] + super(AppTestFunction, self)._keywords()
- def runtest(self):
+ def runtest_perform(self):
target = self.obj
if option.runappdirect:
return target()
@@ -417,7 +446,7 @@
space.setattr(w_instance, space.wrap(name[2:]),
getattr(instance, name))
- def runtest(self):
+ def runtest_perform(self):
target = self.obj
if option.runappdirect:
return target()
Modified: pypy/branch/arm-backend/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt
==============================================================================
--- pypy/branch/arm-backend/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt (original)
+++ pypy/branch/arm-backend/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt Tue Nov 23 10:12:47 2010
@@ -5,8 +5,6 @@
case. So far, this only works for calls with no keyword, no ``*arg``
and no ``**arg`` but it would be easy to extend.
-Gives the best results combined with :config:`objspace.std.withshadowtracking`.
-
For more information, see the section in `Standard Interpreter Optimizations`_.
.. _`Standard Interpreter Optimizations`: ../interpreter-optimizations.html#lookup-method-call-method
Modified: pypy/branch/arm-backend/pypy/doc/docindex.txt
==============================================================================
--- pypy/branch/arm-backend/pypy/doc/docindex.txt (original)
+++ pypy/branch/arm-backend/pypy/doc/docindex.txt Tue Nov 23 10:12:47 2010
@@ -84,7 +84,7 @@
PyPy's own tests `summary`_, daily updated, run through BuildBot infrastructure.
You can also find CPython's compliance tests run with compiled ``pypy-c``
-exeuctables there.
+executables there.
information dating from early 2007:
Modified: pypy/branch/arm-backend/pypy/doc/getting-started.txt
==============================================================================
--- pypy/branch/arm-backend/pypy/doc/getting-started.txt (original)
+++ pypy/branch/arm-backend/pypy/doc/getting-started.txt Tue Nov 23 10:12:47 2010
@@ -18,6 +18,7 @@
translation process - as opposed to encoding low level details into the
language implementation itself. `more...`_
+
.. _Python: http://docs.python.org/ref
.. _`more...`: architecture.html
Modified: pypy/branch/arm-backend/pypy/doc/interpreter-optimizations.txt
==============================================================================
--- pypy/branch/arm-backend/pypy/doc/interpreter-optimizations.txt (original)
+++ pypy/branch/arm-backend/pypy/doc/interpreter-optimizations.txt Tue Nov 23 10:12:47 2010
@@ -153,8 +153,8 @@
dicts:
the representation of the instance dict contains only a list of values.
-You can enable this feature with the :config:`objspace.std.withsharingdict`
-option.
+A more advanced version of sharing dicts, called *map dicts,* is available
+with the :config:`objspace.std.withmapdict` option.
Builtin-Shadowing
+++++++++++++++++
@@ -219,8 +219,7 @@
shadowing the class attribute. If we know that there is no shadowing (since
instance dict tells us that) we can save this lookup on the instance dictionary.
-You can enable this feature with the :config:`objspace.std.withshadowtracking`
-option.
+*This was deprecated and is no longer available.*
Method Caching
Modified: pypy/branch/arm-backend/pypy/interpreter/argument.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/argument.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/argument.py Tue Nov 23 10:12:47 2010
@@ -64,7 +64,7 @@
return not self == other
- # make it look tuply for the annotator
+ # make it look tuply for its use in the annotator
def __len__(self):
return 3
@@ -103,10 +103,11 @@
make_sure_not_resized(self.keywords_w)
make_sure_not_resized(self.arguments_w)
- if ((w_stararg is not None and w_stararg) or
- (w_starstararg is not None and w_starstararg)):
- self._combine_wrapped(w_stararg, w_starstararg)
- # if we have a call where * or ** args are used at the callsite
+ if w_stararg is not None and space.is_true(w_stararg):
+ self._combine_starargs_wrapped(w_stararg)
+ if w_starstararg is not None and space.is_true(w_starstararg):
+ self._combine_starstarargs_wrapped(w_starstararg)
+ # if we have a call where **args are used at the callsite
# we shouldn't let the JIT see the argument matching
self._dont_jit = True
else:
@@ -142,42 +143,48 @@
def _combine_wrapped(self, w_stararg, w_starstararg):
"unpack the *arg and **kwd into arguments_w and keywords_w"
- # unpack the * arguments
if w_stararg is not None:
- self.arguments_w = (self.arguments_w +
- self.space.fixedview(w_stararg))
- # unpack the ** arguments
+ self._combine_starargs_wrapped(w_stararg)
if w_starstararg is not None:
- space = self.space
- if not space.is_true(space.isinstance(w_starstararg, space.w_dict)):
+ self._combine_starstarargs_wrapped(w_starstararg)
+
+ def _combine_starargs_wrapped(self, w_stararg):
+ # unpack the * arguments
+ self.arguments_w = (self.arguments_w +
+ self.space.fixedview(w_stararg))
+
+ def _combine_starstarargs_wrapped(self, w_starstararg):
+ # unpack the ** arguments
+ space = self.space
+ if not space.is_true(space.isinstance(w_starstararg, space.w_dict)):
+ raise OperationError(space.w_TypeError,
+ space.wrap("argument after ** must be "
+ "a dictionary"))
+ keywords_w = [None] * space.int_w(space.len(w_starstararg))
+ keywords = [None] * space.int_w(space.len(w_starstararg))
+ i = 0
+ for w_key in space.unpackiterable(w_starstararg):
+ try:
+ key = space.str_w(w_key)
+ except OperationError, e:
+ if not e.match(space, space.w_TypeError):
+ raise
raise OperationError(space.w_TypeError,
- space.wrap("argument after ** must be "
- "a dictionary"))
- keywords_w = [None] * space.int_w(space.len(w_starstararg))
- keywords = [None] * space.int_w(space.len(w_starstararg))
- i = 0
- for w_key in space.unpackiterable(w_starstararg):
- try:
- key = space.str_w(w_key)
- except OperationError, e:
- if not e.match(space, space.w_TypeError):
- raise
- raise OperationError(space.w_TypeError,
- space.wrap("keywords must be strings"))
- if self.keywords and key in self.keywords:
- raise operationerrfmt(self.space.w_TypeError,
- "got multiple values "
- "for keyword argument "
- "'%s'", key)
- keywords[i] = key
- keywords_w[i] = space.getitem(w_starstararg, w_key)
- i += 1
- if self.keywords is None:
- self.keywords = keywords
- self.keywords_w = keywords_w
- else:
- self.keywords = self.keywords + keywords
- self.keywords_w = self.keywords_w + keywords_w
+ space.wrap("keywords must be strings"))
+ if self.keywords and key in self.keywords:
+ raise operationerrfmt(self.space.w_TypeError,
+ "got multiple values "
+ "for keyword argument "
+ "'%s'", key)
+ keywords[i] = key
+ keywords_w[i] = space.getitem(w_starstararg, w_key)
+ i += 1
+ if self.keywords is None:
+ self.keywords = keywords
+ self.keywords_w = keywords_w
+ else:
+ self.keywords = self.keywords + keywords
+ self.keywords_w = self.keywords_w + keywords_w
def fixedunpack(self, argcount):
"""The simplest argument parsing: get the 'argcount' arguments,
@@ -226,6 +233,10 @@
# argnames = list of formal parameter names
# scope_w = resulting list of wrapped values
#
+
+ # some comments about the JIT: it assumes that signature is a constant,
+ # so all values coming from there can be assumed constant. It assumes
+ # that the length of the defaults_w does not vary too much.
co_argcount = signature.num_argnames() # expected formal arguments, without */**
has_vararg = signature.has_vararg()
has_kwarg = signature.has_kwarg()
@@ -245,12 +256,6 @@
args_w = self.arguments_w
num_args = len(args_w)
- keywords = self.keywords
- keywords_w = self.keywords_w
- num_kwds = 0
- if keywords is not None:
- num_kwds = len(keywords)
-
avail = num_args + upfront
if input_argcount < co_argcount:
@@ -260,15 +265,24 @@
else:
take = num_args
+ # letting the JIT unroll this loop is safe, because take is always
+ # smaller than co_argcount
for i in range(take):
scope_w[i + input_argcount] = args_w[i]
input_argcount += take
+ keywords = self.keywords
+ keywords_w = self.keywords_w
+ num_kwds = 0
+ if keywords is not None:
+ num_kwds = len(keywords)
# the code assumes that keywords can potentially be large, but that
# argnames is typically not too large
num_remainingkwds = num_kwds
used_keywords = None
if keywords:
+ # letting JIT unroll the loop is *only* safe if the callsite didn't
+ # use **args because num_kwds can be arbitrarily large otherwise.
used_keywords = [False] * num_kwds
for i in range(num_kwds):
name = keywords[i]
@@ -276,7 +290,7 @@
if j < 0:
continue
elif j < input_argcount:
- # check that no keyword argument conflicts with these note
+ # check that no keyword argument conflicts with these. note
# that for this purpose we ignore the first blindargs,
# which were put into place by prepend(). This way,
# keywords do not conflict with the hidden extra argument
@@ -388,9 +402,10 @@
space = self.space
w_args = space.newtuple(self.arguments_w)
w_kwds = space.newdict()
- for i in range(len(self.keywords)):
- space.setitem(w_kwds, space.wrap(self.keywords[i]),
- self.keywords_w[i])
+ if self.keywords is not None:
+ for i in range(len(self.keywords)):
+ space.setitem(w_kwds, space.wrap(self.keywords[i]),
+ self.keywords_w[i])
return w_args, w_kwds
class ArgumentsForTranslation(Arguments):
Modified: pypy/branch/arm-backend/pypy/interpreter/astcompiler/test/test_compiler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/astcompiler/test/test_compiler.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/astcompiler/test/test_compiler.py Tue Nov 23 10:12:47 2010
@@ -41,7 +41,7 @@
co_expr = compile(evalexpr, '<evalexpr>', 'eval')
space = self.space
pyco_expr = PyCode._from_code(space, co_expr)
- w_res = pyco_expr.exec_code(space, w_dict, w_dict)
+ w_res = pyco_expr.exec_host_bytecode(space, w_dict, w_dict)
res = space.str_w(space.repr(w_res))
if not isinstance(expected, float):
assert res == repr(expected)
Modified: pypy/branch/arm-backend/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/baseobjspace.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/baseobjspace.py Tue Nov 23 10:12:47 2010
@@ -36,13 +36,10 @@
return space.finditem_str(w_dict, attr)
return None
- def getdictvalue_attr_is_in_class(self, space, attr):
- return self.getdictvalue(space, attr)
-
- def setdictvalue(self, space, attr, w_value, shadows_type=True):
+ def setdictvalue(self, space, attr, w_value):
w_dict = self.getdict()
if w_dict is not None:
- space.setitem_str(w_dict, attr, w_value, shadows_type)
+ space.setitem_str(w_dict, attr, w_value)
return True
return False
@@ -168,6 +165,20 @@
def _call_builtin_destructor(self):
pass # method overridden in typedef.py
+ # hooks that the mapdict implementations needs:
+ def _get_mapdict_map(self):
+ return None
+ def _set_mapdict_map(self, map):
+ raise NotImplementedError
+ def _mapdict_read_storage(self, index):
+ raise NotImplementedError
+ def _mapdict_write_storage(self, index, value):
+ raise NotImplementedError
+ def _mapdict_storage_length(self):
+ raise NotImplementedError
+ def _set_mapdict_storage_and_map(self, storage, map):
+ raise NotImplementedError
+
class Wrappable(W_Root):
"""A subclass of Wrappable is an internal, interpreter-level class
@@ -247,10 +258,9 @@
self.interned_strings = {}
self.actionflag = ActionFlag() # changed by the signal module
+ self.check_signal_action = None # changed by the signal module
self.user_del_action = UserDelAction(self)
self.frame_trace_action = FrameTraceAction(self)
- self.actionflag.register_action(self.user_del_action)
- self.actionflag.register_action(self.frame_trace_action)
from pypy.interpreter.pycode import cpython_magic, default_magic
self.our_magic = default_magic
@@ -643,7 +653,7 @@
"""shortcut for space.int_w(space.hash(w_obj))"""
return self.int_w(self.hash(w_obj))
- def setitem_str(self, w_obj, key, w_value, shadows_type=True):
+ def setitem_str(self, w_obj, key, w_value):
return self.setitem(w_obj, self.wrap(key), w_value)
def finditem_str(self, w_obj, key):
@@ -725,7 +735,7 @@
def unpackiterable(self, w_iterable, expected_length=-1):
"""Unpack an iterable object into a real (interpreter-level) list.
- Raise a real (subclass of) ValueError if the length is wrong."""
+ Raise an OperationError(w_ValueError) if the length is wrong."""
w_iterator = self.iter(w_iterable)
items = []
while True:
Modified: pypy/branch/arm-backend/pypy/interpreter/executioncontext.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/executioncontext.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/executioncontext.py Tue Nov 23 10:12:47 2010
@@ -5,6 +5,8 @@
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib import jit
+TICK_COUNTER_STEP = 100
+
def app_profile_call(space, w_callable, frame, event, w_arg):
space.call_function(w_callable,
space.wrap(frame),
@@ -19,6 +21,9 @@
# XXX self.w_tracefunc, self.profilefunc
# XXX frame.is_being_profiled
+ # XXX [fijal] but they're not. is_being_profiled is guarded a bit all
+ # over the place as well as w_tracefunc
+
def __init__(self, space):
self.space = space
self.topframeref = jit.vref_None
@@ -163,24 +168,19 @@
if self.w_tracefunc is not None:
self._trace(frame, 'return', w_retval)
- def bytecode_trace(self, frame):
+ def bytecode_trace(self, frame, decr_by=TICK_COUNTER_STEP):
"Trace function called before each bytecode."
# this is split into a fast path and a slower path that is
# not invoked every time bytecode_trace() is.
actionflag = self.space.actionflag
- ticker = actionflag.get()
- if actionflag.has_bytecode_counter: # this "if" is constant-folded
- ticker += 1
- actionflag.set(ticker)
- if ticker & actionflag.interesting_bits: # fast check
+ if actionflag.decrement_ticker(decr_by) < 0:
actionflag.action_dispatcher(self, frame) # slow path
bytecode_trace._always_inline_ = True
def bytecode_trace_after_exception(self, frame):
"Like bytecode_trace(), but without increasing the ticker."
actionflag = self.space.actionflag
- ticker = actionflag.get()
- if ticker & actionflag.interesting_bits: # fast check
+ if actionflag.get_ticker() < 0:
actionflag.action_dispatcher(self, frame) # slow path
bytecode_trace_after_exception._always_inline_ = True
@@ -311,125 +311,109 @@
frame.last_exception = last_exception
self.is_tracing -= 1
+ def checksignals(self):
+ """Similar to PyErr_CheckSignals(). If called in the main thread,
+ and if signals are pending for the process, deliver them now
+ (i.e. call the signal handlers)."""
+ if self.space.check_signal_action is not None:
+ self.space.check_signal_action.perform(self, None)
+
def _freeze_(self):
raise Exception("ExecutionContext instances should not be seen during"
" translation. Now is a good time to inspect the"
" traceback and see where this one comes from :-)")
-class AbstractActionFlag:
- """This holds the global 'action flag'. It is a single bitfield
- integer, with bits corresponding to AsyncAction objects that need to
- be immediately triggered. The correspondance from bits to
- AsyncAction instances is built at translation time. We can quickly
- check if there is anything at all to do by checking if any of the
- relevant bits is set. If threads are enabled, they consume the 20
- lower bits to hold a counter incremented at each bytecode, to know
- when to release the GIL.
+class AbstractActionFlag(object):
+ """This holds in an integer the 'ticker'. If threads are enabled,
+ it is decremented at each bytecode; when it reaches zero, we release
+ the GIL. And whether we have threads or not, it is forced to zero
+ whenever we fire any of the asynchronous actions.
"""
def __init__(self):
self._periodic_actions = []
self._nonperiodic_actions = []
- self.unused_bits = self.FREE_BITS[:]
self.has_bytecode_counter = False
- self.interesting_bits = 0
+ self.fired_actions = None
+ self.checkinterval_scaled = 100 * TICK_COUNTER_STEP
self._rebuild_action_dispatcher()
def fire(self, action):
- """Request for the action to be run before the next opcode.
- The action must have been registered at space initalization time."""
- ticker = self.get()
- self.set(ticker | action.bitmask)
-
- def register_action(self, action):
- "NOT_RPYTHON"
- assert isinstance(action, AsyncAction)
- if action.bitmask == 0:
- while True:
- action.bitmask = self.unused_bits.pop(0)
- if not (action.bitmask & self.interesting_bits):
- break
- self.interesting_bits |= action.bitmask
- if action.bitmask & self.BYTECODE_COUNTER_OVERFLOW_BIT:
- assert action.bitmask == self.BYTECODE_COUNTER_OVERFLOW_BIT
- self._periodic_actions.append(action)
+ """Request for the action to be run before the next opcode."""
+ if not action._fired:
+ action._fired = True
+ if self.fired_actions is None:
+ self.fired_actions = []
+ self.fired_actions.append(action)
+ # set the ticker to -1 in order to force action_dispatcher()
+ # to run at the next possible bytecode
+ self.reset_ticker(-1)
+
+ def register_periodic_action(self, action, use_bytecode_counter):
+ """NOT_RPYTHON:
+ Register the PeriodicAsyncAction action to be called whenever the
+ tick counter becomes smaller than 0. If 'use_bytecode_counter' is
+ True, make sure that we decrease the tick counter at every bytecode.
+ This is needed for threads. Note that 'use_bytecode_counter' can be
+ False for signal handling, because whenever the process receives a
+ signal, the tick counter is set to -1 by C code in signals.h.
+ """
+ assert isinstance(action, PeriodicAsyncAction)
+ self._periodic_actions.append(action)
+ if use_bytecode_counter:
self.has_bytecode_counter = True
- self.force_tick_counter()
- else:
- self._nonperiodic_actions.append((action, action.bitmask))
self._rebuild_action_dispatcher()
- def setcheckinterval(self, space, interval):
- if interval < self.CHECK_INTERVAL_MIN:
- interval = self.CHECK_INTERVAL_MIN
- elif interval > self.CHECK_INTERVAL_MAX:
- interval = self.CHECK_INTERVAL_MAX
- space.sys.checkinterval = interval
- self.force_tick_counter()
-
- def force_tick_counter(self):
- # force the tick counter to a valid value -- this actually forces
- # it to reach BYTECODE_COUNTER_OVERFLOW_BIT at the next opcode.
- ticker = self.get()
- ticker &= ~ self.BYTECODE_COUNTER_OVERFLOW_BIT
- ticker |= self.BYTECODE_COUNTER_MASK
- self.set(ticker)
+ def getcheckinterval(self):
+ return self.checkinterval_scaled // TICK_COUNTER_STEP
+
+ def setcheckinterval(self, interval):
+ MAX = sys.maxint // TICK_COUNTER_STEP
+ if interval < 1:
+ interval = 1
+ elif interval > MAX:
+ interval = MAX
+ self.checkinterval_scaled = interval * TICK_COUNTER_STEP
def _rebuild_action_dispatcher(self):
periodic_actions = unrolling_iterable(self._periodic_actions)
- nonperiodic_actions = unrolling_iterable(self._nonperiodic_actions)
- has_bytecode_counter = self.has_bytecode_counter
@jit.dont_look_inside
def action_dispatcher(ec, frame):
- # periodic actions
- if has_bytecode_counter:
- ticker = self.get()
- if ticker & self.BYTECODE_COUNTER_OVERFLOW_BIT:
- # We must run the periodic actions now, but first
- # reset the bytecode counter (the following line
- # works by assuming that we just overflowed the
- # counter, i.e. BYTECODE_COUNTER_OVERFLOW_BIT is
- # set but none of the BYTECODE_COUNTER_MASK bits
- # are).
- ticker -= ec.space.sys.checkinterval
- self.set(ticker)
- for action in periodic_actions:
- action.perform(ec, frame)
+ # periodic actions (first reset the bytecode counter)
+ self.reset_ticker(self.checkinterval_scaled)
+ for action in periodic_actions:
+ action.perform(ec, frame)
# nonperiodic actions
- for action, bitmask in nonperiodic_actions:
- ticker = self.get()
- if ticker & bitmask:
- self.set(ticker & ~ bitmask)
+ list = self.fired_actions
+ if list is not None:
+ self.fired_actions = None
+ for action in list:
+ action._fired = False
action.perform(ec, frame)
action_dispatcher._dont_inline_ = True
self.action_dispatcher = action_dispatcher
- # Bits reserved for the bytecode counter, if used
- BYTECODE_COUNTER_MASK = (1 << 20) - 1
- BYTECODE_COUNTER_OVERFLOW_BIT = (1 << 20)
-
- # Free bits
- FREE_BITS = [1 << _b for _b in range(21, LONG_BIT-1)]
-
- # The acceptable range of values for sys.checkinterval, so that
- # the bytecode_counter fits in 20 bits
- CHECK_INTERVAL_MIN = 1
- CHECK_INTERVAL_MAX = BYTECODE_COUNTER_OVERFLOW_BIT
-
class ActionFlag(AbstractActionFlag):
"""The normal class for space.actionflag. The signal module provides
a different one."""
- _flags = 0
+ _ticker = 0
+
+ def get_ticker(self):
+ return self._ticker
- def get(self):
- return self._flags
+ def reset_ticker(self, value):
+ self._ticker = value
- def set(self, value):
- self._flags = value
+ def decrement_ticker(self, by):
+ value = self._ticker
+ if self.has_bytecode_counter: # this 'if' is constant-folded
+ value -= by
+ self._ticker = value
+ return value
class AsyncAction(object):
@@ -437,7 +421,7 @@
asynchronously with regular bytecode execution, but that still need
to occur between two opcodes, not at a completely random time.
"""
- bitmask = 0 # means 'please choose one bit automatically'
+ _fired = False
def __init__(self, space):
self.space = space
@@ -450,10 +434,11 @@
def fire_after_thread_switch(self):
"""Bit of a hack: fire() the action but only the next time the GIL
is released and re-acquired (i.e. after a potential thread switch).
- Don't call this if threads are not enabled.
+ Don't call this if threads are not enabled. Currently limited to
+ one action (i.e. reserved for CheckSignalAction from module/signal).
"""
from pypy.module.thread.gil import spacestate
- spacestate.set_actionflag_bit_after_thread_switch |= self.bitmask
+ spacestate.action_after_thread_switch = self
def perform(self, executioncontext, frame):
"""To be overridden."""
@@ -463,7 +448,6 @@
"""Abstract base class for actions that occur automatically
every sys.checkinterval bytecodes.
"""
- bitmask = ActionFlag.BYTECODE_COUNTER_OVERFLOW_BIT
class UserDelAction(AsyncAction):
Modified: pypy/branch/arm-backend/pypy/interpreter/function.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/function.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/function.py Tue Nov 23 10:12:47 2010
@@ -430,8 +430,11 @@
self.w_class = w_class # possibly space.w_None
def descr_method__new__(space, w_subtype, w_function, w_instance, w_class=None):
- if space.is_w( w_instance, space.w_None ):
+ if space.is_w(w_instance, space.w_None):
w_instance = None
+ if w_instance is None and space.is_w(w_class, space.w_None):
+ raise OperationError(space.w_TypeError,
+ space.wrap("unbound methods must have class"))
method = space.allocate_instance(Method, w_subtype)
Method.__init__(method, space, w_function, w_instance, w_class)
return space.wrap(method)
@@ -586,12 +589,14 @@
w_klass = space.type(w_obj)
return space.wrap(Method(space, self.w_function, w_klass, space.w_None))
- def descr_classmethod__new__(space, w_type, w_function):
+ def descr_classmethod__new__(space, w_subtype, w_function):
if not space.is_true(space.callable(w_function)):
typename = space.type(w_function).getname(space, '?')
raise operationerrfmt(space.w_TypeError,
"'%s' object is not callable", typename)
- return space.wrap(ClassMethod(w_function))
+ instance = space.allocate_instance(ClassMethod, w_subtype)
+ instance.__init__(w_function)
+ return space.wrap(instance)
class FunctionWithFixedCode(Function):
can_change_code = False
Modified: pypy/branch/arm-backend/pypy/interpreter/gateway.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/gateway.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/gateway.py Tue Nov 23 10:12:47 2010
@@ -28,7 +28,7 @@
# internal non-translatable parts:
import py
-class SignatureBuilder:
+class SignatureBuilder(object):
"NOT_RPYTHON"
def __init__(self, func=None, argnames=None, varargname=None,
kwargname=None, name = None):
@@ -51,7 +51,7 @@
#________________________________________________________________
-class UnwrapSpecRecipe:
+class UnwrapSpecRecipe(object):
"NOT_RPYTHON"
bases_order = [Wrappable, W_Root, ObjSpace, Arguments, object]
Modified: pypy/branch/arm-backend/pypy/interpreter/pycode.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/pycode.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/pycode.py Tue Nov 23 10:12:47 2010
@@ -117,6 +117,10 @@
self._compute_flatcall()
+ if self.space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import init_mapdict_cache
+ init_mapdict_cache(self)
+
def _freeze_(self):
if (self.magic == cpython_magic and
'__pypy__' not in sys.builtin_module_names):
@@ -253,6 +257,12 @@
tuple(self.co_freevars),
tuple(self.co_cellvars) )
+ def exec_host_bytecode(self, w_dict, w_globals, w_locals):
+ from pypy.interpreter.pyframe import CPythonFrame
+ frame = CPythonFrame(self.space, self, w_globals, None)
+ frame.setdictscope(w_locals)
+ return frame.run()
+
def dump(self):
"""A dis.dis() dump of the code object."""
co = self._to_code()
Modified: pypy/branch/arm-backend/pypy/interpreter/pyframe.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/pyframe.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/pyframe.py Tue Nov 23 10:12:47 2010
@@ -13,6 +13,7 @@
from pypy.rlib.rarithmetic import intmask
from pypy.rlib import jit, rstack
from pypy.tool import stdlib_opcode
+from pypy.tool.stdlib_opcode import host_bytecode_spec
# Define some opcodes used
g = globals()
@@ -140,7 +141,8 @@
# the following 'assert' is an annotation hint: it hides from
# the annotator all methods that are defined in PyFrame but
# overridden in the {,Host}FrameClass subclasses of PyFrame.
- assert isinstance(self, self.space.FrameClass)
+ assert (isinstance(self, self.space.FrameClass) or
+ not self.space.config.translating)
executioncontext = self.space.getexecutioncontext()
executioncontext.enter(self)
try:
@@ -634,6 +636,18 @@
return space.wrap(self.builtin is not space.builtin)
return space.w_False
+class CPythonFrame(PyFrame):
+ """
+ Execution of host (CPython) opcodes.
+ """
+
+ bytecode_spec = host_bytecode_spec
+ opcode_method_names = host_bytecode_spec.method_names
+ opcodedesc = host_bytecode_spec.opcodedesc
+ opdescmap = host_bytecode_spec.opdescmap
+ HAVE_ARGUMENT = host_bytecode_spec.HAVE_ARGUMENT
+
+
# ____________________________________________________________
def get_block_class(opname):
Modified: pypy/branch/arm-backend/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/pyopcode.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/pyopcode.py Tue Nov 23 10:12:47 2010
@@ -164,6 +164,9 @@
next_instr = block.handle(self, unroller)
return next_instr
+ def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
+ return self.space.call_function(w_func, w_typ, w_val, w_tb)
+
@jit.unroll_safe
def dispatch_bytecode(self, co_code, next_instr, ec):
space = self.space
@@ -710,9 +713,14 @@
def LOAD_ATTR(self, nameindex, next_instr):
"obj.attributename"
- w_attributename = self.getname_w(nameindex)
w_obj = self.popvalue()
- w_value = self.space.getattr(w_obj, w_attributename)
+ if (self.space.config.objspace.std.withmapdict
+ and not jit.we_are_jitted()):
+ from pypy.objspace.std.mapdict import LOAD_ATTR_caching
+ w_value = LOAD_ATTR_caching(self.getcode(), w_obj, nameindex)
+ else:
+ w_attributename = self.getname_w(nameindex)
+ w_value = self.space.getattr(w_obj, w_attributename)
self.pushvalue(w_value)
LOAD_ATTR._always_inline_ = True
@@ -868,23 +876,43 @@
def WITH_CLEANUP(self, oparg, next_instr):
# see comment in END_FINALLY for stack state
- w_exitfunc = self.popvalue()
- w_unroller = self.peekvalue(2)
+ # This opcode changed a lot between CPython versions
+ if (self.pycode.magic >= 0xa0df2ef
+ # Implementation since 2.7a0: 62191 (introduce SETUP_WITH)
+ or self.pycode.magic >= 0xa0df2d1):
+ # implementation since 2.6a1: 62161 (WITH_CLEANUP optimization)
+ self.popvalue()
+ self.popvalue()
+ w_unroller = self.popvalue()
+ w_exitfunc = self.popvalue()
+ self.pushvalue(w_unroller)
+ self.pushvalue(self.space.w_None)
+ self.pushvalue(self.space.w_None)
+ elif self.pycode.magic >= 0xa0df28c:
+ # Implementation since 2.5a0: 62092 (changed WITH_CLEANUP opcode)
+ w_exitfunc = self.popvalue()
+ w_unroller = self.peekvalue(2)
+ else:
+ raise NotImplementedError("WITH_CLEANUP for CPython <= 2.4")
+
unroller = self.space.interpclass_w(w_unroller)
if isinstance(unroller, SApplicationException):
operr = unroller.operr
- w_result = self.space.call_function(w_exitfunc,
- operr.w_type,
- operr.get_w_value(self.space),
- operr.application_traceback)
+ w_traceback = self.space.wrap(operr.application_traceback)
+ w_result = self.call_contextmanager_exit_function(
+ w_exitfunc,
+ operr.w_type,
+ operr.get_w_value(self.space),
+ w_traceback)
if self.space.is_true(w_result):
# __exit__() returned True -> Swallow the exception.
self.settopvalue(self.space.w_None, 2)
else:
- self.space.call_function(w_exitfunc,
- self.space.w_None,
- self.space.w_None,
- self.space.w_None)
+ self.call_contextmanager_exit_function(
+ w_exitfunc,
+ self.space.w_None,
+ self.space.w_None,
+ self.space.w_None)
@jit.unroll_safe
def call_function(self, oparg, w_star=None, w_starstar=None):
@@ -1112,7 +1140,7 @@
state_pack_variables = staticmethod(state_pack_variables)
-class FrameBlock:
+class FrameBlock(object):
"""Abstract base class for frame blocks from the blockstack,
used by the SETUP_XXX and POP_BLOCK opcodes."""
@@ -1225,10 +1253,18 @@
frame.pushvalue(frame.space.w_None)
return self.handlerposition # jump to the handler
+class WithBlock(FinallyBlock):
+
+ def really_handle(self, frame, unroller):
+ if (frame.space.full_exceptions and
+ isinstance(unroller, SApplicationException)):
+ unroller.operr.normalize_exception(frame.space)
+ return FinallyBlock.really_handle(self, frame, unroller)
block_classes = {'SETUP_LOOP': LoopBlock,
'SETUP_EXCEPT': ExceptBlock,
- 'SETUP_FINALLY': FinallyBlock}
+ 'SETUP_FINALLY': FinallyBlock,
+ 'SETUP_WITH': WithBlock}
### helpers written at the application-level ###
# Some of these functions are expected to be generally useful if other
Modified: pypy/branch/arm-backend/pypy/interpreter/pyparser/pytokenize.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/pyparser/pytokenize.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/pyparser/pytokenize.py Tue Nov 23 10:12:47 2010
@@ -21,40 +21,42 @@
__all__ = [ "tokenize" ]
# ______________________________________________________________________
-# Automatically generated DFA's (with one or two hand tweeks):
-pseudoStatesAccepts = [True, True, True, True, True, True, True, True,
- True, True, False, True, True, True, False, False,
- False, False, True, False, False, True, True, False,
- True, False, True, False, True, False, True, False,
- False, False, True, False, False, False, True]
-
-pseudoStates = [
- {'\t': 0, '\n': 13, '\x0c': 0, '\r': 14, ' ': 0, '!': 10,
- '"': 16, '#': 18, '%': 12, '&': 12,
- "'": 15, '(': 13, ')': 13, '*': 7,
- '+': 12, ',': 13, '-': 12, '.': 6,
- '/': 11, '0': 4, '1': 5, '2': 5,
- '3': 5, '4': 5, '5': 5, '6': 5,
- '7': 5, '8': 5, '9': 5, ':': 13,
- ';': 13, '<': 9, '=': 12, '>': 8,
- '@': 13, 'A': 1,
- 'B': 1, 'C': 1, 'D': 1, 'E': 1,
- 'F': 1, 'G': 1, 'H': 1, 'I': 1,
- 'J': 1, 'K': 1, 'L': 1, 'M': 1,
- 'N': 1, 'O': 1, 'P': 1, 'Q': 1,
- 'R': 2, 'S': 1, 'T': 1, 'U': 3,
- 'V': 1, 'W': 1, 'X': 1, 'Y': 1,
- 'Z': 1, '[': 13, '\\': 17, ']': 13,
- '^': 12, '_': 1, '`': 13, 'a': 1,
- 'b': 1, 'c': 1, 'd': 1, 'e': 1,
- 'f': 1, 'g': 1, 'h': 1, 'i': 1,
- 'j': 1, 'k': 1, 'l': 1, 'm': 1,
- 'n': 1, 'o': 1, 'p': 1, 'q': 1,
- 'r': 2, 's': 1, 't': 1, 'u': 3,
- 'v': 1, 'w': 1, 'x': 1, 'y': 1,
- 'z': 1, '{': 13, '|': 12, '}': 13,
- '~': 13},
+# Automatically generated DFA's
+accepts = [True, True, True, True, True, True, True, True,
+ True, True, False, True, True, True, True, False,
+ False, False, True, True, True, False, True,
+ False, True, False, True, False, False, True,
+ False, False, False, False, True, False, False,
+ False, True]
+states = [
+ # 0
+ {'\t': 0, '\n': 13, '\x0c': 0,
+ '\r': 14, ' ': 0, '!': 10, '"': 16,
+ '#': 18, '%': 12, '&': 12, "'": 15,
+ '(': 13, ')': 13, '*': 7, '+': 12,
+ ',': 13, '-': 12, '.': 6, '/': 11,
+ '0': 4, '1': 5, '2': 5, '3': 5,
+ '4': 5, '5': 5, '6': 5, '7': 5,
+ '8': 5, '9': 5, ':': 13, ';': 13,
+ '<': 9, '=': 12, '>': 8, '@': 13,
+ 'A': 1, 'B': 1, 'C': 1, 'D': 1,
+ 'E': 1, 'F': 1, 'G': 1, 'H': 1,
+ 'I': 1, 'J': 1, 'K': 1, 'L': 1,
+ 'M': 1, 'N': 1, 'O': 1, 'P': 1,
+ 'Q': 1, 'R': 2, 'S': 1, 'T': 1,
+ 'U': 3, 'V': 1, 'W': 1, 'X': 1,
+ 'Y': 1, 'Z': 1, '[': 13, '\\': 17,
+ ']': 13, '^': 12, '_': 1, '`': 13,
+ 'a': 1, 'b': 1, 'c': 1, 'd': 1,
+ 'e': 1, 'f': 1, 'g': 1, 'h': 1,
+ 'i': 1, 'j': 1, 'k': 1, 'l': 1,
+ 'm': 1, 'n': 1, 'o': 1, 'p': 1,
+ 'q': 1, 'r': 2, 's': 1, 't': 1,
+ 'u': 3, 'v': 1, 'w': 1, 'x': 1,
+ 'y': 1, 'z': 1, '{': 13, '|': 12,
+ '}': 13, '~': 13},
+ # 1
{'0': 1, '1': 1, '2': 1, '3': 1,
'4': 1, '5': 1, '6': 1, '7': 1,
'8': 1, '9': 1, 'A': 1, 'B': 1,
@@ -71,7 +73,7 @@
'p': 1, 'q': 1, 'r': 1, 's': 1,
't': 1, 'u': 1, 'v': 1, 'w': 1,
'x': 1, 'y': 1, 'z': 1},
-
+ # 2
{'"': 16, "'": 15, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
@@ -89,7 +91,7 @@
'r': 1, 's': 1, 't': 1, 'u': 1,
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
-
+ # 3
{'"': 16, "'": 15, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
@@ -107,158 +109,187 @@
'r': 2, 's': 1, 't': 1, 'u': 1,
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
-
- {'.': 24, '0': 22, '1': 22, '2': 22,
- '3': 22, '4': 22, '5': 22, '6': 22,
- '7': 22, '8': 23, '9': 23, 'E': 25,
- 'J': 13, 'L': 13, 'X': 21, 'e': 25,
- 'j': 13, 'l': 13, 'x': 21},
-
- {'.': 24, '0': 5, '1': 5, '2': 5,
+ # 4
+ {'.': 22, '0': 20, '1': 20, '2': 20,
+ '3': 20, '4': 20, '5': 20, '6': 20,
+ '7': 20, '8': 21, '9': 21, 'E': 23,
+ 'J': 13, 'L': 13, 'X': 19, 'e': 23,
+ 'j': 13, 'l': 13, 'x': 19},
+ # 5
+ {'.': 22, '0': 5, '1': 5, '2': 5,
'3': 5, '4': 5, '5': 5, '6': 5,
- '7': 5, '8': 5, '9': 5, 'E': 25,
- 'J': 13, 'L': 13, 'e': 25, 'j': 13,
+ '7': 5, '8': 5, '9': 5, 'E': 23,
+ 'J': 13, 'L': 13, 'e': 23, 'j': 13,
'l': 13},
-
- {'0': 26, '1': 26, '2': 26, '3': 26,
- '4': 26, '5': 26, '6': 26, '7': 26,
- '8': 26, '9': 26},
-
+ # 6
+ {'0': 24, '1': 24, '2': 24, '3': 24,
+ '4': 24, '5': 24, '6': 24, '7': 24,
+ '8': 24, '9': 24},
+ # 7
{'*': 12, '=': 13},
-
+ # 8
{'=': 13, '>': 12},
-
- {'=': 13, '<': 12, '>': 13},
-
+ # 9
+ {'<': 12, '=': 13, '>': 13},
+ # 10
{'=': 13},
-
- {'=': 13, '/': 12},
-
+ # 11
+ {'/': 12, '=': 13},
+ # 12
{'=': 13},
-
+ # 13
{},
-
+ # 14
{'\n': 13},
-
- {automata.DEFAULT: 19, '\n': 27, '\\': 29, "'": 28},
-
- {automata.DEFAULT: 20, '"': 30, '\n': 27, '\\': 31},
-
+ # 15
+ {automata.DEFAULT: 28, '\n': 25,
+ '\r': 25, "'": 26, '\\': 27},
+ # 16
+ {automata.DEFAULT: 31, '\n': 25,
+ '\r': 25, '"': 29, '\\': 30},
+ # 17
{'\n': 13, '\r': 14},
-
- {automata.DEFAULT: 18, '\n': 27, '\r': 27},
-
- {automata.DEFAULT: 19, '\n': 27, '\\': 29, "'": 13},
-
- {automata.DEFAULT: 20, '"': 13, '\n': 27, '\\': 31},
-
- {'0': 21, '1': 21, '2': 21, '3': 21,
- '4': 21, '5': 21, '6': 21, '7': 21,
- '8': 21, '9': 21, 'A': 21, 'B': 21,
- 'C': 21, 'D': 21, 'E': 21, 'F': 21,
- 'L': 13, 'a': 21, 'b': 21, 'c': 21,
- 'd': 21, 'e': 21, 'f': 21, 'l': 13},
-
- {'.': 24, '0': 22, '1': 22, '2': 22,
- '3': 22, '4': 22, '5': 22, '6': 22,
- '7': 22, '8': 23, '9': 23, 'E': 25,
- 'J': 13, 'L': 13, 'e': 25, 'j': 13,
+ # 18
+ {automata.DEFAULT: 18, '\n': 25, '\r': 25},
+ # 19
+ {'0': 19, '1': 19, '2': 19, '3': 19,
+ '4': 19, '5': 19, '6': 19, '7': 19,
+ '8': 19, '9': 19, 'A': 19, 'B': 19,
+ 'C': 19, 'D': 19, 'E': 19, 'F': 19,
+ 'L': 13, 'a': 19, 'b': 19, 'c': 19,
+ 'd': 19, 'e': 19, 'f': 19, 'l': 13},
+ # 20
+ {'.': 22, '0': 20, '1': 20, '2': 20,
+ '3': 20, '4': 20, '5': 20, '6': 20,
+ '7': 20, '8': 21, '9': 21, 'E': 23,
+ 'J': 13, 'L': 13, 'e': 23, 'j': 13,
'l': 13},
-
- {'.': 24, '0': 23, '1': 23, '2': 23,
- '3': 23, '4': 23, '5': 23, '6': 23,
- '7': 23, '8': 23, '9': 23, 'E': 25,
- 'J': 13, 'e': 25, 'j': 13},
-
- {'0': 24, '1': 24, '2': 24, '3': 24,
- '4': 24, '5': 24, '6': 24, '7': 24,
- '8': 24, '9': 24, 'E': 32, 'J': 13,
+ # 21
+ {'.': 22, '0': 21, '1': 21, '2': 21,
+ '3': 21, '4': 21, '5': 21, '6': 21,
+ '7': 21, '8': 21, '9': 21, 'E': 23,
+ 'J': 13, 'e': 23, 'j': 13},
+ # 22
+ {'0': 22, '1': 22, '2': 22, '3': 22,
+ '4': 22, '5': 22, '6': 22, '7': 22,
+ '8': 22, '9': 22, 'E': 32, 'J': 13,
'e': 32, 'j': 13},
-
+ # 23
{'+': 33, '-': 33, '0': 34, '1': 34,
'2': 34, '3': 34, '4': 34, '5': 34,
'6': 34, '7': 34, '8': 34, '9': 34},
-
- {'0': 26, '1': 26, '2': 26, '3': 26,
- '4': 26, '5': 26, '6': 26, '7': 26,
- '8': 26, '9': 26, 'E': 32, 'J': 13,
+ # 24
+ {'0': 24, '1': 24, '2': 24, '3': 24,
+ '4': 24, '5': 24, '6': 24, '7': 24,
+ '8': 24, '9': 24, 'E': 32, 'J': 13,
'e': 32, 'j': 13},
-
+ # 25
{},
-
+ # 26
{"'": 13},
-
+ # 27
{automata.DEFAULT: 35, '\n': 13, '\r': 14},
-
+ # 28
+ {automata.DEFAULT: 28, '\n': 25,
+ '\r': 25, "'": 13, '\\': 27},
+ # 29
{'"': 13},
-
+ # 30
{automata.DEFAULT: 36, '\n': 13, '\r': 14},
-
+ # 31
+ {automata.DEFAULT: 31, '\n': 25,
+ '\r': 25, '"': 13, '\\': 30},
+ # 32
{'+': 37, '-': 37, '0': 38, '1': 38,
'2': 38, '3': 38, '4': 38, '5': 38,
'6': 38, '7': 38, '8': 38, '9': 38},
-
-
+ # 33
{'0': 34, '1': 34, '2': 34, '3': 34,
'4': 34, '5': 34, '6': 34, '7': 34,
'8': 34, '9': 34},
-
+ # 34
{'0': 34, '1': 34, '2': 34, '3': 34,
'4': 34, '5': 34, '6': 34, '7': 34,
'8': 34, '9': 34, 'J': 13, 'j': 13},
-
- {automata.DEFAULT: 35, '\n': 27, '\\': 29, "'": 13},
-
- {automata.DEFAULT: 36, '"': 13, '\n': 27, '\\': 31},
-
+ # 35
+ {automata.DEFAULT: 35, '\n': 25,
+ '\r': 25, "'": 13, '\\': 27},
+ # 36
+ {automata.DEFAULT: 36, '\n': 25,
+ '\r': 25, '"': 13, '\\': 30},
+ # 37
{'0': 38, '1': 38, '2': 38, '3': 38,
'4': 38, '5': 38, '6': 38, '7': 38,
'8': 38, '9': 38},
-
+ # 38
{'0': 38, '1': 38, '2': 38, '3': 38,
'4': 38, '5': 38, '6': 38, '7': 38,
'8': 38, '9': 38, 'J': 13, 'j': 13},
]
+pseudoDFA = automata.DFA(states, accepts)
-pseudoDFA = automata.DFA(pseudoStates, pseudoStatesAccepts)
-
-double3StatesAccepts = [False, False, False, False, False, True]
-double3States = [
+accepts = [False, False, False, False, False, True]
+states = [
+ # 0
{automata.DEFAULT: 0, '"': 1, '\\': 2},
+ # 1
{automata.DEFAULT: 4, '"': 3, '\\': 2},
+ # 2
{automata.DEFAULT: 4},
+ # 3
{automata.DEFAULT: 4, '"': 5, '\\': 2},
+ # 4
{automata.DEFAULT: 4, '"': 1, '\\': 2},
+ # 5
{automata.DEFAULT: 4, '"': 5, '\\': 2},
]
-double3DFA = automata.NonGreedyDFA(double3States, double3StatesAccepts)
+double3DFA = automata.NonGreedyDFA(states, accepts)
-single3StatesAccepts = [False, False, False, False, False, True]
-single3States = [
- {automata.DEFAULT: 0, '\\': 2, "'": 1},
- {automata.DEFAULT: 4, '\\': 2, "'": 3},
+accepts = [False, False, False, False, False, True]
+states = [
+ # 0
+ {automata.DEFAULT: 0, "'": 1, '\\': 2},
+ # 1
+ {automata.DEFAULT: 4, "'": 3, '\\': 2},
+ # 2
{automata.DEFAULT: 4},
- {automata.DEFAULT: 4, '\\': 2, "'": 5},
- {automata.DEFAULT: 4, '\\': 2, "'": 1},
- {automata.DEFAULT: 4, '\\': 2, "'": 5},
+ # 3
+ {automata.DEFAULT: 4, "'": 5, '\\': 2},
+ # 4
+ {automata.DEFAULT: 4, "'": 1, '\\': 2},
+ # 5
+ {automata.DEFAULT: 4, "'": 5, '\\': 2},
]
-single3DFA = automata.NonGreedyDFA(single3States, single3StatesAccepts)
+single3DFA = automata.NonGreedyDFA(states, accepts)
-singleStatesAccepts = [False, True, False]
-singleStates = [
- {automata.DEFAULT: 0, '\\': 2, "'": 1},
+accepts = [False, True, False, False]
+states = [
+ # 0
+ {automata.DEFAULT: 0, "'": 1, '\\': 2},
+ # 1
{},
- {automata.DEFAULT: 0},
+ # 2
+ {automata.DEFAULT: 3},
+ # 3
+ {automata.DEFAULT: 3, "'": 1, '\\': 2},
]
-singleDFA = automata.DFA(singleStates, singleStatesAccepts)
+singleDFA = automata.DFA(states, accepts)
-doubleStatesAccepts = [False, True, False]
-doubleStates = [
+accepts = [False, True, False, False]
+states = [
+ # 0
{automata.DEFAULT: 0, '"': 1, '\\': 2},
+ # 1
{},
- {automata.DEFAULT: 0},
+ # 2
+ {automata.DEFAULT: 3},
+ # 3
+ {automata.DEFAULT: 3, '"': 1, '\\': 2},
]
-doubleDFA = automata.DFA(doubleStates, doubleStatesAccepts)
+doubleDFA = automata.DFA(states, accepts)
+
+#_______________________________________________________________________
+# End of automatically generated DFA's
endDFAs = {"'" : singleDFA,
'"' : doubleDFA,
Modified: pypy/branch/arm-backend/pypy/interpreter/pyparser/test/test_pyparse.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/pyparser/test/test_pyparse.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/pyparser/test/test_pyparse.py Tue Nov 23 10:12:47 2010
@@ -92,6 +92,9 @@
exc = py.test.raises(IndentationError, parse, input).value
assert exc.msg == "unindent does not match any outer indentation level"
+ def test_mac_newline(self):
+ self.parse("this_is\ra_mac\rfile")
+
def test_mode(self):
assert self.parse("x = 43*54").type == syms.file_input
tree = self.parse("43**54", "eval")
Modified: pypy/branch/arm-backend/pypy/interpreter/test/test_argument.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/test/test_argument.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/test/test_argument.py Tue Nov 23 10:12:47 2010
@@ -52,12 +52,17 @@
assert y == "d"
assert z == "e"
+class dummy_wrapped_dict(dict):
+ def __nonzero__(self):
+ raise NotImplementedError
class DummySpace(object):
def newtuple(self, items):
return tuple(items)
def is_true(self, obj):
+ if isinstance(obj, dummy_wrapped_dict):
+ return bool(dict(obj))
return bool(obj)
def fixedview(self, it):
@@ -229,7 +234,7 @@
kwds_w = dict(kwds[:i])
keywords = kwds_w.keys()
keywords_w = kwds_w.values()
- w_kwds = dict(kwds[i:])
+ w_kwds = dummy_wrapped_dict(kwds[i:])
if i == 2:
w_kwds = None
assert len(keywords) == len(keywords_w)
@@ -265,7 +270,7 @@
kwds_w = dict(kwds[:i])
keywords = kwds_w.keys()
keywords_w = kwds_w.values()
- w_kwds = dict(kwds[i:])
+ w_kwds = dummy_wrapped_dict(kwds[i:])
if i == 3:
w_kwds = None
args = Arguments(space, [1, 2], keywords, keywords_w, w_starstararg=w_kwds)
@@ -448,7 +453,11 @@
assert set(args.keywords) == set(['a', 'b'])
assert args.keywords_w[args.keywords.index('a')] == 2
assert args.keywords_w[args.keywords.index('b')] == 3
-
+
+ args = Arguments(space, [1])
+ w_args, w_kwds = args.topacked()
+ assert w_args == (1, )
+ assert not w_kwds
class TestErrorHandling(object):
def test_missing_args(self):
Modified: pypy/branch/arm-backend/pypy/interpreter/test/test_executioncontext.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/test/test_executioncontext.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/test/test_executioncontext.py Tue Nov 23 10:12:47 2010
@@ -19,7 +19,6 @@
space = self.space
a1 = DemoAction(space)
- space.actionflag.register_action(a1)
for i in range(20):
# assert does not raise:
space.appexec([], """():
@@ -50,7 +49,7 @@
space = self.space
a2 = DemoAction(space)
- space.actionflag.register_action(a2)
+ space.actionflag.register_periodic_action(a2, True)
try:
for i in range(500):
space.appexec([], """():
@@ -59,7 +58,8 @@
""")
except Finished:
pass
- assert space.sys.checkinterval / 10 < i < space.sys.checkinterval * 3
+ checkinterval = space.actionflag.getcheckinterval()
+ assert checkinterval / 10 < i < checkinterval * 1.1
def test_llprofile(self):
l = []
Modified: pypy/branch/arm-backend/pypy/interpreter/test/test_function.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/test/test_function.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/test/test_function.py Tue Nov 23 10:12:47 2010
@@ -482,6 +482,11 @@
raises(TypeError, m, MyInst(None))
raises(TypeError, m, MyInst(42))
+ def test_invalid_creation(self):
+ import new
+ def f(): pass
+ raises(TypeError, new.instancemethod, f, None)
+
class TestMethod:
def setup_method(self, method):
Modified: pypy/branch/arm-backend/pypy/interpreter/typedef.py
==============================================================================
--- pypy/branch/arm-backend/pypy/interpreter/typedef.py (original)
+++ pypy/branch/arm-backend/pypy/interpreter/typedef.py Tue Nov 23 10:12:47 2010
@@ -133,6 +133,13 @@
typedef = cls.typedef
if wants_dict and typedef.hasdict:
wants_dict = False
+ if config.objspace.std.withmapdict and not typedef.hasdict:
+ # mapdict only works if the type does not already have a dict
+ if wants_del:
+ parentcls = get_unique_interplevel_subclass(config, cls, True, True,
+ False, True)
+ return _usersubclswithfeature(config, parentcls, "del")
+ return _usersubclswithfeature(config, cls, "user", "dict", "weakref", "slots")
# Forest of if's - see the comment above.
if wants_del:
if wants_dict:
@@ -186,10 +193,21 @@
def add(Proto):
for key, value in Proto.__dict__.items():
- if not key.startswith('__') or key == '__del__':
+ if (not key.startswith('__') and not key.startswith('_mixin_')
+ or key == '__del__'):
+ if hasattr(value, "func_name"):
+ value = func_with_new_name(value, value.func_name)
body[key] = value
+ if (config.objspace.std.withmapdict and "dict" in features):
+ from pypy.objspace.std.mapdict import BaseMapdictObject, ObjectMixin
+ add(BaseMapdictObject)
+ add(ObjectMixin)
+ body["user_overridden_class"] = True
+ features = ()
+
if "user" in features: # generic feature needed by all subcls
+
class Proto(object):
user_overridden_class = True
@@ -245,16 +263,10 @@
return self.slots_w[index]
add(Proto)
- wantdict = "dict" in features
- if wantdict and config.objspace.std.withinlineddict:
- from pypy.objspace.std.objectobject import W_ObjectObject
- from pypy.objspace.std.inlinedict import make_mixin
- if supercls is W_ObjectObject:
- Mixin = make_mixin(config)
- add(Mixin)
- wantdict = False
-
- if wantdict:
+ if "dict" in features:
+ base_user_setup = supercls.user_setup.im_func
+ if "user_setup" in body:
+ base_user_setup = body["user_setup"]
class Proto(object):
def getdict(self):
return self.w__dict__
@@ -263,24 +275,13 @@
self.w__dict__ = check_new_dictionary(space, w_dict)
def user_setup(self, space, w_subtype):
- self.space = space
- self.w__class__ = w_subtype
self.w__dict__ = space.newdict(
instance=True, classofinstance=w_subtype)
- self.user_setup_slots(w_subtype.nslots)
+ base_user_setup(self, space, w_subtype)
def setclass(self, space, w_subtype):
# only used by descr_set___class__
self.w__class__ = w_subtype
- if space.config.objspace.std.withshadowtracking:
- self.w__dict__.set_shadows_anything()
-
- def getdictvalue_attr_is_in_class(self, space, name):
- w_dict = self.w__dict__
- if space.config.objspace.std.withshadowtracking:
- if not w_dict.shadows_anything():
- return None
- return space.finditem_str(w_dict, name)
add(Proto)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Tue Nov 23 10:12:47 2010
@@ -173,7 +173,8 @@
reg = r.lr
# XXX free this memory
# XXX allocate correct amount of memory
- mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)*6+9, flavor='raw')
+ mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)*6+9,
+ flavor='raw', track_allocation=False)
# Note, the actual frame depth is one less than the value stored in
# regalloc.frame_manager.frame_depth
self.encode32(mem, 0, regalloc.frame_manager.frame_depth - 1)
Modified: pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py Tue Nov 23 10:12:47 2010
@@ -23,11 +23,6 @@
mach = os.popen('uname -m', 'r').read().strip()
if not mach:
raise ProcessorAutodetectError, "cannot run 'uname -m'"
- if mach == 'x86_64':
- if sys.maxint == 2147483647:
- mach = 'x86' # it's a 64-bit processor but in 32-bits mode, maybe
- else:
- assert sys.maxint == 2 ** 63 - 1
try:
return {'i386': 'x86',
'i486': 'x86',
@@ -40,14 +35,28 @@
'armv7l': 'arm',
}[mach]
except KeyError:
- raise ProcessorAutodetectError, "unsupported processor '%s'" % mach
+ return mach
+
+def autodetect_main_model_and_size():
+ model = autodetect_main_model()
+ if sys.maxint == 2**31-1:
+ model += '_32'
+ elif sys.maxint == 2**63-1:
+ model += '_64'
+ else:
+ raise AssertionError, "bad value for sys.maxint"
+ return model
def autodetect():
model = autodetect_main_model()
- if model == 'x86':
- from pypy.jit.backend.x86.detect_sse2 import detect_sse2
- if not detect_sse2():
- model = 'x86-without-sse2'
+ if sys.maxint == 2**63-1:
+ model += '_64'
+ else:
+ assert sys.maxint == 2**31-1
+ if model == 'x86':
+ from pypy.jit.backend.x86.detect_sse2 import detect_sse2
+ if not detect_sse2():
+ model = 'x86-without-sse2'
return model
def getcpuclassname(backend_name="auto"):
Modified: pypy/branch/arm-backend/pypy/jit/backend/llgraph/llimpl.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llgraph/llimpl.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llgraph/llimpl.py Tue Nov 23 10:12:47 2010
@@ -10,7 +10,7 @@
BoxInt, BoxPtr, BoxObj, BoxFloat,
REF, INT, FLOAT)
from pypy.jit.codewriter import heaptracker
-from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.module.support import LLSupport, OOSupport
from pypy.rpython.llinterp import LLException
@@ -129,7 +129,7 @@
'arraylen_gc' : (('ref',), 'int'),
'call' : (('ref', 'varargs'), 'intorptr'),
'call_assembler' : (('varargs',), 'intorptr'),
- 'cond_call_gc_wb' : (('ptr',), None),
+ 'cond_call_gc_wb' : (('ptr', 'ptr'), None),
'oosend' : (('varargs',), 'intorptr'),
'oosend_pure' : (('varargs',), 'intorptr'),
'guard_true' : (('bool',), None),
@@ -152,7 +152,7 @@
'unicodegetitem' : (('ref', 'int'), 'int'),
'unicodesetitem' : (('ref', 'int', 'int'), 'int'),
'cast_ptr_to_int' : (('ref',), 'int'),
- 'debug_merge_point': (('ref',), None),
+ 'debug_merge_point': (('ref', 'int'), None),
'force_token' : ((), 'int'),
'call_may_force' : (('int', 'varargs'), 'intorptr'),
'guard_not_forced': ((), None),
@@ -305,12 +305,12 @@
loop = _from_opaque(loop)
loop.operations.append(Operation(opnum))
-def compile_add_descr(loop, ofs, type):
+def compile_add_descr(loop, ofs, type, arg_types):
from pypy.jit.backend.llgraph.runner import Descr
loop = _from_opaque(loop)
op = loop.operations[-1]
assert isinstance(type, str) and len(type) == 1
- op.descr = Descr(ofs, type)
+ op.descr = Descr(ofs, type, arg_types=arg_types)
def compile_add_loop_token(loop, descr):
if we_are_translated():
@@ -568,7 +568,7 @@
#
return _op_default_implementation
- def op_debug_merge_point(self, _, value):
+ def op_debug_merge_point(self, _, value, recdepth):
from pypy.jit.metainterp.warmspot import get_stats
loc = ConstPtr(value)._get_str()
get_stats().add_merge_point_location(loc)
@@ -801,7 +801,7 @@
else:
raise TypeError(x)
try:
- return _do_call_common(func, args_in_order)
+ return _do_call_common(func, args_in_order, calldescr)
except LLException, lle:
_last_exception = lle
d = {'v': None,
@@ -810,7 +810,7 @@
FLOAT: 0.0}
return d[calldescr.typeinfo]
- def op_cond_call_gc_wb(self, descr, a):
+ def op_cond_call_gc_wb(self, descr, a, b):
py.test.skip("cond_call_gc_wb not supported")
def op_oosend(self, descr, obj, *args):
@@ -1018,6 +1018,9 @@
if isinstance(TYPE, lltype.Ptr):
if isinstance(x, (int, long, llmemory.AddressAsInt)):
x = llmemory.cast_int_to_adr(x)
+ if TYPE is rffi.VOIDP:
+ # assume that we want a "C-style" cast, without typechecking the value
+ return rffi.cast(TYPE, x)
return llmemory.cast_adr_to_ptr(x, TYPE)
elif TYPE == llmemory.Address:
if isinstance(x, (int, long, llmemory.AddressAsInt)):
@@ -1411,10 +1414,26 @@
def do_call_pushfloat(x):
_call_args_f.append(x)
-def _do_call_common(f, args_in_order=None):
+kind2TYPE = {
+ 'i': lltype.Signed,
+ 'f': lltype.Float,
+ 'v': lltype.Void,
+ }
+
+def _do_call_common(f, args_in_order=None, calldescr=None):
ptr = llmemory.cast_int_to_adr(f).ptr
- FUNC = lltype.typeOf(ptr).TO
- ARGS = FUNC.ARGS
+ PTR = lltype.typeOf(ptr)
+ if PTR == rffi.VOIDP:
+ # it's a pointer to a C function, so we don't have a precise
+ # signature: create one from the descr
+ ARGS = map(kind2TYPE.get, calldescr.arg_types)
+ RESULT = kind2TYPE[calldescr.typeinfo]
+ FUNC = lltype.FuncType(ARGS, RESULT)
+ func_to_call = rffi.cast(lltype.Ptr(FUNC), ptr)
+ else:
+ FUNC = PTR.TO
+ ARGS = FUNC.ARGS
+ func_to_call = ptr._obj._callable
args = cast_call_args(ARGS, _call_args_i, _call_args_r, _call_args_f,
args_in_order)
del _call_args_i[:]
@@ -1426,7 +1445,7 @@
result = llinterp.eval_graph(ptr._obj.graph, args)
# ^^^ may raise, in which case we get an LLException
else:
- result = ptr._obj._callable(*args)
+ result = func_to_call(*args)
return result
def do_call_void(f):
Modified: pypy/branch/arm-backend/pypy/jit/backend/llgraph/runner.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llgraph/runner.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llgraph/runner.py Tue Nov 23 10:12:47 2010
@@ -118,13 +118,13 @@
self._descrs[key] = descr
return descr
- def compile_bridge(self, faildescr, inputargs, operations):
+ def compile_bridge(self, faildescr, inputargs, operations, log=True):
c = llimpl.compile_start()
self._compile_loop_or_bridge(c, inputargs, operations)
old, oldindex = faildescr._compiled_fail
llimpl.compile_redirect_fail(old, oldindex, c)
- def compile_loop(self, inputargs, operations, loopdescr):
+ def compile_loop(self, inputargs, operations, loopdescr, log=True):
"""In a real assembler backend, this should assemble the given
list of operations. Here we just generate a similar CompiledLoop
instance. The code here is RPython, whereas the code in llimpl
@@ -154,7 +154,7 @@
llimpl.compile_add(c, op.getopnum())
descr = op.getdescr()
if isinstance(descr, Descr):
- llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo)
+ llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo, descr.arg_types)
if isinstance(descr, history.LoopToken) and op.getopnum() != rop.JUMP:
llimpl.compile_add_loop_token(c, descr)
if self.is_oo and isinstance(descr, (OODescr, MethDescr)):
@@ -297,6 +297,18 @@
return self.getdescr(0, token[0], extrainfo=extrainfo,
arg_types=''.join(arg_types))
+ def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+ from pypy.jit.backend.llsupport.ffisupport import get_ffi_type_kind
+ arg_types = []
+ for arg in ffi_args:
+ kind = get_ffi_type_kind(arg)
+ if kind != history.VOID:
+ arg_types.append(kind)
+ reskind = get_ffi_type_kind(ffi_result)
+ return self.getdescr(0, reskind, extrainfo=extrainfo,
+ arg_types=''.join(arg_types))
+
+
def grab_exc_value(self):
return llimpl.grab_exc_value()
Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/descr.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llsupport/descr.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/descr.py Tue Nov 23 10:12:47 2010
@@ -82,6 +82,7 @@
_is_pointer_field = False # unless overridden by GcPtrFieldDescr
_is_float_field = False # unless overridden by FloatFieldDescr
+ _is_field_signed = False # unless overridden by XxxFieldDescr
def is_pointer_field(self):
return self._is_pointer_field
@@ -89,6 +90,9 @@
def is_float_field(self):
return self._is_float_field
+ def is_field_signed(self):
+ return self._is_field_signed
+
def repr_of_descr(self):
return '<%s %s %s>' % (self._clsname, self.name, self.offset)
@@ -105,7 +109,7 @@
def getFieldDescrClass(TYPE):
return getDescrClass(TYPE, BaseFieldDescr, GcPtrFieldDescr,
NonGcPtrFieldDescr, 'Field', 'get_field_size',
- '_is_float_field')
+ '_is_float_field', '_is_field_signed')
def get_field_descr(gccache, STRUCT, fieldname):
cache = gccache._cache_field
@@ -126,6 +130,7 @@
# ArrayDescrs
_A = lltype.GcArray(lltype.Signed) # a random gcarray
+_AF = lltype.GcArray(lltype.Float) # an array of C doubles
class BaseArrayDescr(AbstractDescr):
@@ -144,6 +149,7 @@
_is_array_of_pointers = False # unless overridden by GcPtrArrayDescr
_is_array_of_floats = False # unless overridden by FloatArrayDescr
+ _is_item_signed = False # unless overridden by XxxArrayDescr
def is_array_of_pointers(self):
return self._is_array_of_pointers
@@ -151,6 +157,9 @@
def is_array_of_floats(self):
return self._is_array_of_floats
+ def is_item_signed(self):
+ return self._is_item_signed
+
def repr_of_descr(self):
return '<%s>' % self._clsname
@@ -163,16 +172,21 @@
_clsname = 'GcPtrArrayDescr'
_is_array_of_pointers = True
-_CA = rffi.CArray(lltype.Signed)
+class FloatArrayDescr(BaseArrayDescr):
+ _clsname = 'FloatArrayDescr'
+ _is_array_of_floats = True
+ def get_base_size(self, translate_support_code):
+ basesize, _, _ = symbolic.get_array_token(_AF, translate_support_code)
+ return basesize
+ def get_item_size(self, translate_support_code):
+ return symbolic.get_size(lltype.Float, translate_support_code)
class BaseArrayNoLengthDescr(BaseArrayDescr):
def get_base_size(self, translate_support_code):
- basesize, _, _ = symbolic.get_array_token(_CA, translate_support_code)
- return basesize
+ return 0
def get_ofs_length(self, translate_support_code):
- _, _, ofslength = symbolic.get_array_token(_CA, translate_support_code)
- return ofslength
+ return -1
class NonGcPtrArrayNoLengthDescr(BaseArrayNoLengthDescr):
_clsname = 'NonGcPtrArrayNoLengthDescr'
@@ -184,14 +198,16 @@
_is_array_of_pointers = True
def getArrayDescrClass(ARRAY):
+ if ARRAY.OF is lltype.Float:
+ return FloatArrayDescr
return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr,
NonGcPtrArrayDescr, 'Array', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def getArrayNoLengthDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayNoLengthDescr, GcPtrArrayNoLengthDescr,
NonGcPtrArrayNoLengthDescr, 'ArrayNoLength', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def get_array_descr(gccache, ARRAY):
cache = gccache._cache_array
@@ -211,7 +227,8 @@
basesize, itemsize, ofslength = symbolic.get_array_token(ARRAY, False)
assert basesize == arraydescr.get_base_size(False)
assert itemsize == arraydescr.get_item_size(False)
- assert ofslength == arraydescr.get_ofs_length(False)
+ if not ARRAY._hints.get('nolength', False):
+ assert ofslength == arraydescr.get_ofs_length(False)
if isinstance(ARRAY, lltype.GcArray):
gccache.init_array_descr(ARRAY, arraydescr)
cache[ARRAY] = arraydescr
@@ -242,6 +259,9 @@
def get_result_size(self, translate_support_code):
raise NotImplementedError
+ def is_result_signed(self):
+ return False # unless overridden
+
def create_call_stub(self, rtyper, RESULT):
def process(c):
arg = 'args_%s[%d]' % (c, seen[c])
@@ -307,6 +327,30 @@
_return_type = history.INT
call_stub = staticmethod(lambda func, args_i, args_r, args_f: 0)
+ _is_result_signed = False # can be overridden in XxxCallDescr
+ def is_result_signed(self):
+ return self._is_result_signed
+
+class DynamicIntCallDescr(BaseIntCallDescr):
+ """
+ calldescr that works for every integer type, by explicitly passing it the
+ size of the result. Used only by get_call_descr_dynamic
+ """
+ _clsname = 'DynamicIntCallDescr'
+
+ def __init__(self, arg_classes, result_size, result_sign, extrainfo=None):
+ BaseIntCallDescr.__init__(self, arg_classes, extrainfo)
+ assert isinstance(result_sign, bool)
+ self._result_size = chr(result_size)
+ self._result_sign = result_sign
+
+ def get_result_size(self, translate_support_code):
+ return ord(self._result_size)
+
+ def is_result_signed(self):
+ return self._result_sign
+
+
class NonGcPtrCallDescr(BaseIntCallDescr):
_clsname = 'NonGcPtrCallDescr'
def get_result_size(self, translate_support_code):
@@ -341,7 +385,8 @@
return FloatCallDescr
return getDescrClass(RESULT, BaseIntCallDescr, GcPtrCallDescr,
NonGcPtrCallDescr, 'Call', 'get_result_size',
- Ellipsis) # <= floatattrname should not be used here
+ Ellipsis, # <= floatattrname should not be used here
+ '_is_result_signed')
def get_call_descr(gccache, ARGS, RESULT, extrainfo=None):
arg_classes = []
@@ -368,7 +413,8 @@
# ____________________________________________________________
def getDescrClass(TYPE, BaseDescr, GcPtrDescr, NonGcPtrDescr,
- nameprefix, methodname, floatattrname, _cache={}):
+ nameprefix, methodname, floatattrname, signedattrname,
+ _cache={}):
if isinstance(TYPE, lltype.Ptr):
if TYPE.TO._gckind == 'gc':
return GcPtrDescr
@@ -388,6 +434,8 @@
#
if TYPE is lltype.Float:
setattr(Descr, floatattrname, True)
+ elif TYPE is not lltype.Bool and rffi.cast(TYPE, -1) == -1:
+ setattr(Descr, signedattrname, True)
#
_cache[nameprefix, TYPE] = Descr
return Descr
Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/gc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llsupport/gc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/gc.py Tue Nov 23 10:12:47 2010
@@ -19,6 +19,7 @@
# ____________________________________________________________
class GcLLDescription(GcCache):
+ minimal_size_in_nursery = 0
def __init__(self, gcdescr, translator=None, rtyper=None):
GcCache.__init__(self, translator is not None, rtyper)
self.gcdescr = gcdescr
@@ -158,7 +159,7 @@
# used to avoid too many duplications in the GCREF_LISTs.
self.hashtable = lltype.malloc(self.HASHTABLE,
self.HASHTABLE_SIZE+1,
- flavor='raw')
+ flavor='raw', track_allocation=False)
dummy = lltype.direct_ptradd(lltype.direct_arrayitems(self.hashtable),
self.HASHTABLE_SIZE)
dummy = llmemory.cast_ptr_to_adr(dummy)
@@ -252,14 +253,15 @@
def _enlarge_gcmap(self):
newlength = 250 + self._gcmap_maxlength * 2
- newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw')
+ newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw',
+ track_allocation=False)
oldgcmap = self._gcmap
for i in range(self._gcmap_curlength):
newgcmap[i] = oldgcmap[i]
self._gcmap = newgcmap
self._gcmap_maxlength = newlength
if oldgcmap:
- lltype.free(oldgcmap, flavor='raw')
+ lltype.free(oldgcmap, flavor='raw', track_allocation=False)
def get_basic_shape(self, is_64_bit=False):
# XXX: Should this code even really know about stack frame layout of
@@ -308,7 +310,8 @@
# them inside bigger arrays) and we never try to share them.
length = len(shape)
compressed = lltype.malloc(self.CALLSHAPE_ARRAY, length,
- flavor='raw')
+ flavor='raw',
+ track_allocation=False) # memory leak
for i in range(length):
compressed[length-1-i] = rffi.cast(rffi.UCHAR, shape[i])
return llmemory.cast_ptr_to_adr(compressed)
@@ -384,6 +387,7 @@
(self.array_basesize, _, self.array_length_ofs) = \
symbolic.get_array_token(lltype.GcArray(lltype.Signed), True)
self.max_size_of_young_obj = self.GCClass.JIT_max_size_of_young_obj()
+ self.minimal_size_in_nursery=self.GCClass.JIT_minimal_size_in_nursery()
# make a malloc function, with three arguments
def malloc_basic(size, tid):
@@ -404,7 +408,7 @@
self.GC_MALLOC_BASIC = lltype.Ptr(lltype.FuncType(
[lltype.Signed, lltype.Signed], llmemory.GCREF))
self.WB_FUNCPTR = lltype.Ptr(lltype.FuncType(
- [llmemory.Address], lltype.Void))
+ [llmemory.Address, llmemory.Address], lltype.Void))
self.write_barrier_descr = WriteBarrierDescr(self)
#
def malloc_array(itemsize, tid, num_elem):
@@ -466,6 +470,7 @@
def malloc_fixedsize_slowpath(size):
if self.DEBUG:
random_usage_of_xmm_registers()
+ assert size >= self.minimal_size_in_nursery
try:
gcref = llop1.do_malloc_fixedsize_clear(llmemory.GCREF,
0, size, True, False, False)
@@ -550,7 +555,8 @@
# the GC, and call it immediately
llop1 = self.llop1
funcptr = llop1.get_write_barrier_failing_case(self.WB_FUNCPTR)
- funcptr(llmemory.cast_ptr_to_adr(gcref_struct))
+ funcptr(llmemory.cast_ptr_to_adr(gcref_struct),
+ llmemory.cast_ptr_to_adr(gcref_newptr))
def rewrite_assembler(self, cpu, operations):
# Perform two kinds of rewrites in parallel:
@@ -567,6 +573,9 @@
# GETFIELD_RAW from the array 'gcrefs.list'.
#
newops = []
+ # we can only remember one malloc since the next malloc can possibly
+ # collect
+ last_malloc = None
for op in operations:
if op.getopnum() == rop.DEBUG_MERGE_POINT:
continue
@@ -584,29 +593,39 @@
[ConstInt(addr)], box,
self.single_gcref_descr))
op.setarg(i, box)
+ if op.is_malloc():
+ last_malloc = op.result
+ elif op.can_malloc():
+ last_malloc = None
# ---------- write barrier for SETFIELD_GC ----------
if op.getopnum() == rop.SETFIELD_GC:
- v = op.getarg(1)
- if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and
- bool(v.value)): # store a non-NULL
- self._gen_write_barrier(newops, op.getarg(0))
- op = op.copy_and_change(rop.SETFIELD_RAW)
+ val = op.getarg(0)
+ # no need for a write barrier in the case of previous malloc
+ if val is not last_malloc:
+ v = op.getarg(1)
+ if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and
+ bool(v.value)): # store a non-NULL
+ self._gen_write_barrier(newops, op.getarg(0), v)
+ op = op.copy_and_change(rop.SETFIELD_RAW)
# ---------- write barrier for SETARRAYITEM_GC ----------
if op.getopnum() == rop.SETARRAYITEM_GC:
- v = op.getarg(2)
- if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and
- bool(v.value)): # store a non-NULL
- # XXX detect when we should produce a
- # write_barrier_from_array
- self._gen_write_barrier(newops, op.getarg(0))
- op = op.copy_and_change(rop.SETARRAYITEM_RAW)
+ val = op.getarg(0)
+ # no need for a write barrier in the case of previous malloc
+ if val is not last_malloc:
+ v = op.getarg(2)
+ if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and
+ bool(v.value)): # store a non-NULL
+ # XXX detect when we should produce a
+ # write_barrier_from_array
+ self._gen_write_barrier(newops, op.getarg(0), v)
+ op = op.copy_and_change(rop.SETARRAYITEM_RAW)
# ----------
newops.append(op)
del operations[:]
operations.extend(newops)
- def _gen_write_barrier(self, newops, v_base):
- args = [v_base]
+ def _gen_write_barrier(self, newops, v_base, v_value):
+ args = [v_base, v_value]
newops.append(ResOperation(rop.COND_CALL_GC_WB, args, None,
descr=self.write_barrier_descr))
Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/llmodel.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llsupport/llmodel.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/llmodel.py Tue Nov 23 10:12:47 2010
@@ -17,6 +17,7 @@
from pypy.jit.backend.llsupport.descr import get_call_descr
from pypy.jit.backend.llsupport.descr import BaseIntCallDescr, GcPtrCallDescr
from pypy.jit.backend.llsupport.descr import FloatCallDescr, VoidCallDescr
+from pypy.jit.backend.llsupport.ffisupport import get_call_descr_dynamic
from pypy.rpython.annlowlevel import cast_instance_to_base_ptr
@@ -82,7 +83,8 @@
# read back by the machine code reading at the address given by
# pos_exception() and pos_exc_value().
_exception_emulator = lltype.malloc(rffi.CArray(lltype.Signed), 2,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self._exception_emulator = _exception_emulator
def _store_exception(lle):
@@ -210,7 +212,8 @@
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
- return ofs, size
+ sign = fielddescr.is_field_signed()
+ return ofs, size, sign
unpack_fielddescr_size._always_inline_ = True
def arraydescrof(self, A):
@@ -225,12 +228,16 @@
assert isinstance(arraydescr, BaseArrayDescr)
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
- return ofs, size
+ sign = arraydescr.is_item_signed()
+ return ofs, size, sign
unpack_arraydescr_size._always_inline_ = True
def calldescrof(self, FUNC, ARGS, RESULT, extrainfo=None):
return get_call_descr(self.gc_ll_descr, ARGS, RESULT, extrainfo)
+ def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+ return get_call_descr_dynamic(ffi_args, ffi_result, extrainfo)
+
def get_overflow_error(self):
ovf_vtable = self.cast_adr_to_int(self._ovf_error_vtable)
ovf_inst = lltype.cast_opaque_ptr(llmemory.GCREF,
@@ -252,15 +259,21 @@
@specialize.argtype(2)
def bh_getarrayitem_gc_i(self, arraydescr, gcref, itemindex):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- items = rffi.cast(rffi.CArrayPtr(TYPE), items)
- val = items[itemindex]
+ if sign:
+ items = rffi.cast(rffi.CArrayPtr(STYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ items = rffi.cast(rffi.CArrayPtr(UTYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -285,10 +298,10 @@
@specialize.argtype(2)
def bh_setarrayitem_gc_i(self, arraydescr, gcref, itemindex, newvalue):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
items = rffi.cast(rffi.CArrayPtr(TYPE), items)
items[itemindex] = rffi.cast(TYPE, newvalue)
@@ -339,14 +352,22 @@
@specialize.argtype(1)
def _base_do_getfield_i(self, struct, fielddescr):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- val = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)[0]
+ # Note that in the common case where size==sizeof(Signed),
+ # both cases of what follows are doing the same thing.
+ # But gcc is clever enough to figure this out :-)
+ if sign:
+ val = rffi.cast(rffi.CArrayPtr(STYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ val = rffi.cast(rffi.CArrayPtr(UTYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -378,10 +399,10 @@
@specialize.argtype(1)
def _base_do_setfield_i(self, struct, fielddescr, newvalue):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
fieldptr = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)
fieldptr[0] = rffi.cast(TYPE, newvalue)
Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py Tue Nov 23 10:12:47 2010
@@ -365,8 +365,9 @@
arg = op.getarg(j)
if isinstance(arg, Box):
if arg not in start_live:
- print "Bogus arg in operation %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in operation %d at %d" %
+ (op.getopnum(), i))
+
longevity[arg] = (start_live[arg], i)
if op.is_guard():
for arg in op.getfailargs():
@@ -374,8 +375,8 @@
continue
assert isinstance(arg, Box)
if arg not in start_live:
- print "Bogus arg in guard %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in guard %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
for arg in inputargs:
if arg not in longevity:
@@ -394,3 +395,6 @@
loop_consts[inputargs[i]] = i
return loop_consts
+def not_implemented(msg):
+ os.write(2, '[llsupport/regalloc] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/symbolic.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llsupport/symbolic.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/symbolic.py Tue Nov 23 10:12:47 2010
@@ -69,8 +69,9 @@
SIZEOF_INT = get_size(rffi.INT, False)
SIZEOF_FLOAT = get_size(lltype.Float, False)
-unroll_basic_sizes = unrolling_iterable([(lltype.Signed, WORD),
- (lltype.Char, SIZEOF_CHAR),
- (rffi.SHORT, SIZEOF_SHORT),
- (rffi.INT, SIZEOF_INT)])
+unroll_basic_sizes = unrolling_iterable([
+ (lltype.Signed, lltype.Unsigned, WORD),
+ (rffi.SIGNEDCHAR, lltype.Char, SIZEOF_CHAR),
+ (rffi.SHORT, rffi.USHORT, SIZEOF_SHORT),
+ (rffi.INT, rffi.UINT, SIZEOF_INT)])
# does not contain Float ^^^ which must be special-cased
Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_descr.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_descr.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_descr.py Tue Nov 23 10:12:47 2010
@@ -5,6 +5,7 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr
from pypy.jit.metainterp import history
+import struct
def test_get_size_descr():
c0 = GcCache(False)
@@ -83,6 +84,18 @@
assert descr_f.is_float_field()
+def test_get_field_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr_x = get_field_descr(c2, S, 'x')
+ assert descr_x.is_field_signed() == signed
+
+
def test_get_array_descr():
U = lltype.Struct('U')
T = lltype.GcStruct('T')
@@ -118,11 +131,13 @@
assert not descr3.is_array_of_floats()
assert descr4.is_array_of_floats()
#
- WORD = rffi.sizeof(lltype.Signed)
- assert descr1.get_base_size(False) == WORD
- assert descr2.get_base_size(False) == WORD
- assert descr3.get_base_size(False) == WORD
- assert descr4.get_base_size(False) == WORD
+ def get_alignment(code):
+ # Retrieve default alignment for the compiler/platform
+ return struct.calcsize('l' + code) - struct.calcsize(code)
+ assert descr1.get_base_size(False) == get_alignment('c')
+ assert descr2.get_base_size(False) == get_alignment('p')
+ assert descr3.get_base_size(False) == get_alignment('p')
+ assert descr4.get_base_size(False) == get_alignment('d')
assert descr1.get_ofs_length(False) == 0
assert descr2.get_ofs_length(False) == 0
assert descr3.get_ofs_length(False) == 0
@@ -164,6 +179,25 @@
assert descr.get_base_size(False) == 0
assert descr.get_ofs_length(False) == -1
+
+def test_get_array_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, A)
+ assert arraydescr.is_item_signed() == signed
+ #
+ RA = rffi.CArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, RA)
+ assert arraydescr.is_item_signed() == signed
+
+
def test_get_call_descr_not_translated():
c0 = GcCache(False)
descr1 = get_call_descr(c0, [lltype.Char, lltype.Signed], lltype.Char)
@@ -219,6 +253,17 @@
extrainfo = descr3.get_extra_info()
assert extrainfo is None
+def test_get_call_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr1 = get_call_descr(c2, [], RESTYPE)
+ assert descr1.is_result_signed() == signed
+
def test_repr_of_descr():
c0 = GcCache(False)
Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_gc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_gc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_gc.py Tue Nov 23 10:12:47 2010
@@ -6,7 +6,9 @@
from pypy.jit.backend.llsupport.gc import *
from pypy.jit.backend.llsupport import symbolic
from pypy.jit.metainterp.gc import get_description
-
+from pypy.jit.tool.oparser import parse
+from pypy.rpython.lltypesystem.rclass import OBJECT, OBJECT_VTABLE
+from pypy.jit.metainterp.test.test_optimizeopt import equaloplists
def test_boehm():
gc_ll_descr = GcLLDescr_boehm(None, None, None)
@@ -114,7 +116,7 @@
assert gcrootmap._gcmap[i*2+1] == expected_shapeaddr[i]
-class FakeLLOp:
+class FakeLLOp(object):
def __init__(self):
self.record = []
@@ -141,26 +143,26 @@
repr(offset_to_length), p))
return p
- def _write_barrier_failing_case(self, adr_struct):
- self.record.append(('barrier', adr_struct))
+ def _write_barrier_failing_case(self, adr_struct, adr_newptr):
+ self.record.append(('barrier', adr_struct, adr_newptr))
def get_write_barrier_failing_case(self, FPTRTYPE):
return llhelper(FPTRTYPE, self._write_barrier_failing_case)
-class TestFramework:
+class TestFramework(object):
gc = 'hybrid'
def setup_method(self, meth):
- class config_:
- class translation:
+ class config_(object):
+ class translation(object):
gc = self.gc
gcrootfinder = 'asmgcc'
gctransformer = 'framework'
gcremovetypeptr = False
- class FakeTranslator:
+ class FakeTranslator(object):
config = config_
- class FakeCPU:
+ class FakeCPU(object):
def cast_adr_to_int(self, adr):
ptr = llmemory.cast_adr_to_ptr(adr, gc_ll_descr.WB_FUNCPTR)
assert ptr._obj._callable == llop1._write_barrier_failing_case
@@ -239,6 +241,7 @@
s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
r_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, r)
s_adr = llmemory.cast_ptr_to_adr(s)
+ r_adr = llmemory.cast_ptr_to_adr(r)
#
s_hdr.tid &= ~gc_ll_descr.GCClass.JIT_WB_IF_FLAG
gc_ll_descr.do_write_barrier(s_gcref, r_gcref)
@@ -246,7 +249,7 @@
#
s_hdr.tid |= gc_ll_descr.GCClass.JIT_WB_IF_FLAG
gc_ll_descr.do_write_barrier(s_gcref, r_gcref)
- assert self.llop1.record == [('barrier', s_adr)]
+ assert self.llop1.record == [('barrier', s_adr, r_adr)]
def test_gen_write_barrier(self):
gc_ll_descr = self.gc_ll_descr
@@ -254,11 +257,13 @@
#
newops = []
v_base = BoxPtr()
- gc_ll_descr._gen_write_barrier(newops, v_base)
+ v_value = BoxPtr()
+ gc_ll_descr._gen_write_barrier(newops, v_base, v_value)
assert llop1.record == []
assert len(newops) == 1
assert newops[0].getopnum() == rop.COND_CALL_GC_WB
assert newops[0].getarg(0) == v_base
+ assert newops[0].getarg(1) == v_value
assert newops[0].result is None
wbdescr = newops[0].getdescr()
assert isinstance(wbdescr.jit_wb_if_flag, int)
@@ -267,7 +272,7 @@
def test_get_rid_of_debug_merge_point(self):
operations = [
- ResOperation(rop.DEBUG_MERGE_POINT, ['dummy'], None),
+ ResOperation(rop.DEBUG_MERGE_POINT, ['dummy', 2], None),
]
gc_ll_descr = self.gc_ll_descr
gc_ll_descr.rewrite_assembler(None, operations)
@@ -275,11 +280,11 @@
def test_rewrite_assembler_1(self):
# check rewriting of ConstPtrs
- class MyFakeCPU:
+ class MyFakeCPU(object):
def cast_adr_to_int(self, adr):
assert adr == "some fake address"
return 43
- class MyFakeGCRefList:
+ class MyFakeGCRefList(object):
def get_address_of_gcref(self, s_gcref1):
assert s_gcref1 == s_gcref
return "some fake address"
@@ -308,10 +313,10 @@
def test_rewrite_assembler_1_cannot_move(self):
# check rewriting of ConstPtrs
- class MyFakeCPU:
+ class MyFakeCPU(object):
def cast_adr_to_int(self, adr):
xxx # should not be called
- class MyFakeGCRefList:
+ class MyFakeGCRefList(object):
def get_address_of_gcref(self, s_gcref1):
seen.append(s_gcref1)
assert s_gcref1 == s_gcref
@@ -358,6 +363,7 @@
#
assert operations[0].getopnum() == rop.COND_CALL_GC_WB
assert operations[0].getarg(0) == v_base
+ assert operations[0].getarg(1) == v_value
assert operations[0].result is None
#
assert operations[1].getopnum() == rop.SETFIELD_RAW
@@ -381,6 +387,7 @@
#
assert operations[0].getopnum() == rop.COND_CALL_GC_WB
assert operations[0].getarg(0) == v_base
+ assert operations[0].getarg(1) == v_value
assert operations[0].result is None
#
assert operations[1].getopnum() == rop.SETARRAYITEM_RAW
@@ -389,6 +396,68 @@
assert operations[1].getarg(2) == v_value
assert operations[1].getdescr() == array_descr
+ def test_rewrite_assembler_initialization_store(self):
+ S = lltype.GcStruct('S', ('parent', OBJECT),
+ ('x', lltype.Signed))
+ s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True)
+ xdescr = get_field_descr(self.gc_ll_descr, S, 'x')
+ ops = parse("""
+ [p1]
+ p0 = new_with_vtable(ConstClass(s_vtable))
+ setfield_gc(p0, p1, descr=xdescr)
+ jump()
+ """, namespace=locals())
+ expected = parse("""
+ [p1]
+ p0 = new_with_vtable(ConstClass(s_vtable))
+ # no write barrier
+ setfield_gc(p0, p1, descr=xdescr)
+ jump()
+ """, namespace=locals())
+ self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations)
+ equaloplists(ops.operations, expected.operations)
+
+ def test_rewrite_assembler_initialization_store_2(self):
+ S = lltype.GcStruct('S', ('parent', OBJECT),
+ ('x', lltype.Signed))
+ s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True)
+ wbdescr = self.gc_ll_descr.write_barrier_descr
+ xdescr = get_field_descr(self.gc_ll_descr, S, 'x')
+ ops = parse("""
+ [p1]
+ p0 = new_with_vtable(ConstClass(s_vtable))
+ p3 = new_with_vtable(ConstClass(s_vtable))
+ setfield_gc(p0, p1, descr=xdescr)
+ jump()
+ """, namespace=locals())
+ expected = parse("""
+ [p1]
+ p0 = new_with_vtable(ConstClass(s_vtable))
+ p3 = new_with_vtable(ConstClass(s_vtable))
+ cond_call_gc_wb(p0, p1, descr=wbdescr)
+ setfield_raw(p0, p1, descr=xdescr)
+ jump()
+ """, namespace=locals())
+ self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations)
+ equaloplists(ops.operations, expected.operations)
+
+ def test_rewrite_assembler_initialization_store_3(self):
+ A = lltype.GcArray(lltype.Ptr(lltype.GcStruct('S')))
+ arraydescr = get_array_descr(self.gc_ll_descr, A)
+ ops = parse("""
+ [p1]
+ p0 = new_array(3, descr=arraydescr)
+ setarrayitem_gc(p0, 0, p1, descr=arraydescr)
+ jump()
+ """, namespace=locals())
+ expected = parse("""
+ [p1]
+ p0 = new_array(3, descr=arraydescr)
+ setarrayitem_gc(p0, 0, p1, descr=arraydescr)
+ jump()
+ """, namespace=locals())
+ self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations)
+ equaloplists(ops.operations, expected.operations)
class TestFrameworkMiniMark(TestFramework):
gc = 'minimark'
Modified: pypy/branch/arm-backend/pypy/jit/backend/model.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/model.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/model.py Tue Nov 23 10:12:47 2010
@@ -33,14 +33,14 @@
pass
- def compile_loop(self, inputargs, operations, looptoken):
+ def compile_loop(self, inputargs, operations, looptoken, log=True):
"""Assemble the given loop.
Extra attributes should be put in the LoopToken to
point to the compiled loop in assembler.
"""
raise NotImplementedError
- def compile_bridge(self, faildescr, inputargs, operations):
+ def compile_bridge(self, faildescr, inputargs, operations, log=True):
"""Assemble the bridge.
The FailDescr is the descr of the original guard that failed.
"""
Modified: pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py Tue Nov 23 10:12:47 2010
@@ -9,12 +9,13 @@
ConstObj, BoxFloat, ConstFloat)
from pypy.jit.metainterp.resoperation import ResOperation, rop
from pypy.jit.metainterp.typesystem import deref
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi, rclass
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.llinterp import LLException
from pypy.jit.codewriter import heaptracker
+from pypy.rlib.rarithmetic import intmask
class Runner(object):
@@ -421,6 +422,7 @@
assert x == 3.5 - 42
def test_call(self):
+ from pypy.rlib.libffi import types
def func_int(a, b):
return a + b
@@ -428,23 +430,31 @@
return chr(ord(c) + ord(c1))
functions = [
- (func_int, lltype.Signed, 655360),
- (func_int, rffi.SHORT, 1213),
- (func_char, lltype.Char, 12)
+ (func_int, lltype.Signed, types.sint, 655360),
+ (func_int, rffi.SHORT, types.sint16, 1213),
+ (func_char, lltype.Char, types.uchar, 12)
]
- for func, TP, num in functions:
+ for func, TP, ffi_type, num in functions:
cpu = self.cpu
#
FPTR = self.Ptr(self.FuncType([TP, TP], TP))
func_ptr = llhelper(FPTR, func)
FUNC = deref(FPTR)
- calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
funcbox = self.get_funcbox(cpu, func_ptr)
+ # first, try it with the "normal" calldescr
+ calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
res = self.execute_operation(rop.CALL,
[funcbox, BoxInt(num), BoxInt(num)],
'int', descr=calldescr)
assert res.value == 2 * num
+ # then, try it with the dynamic calldescr
+ dyn_calldescr = cpu.calldescrof_dynamic([ffi_type, ffi_type], ffi_type)
+ res = self.execute_operation(rop.CALL,
+ [funcbox, BoxInt(num), BoxInt(num)],
+ 'int', descr=dyn_calldescr)
+ assert res.value == 2 * num
+
if cpu.supports_floats:
def func(f0, f1, f2, f3, f4, f5, f6, i0, i1, f7, f8, f9):
@@ -507,6 +517,24 @@
'int', descr=calldescr)
assert res.value == func_ints(*args)
+ def test_call_to_c_function(self):
+ from pypy.rlib.libffi import CDLL, types, ArgChain
+ from pypy.rpython.lltypesystem.ll2ctypes import libc_name
+ libc = CDLL(libc_name)
+ c_tolower = libc.getpointer('tolower', [types.uchar], types.sint)
+ argchain = ArgChain().arg(ord('A'))
+ assert c_tolower.call(argchain, rffi.INT) == ord('a')
+
+ func_adr = llmemory.cast_ptr_to_adr(c_tolower.funcsym)
+ funcbox = ConstInt(heaptracker.adr2int(func_adr))
+ calldescr = self.cpu.calldescrof_dynamic([types.uchar], types.sint)
+ res = self.execute_operation(rop.CALL,
+ [funcbox, BoxInt(ord('A'))],
+ 'int',
+ descr=calldescr)
+ assert res.value == ord('a')
+
+
def test_field_basic(self):
t_box, T_box = self.alloc_instance(self.T)
fielddescr = self.cpu.fielddescrof(self.S, 'value')
@@ -833,6 +861,23 @@
length_box], 'void')
assert self.look_string(r_box) == "!??cdef?!"
+ def test_copyunicodecontent(self):
+ s_box = self.alloc_unicode(u"abcdef")
+ for s_box in [s_box, s_box.constbox()]:
+ for srcstart_box in [BoxInt(2), ConstInt(2)]:
+ for dststart_box in [BoxInt(3), ConstInt(3)]:
+ for length_box in [BoxInt(4), ConstInt(4)]:
+ for r_box_is_const in [False, True]:
+ r_box = self.alloc_unicode(u"!???????!")
+ if r_box_is_const:
+ r_box = r_box.constbox()
+ self.execute_operation(rop.COPYUNICODECONTENT,
+ [s_box, r_box,
+ srcstart_box,
+ dststart_box,
+ length_box], 'void')
+ assert self.look_unicode(r_box) == u"!??cdef?!"
+
def test_do_unicode_basic(self):
u = self.cpu.bh_newunicode(5)
self.cpu.bh_unicodesetitem(u, 4, 123)
@@ -1227,6 +1272,10 @@
u_box = BoxPtr(lltype.cast_opaque_ptr(llmemory.GCREF, u))
return u_box
+ def look_unicode(self, unicode_box):
+ u = unicode_box.getref(lltype.Ptr(rstr.UNICODE))
+ return u''.join(u.chars)
+
def test_casts(self):
py.test.skip("xxx fix or kill")
@@ -1283,6 +1332,7 @@
descr=fd)
res = self.execute_operation(get_op, [s_box], 'int', descr=fd)
assert res.getint() == 32
+ lltype.free(s, flavor='raw')
def test_new_with_vtable(self):
cpu = self.cpu
@@ -1427,12 +1477,12 @@
assert not excvalue
def test_cond_call_gc_wb(self):
- def func_void(a):
- record.append(a)
+ def func_void(a, b):
+ record.append((a, b))
record = []
#
S = lltype.GcStruct('S', ('tid', lltype.Signed))
- FUNC = self.FuncType([lltype.Ptr(S)], lltype.Void)
+ FUNC = self.FuncType([lltype.Ptr(S), lltype.Signed], lltype.Void)
func_ptr = llhelper(lltype.Ptr(FUNC), func_void)
funcbox = self.get_funcbox(self.cpu, func_ptr)
class WriteBarrierDescr(AbstractDescr):
@@ -1453,10 +1503,10 @@
sgcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
del record[:]
self.execute_operation(rop.COND_CALL_GC_WB,
- [BoxPtr(sgcref)],
+ [BoxPtr(sgcref), ConstInt(-2121)],
'void', descr=WriteBarrierDescr())
if cond:
- assert record == [s]
+ assert record == [(s, -2121)]
else:
assert record == []
@@ -2001,6 +2051,200 @@
assert self.cpu.get_latest_value_float(0) == 13.5
assert called
+ def test_short_result_of_getfield_direct(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getfield_gc_i(lltype.cast_opaque_ptr(llmemory.GCREF, s),
+ descrfld_x)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getfield_compiled(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
+ res = self.execute_operation(rop.GETFIELD_GC, [BoxPtr(s_gcref)],
+ 'int', descr=descrfld_x)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getarrayitem_gc_i(
+ descrarray, lltype.cast_opaque_ptr(llmemory.GCREF, a), 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getarrayitem_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, a)
+ res = self.execute_operation(rop.GETARRAYITEM_GC,
+ [BoxPtr(a_gcref), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_raw_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ x = cpu.bh_getarrayitem_raw_i(descrarray, a_rawint, 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_getarrayitem_raw_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ res = self.execute_operation(rop.GETARRAYITEM_RAW,
+ [BoxInt(a_rawint), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_call_direct(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(
+ separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """],
+ export_symbols=['fn_test_result_of_call'])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ x = self.cpu.bh_call_i(self.get_funcbox(self.cpu, f).value,
+ calldescr, [value], None, None)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_call_compiled(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(
+ separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """],
+ export_symbols=['fn_test_result_of_call'])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ funcbox = self.get_funcbox(self.cpu, f)
+ res = self.execute_operation(rop.CALL, [funcbox, BoxInt(value)],
+ 'int', descr=calldescr)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
class OOtypeBackendTest(BaseBackendTest):
Modified: pypy/branch/arm-backend/pypy/jit/backend/test/test_ll_random.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/test/test_ll_random.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/test/test_ll_random.py Tue Nov 23 10:12:47 2010
@@ -386,6 +386,20 @@
v_string = self.get_string(builder, r)
builder.do(self.opnum, [v_string])
+class AbstractCopyContentOperation(AbstractStringOperation):
+ def produce_into(self, builder, r):
+ v_srcstring = self.get_string(builder, r)
+ v_dststring = self.get_string(builder, r)
+ if v_srcstring.value == v_dststring.value: # because it's not a
+ raise test_random.CannotProduceOperation # memmove(), but memcpy()
+ srclen = len(v_srcstring.getref(self.ptr).chars)
+ dstlen = len(v_dststring.getref(self.ptr).chars)
+ v_length = builder.get_index(min(srclen, dstlen), r)
+ v_srcstart = builder.get_index(srclen - v_length.value + 1, r)
+ v_dststart = builder.get_index(dstlen - v_length.value + 1, r)
+ builder.do(self.opnum, [v_srcstring, v_dststring,
+ v_srcstart, v_dststart, v_length])
+
class StrGetItemOperation(AbstractGetItemOperation, _StrOperation):
pass
@@ -404,6 +418,13 @@
class UnicodeLenOperation(AbstractStringLenOperation, _UnicodeOperation):
pass
+class CopyStrContentOperation(AbstractCopyContentOperation, _StrOperation):
+ pass
+
+class CopyUnicodeContentOperation(AbstractCopyContentOperation,
+ _UnicodeOperation):
+ pass
+
# there are five options in total:
# 1. non raising call and guard_no_exception
@@ -577,6 +598,8 @@
OPERATIONS.append(UnicodeSetItemOperation(rop.UNICODESETITEM))
OPERATIONS.append(StrLenOperation(rop.STRLEN))
OPERATIONS.append(UnicodeLenOperation(rop.UNICODELEN))
+ OPERATIONS.append(CopyStrContentOperation(rop.COPYSTRCONTENT))
+ OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT))
for i in range(2):
OPERATIONS.append(GuardClassOperation(rop.GUARD_CLASS))
Modified: pypy/branch/arm-backend/pypy/jit/backend/test/test_random.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/test/test_random.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/test/test_random.py Tue Nov 23 10:12:47 2010
@@ -1,7 +1,7 @@
import py, sys
from pypy.rlib.rarithmetic import intmask, LONG_BIT
from pypy.rpython.lltypesystem import llmemory
-from pypy.jit.backend.test import conftest as demo_conftest
+from pypy.jit.backend import conftest as demo_conftest
from pypy.jit.metainterp.history import BasicFailDescr, TreeLoop
from pypy.jit.metainterp.history import BoxInt, ConstInt, LoopToken
from pypy.jit.metainterp.history import BoxPtr, ConstPtr
@@ -102,7 +102,7 @@
elif isinstance(v, ConstFloat):
args.append('ConstFloat(%r)' % v.value)
elif isinstance(v, ConstInt):
- args.append('ConstInt(%d)' % v.value)
+ args.append('ConstInt(%s)' % v.value)
else:
raise NotImplementedError(v)
if op.getdescr() is None:
@@ -113,7 +113,7 @@
except AttributeError:
descrstr = ', descr=...'
print >>s, ' ResOperation(rop.%s, [%s], %s%s),' % (
- opname[op.opnum], ', '.join(args), names[op.result], descrstr)
+ opname[op.getopnum()], ', '.join(args), names[op.result], descrstr)
#if getattr(op, 'suboperations', None) is not None:
# subops.append(op)
@@ -189,7 +189,7 @@
v.value)
print >>s, ' op = cpu.execute_token(looptoken)'
if self.should_fail_by is None:
- fail_args = self.loop.operations[-1].args
+ fail_args = self.loop.operations[-1].getarglist()
else:
fail_args = self.should_fail_by.getfailargs()
for i, v in enumerate(fail_args):
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/assembler.py Tue Nov 23 10:12:47 2010
@@ -8,7 +8,8 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.tool.uid import fixid
from pypy.jit.backend.x86.regalloc import (RegAlloc, X86RegisterManager,
- X86XMMRegisterManager, get_ebp_ofs)
+ X86XMMRegisterManager, get_ebp_ofs,
+ _get_scale)
from pypy.jit.backend.x86.arch import (FRAME_FIXED_SIZE, FORCE_INDEX_OFS, WORD,
IS_X86_32, IS_X86_64)
@@ -22,7 +23,8 @@
X86_64_SCRATCH_REG,
X86_64_XMM_SCRATCH_REG,
RegLoc, StackLoc, ConstFloatLoc,
- ImmedLoc, AddressLoc, imm)
+ ImmedLoc, AddressLoc, imm,
+ imm0, imm1)
from pypy.rlib.objectmodel import we_are_translated, specialize
from pypy.jit.backend.x86 import rx86, regloc, codebuf
@@ -244,12 +246,13 @@
f = open_file_as_stream(output_log, "w")
for i in range(len(self.loop_run_counters)):
name, struct = self.loop_run_counters[i]
- f.write(str(struct.i) + " " * (8 - len(str(struct.i))) + name + "\n")
+ f.write(str(name) + ":" + str(struct.i) + "\n")
f.close()
def _build_float_constants(self):
# 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment
- addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw')
+ addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw',
+ track_allocation=False)
if not we_are_translated():
self._keepalive_malloced_float_consts = addr
float_constants = rffi.cast(lltype.Signed, addr)
@@ -292,7 +295,7 @@
self.mc.RET()
self.mc.done()
- def assemble_loop(self, inputargs, operations, looptoken):
+ def assemble_loop(self, inputargs, operations, looptoken, log):
"""adds the following attributes to looptoken:
_x86_loop_code (an integer giving an address)
_x86_bootstrap_code (an integer giving an address)
@@ -300,6 +303,7 @@
_x86_frame_depth
_x86_param_depth
_x86_arglocs
+ _x86_debug_checksum
"""
if not we_are_translated():
# Arguments should be unique
@@ -307,10 +311,11 @@
self.setup()
funcname = self._find_debug_merge_point(operations)
-
+ if log:
+ self._register_counter()
+ operations = self._inject_debugging_code(looptoken, operations)
regalloc = RegAlloc(self, self.cpu.translate_support_code)
- operations = self._inject_debugging_code(operations)
arglocs = regalloc.prepare_loop(inputargs, operations, looptoken)
looptoken._x86_arglocs = arglocs
@@ -332,18 +337,22 @@
self._assemble_bootstrap_direct_call(arglocs, curadr,
frame_depth+param_depth)
#
- debug_print("Loop #", looptoken.number, "has address",
- looptoken._x86_loop_code, "to", self.mc.tell())
+ debug_print("Loop #%d has address %x to %x" % (looptoken.number,
+ looptoken._x86_loop_code,
+ self.mc.tell()))
self.mc.end_function()
self.write_pending_failure_recoveries()
- def assemble_bridge(self, faildescr, inputargs, operations):
+ def assemble_bridge(self, faildescr, inputargs, operations, log):
if not we_are_translated():
# Arguments should be unique
assert len(set(inputargs)) == len(inputargs)
self.setup()
funcname = self._find_debug_merge_point(operations)
+ if log:
+ self._register_counter()
+ operations = self._inject_debugging_code(faildescr, operations)
arglocs = self.rebuild_faillocs_from_descr(
faildescr._x86_failure_recovery_bytecode)
@@ -351,7 +360,6 @@
assert ([loc.assembler() for loc in arglocs] ==
[loc.assembler() for loc in faildescr._x86_debug_faillocs])
regalloc = RegAlloc(self, self.cpu.translate_support_code)
- operations = self._inject_debugging_code(operations)
fail_depths = faildescr._x86_current_depths
regalloc.prepare_bridge(fail_depths, inputargs, arglocs,
operations)
@@ -371,9 +379,8 @@
faildescr._x86_bridge_param_depth = param_depth
# patch the jump from original guard
self.patch_jump_for_descr(faildescr, adr_bridge)
- debug_print("Bridge out of guard",
- descr_number,
- "has address", adr_bridge, "to", self.mc.tell())
+ debug_print("Bridge out of guard %d has address %x to %x" %
+ (descr_number, adr_bridge, self.mc.tell()))
self.mc.end_function()
self.write_pending_failure_recoveries()
@@ -398,11 +405,14 @@
else:
funcname = "<loop %d>" % len(self.loop_run_counters)
# invent the counter, so we don't get too confused
+ return funcname
+
+ def _register_counter(self):
if self._debug:
- struct = lltype.malloc(DEBUG_COUNTER, flavor='raw')
+ struct = lltype.malloc(DEBUG_COUNTER, flavor='raw',
+ track_allocation=False) # known to leak
struct.i = 0
- self.loop_run_counters.append((funcname, struct))
- return funcname
+ self.loop_run_counters.append((len(self.loop_run_counters), struct))
def patch_jump_for_descr(self, faildescr, adr_new_target):
adr_jump_offset = faildescr._x86_adr_jump_offset
@@ -423,9 +433,14 @@
mc.done()
- def _inject_debugging_code(self, operations):
+ @specialize.argtype(1)
+ def _inject_debugging_code(self, looptoken, operations):
if self._debug:
# before doing anything, let's increase a counter
+ s = 0
+ for op in operations:
+ s += op.getopnum()
+ looptoken._x86_debug_checksum = s
c_adr = ConstInt(rffi.cast(lltype.Signed,
self.loop_run_counters[-1][1]))
box = BoxInt()
@@ -440,7 +455,7 @@
# self.mc.PUSH(eax)
# adr = rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])
# self.mc.MOV(eax, heap(adr))
- # self.mc.ADD(eax, imm(1))
+ # self.mc.ADD(eax, imm1)
# self.mc.MOV(heap(adr), eax)
# self.mc.POP(eax)
return operations
@@ -702,16 +717,18 @@
dispatch_opnum = guard_opnum
else:
dispatch_opnum = op.getopnum()
- res = genop_guard_list[dispatch_opnum](self, op, guard_op, guard_token,
- arglocs, resloc)
- faildescr._x86_adr_jump_offset = res
+ genop_guard_list[dispatch_opnum](self, op, guard_op, guard_token,
+ arglocs, resloc)
+ if not we_are_translated():
+ # must be added by the genop_guard_list[]()
+ assert hasattr(faildescr, '_x86_adr_jump_offset')
def regalloc_perform_guard(self, guard_op, faillocs, arglocs, resloc,
current_depths):
self.regalloc_perform_with_guard(None, guard_op, faillocs, arglocs,
resloc, current_depths)
- def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm(0)):
+ def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm0):
self.mc.LEA(result, addr_add(frm, sizereg, baseofs, scale))
def _unaryop(asmop):
@@ -761,15 +778,15 @@
if isinstance(op.getarg(0), Const):
self.mc.CMP(arglocs[1], arglocs[0])
if guard_opnum == rop.GUARD_FALSE:
- return self.implement_guard(guard_token, rev_cond)
+ self.implement_guard(guard_token, rev_cond)
else:
- return self.implement_guard(guard_token, false_rev_cond)
+ self.implement_guard(guard_token, false_rev_cond)
else:
self.mc.CMP(arglocs[0], arglocs[1])
if guard_opnum == rop.GUARD_FALSE:
- return self.implement_guard(guard_token, cond)
+ self.implement_guard(guard_token, cond)
else:
- return self.implement_guard(guard_token, false_cond)
+ self.implement_guard(guard_token, false_cond)
return genop_cmp_guard
def _cmpop_guard_float(cond, false_cond, need_jp):
@@ -783,13 +800,14 @@
if guard_opnum == rop.GUARD_FALSE:
if need_jp:
self.mc.J_il8(rx86.Conditions['P'], 6)
- return self.implement_guard(guard_token, cond)
+ self.implement_guard(guard_token, cond)
else:
if need_jp:
self.mc.J_il8(rx86.Conditions['P'], 2)
self.mc.J_il8(rx86.Conditions[cond], 5)
- return self.implement_guard(guard_token)
- return self.implement_guard(guard_token, false_cond)
+ self.implement_guard(guard_token)
+ else:
+ self.implement_guard(guard_token, false_cond)
return genop_cmp_guard_float
def _emit_call(self, x, arglocs, start=0, tmp=eax):
@@ -951,11 +969,11 @@
self.mc.ensure_bytes_available(16 + guard_token.recovery_stub_size())
if guard_opnum == rop.GUARD_TRUE:
self.mc.J_il8(rx86.Conditions['P'], 6)
- return self.implement_guard(guard_token, 'E')
+ self.implement_guard(guard_token, 'E')
else:
self.mc.J_il8(rx86.Conditions['P'], 2)
self.mc.J_il8(rx86.Conditions['E'], 5)
- return self.implement_guard(guard_token)
+ self.implement_guard(guard_token)
def genop_float_neg(self, op, arglocs, resloc):
# Following what gcc does: res = x ^ 0x8000000000000000
@@ -973,28 +991,28 @@
def genop_guard_int_is_true(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
- return self.implement_guard(guard_token, 'Z')
+ self.implement_guard(guard_token, 'Z')
else:
- return self.implement_guard(guard_token, 'NZ')
+ self.implement_guard(guard_token, 'NZ')
def genop_int_is_true(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['NE'], rl.value)
self.mc.MOVZX8(resloc, rl)
def genop_guard_int_is_zero(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
- return self.implement_guard(guard_token, 'NZ')
+ self.implement_guard(guard_token, 'NZ')
else:
- return self.implement_guard(guard_token, 'Z')
+ self.implement_guard(guard_token, 'Z')
def genop_int_is_zero(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['E'], rl.value)
self.mc.MOVZX8(resloc, rl)
@@ -1050,50 +1068,66 @@
assert result_loc is eax
self.call(self.malloc_unicode_func_addr, arglocs, eax)
- def genop_getfield_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, size_loc = arglocs
- assert isinstance(size_loc, ImmedLoc)
+ # ----------
+
+ def load_from_mem(self, resloc, source_addr, size_loc, sign_loc):
assert isinstance(resloc, RegLoc)
size = size_loc.value
-
- source_addr = AddressLoc(base_loc, ofs_loc)
+ sign = sign_loc.value
if resloc.is_xmm:
self.mc.MOVSD(resloc, source_addr)
+ elif size == WORD:
+ self.mc.MOV(resloc, source_addr)
elif size == 1:
- self.mc.MOVZX8(resloc, source_addr)
+ if sign:
+ self.mc.MOVSX8(resloc, source_addr)
+ else:
+ self.mc.MOVZX8(resloc, source_addr)
elif size == 2:
- self.mc.MOVZX16(resloc, source_addr)
+ if sign:
+ self.mc.MOVSX16(resloc, source_addr)
+ else:
+ self.mc.MOVZX16(resloc, source_addr)
+ elif IS_X86_64 and size == 4:
+ if sign:
+ self.mc.MOVSX32(resloc, source_addr)
+ else:
+ self.mc.MOV32(resloc, source_addr) # zero-extending
+ else:
+ not_implemented("load_from_mem size = %d" % size)
+
+ def save_into_mem(self, dest_addr, value_loc, size_loc):
+ size = size_loc.value
+ if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
+ self.mc.MOVSD(dest_addr, value_loc)
+ elif size == 1:
+ self.mc.MOV8(dest_addr, value_loc.lowest8bits())
+ elif size == 2:
+ self.mc.MOV16(dest_addr, value_loc)
elif size == 4:
- # MOV32 is zero-extending on 64-bit, so this is okay
- self.mc.MOV32(resloc, source_addr)
+ self.mc.MOV32(dest_addr, value_loc)
elif IS_X86_64 and size == 8:
- self.mc.MOV(resloc, source_addr)
+ self.mc.MOV(dest_addr, value_loc)
else:
- raise NotImplementedError("getfield size = %d" % size)
+ not_implemented("save_into_mem size = %d" % size)
+
+ def genop_getfield_gc(self, op, arglocs, resloc):
+ base_loc, ofs_loc, size_loc, sign_loc = arglocs
+ assert isinstance(size_loc, ImmedLoc)
+ source_addr = AddressLoc(base_loc, ofs_loc)
+ self.load_from_mem(resloc, source_addr, size_loc, sign_loc)
genop_getfield_raw = genop_getfield_gc
genop_getfield_raw_pure = genop_getfield_gc
genop_getfield_gc_pure = genop_getfield_gc
def genop_getarrayitem_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, scale, ofs = arglocs
+ base_loc, ofs_loc, size_loc, ofs, sign_loc = arglocs
assert isinstance(ofs, ImmedLoc)
- assert isinstance(scale, ImmedLoc)
- src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale.value)
- if op.result.type == FLOAT:
- self.mc.MOVSD(resloc, src_addr)
- else:
- if scale.value == 0:
- self.mc.MOVZX8(resloc, src_addr)
- elif scale.value == 1:
- self.mc.MOVZX16(resloc, src_addr)
- elif scale.value == 2:
- self.mc.MOV32(resloc, src_addr)
- elif IS_X86_64 and scale.value == 3:
- self.mc.MOV(resloc, src_addr)
- else:
- print "[asmgen]getarrayitem unsupported size: %d" % scale.value
- raise NotImplementedError()
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale)
+ self.load_from_mem(resloc, src_addr, size_loc, sign_loc)
genop_getarrayitem_gc_pure = genop_getarrayitem_gc
genop_getarrayitem_raw = genop_getarrayitem_gc
@@ -1101,40 +1135,16 @@
def genop_discard_setfield_gc(self, op, arglocs):
base_loc, ofs_loc, size_loc, value_loc = arglocs
assert isinstance(size_loc, ImmedLoc)
- size = size_loc.value
dest_addr = AddressLoc(base_loc, ofs_loc)
- if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
- self.mc.MOVSD(dest_addr, value_loc)
- elif IS_X86_64 and size == 8:
- self.mc.MOV(dest_addr, value_loc)
- elif size == 4:
- self.mc.MOV32(dest_addr, value_loc)
- elif size == 2:
- self.mc.MOV16(dest_addr, value_loc)
- elif size == 1:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- print "[asmgen]setfield addr size %d" % size
- raise NotImplementedError("Addr size %d" % size)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_setarrayitem_gc(self, op, arglocs):
- base_loc, ofs_loc, value_loc, scale_loc, baseofs = arglocs
+ base_loc, ofs_loc, value_loc, size_loc, baseofs = arglocs
assert isinstance(baseofs, ImmedLoc)
- assert isinstance(scale_loc, ImmedLoc)
- dest_addr = AddressLoc(base_loc, ofs_loc, scale_loc.value, baseofs.value)
- if op.getarg(2).type == FLOAT:
- self.mc.MOVSD(dest_addr, value_loc)
- else:
- if IS_X86_64 and scale_loc.value == 3:
- self.mc.MOV(dest_addr, value_loc)
- elif scale_loc.value == 2:
- self.mc.MOV32(dest_addr, value_loc)
- elif scale_loc.value == 1:
- self.mc.MOV16(dest_addr, value_loc)
- elif scale_loc.value == 0:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- raise NotImplementedError("scale = %d" % scale_loc.value)
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ dest_addr = AddressLoc(base_loc, ofs_loc, scale, baseofs.value)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_strsetitem(self, op, arglocs):
base_loc, ofs_loc, val_loc = arglocs
@@ -1196,13 +1206,13 @@
def genop_guard_guard_true(self, ign_1, guard_op, guard_token, locs, ign_2):
loc = locs[0]
self.mc.TEST(loc, loc)
- return self.implement_guard(guard_token, 'Z')
+ self.implement_guard(guard_token, 'Z')
genop_guard_guard_nonnull = genop_guard_guard_true
def genop_guard_guard_no_exception(self, ign_1, guard_op, guard_token,
locs, ign_2):
- self.mc.CMP(heap(self.cpu.pos_exception()), imm(0))
- return self.implement_guard(guard_token, 'NZ')
+ self.mc.CMP(heap(self.cpu.pos_exception()), imm0)
+ self.implement_guard(guard_token, 'NZ')
def genop_guard_guard_exception(self, ign_1, guard_op, guard_token,
locs, resloc):
@@ -1210,22 +1220,21 @@
loc1 = locs[1]
self.mc.MOV(loc1, heap(self.cpu.pos_exception()))
self.mc.CMP(loc1, loc)
- addr = self.implement_guard(guard_token, 'NE')
+ self.implement_guard(guard_token, 'NE')
if resloc is not None:
self.mc.MOV(resloc, heap(self.cpu.pos_exc_value()))
- self.mc.MOV(heap(self.cpu.pos_exception()), imm(0))
- self.mc.MOV(heap(self.cpu.pos_exc_value()), imm(0))
- return addr
+ self.mc.MOV(heap(self.cpu.pos_exception()), imm0)
+ self.mc.MOV(heap(self.cpu.pos_exc_value()), imm0)
def _gen_guard_overflow(self, guard_op, guard_token):
guard_opnum = guard_op.getopnum()
if guard_opnum == rop.GUARD_NO_OVERFLOW:
- return self.implement_guard(guard_token, 'O')
+ self.implement_guard(guard_token, 'O')
elif guard_opnum == rop.GUARD_OVERFLOW:
- return self.implement_guard(guard_token, 'NO')
+ self.implement_guard(guard_token, 'NO')
else:
- print "int_xxx_ovf followed by", guard_op.getopname()
- raise AssertionError
+ not_implemented("int_xxx_ovf followed by %s" %
+ guard_op.getopname())
def genop_guard_int_add_ovf(self, op, guard_op, guard_token, arglocs, result_loc):
self.genop_int_add(op, arglocs, result_loc)
@@ -1242,7 +1251,7 @@
def genop_guard_guard_false(self, ign_1, guard_op, guard_token, locs, ign_2):
loc = locs[0]
self.mc.TEST(loc, loc)
- return self.implement_guard(guard_token, 'NZ')
+ self.implement_guard(guard_token, 'NZ')
genop_guard_guard_isnull = genop_guard_guard_false
def genop_guard_guard_value(self, ign_1, guard_op, guard_token, locs, ign_2):
@@ -1251,7 +1260,7 @@
self.mc.UCOMISD(locs[0], locs[1])
else:
self.mc.CMP(locs[0], locs[1])
- return self.implement_guard(guard_token, 'NE')
+ self.implement_guard(guard_token, 'NE')
def _cmp_guard_class(self, locs):
offset = self.cpu.vtable_offset
@@ -1283,12 +1292,12 @@
def genop_guard_guard_class(self, ign_1, guard_op, guard_token, locs, ign_2):
self.mc.ensure_bytes_available(256)
self._cmp_guard_class(locs)
- return self.implement_guard(guard_token, 'NE')
+ self.implement_guard(guard_token, 'NE')
def genop_guard_guard_nonnull_class(self, ign_1, guard_op,
guard_token, locs, ign_2):
self.mc.ensure_bytes_available(256)
- self.mc.CMP(locs[0], imm(1))
+ self.mc.CMP(locs[0], imm1)
# Patched below
self.mc.J_il8(rx86.Conditions['B'], 0)
jb_location = self.mc.get_relative_pos()
@@ -1298,7 +1307,7 @@
assert 0 < offset <= 127
self.mc.overwrite(jb_location-1, [chr(offset)])
#
- return self.implement_guard(guard_token, 'NE')
+ self.implement_guard(guard_token, 'NE')
def implement_guard_recovery(self, guard_opnum, faildescr, failargs,
fail_locs):
@@ -1625,37 +1634,48 @@
def implement_guard(self, guard_token, condition=None):
self.mc.reserve_bytes(guard_token.recovery_stub_size())
self.pending_guard_tokens.append(guard_token)
- # XXX: These jumps are patched later, the self.mc.tell() are just
- # dummy values
+ # These jumps are patched later, the mc.tell() are just
+ # dummy values. Also, use self.mc._mc to avoid triggering a
+ # "buffer full" exactly here.
+ mc = self.mc._mc
if condition:
- self.mc.J_il(rx86.Conditions[condition], self.mc.tell())
+ mc.J_il(rx86.Conditions[condition], mc.tell())
else:
- self.mc.JMP_l(self.mc.tell())
- return self.mc.tell() - 4
+ mc.JMP_l(mc.tell())
+ guard_token.faildescr._x86_adr_jump_offset = mc.tell() - 4
def genop_call(self, op, arglocs, resloc):
sizeloc = arglocs[0]
assert isinstance(sizeloc, ImmedLoc)
size = sizeloc.value
+ signloc = arglocs[1]
if isinstance(op.getarg(0), Const):
x = imm(op.getarg(0).getint())
else:
- x = arglocs[1]
+ x = arglocs[2]
if x is eax:
tmp = ecx
else:
tmp = eax
- self._emit_call(x, arglocs, 2, tmp=tmp)
+ self._emit_call(x, arglocs, 3, tmp=tmp)
+
+ if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
+ self.mc.FSTP_b(resloc.value) # float return
+ elif size == WORD:
+ assert resloc is eax or resloc is xmm0 # a full word
+ elif size == 0:
+ pass # void return
+ else:
+ # use the code in load_from_mem to do the zero- or sign-extension
+ assert resloc is eax
+ if size == 1:
+ srcloc = eax.lowest8bits()
+ else:
+ srcloc = eax
+ self.load_from_mem(eax, srcloc, sizeloc, signloc)
- if isinstance(resloc, StackLoc) and resloc.width == 8 and IS_X86_32:
- self.mc.FSTP_b(resloc.value)
- elif size == 1:
- self.mc.AND_ri(eax.value, 0xff)
- elif size == 2:
- self.mc.AND_ri(eax.value, 0xffff)
-
def genop_guard_call_may_force(self, op, guard_op, guard_token,
arglocs, result_loc):
faildescr = guard_op.getdescr()
@@ -1663,7 +1683,7 @@
self.mc.MOV_bi(FORCE_INDEX_OFS, fail_index)
self.genop_call(op, arglocs, result_loc)
self.mc.CMP_bi(FORCE_INDEX_OFS, 0)
- return self.implement_guard(guard_token, 'L')
+ self.implement_guard(guard_token, 'L')
def genop_guard_call_assembler(self, op, guard_op, guard_token,
arglocs, result_loc):
@@ -1750,16 +1770,21 @@
assert 0 <= offset <= 127
self.mc.overwrite(jmp_location - 1, [chr(offset)])
self.mc.CMP_bi(FORCE_INDEX_OFS, 0)
- return self.implement_guard(guard_token, 'L')
+ self.implement_guard(guard_token, 'L')
def genop_discard_cond_call_gc_wb(self, op, arglocs):
- # use 'mc._mc' directly instead of 'mc', to avoid
- # bad surprizes if the code buffer is mostly full
+ # Write code equivalent to write_barrier() in the GC: it checks
+ # a flag in the object at arglocs[0], and if set, it calls the
+ # function remember_young_pointer() from the GC. The two arguments
+ # to the call are in arglocs[:2]. The rest, arglocs[2:], contains
+ # registers that need to be saved and restored across the call.
descr = op.getdescr()
if we_are_translated():
cls = self.cpu.gc_ll_descr.has_write_barrier_class()
assert cls is not None and isinstance(descr, cls)
loc_base = arglocs[0]
+ # ensure that enough bytes are available to write the whole
+ # following piece of code atomically (for the JZ)
self.mc.ensure_bytes_available(256)
self.mc.TEST8_mi((loc_base.value, descr.jit_wb_if_flag_byteofs),
descr.jit_wb_if_flag_singlebyte)
@@ -1767,35 +1792,39 @@
jz_location = self.mc.get_relative_pos()
# the following is supposed to be the slow path, so whenever possible
# we choose the most compact encoding over the most efficient one.
- # XXX improve a bit, particularly for IS_X86_64.
- for i in range(len(arglocs)-1, -1, -1):
+ if IS_X86_32:
+ limit = -1 # push all arglocs on the stack
+ elif IS_X86_64:
+ limit = 1 # push only arglocs[2:] on the stack
+ for i in range(len(arglocs)-1, limit, -1):
loc = arglocs[i]
if isinstance(loc, RegLoc):
self.mc.PUSH_r(loc.value)
else:
- if IS_X86_64:
- self.mc.MOV_ri(X86_64_SCRATCH_REG.value, loc.getint())
- self.mc.PUSH_r(X86_64_SCRATCH_REG.value)
- else:
- self.mc.PUSH_i32(loc.getint())
-
+ assert not IS_X86_64 # there should only be regs in arglocs[2:]
+ self.mc.PUSH_i32(loc.getint())
if IS_X86_64:
- # We clobber this register to pass the arguments, but that's
+ # We clobber these registers to pass the arguments, but that's
# okay, because consider_cond_call_gc_wb makes sure that any
- # caller-save registers with values in them are present in arglocs,
- # so they are saved on the stack above and restored below
- self.mc.MOV_rs(edi.value, 0)
+ # caller-save registers with values in them are present in
+ # arglocs[2:] too, so they are saved on the stack above and
+ # restored below.
+ remap_frame_layout(self, arglocs[:2], [edi, esi],
+ X86_64_SCRATCH_REG)
# misaligned stack in the call, but it's ok because the write barrier
# is not going to call anything more. Also, this assumes that the
- # write barrier does not touch the xmm registers.
+ # write barrier does not touch the xmm registers. (Slightly delicate
+ # assumption, given that the write barrier can end up calling the
+ # platform's malloc() from AddressStack.append(). XXX may need to
+ # be done properly)
self.mc.CALL(imm(descr.get_write_barrier_fn(self.cpu)))
- for i in range(len(arglocs)):
+ if IS_X86_32:
+ self.mc.ADD_ri(esp.value, 2*WORD)
+ for i in range(2, len(arglocs)):
loc = arglocs[i]
- if isinstance(loc, RegLoc):
- self.mc.POP_r(loc.value)
- else:
- self.mc.ADD_ri(esp.value, WORD) # ignore the pushed constant
+ assert isinstance(loc, RegLoc)
+ self.mc.POP_r(loc.value)
# patch the JZ above
offset = self.mc.get_relative_pos() - jz_location
assert 0 < offset <= 127
@@ -1807,20 +1836,16 @@
self.mc.LEA_rb(resloc.value, FORCE_INDEX_OFS)
def not_implemented_op_discard(self, op, arglocs):
- msg = "not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op(self, op, arglocs, resloc):
- msg = "not implemented operation with res: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with res: %s" %
+ op.getopname())
def not_implemented_op_guard(self, op, guard_op,
failaddr, arglocs, resloc):
- msg = "not implemented operation (guard): %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation (guard): %s" %
+ op.getopname())
def mark_gc_roots(self):
gcrootmap = self.cpu.gc_ll_descr.gcrootmap
@@ -1836,6 +1861,7 @@
def malloc_cond_fixedsize(self, nursery_free_adr, nursery_top_adr,
size, tid):
+ size = max(size, self.cpu.gc_ll_descr.minimal_size_in_nursery)
self.mc.ensure_bytes_available(256)
self.mc.MOV(eax, heap(nursery_free_adr))
self.mc.LEA_rm(edx.value, (eax.value, size))
@@ -1904,3 +1930,7 @@
def heap(addr):
return AddressLoc(ImmedLoc(addr), ImmedLoc(0), 0, 0)
+
+def not_implemented(msg):
+ os.write(2, '[x86/asm] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py Tue Nov 23 10:12:47 2010
@@ -2,6 +2,7 @@
""" Register allocation scheme.
"""
+import os
from pypy.jit.metainterp.history import (Box, Const, ConstInt, ConstPtr,
ResOperation, BoxPtr,
LoopToken, INT, REF, FLOAT)
@@ -40,12 +41,10 @@
return imm(c.value)
elif isinstance(c, ConstPtr):
if we_are_translated() and c.value and rgc.can_move(c.value):
- print "convert_to_imm: ConstPtr needs special care"
- raise AssertionError
+ not_implemented("convert_to_imm: ConstPtr needs special care")
return imm(rffi.cast(lltype.Signed, c.value))
else:
- print "convert_to_imm: got a %s" % c
- raise AssertionError
+ not_implemented("convert_to_imm: got a %s" % c)
class X86_64_RegisterManager(X86RegisterManager):
# r11 omitted because it's used as scratch
@@ -70,8 +69,9 @@
def _get_new_array(self):
n = self.BASE_CONSTANT_SIZE
+ # known to leak
self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n,
- flavor='raw')
+ flavor='raw', track_allocation=False)
self.cur_array_free = n
_get_new_array._dont_inline_ = True
@@ -349,8 +349,8 @@
if op.is_ovf():
if (operations[i + 1].getopnum() != rop.GUARD_NO_OVERFLOW and
operations[i + 1].getopnum() != rop.GUARD_OVERFLOW):
- print "int_xxx_ovf not followed by guard_(no)_overflow"
- raise AssertionError
+ not_implemented("int_xxx_ovf not followed by "
+ "guard_(no)_overflow")
return True
return False
if (operations[i + 1].getopnum() != rop.GUARD_TRUE and
@@ -624,7 +624,13 @@
assert isinstance(calldescr, BaseCallDescr)
assert len(calldescr.arg_classes) == op.numargs() - 1
size = calldescr.get_result_size(self.translate_support_code)
- self._call(op, [imm(size)] + [self.loc(op.getarg(i)) for i in range(op.numargs())],
+ sign = calldescr.is_result_signed()
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self._call(op, [imm(size), sign_loc] +
+ [self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_not_forced_op)
def consider_call(self, op):
@@ -645,7 +651,7 @@
self.rm._sync_var(op.getarg(vable_index))
vable = self.fm.loc(op.getarg(vable_index))
else:
- vable = imm(0)
+ vable = imm0
self._call(op, [imm(size), vable] +
[self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_op)
@@ -653,9 +659,13 @@
def consider_cond_call_gc_wb(self, op):
assert op.result is None
args = op.getarglist()
+ loc_newvalue = self.rm.make_sure_var_in_reg(op.getarg(1), args)
+ # ^^^ we force loc_newvalue in a reg (unless it's a Const),
+ # because it will be needed anyway by the following setfield_gc.
+ # It avoids loading it twice from the memory.
loc_base = self.rm.make_sure_var_in_reg(op.getarg(0), args,
imm_fine=False)
- arglocs = [loc_base]
+ arglocs = [loc_base, loc_newvalue]
# add eax, ecx and edx as extra "arguments" to ensure they are
# saved and restored. Fish in self.rm to know which of these
# registers really need to be saved (a bit of a hack). Moreover,
@@ -731,15 +741,11 @@
loc = self.loc(op.getarg(0))
return self._call(op, [loc])
# boehm GC (XXX kill the following code at some point)
- ofs_items, itemsize, ofs = symbolic.get_array_token(rstr.UNICODE, self.translate_support_code)
- if itemsize == 4:
- return self._malloc_varsize(ofs_items, ofs, 2, op.getarg(0),
- op.result)
- elif itemsize == 2:
- return self._malloc_varsize(ofs_items, ofs, 1, op.getarg(0),
- op.result)
- else:
- assert False, itemsize
+ ofs_items, _, ofs = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ scale = self._get_unicode_item_scale()
+ return self._malloc_varsize(ofs_items, ofs, scale, op.getarg(0),
+ op.result)
def _malloc_varsize(self, ofs_items, ofs_length, scale, v, res_v):
# XXX kill this function at some point
@@ -771,8 +777,9 @@
arglocs.append(self.loc(op.getarg(0)))
return self._call(op, arglocs)
# boehm GC (XXX kill the following code at some point)
- scale_of_field, basesize, ofs_length, _ = (
+ itemsize, basesize, ofs_length, _, _ = (
self._unpack_arraydescr(op.getdescr()))
+ scale_of_field = _get_scale(itemsize)
return self._malloc_varsize(basesize, ofs_length, scale_of_field,
op.getarg(0), op.result)
@@ -782,21 +789,19 @@
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
ptr = arraydescr.is_array_of_pointers()
- scale = 0
- while (1 << scale) < size:
- scale += 1
- assert (1 << scale) == size
- return scale, ofs, ofs_length, ptr
+ sign = arraydescr.is_item_signed()
+ return size, ofs, ofs_length, ptr, sign
def _unpack_fielddescr(self, fielddescr):
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
ptr = fielddescr.is_pointer_field()
- return imm(ofs), imm(size), ptr
+ sign = fielddescr.is_field_signed()
+ return imm(ofs), imm(size), ptr, sign
def consider_setfield_gc(self, op):
- ofs_loc, size_loc, ptr = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, _ = self._unpack_fielddescr(op.getdescr())
assert isinstance(size_loc, ImmedLoc)
if size_loc.value == 1:
need_lower_byte = True
@@ -823,10 +828,10 @@
consider_unicodesetitem = consider_strsetitem
def consider_setarrayitem_gc(self, op):
- scale, ofs, _, ptr = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, _ = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
- if scale == 0:
+ if itemsize == 1:
need_lower_byte = True
else:
need_lower_byte = False
@@ -835,30 +840,39 @@
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.possibly_free_vars(args)
self.PerformDiscard(op, [base_loc, ofs_loc, value_loc,
- imm(scale), imm(ofs)])
+ imm(itemsize), imm(ofs)])
consider_setarrayitem_raw = consider_setarrayitem_gc
def consider_getfield_gc(self, op):
- ofs_loc, size_loc, _ = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, sign = self._unpack_fielddescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
self.rm.possibly_free_vars(args)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, size_loc], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, size_loc, sign_loc], result_loc)
consider_getfield_raw = consider_getfield_gc
consider_getfield_raw_pure = consider_getfield_gc
consider_getfield_gc_pure = consider_getfield_gc
def consider_getarrayitem_gc(self, op):
- scale, ofs, _, _ = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, sign = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.rm.possibly_free_vars_for_op(op)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, imm(scale), imm(ofs)], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, imm(itemsize), imm(ofs),
+ sign_loc], result_loc)
consider_getarrayitem_raw = consider_getarrayitem_gc
consider_getarrayitem_gc_pure = consider_getarrayitem_gc
@@ -912,41 +926,85 @@
consider_unicodegetitem = consider_strgetitem
def consider_copystrcontent(self, op):
+ self._consider_copystrcontent(op, is_unicode=False)
+
+ def consider_copyunicodecontent(self, op):
+ self._consider_copystrcontent(op, is_unicode=True)
+
+ def _consider_copystrcontent(self, op, is_unicode):
# compute the source address
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(args[0], args)
ofs_loc = self.rm.make_sure_var_in_reg(args[2], args)
+ assert args[0] is not args[1] # forbidden case of aliasing
self.rm.possibly_free_var(args[0])
- self.rm.possibly_free_var(args[2])
+ if args[3] is not args[2] is not args[4]: # MESS MESS MESS: don't free
+ self.rm.possibly_free_var(args[2]) # it if ==args[3] or args[4]
srcaddr_box = TempBox()
- srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box)
- self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc)
+ forbidden_vars = [args[1], args[3], args[4], srcaddr_box]
+ srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box, forbidden_vars)
+ self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc,
+ is_unicode=is_unicode)
# compute the destination address
- base_loc = self.rm.make_sure_var_in_reg(args[1], args)
- ofs_loc = self.rm.make_sure_var_in_reg(args[3], args)
+ base_loc = self.rm.make_sure_var_in_reg(args[1], forbidden_vars)
+ ofs_loc = self.rm.make_sure_var_in_reg(args[3], forbidden_vars)
self.rm.possibly_free_var(args[1])
- self.rm.possibly_free_var(args[3])
+ if args[3] is not args[4]: # more of the MESS described above
+ self.rm.possibly_free_var(args[3])
+ forbidden_vars = [args[4], srcaddr_box]
dstaddr_box = TempBox()
- dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box)
- self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc)
+ dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box, forbidden_vars)
+ self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc,
+ is_unicode=is_unicode)
+ # compute the length in bytes
+ length_box = args[4]
+ length_loc = self.loc(length_box)
+ if is_unicode:
+ self.rm.possibly_free_var(length_box)
+ forbidden_vars = [srcaddr_box, dstaddr_box]
+ bytes_box = TempBox()
+ bytes_loc = self.rm.force_allocate_reg(bytes_box, forbidden_vars)
+ scale = self._get_unicode_item_scale()
+ if not (isinstance(length_loc, ImmedLoc) or
+ isinstance(length_loc, RegLoc)):
+ self.assembler.mov(length_loc, bytes_loc)
+ length_loc = bytes_loc
+ self.assembler.load_effective_addr(length_loc, 0, scale, bytes_loc)
+ length_box = bytes_box
+ length_loc = bytes_loc
# call memcpy()
- length_loc = self.loc(args[4])
self.rm.before_call()
self.xrm.before_call()
self.assembler._emit_call(imm(self.assembler.memcpy_addr),
[dstaddr_loc, srcaddr_loc, length_loc])
- self.rm.possibly_free_var(args[4])
+ self.rm.possibly_free_var(length_box)
self.rm.possibly_free_var(dstaddr_box)
self.rm.possibly_free_var(srcaddr_box)
- def _gen_address_inside_string(self, baseloc, ofsloc, resloc):
+ def _gen_address_inside_string(self, baseloc, ofsloc, resloc, is_unicode):
cpu = self.assembler.cpu
- ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR,
+ if is_unicode:
+ ofs_items, _, _ = symbolic.get_array_token(rstr.UNICODE,
self.translate_support_code)
- assert itemsize == 1
- self.assembler.load_effective_addr(ofsloc, ofs_items, 0,
+ scale = self._get_unicode_item_scale()
+ else:
+ ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR,
+ self.translate_support_code)
+ assert itemsize == 1
+ scale = 0
+ self.assembler.load_effective_addr(ofsloc, ofs_items, scale,
resloc, baseloc)
+ def _get_unicode_item_scale(self):
+ _, itemsize, _ = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ if itemsize == 4:
+ return 2
+ elif itemsize == 2:
+ return 1
+ else:
+ raise AssertionError("bad unicode item size")
+
def consider_jump(self, op):
assembler = self.assembler
assert self.jump_target_descr is None
@@ -981,6 +1039,9 @@
def consider_debug_merge_point(self, op):
pass
+ def consider_jit_debug(self, op):
+ pass
+
def get_mark_gc_roots(self, gcrootmap):
shape = gcrootmap.get_basic_shape(IS_X86_64)
for v, val in self.fm.frame_bindings.items():
@@ -1000,15 +1061,11 @@
self.Perform(op, [], loc)
def not_implemented_op(self, op):
- msg = "[regalloc] Not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op_with_guard(self, op, guard_op):
- msg = "[regalloc] Not implemented operation with guard: %s" % (
- op.getopname(),)
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with guard: %s" % (
+ op.getopname(),))
oplist = [RegAlloc.not_implemented_op] * rop._LAST
oplist_with_guard = [RegAlloc.not_implemented_op_with_guard] * rop._LAST
@@ -1042,3 +1099,14 @@
# Returns (ebp-20), (ebp-24), (ebp-28)...
# i.e. the n'th word beyond the fixed frame size.
return -WORD * (FRAME_FIXED_SIZE + position)
+
+def _get_scale(size):
+ assert size == 1 or size == 2 or size == 4 or size == 8
+ if size < 4:
+ return size - 1 # 1, 2 => 0, 1
+ else:
+ return (size >> 2) + 1 # 4, 8 => 2, 3
+
+def not_implemented(msg):
+ os.write(2, '[x86/regalloc] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/regloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/regloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/regloc.py Tue Nov 23 10:12:47 2010
@@ -442,8 +442,11 @@
MOV8 = _binaryop('MOV8')
MOV16 = _16_bit_binaryop('MOV')
MOVZX8 = _binaryop('MOVZX8')
+ MOVSX8 = _binaryop('MOVSX8')
MOVZX16 = _binaryop('MOVZX16')
+ MOVSX16 = _binaryop('MOVSX16')
MOV32 = _binaryop('MOV32')
+ MOVSX32 = _binaryop('MOVSX32')
XCHG = _binaryop('XCHG')
PUSH = _unaryop('PUSH')
@@ -473,6 +476,9 @@
else:
return ImmedLoc(x)
+imm0 = imm(0)
+imm1 = imm(1)
+
all_extra_instructions = [name for name in LocationCodeBuilder.__dict__
if name[0].isupper()]
all_extra_instructions.sort()
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/runner.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/runner.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/runner.py Tue Nov 23 10:12:47 2010
@@ -49,11 +49,13 @@
self.assembler.finish_once()
self.profile_agent.shutdown()
- def compile_loop(self, inputargs, operations, looptoken):
- self.assembler.assemble_loop(inputargs, operations, looptoken)
-
- def compile_bridge(self, faildescr, inputargs, operations):
- self.assembler.assemble_bridge(faildescr, inputargs, operations)
+ def compile_loop(self, inputargs, operations, looptoken, log=True):
+ self.assembler.assemble_loop(inputargs, operations, looptoken,
+ log=log)
+
+ def compile_bridge(self, faildescr, inputargs, operations, log=True):
+ self.assembler.assemble_bridge(faildescr, inputargs, operations,
+ log=log)
def set_future_value_int(self, index, intvalue):
self.assembler.fail_boxes_int.setitem(index, intvalue)
@@ -87,7 +89,9 @@
def execute_token(self, executable_token):
addr = executable_token._x86_bootstrap_code
+ #llop.debug_print(lltype.Void, ">>>> Entering", addr)
func = rffi.cast(lltype.Ptr(self.BOOTSTRAP_TP), addr)
+ #llop.debug_print(lltype.Void, "<<<< Back")
fail_index = self._execute_call(func)
return self.get_fail_descr_from_number(fail_index)
@@ -99,10 +103,7 @@
LLInterpreter.current_interpreter = self.debug_ll_interpreter
res = 0
try:
- #llop.debug_print(lltype.Void, ">>>> Entering",
- # rffi.cast(lltype.Signed, func))
res = func()
- #llop.debug_print(lltype.Void, "<<<< Back")
finally:
if not self.translate_support_code:
LLInterpreter.current_interpreter = prev_interpreter
@@ -114,7 +115,8 @@
return CPU386.cast_adr_to_int(adr)
all_null_registers = lltype.malloc(rffi.LONGP.TO, 24,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
def force(self, addr_of_force_index):
TP = rffi.CArrayPtr(lltype.Signed)
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/rx86.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/rx86.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/rx86.py Tue Nov 23 10:12:47 2010
@@ -642,7 +642,10 @@
define_modrm_modes('MOV8_*i', [rex_w, '\xC6', orbyte(0<<3)], [immediate(2, 'b')], regtype='BYTE')
define_modrm_modes('MOVZX8_r*', [rex_w, '\x0F\xB6', register(1, 8)], regtype='BYTE')
+define_modrm_modes('MOVSX8_r*', [rex_w, '\x0F\xBE', register(1, 8)], regtype='BYTE')
define_modrm_modes('MOVZX16_r*', [rex_w, '\x0F\xB7', register(1, 8)])
+define_modrm_modes('MOVSX16_r*', [rex_w, '\x0F\xBF', register(1, 8)])
+define_modrm_modes('MOVSX32_r*', [rex_w, '\x63', register(1, 8)])
define_modrm_modes('MOVSD_x*', ['\xF2', rex_nw, '\x0F\x10', register(1,8)], regtype='XMM')
define_modrm_modes('MOVSD_*x', ['\xF2', rex_nw, '\x0F\x11', register(2,8)], regtype='XMM')
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_assembler.py Tue Nov 23 10:12:47 2010
@@ -81,7 +81,8 @@
# also test rebuild_faillocs_from_descr(), which should not
# reproduce the holes at all
- bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw')
+ bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw',
+ immortal=True)
for i in range(len(mc.content)):
assert 0 <= mc.content[i] <= 255
bytecode[i] = rffi.cast(rffi.UCHAR, mc.content[i])
@@ -115,7 +116,8 @@
assert withfloats
value = random.random() - 0.5
# make sure it fits into 64 bits
- tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw')
+ tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw',
+ track_allocation=False)
rffi.cast(rffi.DOUBLEP, tmp)[0] = value
return rffi.cast(rffi.DOUBLEP, tmp)[0], tmp[0], tmp[1]
@@ -152,10 +154,12 @@
# prepare the expected target arrays, the descr_bytecode,
# the 'registers' and the 'stack' arrays according to 'content'
- xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1, flavor='raw')
+ xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1,
+ flavor='raw', immortal=True)
registers = rffi.ptradd(xmmregisters, 16)
stacklen = baseloc + 10
- stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw')
+ stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw',
+ immortal=True)
expected_ints = [0] * len(content)
expected_ptrs = [lltype.nullptr(llmemory.GCREF.TO)] * len(content)
expected_floats = [0.0] * len(content)
@@ -221,7 +225,7 @@
descr_bytecode.append(0x00)
descr_bytecode.append(0xCC) # end marker
descr_bytes = lltype.malloc(rffi.UCHARP.TO, len(descr_bytecode),
- flavor='raw')
+ flavor='raw', immortal=True)
for i in range(len(descr_bytecode)):
assert 0 <= descr_bytecode[i] <= 255
descr_bytes[i] = rffi.cast(rffi.UCHAR, descr_bytecode[i])
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_gc_integration.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_gc_integration.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_gc_integration.py Tue Nov 23 10:12:47 2010
@@ -12,7 +12,7 @@
from pypy.jit.backend.detect_cpu import getcpuclass
from pypy.jit.backend.x86.regalloc import RegAlloc
from pypy.jit.backend.x86.arch import WORD, FRAME_FIXED_SIZE
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import rclass, rstr
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_regalloc.py Tue Nov 23 10:12:47 2010
@@ -11,7 +11,7 @@
from pypy.jit.backend.x86.regalloc import RegAlloc, X86RegisterManager,\
FloatConstants, is_comparison_or_ovf_op
from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import rclass, rstr
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_runner.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_runner.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_runner.py Tue Nov 23 10:12:47 2010
@@ -10,7 +10,7 @@
from pypy.jit.metainterp.resoperation import rop
from pypy.jit.metainterp.executor import execute
from pypy.jit.backend.test.runner_test import LLtypeBackendTest
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.tool.udir import udir
import ctypes
import sys
@@ -346,7 +346,7 @@
return self.val
operations = [
- ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("hello")], None),
+ ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("hello"), 0], None),
ResOperation(rop.INT_ADD, [i0, ConstInt(1)], i1),
ResOperation(rop.INT_LE, [i1, ConstInt(9)], i2),
ResOperation(rop.GUARD_TRUE, [i2], None, descr=faildescr1),
@@ -365,7 +365,7 @@
bridge = [
ResOperation(rop.INT_LE, [i1b, ConstInt(19)], i3),
ResOperation(rop.GUARD_TRUE, [i3], None, descr=faildescr2),
- ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("bye")], None),
+ ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("bye"), 0], None),
ResOperation(rop.JUMP, [i1b], None, descr=looptoken),
]
bridge[1].setfailargs([i1b])
@@ -478,6 +478,10 @@
# whether the test segfaults.
assert self.cpu.get_latest_value_int(0) == finished.value
+ def test_overflow_guard_exception(self):
+ for i in range(50):
+ self.test_exceptions()
+
class TestDebuggingAssembler(object):
def setup_method(self, meth):
@@ -493,7 +497,7 @@
def test_debugger_on(self):
loop = """
[i0]
- debug_merge_point('xyz')
+ debug_merge_point('xyz', 0)
i1 = int_add(i0, 1)
i2 = int_ge(i1, 10)
guard_false(i2) []
@@ -506,8 +510,25 @@
self.cpu.execute_token(ops.token)
# check debugging info
name, struct = self.cpu.assembler.loop_run_counters[0]
- assert name == 'xyz'
+ assert name == 0 # 'xyz'
assert struct.i == 10
self.cpu.finish_once()
lines = py.path.local(self.logfile + ".count").readlines()
- assert lines[0] == '10 xyz\n'
+ assert lines[0] == '0:10\n' # '10 xyz\n'
+
+ def test_debugger_checksum(self):
+ loop = """
+ [i0]
+ debug_merge_point('xyz', 0)
+ i1 = int_add(i0, 1)
+ i2 = int_ge(i1, 10)
+ guard_false(i2) []
+ jump(i1)
+ """
+ ops = parse(loop)
+ self.cpu.assembler.set_debug(True)
+ self.cpu.compile_loop(ops.inputargs, ops.operations, ops.token)
+ self.cpu.set_future_value_int(0, 0)
+ self.cpu.execute_token(ops.token)
+ assert ops.token._x86_debug_checksum == sum([op.getopnum()
+ for op in ops.operations])
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_string.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_string.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_string.py Tue Nov 23 10:12:47 2010
@@ -2,8 +2,12 @@
from pypy.jit.metainterp.test import test_string
from pypy.jit.backend.x86.test.test_basic import Jit386Mixin
-class TestString(Jit386Mixin, test_string.StringTests):
+class TestString(Jit386Mixin, test_string.TestLLtype):
# for the individual tests see
# ====> ../../../metainterp/test/test_string.py
- CALL = 'call'
- CALL_PURE = 'call_pure'
+ pass
+
+class TestUnicode(Jit386Mixin, test_string.TestLLtypeUnicode):
+ # for the individual tests see
+ # ====> ../../../metainterp/test/test_string.py
+ pass
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_zrpy_gc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_zrpy_gc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_zrpy_gc.py Tue Nov 23 10:12:47 2010
@@ -191,6 +191,33 @@
def run_orig(self, name, n, x):
self.main_allfuncs(name, n, x)
+ def define_libffi_workaround(cls):
+ # XXX: this is a workaround for a bug in database.py. It seems that
+ # the problem is triggered by optimizeopt/fficall.py, and in
+ # particular by the ``cast_base_ptr_to_instance(Func, llfunc)``: in
+ # these tests, that line is the only place where libffi.Func is
+ # referenced.
+ #
+ # The problem occurs because the gctransformer tries to annotate a
+ # low-level helper to call the __del__ of libffi.Func when it's too
+ # late.
+ #
+ # This workaround works by forcing the annotator (and all the rest of
+ # the toolchain) to see libffi.Func in a "proper" context, not just as
+ # the target of cast_base_ptr_to_instance. Note that the function
+ # below is *never* called by any actual test, it's just annotated.
+ #
+ from pypy.rlib.libffi import get_libc_name, CDLL, types, ArgChain
+ libc_name = get_libc_name()
+ def f(n, x, *args):
+ libc = CDLL(libc_name)
+ ptr = libc.getpointer('labs', [types.slong], types.slong)
+ chain = ArgChain()
+ chain.arg(n)
+ n = ptr.call(chain, lltype.Signed)
+ return (n, x) + args
+ return None, f, None
+
def define_compile_framework_1(cls):
# a moving GC. Supports malloc_varsize_nonmovable. Simple test, works
# without write_barriers and root stack enumeration.
@@ -523,3 +550,29 @@
def test_compile_framework_float(self):
self.run('compile_framework_float')
+
+ def define_compile_framework_minimal_size_in_nursery(self):
+ S = lltype.GcStruct('S') # no fields!
+ T = lltype.GcStruct('T', ('i', lltype.Signed))
+ @unroll_safe
+ def f42(n, x, x0, x1, x2, x3, x4, x5, x6, x7, l, s):
+ lst1 = []
+ lst2 = []
+ i = 0
+ while i < 42:
+ s1 = lltype.malloc(S)
+ t1 = lltype.malloc(T)
+ t1.i = 10000 + i + n
+ lst1.append(s1)
+ lst2.append(t1)
+ i += 1
+ i = 0
+ while i < 42:
+ check(lst2[i].i == 10000 + i + n)
+ i += 1
+ n -= 1
+ return n, x, x0, x1, x2, x3, x4, x5, x6, x7, l, s
+ return None, f42, None
+
+ def test_compile_framework_minimal_size_in_nursery(self):
+ self.run('compile_framework_minimal_size_in_nursery')
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_ztranslation.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_ztranslation.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_ztranslation.py Tue Nov 23 10:12:47 2010
@@ -1,7 +1,8 @@
-import py, os
+import py, os, sys
from pypy.tool.udir import udir
from pypy.rlib.jit import JitDriver, OPTIMIZER_FULL, unroll_parameters
from pypy.rlib.jit import PARAMETERS, dont_look_inside
+from pypy.rlib.jit import hint
from pypy.jit.metainterp.jitprof import Profiler
from pypy.jit.backend.detect_cpu import getcpuclass
from pypy.jit.backend.test.support import CCompiledMixin
@@ -9,6 +10,7 @@
from pypy.translator.translator import TranslationContext
from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64
from pypy.config.translationoption import DEFL_GC
+from pypy.rlib import rgc
class TestTranslationX86(CCompiledMixin):
CPUClass = getcpuclass()
@@ -63,8 +65,32 @@
if k - abs(j): raise ValueError
if k - abs(-j): raise ValueError
return total * 10
- res = self.meta_interp(f, [40, -49])
- assert res == f(40, -49)
+ #
+ from pypy.rpython.lltypesystem import lltype, rffi
+ from pypy.rlib.libffi import types, CDLL, ArgChain
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ jitdriver2 = JitDriver(greens=[], reds = ['i', 'func', 'res', 'x'])
+ def libffi_stuff(i, j):
+ lib = CDLL(libm_name)
+ func = lib.getpointer('fabs', [types.double], types.double)
+ res = 0.0
+ x = float(j)
+ while i > 0:
+ jitdriver2.jit_merge_point(i=i, res=res, func=func, x=x)
+ jitdriver2.can_enter_jit(i=i, res=res, func=func, x=x)
+ func = hint(func, promote=True)
+ argchain = ArgChain()
+ argchain.arg(x)
+ res = func.call(argchain, rffi.DOUBLE)
+ i -= 1
+ return int(res)
+ #
+ def main(i, j):
+ return f(i, j) + libffi_stuff(i, j)
+ expected = main(40, -49)
+ res = self.meta_interp(main, [40, -49])
+ assert res == expected
def test_direct_assembler_call_translates(self):
class Thing(object):
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/tool/viewcode.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/tool/viewcode.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/tool/viewcode.py Tue Nov 23 10:12:47 2010
@@ -37,7 +37,7 @@
'x86_64': 'x86-64',
'i386': 'i386',
}
- objdump = ('objdump -M intel,%(backend)s -b binary -m i386 '
+ objdump = ('objdump -M %(backend)s -b binary -m i386 '
'--adjust-vma=%(origin)d -D %(file)s')
#
f = open(tmpfile, 'wb')
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/assembler.py Tue Nov 23 10:12:47 2010
@@ -233,10 +233,9 @@
addr = llmemory.cast_ptr_to_adr(value)
self.list_of_addr2name.append((addr, name))
- def finished(self):
+ def finished(self, callinfocollection):
# Helper called at the end of assembling. Registers the extra
# functions shown in _callinfo_for_oopspec.
- from pypy.jit.codewriter.effectinfo import _callinfo_for_oopspec
- for _, func in _callinfo_for_oopspec.values():
+ for func in callinfocollection.all_function_addresses_as_int():
func = heaptracker.int2adr(func)
self.see_raw_object(func.ptr)
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/call.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/call.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/call.py Tue Nov 23 10:12:47 2010
@@ -7,7 +7,7 @@
from pypy.jit.codewriter.jitcode import JitCode
from pypy.jit.codewriter.effectinfo import VirtualizableAnalyzer
from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze
-from pypy.jit.codewriter.effectinfo import EffectInfo
+from pypy.jit.codewriter.effectinfo import EffectInfo, CallInfoCollection
from pypy.translator.simplify import get_funcobj, get_functype
from pypy.rpython.lltypesystem import lltype, llmemory
from pypy.translator.backendopt.canraise import RaiseAnalyzer
@@ -23,6 +23,7 @@
self.jitdrivers_sd = jitdrivers_sd
self.jitcodes = {} # map {graph: jitcode}
self.unfinished_graphs = [] # list of graphs with pending jitcodes
+ self.callinfocollection = CallInfoCollection()
if hasattr(cpu, 'rtyper'): # for tests
self.rtyper = cpu.rtyper
translator = self.rtyper.annotator.translator
@@ -237,6 +238,8 @@
effectinfo)
def _canraise(self, op):
+ if op.opname == 'pseudo_call_cannot_raise':
+ return False
try:
return self.raise_analyzer.can_raise(op)
except lltype.DelayedPointer:
@@ -277,3 +280,11 @@
return seen.pop()
else:
return None
+
+ def could_be_green_field(self, GTYPE, fieldname):
+ GTYPE_fieldname = (GTYPE, fieldname)
+ for jd in self.jitdrivers_sd:
+ if jd.greenfield_info is not None:
+ if GTYPE_fieldname in jd.greenfield_info.green_fields:
+ return True
+ return False
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/codewriter.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/codewriter.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/codewriter.py Tue Nov 23 10:12:47 2010
@@ -73,7 +73,7 @@
count += 1
if not count % 500:
log.info("Produced %d jitcodes" % count)
- self.assembler.finished()
+ self.assembler.finished(self.callcontrol.callinfocollection)
heaptracker.finish_registering(self.cpu)
log.info("there are %d JitCode instances." % count)
@@ -95,18 +95,18 @@
print '%s:' % (ssarepr.name,)
print format_assembler(ssarepr)
else:
- dir = udir.ensure("jitcodes", dir=1)
- if portal_jitdriver:
- name = "%02d_portal_runner" % (portal_jitdriver.index,)
- elif ssarepr.name and ssarepr.name != '?':
- name = ssarepr.name
- else:
- name = 'unnamed' % id(ssarepr)
- i = 1
- extra = ''
- while name+extra in self._seen_files:
- i += 1
- extra = '.%d' % i
- self._seen_files.add(name+extra)
- dir.join(name+extra).write(format_assembler(ssarepr))
log.dot()
+ dir = udir.ensure("jitcodes", dir=1)
+ if portal_jitdriver:
+ name = "%02d_portal_runner" % (portal_jitdriver.index,)
+ elif ssarepr.name and ssarepr.name != '?':
+ name = ssarepr.name
+ else:
+ name = 'unnamed' % id(ssarepr)
+ i = 1
+ extra = ''
+ while name+extra in self._seen_files:
+ i += 1
+ extra = '.%d' % i
+ self._seen_files.add(name+extra)
+ dir.join(name+extra).write(format_assembler(ssarepr))
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/effectinfo.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/effectinfo.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/effectinfo.py Tue Nov 23 10:12:47 2010
@@ -18,19 +18,34 @@
# the 'oopspecindex' field is one of the following values:
OS_NONE = 0 # normal case, no oopspec
OS_ARRAYCOPY = 1 # "list.ll_arraycopy"
- OS_STR_CONCAT = 2 # "stroruni.concat"
- OS_UNI_CONCAT = 3 # "stroruni.concat"
- OS_STR_SLICE = 4 # "stroruni.slice"
- OS_UNI_SLICE = 5 # "stroruni.slice"
- OS_STR_EQUAL = 6 # "stroruni.equal"
- OS_UNI_EQUAL = 7 # "stroruni.equal"
- OS_STREQ_SLICE_CHECKNULL = 8 # s2!=NULL and s1[x:x+length]==s2
- OS_STREQ_SLICE_NONNULL = 9 # s1[x:x+length]==s2 (assert s2!=NULL)
- OS_STREQ_SLICE_CHAR = 10 # s1[x:x+length]==char
- OS_STREQ_NONNULL = 11 # s1 == s2 (assert s1!=NULL,s2!=NULL)
- OS_STREQ_NONNULL_CHAR = 12 # s1 == char (assert s1!=NULL)
- OS_STREQ_CHECKNULL_CHAR = 13 # s1!=NULL and s1==char
- OS_STREQ_LENGTHOK = 14 # s1 == s2 (assert len(s1)==len(s2))
+ OS_STR2UNICODE = 2 # "str.str2unicode"
+ #
+ OS_STR_CONCAT = 22 # "stroruni.concat"
+ OS_STR_SLICE = 23 # "stroruni.slice"
+ OS_STR_EQUAL = 24 # "stroruni.equal"
+ OS_STREQ_SLICE_CHECKNULL = 25 # s2!=NULL and s1[x:x+length]==s2
+ OS_STREQ_SLICE_NONNULL = 26 # s1[x:x+length]==s2 (assert s2!=NULL)
+ OS_STREQ_SLICE_CHAR = 27 # s1[x:x+length]==char
+ OS_STREQ_NONNULL = 28 # s1 == s2 (assert s1!=NULL,s2!=NULL)
+ OS_STREQ_NONNULL_CHAR = 29 # s1 == char (assert s1!=NULL)
+ OS_STREQ_CHECKNULL_CHAR = 30 # s1!=NULL and s1==char
+ OS_STREQ_LENGTHOK = 31 # s1 == s2 (assert len(s1)==len(s2))
+ #
+ OS_UNI_CONCAT = 42 #
+ OS_UNI_SLICE = 43 #
+ OS_UNI_EQUAL = 44 #
+ OS_UNIEQ_SLICE_CHECKNULL = 45 #
+ OS_UNIEQ_SLICE_NONNULL = 46 #
+ OS_UNIEQ_SLICE_CHAR = 47 #
+ OS_UNIEQ_NONNULL = 48 # the same for unicode
+ OS_UNIEQ_NONNULL_CHAR = 49 # (must be the same amount as for
+ OS_UNIEQ_CHECKNULL_CHAR = 50 # STR, in the same order)
+ OS_UNIEQ_LENGTHOK = 51 #
+ _OS_offset_uni = OS_UNI_CONCAT - OS_STR_CONCAT
+ #
+ OS_LIBFFI_PREPARE = 60
+ OS_LIBFFI_PUSH_ARG = 61
+ OS_LIBFFI_CALL = 62
def __new__(cls, readonly_descrs_fields,
write_descrs_fields, write_descrs_arrays,
@@ -129,30 +144,44 @@
# ____________________________________________________________
-_callinfo_for_oopspec = {} # {oopspecindex: (calldescr, func_as_int)}
+class CallInfoCollection(object):
+ def __init__(self):
+ # {oopspecindex: (calldescr, func_as_int)}
+ self._callinfo_for_oopspec = {}
-def callinfo_for_oopspec(oopspecindex):
- """A function that returns the calldescr and the function
- address (as an int) of one of the OS_XYZ functions defined above.
- Don't use this if there might be several implementations of the same
- OS_XYZ specialized by type, e.g. OS_ARRAYCOPY."""
- try:
- return _callinfo_for_oopspec[oopspecindex]
- except KeyError:
- return (None, 0)
-
-
-def _funcptr_for_oopspec_memo(oopspecindex):
- from pypy.jit.codewriter import heaptracker
- _, func_as_int = callinfo_for_oopspec(oopspecindex)
- funcadr = heaptracker.int2adr(func_as_int)
- return funcadr.ptr
-_funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo'
-
-def funcptr_for_oopspec(oopspecindex):
- """A memo function that returns a pointer to the function described
- by OS_XYZ (as a real low-level function pointer)."""
- funcptr = _funcptr_for_oopspec_memo(oopspecindex)
- assert funcptr
- return funcptr
-funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(0)'
+ def _freeze_(self):
+ return True
+
+ def add(self, oopspecindex, calldescr, func_as_int):
+ self._callinfo_for_oopspec[oopspecindex] = calldescr, func_as_int
+
+ def has_oopspec(self, oopspecindex):
+ return oopspecindex in self._callinfo_for_oopspec
+
+ def all_function_addresses_as_int(self):
+ return [func for (_, func) in self._callinfo_for_oopspec.values()]
+
+ def callinfo_for_oopspec(self, oopspecindex):
+ """A function that returns the calldescr and the function
+ address (as an int) of one of the OS_XYZ functions defined above.
+ Don't use this if there might be several implementations of the same
+ OS_XYZ specialized by type, e.g. OS_ARRAYCOPY."""
+ try:
+ return self._callinfo_for_oopspec[oopspecindex]
+ except KeyError:
+ return (None, 0)
+
+ def _funcptr_for_oopspec_memo(self, oopspecindex):
+ from pypy.jit.codewriter import heaptracker
+ _, func_as_int = self.callinfo_for_oopspec(oopspecindex)
+ funcadr = heaptracker.int2adr(func_as_int)
+ return funcadr.ptr
+ _funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo'
+
+ def funcptr_for_oopspec(self, oopspecindex):
+ """A memo function that returns a pointer to the function described
+ by OS_XYZ (as a real low-level function pointer)."""
+ funcptr = self._funcptr_for_oopspec_memo(oopspecindex)
+ assert funcptr
+ return funcptr
+ funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(1)'
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/format.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/format.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/format.py Tue Nov 23 10:12:47 2010
@@ -80,7 +80,8 @@
def assert_format(ssarepr, expected):
asm = format_assembler(ssarepr)
- expected = str(py.code.Source(expected)).strip() + '\n'
+ if expected != '':
+ expected = str(py.code.Source(expected)).strip() + '\n'
asmlines = asm.split("\n")
explines = expected.split("\n")
for asm, exp in zip(asmlines, explines):
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/jtransform.py Tue Nov 23 10:12:47 2010
@@ -6,7 +6,7 @@
from pypy.objspace.flow.model import Block, Link, c_last_exception
from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets
from pypy.jit.codewriter import support, heaptracker
-from pypy.jit.codewriter.effectinfo import EffectInfo, _callinfo_for_oopspec
+from pypy.jit.codewriter.effectinfo import EffectInfo
from pypy.jit.codewriter.policy import log
from pypy.jit.metainterp.typesystem import deref, arrayItem
from pypy.rlib import objectmodel
@@ -172,6 +172,7 @@
def rewrite_op_same_as(self, op): pass
def rewrite_op_cast_pointer(self, op): pass
+ def rewrite_op_cast_opaque_ptr(self, op): pass # rlib.rerased
def rewrite_op_cast_primitive(self, op): pass
def rewrite_op_cast_bool_to_int(self, op): pass
def rewrite_op_cast_bool_to_uint(self, op): pass
@@ -316,8 +317,14 @@
prepare = self._handle_list_call
elif oopspec_name.startswith('stroruni.'):
prepare = self._handle_stroruni_call
+ elif oopspec_name == 'str.str2unicode':
+ prepare = self._handle_str2unicode_call
elif oopspec_name.startswith('virtual_ref'):
prepare = self._handle_virtual_ref_call
+ elif oopspec_name.startswith('jit.'):
+ prepare = self._handle_jit_call
+ elif oopspec_name.startswith('libffi_'):
+ prepare = self._handle_libffi_call
else:
prepare = self.prepare_builtin_call
try:
@@ -427,7 +434,8 @@
op.result)
def rewrite_op_free(self, op):
- assert op.args[1].value == 'raw'
+ flags = op.args[1].value
+ assert flags['flavor'] == 'raw'
ARRAY = op.args[0].concretetype.TO
return self._do_builtin_call(op, 'raw_free', [op.args[0]],
extra = (ARRAY,), extrakey = ARRAY)
@@ -519,7 +527,12 @@
# check for deepfrozen structures that force constant-folding
immut = v_inst.concretetype.TO._immutable_field(c_fieldname.value)
if immut:
- pure = '_pure'
+ if (self.callcontrol is not None and
+ self.callcontrol.could_be_green_field(v_inst.concretetype.TO,
+ c_fieldname.value)):
+ pure = '_greenfield'
+ else:
+ pure = '_pure'
if immut == "[*]":
self.immutable_arrays[op.result] = True
else:
@@ -819,6 +832,8 @@
def rewrite_op_jit_marker(self, op):
key = op.args[0].value
jitdriver = op.args[1].value
+ if not jitdriver.active:
+ return []
return getattr(self, 'handle_jit_marker__%s' % key)(op, jitdriver)
def handle_jit_marker__jit_merge_point(self, op, jitdriver):
@@ -828,9 +843,16 @@
"general mix-up of jitdrivers?")
ops = self.promote_greens(op.args[2:], jitdriver)
num_green_args = len(jitdriver.greens)
+ redlists = self.make_three_lists(op.args[2+num_green_args:])
+ for redlist in redlists:
+ for v in redlist:
+ assert isinstance(v, Variable), (
+ "Constant specified red in jit_merge_point()")
+ assert len(dict.fromkeys(redlist)) == len(list(redlist)), (
+ "duplicate red variable on jit_merge_point()")
args = ([Constant(self.portal_jd.index, lltype.Signed)] +
self.make_three_lists(op.args[2:2+num_green_args]) +
- self.make_three_lists(op.args[2+num_green_args:]))
+ redlists)
op1 = SpaceOperation('jit_merge_point', args, None)
op2 = SpaceOperation('-live-', [], None)
# ^^^ we need a -live- for the case of do_recursive_call()
@@ -852,6 +874,17 @@
(self.graph,))
return []
+ def _handle_jit_call(self, op, oopspec_name, args):
+ if oopspec_name == 'jit.debug':
+ return SpaceOperation('jit_debug', args, None)
+ elif oopspec_name == 'jit.assert_green':
+ kind = getkind(args[0].concretetype)
+ return SpaceOperation('%s_assert_green' % kind, args, None)
+ elif oopspec_name == 'jit.current_trace_length':
+ return SpaceOperation('current_trace_length', [], op.result)
+ else:
+ raise AssertionError("missing support for %r" % oopspec_name)
+
# ----------
# Lists.
@@ -870,17 +903,21 @@
prefix = 'do_resizable_'
ARRAY = LIST.items.TO
if self._array_of_voids(ARRAY):
- raise NotSupported("resizable lists of voids")
- descrs = (self.cpu.arraydescrof(ARRAY),
- self.cpu.fielddescrof(LIST, 'length'),
- self.cpu.fielddescrof(LIST, 'items'),
- self.cpu.sizeof(LIST))
+ prefix += 'void_'
+ descrs = ()
+ else:
+ descrs = (self.cpu.arraydescrof(ARRAY),
+ self.cpu.fielddescrof(LIST, 'length'),
+ self.cpu.fielddescrof(LIST, 'items'),
+ self.cpu.sizeof(LIST))
else:
prefix = 'do_fixed_'
if self._array_of_voids(LIST):
- raise NotSupported("fixed lists of voids")
- arraydescr = self.cpu.arraydescrof(LIST)
- descrs = (arraydescr,)
+ prefix += 'void_'
+ descrs = ()
+ else:
+ arraydescr = self.cpu.arraydescrof(LIST)
+ descrs = (arraydescr,)
#
try:
meth = getattr(self, prefix + oopspec_name.replace('.', '_'))
@@ -919,6 +956,11 @@
descr, args[1]], v_posindex)
return v_posindex, [op0, op1]
+ def _prepare_void_list_getset(self, op):
+ non_negative, can_raise = self._get_list_nonneg_canraise_flags(op)
+ if can_raise:
+ raise NotSupported("list operation can raise")
+
def _get_initial_newlist_length(self, op, args):
# normalize number of arguments to the 'newlist' function
if len(args) > 1:
@@ -990,6 +1032,12 @@
def do_fixed_list_ll_arraycopy(self, op, args, arraydescr):
return self._handle_oopspec_call(op, args, EffectInfo.OS_ARRAYCOPY)
+ def do_fixed_void_list_getitem(self, op, args):
+ self._prepare_void_list_getset(op)
+ return []
+ do_fixed_void_list_getitem_foldable = do_fixed_void_list_getitem
+ do_fixed_void_list_setitem = do_fixed_void_list_getitem
+
# ---------- resizable lists ----------
def do_resizable_newlist(self, op, args, arraydescr, lengthdescr,
@@ -1025,17 +1073,26 @@
return SpaceOperation('getfield_gc_i',
[args[0], lengthdescr], op.result)
+ def do_resizable_void_list_getitem(self, op, args):
+ self._prepare_void_list_getset(op)
+ return []
+ do_resizable_void_list_getitem_foldable = do_resizable_void_list_getitem
+ do_resizable_void_list_setitem = do_resizable_void_list_getitem
+
# ----------
# Strings and Unicodes.
- def _handle_oopspec_call(self, op, args, oopspecindex):
+ def _handle_oopspec_call(self, op, args, oopspecindex, extraeffect=None):
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
+ if extraeffect:
+ calldescr.get_extra_info().extraeffect = extraeffect
if isinstance(op.args[0].value, str):
pass # for tests only
else:
func = heaptracker.adr2int(
llmemory.cast_ptr_to_adr(op.args[0].value))
- _callinfo_for_oopspec[oopspecindex] = calldescr, func
+ self.callcontrol.callinfocollection.add(oopspecindex,
+ calldescr, func)
op1 = self.rewrite_call(op, 'residual_call',
[op.args[0], calldescr],
args=args)
@@ -1046,37 +1103,41 @@
def _register_extra_helper(self, oopspecindex, oopspec_name,
argtypes, resulttype):
# a bit hackish
- if oopspecindex in _callinfo_for_oopspec:
+ if self.callcontrol.callinfocollection.has_oopspec(oopspecindex):
return
c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper,
oopspec_name, argtypes,
resulttype)
- op = SpaceOperation('pseudo_call',
+ op = SpaceOperation('pseudo_call_cannot_raise',
[c_func] + [varoftype(T) for T in argtypes],
varoftype(resulttype))
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
- func = heaptracker.adr2int(
- llmemory.cast_ptr_to_adr(c_func.value))
- _callinfo_for_oopspec[oopspecindex] = calldescr, func
+ if isinstance(c_func.value, str): # in tests only
+ func = c_func.value
+ else:
+ func = heaptracker.adr2int(
+ llmemory.cast_ptr_to_adr(c_func.value))
+ self.callcontrol.callinfocollection.add(oopspecindex, calldescr, func)
def _handle_stroruni_call(self, op, oopspec_name, args):
- if args[0].concretetype.TO == rstr.STR:
+ SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
+ if SoU.TO == rstr.STR:
dict = {"stroruni.concat": EffectInfo.OS_STR_CONCAT,
"stroruni.slice": EffectInfo.OS_STR_SLICE,
"stroruni.equal": EffectInfo.OS_STR_EQUAL,
}
- elif args[0].concretetype.TO == rstr.UNICODE:
+ CHR = lltype.Char
+ elif SoU.TO == rstr.UNICODE:
dict = {"stroruni.concat": EffectInfo.OS_UNI_CONCAT,
"stroruni.slice": EffectInfo.OS_UNI_SLICE,
"stroruni.equal": EffectInfo.OS_UNI_EQUAL,
}
+ CHR = lltype.UniChar
else:
assert 0, "args[0].concretetype must be STR or UNICODE"
#
if oopspec_name == "stroruni.equal":
- SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
for otherindex, othername, argtypes, resulttype in [
-
(EffectInfo.OS_STREQ_SLICE_CHECKNULL,
"str.eq_slice_checknull",
[SoU, lltype.Signed, lltype.Signed, SoU],
@@ -1087,7 +1148,7 @@
lltype.Signed),
(EffectInfo.OS_STREQ_SLICE_CHAR,
"str.eq_slice_char",
- [SoU, lltype.Signed, lltype.Signed, lltype.Char],
+ [SoU, lltype.Signed, lltype.Signed, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL,
"str.eq_nonnull",
@@ -1095,22 +1156,27 @@
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL_CHAR,
"str.eq_nonnull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_CHECKNULL_CHAR,
"str.eq_checknull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_LENGTHOK,
"str.eq_lengthok",
[SoU, SoU],
lltype.Signed),
]:
+ if args[0].concretetype.TO == rstr.UNICODE:
+ otherindex += EffectInfo._OS_offset_uni
self._register_extra_helper(otherindex, othername,
argtypes, resulttype)
#
return self._handle_oopspec_call(op, args, dict[oopspec_name])
+ def _handle_str2unicode_call(self, op, oopspec_name, args):
+ return self._handle_oopspec_call(op, args, EffectInfo.OS_STR2UNICODE)
+
# ----------
# VirtualRefs.
@@ -1121,6 +1187,23 @@
vrefinfo.JIT_VIRTUAL_REF)
return SpaceOperation(oopspec_name, list(args), op.result)
+ # -----------
+ # rlib.libffi
+
+ def _handle_libffi_call(self, op, oopspec_name, args):
+ if oopspec_name == 'libffi_prepare_call':
+ oopspecindex = EffectInfo.OS_LIBFFI_PREPARE
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
+ elif oopspec_name.startswith('libffi_push_'):
+ oopspecindex = EffectInfo.OS_LIBFFI_PUSH_ARG
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
+ elif oopspec_name.startswith('libffi_call_'):
+ oopspecindex = EffectInfo.OS_LIBFFI_CALL
+ extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
+ else:
+ assert False, 'unsupported oopspec: %s' % oopspec_name
+ return self._handle_oopspec_call(op, args, oopspecindex, extraeffect)
+
def rewrite_op_jit_force_virtual(self, op):
return self._do_builtin_call(op)
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/support.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/support.py Tue Nov 23 10:12:47 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, rclass
+from pypy.rpython.lltypesystem import lltype, rclass, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython import rlist
from pypy.rpython.lltypesystem import rstr as ll_rstr, rdict as ll_rdict
@@ -8,6 +8,7 @@
from pypy.rpython.ootypesystem import rdict as oo_rdict
from pypy.rpython.llinterp import LLInterpreter
from pypy.rpython.extregistry import ExtRegistryEntry
+from pypy.rpython.annlowlevel import cast_base_ptr_to_instance
from pypy.translator.simplify import get_funcobj
from pypy.translator.unsimplify import split_block
from pypy.objspace.flow.model import Constant
@@ -60,7 +61,7 @@
return rtyper.annotator.translator.graphs[0]
def split_before_jit_merge_point(graph, portalblock, portalopindex):
- """Find the block with 'jit_merge_point' and split just before,
+ """Split the block just before the 'jit_merge_point',
making sure the input args are in the canonical order.
"""
# split the block just before the jit_merge_point()
@@ -217,6 +218,33 @@
else:
return x
+
+# libffi support
+# --------------
+
+def func(llfunc):
+ from pypy.rlib.libffi import Func
+ return cast_base_ptr_to_instance(Func, llfunc)
+
+def _ll_1_libffi_prepare_call(llfunc):
+ return func(llfunc)._prepare()
+
+def _ll_4_libffi_push_int(llfunc, value, ll_args, i):
+ return func(llfunc)._push_int(value, ll_args, i)
+
+def _ll_4_libffi_push_float(llfunc, value, ll_args, i):
+ return func(llfunc)._push_float(value, ll_args, i)
+
+def _ll_3_libffi_call_int(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.LONG)
+
+def _ll_3_libffi_call_float(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.DOUBLE)
+
+def _ll_3_libffi_call_void(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, lltype.Void)
+
+
# in the following calls to builtins, the JIT is allowed to look inside:
inline_calls_to = [
('int_floordiv_ovf_zer', [lltype.Signed, lltype.Signed], lltype.Signed),
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/test/test_codewriter.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/test/test_codewriter.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/test/test_codewriter.py Tue Nov 23 10:12:47 2010
@@ -45,6 +45,7 @@
self.portal_graph = portal_graph
self.portal_runner_ptr = "???"
self.virtualizable_info = None
+ self.greenfield_info = None
def test_loop():
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/test/test_effectinfo.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/test/test_effectinfo.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/test/test_effectinfo.py Tue Nov 23 10:12:47 2010
@@ -1,7 +1,8 @@
from pypy.rpython.lltypesystem.rclass import OBJECT
from pypy.rpython.lltypesystem import lltype
from pypy.rpython.ootypesystem import ootype
-from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze
+from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze,\
+ EffectInfo
class FakeCPU:
def fielddescrof(self, T, fieldname):
@@ -9,6 +10,14 @@
def arraydescrof(self, A):
return ('arraydescr', A)
+def test_no_oopspec_duplicate():
+ # check that all the various EffectInfo.OS_* have unique values
+ oopspecs = set()
+ for name, value in EffectInfo.__dict__.iteritems():
+ if name.startswith('OS_'):
+ assert value not in oopspecs
+ oopspecs.add(value)
+
def test_include_read_field():
S = lltype.GcStruct("S", ("a", lltype.Signed))
effects = frozenset([("readstruct", lltype.Ptr(S), "a")])
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/test/test_jtransform.py Tue Nov 23 10:12:47 2010
@@ -74,10 +74,48 @@
def calldescr_canraise(self, calldescr):
return False
+class FakeCallInfoCollection:
+ def __init__(self):
+ self.seen = []
+ def add(self, oopspecindex, calldescr, func):
+ self.seen.append((oopspecindex, calldescr, func))
+ def has_oopspec(self, oopspecindex):
+ for i, c, f in self.seen:
+ if i == oopspecindex:
+ return True
+ return False
+
class FakeBuiltinCallControl:
+ def __init__(self):
+ self.callinfocollection = FakeCallInfoCollection()
def guess_call_kind(self, op):
return 'builtin'
- def getcalldescr(self, op, oopspecindex):
+ def getcalldescr(self, op, oopspecindex=None):
+ assert oopspecindex is not None # in this test
+ EI = effectinfo.EffectInfo
+ if oopspecindex != EI.OS_ARRAYCOPY:
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ INT = lltype.Signed
+ UNICHAR = lltype.UniChar
+ argtypes = {
+ EI.OS_STR2UNICODE:([PSTR], PUNICODE),
+ EI.OS_STR_CONCAT: ([PSTR, PSTR], PSTR),
+ EI.OS_STR_SLICE: ([PSTR, INT, INT], PSTR),
+ EI.OS_UNI_CONCAT: ([PUNICODE, PUNICODE], PUNICODE),
+ EI.OS_UNI_SLICE: ([PUNICODE, INT, INT], PUNICODE),
+ EI.OS_UNI_EQUAL: ([PUNICODE, PUNICODE], lltype.Bool),
+ EI.OS_UNIEQ_SLICE_CHECKNULL:([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_NONNULL: ([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_CHAR: ([PUNICODE, INT, INT, UNICHAR], INT),
+ EI.OS_UNIEQ_NONNULL: ([PUNICODE, PUNICODE], INT),
+ EI.OS_UNIEQ_NONNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_CHECKNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_LENGTHOK: ([PUNICODE, PUNICODE], INT),
+ }
+ argtypes = argtypes[oopspecindex]
+ assert argtypes[0] == [v.concretetype for v in op.args[1:]]
+ assert argtypes[1] == op.result.concretetype
return 'calldescr-%d' % oopspecindex
def calldescr_canraise(self, calldescr):
return False
@@ -662,6 +700,80 @@
assert block.operations[1].result is None
assert block.exits[0].args == [v1]
+def test_jit_merge_point_1():
+ class FakeJitDriverSD:
+ index = 42
+ class jitdriver:
+ active = True
+ greens = ['green1', 'green2', 'voidgreen3']
+ reds = ['red1', 'red2', 'voidred3']
+ jd = FakeJitDriverSD()
+ v1 = varoftype(lltype.Signed)
+ v2 = varoftype(lltype.Signed)
+ vvoid1 = varoftype(lltype.Void)
+ v3 = varoftype(lltype.Signed)
+ v4 = varoftype(lltype.Signed)
+ vvoid2 = varoftype(lltype.Void)
+ v5 = varoftype(lltype.Void)
+ op = SpaceOperation('jit_marker',
+ [Constant('jit_merge_point', lltype.Void),
+ Constant(jd.jitdriver, lltype.Void),
+ v1, v2, vvoid1, v3, v4, vvoid2], v5)
+ tr = Transformer()
+ tr.portal_jd = jd
+ oplist = tr.rewrite_operation(op)
+ assert len(oplist) == 6
+ assert oplist[0].opname == '-live-'
+ assert oplist[1].opname == 'int_guard_value'
+ assert oplist[1].args == [v1]
+ assert oplist[2].opname == '-live-'
+ assert oplist[3].opname == 'int_guard_value'
+ assert oplist[3].args == [v2]
+ assert oplist[4].opname == 'jit_merge_point'
+ assert oplist[4].args[0].value == 42
+ assert list(oplist[4].args[1]) == [v1, v2]
+ assert list(oplist[4].args[4]) == [v3, v4]
+ assert oplist[5].opname == '-live-'
+
+def test_getfield_gc():
+ S = lltype.GcStruct('S', ('x', lltype.Char))
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_pure():
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_pure'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_greenfield():
+ class FakeCC:
+ def get_vinfo(self, v):
+ return None
+ def could_be_green_field(self, S1, name1):
+ assert S1 is S
+ assert name1 == 'x'
+ return True
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU(), FakeCC()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_greenfield'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
def test_int_abs():
v1 = varoftype(lltype.Signed)
v2 = varoftype(lltype.Signed)
@@ -711,7 +823,8 @@
v2 = varoftype(PSTR)
v3 = varoftype(PSTR)
op = SpaceOperation('direct_call', [const(func), v1, v2], v3)
- tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ cc = FakeBuiltinCallControl()
+ tr = Transformer(FakeCPU(), cc)
op1 = tr.rewrite_operation(op)
assert op1.opname == 'residual_call_r_r'
assert op1.args[0].value == func
@@ -720,9 +833,10 @@
assert op1.result == v3
#
# check the callinfo_for_oopspec
- got = effectinfo.callinfo_for_oopspec(effectinfo.EffectInfo.OS_UNI_CONCAT)
- assert got[0] == op1.args[1] # the calldescr
- assert heaptracker.int2adr(got[1]) == llmemory.cast_ptr_to_adr(func)
+ got = cc.callinfocollection.seen[0]
+ assert got[0] == effectinfo.EffectInfo.OS_UNI_CONCAT
+ assert got[1] == op1.args[1] # the calldescr
+ assert heaptracker.int2adr(got[2]) == llmemory.cast_ptr_to_adr(func)
def test_str_slice():
# test that the oopspec is present and correctly transformed
@@ -766,6 +880,47 @@
assert op1.args[3] == ListOfKind('ref', [v1])
assert op1.result == v4
+def test_str2unicode():
+ # test that the oopspec is present and correctly transformed
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PSTR], PUNICODE)
+ func = lltype.functionptr(FUNC, 'll_str2unicode',
+ _callable=rstr.LLHelpers.ll_str2unicode)
+ v1 = varoftype(PSTR)
+ v2 = varoftype(PUNICODE)
+ op = SpaceOperation('direct_call', [const(func), v1], v2)
+ tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_r'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_STR2UNICODE
+ assert op1.args[2] == ListOfKind('ref', [v1])
+ assert op1.result == v2
+
+def test_unicode_eq_checknull_char():
+ # test that the oopspec is present and correctly transformed
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PUNICODE, PUNICODE], lltype.Bool)
+ func = lltype.functionptr(FUNC, 'll_streq',
+ _callable=rstr.LLHelpers.ll_streq)
+ v1 = varoftype(PUNICODE)
+ v2 = varoftype(PUNICODE)
+ v3 = varoftype(lltype.Bool)
+ op = SpaceOperation('direct_call', [const(func), v1, v2], v3)
+ cc = FakeBuiltinCallControl()
+ tr = Transformer(FakeCPU(), cc)
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_i'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_UNI_EQUAL
+ assert op1.args[2] == ListOfKind('ref', [v1, v2])
+ assert op1.result == v3
+ # test that the OS_UNIEQ_* functions are registered
+ cic = cc.callinfocollection
+ assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL)
+ assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR)
+
def test_list_ll_arraycopy():
from pypy.rlib.rgc import ll_arraycopy
LIST = lltype.GcArray(lltype.Signed)
Modified: pypy/branch/arm-backend/pypy/jit/codewriter/test/test_list.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/codewriter/test/test_list.py (original)
+++ pypy/branch/arm-backend/pypy/jit/codewriter/test/test_list.py Tue Nov 23 10:12:47 2010
@@ -19,6 +19,7 @@
class FakeCPU:
class arraydescrof(AbstractDescr):
def __init__(self, ARRAY):
+ assert ARRAY.OF != lltype.Void
self.ARRAY = ARRAY
def __repr__(self):
return '<ArrayDescr>'
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/blackhole.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/blackhole.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/blackhole.py Tue Nov 23 10:12:47 2010
@@ -760,6 +760,24 @@
def bhimpl_debug_fatalerror(msg):
llop.debug_fatalerror(lltype.Void, msg)
+ @arguments("r", "i", "i", "i", "i")
+ def bhimpl_jit_debug(string, arg1=0, arg2=0, arg3=0, arg4=0):
+ pass
+
+ @arguments("i")
+ def bhimpl_int_assert_green(x):
+ pass
+ @arguments("r")
+ def bhimpl_ref_assert_green(x):
+ pass
+ @arguments("f")
+ def bhimpl_float_assert_green(x):
+ pass
+
+ @arguments(returns="i")
+ def bhimpl_current_trace_length():
+ return -1
+
# ----------
# the main hints and recursive calls
@@ -1073,6 +1091,10 @@
bhimpl_getfield_vable_r = bhimpl_getfield_gc_r
bhimpl_getfield_vable_f = bhimpl_getfield_gc_f
+ bhimpl_getfield_gc_i_greenfield = bhimpl_getfield_gc_i
+ bhimpl_getfield_gc_r_greenfield = bhimpl_getfield_gc_r
+ bhimpl_getfield_gc_f_greenfield = bhimpl_getfield_gc_f
+
@arguments("cpu", "i", "d", returns="i")
def bhimpl_getfield_raw_i(cpu, struct, fielddescr):
return cpu.bh_getfield_raw_i(struct, fielddescr)
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/compile.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/compile.py Tue Nov 23 10:12:47 2010
@@ -1,4 +1,5 @@
+from pypy.rpython.lltypesystem import lltype
from pypy.rpython.ootypesystem import ootype
from pypy.objspace.flow.model import Constant, Variable
from pypy.rlib.objectmodel import we_are_translated
@@ -14,6 +15,7 @@
from pypy.jit.metainterp.specnode import NotSpecNode, more_general_specnodes
from pypy.jit.metainterp.typesystem import llhelper, oohelper
from pypy.jit.metainterp.optimizeutil import InvalidLoop
+from pypy.jit.metainterp.resume import NUMBERING
from pypy.jit.codewriter import heaptracker
def giveup():
@@ -48,13 +50,12 @@
# ____________________________________________________________
-def compile_new_loop(metainterp, old_loop_tokens, greenkey, start):
+def compile_new_loop(metainterp, old_loop_tokens, start):
"""Try to compile a new loop by closing the current history back
to the first operation.
"""
history = metainterp.history
loop = create_empty_loop(metainterp)
- loop.greenkey = greenkey
loop.inputargs = history.inputargs
for box in loop.inputargs:
assert isinstance(box, Box)
@@ -123,7 +124,6 @@
if not we_are_translated():
show_loop(metainterp_sd)
TreeLoop.check_consistency_of(inputargs, operations)
- pass
metainterp_sd.profiler.start_backend()
debug_start("jit-backend")
try:
@@ -208,8 +208,7 @@
}
class ResumeDescr(AbstractFailDescr):
- def __init__(self, original_greenkey):
- self.original_greenkey = original_greenkey
+ pass
class ResumeGuardDescr(ResumeDescr):
_counter = 0 # if < 0, there is one counter per value;
@@ -218,7 +217,7 @@
# this class also gets the following attributes stored by resume.py code
rd_snapshot = None
rd_frame_info_list = None
- rd_numb = None
+ rd_numb = lltype.nullptr(NUMBERING)
rd_consts = None
rd_virtuals = None
rd_pendingfields = None
@@ -228,8 +227,7 @@
CNT_FLOAT = -0x60000000
CNT_MASK = 0x1FFFFFFF
- def __init__(self, metainterp_sd, original_greenkey):
- ResumeDescr.__init__(self, original_greenkey)
+ def __init__(self, metainterp_sd):
self.metainterp_sd = metainterp_sd
def store_final_boxes(self, guard_op, boxes):
@@ -333,14 +331,14 @@
res.rd_pendingfields = self.rd_pendingfields
def _clone_if_mutable(self):
- res = ResumeGuardDescr(self.metainterp_sd, self.original_greenkey)
+ res = ResumeGuardDescr(self.metainterp_sd)
self.copy_all_attrbutes_into(res)
return res
class ResumeGuardForcedDescr(ResumeGuardDescr):
- def __init__(self, metainterp_sd, original_greenkey, jitdriver_sd):
- ResumeGuardDescr.__init__(self, metainterp_sd, original_greenkey)
+ def __init__(self, metainterp_sd, jitdriver_sd):
+ ResumeGuardDescr.__init__(self, metainterp_sd)
self.jitdriver_sd = jitdriver_sd
def handle_fail(self, metainterp_sd, jitdriver_sd):
@@ -370,7 +368,8 @@
from pypy.jit.metainterp.resume import force_from_resumedata
metainterp_sd = self.metainterp_sd
vinfo = self.jitdriver_sd.virtualizable_info
- all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo)
+ ginfo = self.jitdriver_sd.greenfield_info
+ all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo, ginfo)
# The virtualizable data was stored on the real virtualizable above.
# Handle all_virtuals: keep them for later blackholing from the
# future failure of the GUARD_NOT_FORCED
@@ -406,7 +405,6 @@
def _clone_if_mutable(self):
res = ResumeGuardForcedDescr(self.metainterp_sd,
- self.original_greenkey,
self.jitdriver_sd)
self.copy_all_attrbutes_into(res)
return res
@@ -473,9 +471,8 @@
class ResumeFromInterpDescr(ResumeDescr):
- def __init__(self, original_greenkey, redkey):
- ResumeDescr.__init__(self, original_greenkey)
- self.redkey = redkey
+ def __init__(self, original_greenkey):
+ self.original_greenkey = original_greenkey
def compile_and_attach(self, metainterp, new_loop):
# We managed to create a bridge going from the interpreter
@@ -484,10 +481,8 @@
# with completely unoptimized arguments, as in the interpreter.
metainterp_sd = metainterp.staticdata
jitdriver_sd = metainterp.jitdriver_sd
- metainterp.history.inputargs = self.redkey
- new_loop_token = make_loop_token(len(self.redkey), jitdriver_sd)
- new_loop.greenkey = self.original_greenkey
- new_loop.inputargs = self.redkey
+ redargs = new_loop.inputargs
+ new_loop_token = make_loop_token(len(redargs), jitdriver_sd)
new_loop.token = new_loop_token
send_loop_to_backend(metainterp_sd, new_loop, "entry bridge")
# send the new_loop to warmspot.py, to be called directly the next time
@@ -599,5 +594,5 @@
ResOperation(rop.FINISH, finishargs, None, descr=jd.portal_finishtoken)
]
operations[1].setfailargs([])
- cpu.compile_loop(inputargs, operations, loop_token)
+ cpu.compile_loop(inputargs, operations, loop_token, log=False)
return loop_token
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/executor.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/executor.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/executor.py Tue Nov 23 10:12:47 2010
@@ -80,6 +80,9 @@
do_call_loopinvariant = do_call
do_call_may_force = do_call
+def do_call_c(cpu, metainterp, argboxes, descr):
+ raise NotImplementedError("Should never be called directly")
+
def do_getarrayitem_gc(cpu, _, arraybox, indexbox, arraydescr):
array = arraybox.getref_base()
index = indexbox.getint()
@@ -205,8 +208,8 @@
def do_copystrcontent(cpu, _, srcbox, dstbox,
srcstartbox, dststartbox, lengthbox):
- src = srcbox.getptr(lltype.Ptr(rstr.STR))
- dst = dstbox.getptr(lltype.Ptr(rstr.STR))
+ src = srcbox.getref(lltype.Ptr(rstr.STR))
+ dst = dstbox.getref(lltype.Ptr(rstr.STR))
srcstart = srcstartbox.getint()
dststart = dststartbox.getint()
length = lengthbox.getint()
@@ -214,8 +217,8 @@
def do_copyunicodecontent(cpu, _, srcbox, dstbox,
srcstartbox, dststartbox, lengthbox):
- src = srcbox.getptr(lltype.Ptr(rstr.UNICODE))
- dst = dstbox.getptr(lltype.Ptr(rstr.UNICODE))
+ src = srcbox.getref(lltype.Ptr(rstr.UNICODE))
+ dst = dstbox.getref(lltype.Ptr(rstr.UNICODE))
srcstart = srcstartbox.getint()
dststart = dststartbox.getint()
length = lengthbox.getint()
@@ -304,6 +307,7 @@
rop.CALL_ASSEMBLER,
rop.COND_CALL_GC_WB,
rop.DEBUG_MERGE_POINT,
+ rop.JIT_DEBUG,
rop.SETARRAYITEM_RAW,
): # list of opcodes never executed by pyjitpl
continue
@@ -428,6 +432,10 @@
if arity == 3:
func = get_execute_funclist(3, False)[opnum]
return func(cpu, metainterp, argboxes[0], argboxes[1], argboxes[2])
+ if arity == 5: # copystrcontent, copyunicodecontent
+ func = get_execute_funclist(5, False)[opnum]
+ return func(cpu, metainterp, argboxes[0], argboxes[1],
+ argboxes[2], argboxes[3], argboxes[4])
raise NotImplementedError
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/graphpage.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/graphpage.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/graphpage.py Tue Nov 23 10:12:47 2010
@@ -153,7 +153,7 @@
opindex = opstartindex
while True:
op = operations[opindex]
- lines.append(repr(op))
+ lines.append(op.repr(graytext=True))
if is_interesting_guard(op):
tgt = op.getdescr()._debug_suboperations[0]
tgt_g, tgt_i = self.all_operations[tgt]
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/history.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/history.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/history.py Tue Nov 23 10:12:47 2010
@@ -698,6 +698,21 @@
return result
_const_ptr_for_string = {}
+def get_const_ptr_for_unicode(s):
+ from pypy.rpython.annlowlevel import llunicode
+ if not we_are_translated():
+ try:
+ return _const_ptr_for_unicode[s]
+ except KeyError:
+ pass
+ if isinstance(s, str):
+ s = unicode(s)
+ result = ConstPtr(lltype.cast_opaque_ptr(llmemory.GCREF, llunicode(s)))
+ if not we_are_translated():
+ _const_ptr_for_unicode[s] = result
+ return result
+_const_ptr_for_unicode = {}
+
# ____________________________________________________________
# The TreeLoop class contains a loop or a generalized loop, i.e. a tree
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/jitdriver.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/jitdriver.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/jitdriver.py Tue Nov 23 10:12:47 2010
@@ -13,8 +13,10 @@
# self.num_red_args ... pypy.jit.metainterp.warmspot
# self.result_type ... pypy.jit.metainterp.warmspot
# self.virtualizable_info... pypy.jit.metainterp.warmspot
+ # self.greenfield_info ... pypy.jit.metainterp.warmspot
# self.warmstate ... pypy.jit.metainterp.warmspot
# self.handle_jitexc_from_bh pypy.jit.metainterp.warmspot
+ # self.no_loop_header ... pypy.jit.metainterp.warmspot
# self.portal_finishtoken... pypy.jit.metainterp.pyjitpl
# self.index ... pypy.jit.codewriter.call
# self.mainjitcode ... pypy.jit.codewriter.call
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/logger.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/logger.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/logger.py Tue Nov 23 10:12:47 2010
@@ -81,7 +81,8 @@
op = operations[i]
if op.getopnum() == rop.DEBUG_MERGE_POINT:
loc = op.getarg(0)._get_str()
- debug_print("debug_merge_point('%s')" % (loc,))
+ reclev = op.getarg(1).getint()
+ debug_print("debug_merge_point('%s', %s)" % (loc, reclev))
continue
args = ", ".join([self.repr_of_arg(memo, op.getarg(i)) for i in range(op.numargs())])
if op.result is not None:
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimize_nopspec.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/optimize_nopspec.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/optimize_nopspec.py Tue Nov 23 10:12:47 2010
@@ -14,6 +14,9 @@
def _optimize_loop(metainterp_sd, old_loop_tokens, loop):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(loop.inputargs, loop.operations)
+ # XXX the following lines are probably still needed, to discard invalid
+ # loops. bit silly to run a full perfect specialization and throw the
+ # result away.
finder = PerfectSpecializationFinder(cpu)
finder.find_nodes_loop(loop, False)
if old_loop_tokens:
@@ -31,6 +34,7 @@
def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations)
+ # XXX same comment as above applies
finder = BridgeSpecializationFinder(cpu)
finder.find_nodes_bridge(bridge)
if old_loop_tokens:
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/__init__.py Tue Nov 23 10:12:47 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.optimizeopt.intbounds import OptIntBounds
from pypy.jit.metainterp.optimizeopt.virtualize import OptVirtualize
from pypy.jit.metainterp.optimizeopt.heap import OptHeap
+from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall
from pypy.jit.metainterp.optimizeopt.string import OptString
def optimize_loop_1(metainterp_sd, loop, virtuals=True):
@@ -14,8 +15,9 @@
optimizations = [OptIntBounds(),
OptRewrite(),
OptVirtualize(),
-# OptString(),
+ OptString(),
OptHeap(),
+ OptFfiCall(),
]
optimizer = Optimizer(metainterp_sd, loop, optimizations, virtuals)
optimizer.propagate_all_forward()
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/intbounds.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/intbounds.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/intbounds.py Tue Nov 23 10:12:47 2010
@@ -191,6 +191,7 @@
v1.intbound.make_ge(IntLowerBound(0))
optimize_STRLEN = optimize_ARRAYLEN_GC
+ optimize_UNICODELEN = optimize_ARRAYLEN_GC
def make_int_lt(self, box1, box2):
v1 = self.getvalue(box1)
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/optimizer.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/optimizer.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/optimizer.py Tue Nov 23 10:12:47 2010
@@ -66,10 +66,10 @@
assert isinstance(constbox, Const)
self.box = constbox
self.level = LEVEL_CONSTANT
- try:
- val = self.box.getint()
+ if isinstance(constbox, ConstInt):
+ val = constbox.getint()
self.intbound = IntBound(val, val)
- except NotImplementedError:
+ else:
self.intbound = IntUnbounded()
def get_constant_class(self, cpu):
@@ -299,7 +299,9 @@
return CVAL_ZERO
def propagate_all_forward(self):
- self.exception_might_have_happened = False
+ self.exception_might_have_happened = True
+ # ^^^ at least at the start of bridges. For loops, we could set
+ # it to False, but we probably don't care
self.newoperations = []
self.i = 0
while self.i < len(self.loop.operations):
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/rewrite.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/rewrite.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/rewrite.py Tue Nov 23 10:12:47 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.history import ConstInt
from pypy.jit.metainterp.optimizeutil import _findall
from pypy.jit.metainterp.resoperation import rop, ResOperation
+from pypy.jit.codewriter.effectinfo import EffectInfo
class OptRewrite(Optimization):
"""Rewrite operations into equivalent, cheaper operations.
@@ -243,11 +244,11 @@
self.optimizer.exception_might_have_happened = False
def optimize_CALL_LOOPINVARIANT(self, op):
- funcvalue = self.getvalue(op.getarg(0))
- if not funcvalue.is_constant():
- self.emit_operation(op)
- return
- key = make_hashable_int(op.getarg(0).getint())
+ arg = op.getarg(0)
+ # 'arg' must be a Const, because residual_call in codewriter
+ # expects a compile-time constant
+ assert isinstance(arg, Const)
+ key = make_hashable_int(arg.getint())
resvalue = self.optimizer.loop_invariant_results.get(key, None)
if resvalue is not None:
self.make_equal_to(op.result, resvalue)
@@ -323,8 +324,37 @@
## return
## self.emit_operation(op)
-optimize_ops = _findall(OptRewrite, 'optimize_')
-
+ def optimize_CALL(self, op):
+ # dispatch based on 'oopspecindex' to a method that handles
+ # specifically the given oopspec call. For non-oopspec calls,
+ # oopspecindex is just zero.
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ oopspecindex = effectinfo.oopspecindex
+ if oopspecindex == EffectInfo.OS_ARRAYCOPY:
+ if self._optimize_CALL_ARRAYCOPY(op):
+ return
+ self.emit_operation(op)
+ def _optimize_CALL_ARRAYCOPY(self, op):
+ source_value = self.getvalue(op.getarg(1))
+ dest_value = self.getvalue(op.getarg(2))
+ source_start_box = self.get_constant_box(op.getarg(3))
+ dest_start_box = self.get_constant_box(op.getarg(4))
+ length = self.get_constant_box(op.getarg(5))
+ if (source_value.is_virtual() and source_start_box and dest_start_box
+ and length and dest_value.is_virtual()):
+ # XXX optimize the case where dest value is not virtual,
+ # but we still can avoid a mess
+ source_start = source_start_box.getint()
+ dest_start = dest_start_box.getint()
+ for index in range(length.getint()):
+ val = source_value.getitem(index + source_start)
+ dest_value.setitem(index + dest_start, val)
+ return True
+ if length and length.getint() == 0:
+ return True # 0-length arraycopy
+ return False
+
+optimize_ops = _findall(OptRewrite, 'optimize_')
-
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/string.py Tue Nov 23 10:12:47 2010
@@ -3,64 +3,115 @@
from pypy.jit.metainterp.history import Box, BoxInt, BoxPtr
from pypy.jit.metainterp.history import Const, ConstInt, ConstPtr
from pypy.jit.metainterp.history import get_const_ptr_for_string
+from pypy.jit.metainterp.history import get_const_ptr_for_unicode
from pypy.jit.metainterp.resoperation import rop, ResOperation
from pypy.jit.metainterp.optimizeopt import optimizer, virtualize
from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1
from pypy.jit.metainterp.optimizeopt.optimizer import llhelper
from pypy.jit.metainterp.optimizeutil import _findall
-from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec
+from pypy.jit.codewriter.effectinfo import EffectInfo
from pypy.jit.codewriter import heaptracker
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib.objectmodel import specialize, we_are_translated
+
+
+class StrOrUnicode(object):
+ def __init__(self, LLTYPE, hlstr, emptystr, chr,
+ NEWSTR, STRLEN, STRGETITEM, STRSETITEM, COPYSTRCONTENT,
+ OS_offset):
+ self.LLTYPE = LLTYPE
+ self.hlstr = hlstr
+ self.emptystr = emptystr
+ self.chr = chr
+ self.NEWSTR = NEWSTR
+ self.STRLEN = STRLEN
+ self.STRGETITEM = STRGETITEM
+ self.STRSETITEM = STRSETITEM
+ self.COPYSTRCONTENT = COPYSTRCONTENT
+ self.OS_offset = OS_offset
+
+ def _freeze_(self):
+ return True
+
+mode_string = StrOrUnicode(rstr.STR, annlowlevel.hlstr, '', chr,
+ rop.NEWSTR, rop.STRLEN, rop.STRGETITEM,
+ rop.STRSETITEM, rop.COPYSTRCONTENT, 0)
+mode_unicode = StrOrUnicode(rstr.UNICODE, annlowlevel.hlunicode, u'', unichr,
+ rop.NEWUNICODE, rop.UNICODELEN, rop.UNICODEGETITEM,
+ rop.UNICODESETITEM, rop.COPYUNICODECONTENT,
+ EffectInfo._OS_offset_uni)
+
+# ____________________________________________________________
class __extend__(optimizer.OptValue):
"""New methods added to the base class OptValue for this file."""
- def getstrlen(self, newoperations):
- s = self.get_constant_string()
- if s is not None:
- return ConstInt(len(s))
+ def getstrlen(self, newoperations, mode):
+ if mode is mode_string:
+ s = self.get_constant_string_spec(mode_string)
+ if s is not None:
+ return ConstInt(len(s))
else:
- if newoperations is None:
- return None
- self.ensure_nonnull()
- box = self.force_box()
- lengthbox = BoxInt()
- newoperations.append(ResOperation(rop.STRLEN, [box], lengthbox))
- return lengthbox
+ s = self.get_constant_string_spec(mode_unicode)
+ if s is not None:
+ return ConstInt(len(s))
+ if newoperations is None:
+ return None
+ self.ensure_nonnull()
+ box = self.force_box()
+ lengthbox = BoxInt()
+ newoperations.append(ResOperation(mode.STRLEN, [box], lengthbox))
+ return lengthbox
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
if self.is_constant():
- s = self.box.getref(lltype.Ptr(rstr.STR))
- return annlowlevel.hlstr(s)
+ s = self.box.getref(lltype.Ptr(mode.LLTYPE))
+ return mode.hlstr(s)
else:
return None
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
# Copies the pointer-to-string 'self' into the target string
# given by 'targetbox', at the specified offset. Returns the offset
# at the end of the copy.
- lengthbox = self.getstrlen(newoperations)
+ lengthbox = self.getstrlen(newoperations, mode)
srcbox = self.force_box()
return copy_str_content(newoperations, srcbox, targetbox,
- CONST_0, offsetbox, lengthbox)
+ CONST_0, offsetbox, lengthbox, mode)
class VAbstractStringValue(virtualize.AbstractVirtualValue):
- _attrs_ = ()
+ _attrs_ = ('mode',)
+
+ def __init__(self, optimizer, keybox, source_op, mode):
+ virtualize.AbstractVirtualValue.__init__(self, optimizer, keybox,
+ source_op)
+ self.mode = mode
def _really_force(self):
- s = self.get_constant_string()
- if s is not None:
- c_s = get_const_ptr_for_string(s)
- self.make_constant(c_s)
- return
+ if self.mode is mode_string:
+ s = self.get_constant_string_spec(mode_string)
+ if s is not None:
+ c_s = get_const_ptr_for_string(s)
+ self.make_constant(c_s)
+ return
+ else:
+ s = self.get_constant_string_spec(mode_unicode)
+ if s is not None:
+ c_s = get_const_ptr_for_unicode(s)
+ self.make_constant(c_s)
+ return
assert self.source_op is not None
self.box = box = self.source_op.result
newoperations = self.optimizer.newoperations
- lengthbox = self.getstrlen(newoperations)
- newoperations.append(ResOperation(rop.NEWSTR, [lengthbox], box))
- self.string_copy_parts(newoperations, box, CONST_0)
+ lengthbox = self.getstrlen(newoperations, self.mode)
+ op = ResOperation(self.mode.NEWSTR, [lengthbox], box)
+ if not we_are_translated():
+ op.name = 'FORCE'
+ newoperations.append(op)
+ self.string_copy_parts(newoperations, box, CONST_0, self.mode)
class VStringPlainValue(VAbstractStringValue):
@@ -74,7 +125,7 @@
assert 0 <= start <= stop <= len(longerlist)
self._chars = longerlist[start:stop]
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
if self._lengthbox is None:
self._lengthbox = ConstInt(len(self._chars))
return self._lengthbox
@@ -86,18 +137,21 @@
assert isinstance(charvalue, optimizer.OptValue)
self._chars[index] = charvalue
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
for c in self._chars:
if c is optimizer.CVAL_UNINITIALIZED_ZERO or not c.is_constant():
return None
- return ''.join([chr(c.box.getint()) for c in self._chars])
+ return mode.emptystr.join([mode.chr(c.box.getint())
+ for c in self._chars])
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
for i in range(len(self._chars)):
charbox = self._chars[i].force_box()
- newoperations.append(ResOperation(rop.STRSETITEM, [targetbox,
- offsetbox,
- charbox], None))
+ newoperations.append(ResOperation(mode.STRSETITEM, [targetbox,
+ offsetbox,
+ charbox],
+ None))
offsetbox = _int_add(newoperations, offsetbox, CONST_1)
return offsetbox
@@ -109,7 +163,7 @@
value.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrplain()
+ return modifier.make_vstrplain(self.mode is mode_unicode)
class VStringConcatValue(VAbstractStringValue):
@@ -120,23 +174,24 @@
self.right = right
self.lengthbox = lengthbox
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
return self.lengthbox
- def get_constant_string(self):
- s1 = self.left.get_constant_string()
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
+ s1 = self.left.get_constant_string_spec(mode)
if s1 is None:
return None
- s2 = self.right.get_constant_string()
+ s2 = self.right.get_constant_string_spec(mode)
if s2 is None:
return None
return s1 + s2
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
offsetbox = self.left.string_copy_parts(newoperations, targetbox,
- offsetbox)
+ offsetbox, mode)
offsetbox = self.right.string_copy_parts(newoperations, targetbox,
- offsetbox)
+ offsetbox, mode)
return offsetbox
def get_args_for_fail(self, modifier):
@@ -150,7 +205,7 @@
self.right.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrconcat()
+ return modifier.make_vstrconcat(self.mode is mode_unicode)
class VStringSliceValue(VAbstractStringValue):
@@ -162,12 +217,13 @@
self.vstart = vstart
self.vlength = vlength
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
return self.vlength.force_box()
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
if self.vstart.is_constant() and self.vlength.is_constant():
- s1 = self.vstr.get_constant_string()
+ s1 = self.vstr.get_constant_string_spec(mode)
if s1 is None:
return None
start = self.vstart.box.getint()
@@ -177,12 +233,12 @@
return s1[start : start + length]
return None
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
- lengthbox = self.getstrlen(newoperations)
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
+ lengthbox = self.getstrlen(newoperations, mode)
return copy_str_content(newoperations,
self.vstr.force_box(), targetbox,
self.vstart.force_box(), offsetbox,
- lengthbox)
+ lengthbox, mode)
def get_args_for_fail(self, modifier):
if self.box is None and not modifier.already_seen_virtual(self.keybox):
@@ -195,11 +251,11 @@
self.vlength.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrslice()
+ return modifier.make_vstrslice(self.mode is mode_unicode)
def copy_str_content(newoperations, srcbox, targetbox,
- srcoffsetbox, offsetbox, lengthbox):
+ srcoffsetbox, offsetbox, lengthbox, mode):
if isinstance(srcbox, ConstPtr) and isinstance(srcoffsetbox, Const):
M = 5
else:
@@ -208,17 +264,18 @@
# up to M characters are done "inline", i.e. with STRGETITEM/STRSETITEM
# instead of just a COPYSTRCONTENT.
for i in range(lengthbox.value):
- charbox = _strgetitem(newoperations, srcbox, srcoffsetbox)
+ charbox = _strgetitem(newoperations, srcbox, srcoffsetbox, mode)
srcoffsetbox = _int_add(newoperations, srcoffsetbox, CONST_1)
- newoperations.append(ResOperation(rop.STRSETITEM, [targetbox,
- offsetbox,
- charbox], None))
+ newoperations.append(ResOperation(mode.STRSETITEM, [targetbox,
+ offsetbox,
+ charbox],
+ None))
offsetbox = _int_add(newoperations, offsetbox, CONST_1)
else:
nextoffsetbox = _int_add(newoperations, offsetbox, lengthbox)
- op = ResOperation(rop.COPYSTRCONTENT, [srcbox, targetbox,
- srcoffsetbox, offsetbox,
- lengthbox], None)
+ op = ResOperation(mode.COPYSTRCONTENT, [srcbox, targetbox,
+ srcoffsetbox, offsetbox,
+ lengthbox], None)
newoperations.append(op)
offsetbox = nextoffsetbox
return offsetbox
@@ -245,12 +302,16 @@
newoperations.append(ResOperation(rop.INT_SUB, [box1, box2], resbox))
return resbox
-def _strgetitem(newoperations, strbox, indexbox):
+def _strgetitem(newoperations, strbox, indexbox, mode):
if isinstance(strbox, ConstPtr) and isinstance(indexbox, ConstInt):
- s = strbox.getref(lltype.Ptr(rstr.STR))
- return ConstInt(ord(s.chars[indexbox.getint()]))
+ if mode is mode_string:
+ s = strbox.getref(lltype.Ptr(rstr.STR))
+ return ConstInt(ord(s.chars[indexbox.getint()]))
+ else:
+ s = strbox.getref(lltype.Ptr(rstr.UNICODE))
+ return ConstInt(ord(s.chars[indexbox.getint()]))
resbox = BoxInt()
- newoperations.append(ResOperation(rop.STRGETITEM, [strbox, indexbox],
+ newoperations.append(ResOperation(mode.STRGETITEM, [strbox, indexbox],
resbox))
return resbox
@@ -258,62 +319,34 @@
class OptString(optimizer.Optimization):
"Handling of strings and unicodes."
- def make_vstring_plain(self, box, source_op=None):
- vvalue = VStringPlainValue(self.optimizer, box, source_op)
+ def make_vstring_plain(self, box, source_op, mode):
+ vvalue = VStringPlainValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def make_vstring_concat(self, box, source_op=None):
- vvalue = VStringConcatValue(self.optimizer, box, source_op)
+ def make_vstring_concat(self, box, source_op, mode):
+ vvalue = VStringConcatValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def make_vstring_slice(self, box, source_op=None):
- vvalue = VStringSliceValue(self.optimizer, box, source_op)
+ def make_vstring_slice(self, box, source_op, mode):
+ vvalue = VStringSliceValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def optimize_CALL(self, op):
- # dispatch based on 'oopspecindex' to a method that handles
- # specifically the given oopspec call. For non-oopspec calls,
- # oopspecindex is just zero.
- effectinfo = op.getdescr().get_extra_info()
- if effectinfo is not None:
- oopspecindex = effectinfo.oopspecindex
- for value, meth in opt_call_oopspec_ops:
- if oopspecindex == value:
- if meth(self, op):
- return
- self.emit_operation(op)
-
- def opt_call_oopspec_ARRAYCOPY(self, op):
- source_value = self.getvalue(op.getarg(1))
- dest_value = self.getvalue(op.getarg(2))
- source_start_box = self.get_constant_box(op.getarg(3))
- dest_start_box = self.get_constant_box(op.getarg(4))
- length = self.get_constant_box(op.getarg(5))
- if (source_value.is_virtual() and source_start_box and dest_start_box
- and length and dest_value.is_virtual()):
- # XXX optimize the case where dest value is not virtual,
- # but we still can avoid a mess
- source_start = source_start_box.getint()
- dest_start = dest_start_box.getint()
- for index in range(length.getint()):
- val = source_value.getitem(index + source_start)
- dest_value.setitem(index + dest_start, val)
- return True
- if length and length.getint() == 0:
- return True # 0-length arraycopy
- return False
-
def optimize_NEWSTR(self, op):
+ self._optimize_NEWSTR(op, mode_string)
+ def optimize_NEWUNICODE(self, op):
+ self._optimize_NEWSTR(op, mode_unicode)
+
+ def _optimize_NEWSTR(self, op, mode):
length_box = self.get_constant_box(op.getarg(0))
if length_box:
# if the original 'op' did not have a ConstInt as argument,
# build a new one with the ConstInt argument
if not isinstance(op.getarg(0), ConstInt):
- op = ResOperation(rop.NEWSTR, [length_box], op.result)
- vvalue = self.make_vstring_plain(op.result, op)
+ op = ResOperation(mode.NEWSTR, [length_box], op.result)
+ vvalue = self.make_vstring_plain(op.result, op, mode)
vvalue.setup(length_box.getint())
else:
self.getvalue(op.result).ensure_nonnull()
@@ -329,13 +362,20 @@
value.ensure_nonnull()
self.emit_operation(op)
+ optimize_UNICODESETITEM = optimize_STRSETITEM
+
def optimize_STRGETITEM(self, op):
+ self._optimize_STRGETITEM(op, mode_string)
+ def optimize_UNICODEGETITEM(self, op):
+ self._optimize_STRGETITEM(op, mode_unicode)
+
+ def _optimize_STRGETITEM(self, op, mode):
value = self.getvalue(op.getarg(0))
vindex = self.getvalue(op.getarg(1))
- vresult = self.strgetitem(value, vindex)
+ vresult = self.strgetitem(value, vindex, mode)
self.make_equal_to(op.result, vresult)
- def strgetitem(self, value, vindex):
+ def strgetitem(self, value, vindex, mode):
value.ensure_nonnull()
#
if value.is_virtual() and isinstance(value, VStringSliceValue):
@@ -350,28 +390,71 @@
return value.getitem(vindex.box.getint())
#
resbox = _strgetitem(self.optimizer.newoperations,
- value.force_box(),vindex.force_box())
+ value.force_box(),vindex.force_box(), mode)
return self.getvalue(resbox)
def optimize_STRLEN(self, op):
+ self._optimize_STRLEN(op, mode_string)
+ def optimize_UNICODELEN(self, op):
+ self._optimize_STRLEN(op, mode_unicode)
+
+ def _optimize_STRLEN(self, op, mode):
value = self.getvalue(op.getarg(0))
- lengthbox = value.getstrlen(self.optimizer.newoperations)
+ lengthbox = value.getstrlen(self.optimizer.newoperations, mode)
self.make_equal_to(op.result, self.getvalue(lengthbox))
- def opt_call_oopspec_STR_CONCAT(self, op):
+ def optimize_CALL(self, op):
+ # dispatch based on 'oopspecindex' to a method that handles
+ # specifically the given oopspec call. For non-oopspec calls,
+ # oopspecindex is just zero.
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ oopspecindex = effectinfo.oopspecindex
+ for value, meth in opt_call_oopspec_ops:
+ if oopspecindex == value: # a match with the OS_STR_xxx
+ if meth(self, op, mode_string):
+ return
+ break
+ if oopspecindex == value + EffectInfo._OS_offset_uni:
+ # a match with the OS_UNI_xxx
+ if meth(self, op, mode_unicode):
+ return
+ break
+ if oopspecindex == EffectInfo.OS_STR2UNICODE:
+ if self.opt_call_str_STR2UNICODE(op):
+ return
+ self.emit_operation(op)
+
+ def opt_call_str_STR2UNICODE(self, op):
+ # Constant-fold unicode("constant string").
+ # More generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
+ varg = self.getvalue(op.getarg(1))
+ s = varg.get_constant_string_spec(mode_string)
+ if s is None:
+ return False
+ try:
+ u = unicode(s)
+ except UnicodeDecodeError:
+ return False
+ self.make_constant(op.result, get_const_ptr_for_unicode(u))
+ return True
+
+ def opt_call_stroruni_STR_CONCAT(self, op, mode):
vleft = self.getvalue(op.getarg(1))
vright = self.getvalue(op.getarg(2))
vleft.ensure_nonnull()
vright.ensure_nonnull()
newoperations = self.optimizer.newoperations
- len1box = vleft.getstrlen(newoperations)
- len2box = vright.getstrlen(newoperations)
+ len1box = vleft.getstrlen(newoperations, mode)
+ len2box = vright.getstrlen(newoperations, mode)
lengthbox = _int_add(newoperations, len1box, len2box)
- value = self.make_vstring_concat(op.result, op)
+ value = self.make_vstring_concat(op.result, op, mode)
value.setup(vleft, vright, lengthbox)
return True
- def opt_call_oopspec_STR_SLICE(self, op):
+ def opt_call_stroruni_STR_SLICE(self, op, mode):
newoperations = self.optimizer.newoperations
vstr = self.getvalue(op.getarg(1))
vstart = self.getvalue(op.getarg(2))
@@ -380,7 +463,7 @@
if (isinstance(vstr, VStringPlainValue) and vstart.is_constant()
and vstop.is_constant()):
# slicing with constant bounds of a VStringPlainValue
- value = self.make_vstring_plain(op.result, op)
+ value = self.make_vstring_plain(op.result, op, mode)
value.setup_slice(vstr._chars, vstart.box.getint(),
vstop.box.getint())
return True
@@ -398,16 +481,16 @@
vstart.force_box())
vstart = self.getvalue(startbox)
#
- value = self.make_vstring_slice(op.result, op)
+ value = self.make_vstring_slice(op.result, op, mode)
value.setup(vstr, vstart, self.getvalue(lengthbox))
return True
- def opt_call_oopspec_STR_EQUAL(self, op):
+ def opt_call_stroruni_STR_EQUAL(self, op, mode):
v1 = self.getvalue(op.getarg(1))
v2 = self.getvalue(op.getarg(2))
#
- l1box = v1.getstrlen(None)
- l2box = v2.getstrlen(None)
+ l1box = v1.getstrlen(None, mode)
+ l2box = v2.getstrlen(None, mode)
if (l1box is not None and l2box is not None and
isinstance(l1box, ConstInt) and
isinstance(l2box, ConstInt) and
@@ -416,13 +499,13 @@
self.make_constant(op.result, CONST_0)
return True
#
- if self.handle_str_equal_level1(v1, v2, op.result):
+ if self.handle_str_equal_level1(v1, v2, op.result, mode):
return True
- if self.handle_str_equal_level1(v2, v1, op.result):
+ if self.handle_str_equal_level1(v2, v1, op.result, mode):
return True
- if self.handle_str_equal_level2(v1, v2, op.result):
+ if self.handle_str_equal_level2(v1, v2, op.result, mode):
return True
- if self.handle_str_equal_level2(v2, v1, op.result):
+ if self.handle_str_equal_level2(v2, v1, op.result, mode):
return True
#
if v1.is_nonnull() and v2.is_nonnull():
@@ -434,37 +517,37 @@
else:
do = EffectInfo.OS_STREQ_NONNULL
self.generate_modified_call(do, [v1.force_box(),
- v2.force_box()], op.result)
+ v2.force_box()], op.result, mode)
return True
return False
- def handle_str_equal_level1(self, v1, v2, resultbox):
- l2box = v2.getstrlen(None)
+ def handle_str_equal_level1(self, v1, v2, resultbox, mode):
+ l2box = v2.getstrlen(None, mode)
if isinstance(l2box, ConstInt):
if l2box.value == 0:
- lengthbox = v1.getstrlen(self.optimizer.newoperations)
+ lengthbox = v1.getstrlen(self.optimizer.newoperations, mode)
seo = self.optimizer.send_extra_operation
seo(ResOperation(rop.INT_EQ, [lengthbox, CONST_0], resultbox))
return True
if l2box.value == 1:
- l1box = v1.getstrlen(None)
+ l1box = v1.getstrlen(None, mode)
if isinstance(l1box, ConstInt) and l1box.value == 1:
# comparing two single chars
- vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO)
- vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO, mode)
+ vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
seo = self.optimizer.send_extra_operation
seo(ResOperation(rop.INT_EQ, [vchar1.force_box(),
vchar2.force_box()],
resultbox))
return True
if isinstance(v1, VStringSliceValue):
- vchar = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
do = EffectInfo.OS_STREQ_SLICE_CHAR
self.generate_modified_call(do, [v1.vstr.force_box(),
v1.vstart.force_box(),
v1.vlength.force_box(),
vchar.force_box()],
- resultbox)
+ resultbox, mode)
return True
#
if v2.is_null():
@@ -482,17 +565,18 @@
#
return False
- def handle_str_equal_level2(self, v1, v2, resultbox):
- l2box = v2.getstrlen(None)
+ def handle_str_equal_level2(self, v1, v2, resultbox, mode):
+ l2box = v2.getstrlen(None, mode)
if isinstance(l2box, ConstInt):
if l2box.value == 1:
- vchar = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
if v1.is_nonnull():
do = EffectInfo.OS_STREQ_NONNULL_CHAR
else:
do = EffectInfo.OS_STREQ_CHECKNULL_CHAR
self.generate_modified_call(do, [v1.force_box(),
- vchar.force_box()], resultbox)
+ vchar.force_box()], resultbox,
+ mode)
return True
#
if v1.is_virtual() and isinstance(v1, VStringSliceValue):
@@ -503,12 +587,14 @@
self.generate_modified_call(do, [v1.vstr.force_box(),
v1.vstart.force_box(),
v1.vlength.force_box(),
- v2.force_box()], resultbox)
+ v2.force_box()], resultbox, mode)
return True
return False
- def generate_modified_call(self, oopspecindex, args, result):
- calldescr, func = callinfo_for_oopspec(oopspecindex)
+ def generate_modified_call(self, oopspecindex, args, result, mode):
+ oopspecindex += mode.OS_offset
+ cic = self.optimizer.metainterp_sd.callinfocollection
+ calldescr, func = cic.callinfo_for_oopspec(oopspecindex)
op = ResOperation(rop.CALL, [ConstInt(func)] + args, result,
descr=calldescr)
self.optimizer.newoperations.append(op)
@@ -525,7 +611,7 @@
optimize_ops = _findall(OptString, 'optimize_')
def _findall_call_oopspec():
- prefix = 'opt_call_oopspec_'
+ prefix = 'opt_call_stroruni_'
result = []
for name in dir(OptString):
if name.startswith(prefix):
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/virtualize.py Tue Nov 23 10:12:47 2010
@@ -74,6 +74,8 @@
assert self.source_op is not None
# ^^^ This case should not occur any more (see test_bug_3).
#
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
@@ -134,6 +136,11 @@
fielddescrs = self._get_field_descr_list()
return modifier.make_virtual(self.known_class, fielddescrs)
+ def __repr__(self):
+ cls_name = self.known_class.value.adr.ptr._obj._TYPE._name
+ field_names = [field.name for field in self._fields]
+ return "<VirtualValue cls=%s fields=%s>" % (cls_name, field_names)
+
class VStructValue(AbstractVirtualStructValue):
def __init__(self, optimizer, structdescr, keybox, source_op=None):
@@ -165,6 +172,8 @@
def _really_force(self):
assert self.source_op is not None
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/pyjitpl.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/pyjitpl.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/pyjitpl.py Tue Nov 23 10:12:47 2010
@@ -1,4 +1,4 @@
-import py, os
+import py, os, sys
from pypy.rpython.lltypesystem import lltype, llmemory, rclass
from pypy.rlib.objectmodel import we_are_translated
from pypy.rlib.unroll import unrolling_iterable
@@ -14,7 +14,7 @@
from pypy.jit.metainterp.logger import Logger
from pypy.jit.metainterp.jitprof import EmptyProfiler
from pypy.jit.metainterp.jitprof import GUARDS, RECORDED_OPS, ABORT_ESCAPE
-from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG, ABORT_BRIDGE
+from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG
from pypy.jit.metainterp.jitexc import JitException, get_llexception
from pypy.rlib.rarithmetic import intmask
from pypy.rlib.objectmodel import specialize
@@ -94,6 +94,18 @@
else: raise AssertionError(argcode)
outvalue[startindex+i] = reg
+ def _put_back_list_of_boxes(self, outvalue, startindex, position):
+ code = self.bytecode
+ length = ord(code[position])
+ position += 1
+ for i in range(length):
+ index = ord(code[position+i])
+ box = outvalue[startindex+i]
+ if box.type == history.INT: self.registers_i[index] = box
+ elif box.type == history.REF: self.registers_r[index] = box
+ elif box.type == history.FLOAT: self.registers_f[index] = box
+ else: raise AssertionError(box.type)
+
def get_current_position_info(self):
return self.jitcode.get_live_vars_info(self.pc)
@@ -498,6 +510,22 @@
opimpl_getfield_gc_r_pure = _opimpl_getfield_gc_pure_any
opimpl_getfield_gc_f_pure = _opimpl_getfield_gc_pure_any
+ @arguments("orgpc", "box", "descr")
+ def _opimpl_getfield_gc_greenfield_any(self, pc, box, fielddescr):
+ ginfo = self.metainterp.jitdriver_sd.greenfield_info
+ if (ginfo is not None and fielddescr in ginfo.green_field_descrs
+ and not self._nonstandard_virtualizable(pc, box)):
+ # fetch the result, but consider it as a Const box and don't
+ # record any operation
+ resbox = executor.execute(self.metainterp.cpu, self.metainterp,
+ rop.GETFIELD_GC_PURE, fielddescr, box)
+ return resbox.constbox()
+ # fall-back
+ return self.execute_with_descr(rop.GETFIELD_GC_PURE, fielddescr, box)
+ opimpl_getfield_gc_i_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_r_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_f_greenfield = _opimpl_getfield_gc_greenfield_any
+
@arguments("box", "descr", "box")
def _opimpl_setfield_gc_any(self, box, fielddescr, valuebox):
self.execute_with_descr(rop.SETFIELD_GC, fielddescr, box, valuebox)
@@ -529,7 +557,8 @@
def _nonstandard_virtualizable(self, pc, box):
# returns True if 'box' is actually not the "standard" virtualizable
# that is stored in metainterp.virtualizable_boxes[-1]
- if self.metainterp.jitdriver_sd.virtualizable_info is None:
+ if (self.metainterp.jitdriver_sd.virtualizable_info is None and
+ self.metainterp.jitdriver_sd.greenfield_info is None):
return True # can occur in case of multiple JITs
standard_box = self.metainterp.virtualizable_boxes[-1]
if standard_box is box:
@@ -797,14 +826,20 @@
for i in range(num_green_args):
assert isinstance(varargs[i], Const)
- @arguments("orgpc", "int", "boxes3", "boxes3")
- def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, redboxes):
+ @arguments("orgpc", "int", "boxes3", "jitcode_position", "boxes3")
+ def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes,
+ jcposition, redboxes):
+ any_operation = len(self.metainterp.history.operations) > 0
jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex]
self.verify_green_args(jitdriver_sd, greenboxes)
# xxx we may disable the following line in some context later
- self.debug_merge_point(jitdriver_sd, greenboxes)
+ self.debug_merge_point(jitdriver_sd, self.metainterp.in_recursion,
+ greenboxes)
if self.metainterp.seen_loop_header_for_jdindex < 0:
- return
+ if not jitdriver_sd.no_loop_header or not any_operation:
+ return
+ # automatically add a loop_header if there is none
+ self.metainterp.seen_loop_header_for_jdindex = jdindex
#
assert self.metainterp.seen_loop_header_for_jdindex == jdindex, (
"found a loop_header for a JitDriver that does not match "
@@ -821,6 +856,10 @@
self.pc = orgpc
self.metainterp.reached_loop_header(greenboxes, redboxes)
self.pc = saved_pc
+ # no exception, which means that the jit_merge_point did not
+ # close the loop. We have to put the possibly-modified list
+ # 'redboxes' back into the registers where it comes from.
+ put_back_list_of_boxes3(self, jcposition, redboxes)
else:
# warning! careful here. We have to return from the current
# frame containing the jit_merge_point, and then use
@@ -839,13 +878,13 @@
assembler_call=True)
raise ChangeFrame
- def debug_merge_point(self, jitdriver_sd, greenkey):
+ def debug_merge_point(self, jitdriver_sd, in_recursion, greenkey):
# debugging: produce a DEBUG_MERGE_POINT operation
loc = jitdriver_sd.warmstate.get_location_str(greenkey)
debug_print(loc)
constloc = self.metainterp.cpu.ts.conststr(loc)
self.metainterp.history.record(rop.DEBUG_MERGE_POINT,
- [constloc], None)
+ [constloc, ConstInt(in_recursion)], None)
@arguments("box", "label")
def opimpl_goto_if_exception_mismatch(self, vtablebox, next_exc_target):
@@ -893,6 +932,45 @@
msg = box.getref(lltype.Ptr(rstr.STR))
lloperation.llop.debug_fatalerror(msg)
+ @arguments("box", "box", "box", "box", "box")
+ def opimpl_jit_debug(self, stringbox, arg1box, arg2box, arg3box, arg4box):
+ from pypy.rpython.lltypesystem import rstr
+ from pypy.rpython.annlowlevel import hlstr
+ msg = stringbox.getref(lltype.Ptr(rstr.STR))
+ debug_print('jit_debug:', hlstr(msg),
+ arg1box.getint(), arg2box.getint(),
+ arg3box.getint(), arg4box.getint())
+ args = [stringbox, arg1box, arg2box, arg3box, arg4box]
+ i = 4
+ while i > 0 and args[i].getint() == -sys.maxint-1:
+ i -= 1
+ assert i >= 0
+ op = self.metainterp.history.record(rop.JIT_DEBUG, args[:i+1], None)
+ self.metainterp.attach_debug_info(op)
+
+ @arguments("box")
+ def _opimpl_assert_green(self, box):
+ if not isinstance(box, Const):
+ msg = "assert_green failed at %s:%d" % (
+ self.jitcode.name,
+ self.pc)
+ if we_are_translated():
+ from pypy.rpython.annlowlevel import llstr
+ from pypy.rpython.lltypesystem import lloperation
+ lloperation.llop.debug_fatalerror(lltype.Void, llstr(msg))
+ else:
+ from pypy.rlib.jit import AssertGreenFailed
+ raise AssertGreenFailed(msg)
+
+ opimpl_int_assert_green = _opimpl_assert_green
+ opimpl_ref_assert_green = _opimpl_assert_green
+ opimpl_float_assert_green = _opimpl_assert_green
+
+ @arguments()
+ def opimpl_current_trace_length(self):
+ trace_length = len(self.metainterp.history.operations)
+ return ConstInt(trace_length)
+
@arguments("box")
def opimpl_virtual_ref(self, box):
# Details on the content of metainterp.virtualref_boxes:
@@ -987,18 +1065,16 @@
else:
moreargs = list(extraargs)
metainterp_sd = metainterp.staticdata
- original_greenkey = metainterp.resumekey.original_greenkey
if opnum == rop.GUARD_NOT_FORCED:
resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd,
- original_greenkey,
metainterp.jitdriver_sd)
else:
- resumedescr = compile.ResumeGuardDescr(metainterp_sd,
- original_greenkey)
+ resumedescr = compile.ResumeGuardDescr(metainterp_sd)
guard_op = metainterp.history.record(opnum, moreargs, None,
descr=resumedescr)
virtualizable_boxes = None
- if metainterp.jitdriver_sd.virtualizable_info is not None:
+ if (metainterp.jitdriver_sd.virtualizable_info is not None or
+ metainterp.jitdriver_sd.greenfield_info is not None):
virtualizable_boxes = metainterp.virtualizable_boxes
saved_pc = self.pc
if resumepc >= 0:
@@ -1199,6 +1275,7 @@
#
self.jitdrivers_sd = codewriter.callcontrol.jitdrivers_sd
self.virtualref_info = codewriter.callcontrol.virtualref_info
+ self.callinfocollection = codewriter.callcontrol.callinfocollection
self.setup_jitdrivers_sd(optimizer)
#
# store this information for fastpath of call_assembler
@@ -1562,7 +1639,7 @@
assert jitdriver_sd is self.jitdriver_sd
self.create_empty_history()
try:
- original_boxes = self.initialize_original_boxes(jitdriver_sd,*args)
+ original_boxes = self.initialize_original_boxes(jitdriver_sd, *args)
return self._compile_and_run_once(original_boxes)
finally:
self.staticdata.profiler.end_tracing()
@@ -1573,9 +1650,8 @@
self.current_merge_points = [(original_boxes, 0)]
num_green_args = self.jitdriver_sd.num_green_args
original_greenkey = original_boxes[:num_green_args]
- redkey = original_boxes[num_green_args:]
- self.resumekey = compile.ResumeFromInterpDescr(original_greenkey,
- redkey)
+ self.resumekey = compile.ResumeFromInterpDescr(original_greenkey)
+ self.history.inputargs = original_boxes[num_green_args:]
self.seen_loop_header_for_jdindex = -1
try:
self.interpret()
@@ -1597,11 +1673,7 @@
debug_stop('jit-tracing')
def _handle_guard_failure(self, key):
- original_greenkey = key.original_greenkey
- # notice that here we just put the greenkey
- # use -1 to mark that we will have to give up
- # because we cannot reconstruct the beginning of the proper loop
- self.current_merge_points = [(original_greenkey, -1)]
+ self.current_merge_points = []
self.resumekey = key
self.seen_loop_header_for_jdindex = -1
try:
@@ -1646,6 +1718,7 @@
duplicates)
live_arg_boxes += self.virtualizable_boxes
live_arg_boxes.pop()
+ #
assert len(self.virtualref_boxes) == 0, "missing virtual_ref_finish()?"
# Called whenever we reach the 'loop_header' hint.
# First, attempt to make a bridge:
@@ -1664,7 +1737,7 @@
num_green_args = self.jitdriver_sd.num_green_args
for j in range(len(self.current_merge_points)-1, -1, -1):
original_boxes, start = self.current_merge_points[j]
- assert len(original_boxes) == len(live_arg_boxes) or start < 0
+ assert len(original_boxes) == len(live_arg_boxes)
for i in range(num_green_args):
box1 = original_boxes[i]
box2 = live_arg_boxes[i]
@@ -1673,10 +1746,6 @@
break
else:
# Found! Compile it as a loop.
- if start < 0:
- # we cannot reconstruct the beginning of the proper loop
- raise SwitchToBlackhole(ABORT_BRIDGE)
-
# raises in case it works -- which is the common case
self.compile(original_boxes, live_arg_boxes, start)
# creation of the loop was cancelled!
@@ -1741,8 +1810,7 @@
greenkey = original_boxes[:num_green_args]
old_loop_tokens = self.get_compiled_merge_points(greenkey)
self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None)
- loop_token = compile.compile_new_loop(self, old_loop_tokens,
- greenkey, start)
+ loop_token = compile.compile_new_loop(self, old_loop_tokens, start)
if loop_token is not None: # raise if it *worked* correctly
raise GenerateMergePoint(live_arg_boxes, loop_token)
self.history.operations.pop() # remove the JUMP
@@ -1832,6 +1900,7 @@
f.setup_call(original_boxes)
assert self.in_recursion == 0
self.virtualref_boxes = []
+ self.initialize_withgreenfields(original_boxes)
self.initialize_virtualizable(original_boxes)
def initialize_state_from_guard_failure(self, resumedescr):
@@ -1856,6 +1925,14 @@
self.virtualizable_boxes.append(virtualizable_box)
self.initialize_virtualizable_enter()
+ def initialize_withgreenfields(self, original_boxes):
+ ginfo = self.jitdriver_sd.greenfield_info
+ if ginfo is not None:
+ assert self.jitdriver_sd.virtualizable_info is None
+ index = (self.jitdriver_sd.num_green_args +
+ ginfo.red_index)
+ self.virtualizable_boxes = [original_boxes[index]]
+
def initialize_virtualizable_enter(self):
vinfo = self.jitdriver_sd.virtualizable_info
virtualizable_box = self.virtualizable_boxes[-1]
@@ -1949,8 +2026,10 @@
def rebuild_state_after_failure(self, resumedescr):
vinfo = self.jitdriver_sd.virtualizable_info
+ ginfo = self.jitdriver_sd.greenfield_info
self.framestack = []
- boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo)
+ boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo,
+ ginfo)
inputargs_and_holes, virtualizable_boxes, virtualref_boxes = boxlists
#
# virtual refs: make the vrefs point to the freshly allocated virtuals
@@ -1975,6 +2054,12 @@
assert not virtualizable.vable_token
# fill the virtualizable with the local boxes
self.synchronize_virtualizable()
+ #
+ elif self.jitdriver_sd.greenfield_info:
+ self.virtualizable_boxes = virtualizable_boxes
+ else:
+ assert not virtualizable_boxes
+ #
return inputargs_and_holes
def check_synchronized_virtualizable(self):
@@ -2048,7 +2133,8 @@
for i in range(len(boxes)):
if boxes[i] is oldbox:
boxes[i] = newbox
- if self.jitdriver_sd.virtualizable_info is not None:
+ if (self.jitdriver_sd.virtualizable_info is not None or
+ self.jitdriver_sd.greenfield_info is not None):
boxes = self.virtualizable_boxes
for i in range(len(boxes)):
if boxes[i] is oldbox:
@@ -2224,6 +2310,8 @@
else:
raise AssertionError("bad argcode")
position += 1
+ elif argtype == "jitcode_position":
+ value = position
else:
raise AssertionError("bad argtype: %r" % (argtype,))
args += (value,)
@@ -2268,3 +2356,15 @@
argtypes = unrolling_iterable(unboundmethod.argtypes)
handler.func_name = 'handler_' + name
return handler
+
+def put_back_list_of_boxes3(frame, position, newvalue):
+ code = frame.bytecode
+ length1 = ord(code[position])
+ position2 = position + 1 + length1
+ length2 = ord(code[position2])
+ position3 = position2 + 1 + length2
+ length3 = ord(code[position3])
+ assert len(newvalue) == length1 + length2 + length3
+ frame._put_back_list_of_boxes(newvalue, 0, position)
+ frame._put_back_list_of_boxes(newvalue, length1, position2)
+ frame._put_back_list_of_boxes(newvalue, length1 + length2, position3)
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/resoperation.py Tue Nov 23 10:12:47 2010
@@ -93,7 +93,7 @@
def __repr__(self):
return self.repr()
- def repr(self):
+ def repr(self, graytext=False):
# RPython-friendly version
if self.result is not None:
sres = '%s = ' % (self.result,)
@@ -101,6 +101,8 @@
sres = ''
if self.name:
prefix = "%s:%s " % (self.name, self.pc)
+ if graytext:
+ prefix = "\f%s\f" % prefix
else:
prefix = ""
args = self.getarglist()
@@ -141,6 +143,16 @@
def can_raise(self):
return rop._CANRAISE_FIRST <= self.getopnum() <= rop._CANRAISE_LAST
+ def is_malloc(self):
+ # a slightly different meaning from can_malloc
+ return rop._MALLOC_FIRST <= self.getopnum() <= rop._MALLOC_LAST
+
+ def can_malloc(self):
+ return self.is_call() or self.is_malloc()
+
+ def is_call(self):
+ return rop._CALL_FIRST <= self.getopnum() <= rop._CALL_LAST
+
def is_ovf(self):
return rop._OVF_FIRST <= self.getopnum() <= rop._OVF_LAST
@@ -439,9 +451,13 @@
'GETARRAYITEM_RAW/2d',
'GETFIELD_GC/1d',
'GETFIELD_RAW/1d',
+ '_MALLOC_FIRST',
'NEW/0d',
'NEW_WITH_VTABLE/1',
'NEW_ARRAY/1d',
+ 'NEWSTR/1',
+ 'NEWUNICODE/1',
+ '_MALLOC_LAST',
'FORCE_TOKEN/0',
'VIRTUAL_REF/2', # removed before it's passed to the backend
'_NOSIDEEFFECT_LAST', # ----- end of no_side_effect operations -----
@@ -450,26 +466,27 @@
'SETARRAYITEM_RAW/3d',
'SETFIELD_GC/2d',
'SETFIELD_RAW/2d',
- 'NEWSTR/1',
'STRSETITEM/3',
'UNICODESETITEM/3',
- 'NEWUNICODE/1',
#'RUNTIMENEW/1', # ootype operation
- 'COND_CALL_GC_WB/1d', # [objptr] (for the write barrier)
- 'DEBUG_MERGE_POINT/1', # debugging only
+ 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier)
+ 'DEBUG_MERGE_POINT/2', # debugging only
+ 'JIT_DEBUG/*', # debugging only
'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend
'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length
'COPYUNICODECONTENT/5',
'_CANRAISE_FIRST', # ----- start of can_raise operations -----
+ '_CALL_FIRST',
'CALL/*d',
- 'CALL_ASSEMBLER/*d',
+ 'CALL_ASSEMBLER/*d', # call already compiled assembler
'CALL_MAY_FORCE/*d',
'CALL_LOOPINVARIANT/*d',
#'OOSEND', # ootype operation
#'OOSEND_PURE', # ootype operation
'CALL_PURE/*d', # removed before it's passed to the backend
# CALL_PURE(result, func, arg_1,..,arg_n)
+ '_CALL_LAST',
'_CANRAISE_LAST', # ----- end of can_raise operations -----
'_OVF_FIRST', # ----- start of is_ovf operations -----
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/resume.py Tue Nov 23 10:12:47 2010
@@ -4,8 +4,7 @@
from pypy.jit.metainterp.history import INT, REF, FLOAT, HOLE
from pypy.jit.metainterp.resoperation import rop
from pypy.jit.metainterp import jitprof
-from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec
-from pypy.jit.codewriter.effectinfo import funcptr_for_oopspec
+from pypy.jit.codewriter.effectinfo import EffectInfo
from pypy.rpython.lltypesystem import lltype, llmemory, rffi, rstr
from pypy.rlib import rarithmetic
from pypy.rlib.objectmodel import we_are_translated, specialize
@@ -66,12 +65,21 @@
snapshot = Snapshot(snapshot, boxes)
storage.rd_snapshot = snapshot
-class Numbering(object):
- __slots__ = ('prev', 'nums')
-
- def __init__(self, prev, nums):
- self.prev = prev
- self.nums = nums
+#
+# The following is equivalent to the RPython-level declaration:
+#
+# class Numbering: __slots__ = ['prev', 'nums']
+#
+# except that it is more compact in translated programs, because the
+# array 'nums' is inlined in the single NUMBERING object. This is
+# important because this is often the biggest single consumer of memory
+# in a pypy-c-jit.
+#
+NUMBERINGP = lltype.Ptr(lltype.GcForwardReference())
+NUMBERING = lltype.GcStruct('Numbering',
+ ('prev', NUMBERINGP),
+ ('nums', lltype.Array(rffi.SHORT)))
+NUMBERINGP.TO.become(NUMBERING)
TAGMASK = 3
@@ -163,7 +171,7 @@
def number(self, values, snapshot):
if snapshot is None:
- return None, {}, 0
+ return lltype.nullptr(NUMBERING), {}, 0
if snapshot in self.numberings:
numb, liveboxes, v = self.numberings[snapshot]
return numb, liveboxes.copy(), v
@@ -172,7 +180,7 @@
n = len(liveboxes)-v
boxes = snapshot.boxes
length = len(boxes)
- nums = [UNASSIGNED] * length
+ numb = lltype.malloc(NUMBERING, length)
for i in range(length):
box = boxes[i]
value = values.get(box, None)
@@ -191,9 +199,9 @@
tagged = tag(n, TAGBOX)
n += 1
liveboxes[box] = tagged
- nums[i] = tagged
+ numb.nums[i] = tagged
#
- numb = Numbering(numb1, nums)
+ numb.prev = numb1
self.numberings[snapshot] = numb, liveboxes, v
return numb, liveboxes.copy(), v
@@ -255,13 +263,19 @@
def make_varray(self, arraydescr):
return VArrayInfo(arraydescr)
- def make_vstrplain(self):
+ def make_vstrplain(self, is_unicode=False):
+ if is_unicode:
+ return VUniPlainInfo()
return VStrPlainInfo()
- def make_vstrconcat(self):
+ def make_vstrconcat(self, is_unicode=False):
+ if is_unicode:
+ return VUniConcatInfo()
return VStrConcatInfo()
- def make_vstrslice(self):
+ def make_vstrslice(self, is_unicode=False):
+ if is_unicode:
+ return VUniSliceInfo()
return VStrSliceInfo()
def register_virtual_fields(self, virtualbox, fieldboxes):
@@ -292,7 +306,7 @@
# compute the numbering
storage = self.storage
# make sure that nobody attached resume data to this guard yet
- assert storage.rd_numb is None
+ assert not storage.rd_numb
numb, liveboxes_from_env, v = self.memo.number(values,
storage.rd_snapshot)
self.liveboxes_from_env = liveboxes_from_env
@@ -550,6 +564,60 @@
for i in self.fieldnums:
debug_print("\t\t", str(untag(i)))
+
+class VUniPlainInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of the characters of all
+ fieldnums."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ length = len(self.fieldnums)
+ string = decoder.allocate_unicode(length)
+ decoder.virtuals_cache[index] = string
+ for i in range(length):
+ decoder.unicode_setitem(string, i, self.fieldnums[i])
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvuniplaininfo length", len(self.fieldnums))
+
+
+class VUniConcatInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of the concatenation of two
+ other unicode strings."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ # xxx for blackhole resuming, this will build all intermediate
+ # strings and throw them away immediately, which is a bit sub-
+ # efficient. Not sure we care.
+ left, right = self.fieldnums
+ string = decoder.concat_unicodes(left, right)
+ decoder.virtuals_cache[index] = string
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvuniconcatinfo")
+ for i in self.fieldnums:
+ debug_print("\t\t", str(untag(i)))
+
+
+class VUniSliceInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of slicing another
+ unicode string."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ largerstr, start, length = self.fieldnums
+ string = decoder.slice_unicode(largerstr, start, length)
+ decoder.virtuals_cache[index] = string
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvunisliceinfo")
+ for i in self.fieldnums:
+ debug_print("\t\t", str(untag(i)))
+
# ____________________________________________________________
class AbstractResumeDataReader(object):
@@ -629,9 +697,11 @@
# ---------- when resuming for pyjitpl.py, make boxes ----------
-def rebuild_from_resumedata(metainterp, storage, virtualizable_info):
+def rebuild_from_resumedata(metainterp, storage, virtualizable_info,
+ greenfield_info):
resumereader = ResumeDataBoxReader(storage, metainterp)
- boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info)
+ boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info,
+ greenfield_info)
virtualizable_boxes, virtualref_boxes = boxes
frameinfo = storage.rd_frame_info_list
while True:
@@ -661,31 +731,36 @@
self.boxes_f = boxes_f
self._prepare_next_section(info)
- def consume_virtualizable_boxes(self, vinfo, nums):
+ def consume_virtualizable_boxes(self, vinfo, numb):
# we have to ignore the initial part of 'nums' (containing vrefs),
# find the virtualizable from nums[-1], and use it to know how many
# boxes of which type we have to return. This does not write
# anything into the virtualizable.
- virtualizablebox = self.decode_ref(nums[-1])
+ index = len(numb.nums) - 1
+ virtualizablebox = self.decode_ref(numb.nums[index])
virtualizable = vinfo.unwrap_virtualizable_box(virtualizablebox)
- return vinfo.load_list_of_boxes(virtualizable, self, nums)
+ return vinfo.load_list_of_boxes(virtualizable, self, numb)
- def consume_virtualref_boxes(self, nums, end):
+ def consume_virtualref_boxes(self, numb, end):
# Returns a list of boxes, assumed to be all BoxPtrs.
# We leave up to the caller to call vrefinfo.continue_tracing().
assert (end & 1) == 0
- return [self.decode_ref(nums[i]) for i in range(end)]
+ return [self.decode_ref(numb.nums[i]) for i in range(end)]
- def consume_vref_and_vable_boxes(self, vinfo):
- nums = self.cur_numb.nums
- self.cur_numb = self.cur_numb.prev
- if vinfo is None:
- virtualizable_boxes = None
- end = len(nums)
+ def consume_vref_and_vable_boxes(self, vinfo, ginfo):
+ numb = self.cur_numb
+ self.cur_numb = numb.prev
+ if vinfo is not None:
+ virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, numb)
+ end = len(numb.nums) - len(virtualizable_boxes)
+ elif ginfo is not None:
+ index = len(numb.nums) - 1
+ virtualizable_boxes = [self.decode_ref(numb.nums[index])]
+ end = len(numb.nums) - 1
else:
- virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums)
- end = len(nums) - len(virtualizable_boxes)
- virtualref_boxes = self.consume_virtualref_boxes(nums, end)
+ virtualizable_boxes = None
+ end = len(numb.nums)
+ virtualref_boxes = self.consume_virtualref_boxes(numb, end)
return virtualizable_boxes, virtualref_boxes
def allocate_with_vtable(self, known_class):
@@ -709,14 +784,44 @@
strbox, ConstInt(index), charbox)
def concat_strings(self, str1num, str2num):
- calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT)
+ cic = self.metainterp.staticdata.callinfocollection
+ calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT)
str1box = self.decode_box(str1num, REF)
str2box = self.decode_box(str2num, REF)
return self.metainterp.execute_and_record_varargs(
rop.CALL, [ConstInt(func), str1box, str2box], calldescr)
def slice_string(self, strnum, startnum, lengthnum):
- calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_SLICE)
+ cic = self.metainterp.staticdata.callinfocollection
+ calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_SLICE)
+ strbox = self.decode_box(strnum, REF)
+ startbox = self.decode_box(startnum, INT)
+ lengthbox = self.decode_box(lengthnum, INT)
+ stopbox = self.metainterp.execute_and_record(rop.INT_ADD, None,
+ startbox, lengthbox)
+ return self.metainterp.execute_and_record_varargs(
+ rop.CALL, [ConstInt(func), strbox, startbox, stopbox], calldescr)
+
+ def allocate_unicode(self, length):
+ return self.metainterp.execute_and_record(rop.NEWUNICODE,
+ None, ConstInt(length))
+
+ def unicode_setitem(self, strbox, index, charnum):
+ charbox = self.decode_box(charnum, INT)
+ self.metainterp.execute_and_record(rop.UNICODESETITEM, None,
+ strbox, ConstInt(index), charbox)
+
+ def concat_unicodes(self, str1num, str2num):
+ cic = self.metainterp.staticdata.callinfocollection
+ calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT)
+ str1box = self.decode_box(str1num, REF)
+ str2box = self.decode_box(str2num, REF)
+ return self.metainterp.execute_and_record_varargs(
+ rop.CALL, [ConstInt(func), str1box, str2box], calldescr)
+
+ def slice_unicode(self, strnum, startnum, lengthnum):
+ cic = self.metainterp.staticdata.callinfocollection
+ calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE)
strbox = self.decode_box(strnum, REF)
startbox = self.decode_box(startnum, INT)
lengthbox = self.decode_box(lengthnum, INT)
@@ -812,11 +917,12 @@
def blackhole_from_resumedata(blackholeinterpbuilder, jitdriver_sd, storage,
all_virtuals=None):
- resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage,
- all_virtuals)
+ resumereader = ResumeDataDirectReader(blackholeinterpbuilder.metainterp_sd,
+ storage, all_virtuals)
vinfo = jitdriver_sd.virtualizable_info
+ ginfo = jitdriver_sd.greenfield_info
vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
#
# First get a chain of blackhole interpreters whose length is given
# by the depth of rd_frame_info_list. The first one we get must be
@@ -846,11 +952,11 @@
resumereader.done()
return firstbh
-def force_from_resumedata(metainterp_sd, storage, vinfo=None):
- resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage)
+def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo):
+ resumereader = ResumeDataDirectReader(metainterp_sd, storage)
resumereader.handling_async_forcing()
vrefinfo = metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
return resumereader.force_all_virtuals()
class ResumeDataDirectReader(AbstractResumeDataReader):
@@ -861,8 +967,9 @@
# 1: in handle_async_forcing
# 2: resuming from the GUARD_NOT_FORCED
- def __init__(self, cpu, storage, all_virtuals=None):
- self._init(cpu, storage)
+ def __init__(self, metainterp_sd, storage, all_virtuals=None):
+ self._init(metainterp_sd.cpu, storage)
+ self.callinfocollection = metainterp_sd.callinfocollection
if all_virtuals is None: # common case
self._prepare(storage)
else:
@@ -881,23 +988,24 @@
info = blackholeinterp.get_current_position_info()
self._prepare_next_section(info)
- def consume_virtualref_info(self, vrefinfo, nums, end):
+ def consume_virtualref_info(self, vrefinfo, numb, end):
# we have to decode a list of references containing pairs
# [..., virtual, vref, ...] stopping at 'end'
assert (end & 1) == 0
for i in range(0, end, 2):
- virtual = self.decode_ref(nums[i])
- vref = self.decode_ref(nums[i+1])
+ virtual = self.decode_ref(numb.nums[i])
+ vref = self.decode_ref(numb.nums[i+1])
# For each pair, we store the virtual inside the vref.
vrefinfo.continue_tracing(vref, virtual)
- def consume_vable_info(self, vinfo, nums):
+ def consume_vable_info(self, vinfo, numb):
# we have to ignore the initial part of 'nums' (containing vrefs),
# find the virtualizable from nums[-1], load all other values
# from the CPU stack, and copy them into the virtualizable
if vinfo is None:
- return len(nums)
- virtualizable = self.decode_ref(nums[-1])
+ return len(numb.nums)
+ index = len(numb.nums) - 1
+ virtualizable = self.decode_ref(numb.nums[index])
virtualizable = vinfo.cast_gcref_to_vtype(virtualizable)
if self.resume_after_guard_not_forced == 1:
# in the middle of handle_async_forcing()
@@ -909,7 +1017,7 @@
# is and stays 0. Note the call to reset_vable_token() in
# warmstate.py.
assert not virtualizable.vable_token
- return vinfo.write_from_resume_data_partial(virtualizable, self, nums)
+ return vinfo.write_from_resume_data_partial(virtualizable, self, numb)
def load_value_of_type(self, TYPE, tagged):
from pypy.jit.metainterp.warmstate import specialize_value
@@ -925,12 +1033,13 @@
return specialize_value(TYPE, x)
load_value_of_type._annspecialcase_ = 'specialize:arg(1)'
- def consume_vref_and_vable(self, vrefinfo, vinfo):
- nums = self.cur_numb.nums
- self.cur_numb = self.cur_numb.prev
+ def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo):
+ numb = self.cur_numb
+ self.cur_numb = numb.prev
if self.resume_after_guard_not_forced != 2:
- end_vref = self.consume_vable_info(vinfo, nums)
- self.consume_virtualref_info(vrefinfo, nums, end_vref)
+ end_vref = self.consume_vable_info(vinfo, numb)
+ if ginfo is not None: end_vref -= 1
+ self.consume_virtualref_info(vrefinfo, numb, end_vref)
def allocate_with_vtable(self, known_class):
from pypy.jit.metainterp.executor import exec_new_with_vtable
@@ -954,7 +1063,8 @@
str2 = self.decode_ref(str2num)
str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str1)
str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str2)
- funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT)
+ cic = self.callinfocollection
+ funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT)
result = funcptr(str1, str2)
return lltype.cast_opaque_ptr(llmemory.GCREF, result)
@@ -963,7 +1073,35 @@
start = self.decode_int(startnum)
length = self.decode_int(lengthnum)
str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str)
- funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_SLICE)
+ cic = self.callinfocollection
+ funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_SLICE)
+ result = funcptr(str, start, start + length)
+ return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+
+ def allocate_unicode(self, length):
+ return self.cpu.bh_newunicode(length)
+
+ def unicode_setitem(self, str, index, charnum):
+ char = self.decode_int(charnum)
+ self.cpu.bh_unicodesetitem(str, index, char)
+
+ def concat_unicodes(self, str1num, str2num):
+ str1 = self.decode_ref(str1num)
+ str2 = self.decode_ref(str2num)
+ str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1)
+ str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2)
+ cic = self.callinfocollection
+ funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT)
+ result = funcptr(str1, str2)
+ return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+
+ def slice_unicode(self, strnum, startnum, lengthnum):
+ str = self.decode_ref(strnum)
+ start = self.decode_int(startnum)
+ length = self.decode_int(lengthnum)
+ str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str)
+ cic = self.callinfocollection
+ funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE)
result = funcptr(str, start, start + length)
return lltype.cast_opaque_ptr(llmemory.GCREF, result)
@@ -1054,8 +1192,9 @@
'at', compute_unique_id(frameinfo))
frameinfo = frameinfo.prev
numb = storage.rd_numb
- while numb is not None:
- debug_print('\tnumb', str([untag(i) for i in numb.nums]),
+ while numb:
+ debug_print('\tnumb', str([untag(numb.nums[i])
+ for i in range(len(numb.nums))]),
'at', compute_unique_id(numb))
numb = numb.prev
for const in storage.rd_consts:
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/simple_optimize.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/simple_optimize.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/simple_optimize.py Tue Nov 23 10:12:47 2010
@@ -9,11 +9,13 @@
def transform(op):
from pypy.jit.metainterp.history import AbstractDescr
- # Rename CALL_PURE to CALL.
+ # Rename CALL_PURE and CALL_LOOPINVARIANT to CALL.
# Simplify the VIRTUAL_REF_* so that they don't show up in the backend.
if op.getopnum() == rop.CALL_PURE:
op = ResOperation(rop.CALL, op.getarglist()[1:], op.result,
op.getdescr())
+ elif op.getopnum() == rop.CALL_LOOPINVARIANT:
+ op = op.copy_and_change(rop.CALL)
elif op.getopnum() == rop.VIRTUAL_REF:
op = ResOperation(rop.SAME_AS, [op.getarg(0)], op.result)
elif op.getopnum() == rop.VIRTUAL_REF_FINISH:
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_basic.py Tue Nov 23 10:12:47 2010
@@ -2,6 +2,8 @@
import sys
from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside
from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant
+from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed
+from pypy.rlib.jit import unroll_safe, current_trace_length
from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats
from pypy.jit.backend.llgraph import runner
from pypy.jit.metainterp import pyjitpl, history
@@ -44,6 +46,7 @@
num_green_args = 0
portal_graph = graphs[0]
virtualizable_info = None
+ greenfield_info = None
result_type = result_kind
portal_runner_ptr = "???"
@@ -1644,6 +1647,58 @@
res = self.interp_operations(f, [10, 3.5])
assert res == 3.5
+ def test_jit_debug(self):
+ myjitdriver = JitDriver(greens = [], reds = ['x'])
+ class A:
+ pass
+ def f(x):
+ while x > 0:
+ myjitdriver.can_enter_jit(x=x)
+ myjitdriver.jit_merge_point(x=x)
+ jit_debug("hi there:", x)
+ jit_debug("foobar")
+ x -= 1
+ return x
+ res = self.meta_interp(f, [8])
+ assert res == 0
+ self.check_loops(jit_debug=2)
+
+ def test_assert_green(self):
+ def f(x, promote):
+ if promote:
+ x = hint(x, promote=True)
+ assert_green(x)
+ return x
+ res = self.interp_operations(f, [8, 1])
+ assert res == 8
+ py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0])
+
+ def test_current_trace_length(self):
+ myjitdriver = JitDriver(greens = ['g'], reds = ['x'])
+ @dont_look_inside
+ def residual():
+ print "hi there"
+ @unroll_safe
+ def loop(g):
+ y = 0
+ while y < g:
+ residual()
+ y += 1
+ def f(x, g):
+ n = 0
+ while x > 0:
+ myjitdriver.can_enter_jit(x=x, g=g)
+ myjitdriver.jit_merge_point(x=x, g=g)
+ loop(g)
+ x -= 1
+ n = current_trace_length()
+ return n
+ res = self.meta_interp(f, [5, 8])
+ assert 14 < res < 42
+ res = self.meta_interp(f, [5, 2])
+ assert 4 < res < 14
+
+
class TestOOtype(BasicTests, OOJitMixin):
def test_oohash(self):
@@ -1751,7 +1806,7 @@
c = bool(p1)
d = not bool(p2)
return 1000*a + 100*b + 10*c + d
- prebuilt = [lltype.malloc(TP, flavor='raw')] * 2
+ prebuilt = [lltype.malloc(TP, flavor='raw', immortal=True)] * 2
expected = f(0, 1)
assert self.interp_operations(f, [0, 1]) == expected
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_compile.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_compile.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_compile.py Tue Nov 23 10:12:47 2010
@@ -6,8 +6,8 @@
from pypy.jit.metainterp.compile import ResumeGuardCountersInt
from pypy.jit.metainterp.compile import compile_tmp_callback
from pypy.jit.metainterp import optimize, jitprof, typesystem, compile
-from pypy.jit.metainterp.test.oparser import parse
from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin
+from pypy.jit.tool.oparser import parse
def test_insert_loop_token():
@@ -85,7 +85,7 @@
metainterp.history.inputargs = loop.inputargs[:]
#
loop_tokens = []
- loop_token = compile_new_loop(metainterp, loop_tokens, [], 0)
+ loop_token = compile_new_loop(metainterp, loop_tokens, 0)
assert loop_tokens == [loop_token]
assert loop_token.number == 1
assert staticdata.globaldata.loopnumbering == 2
@@ -101,7 +101,7 @@
metainterp.history.operations = loop.operations[:]
metainterp.history.inputargs = loop.inputargs[:]
#
- loop_token_2 = compile_new_loop(metainterp, loop_tokens, [], 0)
+ loop_token_2 = compile_new_loop(metainterp, loop_tokens, 0)
assert loop_token_2 is loop_token
assert loop_tokens == [loop_token]
assert len(cpu.seen) == 0
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_del.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_del.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_del.py Tue Nov 23 10:12:47 2010
@@ -85,6 +85,7 @@
def test_signal_action(self):
from pypy.module.signal.interp_signal import SignalActionFlag
action = SignalActionFlag()
+ action.has_bytecode_counter = True
#
myjitdriver = JitDriver(greens = [], reds = ['n', 'x'])
class X:
@@ -92,17 +93,17 @@
#
def f(n):
x = X()
- while n > 0:
+ action.reset_ticker(n)
+ while True:
myjitdriver.can_enter_jit(n=n, x=x)
myjitdriver.jit_merge_point(n=n, x=x)
x.foo = n
n -= 1
- if action.get() != 0:
+ if action.decrement_ticker(1) < 0:
break
- action.set(0)
return 42
self.meta_interp(f, [20])
- self.check_loops(getfield_raw=1, call=0, call_pure=0)
+ self.check_loops(getfield_raw=1, setfield_raw=1, call=0, call_pure=0)
class TestOOtype(DelTests, OOJitMixin):
def setup_class(cls):
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_exception.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_exception.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_exception.py Tue Nov 23 10:12:47 2010
@@ -1,6 +1,6 @@
import py, sys
from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin
-from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE
+from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE, dont_look_inside
from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask
from pypy.jit.codewriter.policy import StopAtXPolicy
@@ -588,6 +588,33 @@
res = self.interp_operations(f, [99])
assert res == 21
+ def test_bug_exc1_noexc_exc2(self):
+ myjitdriver = JitDriver(greens=[], reds=['i'])
+ @dont_look_inside
+ def rescall(i):
+ if i < 10:
+ raise KeyError
+ if i < 20:
+ return None
+ raise ValueError
+ def f(i):
+ while i < 30:
+ myjitdriver.can_enter_jit(i=i)
+ myjitdriver.jit_merge_point(i=i)
+ try:
+ rescall(i)
+ except KeyError:
+ assert i < 10
+ except ValueError:
+ assert i >= 20
+ else:
+ assert 10 <= i < 20
+ i += 1
+ return i
+ res = self.meta_interp(f, [0], inline=True)
+ assert res == 30
+
+
class MyError(Exception):
def __init__(self, n):
self.n = n
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_jitdriver.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_jitdriver.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_jitdriver.py Tue Nov 23 10:12:47 2010
@@ -72,6 +72,33 @@
# we expect no loop at all for 'loop1': it should always be inlined
self.check_tree_loop_count(2) # 1 x loop, 1 x enter bridge
+ def test_inactive_jitdriver(self):
+ myjitdriver1 = JitDriver(greens=[], reds=['n', 'm'],
+ get_printable_location = getloc1)
+ myjitdriver2 = JitDriver(greens=['g'], reds=['r'],
+ get_printable_location = getloc2)
+ #
+ myjitdriver1.active = False # <===
+ #
+ def loop1(n, m):
+ while n > 0:
+ myjitdriver1.can_enter_jit(n=n, m=m)
+ myjitdriver1.jit_merge_point(n=n, m=m)
+ n -= m
+ return n
+ #
+ def loop2(g, r):
+ while r > 0:
+ myjitdriver2.can_enter_jit(g=g, r=r)
+ myjitdriver2.jit_merge_point(g=g, r=r)
+ r += loop1(r, g) + (-1)
+ return r
+ #
+ res = self.meta_interp(loop2, [4, 40], repeat=7, inline=True)
+ assert res == loop2(4, 40)
+ # we expect no int_sub, but a residual call
+ self.check_loops(int_sub=0, call=1)
+
class TestLLtype(MultipleJitDriversTests, LLJitMixin):
pass
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_logger.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_logger.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_logger.py Tue Nov 23 10:12:47 2010
@@ -1,6 +1,6 @@
import sys
from pypy.rlib import debug
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
from pypy.jit.metainterp import logger
from pypy.jit.metainterp.typesystem import llhelper
from StringIO import StringIO
@@ -97,7 +97,7 @@
def test_debug_merge_point(self):
inp = '''
[]
- debug_merge_point("info")
+ debug_merge_point("info", 0)
'''
loop, oloop = self.reparse(inp, check_equal=False)
assert loop.operations[0].getarg(0)._get_str() == 'info'
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizefindnode.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizefindnode.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizefindnode.py Tue Nov 23 10:12:47 2010
@@ -18,7 +18,7 @@
from pypy.jit.metainterp.specnode import ConstantSpecNode
from pypy.jit.codewriter.effectinfo import EffectInfo
from pypy.jit.codewriter.heaptracker import register_known_gctype, adr2int
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
def test_sort_descrs():
class PseudoDescr(AbstractDescr):
@@ -117,33 +117,32 @@
EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE))
arraycopydescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
EffectInfo([], [], [], oopspecindex=EffectInfo.OS_ARRAYCOPY))
- strconcatdescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_CONCAT))
- slicedescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_SLICE))
- strequaldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_EQUAL))
- streq_slice_checknull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHECKNULL))
- streq_slice_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_NONNULL))
- streq_slice_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHAR))
- streq_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL))
- streq_nonnull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL_CHAR))
- streq_checknull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_CHECKNULL_CHAR))
- streq_lengthok_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_LENGTHOK))
+
+ for _name, _os in [
+ ('strconcatdescr', 'OS_STR_CONCAT'),
+ ('strslicedescr', 'OS_STR_SLICE'),
+ ('strequaldescr', 'OS_STR_EQUAL'),
+ ('streq_slice_checknull_descr', 'OS_STREQ_SLICE_CHECKNULL'),
+ ('streq_slice_nonnull_descr', 'OS_STREQ_SLICE_NONNULL'),
+ ('streq_slice_char_descr', 'OS_STREQ_SLICE_CHAR'),
+ ('streq_nonnull_descr', 'OS_STREQ_NONNULL'),
+ ('streq_nonnull_char_descr', 'OS_STREQ_NONNULL_CHAR'),
+ ('streq_checknull_char_descr', 'OS_STREQ_CHECKNULL_CHAR'),
+ ('streq_lengthok_descr', 'OS_STREQ_LENGTHOK'),
+ ]:
+ _oopspecindex = getattr(EffectInfo, _os)
+ locals()[_name] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+ #
+ _oopspecindex = getattr(EffectInfo, _os.replace('STR', 'UNI'))
+ locals()[_name.replace('str', 'unicode')] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+
+ s2u_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR2UNICODE))
+ #
class LoopToken(AbstractDescr):
pass
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizeopt.py Tue Nov 23 10:12:47 2010
@@ -12,7 +12,7 @@
from pypy.jit.metainterp.jitprof import EmptyProfiler
from pypy.jit.metainterp import executor, compile, resume, history
from pypy.jit.metainterp.resoperation import rop, opname, ResOperation
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
##class FakeFrame(object):
## parent_resumedata_snapshot = None
@@ -41,7 +41,7 @@
b1 = BoxInt()
opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu),
None)
- fdescr = ResumeGuardDescr(None, None)
+ fdescr = ResumeGuardDescr(None)
op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr)
# setup rd data
fi0 = resume.FrameInfo(None, "code0", 11)
@@ -51,12 +51,12 @@
#
opt.store_final_boxes_in_guard(op)
if op.getfailargs() == [b0, b1]:
- assert fdescr.rd_numb.nums == [tag(1, TAGBOX)]
- assert fdescr.rd_numb.prev.nums == [tag(0, TAGBOX)]
+ assert list(fdescr.rd_numb.nums) == [tag(1, TAGBOX)]
+ assert list(fdescr.rd_numb.prev.nums) == [tag(0, TAGBOX)]
else:
assert op.getfailargs() == [b1, b0]
- assert fdescr.rd_numb.nums == [tag(0, TAGBOX)]
- assert fdescr.rd_numb.prev.nums == [tag(1, TAGBOX)]
+ assert list(fdescr.rd_numb.nums) == [tag(0, TAGBOX)]
+ assert list(fdescr.rd_numb.prev.nums) == [tag(1, TAGBOX)]
assert fdescr.rd_virtuals is None
assert fdescr.rd_consts == []
@@ -132,14 +132,21 @@
# ____________________________________________________________
def equaloplists(oplist1, oplist2, strict_fail_args=True, remap={}):
- print '-'*20, 'Comparing lists', '-'*20
+ # try to use the full width of the terminal to display the list
+ # unfortunately, does not work with the default capture method of py.test
+ # (which is fd), you you need to use either -s or --capture=sys, else you
+ # get the standard 80 columns width
+ totwidth = py.io.get_terminal_width()
+ width = totwidth / 2 - 1
+ print ' Comparing lists '.center(totwidth, '-')
+ print '%s| %s' % ('optimized'.center(width), 'expected'.center(width))
for op1, op2 in zip(oplist1, oplist2):
txt1 = str(op1)
txt2 = str(op2)
while txt1 or txt2:
- print '%-39s| %s' % (txt1[:39], txt2[:39])
- txt1 = txt1[39:]
- txt2 = txt2[39:]
+ print '%s| %s' % (txt1[:width].ljust(width), txt2[:width])
+ txt1 = txt1[width:]
+ txt2 = txt2[width:]
assert op1.getopnum() == op2.getopnum()
assert op1.numargs() == op2.numargs()
for i in range(op1.numargs()):
@@ -257,11 +264,17 @@
metainterp_sd = FakeMetaInterpStaticData(self.cpu)
if hasattr(self, 'vrefinfo'):
metainterp_sd.virtualref_info = self.vrefinfo
+ if hasattr(self, 'callinfocollection'):
+ metainterp_sd.callinfocollection = self.callinfocollection
optimize_loop_1(metainterp_sd, loop)
#
expected = self.parse(optops)
print '\n'.join([str(o) for o in loop.operations])
self.assert_equal(loop, expected)
+ return loop
+
+
+class OptimizeOptTest(BaseTestOptimizeOpt):
def test_simple(self):
ops = """
@@ -776,8 +789,12 @@
i3 = call(i2, descr=nonwritedescr)
jump(i1) # the exception is considered lost when we loop back
"""
+ # note that 'guard_no_exception' at the very start must be kept
+ # around: bridges may start with one. (In case of loops we could
+ # remove it, but we probably don't care.)
expected = """
[i]
+ guard_no_exception() []
i1 = int_add(i, 3)
i2 = call(i1, descr=nonwritedescr)
guard_no_exception() [i1, i2]
@@ -796,10 +813,10 @@
guard_value(i2, 1) []
i3 = call_loopinvariant(1, i1, descr=nonwritedescr)
guard_no_exception() []
- guard_value(i2, 1) []
+ guard_value(i3, 1) []
i4 = call_loopinvariant(1, i1, descr=nonwritedescr)
guard_no_exception() []
- guard_value(i2, 1) []
+ guard_value(i4, 1) []
jump(i1)
"""
expected = """
@@ -1379,7 +1396,7 @@
ops = """
[p1]
i1 = getfield_gc(p1, descr=valuedescr)
- debug_merge_point(15)
+ debug_merge_point(15, 0)
i2 = getfield_gc(p1, descr=valuedescr)
escape(i1)
escape(i2)
@@ -1388,7 +1405,7 @@
expected = """
[p1]
i1 = getfield_gc(p1, descr=valuedescr)
- debug_merge_point(15)
+ debug_merge_point(15, 0)
escape(i1)
escape(i1)
jump(p1)
@@ -2643,7 +2660,7 @@
''', rop.GUARD_TRUE)
-class TestLLtype(BaseTestOptimizeOpt, LLtypeMixin):
+class TestLLtype(OptimizeOptTest, LLtypeMixin):
def test_residual_call_does_not_invalidate_caches(self):
ops = """
@@ -3487,7 +3504,7 @@
i0 = strlen(p0)
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_addsub_const(self):
ops = """
@@ -3893,6 +3910,15 @@
"""
self.optimize_loop(ops, 'Not, Not', expected)
+ # ----------
+ def optimize_strunicode_loop(self, ops, spectext, optops):
+ # check with the arguments passed in
+ self.optimize_loop(ops, spectext, optops)
+ # check with replacing 'str' with 'unicode' everywhere
+ self.optimize_loop(ops.replace('str','unicode').replace('s"', 'u"'),
+ spectext,
+ optops.replace('str','unicode').replace('s"', 'u"'))
+
def test_newstr_1(self):
ops = """
[i0]
@@ -3905,7 +3931,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_newstr_2(self):
ops = """
@@ -3921,7 +3947,7 @@
[i0, i1]
jump(i1, i0)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_concat_1(self):
ops = """
@@ -3942,7 +3968,7 @@
copystrcontent(p2, p3, 0, i4, i5)
jump(p2, p3)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_concat_vstr2_str(self):
ops = """
@@ -3965,7 +3991,7 @@
copystrcontent(p2, p3, 0, 2, i4)
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_vstr2(self):
ops = """
@@ -3989,7 +4015,7 @@
i6 = int_add(i5, 1) # will be killed by the backend
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_str_str(self):
ops = """
@@ -4016,12 +4042,12 @@
copystrcontent(p3, p5, 0, i12b, i3b)
jump(p2, p3, p5)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_cstr1(self):
ops = """
[p2]
- p3 = call(0, p2, "x", descr=strconcatdescr)
+ p3 = call(0, p2, s"x", descr=strconcatdescr)
jump(p3)
"""
expected = """
@@ -4035,28 +4061,28 @@
i5 = int_add(i4, 1) # will be killed by the backend
jump(p3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_str_concat_consts(self):
ops = """
[]
- p1 = same_as("ab")
- p2 = same_as("cde")
+ p1 = same_as(s"ab")
+ p2 = same_as(s"cde")
p3 = call(0, p1, p2, descr=strconcatdescr)
escape(p3)
jump()
"""
expected = """
[]
- escape("abcde")
+ escape(s"abcde")
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_strunicode_loop(ops, '', expected)
def test_str_slice_1(self):
ops = """
[p1, i1, i2]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
jump(p2, i1, i2)
"""
expected = """
@@ -4066,12 +4092,12 @@
copystrcontent(p1, p2, i1, 0, i3)
jump(p2, i1, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_slice_2(self):
ops = """
[p1, i2]
- p2 = call(0, p1, 0, i2, descr=slicedescr)
+ p2 = call(0, p1, 0, i2, descr=strslicedescr)
jump(p2, i2)
"""
expected = """
@@ -4080,13 +4106,13 @@
copystrcontent(p1, p2, 0, 0, i2)
jump(p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_slice_3(self):
ops = """
[p1, i1, i2, i3, i4]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
- p3 = call(0, p2, i3, i4, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
+ p3 = call(0, p2, i3, i4, descr=strslicedescr)
jump(p3, i1, i2, i3, i4)
"""
expected = """
@@ -4098,12 +4124,12 @@
copystrcontent(p1, p3, i6, 0, i5)
jump(p3, i1, i2, i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not, Not', expected)
def test_str_slice_getitem1(self):
ops = """
[p1, i1, i2, i3]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
i4 = strgetitem(p2, i3)
escape(i4)
jump(p1, i1, i2, i3)
@@ -4116,7 +4142,7 @@
escape(i4)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
def test_str_slice_plain(self):
ops = """
@@ -4124,7 +4150,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i3)
strsetitem(p1, 1, i4)
- p2 = call(0, p1, 1, 2, descr=slicedescr)
+ p2 = call(0, p1, 1, 2, descr=strslicedescr)
i5 = strgetitem(p2, 0)
escape(i5)
jump(i3, i4)
@@ -4134,12 +4160,12 @@
escape(i4)
jump(i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_slice_concat(self):
ops = """
[p1, i1, i2, p2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
p4 = call(0, p3, p2, descr=strconcatdescr)
jump(p4, i1, i2, p2)
"""
@@ -4155,27 +4181,25 @@
copystrcontent(p2, p4, 0, i3, i4b)
jump(p4, i1, i2, p2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
# ----------
- def optimize_loop_extradescrs(self, ops, spectext, optops):
+ def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops):
from pypy.jit.metainterp.optimizeopt import string
- def my_callinfo_for_oopspec(oopspecindex):
- calldescrtype = type(LLtypeMixin.strequaldescr)
- for value in LLtypeMixin.__dict__.values():
- if isinstance(value, calldescrtype):
- if (value.get_extra_info() and
- value.get_extra_info().oopspecindex == oopspecindex):
- # returns 0 for 'func' in this test
- return value, 0
- raise AssertionError("not found: oopspecindex=%d" % oopspecindex)
+ class FakeCallInfoCollection:
+ def callinfo_for_oopspec(self, oopspecindex):
+ calldescrtype = type(LLtypeMixin.strequaldescr)
+ for value in LLtypeMixin.__dict__.values():
+ if isinstance(value, calldescrtype):
+ extra = value.get_extra_info()
+ if extra and extra.oopspecindex == oopspecindex:
+ # returns 0 for 'func' in this test
+ return value, 0
+ raise AssertionError("not found: oopspecindex=%d" %
+ oopspecindex)
#
- saved = string.callinfo_for_oopspec
- try:
- string.callinfo_for_oopspec = my_callinfo_for_oopspec
- self.optimize_loop(ops, spectext, optops)
- finally:
- string.callinfo_for_oopspec = saved
+ self.callinfocollection = FakeCallInfoCollection()
+ self.optimize_strunicode_loop(ops, spectext, optops)
def test_str_equal_noop1(self):
ops = """
@@ -4184,7 +4208,7 @@
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', ops)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', ops)
def test_str_equal_noop2(self):
ops = """
@@ -4209,12 +4233,13 @@
escape(i0)
jump(p1, p2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ expected)
def test_str_equal_slice1(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p4, p3, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4226,12 +4251,13 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice2(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4243,13 +4269,14 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice3(self):
ops = """
[p1, i1, i2, p3]
guard_nonnull(p3) []
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4262,13 +4289,14 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice4(self):
ops = """
[p1, i1, i2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
- i0 = call(0, p3, "x", descr=strequaldescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
+ i0 = call(0, p3, s"x", descr=strequaldescr)
escape(i0)
jump(p1, i1, i2)
"""
@@ -4279,12 +4307,13 @@
escape(i0)
jump(p1, i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ expected)
def test_str_equal_slice5(self):
ops = """
[p1, i1, i2, i3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
p5 = newstr(1)
strsetitem(p5, 0, i3)
i0 = call(0, p5, p4, descr=strequaldescr)
@@ -4298,7 +4327,8 @@
escape(i0)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_none1(self):
ops = """
@@ -4313,7 +4343,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_none2(self):
ops = """
@@ -4328,30 +4358,30 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull1(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=strequaldescr)
+ i0 = call(0, p1, s"hello world", descr=strequaldescr)
escape(i0)
jump(p1)
"""
expected = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=streq_nonnull_descr)
+ i0 = call(0, p1, s"hello world", descr=streq_nonnull_descr)
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull2(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4363,13 +4393,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull3(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4380,13 +4410,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull4(self):
ops = """
[p1, p2]
p4 = call(0, p1, p2, descr=strconcatdescr)
- i0 = call(0, "hello world", p4, descr=strequaldescr)
+ i0 = call(0, s"hello world", p4, descr=strequaldescr)
escape(i0)
jump(p1, p2)
"""
@@ -4401,17 +4431,17 @@
i5 = strlen(p2)
i6 = int_add(i4, i5) # will be killed by the backend
copystrcontent(p2, p4, 0, i4, i5)
- i0 = call(0, "hello world", p4, descr=streq_nonnull_descr)
+ i0 = call(0, s"hello world", p4, descr=streq_nonnull_descr)
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
def test_str_equal_chars0(self):
ops = """
[i1]
p1 = newstr(0)
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4420,14 +4450,14 @@
escape(1)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_chars1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4437,7 +4467,7 @@
escape(i0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_chars2(self):
ops = """
@@ -4445,7 +4475,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=strequaldescr)
+ i0 = call(0, p1, s"xy", descr=strequaldescr)
escape(i0)
jump(i1, i2)
"""
@@ -4454,16 +4484,16 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=streq_lengthok_descr)
+ i0 = call(0, p1, s"xy", descr=streq_lengthok_descr)
escape(i0)
jump(i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
def test_str_equal_chars3(self):
ops = """
[p1]
- i0 = call(0, "x", p1, descr=strequaldescr)
+ i0 = call(0, s"x", p1, descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4473,14 +4503,14 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_lengthmismatch1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, "xy", p1, descr=strequaldescr)
+ i0 = call(0, s"xy", p1, descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4489,13 +4519,36 @@
escape(0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+
+ def test_str2unicode_constant(self):
+ ops = """
+ []
+ p0 = call(0, "xy", descr=s2u_descr) # string -> unicode
+ escape(p0)
+ jump()
+ """
+ expected = """
+ []
+ escape(u"xy")
+ jump()
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, '', expected)
- # XXX unicode operations
- # XXX str2unicode
+ def test_str2unicode_nonconstant(self):
+ ops = """
+ [p0]
+ p1 = call(0, p0, descr=s2u_descr) # string -> unicode
+ escape(p1)
+ jump(p1)
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', ops)
+ # more generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
-##class TestOOtype(BaseTestOptimizeOpt, OOtypeMixin):
+##class TestOOtype(OptimizeOptTest, OOtypeMixin):
## def test_instanceof(self):
## ops = """
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_recursive.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_recursive.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_recursive.py Tue Nov 23 10:12:47 2010
@@ -1142,6 +1142,19 @@
res = self.meta_interp(main, [], inline=True, trace_limit=tlimit)
assert ''.join(res.chars) == 'ABCDEFGHIabcdefghijJ' * 5
+ def test_no_duplicates_bug(self):
+ driver = JitDriver(greens = ['codeno'], reds = ['i'],
+ get_printable_location = lambda codeno: str(codeno))
+ def portal(codeno, i):
+ while i > 0:
+ driver.can_enter_jit(codeno=codeno, i=i)
+ driver.jit_merge_point(codeno=codeno, i=i)
+ if codeno > 0:
+ break
+ portal(i, i)
+ i -= 1
+ self.meta_interp(portal, [0, 10], inline=True)
+
class TestLLtype(RecursiveTests, LLJitMixin):
pass
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resoperation.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resoperation.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resoperation.py Tue Nov 23 10:12:47 2010
@@ -61,3 +61,10 @@
assert op.getarglist() == ['a', 'b']
assert op.result == 'c'
assert op.getdescr() is mydescr
+
+def test_can_malloc():
+ mydescr = AbstractDescr()
+ assert rop.ResOperation(rop.rop.NEW, [], 'b').can_malloc()
+ call = rop.ResOperation(rop.rop.CALL, ['a', 'b'], 'c', descr=mydescr)
+ assert call.can_malloc()
+ assert not rop.ResOperation(rop.rop.INT_ADD, ['a', 'b'], 'c').can_malloc()
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resume.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resume.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resume.py Tue Nov 23 10:12:47 2010
@@ -51,6 +51,7 @@
class MyMetaInterp:
_already_allocated_resume_virtuals = None
+ callinfocollection = None
def __init__(self, cpu=None):
if cpu is None:
@@ -141,6 +142,13 @@
assert bh.written_f == expected_f
+def Numbering(prev, nums):
+ numb = lltype.malloc(NUMBERING, len(nums))
+ numb.prev = prev or lltype.nullptr(NUMBERING)
+ for i in range(len(nums)):
+ numb.nums[i] = nums[i]
+ return numb
+
def test_simple_read():
#b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()]
c1, c2, c3 = [ConstInt(111), ConstInt(222), ConstInt(333)]
@@ -156,12 +164,12 @@
storage.rd_numb = numb
#
cpu = MyCPU([42, gcref1, -66])
- reader = ResumeDataDirectReader(cpu, storage)
+ metainterp = MyMetaInterp(cpu)
+ reader = ResumeDataDirectReader(metainterp, storage)
_next_section(reader, 42, 111, gcrefnull, 42, gcref1)
_next_section(reader, 222, 333)
_next_section(reader, 42, gcref1, -66)
#
- metainterp = MyMetaInterp(cpu)
reader = ResumeDataBoxReader(storage, metainterp)
bi, br, bf = [None]*3, [None]*2, [None]*0
info = MyBlackholeInterp([lltype.Signed, lltype.Signed,
@@ -193,7 +201,7 @@
storage.rd_numb = numb
#
cpu = MyCPU([])
- reader = ResumeDataDirectReader(cpu, storage)
+ reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage)
_next_section(reader, 100)
@@ -211,7 +219,7 @@
class FakeMetainterp(object):
_already_allocated_resume_virtuals = None
cpu = None
- reader = ResumeDataDirectReader(None, FakeStorage())
+ reader = ResumeDataDirectReader(MyMetaInterp(None), FakeStorage())
assert reader.force_all_virtuals() == ["allocated", reader.virtual_default]
# ____________________________________________________________
@@ -240,6 +248,17 @@
return FakeBuiltObject(strconcat=[left, right])
def slice_string(self, str, start, length):
return FakeBuiltObject(strslice=[str, start, length])
+ def allocate_unicode(self, length):
+ return FakeBuiltObject(unistring=[None]*length)
+ def unicode_setitem(self, unistring, i, fieldnum):
+ value, tag = untag(fieldnum)
+ assert tag == TAGINT
+ assert 0 <= i < len(unistring.unistring)
+ unistring.unistring[i] = value
+ def concat_unicodes(self, left, right):
+ return FakeBuiltObject(uniconcat=[left, right])
+ def slice_unicode(self, str, start, length):
+ return FakeBuiltObject(unislice=[str, start, length])
class FakeBuiltObject(object):
def __init__(self, **kwds):
@@ -304,6 +323,30 @@
assert reader.force_all_virtuals() == [
FakeBuiltObject(strslice=info.fieldnums)]
+def test_vuniplaininfo():
+ info = VUniPlainInfo()
+ info.fieldnums = [tag(60, TAGINT)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(unistring=[60])]
+
+def test_vuniconcatinfo():
+ info = VUniConcatInfo()
+ info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(uniconcat=info.fieldnums)]
+
+def test_vunisliceinfo():
+ info = VUniSliceInfo()
+ info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX), tag(30, TAGBOX)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(unislice=info.fieldnums)]
+
# ____________________________________________________________
@@ -355,15 +398,15 @@
assert fi1.pc == 3
def test_Numbering_create():
- l = [1, 2]
+ l = [rffi.r_short(1), rffi.r_short(2)]
numb = Numbering(None, l)
- assert numb.prev is None
- assert numb.nums is l
+ assert not numb.prev
+ assert list(numb.nums) == l
- l1 = ['b3']
+ l1 = [rffi.r_short(3)]
numb1 = Numbering(numb, l1)
- assert numb1.prev is numb
- assert numb1.nums is l1
+ assert numb1.prev == numb
+ assert list(numb1.nums) == l1
def test_capture_resumedata():
b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()]
@@ -729,11 +772,12 @@
assert liveboxes == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX),
b3: tag(2, TAGBOX)}
- assert numb.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX),
- tag(1, TAGINT)]
- assert numb.prev.nums == [tag(0, TAGBOX), tag(1, TAGINT), tag(1, TAGBOX),
- tag(0, TAGBOX), tag(2, TAGINT)]
- assert numb.prev.prev is None
+ assert list(numb.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX),
+ tag(1, TAGINT)]
+ assert list(numb.prev.nums) == [tag(0, TAGBOX), tag(1, TAGINT),
+ tag(1, TAGBOX),
+ tag(0, TAGBOX), tag(2, TAGINT)]
+ assert not numb.prev.prev
numb2, liveboxes2, v = memo.number({}, snap2)
assert v == 0
@@ -741,9 +785,9 @@
assert liveboxes2 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX),
b3: tag(2, TAGBOX)}
assert liveboxes2 is not liveboxes
- assert numb2.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX),
- tag(3, TAGINT)]
- assert numb2.prev is numb.prev
+ assert list(numb2.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX),
+ tag(3, TAGINT)]
+ assert numb2.prev == numb.prev
env3 = [c3, b3, b1, c3]
snap3 = Snapshot(snap, env3)
@@ -764,9 +808,9 @@
assert v == 0
assert liveboxes3 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX)}
- assert numb3.nums == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX),
- tag(3, TAGINT)]
- assert numb3.prev is numb.prev
+ assert list(numb3.nums) == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX),
+ tag(3, TAGINT)]
+ assert numb3.prev == numb.prev
# virtual
env4 = [c3, b4, b1, c3]
@@ -777,9 +821,9 @@
assert liveboxes4 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX),
b4: tag(0, TAGVIRTUAL)}
- assert numb4.nums == [tag(3, TAGINT), tag(0, TAGVIRTUAL), tag(0, TAGBOX),
- tag(3, TAGINT)]
- assert numb4.prev is numb.prev
+ assert list(numb4.nums) == [tag(3, TAGINT), tag(0, TAGVIRTUAL),
+ tag(0, TAGBOX), tag(3, TAGINT)]
+ assert numb4.prev == numb.prev
env5 = [b1, b4, b5]
snap5 = Snapshot(snap4, env5)
@@ -790,9 +834,9 @@
assert liveboxes5 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX),
b4: tag(0, TAGVIRTUAL), b5: tag(1, TAGVIRTUAL)}
- assert numb5.nums == [tag(0, TAGBOX), tag(0, TAGVIRTUAL),
- tag(1, TAGVIRTUAL)]
- assert numb5.prev is numb4
+ assert list(numb5.nums) == [tag(0, TAGBOX), tag(0, TAGVIRTUAL),
+ tag(1, TAGVIRTUAL)]
+ assert numb5.prev == numb4
def test_ResumeDataLoopMemo_number_boxes():
memo = ResumeDataLoopMemo(FakeMetaInterpStaticData())
@@ -890,7 +934,7 @@
liveboxes = modifier.finish({})
assert storage.rd_snapshot is None
cpu = MyCPU([])
- reader = ResumeDataDirectReader(cpu, storage)
+ reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage)
_next_section(reader, sys.maxint, 2**16, -65)
_next_section(reader, 2, 3)
_next_section(reader, sys.maxint, 1, sys.maxint, 2**16)
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_string.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_string.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_string.py Tue Nov 23 10:12:47 2010
@@ -6,14 +6,17 @@
class StringTests:
+ _str, _chr = str, chr
+
def test_eq_residual(self):
+ _str = self._str
jitdriver = JitDriver(greens = [], reds = ['n', 'i', 's'])
- global_s = "hello"
+ global_s = _str("hello")
def f(n, b, s):
if b:
- s += "ello"
+ s += _str("ello")
else:
- s += "allo"
+ s += _str("allo")
i = 0
while n > 0:
jitdriver.can_enter_jit(s=s, n=n, i=i)
@@ -21,18 +24,19 @@
n -= 1 + (s == global_s)
i += 1
return i
- res = self.meta_interp(f, [10, True, 'h'], listops=True)
+ res = self.meta_interp(f, [10, True, _str('h')], listops=True)
assert res == 5
self.check_loops(**{self.CALL: 1, self.CALL_PURE: 0})
def test_eq_folded(self):
+ _str = self._str
jitdriver = JitDriver(greens = ['s'], reds = ['n', 'i'])
- global_s = "hello"
+ global_s = _str("hello")
def f(n, b, s):
if b:
- s += "ello"
+ s += _str("ello")
else:
- s += "allo"
+ s += _str("allo")
i = 0
while n > 0:
jitdriver.can_enter_jit(s=s, n=n, i=i)
@@ -40,31 +44,18 @@
n -= 1 + (s == global_s)
i += 1
return i
- res = self.meta_interp(f, [10, True, 'h'], listops=True)
+ res = self.meta_interp(f, [10, True, _str('h')], listops=True)
assert res == 5
self.check_loops(**{self.CALL: 0, self.CALL_PURE: 0})
def test_newstr(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['n', 'm'])
def f(n, m):
while True:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- bytecode = 'adlfkj' + chr(n)
- res = bytecode[n]
- m -= 1
- if m < 0:
- return ord(res)
- res = self.meta_interp(f, [6, 10])
- assert res == 6
-
- def test_newunicode(self):
- jitdriver = JitDriver(greens = [], reds = ['n', 'm'])
- def f(n, m):
- while True:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- bytecode = u'adlfkj' + unichr(n)
+ bytecode = _str('adlfkj') + _chr(n)
res = bytecode[n]
m -= 1
if m < 0:
@@ -73,95 +64,96 @@
assert res == 6
def test_char2string_pure(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n):
- while n > 0:
- jitdriver.can_enter_jit(n=n)
- jitdriver.jit_merge_point(n=n)
- s = dochr(n)
- if not we_are_jitted():
- s += s # forces to be a string
- if n > 100:
- escape(s)
- n -= 1
- return 42
- self.meta_interp(f, [6])
- self.check_loops(newstr=0, strsetitem=0, strlen=0,
- newunicode=0, unicodesetitem=0, unicodelen=0)
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n):
+ while n > 0:
+ jitdriver.can_enter_jit(n=n)
+ jitdriver.jit_merge_point(n=n)
+ s = _chr(n)
+ if not we_are_jitted():
+ s += s # forces to be a string
+ if n > 100:
+ escape(s)
+ n -= 1
+ return 42
+ self.meta_interp(f, [6])
+ self.check_loops(newstr=0, strsetitem=0, strlen=0,
+ newunicode=0, unicodesetitem=0, unicodelen=0)
def test_char2string_escape(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['n', 'total'])
- @dont_look_inside
- def escape(x):
- return ord(x[0])
- def f(n):
- total = 0
- while n > 0:
- jitdriver.can_enter_jit(n=n, total=total)
- jitdriver.jit_merge_point(n=n, total=total)
- s = dochr(n)
- if not we_are_jitted():
- s += s # forces to be a string
- total += escape(s)
- n -= 1
- return total
- res = self.meta_interp(f, [6])
- assert res == 21
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['n', 'total'])
+ @dont_look_inside
+ def escape(x):
+ return ord(x[0])
+ def f(n):
+ total = 0
+ while n > 0:
+ jitdriver.can_enter_jit(n=n, total=total)
+ jitdriver.jit_merge_point(n=n, total=total)
+ s = _chr(n)
+ if not we_are_jitted():
+ s += s # forces to be a string
+ total += escape(s)
+ n -= 1
+ return total
+ res = self.meta_interp(f, [6])
+ assert res == 21
def test_char2string2char(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'total'])
- def f(m):
- total = 0
- while m > 0:
- jitdriver.can_enter_jit(m=m, total=total)
- jitdriver.jit_merge_point(m=m, total=total)
- string = dochr(m)
- if m > 100:
- string += string # forces to be a string
- # read back the character
- c = string[0]
- total += ord(c)
- m -= 1
- return total
- res = self.meta_interp(f, [6])
- assert res == 21
- self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0,
- newunicode=0, unicodegetitem=0, unicodesetitem=0,
- unicodelen=0)
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['m', 'total'])
+ def f(m):
+ total = 0
+ while m > 0:
+ jitdriver.can_enter_jit(m=m, total=total)
+ jitdriver.jit_merge_point(m=m, total=total)
+ string = _chr(m)
+ if m > 100:
+ string += string # forces to be a string
+ # read back the character
+ c = string[0]
+ total += ord(c)
+ m -= 1
+ return total
+ res = self.meta_interp(f, [6])
+ assert res == 21
+ self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0,
+ newunicode=0, unicodegetitem=0, unicodesetitem=0,
+ unicodelen=0)
def test_strconcat_pure(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- mylist = [somestr+str(i) for i in range(10)]
- def f(n, m):
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + mylist[m]
- if m > 100:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [6, 7])
- self.check_loops(newstr=0, strsetitem=0,
- newunicode=0, unicodesetitem=0,
- call=0, call_pure=0)
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ mylist = [_str("abc") + _str(i) for i in range(10)]
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = mylist[n] + mylist[m]
+ if m > 100:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(newstr=0, strsetitem=0,
+ newunicode=0, unicodesetitem=0,
+ call=0, call_pure=0)
def test_strconcat_escape_str_str(self):
+ _str = self._str
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
@@ -171,46 +163,64 @@
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=0, copystrcontent=2,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=0, copystrcontent=2,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=0,
+ copyunicodecontent=2,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_str_char(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + chr(m)
+ s = mylist[n] + _chr(m)
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=1,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_char_str(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = chr(n) + mylist[m]
+ s = _chr(n) + mylist[m]
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=1,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_char_char(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
@@ -219,91 +229,132 @@
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = chr(n) + chr(m)
+ s = _chr(n) + _chr(m)
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=2, copystrcontent=0,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=2, copystrcontent=0,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=2,
+ copyunicodecontent=0,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_str_char_str(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + chr(n) + mylist[m]
+ s = mylist[n] + _chr(n) + mylist[m]
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=2,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=2,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=2,
+ call=1, call_pure=0) # escape
def test_strconcat_guard_fail(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- mylist = [somestr+str(i) for i in range(12)]
- def f(n, m):
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + mylist[m]
- if m & 1:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [6, 10])
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ mylist = [_str("abc") + _str(i) for i in range(12)]
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = mylist[n] + mylist[m]
+ if m & 1:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 10])
def test_strslice(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n, m):
- assert n >= 0
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = "foobarbazetc"[m:n]
- if m <= 5:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [10, 10])
+ _str = self._str
+ longstring = _str("foobarbazetc")
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n, m):
+ assert n >= 0
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = longstring[m:n]
+ if m <= 5:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [10, 10])
def test_streq_char(self):
- for somestr in ["?abcdefg", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n, m):
- assert n >= 0
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = somestr[:m]
- escape(s == "?")
- m -= 1
- return 42
- self.meta_interp(f, [6, 7])
- self.check_loops(newstr=0, newunicode=0)
-
-
-class TestOOtype(StringTests, OOJitMixin):
- CALL = "oosend"
- CALL_PURE = "oosend_pure"
+ _str = self._str
+ longstring = _str("?abcdefg")
+ somechar = _str("?")
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n, m):
+ assert n >= 0
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = longstring[:m]
+ escape(s == somechar)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(newstr=0, newunicode=0)
+
+
+#class TestOOtype(StringTests, OOJitMixin):
+# CALL = "oosend"
+# CALL_PURE = "oosend_pure"
class TestLLtype(StringTests, LLJitMixin):
CALL = "call"
CALL_PURE = "call_pure"
+
+class TestLLtypeUnicode(TestLLtype):
+ _str, _chr = unicode, unichr
+
+ def test_str2unicode(self):
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ class Foo:
+ pass
+ @dont_look_inside
+ def escape(x):
+ assert x == _str("6y")
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ foo = Foo()
+ foo.y = chr(m)
+ foo.y = "y"
+ s = _str(str(n)) + _str(foo.y)
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(call=3, # str(), _str(), escape()
+ newunicode=1, unicodegetitem=0,
+ unicodesetitem=1, copyunicodecontent=1)
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_virtualref.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_virtualref.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_virtualref.py Tue Nov 23 10:12:47 2010
@@ -88,12 +88,16 @@
cpu.get_latest_value_int = lambda i:guard_op.getfailargs()[i].getint()
cpu.get_latest_value_ref = lambda i:guard_op.getfailargs()[i].getref_base()
cpu.clear_latest_values = lambda count: None
- resumereader = ResumeDataDirectReader(cpu, guard_op.getdescr())
+ class FakeMetaInterpSd:
+ callinfocollection = None
+ FakeMetaInterpSd.cpu = cpu
+ resumereader = ResumeDataDirectReader(FakeMetaInterpSd(),
+ guard_op.getdescr())
vrefinfo = self.metainterp.staticdata.virtualref_info
lst = []
vrefinfo.continue_tracing = lambda vref, virtual: \
lst.append((vref, virtual))
- resumereader.consume_vref_and_vable(vrefinfo, None)
+ resumereader.consume_vref_and_vable(vrefinfo, None, None)
del vrefinfo.continue_tracing
assert len(lst) == 1
lltype.cast_opaque_ptr(lltype.Ptr(JIT_VIRTUAL_REF),
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_warmspot.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_warmspot.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_warmspot.py Tue Nov 23 10:12:47 2010
@@ -296,6 +296,69 @@
assert res == 1
self.check_loops(int_add=1) # I get 13 without the loop_header()
+ def test_omit_can_enter_jit(self):
+ # Simple test comparing the effects of always giving a can_enter_jit(),
+ # or not giving any. Mostly equivalent, except that if given, it is
+ # ignored the first time, and so it ends up taking one extra loop to
+ # start JITting.
+ mydriver = JitDriver(greens=[], reds=['m'])
+ #
+ for i2 in range(10):
+ def f2(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f2, [i2])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f2: no loop generated for i2==%d" % i2
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ for i1 in range(10):
+ def f1(m):
+ while m > 0:
+ mydriver.can_enter_jit(m=m)
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f1, [i1])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f1: no loop generated for i1==%d" % i1
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ assert i1 - 1 == i2
+
+ def test_no_loop_at_all(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f2(m):
+ mydriver.jit_merge_point(m=m)
+ return m - 1
+ def f1(m):
+ while m > 0:
+ m = f2(m)
+ self.meta_interp(f1, [8])
+ # it should generate one "loop" only, which ends in a FINISH
+ # corresponding to the return from f2.
+ self.check_tree_loop_count(1)
+ self.check_loop_count(0)
+
+ def test_simple_loop(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f1(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m = m - 1
+ self.meta_interp(f1, [8])
+ self.check_loop_count(1)
+ self.check_loops({'int_sub': 1, 'int_gt': 1, 'guard_true': 1,
+ 'jump': 1})
+
class TestLLWarmspot(WarmspotTests, LLJitMixin):
CPUClass = runner.LLtypeCPU
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_ztranslation.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_ztranslation.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_ztranslation.py Tue Nov 23 10:12:47 2010
@@ -79,7 +79,7 @@
res = ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass,
type_system=self.type_system)
assert res == main(40, 5)
- res = rpython_ll_meta_interp(main, [40, 5], loops=2,
+ res = rpython_ll_meta_interp(main, [40, 5],
CPUClass=self.CPUClass,
type_system=self.type_system,
optimizer=OPTIMIZER_FULL,
@@ -120,7 +120,7 @@
res = ll_meta_interp(main, [40], CPUClass=self.CPUClass,
type_system=self.type_system)
assert res == main(40)
- res = rpython_ll_meta_interp(main, [40], loops=2, CPUClass=self.CPUClass,
+ res = rpython_ll_meta_interp(main, [40], CPUClass=self.CPUClass,
type_system=self.type_system,
optimizer=OPTIMIZER_FULL,
ProfilerClass=Profiler)
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/virtualizable.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/virtualizable.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/virtualizable.py Tue Nov 23 10:12:47 2010
@@ -100,48 +100,48 @@
i = i + 1
assert len(boxes) == i + 1
#
- def write_from_resume_data_partial(virtualizable, reader, nums):
+ def write_from_resume_data_partial(virtualizable, reader, numb):
# Load values from the reader (see resume.py) described by
# the list of numbers 'nums', and write them in their proper
# place in the 'virtualizable'. This works from the end of
# the list and returns the index in 'nums' of the start of
# the virtualizable data found, allowing the caller to do
# further processing with the start of the list.
- i = len(nums) - 1
+ i = len(numb.nums) - 1
assert i >= 0
for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev:
lst = getattr(virtualizable, fieldname)
for j in range(getlength(lst)-1, -1, -1):
i -= 1
assert i >= 0
- x = reader.load_value_of_type(ARRAYITEMTYPE, nums[i])
+ x = reader.load_value_of_type(ARRAYITEMTYPE, numb.nums[i])
setarrayitem(lst, j, x)
for FIELDTYPE, fieldname in unroll_static_fields_rev:
i -= 1
assert i >= 0
- x = reader.load_value_of_type(FIELDTYPE, nums[i])
+ x = reader.load_value_of_type(FIELDTYPE, numb.nums[i])
setattr(virtualizable, fieldname, x)
return i
#
- def load_list_of_boxes(virtualizable, reader, nums):
+ def load_list_of_boxes(virtualizable, reader, numb):
# Uses 'virtualizable' only to know the length of the arrays;
# does not write anything into it. The returned list is in
# the format expected of virtualizable_boxes, so it ends in
# the virtualizable itself.
- i = len(nums) - 1
+ i = len(numb.nums) - 1
assert i >= 0
- boxes = [reader.decode_box_of_type(self.VTYPEPTR, nums[i])]
+ boxes = [reader.decode_box_of_type(self.VTYPEPTR, numb.nums[i])]
for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev:
lst = getattr(virtualizable, fieldname)
for j in range(getlength(lst)-1, -1, -1):
i -= 1
assert i >= 0
- box = reader.decode_box_of_type(ARRAYITEMTYPE, nums[i])
+ box = reader.decode_box_of_type(ARRAYITEMTYPE,numb.nums[i])
boxes.append(box)
for FIELDTYPE, fieldname in unroll_static_fields_rev:
i -= 1
assert i >= 0
- box = reader.decode_box_of_type(FIELDTYPE, nums[i])
+ box = reader.decode_box_of_type(FIELDTYPE, numb.nums[i])
boxes.append(box)
boxes.reverse()
return boxes
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/virtualref.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/virtualref.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/virtualref.py Tue Nov 23 10:12:47 2010
@@ -16,7 +16,8 @@
('virtualref_index', lltype.Signed),
('forced', rclass.OBJECTPTR))
self.jit_virtual_ref_vtable = lltype.malloc(rclass.OBJECT_VTABLE,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self.jit_virtual_ref_vtable.name = rclass.alloc_array_name(
'jit_virtual_ref')
# build some constants
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/warmspot.py Tue Nov 23 10:12:47 2010
@@ -67,9 +67,16 @@
def jittify_and_run(interp, graph, args, repeat=1,
backendopt=False, trace_limit=sys.maxint,
debug_level=DEBUG_STEPS, inline=False, **kwds):
+ from pypy.config.config import ConfigError
translator = interp.typer.annotator.translator
- translator.config.translation.gc = "boehm"
- translator.config.translation.list_comprehension_operations = True
+ try:
+ translator.config.translation.gc = "boehm"
+ except ConfigError:
+ pass
+ try:
+ translator.config.translation.list_comprehension_operations = True
+ except ConfigError:
+ pass
warmrunnerdesc = WarmRunnerDesc(translator, backendopt=backendopt, **kwds)
for jd in warmrunnerdesc.jitdrivers_sd:
jd.warmstate.set_param_threshold(3) # for tests
@@ -91,8 +98,7 @@
repeat -= 1
return res
-def rpython_ll_meta_interp(function, args, backendopt=True,
- loops='not used right now', **kwds):
+def rpython_ll_meta_interp(function, args, backendopt=True, **kwds):
return ll_meta_interp(function, args, backendopt=backendopt,
translate_support_code=True, **kwds)
@@ -103,15 +109,16 @@
for i in range(len(block.operations)):
op = block.operations[i]
if (op.opname == 'jit_marker' and
- op.args[0].value == marker_name):
+ op.args[0].value == marker_name and
+ op.args[1].value.active): # the jitdriver
results.append((graph, block, i))
return results
def find_can_enter_jit(graphs):
- results = _find_jit_marker(graphs, 'can_enter_jit')
- if not results:
- raise Exception("no can_enter_jit found!")
- return results
+ return _find_jit_marker(graphs, 'can_enter_jit')
+
+def find_loop_headers(graphs):
+ return _find_jit_marker(graphs, 'loop_header')
def find_jit_merge_points(graphs):
results = _find_jit_marker(graphs, 'jit_merge_point')
@@ -204,9 +211,9 @@
"there are multiple jit_merge_points with the same jitdriver"
def split_graph_and_record_jitdriver(self, graph, block, pos):
- jd = JitDriverStaticData()
- jd._jit_merge_point_pos = (graph, block, pos)
op = block.operations[pos]
+ jd = JitDriverStaticData()
+ jd._jit_merge_point_pos = (graph, op)
args = op.args[2:]
s_binding = self.translator.annotator.binding
jd._portal_args_s = [s_binding(v) for v in args]
@@ -279,10 +286,20 @@
def make_virtualizable_infos(self):
vinfos = {}
for jd in self.jitdrivers_sd:
+ #
+ jd.greenfield_info = None
+ for name in jd.jitdriver.greens:
+ if '.' in name:
+ from pypy.jit.metainterp.greenfield import GreenFieldInfo
+ jd.greenfield_info = GreenFieldInfo(self.cpu, jd)
+ break
+ #
if not jd.jitdriver.virtualizables:
jd.virtualizable_info = None
jd.index_of_virtualizable = -1
continue
+ else:
+ assert jd.greenfield_info is None, "XXX not supported yet"
#
jitdriver = jd.jitdriver
assert len(jitdriver.virtualizables) == 1 # for now
@@ -450,8 +467,7 @@
self.make_args_specification(jd)
def make_args_specification(self, jd):
- graph, block, index = jd._jit_merge_point_pos
- op = block.operations[index]
+ graph, op = jd._jit_merge_point_pos
greens_v, reds_v = support.decode_hp_hint_args(op)
ALLARGS = [v.concretetype for v in (greens_v + reds_v)]
jd._green_args_spec = [v.concretetype for v in greens_v]
@@ -467,26 +483,37 @@
[lltype.Signed, llmemory.GCREF], RESTYPE)
def rewrite_can_enter_jits(self):
- can_enter_jits = find_can_enter_jit(self.translator.graphs)
sublists = {}
for jd in self.jitdrivers_sd:
- sublists[jd.jitdriver] = []
+ sublists[jd.jitdriver] = jd, []
+ jd.no_loop_header = True
+ #
+ loop_headers = find_loop_headers(self.translator.graphs)
+ for graph, block, index in loop_headers:
+ op = block.operations[index]
+ jitdriver = op.args[1].value
+ assert jitdriver in sublists, \
+ "loop_header with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
+ jd.no_loop_header = False
+ #
+ can_enter_jits = find_can_enter_jit(self.translator.graphs)
for graph, block, index in can_enter_jits:
op = block.operations[index]
jitdriver = op.args[1].value
assert jitdriver in sublists, \
"can_enter_jit with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
origportalgraph = jd._jit_merge_point_pos[0]
if graph is not origportalgraph:
- sublists[jitdriver].append((graph, block, index))
+ sublist.append((graph, block, index))
+ jd.no_loop_header = False
else:
pass # a 'can_enter_jit' before the 'jit-merge_point', but
# originally in the same function: we ignore it here
# see e.g. test_jitdriver.test_simple
for jd in self.jitdrivers_sd:
- sublist = sublists[jd.jitdriver]
- assert len(sublist) > 0, \
- "found no can_enter_jit for %r" % (jd.jitdriver,)
+ _, sublist = sublists[jd.jitdriver]
self.rewrite_can_enter_jit(jd, sublist)
def rewrite_can_enter_jit(self, jd, can_enter_jits):
@@ -494,6 +521,19 @@
FUNCPTR = jd._PTR_JIT_ENTER_FUNCTYPE
jit_enter_fnptr = self.helper_func(FUNCPTR, jd._maybe_enter_jit_fn)
+ if len(can_enter_jits) == 0:
+ # see test_warmspot.test_no_loop_at_all
+ operations = jd.portal_graph.startblock.operations
+ op1 = operations[0]
+ assert (op1.opname == 'jit_marker' and
+ op1.args[0].value == 'jit_merge_point')
+ op0 = SpaceOperation(
+ 'jit_marker',
+ [Constant('can_enter_jit', lltype.Void)] + op1.args[1:],
+ None)
+ operations.insert(0, op0)
+ can_enter_jits = [(jd.portal_graph, jd.portal_graph.startblock, 0)]
+
for graph, block, index in can_enter_jits:
if graph is jd._jit_merge_point_pos[0]:
continue
@@ -702,8 +742,14 @@
# ____________________________________________________________
# Now mutate origportalgraph to end with a call to portal_runner_ptr
#
- _, origblock, origindex = jd._jit_merge_point_pos
- op = origblock.operations[origindex]
+ _, op = jd._jit_merge_point_pos
+ for origblock in origportalgraph.iterblocks():
+ if op in origblock.operations:
+ break
+ else:
+ assert False, "lost the operation %r in the graph %r" % (
+ op, origportalgraph)
+ origindex = origblock.operations.index(op)
assert op.opname == 'jit_marker'
assert op.args[0].value == 'jit_merge_point'
greens_v, reds_v = support.decode_hp_hint_args(op)
Modified: pypy/branch/arm-backend/pypy/jit/metainterp/warmstate.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/metainterp/warmstate.py (original)
+++ pypy/branch/arm-backend/pypy/jit/metainterp/warmstate.py Tue Nov 23 10:12:47 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, llmemory, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import hlstr, llstr, cast_base_ptr_to_instance
from pypy.rpython.annlowlevel import cast_object_to_ptr
@@ -24,7 +24,11 @@
"""
INPUT = lltype.typeOf(x)
if INPUT is lltype.Signed:
- return lltype.cast_primitive(TYPE, x) # XXX missing: Ptr(non-gc)
+ if isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'raw':
+ # non-gc pointer
+ return rffi.cast(TYPE, x)
+ else:
+ return lltype.cast_primitive(TYPE, x)
elif INPUT is lltype.Float:
assert TYPE is lltype.Float
return x
@@ -172,6 +176,9 @@
meth(default_value)
def set_param_threshold(self, threshold):
+ if threshold < 0:
+ self.increment_threshold = 0 # never reach the THRESHOLD_LIMIT
+ return
if threshold < 2:
threshold = 2
self.increment_threshold = (self.THRESHOLD_LIMIT // threshold) + 1
Modified: pypy/branch/arm-backend/pypy/jit/tl/pypyjit.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/tl/pypyjit.py (original)
+++ pypy/branch/arm-backend/pypy/jit/tl/pypyjit.py Tue Nov 23 10:12:47 2010
@@ -38,10 +38,12 @@
config.objspace.allworkingmodules = False
config.objspace.usemodules.pypyjit = True
config.objspace.usemodules.array = True
-config.objspace.usemodules._weakref = False
+config.objspace.usemodules._weakref = True
config.objspace.usemodules._sre = False
+#
+config.objspace.usemodules._ffi = True
+#
set_pypy_opt_level(config, level='jit')
-config.objspace.std.withinlineddict = True
if BACKEND == 'c':
config.objspace.std.multimethods = 'mrd'
Modified: pypy/branch/arm-backend/pypy/jit/tl/pypyjit_child.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/tl/pypyjit_child.py (original)
+++ pypy/branch/arm-backend/pypy/jit/tl/pypyjit_child.py Tue Nov 23 10:12:47 2010
@@ -16,7 +16,7 @@
interp.heap.malloc_nonmovable = returns_null # XXX
from pypy.jit.backend.llgraph.runner import LLtypeCPU
- LLtypeCPU.supports_floats = False # for now
+ #LLtypeCPU.supports_floats = False # for now
apply_jit(interp, graph, LLtypeCPU)
Modified: pypy/branch/arm-backend/pypy/jit/tl/spli/interpreter.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/tl/spli/interpreter.py (original)
+++ pypy/branch/arm-backend/pypy/jit/tl/spli/interpreter.py Tue Nov 23 10:12:47 2010
@@ -1,12 +1,14 @@
import os
-from pypy.tool import stdlib_opcode as opcode
+from pypy.tool import stdlib_opcode
from pypy.jit.tl.spli import objects, pycode
-from pypy.tool.stdlib_opcode import unrolling_opcode_descs
-from pypy.tool.stdlib_opcode import opcode_method_names
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib.jit import JitDriver, hint, dont_look_inside
from pypy.rlib.objectmodel import we_are_translated
+opcode_method_names = stdlib_opcode.host_bytecode_spec.method_names
+unrolling_opcode_descs = unrolling_iterable(
+ stdlib_opcode.host_bytecode_spec.ordered_opdescs)
+HAVE_ARGUMENT = stdlib_opcode.host_HAVE_ARGUMENT
compare_ops = [
"cmp_lt", # "<"
@@ -79,7 +81,7 @@
self.stack_depth = hint(self.stack_depth, promote=True)
op = ord(code[instr_index])
instr_index += 1
- if op >= opcode.HAVE_ARGUMENT:
+ if op >= HAVE_ARGUMENT:
low = ord(code[instr_index])
hi = ord(code[instr_index + 1])
oparg = (hi << 8) | low
@@ -183,6 +185,12 @@
next_instr += arg
return next_instr
+ def POP_JUMP_IF_FALSE(self, arg, next_instr, code):
+ w_cond = self.pop()
+ if not w_cond.is_true():
+ next_instr = arg
+ return next_instr
+
def JUMP_FORWARD(self, arg, next_instr, code):
return next_instr + arg
Modified: pypy/branch/arm-backend/pypy/jit/tool/loopviewer.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/tool/loopviewer.py (original)
+++ pypy/branch/arm-backend/pypy/jit/tool/loopviewer.py Tue Nov 23 10:12:47 2010
@@ -9,7 +9,7 @@
import optparse
from pprint import pprint
from pypy.tool import logparser
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.history import ConstInt
from pypy.rpython.lltypesystem import llmemory, lltype
Modified: pypy/branch/arm-backend/pypy/jit/tool/pypytrace-mode.el
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/tool/pypytrace-mode.el (original)
+++ pypy/branch/arm-backend/pypy/jit/tool/pypytrace-mode.el Tue Nov 23 10:12:47 2010
@@ -26,7 +26,7 @@
("<.*FieldDescr \\([^ ]*\\)" (1 'font-lock-variable-name-face))
;; comment out debug_merge_point, but then highlight specific part of it
("^debug_merge_point.*" . font-lock-comment-face)
- ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)')"
+ ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)"
(1 'compilation-warning t)
(2 'escape-glyph t)
(3 'font-lock-string-face t)
Modified: pypy/branch/arm-backend/pypy/jit/tool/showstats.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/tool/showstats.py (original)
+++ pypy/branch/arm-backend/pypy/jit/tool/showstats.py Tue Nov 23 10:12:47 2010
@@ -4,15 +4,16 @@
import autopath
import sys, py
from pypy.tool import logparser
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.resoperation import rop
from pypy.rpython.lltypesystem import lltype, llmemory
def main(argv):
log = logparser.parse_log_file(argv[0])
+ log_count_lines = open(argv[0] + '.count').readlines()
parts = logparser.extract_category(log, "jit-log-opt-")
for i, oplist in enumerate(parts):
- loop = parse(oplist, no_namespace=True)
+ loop = parse(oplist, no_namespace=True, nonstrict=True)
num_ops = 0
num_dmp = 0
num_guards = 0
@@ -23,7 +24,11 @@
num_ops += 1
if op.is_guard():
num_guards += 1
- print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp)
-
+ if num_dmp == 0:
+ print "Loop #%d, length: %d, opcodes: %d, guards: %d" % (i, num_ops, num_dmp, num_guards)
+ else:
+ print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp)
+ print loop.comment, "run", log_count_lines[i].split(":")[1].strip(), "times"
+
if __name__ == '__main__':
main(sys.argv[1:])
Modified: pypy/branch/arm-backend/pypy/jit/tool/test/test_traceviewer.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/tool/test/test_traceviewer.py (original)
+++ pypy/branch/arm-backend/pypy/jit/tool/test/test_traceviewer.py Tue Nov 23 10:12:47 2010
@@ -19,7 +19,7 @@
def test_no_of_loops(self):
data = [preparse("""
# Loop 0 : loop with 39 ops
- debug_merge_point('')
+ debug_merge_point('', 0)
guard_class(p4, 141310752, descr=<Guard5>) [p0, p1]
p60 = getfield_gc(p4, descr=<GcPtrFieldDescr 16>)
guard_nonnull(p60, descr=<Guard6>) [p0, p1]
@@ -51,7 +51,7 @@
assert loop.right.content == 'extra'
def test_postparse(self):
- real_loops = [FinalBlock("debug_merge_point('<code object _runCallbacks, file '/tmp/x/twisted-trunk/twisted/internet/defer.py', line 357> #40 POP_TOP')", None)]
+ real_loops = [FinalBlock("debug_merge_point('<code object _runCallbacks, file '/tmp/x/twisted-trunk/twisted/internet/defer.py', line 357> #40 POP_TOP', 0)", None)]
postprocess(real_loops, real_loops[:], {})
assert real_loops[0].header.startswith("_runCallbacks, file '/tmp/x/twisted-trunk/twisted/internet/defer.py', line 357")
Modified: pypy/branch/arm-backend/pypy/jit/tool/traceviewer.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/tool/traceviewer.py (original)
+++ pypy/branch/arm-backend/pypy/jit/tool/traceviewer.py Tue Nov 23 10:12:47 2010
@@ -253,10 +253,10 @@
def main(loopfile, use_threshold, view=True):
countname = py.path.local(loopfile + '.count')
if countname.check():
- counts = [re.split('(<code)|(<loop)', line, maxsplit=1)
- for line in countname.readlines()]
- counts = Counts([('<code' + k.strip("\n"), int(v.strip('\n').strip()))
- for v, _, _, k in counts])
+ #counts = [line.split(':', 1) for line in countname.readlines()]
+ #counts = Counts([('<code' + k.strip("\n"), int(v.strip('\n').strip()))
+ # for v, k in counts])
+ counts = Counts([])
l = list(sorted(counts.values()))
if len(l) > 20 and use_threshold:
counts.threshold = l[-20]
Modified: pypy/branch/arm-backend/pypy/module/__builtin__/descriptor.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/__builtin__/descriptor.py (original)
+++ pypy/branch/arm-backend/pypy/module/__builtin__/descriptor.py Tue Nov 23 10:12:47 2010
@@ -96,6 +96,8 @@
)
class W_Property(Wrappable):
+ _immutable_fields_ = ["w_fget", "w_fset", "w_fdel"]
+
def init(self, space, w_fget=None, w_fset=None, w_fdel=None, w_doc=None):
self.w_fget = w_fget
self.w_fset = w_fset
@@ -183,4 +185,3 @@
fget = interp_attrproperty_w('w_fget', W_Property),
fset = interp_attrproperty_w('w_fset', W_Property),
)
-
Modified: pypy/branch/arm-backend/pypy/module/__builtin__/interp_classobj.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/__builtin__/interp_classobj.py (original)
+++ pypy/branch/arm-backend/pypy/module/__builtin__/interp_classobj.py Tue Nov 23 10:12:47 2010
@@ -2,9 +2,11 @@
from pypy.interpreter.error import OperationError, operationerrfmt
from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, applevel
from pypy.interpreter.gateway import interp2app, ObjSpace
-from pypy.interpreter.typedef import TypeDef, make_weakref_descr
+from pypy.interpreter.typedef import TypeDef
from pypy.interpreter.argument import Arguments
from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.typedef import GetSetProperty, descr_get_dict
+from pypy.interpreter.typedef import descr_set_dict
from pypy.rlib.rarithmetic import r_uint, intmask
from pypy.rlib.objectmodel import compute_identity_hash
from pypy.rlib.debug import make_sure_not_resized
@@ -57,6 +59,14 @@
self.bases_w = bases
self.w_dict = w_dict
+ def instantiate(self, space):
+ cache = space.fromcache(Cache)
+ if self.lookup(space, '__del__') is not None:
+ w_inst = cache.cls_with_del(space, self)
+ else:
+ w_inst = cache.cls_without_del(space, self)
+ return w_inst
+
def getdict(self):
return self.w_dict
@@ -100,15 +110,15 @@
return False
@jit.unroll_safe
- def lookup(self, space, w_attr):
+ def lookup(self, space, attr):
# returns w_value or interplevel None
- w_result = space.finditem(self.w_dict, w_attr)
+ w_result = space.finditem_str(self.w_dict, attr)
if w_result is not None:
return w_result
for base in self.bases_w:
# XXX fix annotation of bases_w to be a list of W_ClassObjects
assert isinstance(base, W_ClassObject)
- w_result = base.lookup(space, w_attr)
+ w_result = base.lookup(space, attr)
if w_result is not None:
return w_result
return None
@@ -122,7 +132,7 @@
return space.wrap(self.name)
elif name == "__bases__":
return space.newtuple(self.bases_w)
- w_value = self.lookup(space, w_attr)
+ w_value = self.lookup(space, name)
if w_value is None:
raise operationerrfmt(
space.w_AttributeError,
@@ -147,7 +157,7 @@
self.setbases(space, w_value)
return
elif name == "__del__":
- if self.lookup(space, w_attr) is None:
+ if self.lookup(space, name) is None:
msg = ("a __del__ method added to an existing class "
"will not be called")
space.warn(msg, space.w_RuntimeWarning)
@@ -195,13 +205,20 @@
# NOT_RPYTHON
return '<W_ClassObject(%s)>' % self.name
+class Cache:
+ def __init__(self, space):
+ from pypy.interpreter.typedef import _usersubclswithfeature
+ # evil
+ self.cls_without_del = _usersubclswithfeature(
+ space.config, W_InstanceObject, "dict", "weakref")
+ self.cls_with_del = _usersubclswithfeature(
+ space.config, self.cls_without_del, "del")
+
+
def class_descr_call(space, w_self, __args__):
self = space.interp_w(W_ClassObject, w_self)
- if self.lookup(space, space.wrap('__del__')) is not None:
- w_inst = W_InstanceObjectWithDel(space, self)
- else:
- w_inst = W_InstanceObject(space, self)
- w_init = w_inst.getattr_from_class(space, space.wrap('__init__'))
+ w_inst = self.instantiate(space)
+ w_init = w_inst.getattr_from_class(space, '__init__')
if w_init is not None:
w_result = space.call_args(w_init, __args__)
if not space.is_w(w_result, space.w_None):
@@ -234,7 +251,7 @@
def make_unary_instance_method(name):
def unaryop(self, space):
- w_meth = self.getattr(space, space.wrap(name), True)
+ w_meth = self.getattr(space, name, True)
return space.call_function(w_meth)
unaryop.func_name = name
return unaryop
@@ -242,7 +259,7 @@
def make_binary_returning_notimplemented_instance_method(name):
def binaryop(self, space, w_other):
try:
- w_meth = self.getattr(space, space.wrap(name), False)
+ w_meth = self.getattr(space, name, False)
except OperationError, e:
if e.match(space, space.w_AttributeError):
return space.w_NotImplemented
@@ -267,7 +284,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_meth = self.getattr(space, space.wrap(specialname), False)
+ w_meth = self.getattr(space, specialname, False)
if w_meth is None:
return space.w_NotImplemented
return space.call_function(w_meth, w_b)
@@ -278,7 +295,7 @@
def rbinaryop(self, space, w_other):
w_a, w_b = _coerce_helper(space, self, w_other)
if w_a is None or w_a is self:
- w_meth = self.getattr(space, space.wrap(rspecialname), False)
+ w_meth = self.getattr(space, rspecialname, False)
if w_meth is None:
return space.w_NotImplemented
return space.call_function(w_meth, w_other)
@@ -302,46 +319,34 @@
raise OperationError(
space.w_TypeError,
space.wrap("instance() first arg must be class"))
- if space.is_w(w_dict, space.w_None):
- w_dict = None
- elif not space.is_true(space.isinstance(w_dict, space.w_dict)):
- raise OperationError(
- space.w_TypeError,
- space.wrap("instance() second arg must be dictionary or None"))
- return W_InstanceObject(space, w_class, w_dict)
+ w_result = w_class.instantiate(space)
+ if not space.is_w(w_dict, space.w_None):
+ w_result.setdict(space, w_dict)
+ return w_result
class W_InstanceObject(Wrappable):
- def __init__(self, space, w_class, w_dict=None):
- if w_dict is None:
- w_dict = space.newdict(instance=True)
+ def __init__(self, space, w_class):
+ # note that user_setup is overridden by the typedef.py machinery
+ self.user_setup(space, space.gettypeobject(self.typedef))
assert isinstance(w_class, W_ClassObject)
self.w_class = w_class
- self.w_dict = w_dict
- self.space = space
-
- def getdict(self):
- return self.w_dict
- def setdict(self, space, w_dict):
- if (w_dict is None or
- not space.is_true(space.isinstance(w_dict, space.w_dict))):
- raise OperationError(
- space.w_TypeError,
- space.wrap("__dict__ must be a dictionary object"))
- self.w_dict = w_dict
+ def user_setup(self, space, w_subtype):
+ self.space = space
- def setclass(self, space, w_class):
+ def set_oldstyle_class(self, space, w_class):
if w_class is None or not isinstance(w_class, W_ClassObject):
raise OperationError(
space.w_TypeError,
space.wrap("__class__ must be set to a class"))
self.w_class = w_class
- def getattr_from_class(self, space, w_name):
+ def getattr_from_class(self, space, name):
# Look up w_name in the class dict, and call its __get__.
# This method ignores the instance dict and the __getattr__.
# Returns None if not found.
- w_value = self.w_class.lookup(space, w_name)
+ assert isinstance(name, str)
+ w_value = self.w_class.lookup(space, name)
if w_value is None:
return None
w_descr_get = space.lookup(w_value, '__get__')
@@ -349,19 +354,20 @@
return w_value
return space.call_function(w_descr_get, w_value, self, self.w_class)
- def getattr(self, space, w_name, exc=True):
+ def getattr(self, space, name, exc=True):
# Normal getattr rules: look up w_name in the instance dict,
# in the class dict, and then via a call to __getatttr__.
- w_result = space.finditem(self.w_dict, w_name)
+ assert isinstance(name, str)
+ w_result = self.getdictvalue(space, name)
if w_result is not None:
return w_result
- w_result = self.getattr_from_class(space, w_name)
+ w_result = self.getattr_from_class(space, name)
if w_result is not None:
return w_result
- w_meth = self.getattr_from_class(space, space.wrap('__getattr__'))
+ w_meth = self.getattr_from_class(space, '__getattr__')
if w_meth is not None:
try:
- return space.call_function(w_meth, w_name)
+ return space.call_function(w_meth, space.wrap(name))
except OperationError, e:
if not exc and e.match(space, space.w_AttributeError):
return None # eat the AttributeError
@@ -371,7 +377,7 @@
raise operationerrfmt(
space.w_AttributeError,
"%s instance has no attribute '%s'",
- self.w_class.name, space.str_w(w_name))
+ self.w_class.name, name)
else:
return None
@@ -379,24 +385,25 @@
name = space.str_w(w_attr)
if len(name) >= 8 and name[0] == '_':
if name == "__dict__":
- return self.w_dict
+ return self.getdict()
elif name == "__class__":
return self.w_class
- return self.getattr(space, w_attr)
+ return self.getattr(space, name)
def descr_setattr(self, space, w_name, w_value):
name = unwrap_attr(space, w_name)
- w_meth = self.getattr_from_class(space, space.wrap('__setattr__'))
+ w_meth = self.getattr_from_class(space, '__setattr__')
if name and name[0] == "_":
if name == '__dict__':
self.setdict(space, w_value)
return
if name == '__class__':
- self.setclass(space, w_value)
+ self.set_oldstyle_class(space, w_value)
return
if name == '__del__' and w_meth is None:
- if (not isinstance(self, W_InstanceObjectWithDel)
- and space.finditem(self.w_dict, w_name) is None):
+ cache = space.fromcache(Cache)
+ if (not isinstance(self, cache.cls_with_del)
+ and self.getdictvalue(space, '__del__') is None):
msg = ("a __del__ method added to an instance "
"with no __del__ in the class will not be called")
space.warn(msg, space.w_RuntimeWarning)
@@ -410,13 +417,13 @@
if name and name[0] == "_":
if name == '__dict__':
# use setdict to raise the error
- self.setdict(space, None)
+ self.setdict(space, space.w_None)
return
elif name == '__class__':
- # use setclass to raise the error
- self.setclass(space, None)
+ # use set_oldstyle_class to raise the error
+ self.set_oldstyle_class(space, None)
return
- w_meth = self.getattr_from_class(space, space.wrap('__delattr__'))
+ w_meth = self.getattr_from_class(space, '__delattr__')
if w_meth is not None:
space.call_function(w_meth, w_name)
else:
@@ -427,7 +434,7 @@
self.w_class.name, name)
def descr_repr(self, space):
- w_meth = self.getattr(space, space.wrap('__repr__'), False)
+ w_meth = self.getattr(space, '__repr__', False)
if w_meth is None:
w_class = self.w_class
mod = w_class.get_module_string(space)
@@ -435,19 +442,19 @@
return space.call_function(w_meth)
def descr_str(self, space):
- w_meth = self.getattr(space, space.wrap('__str__'), False)
+ w_meth = self.getattr(space, '__str__', False)
if w_meth is None:
return self.descr_repr(space)
return space.call_function(w_meth)
def descr_unicode(self, space):
- w_meth = self.getattr(space, space.wrap('__unicode__'), False)
+ w_meth = self.getattr(space, '__unicode__', False)
if w_meth is None:
return self.descr_str(space)
return space.call_function(w_meth)
def descr_len(self, space):
- w_meth = self.getattr(space, space.wrap('__len__'))
+ w_meth = self.getattr(space, '__len__')
w_result = space.call_function(w_meth)
if space.is_true(space.isinstance(w_result, space.w_int)):
if space.is_true(space.lt(w_result, space.wrap(0))):
@@ -460,22 +467,22 @@
space.wrap("__len__() should return an int"))
def descr_getitem(self, space, w_key):
- w_meth = self.getattr(space, space.wrap('__getitem__'))
+ w_meth = self.getattr(space, '__getitem__')
return space.call_function(w_meth, w_key)
def descr_setitem(self, space, w_key, w_value):
- w_meth = self.getattr(space, space.wrap('__setitem__'))
+ w_meth = self.getattr(space, '__setitem__')
space.call_function(w_meth, w_key, w_value)
def descr_delitem(self, space, w_key):
- w_meth = self.getattr(space, space.wrap('__delitem__'))
+ w_meth = self.getattr(space, '__delitem__')
space.call_function(w_meth, w_key)
def descr_iter(self, space):
- w_meth = self.getattr(space, space.wrap('__iter__'), False)
+ w_meth = self.getattr(space, '__iter__', False)
if w_meth is not None:
return space.call_function(w_meth)
- w_meth = self.getattr(space, space.wrap('__getitem__'), False)
+ w_meth = self.getattr(space, '__getitem__', False)
if w_meth is None:
raise OperationError(
space.w_TypeError,
@@ -485,14 +492,14 @@
# don't see the point
def descr_getslice(self, space, w_i, w_j):
- w_meth = self.getattr(space, space.wrap('__getslice__'), False)
+ w_meth = self.getattr(space, '__getslice__', False)
if w_meth is not None:
return space.call_function(w_meth, w_i, w_j)
else:
return space.getitem(self, space.newslice(w_i, w_j, space.w_None))
def descr_setslice(self, space, w_i, w_j, w_sequence):
- w_meth = self.getattr(space, space.wrap('__setslice__'), False)
+ w_meth = self.getattr(space, '__setslice__', False)
if w_meth is not None:
space.call_function(w_meth, w_i, w_j, w_sequence)
else:
@@ -500,20 +507,20 @@
w_sequence)
def descr_delslice(self, space, w_i, w_j):
- w_meth = self.getattr(space, space.wrap('__delslice__'), False)
+ w_meth = self.getattr(space, '__delslice__', False)
if w_meth is not None:
space.call_function(w_meth, w_i, w_j)
else:
return space.delitem(self, space.newslice(w_i, w_j, space.w_None))
def descr_call(self, space, __args__):
- w_meth = self.getattr(space, space.wrap('__call__'))
+ w_meth = self.getattr(space, '__call__')
return space.call_args(w_meth, __args__)
def descr_nonzero(self, space):
- w_func = self.getattr(space, space.wrap('__nonzero__'), False)
+ w_func = self.getattr(space, '__nonzero__', False)
if w_func is None:
- w_func = self.getattr(space, space.wrap('__len__'), False)
+ w_func = self.getattr(space, '__len__', False)
if w_func is None:
return space.w_True
w_result = space.call_function(w_func)
@@ -537,7 +544,7 @@
not isinstance(w_b, W_InstanceObject)):
return space.cmp(w_a, w_b)
if isinstance(w_a, W_InstanceObject):
- w_func = w_a.getattr(space, space.wrap('__cmp__'), False)
+ w_func = w_a.getattr(space, '__cmp__', False)
if w_func is not None:
w_res = space.call_function(w_func, w_b)
if space.is_w(w_res, space.w_NotImplemented):
@@ -556,7 +563,7 @@
return space.wrap(-1)
return space.wrap(0)
if isinstance(w_b, W_InstanceObject):
- w_func = w_b.getattr(space, space.wrap('__cmp__'), False)
+ w_func = w_b.getattr(space, '__cmp__', False)
if w_func is not None:
w_res = space.call_function(w_func, w_a)
if space.is_w(w_res, space.w_NotImplemented):
@@ -577,10 +584,10 @@
return space.w_NotImplemented
def descr_hash(self, space):
- w_func = self.getattr(space, space.wrap('__hash__'), False)
+ w_func = self.getattr(space, '__hash__', False)
if w_func is None:
- w_eq = self.getattr(space, space.wrap('__eq__'), False)
- w_cmp = self.getattr(space, space.wrap('__cmp__'), False)
+ w_eq = self.getattr(space, '__eq__', False)
+ w_cmp = self.getattr(space, '__cmp__', False)
if w_eq is not None or w_cmp is not None:
raise OperationError(space.w_TypeError,
space.wrap("unhashable instance"))
@@ -595,7 +602,7 @@
return w_ret
def descr_index(self, space):
- w_func = self.getattr(space, space.wrap('__index__'), False)
+ w_func = self.getattr(space, '__index__', False)
if w_func is not None:
return space.call_function(w_func)
raise OperationError(
@@ -603,7 +610,7 @@
space.wrap("object cannot be interpreted as an index"))
def descr_contains(self, space, w_obj):
- w_func = self.getattr(space, space.wrap('__contains__'), False)
+ w_func = self.getattr(space, '__contains__', False)
if w_func is not None:
return space.wrap(space.is_true(space.call_function(w_func, w_obj)))
# now do it ourselves
@@ -626,7 +633,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_func = self.getattr(space, space.wrap('__pow__'), False)
+ w_func = self.getattr(space, '__pow__', False)
if w_func is not None:
return space.call_function(w_func, w_other)
return space.w_NotImplemented
@@ -634,7 +641,7 @@
return space.pow(w_a, w_b, space.w_None)
else:
# CPython also doesn't try coercion in this case
- w_func = self.getattr(space, space.wrap('__pow__'), False)
+ w_func = self.getattr(space, '__pow__', False)
if w_func is not None:
return space.call_function(w_func, w_other, w_modulo)
return space.w_NotImplemented
@@ -646,7 +653,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_func = self.getattr(space, space.wrap('__rpow__'), False)
+ w_func = self.getattr(space, '__rpow__', False)
if w_func is not None:
return space.call_function(w_func, w_other)
return space.w_NotImplemented
@@ -654,13 +661,13 @@
return space.pow(w_b, w_a, space.w_None)
else:
# CPython also doesn't try coercion in this case
- w_func = self.getattr(space, space.wrap('__rpow__'), False)
+ w_func = self.getattr(space, '__rpow__', False)
if w_func is not None:
return space.call_function(w_func, w_other, w_modulo)
return space.w_NotImplemented
def descr_next(self, space):
- w_func = self.getattr(space, space.wrap('next'), False)
+ w_func = self.getattr(space, 'next', False)
if w_func is None:
raise OperationError(space.w_TypeError,
space.wrap("instance has no next() method"))
@@ -669,10 +676,9 @@
def descr_del(self, space):
# Note that this is called from executioncontext.UserDelAction
# via the space.userdel() method.
- w_name = space.wrap('__del__')
- w_func = space.finditem(self.w_dict, w_name)
+ w_func = self.getdictvalue(space, '__del__')
if w_func is None:
- w_func = self.getattr_from_class(space, w_name)
+ w_func = self.getattr_from_class(space, '__del__')
if w_func is not None:
space.call_function(w_func)
@@ -717,6 +723,14 @@
rmeth,
unwrap_spec=["self", ObjSpace, W_Root])
+
+def descr_del_dict(space, w_inst):
+ # use setdict to raise the error
+ w_inst.setdict(space, space.w_None)
+
+dict_descr = GetSetProperty(descr_get_dict, descr_set_dict, descr_del_dict)
+dict_descr.name = '__dict__'
+
W_InstanceObject.typedef = TypeDef("instance",
__new__ = interp2app(descr_instance_new),
__getattribute__ = interp2app(W_InstanceObject.descr_getattribute,
@@ -766,12 +780,9 @@
unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
next = interp2app(W_InstanceObject.descr_next,
unwrap_spec=['self', ObjSpace]),
- __weakref__ = make_weakref_descr(W_InstanceObject),
__del__ = interp2app(W_InstanceObject.descr_del,
unwrap_spec=['self', ObjSpace]),
+ __dict__ = dict_descr,
**rawdict
)
-
-class W_InstanceObjectWithDel(W_InstanceObject):
- def __del__(self):
- self._enqueue_for_destruction(self.space)
+W_InstanceObject.typedef.acceptable_as_base_class = False
Modified: pypy/branch/arm-backend/pypy/module/__builtin__/test/test_classobj.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/__builtin__/test/test_classobj.py (original)
+++ pypy/branch/arm-backend/pypy/module/__builtin__/test/test_classobj.py Tue Nov 23 10:12:47 2010
@@ -928,29 +928,33 @@
assert x is b
assert y == 5
-
-class AppTestOldStyleSharing(AppTestOldstyle):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
- if option.runappdirect:
- py.test.skip("can only be run on py.py")
- def is_sharing(space, w_inst):
- from pypy.objspace.std.sharingdict import SharedDictImplementation
- w_d = w_inst.getdict()
- return space.wrap(isinstance(w_d, SharedDictImplementation) and w_d.r_dict_content is None)
- cls.w_is_sharing = cls.space.wrap(gateway.interp2app(is_sharing))
-
-
- def test_real_sharing(self):
+ def test_cant_subclass_instance(self):
class A:
- def __init__(self):
- self.x = 42
- A1, A2, A3 = A(), A(), A()
- assert self.is_sharing(A3)
- assert self.is_sharing(A2)
- assert self.is_sharing(A1)
+ pass
+ try:
+ class B(type(A())):
+ pass
+ except TypeError:
+ pass
+ else:
+ assert 0, "should have raised"
+
+ def test_dict_descriptor(self):
+ import sys
+ if not hasattr(sys, 'pypy_objspaceclass'):
+ skip("on CPython old-style instances don't have a __dict__ descriptor")
+ class A:
+ pass
+ a = A()
+ a.x = 1
+ descr = type(a).__dict__['__dict__']
+ assert descr.__get__(a) == {'x': 1}
+ descr.__set__(a, {'x': 2})
+ assert a.x == 2
+ raises(TypeError, descr.__delete__, a)
+
-class AppTestOldStyleModDict(object):
+class AppTestOldStyleClassStrDict(object):
def setup_class(cls):
if option.runappdirect:
py.test.skip("can only be run on py.py")
@@ -966,3 +970,22 @@
a = 1
b = 2
assert self.is_strdict(A)
+
+class AppTestOldStyleMapDict(AppTestOldstyle):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withmapdict": True})
+ if option.runappdirect:
+ py.test.skip("can only be run on py.py")
+ def has_mapdict(space, w_inst):
+ return space.wrap(w_inst._get_mapdict_map() is not None)
+ cls.w_has_mapdict = cls.space.wrap(gateway.interp2app(has_mapdict))
+
+
+ def test_has_mapdict(self):
+ class A:
+ def __init__(self):
+ self.x = 42
+ a = A()
+ assert a.x == 42
+ assert self.has_mapdict(a)
+
Modified: pypy/branch/arm-backend/pypy/module/__pypy__/__init__.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/__pypy__/__init__.py (original)
+++ pypy/branch/arm-backend/pypy/module/__pypy__/__init__.py Tue Nov 23 10:12:47 2010
@@ -23,5 +23,12 @@
'interp_magic.method_cache_counter')
self.extra_interpdef('reset_method_cache_counter',
'interp_magic.reset_method_cache_counter')
+ if self.space.config.objspace.std.withmapdict:
+ self.extra_interpdef('mapdict_cache_counter',
+ 'interp_magic.mapdict_cache_counter')
PYC_MAGIC = get_pyc_magic(self.space)
self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC)
+ #
+ from pypy.jit.backend import detect_cpu
+ model = detect_cpu.autodetect_main_model_and_size()
+ self.extra_interpdef('cpumodel', 'space.wrap(%r)' % model)
Modified: pypy/branch/arm-backend/pypy/module/__pypy__/interp_magic.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/__pypy__/interp_magic.py (original)
+++ pypy/branch/arm-backend/pypy/module/__pypy__/interp_magic.py Tue Nov 23 10:12:47 2010
@@ -2,6 +2,7 @@
from pypy.interpreter.gateway import ObjSpace
from pypy.rlib.objectmodel import we_are_translated
from pypy.objspace.std.typeobject import MethodCache
+from pypy.objspace.std.mapdict import IndexCache
def internal_repr(space, w_object):
return space.wrap('%r' % (w_object,))
@@ -36,4 +37,17 @@
cache = space.fromcache(MethodCache)
cache.misses = {}
cache.hits = {}
-
+ if space.config.objspace.std.withmapdict:
+ cache = space.fromcache(IndexCache)
+ cache.misses = {}
+ cache.hits = {}
+
+def mapdict_cache_counter(space, name):
+ """Return a tuple (index_cache_hits, index_cache_misses) for lookups
+ in the mapdict cache with the given attribute name."""
+ assert space.config.objspace.std.withmethodcachecounter
+ assert space.config.objspace.std.withmapdict
+ cache = space.fromcache(IndexCache)
+ return space.newtuple([space.newint(cache.hits.get(name, 0)),
+ space.newint(cache.misses.get(name, 0))])
+mapdict_cache_counter.unwrap_spec = [ObjSpace, str]
Modified: pypy/branch/arm-backend/pypy/module/__pypy__/test/test_special.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/__pypy__/test/test_special.py (original)
+++ pypy/branch/arm-backend/pypy/module/__pypy__/test/test_special.py Tue Nov 23 10:12:47 2010
@@ -17,3 +17,7 @@
from __pypy__ import isfake
import select
assert isfake(select)
+
+ def test_cpumodel(self):
+ import __pypy__
+ assert hasattr(__pypy__, 'cpumodel')
Modified: pypy/branch/arm-backend/pypy/module/_rawffi/__init__.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_rawffi/__init__.py (original)
+++ pypy/branch/arm-backend/pypy/module/_rawffi/__init__.py Tue Nov 23 10:12:47 2010
@@ -1,5 +1,5 @@
-""" Low-level interface to libffi
+""" Low-level interface to clibffi
"""
from pypy.interpreter.mixedmodule import MixedModule
@@ -39,11 +39,11 @@
if hasattr(interp_rawffi, 'check_HRESULT'):
Module.interpleveldefs['check_HRESULT'] = 'interp_rawffi.check_HRESULT'
- from pypy.rlib import libffi
+ from pypy.rlib import clibffi
for name in ['FUNCFLAG_STDCALL', 'FUNCFLAG_CDECL', 'FUNCFLAG_PYTHONAPI',
]:
- if hasattr(libffi, name):
- Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(libffi, name)
+ if hasattr(clibffi, name):
+ Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(clibffi, name)
super(Module, cls).buildloaders()
buildloaders = classmethod(buildloaders)
Modified: pypy/branch/arm-backend/pypy/module/_rawffi/array.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_rawffi/array.py (original)
+++ pypy/branch/arm-backend/pypy/module/_rawffi/array.py Tue Nov 23 10:12:47 2010
@@ -97,7 +97,15 @@
class W_ArrayInstance(W_DataInstance):
def __init__(self, space, shape, length, address=r_uint(0)):
- W_DataInstance.__init__(self, space, shape.size * length, address)
+ # Workaround for a strange behavior of libffi: make sure that
+ # we always have at least 8 bytes. For W_ArrayInstances that are
+ # used as the result value of a function call, ffi_call() writes
+ # 8 bytes into it even if the function's result type asks for less.
+ # This strange behavior is documented.
+ memsize = shape.size * length
+ if memsize < 8:
+ memsize = 8
+ W_DataInstance.__init__(self, space, memsize, address)
self.length = length
self.shape = shape
Modified: pypy/branch/arm-backend/pypy/module/_rawffi/callback.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_rawffi/callback.py (original)
+++ pypy/branch/arm-backend/pypy/module/_rawffi/callback.py Tue Nov 23 10:12:47 2010
@@ -8,8 +8,8 @@
from pypy.module._rawffi.array import get_elem, push_elem
from pypy.module._rawffi.interp_rawffi import W_DataInstance, letter2tp, \
wrap_value, unwrap_value, unwrap_truncate_int
-from pypy.rlib.libffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
-from pypy.rlib.libffi import ffi_type_void
+from pypy.rlib.clibffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
+from pypy.rlib.clibffi import ffi_type_void
from pypy.module._rawffi.tracker import tracker
from pypy.interpreter.error import OperationError
from pypy.interpreter import gateway
Modified: pypy/branch/arm-backend/pypy/module/_rawffi/interp_rawffi.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_rawffi/interp_rawffi.py (original)
+++ pypy/branch/arm-backend/pypy/module/_rawffi/interp_rawffi.py Tue Nov 23 10:12:47 2010
@@ -5,7 +5,7 @@
from pypy.interpreter.gateway import interp2app, NoneNotWrapped
from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.rlib.libffi import *
+from pypy.rlib.clibffi import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.rlib.unroll import unrolling_iterable
Modified: pypy/branch/arm-backend/pypy/module/_rawffi/structure.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_rawffi/structure.py (original)
+++ pypy/branch/arm-backend/pypy/module/_rawffi/structure.py Tue Nov 23 10:12:47 2010
@@ -15,7 +15,7 @@
from pypy.module._rawffi.interp_rawffi import wrap_value, unwrap_value
from pypy.module._rawffi.interp_rawffi import unpack_shape_with_length
from pypy.module._rawffi.interp_rawffi import size_alignment
-from pypy.rlib import libffi
+from pypy.rlib import clibffi
from pypy.rlib.rarithmetic import intmask, r_uint
def unpack_fields(space, w_fields):
@@ -34,7 +34,7 @@
def round_up(size, alignment):
return (size + alignment - 1) & -alignment
-def size_alignment_pos(fields):
+def size_alignment_pos(fields, is_union=False):
size = 0
alignment = 1
pos = []
@@ -42,16 +42,20 @@
# fieldtype is a W_Array
fieldsize = fieldtype.size
fieldalignment = fieldtype.alignment
- size = round_up(size, fieldalignment)
alignment = max(alignment, fieldalignment)
- pos.append(size)
- size += intmask(fieldsize)
+ if is_union:
+ pos.append(0)
+ size = max(size, fieldsize)
+ else:
+ size = round_up(size, fieldalignment)
+ pos.append(size)
+ size += intmask(fieldsize)
size = round_up(size, alignment)
return size, alignment, pos
class W_Structure(W_DataShape):
- def __init__(self, space, fields, size, alignment):
+ def __init__(self, space, fields, size, alignment, is_union=False):
name_to_index = {}
if fields is not None:
for i in range(len(fields)):
@@ -60,7 +64,7 @@
raise operationerrfmt(space.w_ValueError,
"duplicate field name %s", name)
name_to_index[name] = i
- size, alignment, pos = size_alignment_pos(fields)
+ size, alignment, pos = size_alignment_pos(fields, is_union)
else: # opaque case
fields = []
pos = []
@@ -104,14 +108,14 @@
descr_fieldoffset.unwrap_spec = ['self', ObjSpace, str]
# get the corresponding ffi_type
- ffi_struct = lltype.nullptr(libffi.FFI_STRUCT_P.TO)
+ ffi_struct = lltype.nullptr(clibffi.FFI_STRUCT_P.TO)
def get_basic_ffi_type(self):
if not self.ffi_struct:
# Repeated fields are delicate. Consider for example
# struct { int a[5]; }
# or struct { struct {int x;} a[5]; }
- # Seeing no corresponding doc in libffi, let's just repeat
+ # Seeing no corresponding doc in clibffi, let's just repeat
# the field 5 times...
fieldtypes = []
for name, tp in self.fields:
@@ -122,7 +126,7 @@
while count + basic_size <= total_size:
fieldtypes.append(basic_ffi_type)
count += basic_size
- self.ffi_struct = libffi.make_struct_ffitype_e(self.size,
+ self.ffi_struct = clibffi.make_struct_ffitype_e(self.size,
self.alignment,
fieldtypes)
return self.ffi_struct.ffistruct
@@ -133,15 +137,17 @@
-def descr_new_structure(space, w_type, w_shapeinfo):
+def descr_new_structure(space, w_type, w_shapeinfo, union=0):
+ is_union = bool(union)
if space.is_true(space.isinstance(w_shapeinfo, space.w_tuple)):
w_size, w_alignment = space.fixedview(w_shapeinfo, expected_length=2)
S = W_Structure(space, None, space.int_w(w_size),
- space.int_w(w_alignment))
+ space.int_w(w_alignment), is_union)
else:
fields = unpack_fields(space, w_shapeinfo)
- S = W_Structure(space, fields, 0, 0)
+ S = W_Structure(space, fields, 0, 0, is_union)
return space.wrap(S)
+descr_new_structure.unwrap_spec = [ObjSpace, W_Root, W_Root, int]
W_Structure.typedef = TypeDef(
'Structure',
Modified: pypy/branch/arm-backend/pypy/module/_rawffi/test/test__rawffi.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_rawffi/test/test__rawffi.py (original)
+++ pypy/branch/arm-backend/pypy/module/_rawffi/test/test__rawffi.py Tue Nov 23 10:12:47 2010
@@ -70,7 +70,7 @@
return s[num];
}
- char *char_check(char x, char y)
+ const char *char_check(char x, char y)
{
if (y == static_str[0])
return static_str;
@@ -185,13 +185,14 @@
sum_x_y
give perturb get_s2a check_s2a
AAA_first_ordinal_function
+ ret_un_func
""".split()
eci = ExternalCompilationInfo(export_symbols=symbols)
return str(platform.compile([c_file], eci, 'x', standalone=False))
prepare_c_example = staticmethod(prepare_c_example)
def setup_class(cls):
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
space = gettestobjspace(usemodules=('_rawffi', 'struct'))
cls.space = space
cls.w_lib_name = space.wrap(cls.prepare_c_example())
@@ -295,6 +296,7 @@
assert _rawffi.charp2string(res[0]) is None
arg1.free()
arg2.free()
+ a.free()
def test_raw_callable(self):
import _rawffi
@@ -945,14 +947,15 @@
assert a[4] == 't'
def test_union(self):
- skip("segfaulting")
import _rawffi
longsize = _rawffi.sizeof('l')
- S = _rawffi.Structure((longsize, longsize))
+ S = _rawffi.Structure([('x', 'h'), ('y', 'l')], union=True)
s = S(autofree=False)
+ s.x = 12345
lib = _rawffi.CDLL(self.lib_name)
f = lib.ptr('ret_un_func', [(S, 1)], (S, 1))
ret = f(s)
+ assert ret.y == 1234500, "ret.y == %d" % (ret.y,)
s.free()
class AppTestAutoFree:
Modified: pypy/branch/arm-backend/pypy/module/_rawffi/test/test_nested.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_rawffi/test/test_nested.py (original)
+++ pypy/branch/arm-backend/pypy/module/_rawffi/test/test_nested.py Tue Nov 23 10:12:47 2010
@@ -107,7 +107,6 @@
assert S.fieldoffset('x') == 0
assert S.fieldoffset('ar') == A5alignment
s = S()
- s = S()
s.x = 'G'
raises(TypeError, 's.ar')
assert s.fieldaddress('ar') == s.buffer + S.fieldoffset('ar')
Modified: pypy/branch/arm-backend/pypy/module/_socket/test/test_sock_app.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_socket/test/test_sock_app.py (original)
+++ pypy/branch/arm-backend/pypy/module/_socket/test/test_sock_app.py Tue Nov 23 10:12:47 2010
@@ -254,6 +254,7 @@
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
fd = s.fileno()
w_obj = rsocket.make_address(c_addr, addrlen).as_object(fd, space)
+ lltype.free(c_addr_ll, flavor='raw')
assert space.is_true(space.eq(w_obj, space.newtuple([
space.wrap('lo'),
space.wrap(socket.ntohs(8)),
Modified: pypy/branch/arm-backend/pypy/module/_sre/interp_sre.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_sre/interp_sre.py (original)
+++ pypy/branch/arm-backend/pypy/module/_sre/interp_sre.py Tue Nov 23 10:12:47 2010
@@ -290,10 +290,8 @@
def SRE_Pattern__new__(space, w_subtype, w_pattern, flags, w_code,
groups=0, w_groupindex=None, w_indexgroup=None):
n = space.int_w(space.len(w_code))
- code = [0] * n
- for i in range(n):
- x = space.uint_w(space.getitem(w_code, space.wrap(i)))
- code[i] = intmask(x)
+ code = [intmask(space.uint_w(space.getitem(w_code, space.wrap(i))))
+ for i in range(n)]
#
w_srepat = space.allocate_instance(W_SRE_Pattern, w_subtype)
srepat = space.interp_w(W_SRE_Pattern, w_srepat)
Modified: pypy/branch/arm-backend/pypy/module/_weakref/interp__weakref.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_weakref/interp__weakref.py (original)
+++ pypy/branch/arm-backend/pypy/module/_weakref/interp__weakref.py Tue Nov 23 10:12:47 2010
@@ -7,7 +7,7 @@
import weakref
-class WeakrefLifeline(object):
+class WeakrefLifeline(W_Root):
def __init__(self, space):
self.space = space # this is here for W_Root.clear_all_weakrefs()
self.refs_weak = []
Modified: pypy/branch/arm-backend/pypy/module/_winreg/interp_winreg.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/_winreg/interp_winreg.py (original)
+++ pypy/branch/arm-backend/pypy/module/_winreg/interp_winreg.py Tue Nov 23 10:12:47 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
from pypy.interpreter.baseobjspace import Wrappable
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.gateway import interp2app
@@ -212,13 +213,10 @@
subkey = None
else:
subkey = space.str_w(w_subkey)
- dataptr = rffi.str2charp(value)
- try:
+ with rffi.scoped_str2charp(value) as dataptr:
ret = rwinreg.RegSetValue(hkey, subkey, rwinreg.REG_SZ, dataptr, len(value))
- finally:
- rffi.free_charp(dataptr)
- if ret != 0:
- raiseWindowsError(space, ret, 'RegSetValue')
+ if ret != 0:
+ raiseWindowsError(space, ret, 'RegSetValue')
SetValue.unwrap_spec = [ObjSpace, W_Root, W_Root, int, str]
def QueryValue(space, w_hkey, w_subkey):
@@ -237,23 +235,15 @@
subkey = None
else:
subkey = space.str_w(w_subkey)
- bufsize_p = lltype.malloc(rwin32.PLONG.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.PLONG.TO, 1) as bufsize_p:
ret = rwinreg.RegQueryValue(hkey, subkey, None, bufsize_p)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValue')
- buf = lltype.malloc(rffi.CCHARP.TO, bufsize_p[0], flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, bufsize_p[0]) as buf:
ret = rwinreg.RegQueryValue(hkey, subkey, buf, bufsize_p)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValue')
return space.wrap(rffi.charp2strn(buf, bufsize_p[0] - 1))
- finally:
- lltype.free(buf, flavor='raw')
- finally:
- lltype.free(bufsize_p, flavor='raw')
- if ret != 0:
- raiseWindowsError(space, ret, 'RegQueryValue')
QueryValue.unwrap_spec = [ObjSpace, W_Root, W_Root]
def convert_to_regdata(space, w_value, typ):
@@ -412,16 +402,14 @@
value_name is a string indicating the value to query"""
hkey = hkey_w(w_hkey, space)
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
- retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize:
ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, null_dword,
None, retDataSize)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValueEx')
- databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0], flavor='raw')
- try:
- retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType:
ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword,
retType, databuf, retDataSize)
@@ -432,12 +420,6 @@
retDataSize[0], retType[0]),
space.wrap(retType[0]),
])
- finally:
- lltype.free(retType, flavor='raw')
- finally:
- lltype.free(databuf, flavor='raw')
- finally:
- lltype.free(retDataSize, flavor='raw')
QueryValueEx.unwrap_spec = [ObjSpace, W_Root, str]
@@ -454,14 +436,11 @@
The return value is the handle of the opened key.
If the function fails, an exception is raised."""
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegCreateKey(hkey, subkey, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'CreateKey')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
CreateKey.unwrap_spec = [ObjSpace, W_Root, str]
def DeleteKey(space, w_hkey, subkey):
@@ -504,14 +483,11 @@
The result is a new handle to the specified key
If the function fails, an EnvironmentError exception is raised."""
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegOpenKeyEx(hkey, subkey, res, sam, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'RegOpenKeyEx')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
OpenKey.unwrap_spec = [ObjSpace, W_Root, str, int, rffi.r_uint]
def EnumValue(space, w_hkey, index):
@@ -531,10 +507,8 @@
hkey = hkey_w(w_hkey, space)
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
- retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize:
ret = rwinreg.RegQueryInfoKey(
hkey, None, null_dword, null_dword,
null_dword, null_dword, null_dword,
@@ -546,14 +520,9 @@
retValueSize[0] += 1
retDataSize[0] += 1
- valuebuf = lltype.malloc(rffi.CCHARP.TO, retValueSize[0],
- flavor='raw')
- try:
- databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0],
- flavor='raw')
- try:
- retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retValueSize[0]) as valuebuf:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType:
ret = rwinreg.RegEnumValue(
hkey, index, valuebuf, retValueSize,
null_dword, retType, databuf, retDataSize)
@@ -566,16 +535,6 @@
retDataSize[0], retType[0]),
space.wrap(retType[0]),
])
- finally:
- lltype.free(retType, flavor='raw')
- finally:
- lltype.free(databuf, flavor='raw')
- finally:
- lltype.free(valuebuf, flavor='raw')
- finally:
- lltype.free(retDataSize, flavor='raw')
- finally:
- lltype.free(retValueSize, flavor='raw')
EnumValue.unwrap_spec = [ObjSpace, W_Root, int]
@@ -592,10 +551,8 @@
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
# max key name length is 255
- buf = lltype.malloc(rffi.CCHARP.TO, 256, flavor='raw')
- try:
- retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, 256) as buf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize:
retValueSize[0] = 256 # includes NULL terminator
ret = rwinreg.RegEnumKeyEx(hkey, index, buf, retValueSize,
null_dword, None, null_dword,
@@ -603,10 +560,6 @@
if ret != 0:
raiseWindowsError(space, ret, 'RegEnumKeyEx')
return space.wrap(rffi.charp2str(buf))
- finally:
- lltype.free(retValueSize, flavor='raw')
- finally:
- lltype.free(buf, flavor='raw')
EnumKey.unwrap_spec = [ObjSpace, W_Root, int]
@@ -621,12 +574,9 @@
A long integer that identifies when the key was last modified (if available)
as 100's of nanoseconds since Jan 1, 1600."""
hkey = hkey_w(w_hkey, space)
- nSubKeys = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- nValues = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- ft = lltype.malloc(rwin32.PFILETIME.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nSubKeys:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nValues:
+ with lltype.scoped_alloc(rwin32.PFILETIME.TO, 1) as ft:
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
ret = rwinreg.RegQueryInfoKey(
hkey, None, null_dword, null_dword,
@@ -640,12 +590,6 @@
return space.newtuple([space.wrap(nSubKeys[0]),
space.wrap(nValues[0]),
space.wrap(l)])
- finally:
- lltype.free(ft, flavor='raw')
- finally:
- lltype.free(nValues, flavor='raw')
- finally:
- lltype.free(nSubKeys, flavor='raw')
QueryInfoKey.unwrap_spec = [ObjSpace, W_Root]
def str_or_None_w(space, w_obj):
@@ -666,12 +610,9 @@
If the function fails, an EnvironmentError exception is raised."""
machine = str_or_None_w(space, w_machine)
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegConnectRegistry(machine, hkey, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'RegConnectRegistry')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
ConnectRegistry.unwrap_spec = [ObjSpace, W_Root, W_Root]
Modified: pypy/branch/arm-backend/pypy/module/array/interp_array.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/array/interp_array.py (original)
+++ pypy/branch/arm-backend/pypy/module/array/interp_array.py Tue Nov 23 10:12:47 2010
@@ -27,7 +27,7 @@
typecode = typecode[0]
if space.is_w(w_cls, space.gettypeobject(W_ArrayBase.typedef)):
- if len(w_args.keywords_w) > 0:
+ if w_args.keywords: # XXX this might be forbidden fishing
msg = 'array.array() does not take keyword arguments'
raise OperationError(space.w_TypeError, space.wrap(msg))
@@ -192,32 +192,30 @@
mytype.bytes
raise OperationError(space.w_OverflowError,
space.wrap(msg))
- elif mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w':
+ return rffi.cast(mytype.itemtype, item)
+ if mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w':
if len(item) != 1:
msg = 'array item must be char'
raise OperationError(space.w_TypeError, space.wrap(msg))
item = item[0]
-
+ return rffi.cast(mytype.itemtype, item)
+ #
+ # "regular" case: it fits in an rpython integer (lltype.Signed)
+ result = rffi.cast(mytype.itemtype, item)
if mytype.canoverflow:
- msg = None
- if mytype.signed:
- if item < -1 << (mytype.bytes * 8 - 1):
+ if rffi.cast(lltype.Signed, result) != item:
+ # overflow. build the correct message
+ if item < 0:
msg = ('signed %d-byte integer is less than minimum' %
mytype.bytes)
- elif item > (1 << (mytype.bytes * 8 - 1)) - 1:
+ else:
msg = ('signed %d-byte integer is greater than maximum'
% mytype.bytes)
- else:
- if item < 0:
- msg = ('unsigned %d-byte integer is less than minimum'
- % mytype.bytes)
- elif item > (1 << (mytype.bytes * 8)) - 1:
- msg = ('unsigned %d-byte integer is greater'
- ' than maximum' % mytype.bytes)
- if msg is not None:
+ if not mytype.signed:
+ msg = 'un' + msg # 'signed' => 'unsigned'
raise OperationError(space.w_OverflowError,
space.wrap(msg))
- return rffi.cast(mytype.itemtype, item)
+ return result
def __del__(self):
self.setlen(0)
Modified: pypy/branch/arm-backend/pypy/module/bz2/interp_bz2.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/bz2/interp_bz2.py (original)
+++ pypy/branch/arm-backend/pypy/module/bz2/interp_bz2.py Tue Nov 23 10:12:47 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
from pypy.rpython.tool import rffi_platform as platform
from pypy.rpython.lltypesystem import rffi
from pypy.rpython.lltypesystem import lltype
@@ -225,6 +226,11 @@
if self.current_size > 0:
rffi.keep_buffer_alive_until_here(self.raw_buf, self.gc_buf)
+ def __enter__(self):
+ return self
+ def __exit__(self, *args):
+ self.free()
+
# ____________________________________________________________
#
# Make the BZ2File type by internally inheriting from W_File.
@@ -531,33 +537,30 @@
if not self.running:
raise OperationError(self.space.w_ValueError,
self.space.wrap("this object was already flushed"))
-
- out = OutBuffer(self.bzs)
+
in_bufsize = datasize
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(datasize):
- in_buf[i] = data[i]
- try:
-
- self.bzs.c_next_in = in_buf
- rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
+ with OutBuffer(self.bzs) as out:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
- while True:
- bzerror = BZ2_bzCompress(self.bzs, BZ_RUN)
- if bzerror != BZ_RUN_OK:
- _catch_bz2_error(self.space, bzerror)
+ for i in range(datasize):
+ in_buf[i] = data[i]
- if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
- break
- elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
+ self.bzs.c_next_in = in_buf
+ rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
- res = out.make_result_string()
- return self.space.wrap(res)
- finally:
- lltype.free(in_buf, flavor='raw')
- out.free()
+ while True:
+ bzerror = BZ2_bzCompress(self.bzs, BZ_RUN)
+ if bzerror != BZ_RUN_OK:
+ _catch_bz2_error(self.space, bzerror)
+
+ if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
+ break
+ elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
+ return self.space.wrap(res)
compress.unwrap_spec = ['self', 'bufferstr']
@@ -566,9 +569,8 @@
raise OperationError(self.space.w_ValueError,
self.space.wrap("this object was already flushed"))
self.running = False
-
- out = OutBuffer(self.bzs)
- try:
+
+ with OutBuffer(self.bzs) as out:
while True:
bzerror = BZ2_bzCompress(self.bzs, BZ_FINISH)
if bzerror == BZ_STREAM_END:
@@ -581,8 +583,6 @@
res = out.make_result_string()
return self.space.wrap(res)
- finally:
- out.free()
flush.unwrap_spec = ['self']
W_BZ2Compressor.typedef = TypeDef("BZ2Compressor",
@@ -641,38 +641,37 @@
if not self.running:
raise OperationError(self.space.w_EOFError,
self.space.wrap("end of stream was already found"))
-
+
in_bufsize = len(data)
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
- out = OutBuffer(self.bzs)
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
self.bzs.c_next_in = in_buf
rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
- while True:
- bzerror = BZ2_bzDecompress(self.bzs)
- if bzerror == BZ_STREAM_END:
- if rffi.getintfield(self.bzs, 'c_avail_in') != 0:
- unused = [self.bzs.c_next_in[i] for i in range(rffi.getintfield(self.bzs, 'c_avail_in'))]
- self.unused_data = "".join(unused)
- self.running = False
- break
- if bzerror != BZ_OK:
- _catch_bz2_error(self.space, bzerror)
-
- if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
- break
- elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
+ with OutBuffer(self.bzs) as out:
+ while True:
+ bzerror = BZ2_bzDecompress(self.bzs)
+ if bzerror == BZ_STREAM_END:
+ if rffi.getintfield(self.bzs, 'c_avail_in') != 0:
+ unused = [self.bzs.c_next_in[i]
+ for i in range(
+ rffi.getintfield(self.bzs,
+ 'c_avail_in'))]
+ self.unused_data = "".join(unused)
+ self.running = False
+ break
+ if bzerror != BZ_OK:
+ _catch_bz2_error(self.space, bzerror)
+
+ if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
+ break
+ elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
- res = out.make_result_string()
- return self.space.wrap(res)
- finally:
- lltype.free(in_buf, flavor='raw')
- out.free()
+ res = out.make_result_string()
+ return self.space.wrap(res)
decompress.unwrap_spec = ['self', 'bufferstr']
@@ -695,43 +694,39 @@
if compresslevel < 1 or compresslevel > 9:
raise OperationError(space.w_ValueError,
space.wrap("compresslevel must be between 1 and 9"))
-
- bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
- in_bufsize = len(data)
- # conforming to bz2 manual, this is large enough to fit compressed
- # data in one shot. We will check it later anyway.
- out = OutBuffer(bzs, in_bufsize + (in_bufsize / 100 + 1) + 600)
-
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
-
- try:
- bzs.c_next_in = in_buf
- rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
- bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0)
- if bzerror != BZ_OK:
- _catch_bz2_error(space, bzerror)
+ with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs:
+ in_bufsize = len(data)
+
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
+ bzs.c_next_in = in_buf
+ rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
+
+ # conforming to bz2 manual, this is large enough to fit compressed
+ # data in one shot. We will check it later anyway.
+ with OutBuffer(bzs,
+ in_bufsize + (in_bufsize / 100 + 1) + 600) as out:
+
+ bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0)
+ if bzerror != BZ_OK:
+ _catch_bz2_error(space, bzerror)
- while True:
- bzerror = BZ2_bzCompress(bzs, BZ_FINISH)
- if bzerror == BZ_STREAM_END:
- break
- elif bzerror != BZ_FINISH_OK:
+ while True:
+ bzerror = BZ2_bzCompress(bzs, BZ_FINISH)
+ if bzerror == BZ_STREAM_END:
+ break
+ elif bzerror != BZ_FINISH_OK:
+ BZ2_bzCompressEnd(bzs)
+ _catch_bz2_error(space, bzerror)
+
+ if rffi.getintfield(bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
BZ2_bzCompressEnd(bzs)
- _catch_bz2_error(space, bzerror)
-
- if rffi.getintfield(bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
-
- res = out.make_result_string()
- BZ2_bzCompressEnd(bzs)
- return space.wrap(res)
- finally:
- lltype.free(bzs, flavor='raw')
- lltype.free(in_buf, flavor='raw')
- out.free()
+ return space.wrap(res)
compress.unwrap_spec = [ObjSpace, 'bufferstr', int]
def decompress(space, data):
@@ -744,40 +739,34 @@
if in_bufsize == 0:
return space.wrap("")
- bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
-
- out = OutBuffer(bzs)
- try:
- bzs.c_next_in = in_buf
- rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
-
- bzerror = BZ2_bzDecompressInit(bzs, 0, 0)
- if bzerror != BZ_OK:
- _catch_bz2_error(space, bzerror)
-
- while True:
- bzerror = BZ2_bzDecompress(bzs)
- if bzerror == BZ_STREAM_END:
- break
- if bzerror != BZ_OK:
- BZ2_bzDecompressEnd(bzs)
- _catch_bz2_error(space, bzerror)
-
- if rffi.getintfield(bzs, 'c_avail_in') == 0:
+ with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
+ bzs.c_next_in = in_buf
+ rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
+
+ with OutBuffer(bzs) as out:
+ bzerror = BZ2_bzDecompressInit(bzs, 0, 0)
+ if bzerror != BZ_OK:
+ _catch_bz2_error(space, bzerror)
+
+ while True:
+ bzerror = BZ2_bzDecompress(bzs)
+ if bzerror == BZ_STREAM_END:
+ break
+ if bzerror != BZ_OK:
+ BZ2_bzDecompressEnd(bzs)
+ _catch_bz2_error(space, bzerror)
+
+ if rffi.getintfield(bzs, 'c_avail_in') == 0:
+ BZ2_bzDecompressEnd(bzs)
+ raise OperationError(space.w_ValueError, space.wrap(
+ "couldn't find end of stream"))
+ elif rffi.getintfield(bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
BZ2_bzDecompressEnd(bzs)
- raise OperationError(space.w_ValueError,
- space.wrap("couldn't find end of stream"))
- elif rffi.getintfield(bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
-
- res = out.make_result_string()
- BZ2_bzDecompressEnd(bzs)
- return space.wrap(res)
- finally:
- lltype.free(bzs, flavor='raw')
- lltype.free(in_buf, flavor='raw')
- out.free()
+ return space.wrap(res)
decompress.unwrap_spec = [ObjSpace, 'bufferstr']
Modified: pypy/branch/arm-backend/pypy/module/cpyext/api.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/cpyext/api.py (original)
+++ pypy/branch/arm-backend/pypy/module/cpyext/api.py Tue Nov 23 10:12:47 2010
@@ -226,7 +226,7 @@
def unwrapper(space, *args):
from pypy.module.cpyext.pyobject import Py_DecRef
from pypy.module.cpyext.pyobject import make_ref, from_ref
- from pypy.module.cpyext.pyobject import BorrowPair
+ from pypy.module.cpyext.pyobject import Reference
newargs = ()
to_decref = []
assert len(args) == len(api_function.argtypes)
@@ -270,8 +270,8 @@
return api_function.error_value
if res is None:
return None
- elif isinstance(res, BorrowPair):
- return res.w_borrowed
+ elif isinstance(res, Reference):
+ return res.get_wrapped(space)
else:
return res
finally:
@@ -473,7 +473,7 @@
@specialize.ll()
def wrapper(*args):
from pypy.module.cpyext.pyobject import make_ref, from_ref
- from pypy.module.cpyext.pyobject import BorrowPair
+ from pypy.module.cpyext.pyobject import Reference
# we hope that malloc removal removes the newtuple() that is
# inserted exactly here by the varargs specializer
llop.gc_stack_bottom(lltype.Void) # marker for trackgcroot.py
@@ -525,7 +525,7 @@
elif is_PyObject(callable.api_func.restype):
if result is None:
retval = make_ref(space, None)
- elif isinstance(result, BorrowPair):
+ elif isinstance(result, Reference):
retval = result.get_ref(space)
elif not rffi._isllptr(result):
retval = rffi.cast(callable.api_func.restype,
@@ -908,8 +908,10 @@
from pypy.rlib import rdynload
try:
ll_libname = rffi.str2charp(path)
- dll = rdynload.dlopen(ll_libname)
- lltype.free(ll_libname, flavor='raw')
+ try:
+ dll = rdynload.dlopen(ll_libname)
+ finally:
+ lltype.free(ll_libname, flavor='raw')
except rdynload.DLOpenError, e:
raise operationerrfmt(
space.w_ImportError,
Modified: pypy/branch/arm-backend/pypy/module/cpyext/cdatetime.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/cpyext/cdatetime.py (original)
+++ pypy/branch/arm-backend/pypy/module/cpyext/cdatetime.py Tue Nov 23 10:12:47 2010
@@ -4,7 +4,7 @@
from pypy.module.cpyext.api import (
cpython_api, CANNOT_FAIL, cpython_struct, PyObjectFields)
from pypy.module.cpyext.import_ import PyImport_Import
-from pypy.module.cpyext.typeobject import PyTypeObjectPtr
+from pypy.module.cpyext.typeobject import PyTypeObjectPtr, render_immortal
from pypy.module.cpyext.state import State
from pypy.interpreter.error import OperationError
from pypy.tool.sourcetools import func_renamer
@@ -22,25 +22,34 @@
@cpython_api([], lltype.Ptr(PyDateTime_CAPI),
error=lltype.nullptr(PyDateTime_CAPI))
def _PyDateTime_Import(space):
- datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw')
+ datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw',
+ track_allocation=False)
if not we_are_translated():
datetimeAPI_dealloc(space)
space.fromcache(State).datetimeAPI = datetimeAPI
w_datetime = PyImport_Import(space, space.wrap("datetime"))
+
w_type = space.getattr(w_datetime, space.wrap("date"))
datetimeAPI.c_DateType = rffi.cast(
PyTypeObjectPtr, make_ref(space, w_type))
+ render_immortal(datetimeAPI.c_DateType, w_type)
+
w_type = space.getattr(w_datetime, space.wrap("datetime"))
datetimeAPI.c_DateTimeType = rffi.cast(
PyTypeObjectPtr, make_ref(space, w_type))
+ render_immortal(datetimeAPI.c_DateTimeType, w_type)
+
w_type = space.getattr(w_datetime, space.wrap("time"))
datetimeAPI.c_TimeType = rffi.cast(
PyTypeObjectPtr, make_ref(space, w_type))
+ render_immortal(datetimeAPI.c_TimeType, w_type)
+
w_type = space.getattr(w_datetime, space.wrap("timedelta"))
datetimeAPI.c_DeltaType = rffi.cast(
PyTypeObjectPtr, make_ref(space, w_type))
+ render_immortal(datetimeAPI.c_DeltaType, w_type)
return datetimeAPI
Modified: pypy/branch/arm-backend/pypy/module/cpyext/classobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/cpyext/classobject.py (original)
+++ pypy/branch/arm-backend/pypy/module/cpyext/classobject.py Tue Nov 23 10:12:47 2010
@@ -15,16 +15,20 @@
class is the class of new object. The dict parameter will be used as the
object's __dict__; if NULL, a new dictionary will be created for the
instance."""
- if not PyClass_Check(space, w_class):
+ if not isinstance(w_class, W_ClassObject):
return PyErr_BadInternalCall(space)
- return W_InstanceObject(space, w_class, w_dict)
+ w_result = w_class.instantiate(space)
+ if w_dict is not None:
+ w_result.setdict(space, w_dict)
+ return w_result
@cpython_api([PyObject, PyObject], PyObject, error=CANNOT_FAIL)
def _PyInstance_Lookup(space, w_instance, w_name):
+ name = space.str_w(w_name)
assert isinstance(w_instance, W_InstanceObject)
- w_result = space.finditem(w_instance.w_dict, w_name)
+ w_result = w_instance.getdictvalue(space, name)
if w_result is not None:
return w_result
- return w_instance.w_class.lookup(space, w_name)
+ return w_instance.w_class.lookup(space, name)
Modified: pypy/branch/arm-backend/pypy/module/cpyext/presetup.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/cpyext/presetup.py (original)
+++ pypy/branch/arm-backend/pypy/module/cpyext/presetup.py Tue Nov 23 10:12:47 2010
@@ -21,6 +21,8 @@
from pypy.conftest import gettestobjspace
from pypy.module.cpyext.api import build_bridge
+from pypy.module.imp.importing import get_so_extension
+
usemodules = ['cpyext', 'thread']
if sys.platform == 'win32':
usemodules.append('_winreg') # necessary in distutils
@@ -35,6 +37,7 @@
def patch_distutils():
sysconfig.get_python_inc = get_python_inc
+ sysconfig.get_config_vars()['SO'] = get_so_extension(space)
patch_distutils()
Modified: pypy/branch/arm-backend/pypy/module/cpyext/pyobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/cpyext/pyobject.py (original)
+++ pypy/branch/arm-backend/pypy/module/cpyext/pyobject.py Tue Nov 23 10:12:47 2010
@@ -424,7 +424,18 @@
state = space.fromcache(RefcountState)
return state.make_borrowed(w_container, w_borrowed)
-class BorrowPair:
+class Reference:
+ def __init__(self, pyobj):
+ assert not isinstance(pyobj, W_Root)
+ self.pyobj = pyobj
+
+ def get_ref(self, space):
+ return self.pyobj
+
+ def get_wrapped(self, space):
+ return from_ref(space, self.pyobj)
+
+class BorrowPair(Reference):
"""
Delays the creation of a borrowed reference.
"""
@@ -435,6 +446,9 @@
def get_ref(self, space):
return make_borrowed_ref(space, self.w_container, self.w_borrowed)
+ def get_wrapped(self, space):
+ return self.w_borrowed
+
def borrow_from(container, borrowed):
return BorrowPair(container, borrowed)
Modified: pypy/branch/arm-backend/pypy/module/cpyext/test/test_borrow.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/cpyext/test/test_borrow.py (original)
+++ pypy/branch/arm-backend/pypy/module/cpyext/test/test_borrow.py Tue Nov 23 10:12:47 2010
@@ -31,6 +31,7 @@
g = PyTuple_GetItem(t, 0); // borrows reference again
printf("Refcnt4: %i\\n", f->ob_refcnt);
printf("COMPARE: %i\\n", f == g);
+ fflush(stdout);
Py_DECREF(t);
Py_RETURN_TRUE;
"""),
Modified: pypy/branch/arm-backend/pypy/module/cpyext/test/test_cpyext.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/cpyext/test/test_cpyext.py (original)
+++ pypy/branch/arm-backend/pypy/module/cpyext/test/test_cpyext.py Tue Nov 23 10:12:47 2010
@@ -16,6 +16,7 @@
from pypy.module.cpyext.pyobject import Py_DecRef, InvalidPointerException
from pypy.translator.goal import autopath
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
@api.cpython_api([], api.PyObject)
def PyPy_Crash1(space):
@@ -41,7 +42,7 @@
raises(ImportError, cpyext.load_module, "missing.file", "foo")
raises(ImportError, cpyext.load_module, self.libc, "invalid.function")
-def compile_module(modname, **kwds):
+def compile_module(space, modname, **kwds):
"""
Build an extension module and return the filename of the resulting native
code file.
@@ -64,10 +65,8 @@
[], eci,
outputfilename=str(dirname/modname),
standalone=False)
- if sys.platform == 'win32':
- pydname = soname.new(purebasename=modname, ext='.pyd')
- else:
- pydname = soname.new(purebasename=modname, ext='.so')
+ from pypy.module.imp.importing import get_so_extension
+ pydname = soname.new(purebasename=modname, ext=get_so_extension(space))
soname.rename(pydname)
return str(pydname)
@@ -78,7 +77,6 @@
self.frozen_refcounts[w_obj] = obj.c_ob_refcnt
#state.print_refcounts()
self.frozen_ll2callocations = set(ll2ctypes.ALLOCATED.values())
- lltype.start_tracking_allocations()
class LeakCheckingTest(object):
def check_and_print_leaks(self):
@@ -126,17 +124,8 @@
for w_obj in lost_objects_w:
print >>sys.stderr, "Lost object %r" % (w_obj, )
leaking = True
- for llvalue in set(ll2ctypes.ALLOCATED.values()) - self.frozen_ll2callocations:
- if getattr(llvalue, "_traceback", None): # this means that the allocation should be tracked
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (ll2ctypes)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
- for llvalue in lltype.ALLOCATED.keys():
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (llvalue)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
-
- lltype.stop_tracking_allocations()
+ # the actual low-level leak checking is done by pypy.tool.leakfinder,
+ # enabled automatically by pypy.conftest.
return leaking
class AppTestCpythonExtensionBase(LeakCheckingTest):
@@ -162,7 +151,7 @@
kwds["link_files"] = [str(api_library + '.so')]
if sys.platform == 'linux2':
kwds["compile_extra"]=["-Werror=implicit-function-declaration"]
- return compile_module(name, **kwds)
+ return compile_module(self.space, name, **kwds)
def import_module(self, name, init=None, body='', load_it=True, filename=None):
Modified: pypy/branch/arm-backend/pypy/module/cpyext/test/test_unicodeobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/cpyext/test/test_unicodeobject.py (original)
+++ pypy/branch/arm-backend/pypy/module/cpyext/test/test_unicodeobject.py Tue Nov 23 10:12:47 2010
@@ -177,13 +177,14 @@
encoded_charp = rffi.str2charp(encoded)
strict_charp = rffi.str2charp("strict")
if endian is not None:
- pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
if endian < 0:
- pendian[0] = -1
+ value = -1
elif endian > 0:
- pendian[0] = 1
+ value = 1
else:
- pendian[0] = 0
+ value = 0
+ pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
+ pendian[0] = rffi.cast(rffi.INT, value)
else:
pendian = None
Modified: pypy/branch/arm-backend/pypy/module/cpyext/typeobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/cpyext/typeobject.py (original)
+++ pypy/branch/arm-backend/pypy/module/cpyext/typeobject.py Tue Nov 23 10:12:47 2010
@@ -182,10 +182,10 @@
subtype = rffi.cast(PyTypeObjectPtr, make_ref(space, w_subtype))
try:
- obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds)
+ w_obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds)
finally:
Py_DecRef(space, w_subtype)
- return obj
+ return w_obj
@specialize.memo()
def get_new_method_def(space):
@@ -193,10 +193,14 @@
if state.new_method_def:
return state.new_method_def
from pypy.module.cpyext.modsupport import PyMethodDef
- ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True)
+ ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True,
+ immortal=True)
ptr.c_ml_name = rffi.str2charp("__new__")
+ lltype.render_immortal(ptr.c_ml_name)
rffi.setintfield(ptr, 'c_ml_flags', METH_VARARGS | METH_KEYWORDS)
- ptr.c_ml_doc = rffi.str2charp("T.__new__(S, ...) -> a new object with type S, a subtype of T")
+ ptr.c_ml_doc = rffi.str2charp(
+ "T.__new__(S, ...) -> a new object with type S, a subtype of T")
+ lltype.render_immortal(ptr.c_ml_doc)
state.new_method_def = ptr
return ptr
@@ -429,6 +433,12 @@
finish_type_1(space, pto)
finish_type_2(space, pto, w_type)
+ if space.type(w_type).is_cpytype():
+ # XXX Types with a C metatype are never freed, try to see why...
+ render_immortal(pto, w_type)
+ lltype.render_immortal(pto)
+ lltype.render_immortal(pto.c_tp_name)
+
pto.c_tp_basicsize = rffi.sizeof(typedescr.basestruct)
if pto.c_tp_base:
if pto.c_tp_base.c_tp_basicsize > pto.c_tp_basicsize:
@@ -534,12 +544,25 @@
w_obj.ready()
finish_type_2(space, py_type, w_obj)
+ render_immortal(py_type, w_obj)
state = space.fromcache(RefcountState)
state.non_heaptypes_w.append(w_obj)
return w_obj
+def render_immortal(py_type, w_obj):
+ lltype.render_immortal(py_type.c_tp_bases)
+ lltype.render_immortal(py_type.c_tp_mro)
+
+ assert isinstance(w_obj, W_TypeObject)
+ if w_obj.is_cpytype():
+ lltype.render_immortal(py_type.c_tp_dict)
+ else:
+ lltype.render_immortal(py_type.c_tp_name)
+ if not w_obj.is_cpytype() and w_obj.is_heaptype():
+ lltype.render_immortal(py_type)
+
def finish_type_1(space, pto):
"""
Sets up tp_bases, necessary before creating the interpreter type.
Modified: pypy/branch/arm-backend/pypy/module/gc/__init__.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/gc/__init__.py (original)
+++ pypy/branch/arm-backend/pypy/module/gc/__init__.py Tue Nov 23 10:12:47 2010
@@ -29,6 +29,7 @@
'get_referents': 'referents.get_referents',
'get_referrers': 'referents.get_referrers',
'_dump_rpy_heap': 'referents._dump_rpy_heap',
+ 'get_typeids_z': 'referents.get_typeids_z',
'GcRef': 'referents.W_GcRef',
})
MixedModule.__init__(self, space, w_name)
Modified: pypy/branch/arm-backend/pypy/module/gc/app_referents.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/gc/app_referents.py (original)
+++ pypy/branch/arm-backend/pypy/module/gc/app_referents.py Tue Nov 23 10:12:47 2010
@@ -14,11 +14,25 @@
and [addr1]..[addrn] are addresses of other objects that this object
points to. The full dump is a list of such objects, with a marker
[0][0][0][-1] inserted after all GC roots, before all non-roots.
+
+ If the argument is a filename and the 'zlib' module is available,
+ we also write a 'typeids.txt' in the same directory, if none exists.
"""
if isinstance(file, str):
f = open(file, 'wb')
gc._dump_rpy_heap(f.fileno())
f.close()
+ try:
+ import zlib, os
+ except ImportError:
+ pass
+ else:
+ filename2 = os.path.join(os.path.dirname(file), 'typeids.txt')
+ if not os.path.exists(filename2):
+ data = zlib.decompress(gc.get_typeids_z())
+ f = open(filename2, 'wb')
+ f.write(data)
+ f.close()
else:
if isinstance(file, int):
fd = file
Modified: pypy/branch/arm-backend/pypy/module/gc/interp_gc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/gc/interp_gc.py (original)
+++ pypy/branch/arm-backend/pypy/module/gc/interp_gc.py Tue Nov 23 10:12:47 2010
@@ -5,6 +5,15 @@
def collect(space):
"Run a full collection."
+ # First clear the method cache. See test_gc for an example of why.
+ if space.config.objspace.std.withmethodcache:
+ from pypy.objspace.std.typeobject import MethodCache
+ cache = space.fromcache(MethodCache)
+ cache.clear()
+ if space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import IndexCache
+ cache = space.fromcache(IndexCache)
+ cache.clear()
rgc.collect()
return space.wrap(0)
Modified: pypy/branch/arm-backend/pypy/module/gc/referents.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/gc/referents.py (original)
+++ pypy/branch/arm-backend/pypy/module/gc/referents.py Tue Nov 23 10:12:47 2010
@@ -15,8 +15,16 @@
def try_cast_gcref_to_w_root(gcref):
w_obj = rgc.try_cast_gcref_to_instance(W_Root, gcref)
- if not we_are_translated() and not hasattr(w_obj, 'typedef'):
- w_obj = None
+ # Ignore the instances of W_Root that are not really valid as Python
+ # objects. There is e.g. WeakrefLifeline in module/_weakref that
+ # inherits from W_Root for internal reasons. Such instances don't
+ # have a typedef at all (or have a null typedef after translation).
+ if not we_are_translated():
+ if not hasattr(w_obj, 'typedef'):
+ return None
+ else:
+ if w_obj is None or not w_obj.typedef:
+ return None
return w_obj
def wrap(space, gcref):
@@ -169,3 +177,9 @@
if not ok:
raise missing_operation(space)
_dump_rpy_heap.unwrap_spec = [ObjSpace, int]
+
+def get_typeids_z(space):
+ a = rgc.get_typeids_z()
+ s = ''.join([a[i] for i in range(len(a))])
+ return space.wrap(s)
+get_typeids_z.unwrap_spec = [ObjSpace]
Modified: pypy/branch/arm-backend/pypy/module/gc/test/test_gc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/gc/test/test_gc.py (original)
+++ pypy/branch/arm-backend/pypy/module/gc/test/test_gc.py Tue Nov 23 10:12:47 2010
@@ -103,3 +103,49 @@
import gc
gc.dump_heap_stats(self.fname)
+
+class AppTestGcMethodCache(object):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True})
+
+ def test_clear_method_cache(self):
+ import gc, weakref
+ rlist = []
+ def f():
+ class C(object):
+ def f(self):
+ pass
+ C().f() # Fill the method cache
+ rlist.append(weakref.ref(C))
+ for i in range(10):
+ f()
+ gc.collect() # the classes C should all go away here
+ # the last class won't go in mapdict, as long as the code object of f
+ # is around
+ rlist.pop()
+ for r in rlist:
+ assert r() is None
+
+class AppTestGcMapDictIndexCache(AppTestGcMethodCache):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True,
+ "objspace.std.withmapdict": True})
+
+
+ def test_clear_index_cache(self):
+ import gc, weakref
+ rlist = []
+ def f():
+ class C(object):
+ def f(self):
+ pass
+ c = C()
+ c.x = 1
+ getattr(c, "x") # fill the index cache without using the local cache
+ getattr(c, "x")
+ rlist.append(weakref.ref(C))
+ for i in range(5):
+ f()
+ gc.collect() # the classes C should all go away here
+ for r in rlist:
+ assert r() is None
Modified: pypy/branch/arm-backend/pypy/module/imp/importing.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/imp/importing.py (original)
+++ pypy/branch/arm-backend/pypy/module/imp/importing.py Tue Nov 23 10:12:47 2010
@@ -12,7 +12,7 @@
from pypy.rlib import streamio, jit
from pypy.rlib.streamio import StreamErrors
from pypy.rlib.rarithmetic import intmask
-from pypy.rlib.objectmodel import we_are_translated
+from pypy.rlib.objectmodel import we_are_translated, specialize
SEARCH_ERROR = 0
PY_SOURCE = 1
@@ -25,10 +25,26 @@
# PY_CODERESOURCE = 8
IMP_HOOK = 9
-if sys.platform.startswith('win'):
- so_extension = ".pyd"
+if sys.platform == 'win32':
+ SO = ".pyd"
else:
- so_extension = ".so"
+ SO = ".so"
+DEFAULT_SOABI = 'pypy-14'
+
+ at specialize.memo()
+def get_so_extension(space):
+ if space.config.objspace.soabi is not None:
+ soabi = space.config.objspace.soabi
+ else:
+ soabi = DEFAULT_SOABI
+
+ if not soabi:
+ return SO
+
+ if not space.config.translating:
+ soabi += 'i'
+
+ return '.' + soabi + SO
def find_modtype(space, filepart):
"""Check which kind of module to import for the given filepart,
@@ -53,6 +69,7 @@
return PY_COMPILED, ".pyc", "rb"
if space.config.objspace.usemodules.cpyext:
+ so_extension = get_so_extension(space)
pydfile = filepart + so_extension
if os.path.exists(pydfile) and case_ok(pydfile):
return C_EXTENSION, so_extension, "rb"
@@ -122,7 +139,7 @@
n = len(ctxt_name_prefix_parts)-level+1
assert n>=0
ctxt_name_prefix_parts = ctxt_name_prefix_parts[:n]
- if ctxt_w_path is None: # plain module
+ if ctxt_name_prefix_parts and ctxt_w_path is None: # plain module
ctxt_name_prefix_parts.pop()
if ctxt_name_prefix_parts:
rel_modulename = '.'.join(ctxt_name_prefix_parts)
Modified: pypy/branch/arm-backend/pypy/module/imp/interp_imp.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/imp/interp_imp.py (original)
+++ pypy/branch/arm-backend/pypy/module/imp/interp_imp.py Tue Nov 23 10:12:47 2010
@@ -8,10 +8,16 @@
def get_suffixes(space):
w = space.wrap
- return space.newlist([
+ suffixes_w = []
+ if space.config.objspace.usemodules.cpyext:
+ suffixes_w.append(
+ space.newtuple([w(importing.get_so_extension(space)),
+ w('rb'), w(importing.C_EXTENSION)]))
+ suffixes_w.extend([
space.newtuple([w('.py'), w('U'), w(importing.PY_SOURCE)]),
space.newtuple([w('.pyc'), w('rb'), w(importing.PY_COMPILED)]),
])
+ return space.newlist(suffixes_w)
def get_magic(space):
x = importing.get_pyc_magic(space)
Modified: pypy/branch/arm-backend/pypy/module/imp/test/test_app.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/imp/test/test_app.py (original)
+++ pypy/branch/arm-backend/pypy/module/imp/test/test_app.py Tue Nov 23 10:12:47 2010
@@ -47,6 +47,9 @@
elif mode == self.imp.PY_COMPILED:
assert suffix in ('.pyc', '.pyo')
assert type == 'rb'
+ elif mode == self.imp.C_EXTENSION:
+ assert suffix.endswith(('.pyd', '.so'))
+ assert type == 'rb'
def test_obscure_functions(self):
Modified: pypy/branch/arm-backend/pypy/module/imp/test/test_import.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/imp/test/test_import.py (original)
+++ pypy/branch/arm-backend/pypy/module/imp/test/test_import.py Tue Nov 23 10:12:47 2010
@@ -360,6 +360,12 @@
""".rstrip()
raises(ValueError, imp)
+ def test_future_relative_import_error_when_in_non_package2(self):
+ exec """def imp():
+ from .. import inpackage
+ """.rstrip()
+ raises(ValueError, imp)
+
def test_relative_import_with___name__(self):
import sys
mydict = {'__name__': 'sys.foo'}
@@ -467,6 +473,17 @@
except ImportError:
pass
+class TestAbi:
+ def test_abi_tag(self):
+ space1 = gettestobjspace(soabi='TEST')
+ space2 = gettestobjspace(soabi='')
+ if sys.platform == 'win32':
+ assert importing.get_so_extension(space1) == '.TESTi.pyd'
+ assert importing.get_so_extension(space2) == '.pyd'
+ else:
+ assert importing.get_so_extension(space1) == '.TESTi.so'
+ assert importing.get_so_extension(space2) == '.so'
+
def _getlong(data):
x = marshal.dumps(data)
return x[-4:]
@@ -784,6 +801,7 @@
extrapath = udir.ensure("pythonpath", dir=1)
extrapath.join("urllib.py").write("print 42\n")
old = os.environ.get('PYTHONPATH', None)
+ oldlang = os.environ.pop('LANG', None)
try:
os.environ['PYTHONPATH'] = str(extrapath)
output = py.process.cmdexec('''"%s" "%s" -c "import urllib"''' %
@@ -792,6 +810,8 @@
finally:
if old:
os.environ['PYTHONPATH'] = old
+ if oldlang:
+ os.environ['LANG'] = oldlang
class AppTestImportHooks(object):
def test_meta_path(self):
Modified: pypy/branch/arm-backend/pypy/module/parser/test/test_parser.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/parser/test/test_parser.py (original)
+++ pypy/branch/arm-backend/pypy/module/parser/test/test_parser.py Tue Nov 23 10:12:47 2010
@@ -10,6 +10,9 @@
cls.w_m = space.appexec([], """():
import parser
return parser""")
+ cls.w_symbol = space.appexec([], """():
+ import symbol
+ return symbol""")
class AppTestParser(ParserModuleTest):
@@ -36,7 +39,7 @@
seq = getattr(s, meth)()
assert isinstance(seq, tp)
assert len(seq) == 4
- assert seq[0] == 286
+ assert seq[0] == self.symbol.file_input
assert len(seq[2]) == 2
assert len(seq[3]) == 2
assert seq[2][0] == 4
Modified: pypy/branch/arm-backend/pypy/module/posix/interp_posix.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/posix/interp_posix.py (original)
+++ pypy/branch/arm-backend/pypy/module/posix/interp_posix.py Tue Nov 23 10:12:47 2010
@@ -454,7 +454,8 @@
self.w_environ = space.newdict()
if _WIN:
self.cryptProviderPtr = lltype.malloc(
- rffi.CArray(HCRYPTPROV), 1, zero=True, flavor='raw')
+ rffi.CArray(HCRYPTPROV), 1, zero=True,
+ flavor='raw', immortal=True)
def startup(self, space):
_convertenviron(space, self.w_environ)
def _freeze_(self):
Modified: pypy/branch/arm-backend/pypy/module/pypyjit/__init__.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/pypyjit/__init__.py (original)
+++ pypy/branch/arm-backend/pypy/module/pypyjit/__init__.py Tue Nov 23 10:12:47 2010
@@ -6,6 +6,7 @@
interpleveldefs = {
'set_param': 'interp_jit.set_param',
+ 'residual_call': 'interp_jit.residual_call',
}
def setup_after_space_initialization(self):
@@ -14,6 +15,5 @@
# add the 'defaults' attribute
from pypy.rlib.jit import PARAMETERS
space = self.space
- # XXX this is not really the default compiled into a pypy-c-jit XXX
w_obj = space.wrap(PARAMETERS)
space.setattr(space.wrap(self), space.wrap('defaults'), w_obj)
Modified: pypy/branch/arm-backend/pypy/module/pypyjit/interp_jit.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/pypyjit/interp_jit.py (original)
+++ pypy/branch/arm-backend/pypy/module/pypyjit/interp_jit.py Tue Nov 23 10:12:47 2010
@@ -5,10 +5,11 @@
from pypy.tool.pairtype import extendabletype
from pypy.rlib.rarithmetic import r_uint, intmask
-from pypy.rlib.jit import JitDriver, hint, we_are_jitted
+from pypy.rlib.jit import JitDriver, hint, we_are_jitted, dont_look_inside
+from pypy.rlib.jit import current_trace_length
import pypy.interpreter.pyopcode # for side-effects
from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import ObjSpace, Arguments
+from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root
from pypy.interpreter.pycode import PyCode, CO_GENERATOR
from pypy.interpreter.pyframe import PyFrame
from pypy.interpreter.pyopcode import ExitFrame
@@ -80,9 +81,22 @@
def jump_absolute(self, jumpto, _, ec=None):
if we_are_jitted():
+ # Normally, the tick counter is decremented by 100 for every
+ # Python opcode. Here, to better support JIT compilation of
+ # small loops, we decrement it by a possibly smaller constant.
+ # We get the maximum 100 when the (unoptimized) trace length
+ # is at least 3200 (a bit randomly).
+ trace_length = r_uint(current_trace_length())
+ decr_by = trace_length // 32
+ if decr_by < 1:
+ decr_by = 1
+ elif decr_by > 100: # also if current_trace_length() returned -1
+ decr_by = 100
+ #
self.last_instr = intmask(jumpto)
- ec.bytecode_trace(self)
+ ec.bytecode_trace(self, intmask(decr_by))
jumpto = r_uint(self.last_instr)
+ #
pypyjitdriver.can_enter_jit(frame=self, ec=ec, next_instr=jumpto,
pycode=self.getcode())
return jumpto
@@ -131,3 +145,10 @@
"no JIT parameter '%s'", key)
set_param.unwrap_spec = [ObjSpace, Arguments]
+
+ at dont_look_inside
+def residual_call(space, w_callable, args):
+ '''For testing. Invokes callable(...), but without letting
+ the JIT follow the call.'''
+ return space.call_args(w_callable, args)
+residual_call.unwrap_spec = [ObjSpace, W_Root, Arguments]
Modified: pypy/branch/arm-backend/pypy/module/pypyjit/policy.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/pypyjit/policy.py (original)
+++ pypy/branch/arm-backend/pypy/module/pypyjit/policy.py Tue Nov 23 10:12:47 2010
@@ -6,12 +6,13 @@
if (modname == '__builtin__.operation' or
modname == '__builtin__.abstractinst' or
modname == '__builtin__.interp_classobj' or
- modname == '__builtin__.functional'):
+ modname == '__builtin__.functional' or
+ modname == '__builtin__.descriptor'):
return True
if '.' in modname:
modname, _ = modname.split('.', 1)
if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions',
- 'imp', 'sys', 'array']:
+ 'imp', 'sys', 'array', '_ffi']:
return True
return False
@@ -19,7 +20,7 @@
# this function should never actually return True directly
# but instead call the base implementation
mod = func.__module__ or '?'
-
+
if mod.startswith('pypy.objspace.'):
# gc_id operation
if func.__name__ == 'id__ANY':
@@ -36,5 +37,5 @@
modname = mod[len('pypy.module.'):]
if not self.look_inside_pypy_module(modname):
return False
-
+
return True
Modified: pypy/branch/arm-backend/pypy/module/pypyjit/test/test_policy.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/pypyjit/test/test_policy.py (original)
+++ pypy/branch/arm-backend/pypy/module/pypyjit/test/test_policy.py Tue Nov 23 10:12:47 2010
@@ -12,7 +12,7 @@
def test_rlocale():
from pypy.rlib.rlocale import setlocale
- assert not pypypolicy.look_inside_function(setlocale)
+ assert not pypypolicy.look_inside_function(setlocale)
def test_geninterp():
d = {'_geninterp_': True}
@@ -28,6 +28,10 @@
from pypy.interpreter.pyparser import parser
assert not pypypolicy.look_inside_function(parser.Grammar.__init__.im_func)
+def test_property():
+ from pypy.module.__builtin__.descriptor import W_Property
+ assert pypypolicy.look_inside_function(W_Property.get.im_func)
+
def test_pypy_module():
from pypy.module._random.interp_random import W_Random
assert not pypypolicy.look_inside_function(W_Random.random)
@@ -35,6 +39,7 @@
assert pypypolicy.look_inside_pypy_module('__builtin__.operation')
assert pypypolicy.look_inside_pypy_module('__builtin__.abstractinst')
assert pypypolicy.look_inside_pypy_module('__builtin__.functional')
+ assert pypypolicy.look_inside_pypy_module('__builtin__.descriptor')
assert pypypolicy.look_inside_pypy_module('exceptions.interp_exceptions')
for modname in 'pypyjit', 'signal', 'micronumpy', 'math', 'imp':
assert pypypolicy.look_inside_pypy_module(modname)
@@ -42,4 +47,3 @@
def test_see_jit_module():
assert pypypolicy.look_inside_pypy_module('pypyjit.interp_jit')
-
Modified: pypy/branch/arm-backend/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/branch/arm-backend/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 23 10:12:47 2010
@@ -79,8 +79,11 @@
class PyPyCJITTests(object):
- def run_source(self, source, expected_max_ops, *testcases):
+ def run_source(self, source, expected_max_ops, *testcases, **kwds):
assert isinstance(expected_max_ops, int)
+ threshold = kwds.pop('threshold', 3)
+ if kwds:
+ raise TypeError, 'Unsupported keyword arguments: %s' % kwds.keys()
source = py.code.Source(source)
filepath = self.tmpdir.join('case%d.py' % self.counter)
logfilepath = filepath.new(ext='.log')
@@ -90,9 +93,12 @@
# some support code...
print >> f, py.code.Source("""
import sys
+ # we don't want to see the small bridges created
+ # by the checkinterval reaching the limit
+ sys.setcheckinterval(10000000)
try: # make the file runnable by CPython
import pypyjit
- pypyjit.set_param(threshold=3)
+ pypyjit.set_param(threshold=%d)
except ImportError:
pass
@@ -102,7 +108,7 @@
print >> sys.stderr, 'got:', repr(result)
assert result == expected
assert type(result) is type(expected)
- """)
+ """ % threshold)
for testcase in testcases * 2:
print >> f, "check(%r, %r)" % testcase
print >> f, "print 'OK :-)'"
@@ -116,6 +122,8 @@
result = child_stdout.read()
child_stdout.close()
assert result
+ if result.strip().startswith('SKIP:'):
+ py.test.skip(result.strip())
assert result.splitlines()[-1].strip() == 'OK :-)'
self.parse_loops(logfilepath)
self.print_loops()
@@ -123,16 +131,18 @@
if self.total_ops > expected_max_ops:
assert 0, "too many operations: got %d, expected maximum %d" % (
self.total_ops, expected_max_ops)
+ return result
def parse_loops(self, opslogfile):
- from pypy.jit.metainterp.test.oparser import parse
+ from pypy.jit.tool.oparser import parse
from pypy.tool import logparser
assert opslogfile.check()
log = logparser.parse_log_file(str(opslogfile))
parts = logparser.extract_category(log, 'jit-log-opt-')
+ self.rawloops = [part for part in parts
+ if not from_entry_bridge(part, parts)]
# skip entry bridges, they can contain random things
- self.loops = [parse(part, no_namespace=True) for part in parts
- if not from_entry_bridge(part, parts)]
+ self.loops = [parse(part, no_namespace=True) for part in self.rawloops]
self.sliced_loops = [] # contains all bytecodes of all loops
self.total_ops = 0
for loop in self.loops:
@@ -156,12 +166,11 @@
return [ops for ops in self.sliced_loops if ops.bytecode == name]
def print_loops(self):
- for loop in self.loops:
+ for rawloop in self.rawloops:
print
print '@' * 79
print
- for op in loop.operations:
- print op
+ print rawloop.rstrip()
print
print '@' * 79
@@ -272,7 +281,7 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 7
+ assert len(ops[0].get_opnames("guard")) <= 2
assert not ops[1] # second LOOKUP_METHOD folded away
ops = self.get_by_bytecode("CALL_METHOD")
@@ -283,12 +292,16 @@
else:
assert not bytecode.get_opnames("call")
assert not bytecode.get_opnames("new")
- assert len(bytecode.get_opnames("guard")) <= 9
+ assert len(bytecode.get_opnames("guard")) <= 6
assert len(ops[1]) < len(ops[0])
ops = self.get_by_bytecode("LOAD_ATTR")
assert len(ops) == 2
- assert ops[0].get_opnames() == ["getfield_gc", "getarrayitem_gc",
+ # With mapdict, we get fast access to (so far) the 5 first
+ # attributes, which means it is done with only the following
+ # operations. (For the other attributes there is additionally
+ # a getarrayitem_gc.)
+ assert ops[0].get_opnames() == ["getfield_gc",
"guard_nonnull_class"]
assert not ops[1] # second LOAD_ATTR folded away
@@ -317,8 +330,8 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 7
- assert len(ops[0].get_opnames("getfield")) < 6
+ assert len(ops[0].get_opnames("guard")) <= 2
+ assert len(ops[0].get_opnames("getfield")) <= 4
assert not ops[1] # second LOOKUP_METHOD folded away
def test_default_and_kw(self):
@@ -364,10 +377,75 @@
([1000], 49500),
([10000], 495000),
([100000], 4950000))
- assert len(self.loops) == 2
+ assert len(self.loops) == 3
op, = self.get_by_bytecode("CALL_FUNCTION_KW")
# XXX a bit too many guards, but better than before
- assert len(op.get_opnames("guard")) <= 10
+ assert len(op.get_opnames("guard")) <= 12
+
+ def test_stararg_virtual(self):
+ self.run_source('''
+ d = {}
+
+ def g(*args):
+ return len(args)
+ def h(a, b, c):
+ return c
+
+ def main(x):
+ s = 0
+ for i in range(x):
+ l = [i, x, 2]
+ s += g(*l)
+ s += h(*l)
+ s += g(i, x, 2)
+ for i in range(x):
+ l = [x, 2]
+ s += g(i, *l)
+ s += h(i, *l)
+ return s
+ ''', 100000, ([100], 1300),
+ ([1000], 13000),
+ ([10000], 130000),
+ ([100000], 1300000))
+ assert len(self.loops) == 2
+ ops = self.get_by_bytecode("CALL_FUNCTION_VAR")
+ assert len(ops) == 4
+ for op in ops:
+ assert len(op.get_opnames("new")) == 0
+ assert len(op.get_opnames("call_may_force")) == 0
+
+ ops = self.get_by_bytecode("CALL_FUNCTION")
+ for op in ops:
+ assert len(op.get_opnames("new")) == 0
+ assert len(op.get_opnames("call_may_force")) == 0
+
+ def test_stararg(self):
+ self.run_source('''
+ d = {}
+
+ def g(*args):
+ return args[-1]
+ def h(*args):
+ return len(args)
+
+ def main(x):
+ s = 0
+ l = []
+ i = 0
+ while i < x:
+ l.append(1)
+ s += g(*l)
+ i = h(*l)
+ return s
+ ''', 100000, ([100], 100),
+ ([1000], 1000),
+ ([2000], 2000),
+ ([4000], 4000))
+ assert len(self.loops) == 1
+ ops = self.get_by_bytecode("CALL_FUNCTION_VAR")
+ for op in ops:
+ assert len(op.get_opnames("new_with_vtable")) == 0
+ assert len(op.get_opnames("call_may_force")) == 0
def test_virtual_instance(self):
self.run_source('''
@@ -382,7 +460,7 @@
a.x = 2
i = i + a.x
return i
- ''', 67,
+ ''', 69,
([20], 20),
([31], 32))
@@ -390,7 +468,7 @@
self.get_by_bytecode("CALL_FUNCTION"))
assert not callA.get_opnames("call")
assert not callA.get_opnames("new")
- assert len(callA.get_opnames("guard")) <= 8
+ assert len(callA.get_opnames("guard")) <= 2
assert not callisinstance1.get_opnames("call")
assert not callisinstance1.get_opnames("new")
assert len(callisinstance1.get_opnames("guard")) <= 2
@@ -554,17 +632,13 @@
def test_blockstack_virtualizable(self):
self.run_source('''
- def g(k):
- s = 0
- for i in range(k, k+2):
- s += 1
- return s
+ from pypyjit import residual_call
def main():
i = 0
while i < 100:
try:
- g(i)
+ residual_call(len, [])
except:
pass
i += 1
@@ -606,16 +680,17 @@
# call that can raise is not exchanged into getarrayitem_gc
def test_overflow_checking(self):
+ startvalue = sys.maxint - 2147483647
self.run_source('''
def main():
def f(a,b):
if a < 0: return -1
return a-b
- total = 0
+ total = %d
for i in range(100000):
total += f(i, 5)
return total
- ''', 170, ([], 4999450000L))
+ ''' % startvalue, 170, ([], startvalue + 4999450000L))
def test_boolrewrite_invers(self):
for a, b, res, ops in (('2000', '2000', 20001000, 51),
@@ -742,6 +817,8 @@
'''%(op1, float(a)/4.0, float(b)/4.0, op2), 109, ([], res))
def test_boolrewrite_ptr(self):
+ # XXX this test is way too imprecise in what it is actually testing
+ # it should count the number of guards instead
compares = ('a == b', 'b == a', 'a != b', 'b != a', 'a == c', 'c != b')
for e1 in compares:
for e2 in compares:
@@ -765,7 +842,7 @@
print
print 'Test:', e1, e2, n, res
self.run_source('''
- class tst:
+ class tst(object):
pass
def main():
a = tst()
@@ -781,24 +858,6 @@
return sa
'''%(e1, e2), n, ([], res))
- def test_boolrewrite_ptr_single(self):
- self.run_source('''
- class tst:
- pass
- def main():
- a = tst()
- b = tst()
- c = tst()
- sa = 0
- for i in range(1000):
- if a == b: sa += 1
- else: sa += 2
- if a != b: sa += 10000
- else: sa += 20000
- if i > 750: a = b
- return sa
- ''', 215, ([], 12481752))
-
def test_array_sum(self):
for tc, maxops in zip('bhilBHILfd', (38,) * 6 + (40, 40, 41, 38)):
res = 19352859
@@ -847,7 +906,12 @@
''', 65, ([], 122880))
def test_array_intimg(self):
- for tc, maxops in zip('ilILd', (67, 67, 69, 69, 61)):
+ # XXX this test is way too imprecise in what it is actually testing
+ # it should count the number of guards instead
+ for tc, maxops in zip('ilILd', (67, 67, 70, 70, 61)):
+ print
+ print '='*65
+ print '='*20, 'running test for tc=%r' % (tc,), '='*20
res = 73574560
if tc in 'IL':
res = long(res)
@@ -1130,6 +1194,44 @@
return sa
''', 88, ([], 1997001))
+ def test__ffi_call(self):
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ out = self.run_source('''
+ def main():
+ try:
+ from _ffi import CDLL, types
+ except ImportError:
+ sys.stdout.write('SKIP: cannot import _ffi')
+ return 0
+
+ libm = CDLL('%(libm_name)s')
+ pow = libm.getfunc('pow', [types.double, types.double],
+ types.double)
+ print pow.getaddr()
+ i = 0
+ res = 0
+ while i < 2000:
+ res += pow(2, 3)
+ i += 1
+ return res
+ ''' % locals(),
+ 76, ([], 8.0*2000), threshold=1000)
+ pow_addr = int(out.splitlines()[0])
+ ops = self.get_by_bytecode('CALL_FUNCTION')
+ assert len(ops) == 2 # we get two loops, because of specialization
+ call_function = ops[0]
+ last_ops = [op.getopname() for op in call_function[-5:]]
+ assert last_ops == ['force_token',
+ 'setfield_gc',
+ 'call_may_force',
+ 'guard_not_forced',
+ 'guard_no_exception']
+ call = call_function[-3]
+ assert call.getarg(0).value == pow_addr
+ assert call.getarg(1).value == 2.0
+ assert call.getarg(2).value == 3.0
+
# test_circular
class AppTestJIT(PyPyCJITTests):
@@ -1155,6 +1257,17 @@
cls.pypy_c = option.pypy_c
+def test_interface_residual_call():
+ space = gettestobjspace(usemodules=['pypyjit'])
+ space.appexec([], """():
+ import pypyjit
+ def f(*args, **kwds):
+ return (args, kwds)
+ res = pypyjit.residual_call(f, 4, x=6)
+ assert res == ((4,), {'x': 6})
+ """)
+
+
def has_info(pypy_c, option):
g = os.popen('"%s" --info' % pypy_c, 'r')
lines = g.readlines()
Modified: pypy/branch/arm-backend/pypy/module/rctime/interp_time.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/rctime/interp_time.py (original)
+++ pypy/branch/arm-backend/pypy/module/rctime/interp_time.py Tue Nov 23 10:12:47 2010
@@ -69,7 +69,7 @@
CLOCKS_PER_SEC = cConfig.CLOCKS_PER_SEC
clock_t = cConfig.clock_t
tm = cConfig.tm
-glob_buf = lltype.malloc(tm, flavor='raw', zero=True)
+glob_buf = lltype.malloc(tm, flavor='raw', zero=True, immortal=True)
if cConfig.has_gettimeofday:
c_gettimeofday = external('gettimeofday', [rffi.VOIDP, rffi.VOIDP], rffi.INT)
Modified: pypy/branch/arm-backend/pypy/module/signal/__init__.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/signal/__init__.py (original)
+++ pypy/branch/arm-backend/pypy/module/signal/__init__.py Tue Nov 23 10:12:47 2010
@@ -34,9 +34,7 @@
MixedModule.__init__(self, space, *args)
# add the signal-checking callback as an action on the space
space.check_signal_action = interp_signal.CheckSignalAction(space)
- space.actionflag.register_action(space.check_signal_action)
- # use the C-level pypysig_occurred variable as the action flag
- # (the result is that the C-level signal handler will directly
- # set the flag for the CheckSignalAction)
+ space.actionflag.register_periodic_action(space.check_signal_action,
+ use_bytecode_counter=False)
space.actionflag.__class__ = interp_signal.SignalActionFlag
# xxx yes I know the previous line is a hack
Modified: pypy/branch/arm-backend/pypy/module/signal/interp_signal.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/signal/interp_signal.py (original)
+++ pypy/branch/arm-backend/pypy/module/signal/interp_signal.py Tue Nov 23 10:12:47 2010
@@ -1,7 +1,7 @@
from pypy.interpreter.error import OperationError
from pypy.interpreter.baseobjspace import W_Root, ObjSpace
from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag
-from pypy.rlib.rarithmetic import LONG_BIT, intmask
+from pypy.interpreter.executioncontext import PeriodicAsyncAction
import signal as cpy_signal
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.translator.tool.cbuild import ExternalCompilationInfo
@@ -53,19 +53,29 @@
class SignalActionFlag(AbstractActionFlag):
- def get(self):
+ # This class uses the C-level pypysig_counter variable as the tick
+ # counter. The C-level signal handler will reset it to -1 whenever
+ # a signal is received.
+
+ def get_ticker(self):
p = pypysig_getaddr_occurred()
return p.c_value
- def set(self, value):
+
+ def reset_ticker(self, value):
p = pypysig_getaddr_occurred()
p.c_value = value
+ def decrement_ticker(self, by):
+ p = pypysig_getaddr_occurred()
+ value = p.c_value
+ if self.has_bytecode_counter: # this 'if' is constant-folded
+ value -= by
+ p.c_value = value
+ return value
-class CheckSignalAction(AsyncAction):
- """An action that is automatically invoked when a signal is received."""
- # The C-level signal handler sets the highest bit of pypysig_occurred:
- bitmask = intmask(1 << (LONG_BIT-1))
+class CheckSignalAction(PeriodicAsyncAction):
+ """An action that is automatically invoked when a signal is received."""
def __init__(self, space):
AsyncAction.__init__(self, space)
@@ -74,7 +84,6 @@
# need a helper action in case signals arrive in a non-main thread
self.pending_signals = {}
self.reissue_signal_action = ReissueSignalAction(space)
- space.actionflag.register_action(self.reissue_signal_action)
else:
self.reissue_signal_action = None
Modified: pypy/branch/arm-backend/pypy/module/signal/test/test_signal.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/signal/test/test_signal.py (original)
+++ pypy/branch/arm-backend/pypy/module/signal/test/test_signal.py Tue Nov 23 10:12:47 2010
@@ -1,6 +1,38 @@
import os, py
+import signal as cpy_signal
from pypy.conftest import gettestobjspace
+
+class TestCheckSignals:
+
+ def setup_class(cls):
+ if not hasattr(os, 'kill') or not hasattr(os, 'getpid'):
+ py.test.skip("requires os.kill() and os.getpid()")
+ cls.space = gettestobjspace(usemodules=['signal'])
+
+ def test_checksignals(self):
+ space = self.space
+ w_received = space.appexec([], """():
+ import signal
+ received = []
+ def myhandler(signum, frame):
+ received.append(signum)
+ signal.signal(signal.SIGUSR1, myhandler)
+ return received""")
+ #
+ assert not space.is_true(w_received)
+ #
+ # send the signal now
+ os.kill(os.getpid(), cpy_signal.SIGUSR1)
+ #
+ # myhandler() should not be immediately called
+ assert not space.is_true(w_received)
+ #
+ # calling ec.checksignals() should call it
+ space.getexecutioncontext().checksignals()
+ assert space.is_true(w_received)
+
+
class AppTestSignal:
def setup_class(cls):
@@ -24,18 +56,12 @@
signal.signal(signal.SIGUSR1, myhandler)
posix.kill(posix.getpid(), signal.SIGUSR1)
- for i in range(10000):
- # wait a bit for the signal to be delivered to the handler
- if received:
- break
+ # the signal should be delivered to the handler immediately
assert received == [signal.SIGUSR1]
del received[:]
posix.kill(posix.getpid(), signal.SIGUSR1)
- for i in range(10000):
- # wait a bit for the signal to be delivered to the handler
- if received:
- break
+ # the signal should be delivered to the handler immediately
assert received == [signal.SIGUSR1]
del received[:]
Modified: pypy/branch/arm-backend/pypy/module/sys/__init__.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/sys/__init__.py (original)
+++ pypy/branch/arm-backend/pypy/module/sys/__init__.py Tue Nov 23 10:12:47 2010
@@ -7,13 +7,14 @@
"""Sys Builtin Module. """
def __init__(self, space, w_name):
"""NOT_RPYTHON""" # because parent __init__ isn't
+ if space.config.translating:
+ del self.__class__.interpleveldefs['pypy_getudir']
super(Module, self).__init__(space, w_name)
- self.checkinterval = 100
self.recursionlimit = 100
self.w_default_encoder = None
self.defaultencoding = "ascii"
self.filesystemencoding = None
-
+
interpleveldefs = {
'__name__' : '(space.wrap("sys"))',
'__doc__' : '(space.wrap("PyPy sys module"))',
@@ -37,7 +38,7 @@
'argv' : 'state.get(space).w_argv',
'warnoptions' : 'state.get(space).w_warnoptions',
'builtin_module_names' : 'state.w_None',
- 'pypy_getudir' : 'state.pypy_getudir',
+ 'pypy_getudir' : 'state.pypy_getudir', # not translated
'pypy_initial_path' : 'state.pypy_initial_path',
'_getframe' : 'vm._getframe',
Modified: pypy/branch/arm-backend/pypy/module/sys/state.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/sys/state.py (original)
+++ pypy/branch/arm-backend/pypy/module/sys/state.py Tue Nov 23 10:12:47 2010
@@ -33,6 +33,8 @@
raise OSError(errno.ENOTDIR, path)
+platform = sys.platform
+
def getinitialpath(prefix):
from pypy.module.sys.version import CPYTHON_VERSION
dirname = '%d.%d.%d' % (CPYTHON_VERSION[0],
@@ -51,6 +53,15 @@
importlist.append(lib_pypy)
importlist.append(python_std_lib_modified)
importlist.append(python_std_lib)
+ #
+ # List here the extra platform-specific paths.
+ if platform != 'win32':
+ importlist.append(os.path.join(python_std_lib, 'plat-'+platform))
+ if platform == 'darwin':
+ platmac = os.path.join(python_std_lib, 'plat-mac')
+ importlist.append(platmac)
+ importlist.append(os.path.join(platmac, 'lib-scriptpackages'))
+ #
return importlist
def pypy_initial_path(space, srcdir):
@@ -95,15 +106,8 @@
def getio(space):
return space.fromcache(IOState)
-def _pypy_getudir(space):
- """NOT_RPYTHON"""
+def pypy_getudir(space):
+ """NOT_RPYTHON
+ (should be removed from interpleveldefs before translation)"""
from pypy.tool.udir import udir
return space.wrap(str(udir))
-_pypy_getudir._annspecialcase_ = "override:ignore"
-
-# we need the indirection because this function will live in a dictionary with other
-# RPYTHON functions and share call sites with them. Better it not be a special-case
-# directly.
-def pypy_getudir(space):
- return _pypy_getudir(space)
-
Modified: pypy/branch/arm-backend/pypy/module/sys/test/test_initialpath.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/sys/test/test_initialpath.py (original)
+++ pypy/branch/arm-backend/pypy/module/sys/test/test_initialpath.py Tue Nov 23 10:12:47 2010
@@ -15,4 +15,5 @@
def test_stdlib_in_prefix(tmpdir):
dirs = build_hierarchy(tmpdir)
path = getinitialpath(str(tmpdir))
- assert path == map(str, dirs)
+ # we get at least 'dirs', and maybe more (e.g. plat-linux2)
+ assert path[:len(dirs)] == map(str, dirs)
Modified: pypy/branch/arm-backend/pypy/module/sys/version.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/sys/version.py (original)
+++ pypy/branch/arm-backend/pypy/module/sys/version.py Tue Nov 23 10:12:47 2010
@@ -8,7 +8,7 @@
CPYTHON_VERSION = (2, 5, 2, "beta", 42) #XXX # sync patchlevel.h
CPYTHON_API_VERSION = 1012 #XXX # sync with include/modsupport.h
-PYPY_VERSION = (1, 3, 0, "beta", '?') #XXX # sync patchlevel.h
+PYPY_VERSION = (1, 4, 0, "beta", '?') #XXX # sync patchlevel.h
# the last item is replaced by the svn revision ^^^
TRIM_URL_UP_TO = 'svn/pypy/'
Modified: pypy/branch/arm-backend/pypy/module/sys/vm.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/sys/vm.py (original)
+++ pypy/branch/arm-backend/pypy/module/sys/vm.py Tue Nov 23 10:12:47 2010
@@ -67,7 +67,7 @@
def setcheckinterval(space, interval):
"""Tell the Python interpreter to check for asynchronous events every
n instructions. This also affects how often thread switches occur."""
- space.actionflag.setcheckinterval(space, interval)
+ space.actionflag.setcheckinterval(interval)
setcheckinterval.unwrap_spec = [ObjSpace, int]
def getcheckinterval(space):
@@ -77,7 +77,7 @@
# return 0. The idea is that according to the CPython docs, <= 0
# means "check every virtual instruction, maximizing responsiveness
# as well as overhead".
- result = space.sys.checkinterval
+ result = space.actionflag.getcheckinterval()
if result <= 1:
result = 0
return space.wrap(result)
Modified: pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
==============================================================================
--- pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c (original)
+++ pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c Tue Nov 23 10:12:47 2010
@@ -533,3 +533,7 @@
return inp;
}
+int my_unused_function(void)
+{
+ return 42;
+}
Modified: pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py (original)
+++ pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py Tue Nov 23 10:12:47 2010
@@ -148,3 +148,7 @@
# but it segfaults for some reason.
if sys.platform == 'win32':
assert f() == 0x12345678
+
+ def test_restype(self):
+ foo = lib.my_unused_function
+ assert foo.restype is c_int # by default
Modified: pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py (original)
+++ pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py Tue Nov 23 10:12:47 2010
@@ -29,6 +29,7 @@
class RECT(Structure):
_fields_ = [("left", c_int), ("top", c_int),
("right", c_int), ("bottom", c_int)]
+
class TestFunctions(BaseCTypesTestChecker):
def test_mro(self):
@@ -392,6 +393,18 @@
result = f("abcd", ord("b"))
assert result == "bcd"
+ def test_caching_bug_1(self):
+ # the same test as test_call_some_args, with two extra lines
+ # in the middle that trigger caching in f._ptr, which then
+ # makes the last two lines fail
+ f = dll.my_strchr
+ f.argtypes = [c_char_p, c_int]
+ f.restype = c_char_p
+ result = f("abcd", ord("b"))
+ assert result == "bcd"
+ result = f("abcd", ord("b"), 42)
+ assert result == "bcd"
+
def test_sf1651235(self):
py.test.skip("we are less strict in checking callback parameters")
# see http://www.python.org/sf/1651235
Modified: pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py (original)
+++ pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py Tue Nov 23 10:12:47 2010
@@ -10,7 +10,7 @@
"""
def test_array_of_pointers(self):
# tests array item assignements & pointer.contents = ...
- A = POINTER(c_int) * 24
+ A = POINTER(c_long) * 24
a = A()
l = c_long(2)
p = pointer(l)
Modified: pypy/branch/arm-backend/pypy/module/thread/gil.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/thread/gil.py (original)
+++ pypy/branch/arm-backend/pypy/module/thread/gil.py Tue Nov 23 10:12:47 2010
@@ -20,7 +20,8 @@
def initialize(self, space):
# add the GIL-releasing callback as an action on the space
- space.actionflag.register_action(GILReleaseAction(space))
+ space.actionflag.register_periodic_action(GILReleaseAction(space),
+ use_bytecode_counter=True)
def setup_threads(self, space):
"""Enable threads in the object space, if they haven't already been."""
@@ -44,7 +45,6 @@
# test_compile_lock. As a workaround, we repatch these global
# fields systematically.
spacestate.ll_GIL = self.ll_GIL
- spacestate.actionflag = space.actionflag
invoke_around_extcall(before_external_call, after_external_call)
return result
@@ -68,18 +68,17 @@
def _freeze_(self):
self.ll_GIL = thread.null_ll_lock
- self.actionflag = None
- self.set_actionflag_bit_after_thread_switch = 0
+ self.action_after_thread_switch = None
+ # ^^^ set by AsyncAction.fire_after_thread_switch()
return False
def after_thread_switch(self):
# this is support logic for the signal module, to help it deliver
# signals to the main thread.
- actionflag = self.actionflag
- if actionflag is not None:
- flag = actionflag.get()
- flag |= self.set_actionflag_bit_after_thread_switch
- actionflag.set(flag)
+ action = self.action_after_thread_switch
+ if action is not None:
+ self.action_after_thread_switch = None
+ action.fire()
spacestate = SpaceState()
spacestate._freeze_()
Modified: pypy/branch/arm-backend/pypy/module/thread/ll_thread.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/thread/ll_thread.py (original)
+++ pypy/branch/arm-backend/pypy/module/thread/ll_thread.py Tue Nov 23 10:12:47 2010
@@ -111,7 +111,7 @@
c_thread_releaselock(self._lock)
def __del__(self):
- lltype.free(self._lock, flavor='raw')
+ lltype.free(self._lock, flavor='raw', track_allocation=False)
# ____________________________________________________________
#
@@ -128,10 +128,13 @@
null_ll_lock = lltype.nullptr(TLOCKP.TO)
def allocate_ll_lock():
- ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw')
+ # track_allocation=False here; be careful to lltype.free() it. The
+ # reason it is set to False is that we get it from all app-level
+ # lock objects, as well as from the GIL, which exists at shutdown.
+ ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw', track_allocation=False)
res = c_thread_lock_init(ll_lock)
if res == -1:
- lltype.free(ll_lock, flavor='raw')
+ lltype.free(ll_lock, flavor='raw', track_allocation=False)
raise error("out of resources")
return ll_lock
Modified: pypy/branch/arm-backend/pypy/module/thread/test/test_gil.py
==============================================================================
--- pypy/branch/arm-backend/pypy/module/thread/test/test_gil.py (original)
+++ pypy/branch/arm-backend/pypy/module/thread/test/test_gil.py Tue Nov 23 10:12:47 2010
@@ -8,7 +8,7 @@
pass
class FakeActionFlag(object):
- def register_action(self, action):
+ def register_periodic_action(self, action, use_bytecode_counter):
pass
def get(self):
return 0
Modified: pypy/branch/arm-backend/pypy/objspace/descroperation.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/descroperation.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/descroperation.py Tue Nov 23 10:12:47 2010
@@ -51,7 +51,7 @@
return space.is_w(w_typ1, w_typ2)
-class Object:
+class Object(object):
def descr__getattribute__(space, w_obj, w_name):
name = space.str_w(w_name)
w_descr = space.lookup(w_obj, name)
@@ -64,9 +64,7 @@
w_type = space.type(w_obj)
return space.get_and_call_function(w_get, w_descr, w_obj,
w_type)
- w_value = w_obj.getdictvalue_attr_is_in_class(space, name)
- else:
- w_value = w_obj.getdictvalue(space, name)
+ w_value = w_obj.getdictvalue(space, name)
if w_value is not None:
return w_value
if w_descr is not None:
@@ -76,13 +74,11 @@
def descr__setattr__(space, w_obj, w_name, w_value):
name = space.str_w(w_name)
w_descr = space.lookup(w_obj, name)
- shadows_type = False
if w_descr is not None:
if space.is_data_descr(w_descr):
space.set(w_descr, w_obj, w_value)
return
- shadows_type = True
- if w_obj.setdictvalue(space, name, w_value, shadows_type):
+ if w_obj.setdictvalue(space, name, w_value):
return
raiseattrerror(space, w_obj, name, w_descr)
@@ -100,7 +96,7 @@
def descr__init__(space, w_obj, __args__):
pass
-class DescrOperation:
+class DescrOperation(object):
_mixin_ = True
def is_data_descr(space, w_obj):
@@ -545,7 +541,7 @@
# what is the maximum value slices can get on CPython?
# we need to stick to that value, because fake.py etc.
-class Temp:
+class Temp(object):
def __getslice__(self, i, j):
return j
slice_max = Temp()[:]
Modified: pypy/branch/arm-backend/pypy/objspace/flow/flowcontext.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/flow/flowcontext.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/flow/flowcontext.py Tue Nov 23 10:12:47 2010
@@ -460,7 +460,22 @@
else:
v = self.peekvalue(oparg - 1)
self.space.call_method(v, 'append', w)
-
+
+ # `with` statement
+
+ def SETUP_WITH(self, offsettoend, next_instr):
+ # A simpler version than the 'real' 2.7 one:
+ # directly call manager.__enter__(), don't use special lookup functions
+ # which don't make sense on the RPython type system.
+ from pypy.interpreter.pyopcode import WithBlock
+ w_manager = self.peekvalue()
+ w_exit = self.space.getattr(w_manager, self.space.wrap("__exit__"))
+ self.settopvalue(w_exit)
+ w_result = self.space.call_method(w_manager, "__enter__")
+ block = WithBlock(self, next_instr + offsettoend)
+ self.append_block(block)
+ self.pushvalue(w_result)
+
# XXX Unimplemented 2.7 opcodes ----------------
# Set literals, set comprehensions
@@ -476,12 +491,6 @@
def MAP_ADD(self, oparg, next_instr):
raise NotImplementedError("MAP_ADD")
- # `with` statement
-
- def SETUP_WITH(self, oparg, next_instr):
- raise NotImplementedError("SETUP_WITH")
-
-
def make_arguments(self, nargs):
return ArgumentsForTranslation(self.space, self.peekvalues(nargs))
def argument_factory(self, *args):
@@ -493,3 +502,12 @@
raise operr
return pyframe.PyFrame.handle_operation_error(self, ec, operr,
*args, **kwds)
+
+ def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
+ if w_typ is not self.space.w_None:
+ # The annotator won't allow to merge exception types with None.
+ # Replace it with an object which will break translation when used
+ # (except maybe with 'exc_typ is None')
+ w_typ = self.space.wrap(self.space)
+ return self.space.call_function(w_func, w_typ, w_val, w_tb)
+
Modified: pypy/branch/arm-backend/pypy/objspace/flow/model.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/flow/model.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/flow/model.py Tue Nov 23 10:12:47 2010
@@ -355,7 +355,7 @@
return "%r = %s(%s)" % (self.result, self.opname,
", ".join(map(repr, self.args)))
-class Atom:
+class Atom(object):
def __init__(self, name):
self.__name__ = name # make save_global happy
def __repr__(self):
Modified: pypy/branch/arm-backend/pypy/objspace/flow/objspace.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/flow/objspace.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/flow/objspace.py Tue Nov 23 10:12:47 2010
@@ -210,6 +210,11 @@
check_class = self.unwrap(w_check_class)
except UnwrapException:
raise Exception, "non-constant except guard"
+ if check_class in (NotImplementedError, AssertionError):
+ # if we are in geninterp, we cannot catch these exceptions
+ if not self.config.translation.builtins_can_raise_exceptions:
+ raise error.FlowingError("Catching %s is not valid in RPython" %
+ check_class.__name__)
if not isinstance(check_class, tuple):
# the simple case
return ObjSpace.exception_match(self, w_exc_type, w_check_class)
Modified: pypy/branch/arm-backend/pypy/objspace/flow/test/test_objspace.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/flow/test/test_objspace.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/flow/test/test_objspace.py Tue Nov 23 10:12:47 2010
@@ -1,10 +1,11 @@
+from __future__ import with_statement
import new
import py
from pypy.objspace.flow.model import Constant, Block, Link, Variable, traverse
from pypy.objspace.flow.model import flatten, mkentrymap, c_last_exception
from pypy.interpreter.argument import Arguments
from pypy.translator.simplify import simplify_graph
-from pypy.objspace.flow.objspace import FlowObjSpace
+from pypy.objspace.flow.objspace import FlowObjSpace, error
from pypy.objspace.flow import objspace, flowcontext
from pypy import conftest
from pypy.tool.stdlib_opcode import bytecode_spec
@@ -828,6 +829,25 @@
simplify_graph(graph)
assert self.all_operations(graph) == {'getitem': 1}
+ def test_context_manager(self):
+ def f(c, x):
+ with x:
+ pass
+ graph = self.codetest(f)
+ # 2 method calls: x.__enter__() and x.__exit__(None, None, None)
+ assert self.all_operations(graph) == {'getattr': 2,
+ 'simple_call': 2}
+ #
+ def g(): pass
+ def f(c, x):
+ with x:
+ g()
+ graph = self.codetest(f)
+ assert self.all_operations(graph) == {
+ 'getattr': 2, # __enter__ and __exit__
+ 'simple_call': 4, # __enter__, g and 2 possible calls to __exit__
+ 'is_true': 1} # check the result of __exit__()
+
def monkey_patch_code(self, code, stacksize, flags, codestring, names, varnames):
c = code
return new.code(c.co_argcount, c.co_nlocals, stacksize, flags,
@@ -933,6 +953,22 @@
assert op.args[0] == Constant(g)
+ def test_cannot_catch_special_exceptions(self):
+ def f():
+ try:
+ f()
+ except NotImplementedError:
+ pass
+ py.test.raises(error.FlowingError, "self.codetest(f)")
+ #
+ def f():
+ try:
+ f()
+ except AssertionError:
+ pass
+ py.test.raises(error.FlowingError, "self.codetest(f)")
+
+
class TestFlowObjSpaceDelay(Base):
def setup_class(cls):
cls.space = FlowObjSpace()
@@ -993,6 +1029,15 @@
expected.sort()
assert excfound == expected
+ def test_can_catch_special_exceptions(self):
+ def f():
+ try:
+ f()
+ except NotImplementedError:
+ pass
+ graph = self.codetest(f)
+ # assert did not crash
+
DATA = {'x': 5,
'y': 6}
Modified: pypy/branch/arm-backend/pypy/objspace/std/callmethod.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/callmethod.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/callmethod.py Tue Nov 23 10:12:47 2010
@@ -13,6 +13,9 @@
from pypy.interpreter import function
from pypy.objspace.descroperation import object_getattribute
from pypy.rlib import jit, rstack # for resume points
+from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict, \
+ LOOKUP_METHOD_mapdict_fill_cache_method
+
# This module exports two extra methods for StdObjSpaceFrame implementing
# the LOOKUP_METHOD and CALL_METHOD opcodes in an efficient way, as well
@@ -30,6 +33,13 @@
#
space = f.space
w_obj = f.popvalue()
+
+ if space.config.objspace.std.withmapdict and not jit.we_are_jitted():
+ # mapdict has an extra-fast version of this function
+ from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict
+ if LOOKUP_METHOD_mapdict(f, nameindex, w_obj):
+ return
+
w_name = f.getname_w(nameindex)
w_value = None
@@ -44,12 +54,17 @@
else:
typ = type(w_descr)
if typ is function.Function or typ is function.FunctionWithFixedCode:
- w_value = w_obj.getdictvalue_attr_is_in_class(space, name)
+ w_value = w_obj.getdictvalue(space, name)
if w_value is None:
# fast method path: a function object in the class,
# nothing in the instance
f.pushvalue(w_descr)
f.pushvalue(w_obj)
+ if (space.config.objspace.std.withmapdict and
+ not jit.we_are_jitted()):
+ # let mapdict cache stuff
+ LOOKUP_METHOD_mapdict_fill_cache_method(
+ f.getcode(), nameindex, w_obj, w_type, w_descr)
return
if w_value is None:
w_value = space.getattr(w_obj, w_name)
@@ -103,7 +118,7 @@
w_descr = space.lookup(w_obj, methname)
typ = type(w_descr)
if typ is function.Function or typ is function.FunctionWithFixedCode:
- w_value = w_obj.getdictvalue_attr_is_in_class(space, methname)
+ w_value = w_obj.getdictvalue(space, methname)
if w_value is None:
# fast method path: a function object in the class,
# nothing in the instance
Modified: pypy/branch/arm-backend/pypy/objspace/std/celldict.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/celldict.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/celldict.py Tue Nov 23 10:12:47 2010
@@ -45,9 +45,9 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
- def impl_setitem_str(self, name, w_value, shadows_type=True):
+ def impl_setitem_str(self, name, w_value):
self.getcell(name, True).w_value = w_value
def impl_delitem(self, w_key):
@@ -66,7 +66,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
# inefficient, but do we care?
@@ -85,7 +85,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_lookup)
+ return self._as_rdict().impl_fallback_getitem(w_lookup)
def impl_getitem_str(self, lookup):
res = self.getcell(lookup, False)
Modified: pypy/branch/arm-backend/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/dictmultiobject.py Tue Nov 23 10:12:47 2010
@@ -49,14 +49,6 @@
elif space.config.objspace.std.withdictmeasurement:
assert w_type is None
return MeasuringDictImplementation(space)
- elif space.config.objspace.std.withsharingdict and instance:
- from pypy.objspace.std.sharingdict import SharedDictImplementation
- assert w_type is None
- return SharedDictImplementation(space)
- elif (space.config.objspace.std.withshadowtracking and instance and
- classofinstance is not None):
- assert w_type is None
- return ShadowDetectingDictImplementation(space, classofinstance)
elif instance or strdict or module:
assert w_type is None
return StrDictImplementation(space)
@@ -102,17 +94,17 @@
else:
return None
- # _________________________________________________________________
+ # _________________________________________________________________
# implementation methods
def impl_getitem(self, w_key):
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_getitem_str(self, w_key):
+ def impl_getitem_str(self, key):
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
raise NotImplementedError("abstract base class")
def impl_setitem(self, w_key, w_value):
@@ -120,7 +112,7 @@
def impl_delitem(self, w_key):
raise NotImplementedError("abstract base class")
-
+
def impl_length(self):
raise NotImplementedError("abstract base class")
@@ -165,12 +157,15 @@
key = OPTIMIZED_BUILTINS[i]
return self.impl_getitem_str(key)
- # this method will only be seen whan a certain config option is used
- def impl_shadows_anything(self):
- return True
-
- def impl_set_shadows_anything(self):
- pass
+ def impl_popitem(self):
+ # default implementation
+ space = self.space
+ iterator = self.impl_iter()
+ w_key, w_value = iterator.next()
+ if w_key is None:
+ raise KeyError
+ self.impl_delitem(w_key)
+ return w_key, w_value
# _________________________________________________________________
# fallback implementation methods
@@ -178,7 +173,7 @@
def impl_fallback_setitem(self, w_key, w_value):
self.r_dict_content[w_key] = w_value
- def impl_fallback_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_fallback_setitem_str(self, key, w_value):
return self.impl_fallback_setitem(self.space.wrap(key), w_value)
def impl_fallback_delitem(self, w_key):
@@ -211,18 +206,15 @@
key = OPTIMIZED_BUILTINS[i]
return self.impl_fallback_getitem_str(key)
- def impl_fallback_shadows_anything(self):
- return True
-
- def impl_fallback_set_shadows_anything(self):
- pass
+ def impl_fallback_popitem(self):
+ return self.r_dict_content.popitem()
implementation_methods = [
("getitem", 1),
("getitem_str", 1),
("length", 0),
- ("setitem_str", 3),
+ ("setitem_str", 2),
("setitem", 2),
("delitem", 1),
("iter", 0),
@@ -231,8 +223,7 @@
("keys", 0),
("clear", 0),
("get_builtin_indexed", 1),
- ("shadows_anything", 0),
- ("set_shadows_anything", 0),
+ ("popitem", 0),
]
@@ -310,9 +301,9 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
self.content[key] = w_value
def impl_delitem(self, w_key):
@@ -324,7 +315,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
return len(self.content)
@@ -344,7 +335,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_key)
+ return self._as_rdict().impl_fallback_getitem(w_key)
def impl_iter(self):
return StrIteratorImplementation(self.space, self)
@@ -388,47 +379,12 @@
return None, None
-class ShadowDetectingDictImplementation(StrDictImplementation):
- def __init__(self, space, w_type):
- StrDictImplementation.__init__(self, space)
- self.w_type = w_type
- self.original_version_tag = w_type.version_tag()
- if self.original_version_tag is None:
- self._shadows_anything = True
- else:
- self._shadows_anything = False
-
- def impl_setitem_str(self, key, w_value, shadows_type=True):
- if shadows_type:
- self._shadows_anything = True
- StrDictImplementation.impl_setitem_str(
- self, key, w_value, shadows_type)
-
- def impl_setitem(self, w_key, w_value):
- space = self.space
- if space.is_w(space.type(w_key), space.w_str):
- if not self._shadows_anything:
- w_obj = self.w_type.lookup(space.str_w(w_key))
- if w_obj is not None:
- self._shadows_anything = True
- StrDictImplementation.impl_setitem_str(
- self, self.space.str_w(w_key), w_value, False)
- else:
- self._as_rdict().setitem(w_key, w_value)
-
- def impl_shadows_anything(self):
- return (self._shadows_anything or
- self.w_type.version_tag() is not self.original_version_tag)
-
- def impl_set_shadows_anything(self):
- self._shadows_anything = True
-
class WaryDictImplementation(StrDictImplementation):
def __init__(self, space):
StrDictImplementation.__init__(self, space)
self.shadowed = [None] * len(BUILTIN_TO_INDEX)
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
i = BUILTIN_TO_INDEX.get(key, -1)
if i != -1:
self.shadowed[i] = w_value
@@ -446,7 +402,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_get_builtin_indexed(self, i):
return self.shadowed[i]
@@ -558,7 +514,7 @@
self.info.writes += 1
self.content[w_key] = w_value
self.info.maxcontents = max(self.info.maxcontents, len(self.content))
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
self.info.setitem_strs += 1
self.impl_setitem(self.space.wrap(key), w_value)
def impl_delitem(self, w_key):
@@ -641,30 +597,50 @@
init_signature = Signature(['seq_or_map'], None, 'kwargs')
init_defaults = [None]
-def init__DictMulti(space, w_dict, __args__):
- w_src, w_kwds = __args__.parse_obj(
- None, 'dict',
- init_signature, # signature
- init_defaults) # default argument
- if w_src is None:
- pass
- elif space.findattr(w_src, space.wrap("keys")) is None:
- list_of_w_pairs = space.listview(w_src)
- for w_pair in list_of_w_pairs:
+def update1(space, w_dict, w_data):
+ if space.findattr(w_data, space.wrap("keys")) is None:
+ # no 'keys' method, so we assume it is a sequence of pairs
+ for w_pair in space.listview(w_data):
pair = space.fixedview(w_pair)
- if len(pair)!=2:
+ if len(pair) != 2:
raise OperationError(space.w_ValueError,
- space.wrap("dict() takes a sequence of pairs"))
- w_k, w_v = pair
- w_dict.setitem(w_k, w_v)
+ space.wrap("sequence of pairs expected"))
+ w_key, w_value = pair
+ w_dict.setitem(w_key, w_value)
else:
- if space.is_true(w_src):
- from pypy.objspace.std.dicttype import update1
- update1(space, w_dict, w_src)
+ if isinstance(w_data, W_DictMultiObject): # optimization case only
+ update1_dict_dict(space, w_dict, w_data)
+ else:
+ # general case -- "for k in o.keys(): dict.__setitem__(d, k, o[k])"
+ w_keys = space.call_method(w_data, "keys")
+ for w_key in space.listview(w_keys):
+ w_value = space.getitem(w_data, w_key)
+ w_dict.setitem(w_key, w_value)
+
+def update1_dict_dict(space, w_dict, w_data):
+ iterator = w_data.iter()
+ while 1:
+ w_key, w_value = iterator.next()
+ if w_key is None:
+ break
+ w_dict.setitem(w_key, w_value)
+
+def init_or_update(space, w_dict, __args__, funcname):
+ w_src, w_kwds = __args__.parse_obj(
+ None, funcname,
+ init_signature, # signature
+ init_defaults) # default argument
+ if w_src is not None:
+ update1(space, w_dict, w_src)
if space.is_true(w_kwds):
- from pypy.objspace.std.dicttype import update1
update1(space, w_dict, w_kwds)
+def init__DictMulti(space, w_dict, __args__):
+ init_or_update(space, w_dict, __args__, 'dict')
+
+def dict_update__DictMulti(space, w_dict, __args__):
+ init_or_update(space, w_dict, __args__, 'dict.update')
+
def getitem__DictMulti_ANY(space, w_dict, w_key):
w_value = w_dict.getitem(w_key)
if w_value is not None:
@@ -758,9 +734,8 @@
return w_res
def dict_copy__DictMulti(space, w_self):
- from pypy.objspace.std.dicttype import update1
w_new = W_DictMultiObject.allocate_and_init_instance(space)
- update1(space, w_new, w_self)
+ update1_dict_dict(space, w_new, w_self)
return w_new
def dict_items__DictMulti(space, w_self):
@@ -791,6 +766,15 @@
else:
return w_default
+def dict_setdefault__DictMulti_ANY_ANY(space, w_dict, w_key, w_default):
+ # XXX should be more efficient, with only one dict lookup
+ w_value = w_dict.getitem(w_key)
+ if w_value is not None:
+ return w_value
+ else:
+ w_dict.setitem(w_key, w_default)
+ return w_default
+
def dict_pop__DictMulti_ANY(space, w_dict, w_key, w_defaults):
defaults = space.listview(w_defaults)
len_defaults = len(defaults)
@@ -808,6 +792,14 @@
w_dict.delitem(w_key)
return w_item
+def dict_popitem__DictMulti(space, w_dict):
+ try:
+ w_key, w_value = w_dict.popitem()
+ except KeyError:
+ raise OperationError(space.w_KeyError,
+ space.wrap("popitem(): dictionary is empty"))
+ return space.newtuple([w_key, w_value])
+
app = gateway.applevel('''
def dictrepr(currently_in_repr, d):
# Now we only handle one implementation of dicts, this one.
Modified: pypy/branch/arm-backend/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/dicttype.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/dicttype.py Tue Nov 23 10:12:47 2010
@@ -53,106 +53,21 @@
# This can return when multimethods have been fixed
#dict_str = StdObjSpace.str
-# default application-level implementations for some operations
-# most of these (notably not popitem and update*) are overwritten
-# in dictmultiobject
-# gateway is imported in the stdtypedef module
-app = gateway.applevel('''
-
- # in the following functions we use dict.__setitem__ instead of
- # d[k]=... because when a subclass of dict override __setitem__,
- # CPython does not call it when doing builtin operations. The
- # same for other operations.
-
- def update1(d, o):
- if hasattr(o, 'keys'):
- for k in o.keys():
- dict.__setitem__(d, k, o[k])
- else:
- for k,v in o:
- dict.__setitem__(d, k, v)
-
- def update(d, *args, **kwargs):
- len_args = len(args)
- if len_args == 1:
- update1(d, args[0])
- elif len_args > 1:
- raise TypeError("update takes at most 1 (non-keyword) argument")
- if kwargs:
- update1(d, kwargs)
-
- def popitem(d):
- for k in dict.iterkeys(d):
- break
- else:
- raise KeyError("popitem(): dictionary is empty")
- v = dict.__getitem__(d, k)
- dict.__delitem__(d, k)
- return k, v
-
- def get(d, k, v=None):
- if k in d:
- return dict.__getitem__(d, k)
- else:
- return v
-
- def setdefault(d, k, v=None):
- if k in d:
- return dict.__getitem__(d, k)
- else:
- dict.__setitem__(d, k, v)
- return v
-
- def pop(d, k, defaults): # XXX defaults is actually *defaults
- if len(defaults) > 1:
- raise TypeError, "pop expected at most 2 arguments, got %d" % (
- 1 + len(defaults))
- try:
- v = dict.__getitem__(d, k)
- dict.__delitem__(d, k)
- except KeyError, e:
- if defaults:
- return defaults[0]
- else:
- raise e
- return v
-
- def iteritems(d):
- return iter(dict.items(d))
-
- def iterkeys(d):
- return iter(dict.keys(d))
-
- def itervalues(d):
- return iter(dict.values(d))
-''', filename=__file__)
-
-dict_update__ANY = app.interphook("update")
-dict_popitem__ANY = app.interphook("popitem")
-dict_get__ANY_ANY_ANY = app.interphook("get")
-dict_setdefault__ANY_ANY_ANY = app.interphook("setdefault")
-dict_pop__ANY_ANY = app.interphook("pop")
-dict_iteritems__ANY = app.interphook("iteritems")
-dict_iterkeys__ANY = app.interphook("iterkeys")
-dict_itervalues__ANY = app.interphook("itervalues")
-update1 = app.interphook("update1")
-
register_all(vars(), globals())
@gateway.unwrap_spec(ObjSpace, W_Root, W_Root, W_Root)
def descr_fromkeys(space, w_type, w_keys, w_fill=None):
+ from pypy.objspace.std.dictmultiobject import W_DictMultiObject
if w_fill is None:
w_fill = space.w_None
- w_dict = space.call_function(w_type)
- w_iter = space.iter(w_keys)
- while True:
- try:
- w_key = space.next(w_iter)
- except OperationError, e:
- if not e.match(space, space.w_StopIteration):
- raise
- break
- space.setitem(w_dict, w_key, w_fill)
+ if w_type is space.w_dict:
+ w_dict = W_DictMultiObject.allocate_and_init_instance(space, w_type)
+ for w_key in space.listview(w_keys):
+ w_dict.setitem(w_key, w_fill)
+ else:
+ w_dict = space.call_function(w_type)
+ for w_key in space.listview(w_keys):
+ space.setitem(w_dict, w_key, w_fill)
return w_dict
Modified: pypy/branch/arm-backend/pypy/objspace/std/fake.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/fake.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/fake.py Tue Nov 23 10:12:47 2010
@@ -21,7 +21,6 @@
#debug_print("faking obj %s" % x)
ft = fake_type(type(x))
return ft(space, x)
-fake_object._annspecialcase_ = "override:fake_object"
import sys
@@ -47,7 +46,6 @@
w_exc = space.wrap(exc)
w_value = space.wrap(value)
raise OperationError, OperationError(w_exc, w_value), tb
-wrap_exception._annspecialcase_ = "override:ignore"
def fake_type(cpy_type):
assert type(cpy_type) is type
Modified: pypy/branch/arm-backend/pypy/objspace/std/frozensettype.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/frozensettype.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/frozensettype.py Tue Nov 23 10:12:47 2010
@@ -39,13 +39,13 @@
def descr__frozenset__new__(space, w_frozensettype,
w_iterable=gateway.NoneNotWrapped):
from pypy.objspace.std.setobject import W_FrozensetObject
- from pypy.objspace.std.setobject import _is_frozenset_exact
+ from pypy.objspace.std.setobject import make_setdata_from_w_iterable
if (space.is_w(w_frozensettype, space.w_frozenset) and
- _is_frozenset_exact(w_iterable)):
+ w_iterable is not None and type(w_iterable) is W_FrozensetObject):
return w_iterable
w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype)
- W_FrozensetObject.__init__(w_obj, space, None)
-
+ data = make_setdata_from_w_iterable(space, w_iterable)
+ W_FrozensetObject.__init__(w_obj, space, data)
return w_obj
frozenset_typedef = StdTypeDef("frozenset",
Modified: pypy/branch/arm-backend/pypy/objspace/std/inttype.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/inttype.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/inttype.py Tue Nov 23 10:12:47 2010
@@ -1,7 +1,7 @@
from pypy.interpreter import gateway
from pypy.interpreter.error import OperationError
from pypy.objspace.std.stdtypedef import StdTypeDef
-from pypy.objspace.std.strutil import (string_to_int, string_to_w_long,
+from pypy.objspace.std.strutil import (string_to_int, string_to_bigint,
ParseStringError,
ParseStringOverflowError)
from pypy.rlib.rarithmetic import r_uint
@@ -45,10 +45,12 @@
def retry_to_w_long(space, parser, base=0):
parser.rewind()
try:
- return string_to_w_long(space, None, base=base, parser=parser)
+ bigint = string_to_bigint(None, base=base, parser=parser)
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
+ from pypy.objspace.std.longobject import W_LongObject
+ return W_LongObject(bigint)
def descr__new__(space, w_inttype, w_x=0, w_base=gateway.NoneNotWrapped):
from pypy.objspace.std.intobject import W_IntObject
Modified: pypy/branch/arm-backend/pypy/objspace/std/listobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/listobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/listobject.py Tue Nov 23 10:12:47 2010
@@ -283,7 +283,7 @@
elif start >= 0:
del items[start:start+delta]
else:
- assert delta==0
+ assert delta==0 # start<0 is only possible with slicelength==0
elif len2 != slicelength: # No resize for extended slices
raise operationerrfmt(space.w_ValueError, "attempt to "
"assign sequence of size %d to extended slice of size %d",
Modified: pypy/branch/arm-backend/pypy/objspace/std/longtype.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/longtype.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/longtype.py Tue Nov 23 10:12:47 2010
@@ -1,7 +1,7 @@
from pypy.interpreter.error import OperationError
from pypy.interpreter import gateway
from pypy.objspace.std.stdtypedef import StdTypeDef
-from pypy.objspace.std.strutil import string_to_w_long, ParseStringError
+from pypy.objspace.std.strutil import string_to_bigint, ParseStringError
def descr__new__(space, w_longtype, w_x=0, w_base=gateway.NoneNotWrapped):
from pypy.objspace.std.longobject import W_LongObject
@@ -9,10 +9,10 @@
if w_base is None:
# check for easy cases
if type(w_value) is W_LongObject:
- pass
+ bigint = w_value.num
elif space.is_true(space.isinstance(w_value, space.w_str)):
try:
- w_value = string_to_w_long(space, space.str_w(w_value))
+ bigint = string_to_bigint(space.str_w(w_value))
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
@@ -22,7 +22,7 @@
from pypy.objspace.std.ropeunicodeobject import unicode_to_decimal_w
else:
from pypy.objspace.std.unicodeobject import unicode_to_decimal_w
- w_value = string_to_w_long(space, unicode_to_decimal_w(space, w_value))
+ bigint = string_to_bigint(unicode_to_decimal_w(space, w_value))
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
@@ -35,10 +35,11 @@
if space.is_true(space.isinstance(w_obj, space.w_long)):
assert isinstance(w_obj, W_LongObject) # XXX this could fail!
# XXX find a way to do that even if w_obj is not a W_LongObject
- w_value = w_obj
+ bigint = w_obj.num
elif space.is_true(space.isinstance(w_obj, space.w_int)):
+ from pypy.rlib.rbigint import rbigint
intval = space.int_w(w_obj)
- w_value = W_LongObject.fromint(space, intval)
+ bigint = rbigint.fromint(intval)
else:
raise OperationError(space.w_ValueError,
space.wrap("value can't be converted to long"))
@@ -56,13 +57,13 @@
space.wrap("long() can't convert non-string "
"with explicit base"))
try:
- w_value = string_to_w_long(space, s, base)
+ bigint = string_to_bigint(s, base)
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
w_obj = space.allocate_instance(W_LongObject, w_longtype)
- W_LongObject.__init__(w_obj, w_value.num)
+ W_LongObject.__init__(w_obj, bigint)
return w_obj
# ____________________________________________________________
Modified: pypy/branch/arm-backend/pypy/objspace/std/model.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/model.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/model.py Tue Nov 23 10:12:47 2010
@@ -18,6 +18,7 @@
"withsmallint" : ["smallintobject.W_SmallIntObject"],
"withstrslice" : ["strsliceobject.W_StringSliceObject"],
"withstrjoin" : ["strjoinobject.W_StringJoinObject"],
+ "withstrbuf" : ["strbufobject.W_StringBufferObject"],
"withrope" : ["ropeobject.W_RopeObject",
"ropeobject.W_RopeIterObject"],
"withropeunicode": ["ropeunicodeobject.W_RopeUnicodeObject",
@@ -75,6 +76,7 @@
from pypy.objspace.std import ropeunicodeobject
from pypy.objspace.std import strsliceobject
from pypy.objspace.std import strjoinobject
+ from pypy.objspace.std import strbufobject
from pypy.objspace.std import typeobject
from pypy.objspace.std import sliceobject
from pypy.objspace.std import longobject
@@ -222,6 +224,13 @@
(unicodeobject.W_UnicodeObject,
strjoinobject.delegate_join2unicode)
]
+ elif config.objspace.std.withstrbuf:
+ self.typeorder[strbufobject.W_StringBufferObject] += [
+ (stringobject.W_StringObject,
+ strbufobject.delegate_buf2str),
+ (unicodeobject.W_UnicodeObject,
+ strbufobject.delegate_buf2unicode)
+ ]
if config.objspace.std.withrangelist:
self.typeorder[rangeobject.W_RangeListObject] += [
(listobject.W_ListObject,
Modified: pypy/branch/arm-backend/pypy/objspace/std/objspace.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/objspace.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/objspace.py Tue Nov 23 10:12:47 2010
@@ -23,6 +23,7 @@
from pypy.objspace.std.listobject import W_ListObject
from pypy.objspace.std.longobject import W_LongObject
from pypy.objspace.std.noneobject import W_NoneObject
+from pypy.objspace.std.objectobject import W_ObjectObject
from pypy.objspace.std.ropeobject import W_RopeObject
from pypy.objspace.std.iterobject import W_SeqIterObject
from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
@@ -242,7 +243,6 @@
w_result = getattr(self, 'w_' + x.__name__)
return w_result
return None
- wrap_exception_cls._annspecialcase_ = "override:wrap_exception_cls"
def unwrap(self, w_obj):
if isinstance(w_obj, Wrappable):
@@ -318,9 +318,14 @@
w_subtype = w_type.check_user_subclass(w_subtype)
if cls.typedef.applevel_subclasses_base is not None:
cls = cls.typedef.applevel_subclasses_base
- subcls = get_unique_interplevel_subclass(
- self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0,
- w_subtype.needsdel, w_subtype.weakrefable)
+ if (self.config.objspace.std.withmapdict and cls is W_ObjectObject
+ and not w_subtype.needsdel):
+ from pypy.objspace.std.mapdict import get_subclass_of_correct_size
+ subcls = get_subclass_of_correct_size(self, cls, w_subtype)
+ else:
+ subcls = get_unique_interplevel_subclass(
+ self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0,
+ w_subtype.needsdel, w_subtype.weakrefable)
instance = instantiate(subcls)
assert isinstance(instance, cls)
instance.user_setup(self, w_subtype)
@@ -432,7 +437,7 @@
if is_data:
w_get = self.lookup(w_descr, "__get__")
if w_get is None:
- w_value = w_obj.getdictvalue_attr_is_in_class(self, name)
+ w_value = w_obj.getdictvalue(self, name)
if w_value is not None:
return w_value
if not is_data:
@@ -484,14 +489,12 @@
return w_obj.getitem(w_key)
return ObjSpace.finditem(self, w_obj, w_key)
- def setitem_str(self, w_obj, key, w_value, shadows_type=True):
+ def setitem_str(self, w_obj, key, w_value):
""" Same as setitem, but takes string instead of any wrapped object
-
- XXX what shadows_type means???
"""
if (isinstance(w_obj, W_DictMultiObject) and
not w_obj.user_overridden_class):
- w_obj.setitem_str(key, w_value, shadows_type)
+ w_obj.setitem_str(key, w_value)
else:
self.setitem(w_obj, self.wrap(key), w_value)
Modified: pypy/branch/arm-backend/pypy/objspace/std/proxyobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/proxyobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/proxyobject.py Tue Nov 23 10:12:47 2010
@@ -43,7 +43,7 @@
raise
return None
- def setdictvalue(self, space, attr, w_value, shadows_type=True):
+ def setdictvalue(self, space, attr, w_value):
try:
space.call_function(self.w_controller, space.wrap('__setattr__'),
space.wrap(attr), w_value)
Modified: pypy/branch/arm-backend/pypy/objspace/std/setobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/setobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/setobject.py Tue Nov 23 10:12:47 2010
@@ -21,11 +21,10 @@
return False
- def __init__(w_self, space, setdata=None):
- if setdata is None:
- w_self.setdata = r_dict(space.eq_w, space.hash_w)
- else:
- w_self.setdata = setdata.copy()
+ def __init__(w_self, space, setdata):
+ """Initialize the set by taking ownership of 'setdata'."""
+ assert setdata is not None
+ w_self.setdata = setdata
def __repr__(w_self):
"""representation for debugging purposes"""
@@ -33,6 +32,7 @@
return "<%s(%s)>" % (w_self.__class__.__name__, ', '.join(reprlist))
def _newobj(w_self, space, rdict_w=None):
+ """Make a new set or frozenset by taking ownership of 'rdict_w'."""
#return space.call(space.type(w_self),W_SetIterObject(rdict_w))
objtype = type(w_self)
if objtype is W_SetObject:
@@ -55,10 +55,7 @@
class W_FrozensetObject(W_BaseSetObject):
from pypy.objspace.std.frozensettype import frozenset_typedef as typedef
-
- def __init__(w_self, space, setdata):
- W_BaseSetObject.__init__(w_self, space, setdata)
- w_self.hash = -1
+ hash = 0
registerimplementation(W_BaseSetObject)
registerimplementation(W_SetObject)
@@ -109,8 +106,14 @@
# some helper functions
+def newset(space):
+ return r_dict(space.eq_w, space.hash_w)
+
def make_setdata_from_w_iterable(space, w_iterable=None):
- data = r_dict(space.eq_w, space.hash_w)
+ """Return a new r_dict with the content of w_iterable."""
+ if isinstance(w_iterable, W_BaseSetObject):
+ return w_iterable.setdata.copy()
+ data = newset(space)
if w_iterable is not None:
for w_item in space.listview(w_iterable):
data[w_item] = None
@@ -119,7 +122,7 @@
def _initialize_set(space, w_obj, w_iterable=None):
w_obj.setdata.clear()
if w_iterable is not None:
- w_obj.setdata.update(make_setdata_from_w_iterable(space, w_iterable))
+ w_obj.setdata = make_setdata_from_w_iterable(space, w_iterable)
def _convert_set_to_frozenset(space, w_obj):
if space.is_true(space.isinstance(w_obj, space.w_set)):
@@ -130,12 +133,6 @@
# helper functions for set operation on dicts
-def _is_frozenset_exact(w_obj):
- if (w_obj is not None) and (type(w_obj) is W_FrozensetObject):
- return True
- else:
- return False
-
def _is_eq(ld, rd):
if len(ld) != len(rd):
return False
@@ -144,66 +141,41 @@
return False
return True
-def _union_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- ld.update(rdict)
- return ld, rdict
-
-def _difference_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- del_list_w = []
+def _difference_dict(space, ld, rd):
+ result = newset(space)
for w_key in ld:
- if w_key in rdict:
- del_list_w.append(w_key)
- for w_key in del_list_w:
- del ld[w_key]
-
- return ld, rdict
-
-def _intersection_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- del_list_w = []
- for w_key in ld:
- if w_key not in rdict:
- del_list_w.append(w_key)
-
- for w_key in del_list_w:
- del ld[w_key]
-
- return ld, rdict
-
+ if w_key not in rd:
+ result[w_key] = None
+ return result
-def _symmetric_difference_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
+def _difference_dict_update(space, ld, rd):
+ if ld is rd:
+ ld.clear() # for the case 'a.difference_update(a)'
else:
- ld = ldict.copy()
- del_list_w = []
- add_list_w = []
+ for w_key in rd:
+ try:
+ del ld[w_key]
+ except KeyError:
+ pass
+
+def _intersection_dict(space, ld, rd):
+ result = newset(space)
+ if len(ld) > len(rd):
+ ld, rd = rd, ld # loop over the smaller dict
for w_key in ld:
- if w_key in rdict:
- del_list_w.append(w_key)
+ if w_key in rd:
+ result[w_key] = None
+ return result
- for w_key in rdict:
+def _symmetric_difference_dict(space, ld, rd):
+ result = newset(space)
+ for w_key in ld:
+ if w_key not in rd:
+ result[w_key] = None
+ for w_key in rd:
if w_key not in ld:
- add_list_w.append(w_key)
-
- for w_key in del_list_w:
- del ld[w_key]
-
- for w_key in add_list_w:
- ld[w_key] = None
-
- return ld, rdict
+ result[w_key] = None
+ return result
def _issubset_dict(ldict, rdict):
if len(ldict) > len(rdict):
@@ -220,14 +192,13 @@
def set_update__Set_BaseSet(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _union_dict(ld, rd, True)
- return space.w_None
+ ld.update(rd)
def set_update__Set_ANY(space, w_left, w_other):
"""Update a set with the union of itself and another."""
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _union_dict(ld, rd, True)
- return space.w_None
+ ld = w_left.setdata
+ for w_item in space.listview(w_other):
+ ld[w_item] = None
def inplace_or__Set_Set(space, w_left, w_other):
set_update__Set_BaseSet(space, w_left, w_other)
@@ -241,25 +212,23 @@
This has no effect if the element is already present.
"""
w_left.setdata[w_other] = None
- return space.w_None
def set_copy__Set(space, w_set):
- return w_set._newobj(space,w_set.setdata)
+ return w_set._newobj(space, w_set.setdata.copy())
def frozenset_copy__Frozenset(space, w_left):
- if _is_frozenset_exact(w_left):
+ if type(w_left) is W_FrozensetObject:
return w_left
else:
- return set_copy__Set(space,w_left)
+ return set_copy__Set(space, w_left)
def set_clear__Set(space, w_left):
w_left.setdata.clear()
- return space.w_None
def set_difference__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _difference_dict(ld, rd, False)
+ new_ld = _difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
set_difference__Set_Frozenset = set_difference__Set_Set
@@ -272,7 +241,7 @@
def set_difference__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _difference_dict(ld, rd, False)
+ new_ld = _difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
frozenset_difference__Frozenset_ANY = set_difference__Set_ANY
@@ -281,15 +250,17 @@
def set_difference_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _difference_dict(ld, rd, True)
- return space.w_None
+ _difference_dict_update(space, ld, rd)
set_difference_update__Set_Frozenset = set_difference_update__Set_Set
def set_difference_update__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _difference_dict(ld, rd, True)
- return space.w_None
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ try:
+ del ld[w_key]
+ except KeyError:
+ pass
def inplace_sub__Set_Set(space, w_left, w_other):
set_difference_update__Set_Set(space, w_left, w_other)
@@ -467,7 +438,7 @@
def hash__Frozenset(space, w_set):
multi = r_uint(1822399083) + r_uint(1822399083) + 1
- if w_set.hash != -1:
+ if w_set.hash != 0:
return space.wrap(w_set.hash)
hash = 1927868237
hash *= (len(w_set.setdata) + 1)
@@ -476,7 +447,7 @@
value = ((h ^ (h << 16) ^ 89869747) * multi)
hash = intmask(hash ^ value)
hash = hash * 69069 + 907133923
- if hash == -1:
+ if hash == 0:
hash = 590923713
hash = intmask(hash)
w_set.hash = hash
@@ -484,29 +455,31 @@
return space.wrap(hash)
def set_pop__Set(space, w_left):
- if len(w_left.setdata) == 0:
+ for w_key in w_left.setdata:
+ break
+ else:
raise OperationError(space.w_KeyError,
space.wrap('pop from an empty set'))
- w_keys = w_left.setdata.keys()
- w_value = w_keys[0]
- del w_left.setdata[w_value]
-
- return w_value
+ del w_left.setdata[w_key]
+ return w_key
def set_intersection__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _intersection_dict(ld, rd, False)
- return w_left._newobj(space,new_ld)
+ new_ld = _intersection_dict(space, ld, rd)
+ return w_left._newobj(space, new_ld)
set_intersection__Set_Frozenset = set_intersection__Set_Set
set_intersection__Frozenset_Frozenset = set_intersection__Set_Set
set_intersection__Frozenset_Set = set_intersection__Set_Set
def set_intersection__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _intersection_dict(ld, rd, False)
- return w_left._newobj(space,new_ld)
+ result = newset(space)
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ if w_key in ld:
+ result[w_key] = None
+ return w_left._newobj(space, result)
frozenset_intersection__Frozenset_ANY = set_intersection__Set_ANY
@@ -518,15 +491,18 @@
def set_intersection_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _intersection_dict(ld, rd, True)
- return space.w_None
+ new_ld = _intersection_dict(space, ld, rd)
+ w_left.setdata = new_ld
set_intersection_update__Set_Frozenset = set_intersection_update__Set_Set
def set_intersection_update__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _intersection_dict(ld, rd, True)
- return space.w_None
+ result = newset(space)
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ if w_key in ld:
+ result[w_key] = None
+ w_left.setdata = result
def inplace_and__Set_Set(space, w_left, w_other):
set_intersection_update__Set_Set(space, w_left, w_other)
@@ -537,7 +513,7 @@
def set_symmetric_difference__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _symmetric_difference_dict(ld, rd, False)
+ new_ld = _symmetric_difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
set_symmetric_difference__Set_Frozenset = set_symmetric_difference__Set_Set
@@ -553,7 +529,7 @@
def set_symmetric_difference__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _symmetric_difference_dict(ld, rd, False)
+ new_ld = _symmetric_difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
frozenset_symmetric_difference__Frozenset_ANY = \
@@ -562,16 +538,16 @@
def set_symmetric_difference_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _symmetric_difference_dict(ld, rd, True)
- return space.w_None
+ new_ld = _symmetric_difference_dict(space, ld, rd)
+ w_left.setdata = new_ld
set_symmetric_difference_update__Set_Frozenset = \
set_symmetric_difference_update__Set_Set
def set_symmetric_difference_update__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _symmetric_difference_dict(ld, rd, True)
- return space.w_None
+ new_ld = _symmetric_difference_dict(space, ld, rd)
+ w_left.setdata = new_ld
def inplace_xor__Set_Set(space, w_left, w_other):
set_symmetric_difference_update__Set_Set(space, w_left, w_other)
@@ -582,8 +558,9 @@
def set_union__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _union_dict(ld, rd, False)
- return w_left._newobj(space, new_ld)
+ result = ld.copy()
+ result.update(rd)
+ return w_left._newobj(space, result)
set_union__Set_Frozenset = set_union__Set_Set
set_union__Frozenset_Set = set_union__Set_Set
@@ -595,9 +572,11 @@
def set_union__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _union_dict(ld, rd, False)
- return w_left._newobj(space, new_ld)
+ ld = w_left.setdata
+ result = ld.copy()
+ for w_key in space.listview(w_other):
+ result[w_key] = None
+ return w_left._newobj(space, result)
frozenset_union__Frozenset_ANY = set_union__Set_ANY
@@ -629,15 +608,6 @@
init_defaults)
_initialize_set(space, w_set, w_iterable)
-def init__Frozenset(space, w_set, __args__):
- w_iterable, = __args__.parse_obj(
- None, 'set',
- init_signature,
- init_defaults)
- if w_set.hash == -1:
- _initialize_set(space, w_set, w_iterable)
- hash__Frozenset(space, w_set)
-
app = gateway.applevel("""
def setrepr(currently_in_repr, s):
'The app-level part of repr().'
Modified: pypy/branch/arm-backend/pypy/objspace/std/settype.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/settype.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/settype.py Tue Nov 23 10:12:47 2010
@@ -66,9 +66,9 @@
register_all(vars(), globals())
def descr__new__(space, w_settype, __args__):
- from pypy.objspace.std.setobject import W_SetObject
+ from pypy.objspace.std.setobject import W_SetObject, newset
w_obj = space.allocate_instance(W_SetObject, w_settype)
- W_SetObject.__init__(w_obj, space, None)
+ W_SetObject.__init__(w_obj, space, newset(space))
return w_obj
set_typedef = StdTypeDef("set",
Modified: pypy/branch/arm-backend/pypy/objspace/std/smallintobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/smallintobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/smallintobject.py Tue Nov 23 10:12:47 2010
@@ -2,18 +2,13 @@
Implementation of small ints, stored as odd-valued pointers in the
translated PyPy. To enable them, see inttype.py.
"""
-import types
-from pypy.interpreter.error import OperationError
from pypy.objspace.std import intobject
from pypy.objspace.std.model import registerimplementation, W_Object
from pypy.objspace.std.register_all import register_all
-from pypy.objspace.std.multimethod import FailedToImplementArgs
from pypy.objspace.std.noneobject import W_NoneObject
-from pypy.rlib.rarithmetic import ovfcheck, ovfcheck_lshift, LONG_BIT, r_uint
-from pypy.objspace.std.inttype import wrapint
-from pypy.objspace.std.intobject import W_IntObject, _impl_int_int_pow
+from pypy.objspace.std.intobject import W_IntObject
from pypy.rlib.objectmodel import UnboxedValue
-from pypy.rlib.rbigint import rbigint
+from pypy.tool.sourcetools import func_with_new_name
class W_SmallIntObject(W_Object, UnboxedValue):
@@ -46,9 +41,7 @@
new_name = name.replace("Int", "SmallInt")
# Copy the function, so the annotator specializes it for
# W_SmallIntObject.
- ns[new_name] = types.FunctionType(func.func_code, ns, new_name,
- func.func_defaults,
- func.func_closure)
+ ns[new_name] = func_with_new_name(func, new_name)
ns["get_integer"] = ns["pos__SmallInt"] = ns["int__SmallInt"]
ns["get_negint"] = ns["neg__SmallInt"]
Modified: pypy/branch/arm-backend/pypy/objspace/std/stringobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/stringobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/stringobject.py Tue Nov 23 10:12:47 2010
@@ -14,7 +14,7 @@
from pypy.rlib.rstring import StringBuilder, string_repeat
from pypy.interpreter.buffer import StringBuffer
-from pypy.objspace.std.stringtype import sliced, joined, wrapstr, wrapchar, \
+from pypy.objspace.std.stringtype import sliced, wrapstr, wrapchar, \
stringendswith, stringstartswith, joined2
from pypy.objspace.std.formatting import mod_format
@@ -937,10 +937,17 @@
string = w_string._value
chars = []
- for char in string:
- w_char = W_StringObject.PREBUILT[ord(char)]
- if not space.is_true(space.contains(w_deletechars, w_char)):
- chars.append(table[ord(char)])
+ deletechars = space.str_w(w_deletechars)
+ if len(deletechars) == 0:
+ for char in string:
+ chars.append(table[ord(char)])
+ else:
+ deletion_table = [False] * 256
+ for c in deletechars:
+ deletion_table[ord(c)] = True
+ for char in string:
+ if not deletion_table[ord(char)]:
+ chars.append(table[ord(char)])
return W_StringObject(''.join(chars))
def str_decode__String_ANY_ANY(space, w_string, w_encoding=None, w_errors=None):
Modified: pypy/branch/arm-backend/pypy/objspace/std/stringtype.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/stringtype.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/stringtype.py Tue Nov 23 10:12:47 2010
@@ -55,19 +55,14 @@
return W_StringSliceObject(s, start, stop)
return wrapstr(space, s[start:stop])
-def joined(space, strlist):
- assert not space.config.objspace.std.withrope
- if space.config.objspace.std.withstrjoin:
- from pypy.objspace.std.strjoinobject import W_StringJoinObject
- return W_StringJoinObject(strlist)
- else:
- return wrapstr(space, "".join(strlist))
-
def joined2(space, str1, str2):
assert not space.config.objspace.std.withrope
if space.config.objspace.std.withstrjoin:
from pypy.objspace.std.strjoinobject import W_StringJoinObject
return W_StringJoinObject([str1, str2])
+ elif space.config.objspace.std.withstrbuf:
+ from pypy.objspace.std.strbufobject import joined2
+ return joined2(str1, str2)
else:
return wrapstr(space, str1 + str2)
Modified: pypy/branch/arm-backend/pypy/objspace/std/strutil.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/strutil.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/strutil.py Tue Nov 23 10:12:47 2010
@@ -4,6 +4,7 @@
from pypy.rlib.rarithmetic import ovfcheck, break_up_float, parts_to_float,\
INFINITY, NAN
+from pypy.rlib.rbigint import rbigint, parse_digit_string
from pypy.interpreter.error import OperationError
import math
@@ -91,9 +92,10 @@
return -1
def string_to_int(s, base=10):
- """Utility to converts a string to an integer (or possibly a long).
+ """Utility to converts a string to an integer.
If base is 0, the proper base is guessed based on the leading
characters of 's'. Raises ParseStringError in case of error.
+ Raises ParseStringOverflowError in case the result does not fit.
"""
s = literal = strip_spaces(s)
p = NumberStringParser(s, literal, base, 'int')
@@ -113,11 +115,9 @@
except OverflowError:
raise ParseStringOverflowError(p)
-def string_to_long(space, s, base=10, parser=None):
- return string_to_w_long(space, s, base, parser).longval()
-
-def string_to_w_long(space, s, base=10, parser=None):
- """As string_to_int(), but ignores an optional 'l' or 'L' suffix."""
+def string_to_bigint(s, base=10, parser=None):
+ """As string_to_int(), but ignores an optional 'l' or 'L' suffix
+ and returns an rbigint."""
if parser is None:
s = literal = strip_spaces(s)
if (s.endswith('l') or s.endswith('L')) and base < 22:
@@ -126,18 +126,7 @@
p = NumberStringParser(s, literal, base, 'long')
else:
p = parser
- w_base = space.newlong(p.base)
- w_result = space.newlong(0)
- while True:
- digit = p.next_digit()
- if digit == -1:
- if p.sign == -1:
- w_result = space.neg(w_result)
- # XXX grumble
- from pypy.objspace.std.longobject import W_LongObject
- assert isinstance(w_result, W_LongObject)
- return w_result
- w_result = space.add(space.mul(w_result,w_base), space.newlong(digit))
+ return parse_digit_string(p)
# Tim's comment:
# 57 bits are more than needed in any case.
Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/test/test_dictmultiobject.py Tue Nov 23 10:12:47 2010
@@ -4,7 +4,6 @@
StrDictImplementation
from pypy.objspace.std.celldict import ModuleDictImplementation
-from pypy.objspace.std.sharingdict import SharedDictImplementation
from pypy.conftest import gettestobjspace
@@ -238,7 +237,17 @@
it1 = d.popitem()
assert len(d) == 0
assert (it!=it1) and (it1==(1,2) or it1==(3,4))
-
+ raises(KeyError, d.popitem)
+
+ def test_popitem_2(self):
+ class A(object):
+ pass
+ d = A().__dict__
+ d['x'] = 5
+ it1 = d.popitem()
+ assert it1 == ('x', 5)
+ raises(KeyError, d.popitem)
+
def test_setdefault(self):
d = {1:2, 3:4}
dd = d.copy()
@@ -446,7 +455,9 @@
d1 = {}
d2 = D(a='foo')
d1.update(d2)
- assert d1['a'] == 42 # fails on CPython, d1['a'] == 'foo'
+ assert d1['a'] == 'foo'
+ # a bit of an obscure case: now (from r78295) we get the same result
+ # as CPython does
def test_index_keyerror_unpacking(self):
d = {}
@@ -510,32 +521,6 @@
assert getattr(a, s) == 42
-class TestW_DictSharing(TestW_DictObject):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
-
-class AppTest_DictSharing(AppTest_DictObject):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
-
- def test_values_does_not_share(self):
- class A(object):
- pass
- a = A()
- a.abc = 12
- l = a.__dict__.values()
- assert l == [12]
- l[0] = 24
- assert a.abc == 12
-
- def test_items(self):
- class A(object):
- pass
- a = A()
- a.abc = 12
- a.__dict__.items() == [("abc", 12)]
-
-
class AppTestModuleDict(object):
def setup_class(cls):
cls.space = gettestobjspace(**{"objspace.std.withcelldict": True})
@@ -602,6 +587,15 @@
classofinstance=classofinstance,
from_strdict_shared=from_strdict_shared)
+ def finditem_str(self, w_dict, s):
+ return w_dict.getitem_str(s) # assume it's a multidict
+
+ def setitem_str(self, w_dict, s, w_value):
+ return w_dict.setitem_str(s, w_value) # assume it's a multidict
+
+ def delitem(self, w_dict, w_s):
+ return w_dict.delitem(w_s) # assume it's a multidict
+
def allocate_instance(self, cls, type):
return object.__new__(cls)
@@ -611,7 +605,7 @@
w_StopIteration = StopIteration
w_None = None
StringObjectCls = FakeString
- w_dict = None
+ w_dict = W_DictMultiObject
iter = iter
fixedview = list
listview = list
@@ -620,10 +614,9 @@
class objspace:
class std:
withdictmeasurement = False
- withsharingdict = False
withsmalldicts = False
withcelldict = False
- withshadowtracking = False
+ withmethodcache = False
class opcodes:
CALL_LIKELY_BUILTIN = False
@@ -687,6 +680,14 @@
assert self.impl.length() == 0
self.check_not_devolved()
+ def test_clear(self):
+ self.fill_impl()
+ assert self.impl.length() == 2
+ self.impl.clear()
+ assert self.impl.length() == 0
+ self.check_not_devolved()
+
+
def test_keys(self):
self.fill_impl()
keys = self.impl.keys()
@@ -750,9 +751,6 @@
string = "int"
string2 = "isinstance"
-class TestSharedDictImplementation(BaseTestRDictImplementation):
- ImplementionClass = SharedDictImplementation
-
class BaseTestDevolvedDictImplementation(BaseTestRDictImplementation):
def fill_impl(self):
@@ -774,8 +772,6 @@
string = "int"
string2 = "isinstance"
-class TestDevolvedSharedDictImplementation(BaseTestDevolvedDictImplementation):
- ImplementionClass = SharedDictImplementation
def test_module_uses_strdict():
fakespace = FakeSpace()
Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_longobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/test/test_longobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/test/test_longobject.py Tue Nov 23 10:12:47 2010
@@ -41,56 +41,64 @@
class AppTestLong:
def test_add(self):
- assert int(123L + 12443L) == 123 + 12443
- assert -20 + 2 + 3L + True == -14L
+ x = 123L
+ assert int(x + 12443L) == 123 + 12443
+ x = -20
+ assert x + 2 + 3L + True == -14L
def test_sub(self):
- assert int(58543L - 12332L) == 58543 - 12332
- assert 237123838281233L * 12 == 237123838281233L * 12L
+ x = 58543L
+ assert int(x - 12332L) == 58543 - 12332
+ x = 237123838281233L
+ assert x * 12 == x * 12L
def test_mul(self):
- assert 363L * 2 ** 40 == 363L << 40
+ x = 363L
+ assert x * 2 ** 40 == x << 40
def test_truediv(self):
exec "from __future__ import division; a = 31415926L / 10000000L"
assert a == 3.1415926
def test_floordiv(self):
- a = 31415926L // 10000000L
+ x = 31415926L
+ a = x // 10000000L
assert a == 3L
def test_compare(self):
+ Z = 0
+ ZL = 0L
for BIG in (1L, 1L << 62, 1L << 9999):
- assert 0 == 0L
- assert not (0 != 0L)
- assert 0L == 0
- assert not (0L != 0)
- assert not (0 == BIG)
- assert 0 != BIG
- assert not (BIG == 0)
- assert BIG != 0
- assert not (0L == BIG)
- assert 0L != BIG
- assert 0 <= 0L
- assert not (0 < 0L)
- assert 0 <= BIG
- assert 0 < BIG
- assert not (BIG <= 0)
- assert not (BIG < 0)
- assert 0L <= 0L
- assert not (0L < 0L)
- assert 0L <= BIG
- assert 0L < BIG
- assert not (BIG <= 0L)
- assert not (BIG < 0L)
- assert not (0 <= -BIG)
- assert not (0 < -BIG)
- assert -BIG <= 0
- assert -BIG < 0
- assert not (0L <= -BIG)
- assert not (0L < -BIG)
- assert -BIG <= 0L
- assert -BIG < 0L
+ assert Z == ZL
+ assert not (Z != ZL)
+ assert ZL == Z
+ assert not (ZL != Z)
+ assert not (Z == BIG)
+ assert Z != BIG
+ assert not (BIG == Z)
+ assert BIG != Z
+ assert not (ZL == BIG)
+ assert ZL != BIG
+ assert Z <= ZL
+ assert not (Z < ZL)
+ assert Z <= BIG
+ assert Z < BIG
+ assert not (BIG <= Z)
+ assert not (BIG < Z)
+ assert ZL <= ZL
+ assert not (ZL < ZL)
+ assert ZL <= BIG
+ assert ZL < BIG
+ assert not (BIG <= ZL)
+ assert not (BIG < ZL)
+ assert not (Z <= -BIG)
+ assert not (Z < -BIG)
+ assert -BIG <= Z
+ assert -BIG < Z
+ assert not (ZL <= -BIG)
+ assert not (ZL < -BIG)
+ assert -BIG <= ZL
+ assert -BIG < ZL
#
assert not (BIG < int(BIG))
assert (BIG <= int(BIG))
@@ -137,7 +145,8 @@
def test_conversion(self):
class long2(long):
pass
- x = long2(1L<<100)
+ x = 1L
+ x = long2(x<<100)
y = int(x)
assert type(y) == long
assert type(+long2(5)) is long
@@ -152,7 +161,8 @@
assert type(long2(5) // 1) is long
def test_pow(self):
- assert pow(0L, 0L, 1L) == 0L
+ x = 0L
+ assert pow(x, 0L, 1L) == 0L
def test_getnewargs(self):
assert 0L .__getnewargs__() == (0L,)
@@ -193,13 +203,14 @@
assert oct(01234567012345670L) == '01234567012345670L'
def test_bits(self):
- assert 0xAAAAAAAAL | 0x55555555L == 0xFFFFFFFFL
- assert 0xAAAAAAAAL & 0x55555555L == 0x00000000L
- assert 0xAAAAAAAAL ^ 0x55555555L == 0xFFFFFFFFL
- assert -0xAAAAAAAAL | 0x55555555L == -0xAAAAAAA9L
- assert 0xAAAAAAAAL | 0x555555555L == 0x5FFFFFFFFL
- assert 0xAAAAAAAAL & 0x555555555L == 0x000000000L
- assert 0xAAAAAAAAL ^ 0x555555555L == 0x5FFFFFFFFL
+ x = 0xAAAAAAAAL
+ assert x | 0x55555555L == 0xFFFFFFFFL
+ assert x & 0x55555555L == 0x00000000L
+ assert x ^ 0x55555555L == 0xFFFFFFFFL
+ assert -x | 0x55555555L == -0xAAAAAAA9L
+ assert x | 0x555555555L == 0x5FFFFFFFFL
+ assert x & 0x555555555L == 0x000000000L
+ assert x ^ 0x555555555L == 0x5FFFFFFFFL
def test_hash(self):
# ints have the same hash as equal longs
@@ -229,7 +240,8 @@
def test_huge_longs(self):
import operator
- huge = 1L << 40000L
+ x = 1L
+ huge = x << 40000L
raises(OverflowError, float, huge)
raises(OverflowError, operator.truediv, huge, 3)
raises(OverflowError, operator.truediv, huge, 3L)
@@ -242,3 +254,7 @@
class myotherlong(long):
pass
assert long(myotherlong(21)) == 21L
+
+ def test_negative_zero(self):
+ x = eval("-0L")
+ assert x == 0L
Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_setobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/test/test_setobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/test/test_setobject.py Tue Nov 23 10:12:47 2010
@@ -10,6 +10,7 @@
import py.test
from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
from pypy.objspace.std.setobject import _initialize_set
+from pypy.objspace.std.setobject import newset, make_setdata_from_w_iterable
from pypy.objspace.std.setobject import set_intersection__Set_Set
from pypy.objspace.std.setobject import set_intersection__Set_ANY
from pypy.objspace.std.setobject import eq__Set_Set
@@ -28,12 +29,12 @@
self.false = self.space.w_False
def test_and(self):
- s = W_SetObject(self.space, None)
+ s = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, s, self.word)
- t0 = W_SetObject(self.space, None)
+ t0 = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, t0, self.otherword)
- t1 = W_FrozensetObject(self.space, None)
- _initialize_set(self.space, t1, self.otherword)
+ t1 = W_FrozensetObject(self.space,
+ make_setdata_from_w_iterable(self.space, self.otherword))
r0 = set_intersection__Set_Set(self.space, s, t0)
r1 = set_intersection__Set_Set(self.space, s, t1)
assert eq__Set_Set(self.space, r0, r1) == self.true
@@ -41,9 +42,9 @@
assert eq__Set_Set(self.space, r0, sr) == self.true
def test_compare(self):
- s = W_SetObject(self.space, None)
+ s = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, s, self.word)
- t = W_SetObject(self.space, None)
+ t = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, t, self.word)
assert self.space.eq_w(s,t)
u = self.space.wrap(set('simsalabim'))
@@ -56,6 +57,11 @@
b = a | set('abc')
assert type(b) is subset
+ def test_union(self):
+ a = set([4, 5])
+ b = a.union([5, 7])
+ assert sorted(b) == [4, 5, 7]
+
def test_compare(self):
raises(TypeError, cmp, set('abc'), set('abd'))
assert set('abc') != 'abc'
Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_strutil.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/test/test_strutil.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/test/test_strutil.py Tue Nov 23 10:12:47 2010
@@ -25,7 +25,7 @@
]
for s, expected in cases:
assert string_to_int(s) == expected
- assert string_to_w_long(space, s).longval() == expected
+ assert string_to_bigint(s).tolong() == expected
def test_string_to_int_base(self):
space = self.space
@@ -119,17 +119,16 @@
raises(ParseStringError, string_to_int, '+'+s, base)
raises(ParseStringError, string_to_int, '-'+s, base)
- def test_string_to_w_long(self):
- space = self.space
- assert string_to_w_long(space, '123L').longval() == 123
- assert string_to_w_long(space, '123L ').longval() == 123
- raises(ParseStringError, string_to_w_long, space, 'L')
- raises(ParseStringError, string_to_w_long, space, 'L ')
- assert string_to_w_long(space, '123L', 4).longval() == 27
- assert string_to_w_long(space, '123L', 30).longval() == 27000 + 1800 + 90 + 21
- assert string_to_w_long(space, '123L', 22).longval() == 10648 + 968 + 66 + 21
- assert string_to_w_long(space, '123L', 21).longval() == 441 + 42 + 3
- assert string_to_w_long(space, '1891234174197319').longval() == 1891234174197319
+ def test_string_to_bigint(self):
+ assert string_to_bigint('123L').tolong() == 123
+ assert string_to_bigint('123L ').tolong() == 123
+ raises(ParseStringError, string_to_bigint, 'L')
+ raises(ParseStringError, string_to_bigint, 'L ')
+ assert string_to_bigint('123L', 4).tolong() == 27
+ assert string_to_bigint('123L', 30).tolong() == 27000 + 1800 + 90 + 21
+ assert string_to_bigint('123L', 22).tolong() == 10648 + 968 + 66 + 21
+ assert string_to_bigint('123L', 21).tolong() == 441 + 42 + 3
+ assert string_to_bigint('1891234174197319').tolong() == 1891234174197319
def test_string_to_float(self):
def string_to_float(x):
Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_typeobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/test/test_typeobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/test/test_typeobject.py Tue Nov 23 10:12:47 2010
@@ -1085,4 +1085,12 @@
assert b == 1
-
+ def test_slots_with_method_in_class(self):
+ # this works in cpython...
+ class A(object):
+ __slots__ = ["f"]
+ def f(self, x):
+ return x + 1
+ a = A()
+ assert a.f(1) == 2
+
Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_versionedtype.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/test/test_versionedtype.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/test/test_versionedtype.py Tue Nov 23 10:12:47 2010
@@ -12,10 +12,12 @@
def f(self): pass
class B(A):
pass
- class metatype(type):
+ class X:
+ pass
+ class Y(object):
+ pass
+ class C(Y, X):
pass
- class C(object):
- __metaclass__ = metatype
return A, B, C
""")
return space.unpackiterable(w_types)
Modified: pypy/branch/arm-backend/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/std/typeobject.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/std/typeobject.py Tue Nov 23 10:12:47 2010
@@ -60,6 +60,15 @@
self.hits = {}
self.misses = {}
+ def clear(self):
+ None_None = (None, None)
+ for i in range(len(self.versions)):
+ self.versions[i] = None
+ for i in range(len(self.names)):
+ self.names[i] = None
+ for i in range(len(self.lookup_where)):
+ self.lookup_where[i] = None_None
+
class W_TypeObject(W_Object):
from pypy.objspace.std.typetype import type_typedef as typedef
@@ -75,7 +84,9 @@
'weakrefable',
'hasdict',
'nslots',
- 'instancetypedef']
+ 'instancetypedef',
+ 'terminator',
+ ]
# for config.objspace.std.getattributeshortcut
# (False is a conservative default, fixed during real usage)
@@ -102,20 +113,24 @@
if overridetypedef is not None:
setup_builtin_type(w_self)
- custom_metaclass = False
else:
setup_user_defined_type(w_self)
- custom_metaclass = not space.is_w(space.type(w_self), space.w_type)
w_self.w_same_layout_as = get_parent_layout(w_self)
if space.config.objspace.std.withtypeversion:
- if custom_metaclass or not is_mro_purely_of_types(w_self.mro_w):
+ if not is_mro_purely_of_types(w_self.mro_w):
pass
else:
# the _version_tag should change, whenever the content of
# dict_w of any of the types in the mro changes, or if the mro
# itself changes
w_self._version_tag = VersionTag()
+ if space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import DictTerminator, NoDictTerminator
+ if w_self.hasdict:
+ w_self.terminator = DictTerminator(space, w_self)
+ else:
+ w_self.terminator = NoDictTerminator(space, w_self)
def mutated(w_self):
space = w_self.space
@@ -551,11 +566,13 @@
space.wrap('__slots__ must be identifiers'))
# create member
slot_name = _mangle(slot_name, w_self.name)
- # Force interning of slot names.
- slot_name = space.str_w(space.new_interned_str(slot_name))
- member = Member(w_self.nslots, slot_name, w_self)
- w_self.dict_w[slot_name] = space.wrap(member)
- w_self.nslots += 1
+ if slot_name not in w_self.dict_w:
+ # Force interning of slot names.
+ slot_name = space.str_w(space.new_interned_str(slot_name))
+ # in cpython it is ignored less, but we probably don't care
+ member = Member(w_self.nslots, slot_name, w_self)
+ w_self.dict_w[slot_name] = space.wrap(member)
+ w_self.nslots += 1
def create_dict_slot(w_self):
if not w_self.hasdict:
Modified: pypy/branch/arm-backend/pypy/objspace/test/test_descroperation.py
==============================================================================
--- pypy/branch/arm-backend/pypy/objspace/test/test_descroperation.py (original)
+++ pypy/branch/arm-backend/pypy/objspace/test/test_descroperation.py Tue Nov 23 10:12:47 2010
@@ -400,7 +400,7 @@
class D(object):
def __init__(self, a):
self.a = a
-
+
assert A(1) == B(1)
assert B(1) == A(1)
assert A(1) == C(1)
@@ -457,6 +457,20 @@
else:
assert False, "did not raise"
+ def test_attribute_error(self):
+ class classmethodonly(classmethod):
+ def __get__(self, instance, type):
+ if instance is not None:
+ raise AttributeError("Must be called on a class, not an instance.")
+ return super(classmethodonly, self).__get__(instance, type)
+
+ class A(object):
+ @classmethodonly
+ def a(cls):
+ return 3
+
+ raises(AttributeError, lambda: A().a)
+
class AppTestWithBuiltinShortcut(AppTest_Descroperation):
OPTIONS = {'objspace.std.builtinshortcut': True}
Modified: pypy/branch/arm-backend/pypy/rlib/_rsocket_rffi.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/_rsocket_rffi.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/_rsocket_rffi.py Tue Nov 23 10:12:47 2010
@@ -324,10 +324,11 @@
('sll_hatype', rffi.INT),
('sll_addr', rffi.CFixedArray(rffi.CHAR, 8)),
('sll_halen', rffi.INT)],
- )
+ ifdef='AF_PACKET')
CConfig.ifreq = platform.Struct('struct ifreq', [('ifr_ifindex', rffi.INT),
- ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))])
+ ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))],
+ ifdef='AF_PACKET')
if _WIN32:
CConfig.WSAEVENT = platform.SimpleType('WSAEVENT', rffi.VOIDP)
@@ -532,8 +533,9 @@
socketpair_t = rffi.CArray(socketfd_type)
socketpair = external('socketpair', [rffi.INT, rffi.INT, rffi.INT,
lltype.Ptr(socketpair_t)], rffi.INT)
- ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)],
- rffi.INT)
+ if ifreq is not None:
+ ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)],
+ rffi.INT)
if _WIN32:
ioctlsocket = external('ioctlsocket',
Modified: pypy/branch/arm-backend/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/jit.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/jit.py Tue Nov 23 10:12:47 2010
@@ -4,6 +4,7 @@
from pypy.rlib.objectmodel import CDefinedIntSymbolic
from pypy.rlib.objectmodel import keepalive_until_here
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib.nonconst import NonConstant
def purefunction(func):
""" Decorate a function as pure. Pure means precisely that:
@@ -77,6 +78,12 @@
return result
return decorator
+def oopspec(spec):
+ def decorator(func):
+ func.oopspec = spec
+ return func
+ return decorator
+
class Entry(ExtRegistryEntry):
_about_ = hint
@@ -139,6 +146,32 @@
return hop.inputconst(lltype.Signed, _we_are_jitted)
+def current_trace_length():
+ """During JIT tracing, returns the current trace length (as a constant).
+ If not tracing, returns -1."""
+ if NonConstant(False):
+ return 73
+ return -1
+current_trace_length.oopspec = 'jit.current_trace_length()'
+
+def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1,
+ arg3=-sys.maxint-1, arg4=-sys.maxint-1):
+ """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in
+ the graphs. Should not be left after debugging."""
+ keepalive_until_here(string) # otherwise the whole function call is removed
+jit_debug.oopspec = 'jit.debug(string, arg1, arg2, arg3, arg4)'
+
+def assert_green(value):
+ """Very strong assert: checks that 'value' is a green
+ (a JIT compile-time constant)."""
+ keepalive_until_here(value)
+assert_green._annspecialcase_ = 'specialize:argtype(0)'
+assert_green.oopspec = 'jit.assert_green(value)'
+
+class AssertGreenFailed(Exception):
+ pass
+
+
##def force_virtualizable(virtualizable):
## pass
@@ -250,8 +283,9 @@
several independent JITting interpreters in it.
"""
+ active = True # if set to False, this JitDriver is ignored
virtualizables = []
-
+
def __init__(self, greens=None, reds=None, virtualizables=None,
get_jitcell_at=None, set_jitcell_at=None,
get_printable_location=None, confirm_enter_jit=None,
@@ -266,7 +300,8 @@
self.virtualizables = virtualizables
for v in self.virtualizables:
assert v in self.reds
- self._alllivevars = dict.fromkeys(self.greens + self.reds)
+ self._alllivevars = dict.fromkeys(
+ [name for name in self.greens + self.reds if '.' not in name])
self._make_extregistryentries()
self.get_jitcell_at = get_jitcell_at
self.set_jitcell_at = set_jitcell_at
@@ -355,10 +390,16 @@
def compute_result_annotation(self, **kwds_s):
from pypy.annotation import model as annmodel
+
+ if self.instance.__name__ == 'jit_merge_point':
+ if not self.annotate_hooks(**kwds_s):
+ return None # wrong order, try again later
+
driver = self.instance.im_self
keys = kwds_s.keys()
keys.sort()
- expected = ['s_' + name for name in driver.greens + driver.reds]
+ expected = ['s_' + name for name in driver.greens + driver.reds
+ if '.' not in name]
expected.sort()
if keys != expected:
raise JitHintError("%s expects the following keyword "
@@ -382,30 +423,35 @@
key[2:])
cache[key] = s_value
- if self.instance.__name__ == 'jit_merge_point':
- self.annotate_hooks(**kwds_s)
-
return annmodel.s_None
def annotate_hooks(self, **kwds_s):
driver = self.instance.im_self
s_jitcell = self.bookkeeper.valueoftype(BaseJitCell)
- self.annotate_hook(driver.get_jitcell_at, driver.greens, **kwds_s)
- self.annotate_hook(driver.set_jitcell_at, driver.greens, [s_jitcell],
- **kwds_s)
- self.annotate_hook(driver.get_printable_location, driver.greens, **kwds_s)
+ h = self.annotate_hook
+ return (h(driver.get_jitcell_at, driver.greens, **kwds_s)
+ and h(driver.set_jitcell_at, driver.greens, [s_jitcell], **kwds_s)
+ and h(driver.get_printable_location, driver.greens, **kwds_s))
def annotate_hook(self, func, variables, args_s=[], **kwds_s):
if func is None:
- return
+ return True
bk = self.bookkeeper
s_func = bk.immutablevalue(func)
uniquekey = 'jitdriver.%s' % func.func_name
args_s = args_s[:]
for name in variables:
- s_arg = kwds_s['s_' + name]
+ if '.' not in name:
+ s_arg = kwds_s['s_' + name]
+ else:
+ objname, fieldname = name.split('.')
+ s_instance = kwds_s['s_' + objname]
+ s_arg = s_instance.classdef.about_attribute(fieldname)
+ if s_arg is None:
+ return False # wrong order, try again later
args_s.append(s_arg)
bk.emulate_pbc_call(uniquekey, s_func, args_s)
+ return True
def specialize_call(self, hop, **kwds_i):
# XXX to be complete, this could also check that the concretetype
@@ -416,9 +462,42 @@
greens_v = []
reds_v = []
for name in driver.greens:
- i = kwds_i['i_' + name]
- r_green = hop.args_r[i]
- v_green = hop.inputarg(r_green, arg=i)
+ if '.' not in name:
+ i = kwds_i['i_' + name]
+ r_green = hop.args_r[i]
+ v_green = hop.inputarg(r_green, arg=i)
+ else:
+ if hop.rtyper.type_system.name == 'ootypesystem':
+ py.test.skip("lltype only")
+ objname, fieldname = name.split('.') # see test_green_field
+ assert objname in driver.reds
+ i = kwds_i['i_' + objname]
+ s_red = hop.args_s[i]
+ r_red = hop.args_r[i]
+ while True:
+ try:
+ mangled_name, r_field = r_red._get_field(fieldname)
+ break
+ except KeyError:
+ pass
+ assert r_red.rbase is not None, (
+ "field %r not found in %r" % (name,
+ r_red.lowleveltype.TO))
+ r_red = r_red.rbase
+ GTYPE = r_red.lowleveltype.TO
+ assert GTYPE._immutable_field(mangled_name), (
+ "field %r must be declared as immutable" % name)
+ if not hasattr(driver, 'll_greenfields'):
+ driver.ll_greenfields = {}
+ driver.ll_greenfields[name] = GTYPE, mangled_name
+ #
+ v_red = hop.inputarg(r_red, arg=i)
+ c_llname = hop.inputconst(lltype.Void, mangled_name)
+ v_green = hop.genop('getfield', [v_red, c_llname],
+ resulttype = r_field)
+ s_green = s_red.classdef.about_attribute(fieldname)
+ assert s_green is not None
+ hop.rtyper.annotator.setbinding(v_green, s_green)
greens_v.append(v_green)
for name in driver.reds:
i = kwds_i['i_' + name]
Modified: pypy/branch/arm-backend/pypy/rlib/objectmodel.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/objectmodel.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/objectmodel.py Tue Nov 23 10:12:47 2010
@@ -475,6 +475,10 @@
def setdefault(self, key, default):
return self._dict.setdefault(_r_dictkey(self, key), default)
+ def popitem(self):
+ dk, value = self._dict.popitem()
+ return dk.key, value
+
def copy(self):
result = r_dict(self.key_eq, self.key_hash)
result.update(self)
Modified: pypy/branch/arm-backend/pypy/rlib/rarithmetic.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/rarithmetic.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/rarithmetic.py Tue Nov 23 10:12:47 2010
@@ -501,6 +501,11 @@
def __cmp__(self, other):
raise TypeError("not supported on r_singlefloat instances")
+ def __eq__(self, other):
+ return self.__class__ is other.__class__ and self._bytes == other._bytes
+
+ def __ne__(self, other):
+ return not self.__eq__(other)
class For_r_singlefloat_values_Entry(extregistry.ExtRegistryEntry):
_type_ = r_singlefloat
Modified: pypy/branch/arm-backend/pypy/rlib/rbigint.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/rbigint.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/rbigint.py Tue Nov 23 10:12:47 2010
@@ -1596,11 +1596,16 @@
# a few internal helpers
-DEC_PER_DIGIT = 1
-while int('9' * DEC_PER_DIGIT) < MASK:
- DEC_PER_DIGIT += 1
-DEC_PER_DIGIT -= 1
-DEC_MAX = 10 ** DEC_PER_DIGIT
+def digits_max_for_base(base):
+ dec_per_digit = 1
+ while base ** dec_per_digit < MASK:
+ dec_per_digit += 1
+ dec_per_digit -= 1
+ return base ** dec_per_digit
+
+BASE_MAX = [0, 0] + [digits_max_for_base(_base) for _base in range(2, 37)]
+DEC_MAX = digits_max_for_base(10)
+assert DEC_MAX == BASE_MAX[10]
def _decimalstr_to_bigint(s):
# a string that has been already parsed to be decimal and valid,
@@ -1615,7 +1620,6 @@
p += 1
a = rbigint.fromint(0)
- cnt = DEC_PER_DIGIT
tens = 1
dig = 0
ord0 = ord('0')
@@ -1627,8 +1631,26 @@
a = _muladd1(a, tens, dig)
tens = 1
dig = 0
- if sign:
+ if sign and a.sign == 1:
a.sign = -1
return a
-
+def parse_digit_string(parser):
+ # helper for objspace.std.strutil
+ a = rbigint.fromint(0)
+ base = parser.base
+ digitmax = BASE_MAX[base]
+ tens, dig = 1, 0
+ while True:
+ digit = parser.next_digit()
+ if tens == digitmax or digit < 0:
+ a = _muladd1(a, tens, dig)
+ if digit < 0:
+ break
+ dig = digit
+ tens = base
+ else:
+ dig = dig * base + digit
+ tens *= base
+ a.sign *= parser.sign
+ return a
Modified: pypy/branch/arm-backend/pypy/rlib/rgc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/rgc.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/rgc.py Tue Nov 23 10:12:47 2010
@@ -170,7 +170,14 @@
return hop.genop('gc_set_max_heap_size', [v_nbytes],
resulttype=lltype.Void)
-def can_move(p): # NB. must not be called with NULL pointers
+def can_move(p):
+ """Check if the GC object 'p' is at an address that can move.
+ Must not be called with None. With non-moving GCs, it is always False.
+ With some moving GCs like the SemiSpace GC, it is always True.
+ With other moving GCs like the MiniMark GC, it can be True for some
+ time, then False for the same object, when we are sure that it won't
+ move any more.
+ """
return True
class CanMoveEntry(ExtRegistryEntry):
@@ -372,6 +379,11 @@
"NOT_RPYTHON"
raise NotImplementedError
+def get_typeids_z():
+ "NOT_RPYTHON"
+ raise NotImplementedError
+
+ARRAY_OF_CHAR = lltype.Array(lltype.Char)
NULL_GCREF = lltype.nullptr(llmemory.GCREF.TO)
class _GcRef(object):
@@ -523,3 +535,12 @@
vlist = hop.inputargs(lltype.Signed)
hop.exception_is_here()
return hop.genop('gc_dump_rpy_heap', vlist, resulttype = hop.r_result)
+
+class Entry(ExtRegistryEntry):
+ _about_ = get_typeids_z
+ def compute_result_annotation(self):
+ from pypy.annotation.model import SomePtr
+ return SomePtr(lltype.Ptr(ARRAY_OF_CHAR))
+ def specialize_call(self, hop):
+ hop.exception_is_here()
+ return hop.genop('gc_typeids_z', [], resulttype = hop.r_result)
Modified: pypy/branch/arm-backend/pypy/rlib/rmmap.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/rmmap.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/rmmap.py Tue Nov 23 10:12:47 2010
@@ -50,7 +50,7 @@
constant_names = ['MAP_SHARED', 'MAP_PRIVATE',
'PROT_READ', 'PROT_WRITE',
'MS_SYNC']
- opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS',
+ opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS', 'MAP_NORESERVE',
'PROT_EXEC',
'MAP_DENYWRITE', 'MAP_EXECUTABLE']
for name in constant_names:
Modified: pypy/branch/arm-backend/pypy/rlib/rsre/rsre_char.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/rsre/rsre_char.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/rsre/rsre_char.py Tue Nov 23 10:12:47 2010
@@ -4,6 +4,7 @@
import sys
from pypy.rlib.rlocale import tolower, isalnum
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib import jit
# Note: the unicode parts of this module require you to call
# rsre_char.set_unicode_db() first, to select one of the modules
@@ -43,6 +44,7 @@
# XXX can we import those safely from sre_constants?
SRE_INFO_PREFIX = 1
SRE_INFO_LITERAL = 2
+SRE_INFO_CHARSET = 4
SRE_FLAG_LOCALE = 4 # honour system locale
SRE_FLAG_UNICODE = 32 # use unicode locale
OPCODE_INFO = 17
@@ -64,33 +66,27 @@
#### Category helpers
-ascii_char_info = [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 6, 2,
-2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 25, 25, 25, 25, 25, 25, 25,
-25, 25, 0, 0, 0, 0, 0, 0, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0,
-0, 0, 16, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0, 0, 0, 0 ]
-
+is_a_word = [(chr(i).isalnum() or chr(i) == '_') for i in range(256)]
linebreak = ord("\n")
underline = ord("_")
def is_digit(code):
- return code < 128 and (ascii_char_info[code] & 1 != 0)
+ return code <= 57 and code >= 48
def is_uni_digit(code):
assert unicodedb is not None
return unicodedb.isdigit(code)
def is_space(code):
- return code < 128 and (ascii_char_info[code] & 2 != 0)
+ return code == 32 or (code <= 13 and code >= 9)
def is_uni_space(code):
assert unicodedb is not None
return unicodedb.isspace(code)
def is_word(code):
- return code < 128 and (ascii_char_info[code] & 16 != 0)
+ assert code >= 0
+ return code < 256 and is_a_word[code]
def is_uni_word(code):
assert unicodedb is not None
@@ -142,6 +138,7 @@
SET_OK = -1
SET_NOT_OK = -2
+ at jit.unroll_safe
def check_charset(pattern, ppos, char_code):
"""Checks whether a character matches set of arbitrary length.
The set starts at pattern[ppos]."""
Modified: pypy/branch/arm-backend/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/rsre/rsre_core.py Tue Nov 23 10:12:47 2010
@@ -1,9 +1,11 @@
import sys
-from pypy.rlib.debug import check_nonneg
+from pypy.rlib.debug import check_nonneg, make_sure_not_modified
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib.rsre import rsre_char
from pypy.tool.sourcetools import func_with_new_name
from pypy.rlib.objectmodel import we_are_translated
+from pypy.rlib import jit
+from pypy.rlib.rsre.rsre_jit import install_jitdriver, install_jitdriver_spec
OPCODE_FAILURE = 0
@@ -56,16 +58,19 @@
_seen_specname[specname] = True
# Install a copy of the function under the name '_spec_funcname' in each
# concrete subclass
+ specialized_methods = []
for prefix, concreteclass in [('str', StrMatchContext),
('uni', UnicodeMatchContext)]:
newfunc = func_with_new_name(func, prefix + specname)
assert not hasattr(concreteclass, specname)
setattr(concreteclass, specname, newfunc)
+ specialized_methods.append(newfunc)
# Return a dispatcher function, specialized on the exact type of 'ctx'
def dispatch(ctx, *args):
return getattr(ctx, specname)(*args)
dispatch._annspecialcase_ = 'specialize:argtype(0)'
- return dispatch
+ dispatch._specialized_methods_ = specialized_methods
+ return func_with_new_name(dispatch, specname)
# ____________________________________________________________
@@ -75,6 +80,7 @@
class AbstractMatchContext(object):
"""Abstract base class"""
+ _immutable_fields_ = ['pattern[*]', 'flags', 'end']
match_start = 0
match_end = 0
match_marks = None
@@ -164,6 +170,8 @@
def __init__(self, pattern, string, match_start, end, flags):
AbstractMatchContext.__init__(self, pattern, match_start, end, flags)
self._string = string
+ if not we_are_translated() and isinstance(string, unicode):
+ self.flags |= rsre_char.SRE_FLAG_UNICODE # for rsre_re.py
def str(self, index):
check_nonneg(index)
@@ -238,8 +246,9 @@
self.start_ptr = ptr
self.start_marks = marks
+ @jit.unroll_safe
def find_first_result(self, ctx):
- ppos = self.ppos
+ ppos = jit.hint(self.ppos, promote=True)
while ctx.pat(ppos):
result = sre_match(ctx, ppos + 1, self.start_ptr, self.start_marks)
ppos += ctx.pat(ppos)
@@ -250,6 +259,10 @@
find_next_result = find_first_result
class RepeatOneMatchResult(MatchResult):
+ install_jitdriver('RepeatOne',
+ greens=['nextppos', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(1, 0)) # indices in 'greens'
def __init__(self, nextppos, minptr, ptr, marks):
self.nextppos = nextppos
@@ -259,8 +272,11 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
while ptr >= self.minptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ ctx.jitdriver_RepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos)
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
ptr -= 1
if result is not None:
self.subresult = result
@@ -270,6 +286,10 @@
class MinRepeatOneMatchResult(MatchResult):
+ install_jitdriver('MinRepeatOne',
+ greens=['nextppos', 'ppos3', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(2, 0)) # indices in 'greens'
def __init__(self, nextppos, ppos3, maxptr, ptr, marks):
self.nextppos = nextppos
@@ -280,29 +300,32 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
+ ppos3 = self.ppos3
while ptr <= self.maxptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ ctx.jitdriver_MinRepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos, ppos3=ppos3)
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
if result is not None:
self.subresult = result
self.start_ptr = ptr
return self
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, ppos3):
break
ptr += 1
def find_next_result(self, ctx):
ptr = self.start_ptr
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, self.ppos3):
return
self.start_ptr = ptr + 1
return self.find_first_result(ctx)
- def next_char_ok(self, ctx, ptr):
+ def next_char_ok(self, ctx, ptr, ppos):
if ptr == ctx.end:
return False
- ppos = self.ppos3
op = ctx.pat(ppos)
- for op1, (checkerfn, _) in unroll_char_checker:
+ for op1, checkerfn in unroll_char_checker:
if op1 == op:
return checkerfn(ctx, ptr, ppos)
raise Error("next_char_ok[%d]" % op)
@@ -325,41 +348,34 @@
self.next = next # chained list
class MaxUntilMatchResult(AbstractUntilMatchResult):
+ install_jitdriver('MaxUntil',
+ greens=['ppos', 'tailppos', 'match_more', 'ctx.pattern'],
+ reds=['ptr', 'marks', 'self', 'ctx'],
+ debugprint=(3, 0, 2))
def find_first_result(self, ctx):
- enum = sre_match(ctx, self.ppos + 3, self.cur_ptr, self.cur_marks)
- return self.search_next(ctx, enum, resume=False)
+ return self.search_next(ctx, match_more=True)
def find_next_result(self, ctx):
- return self.search_next(ctx, None, resume=True)
+ return self.search_next(ctx, match_more=False)
- def search_next(self, ctx, enum, resume):
+ def search_next(self, ctx, match_more):
ppos = self.ppos
- min = ctx.pat(ppos+1)
- max = ctx.pat(ppos+2)
+ tailppos = self.tailppos
ptr = self.cur_ptr
marks = self.cur_marks
while True:
- while True:
- if (enum is not None and
- (ptr != ctx.match_end or self.num_pending < min)):
- # ^^^^^^^^^^ zero-width match protection
- # matched one more 'item'. record it and continue.
- self.pending = Pending(ptr, marks, enum, self.pending)
- self.num_pending += 1
- ptr = ctx.match_end
- marks = ctx.match_marks
- break
- # 'item' no longer matches.
- if not resume and self.num_pending >= min:
- # try to match 'tail' if we have enough 'item'
- result = sre_match(ctx, self.tailppos, ptr, marks)
- if result is not None:
- self.subresult = result
- self.cur_ptr = ptr
- self.cur_marks = marks
- return self
- resume = False
+ ctx.jitdriver_MaxUntil.jit_merge_point(
+ ppos=ppos, tailppos=tailppos, match_more=match_more,
+ ptr=ptr, marks=marks, self=self, ctx=ctx)
+ if match_more:
+ max = ctx.pat(ppos+2)
+ if max == 65535 or self.num_pending < max:
+ # try to match one more 'item'
+ enum = sre_match(ctx, ppos + 3, ptr, marks)
+ else:
+ enum = None # 'max' reached, no more matches
+ else:
p = self.pending
if p is None:
return
@@ -369,11 +385,27 @@
marks = p.marks
enum = p.enum.move_to_next_result(ctx)
#
- if max == 65535 or self.num_pending < max:
- # try to match one more 'item'
- enum = sre_match(ctx, ppos + 3, ptr, marks)
+ min = ctx.pat(ppos+1)
+ if (enum is not None and
+ (ptr != ctx.match_end or self.num_pending < min)):
+ # ^^^^^^^^^^ zero-width match protection
+ # matched one more 'item'. record it and continue.
+ self.pending = Pending(ptr, marks, enum, self.pending)
+ self.num_pending += 1
+ ptr = ctx.match_end
+ marks = ctx.match_marks
+ match_more = True
else:
- enum = None # 'max' reached, no more matches
+ # 'item' no longer matches.
+ if self.num_pending >= min:
+ # try to match 'tail' if we have enough 'item'
+ result = sre_match(ctx, tailppos, ptr, marks)
+ if result is not None:
+ self.subresult = result
+ self.cur_ptr = ptr
+ self.cur_marks = marks
+ return self
+ match_more = False
class MinUntilMatchResult(AbstractUntilMatchResult):
@@ -384,6 +416,7 @@
return self.search_next(ctx, resume=True)
def search_next(self, ctx, resume):
+ # XXX missing jit support here
ppos = self.ppos
min = ctx.pat(ppos+1)
max = ctx.pat(ppos+2)
@@ -429,6 +462,7 @@
# ____________________________________________________________
@specializectx
+ at jit.unroll_safe
def sre_match(ctx, ppos, ptr, marks):
"""Returns either None or a MatchResult object. Usually we only need
the first result, but there is the case of REPEAT...UNTIL where we
@@ -437,6 +471,13 @@
while True:
op = ctx.pat(ppos)
ppos += 1
+ make_sure_not_modified(ctx.pattern)
+
+ #jit.jit_debug("sre_match", op, ppos, ptr)
+ #
+ # When using the JIT, calls to sre_match() must always have a constant
+ # (green) argument for 'ppos'. If not, the following assert fails.
+ jit.assert_green(op)
if op == OPCODE_FAILURE:
return
@@ -712,13 +753,23 @@
@specializectx
def find_repetition_end(ctx, ppos, ptr, maxcount):
end = ctx.end
- # adjust end
- if maxcount != 65535:
+ if maxcount <= 1:
+ if maxcount == 1 and ptr < end:
+ # Relatively common case: maxcount == 1. If we are not at the
+ # end of the string, it's done by a single direct check.
+ op = ctx.pat(ppos)
+ for op1, checkerfn in unroll_char_checker:
+ if op1 == op:
+ if checkerfn(ctx, ptr, ppos):
+ return ptr + 1
+ return ptr
+ elif maxcount != 65535:
+ # adjust end
end1 = ptr + maxcount
if end1 <= end:
end = end1
op = ctx.pat(ppos)
- for op1, (_, fre) in unroll_char_checker:
+ for op1, fre in unroll_fre_checker:
if op1 == op:
return fre(ctx, ptr, end, ppos)
raise Error("rsre.find_repetition_end[%d]" % op)
@@ -751,23 +802,60 @@
if checkerfn == match_ANY_ALL:
def fre(ctx, ptr, end, ppos):
return end
+ elif checkerfn == match_IN:
+ install_jitdriver_spec('MatchIn',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchIn.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
+ elif checkerfn == match_IN_IGNORE:
+ install_jitdriver_spec('MatchInIgnore',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchInIgnore.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
else:
+ # in the other cases, the fre() function is not JITted at all
+ # and is present as a residual call.
+ @specializectx
def fre(ctx, ptr, end, ppos):
while ptr < end and checkerfn(ctx, ptr, ppos):
ptr += 1
return ptr
- return checkerfn, fre
+ fre = func_with_new_name(fre, 'fre_' + checkerfn.__name__)
+ return fre
+
+unroll_char_checker = [
+ (OPCODE_ANY, match_ANY),
+ (OPCODE_ANY_ALL, match_ANY_ALL),
+ (OPCODE_IN, match_IN),
+ (OPCODE_IN_IGNORE, match_IN_IGNORE),
+ (OPCODE_LITERAL, match_LITERAL),
+ (OPCODE_LITERAL_IGNORE, match_LITERAL_IGNORE),
+ (OPCODE_NOT_LITERAL, match_NOT_LITERAL),
+ (OPCODE_NOT_LITERAL_IGNORE, match_NOT_LITERAL_IGNORE),
+ ]
+unroll_fre_checker = [(_op, _make_fre(_fn))
+ for (_op, _fn) in unroll_char_checker]
-unroll_char_checker = unrolling_iterable([
- (OPCODE_ANY, _make_fre(match_ANY)),
- (OPCODE_ANY_ALL, _make_fre(match_ANY_ALL)),
- (OPCODE_IN, _make_fre(match_IN)),
- (OPCODE_IN_IGNORE, _make_fre(match_IN_IGNORE)),
- (OPCODE_LITERAL, _make_fre(match_LITERAL)),
- (OPCODE_LITERAL_IGNORE, _make_fre(match_LITERAL_IGNORE)),
- (OPCODE_NOT_LITERAL, _make_fre(match_NOT_LITERAL)),
- (OPCODE_NOT_LITERAL_IGNORE, _make_fre(match_NOT_LITERAL_IGNORE)),
- ])
+unroll_char_checker = unrolling_iterable(unroll_char_checker)
+unroll_fre_checker = unrolling_iterable(unroll_fre_checker)
##### At dispatch
@@ -873,74 +961,139 @@
else:
return None
+install_jitdriver('Match',
+ greens=['ctx.pattern'], reds=['ctx'],
+ debugprint=(0,))
+
def match_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
+ ctx.jitdriver_Match.jit_merge_point(ctx=ctx)
return sre_match(ctx, 0, ctx.match_start, None) is not None
def search_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
- if ctx.pat(0) == OPCODE_INFO:
- if ctx.pat(2) & rsre_char.SRE_INFO_PREFIX and ctx.pat(5) > 1:
- return fast_search(ctx)
- return regular_search(ctx)
+ base = 0
+ charset = False
+ if ctx.pat(base) == OPCODE_INFO:
+ flags = ctx.pat(2)
+ if flags & rsre_char.SRE_INFO_PREFIX:
+ if ctx.pat(5) > 1:
+ return fast_search(ctx)
+ else:
+ charset = (flags & rsre_char.SRE_INFO_CHARSET)
+ base += 1 + ctx.pat(1)
+ if ctx.pat(base) == OPCODE_LITERAL:
+ return literal_search(ctx, base)
+ if charset:
+ return charset_search(ctx, base)
+ return regular_search(ctx, base)
+
+install_jitdriver('RegularSearch',
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
-def regular_search(ctx):
+def regular_search(ctx, base):
start = ctx.match_start
while start <= ctx.end:
- if sre_match(ctx, 0, start, None) is not None:
+ ctx.jitdriver_RegularSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if sre_match(ctx, base, start, None) is not None:
ctx.match_start = start
return True
start += 1
return False
+install_jitdriver_spec("LiteralSearch",
+ greens=['base', 'character', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(2, 0, 1))
+ at specializectx
+def literal_search(ctx, base):
+ # pattern starts with a literal character. this is used
+ # for short prefixes, and if fast search is disabled
+ character = ctx.pat(base + 1)
+ base += 2
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_LiteralSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base, character=character)
+ if ctx.str(start) == character:
+ if sre_match(ctx, base, start + 1, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec("CharsetSearch",
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
+ at specializectx
+def charset_search(ctx, base):
+ # pattern starts with a character from a known set
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_CharsetSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if rsre_char.check_charset(ctx.pattern, 5, ctx.str(start)):
+ if sre_match(ctx, base, start, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec('FastSearch',
+ greens=['i', 'prefix_len', 'ctx.pattern'],
+ reds=['string_position', 'ctx'],
+ debugprint=(2, 0))
@specializectx
def fast_search(ctx):
# skips forward in a string as fast as possible using information from
# an optimization info block
# <INFO> <1=skip> <2=flags> <3=min> <4=...>
# <5=length> <6=skip> <7=prefix data> <overlap data>
- flags = ctx.pat(2)
+ string_position = ctx.match_start
+ if string_position >= ctx.end:
+ return False
prefix_len = ctx.pat(5)
assert prefix_len >= 0
- prefix_skip = ctx.pat(6)
- assert prefix_skip >= 0
- overlap_offset = 7 + prefix_len - 1
- assert overlap_offset >= 0
- pattern_offset = ctx.pat(1) + 1
- ppos_start = pattern_offset + 2 * prefix_skip
- assert ppos_start >= 0
i = 0
- string_position = ctx.match_start
- end = ctx.end
- while string_position < end:
- while True:
- char_ord = ctx.str(string_position)
- if char_ord != ctx.pat(7 + i):
- if i == 0:
- break
- else:
- i = ctx.pat(overlap_offset + i)
- else:
- i += 1
- if i == prefix_len:
- # found a potential match
- start = string_position + 1 - prefix_len
- assert start >= 0
- ptr = start + prefix_skip
- if flags & rsre_char.SRE_INFO_LITERAL:
- # matched all of pure literal pattern
- ctx.match_start = start
- ctx.match_end = ptr
- ctx.match_marks = None
- return True
- if sre_match(ctx, ppos_start, ptr, None) is not None:
- ctx.match_start = start
- return True
- i = ctx.pat(overlap_offset + i)
- break
+ while True:
+ ctx.jitdriver_FastSearch.jit_merge_point(ctx=ctx,
+ string_position=string_position, i=i, prefix_len=prefix_len)
+ char_ord = ctx.str(string_position)
+ if char_ord != ctx.pat(7 + i):
+ if i > 0:
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
+ continue
+ else:
+ i += 1
+ if i == prefix_len:
+ # found a potential match
+ start = string_position + 1 - prefix_len
+ assert start >= 0
+ prefix_skip = ctx.pat(6)
+ ptr = start + prefix_skip
+ #flags = ctx.pat(2)
+ #if flags & rsre_char.SRE_INFO_LITERAL:
+ # # matched all of pure literal pattern
+ # ctx.match_start = start
+ # ctx.match_end = ptr
+ # ctx.match_marks = None
+ # return True
+ pattern_offset = ctx.pat(1) + 1
+ ppos_start = pattern_offset + 2 * prefix_skip
+ if sre_match(ctx, ppos_start, ptr, None) is not None:
+ ctx.match_start = start
+ return True
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
string_position += 1
- return False
+ if string_position >= ctx.end:
+ return False
Modified: pypy/branch/arm-backend/pypy/rlib/rsre/test/test_match.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/rsre/test/test_match.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/rsre/test/test_match.py Tue Nov 23 10:12:47 2010
@@ -1,24 +1,49 @@
-import _sre, re, sre_compile
-from pypy.rlib.rsre import rsre_core
+import re
+from pypy.rlib.rsre import rsre_core, rsre_char
-def get_code(regexp, flags=0, allargs=False):
- class GotIt(Exception):
- pass
- def my_compile(pattern, flags, code, *args):
- print code
- raise GotIt(code, flags, args)
- saved = _sre.compile
- try:
- _sre.compile = my_compile
- try:
- sre_compile.compile(regexp, flags)
- except GotIt, e:
- pass
+def get_hacked_sre_compile(my_compile):
+ """Return a copy of the sre_compile module for which the _sre
+ module is a custom module that has _sre.compile == my_compile
+ and CODESIZE == rsre_char.CODESIZE.
+ """
+ import sre_compile, __builtin__, new
+ sre_hacked = new.module("_sre_hacked")
+ sre_hacked.compile = my_compile
+ sre_hacked.MAGIC = sre_compile.MAGIC
+ sre_hacked.CODESIZE = rsre_char.CODESIZE
+ sre_hacked.getlower = rsre_char.getlower
+ def my_import(name, *args):
+ if name == '_sre':
+ return sre_hacked
else:
- raise ValueError("did not reach _sre.compile()!")
+ return default_import(name, *args)
+ src = sre_compile.__file__
+ if src.lower().endswith('.pyc') or src.lower().endswith('.pyo'):
+ src = src[:-1]
+ mod = new.module("sre_compile_hacked")
+ default_import = __import__
+ try:
+ __builtin__.__import__ = my_import
+ execfile(src, mod.__dict__)
finally:
- _sre.compile = saved
+ __builtin__.__import__ = default_import
+ return mod
+
+class GotIt(Exception):
+ pass
+def my_compile(pattern, flags, code, *args):
+ print code
+ raise GotIt(code, flags, args)
+sre_compile_hacked = get_hacked_sre_compile(my_compile)
+
+def get_code(regexp, flags=0, allargs=False):
+ try:
+ sre_compile_hacked.compile(regexp, flags)
+ except GotIt, e:
+ pass
+ else:
+ raise ValueError("did not reach _sre.compile()!")
if allargs:
return e.args
else:
Modified: pypy/branch/arm-backend/pypy/rlib/rstring.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/rstring.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/rstring.py Tue Nov 23 10:12:47 2010
@@ -54,6 +54,7 @@
self.l = []
def append(self, s):
+ assert isinstance(s, self._type)
self.l.append(s)
def append_slice(self, s, start, end):
@@ -63,11 +64,16 @@
def append_multiple_char(self, c, times):
self.l.append(c * times)
+ def getlength(self):
+ return len(self.build())
+
class StringBuilder(AbstractStringBuilder):
+ _type = str
def build(self):
return "".join(self.l)
class UnicodeBuilder(AbstractStringBuilder):
+ _type = unicode
def build(self):
return u''.join(self.l)
@@ -121,9 +127,12 @@
assert s_times.nonneg
return s_None
+ def method_getlength(self):
+ return SomeInteger(nonneg=True)
+
def method_build(self):
return SomeString()
-
+
def rtyper_makerepr(self, rtyper):
return rtyper.type_system.rbuilder.stringbuilder_repr
@@ -146,6 +155,9 @@
assert s_times.nonneg
return s_None
+ def method_getlength(self):
+ return SomeInteger(nonneg=True)
+
def method_build(self):
return SomeUnicodeString()
Modified: pypy/branch/arm-backend/pypy/rlib/streamio.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/streamio.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/streamio.py Tue Nov 23 10:12:47 2010
@@ -12,7 +12,7 @@
* some other methods also have no default parameters.
* close() should be called exactly once and no further operations performed;
there is no __del__() closing the stream for you.
- * some methods may raise NotImplementedError.
+ * some methods may raise MyNotImplementedError.
* peek() returns some (or no) characters that have already been read ahead.
* flushable() returns True/False if flushing that stream is useful/pointless.
@@ -54,6 +54,12 @@
('a', True): O_RDWR | O_CREAT,
}
+class MyNotImplementedError(Exception):
+ """
+ Catching NotImplementedError is not RPython, so we use this custom class
+ instead of it
+ """
+
# ____________________________________________________________
@@ -209,16 +215,16 @@
some methods."""
def read(self, n):
- raise NotImplementedError
+ raise MyNotImplementedError
def write(self, data):
- raise NotImplementedError
+ raise MyNotImplementedError
def tell(self):
- raise NotImplementedError
+ raise MyNotImplementedError
def seek(self, offset, whence):
- raise NotImplementedError
+ raise MyNotImplementedError
def readall(self):
bufsize = 8192
@@ -251,7 +257,7 @@
return ''.join(result)
def truncate(self, size):
- raise NotImplementedError
+ raise MyNotImplementedError
def flush_buffers(self):
pass
@@ -487,7 +493,7 @@
if self.lines or self.buf:
try:
self.do_seek(self.tell(), 0)
- except NotImplementedError:
+ except MyNotImplementedError:
pass
else:
self.lines = []
@@ -535,14 +541,14 @@
self.buf = ""
try:
self.do_seek(offset, 1)
- except NotImplementedError:
+ except MyNotImplementedError:
intoffset = offset2int(offset)
self.read(intoffset)
return
if whence == 2:
try:
self.do_seek(offset, 2)
- except NotImplementedError:
+ except MyNotImplementedError:
pass
else:
self.lines = []
@@ -1019,7 +1025,7 @@
if self.buf:
try:
self.base.seek(-len(self.buf), 1)
- except NotImplementedError:
+ except MyNotImplementedError:
pass
else:
self.buf = ""
Modified: pypy/branch/arm-backend/pypy/rlib/test/test_jit.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/test/test_jit.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/test/test_jit.py Tue Nov 23 10:12:47 2010
@@ -1,10 +1,17 @@
import py
+from pypy.conftest import option
from pypy.rlib.jit import hint, we_are_jitted, JitDriver, purefunction_promote
-from pypy.rlib.jit import JitHintError
+from pypy.rlib.jit import JitHintError, oopspec
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem import lltype
+def test_oopspec():
+ @oopspec('foobar')
+ def fn():
+ pass
+ assert fn.oopspec == 'foobar'
+
class BaseTestJIT(BaseRtypingTest):
def test_hint(self):
def f():
@@ -104,6 +111,26 @@
return n
py.test.raises(JitHintError, self.gengraph, fn, [int])
+ def test_green_field(self):
+ def get_printable_location(xfoo):
+ return str(ord(xfoo)) # xfoo must be annotated as a character
+ myjitdriver = JitDriver(greens=['x.foo'], reds=['n', 'x'],
+ get_printable_location=get_printable_location)
+ class A(object):
+ _immutable_fields_ = ['foo']
+ def fn(n):
+ x = A()
+ x.foo = chr(n)
+ while n > 0:
+ myjitdriver.can_enter_jit(x=x, n=n)
+ myjitdriver.jit_merge_point(x=x, n=n)
+ n -= 1
+ return n
+ t = self.gengraph(fn, [int])[0]
+ if option.view:
+ t.view()
+ # assert did not raise
+
class TestJITLLtype(BaseTestJIT, LLRtypeMixin):
pass
Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rarithmetic.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/test/test_rarithmetic.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/test/test_rarithmetic.py Tue Nov 23 10:12:47 2010
@@ -325,6 +325,15 @@
assert float(x) != 2.1
assert abs(float(x) - 2.1) < 1E-6
+def test_r_singlefloat_eq():
+ x = r_singlefloat(2.5) # exact number
+ y = r_singlefloat(2.5)
+ assert x == y
+ assert not x != y
+ assert not x == 2.5
+ assert x != 2.5
+ py.test.raises(TypeError, "x>y")
+
class BaseTestRarithmetic(BaseRtypingTest):
def test_formatd(self):
from pypy.rlib.rarithmetic import formatd
@@ -358,7 +367,17 @@
assert res == 1.0
res = self.interpret(f, [1])
- assert res == 1e-100
+ assert res == 1e-100
+
+ def test_compare_singlefloat_crashes(self):
+ from pypy.rlib.rarithmetic import r_singlefloat
+ from pypy.rpython.error import MissingRTypeOperation
+ def f(x):
+ a = r_singlefloat(x)
+ b = r_singlefloat(x+1)
+ return a == b
+ py.test.raises(MissingRTypeOperation, "self.interpret(f, [42.0])")
+
class TestLLtype(BaseTestRarithmetic, LLRtypeMixin):
pass
Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rbigint.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/test/test_rbigint.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/test/test_rbigint.py Tue Nov 23 10:12:47 2010
@@ -112,6 +112,23 @@
assert rbigint.fromrarith_int(r_uint(2*sys.maxint+1)).eq(
rbigint.fromlong(2*sys.maxint+1))
+ def test_fromdecimalstr(self):
+ x = rbigint.fromdecimalstr("12345678901234567890523897987")
+ assert x.tolong() == 12345678901234567890523897987L
+ assert x.tobool() is True
+ x = rbigint.fromdecimalstr("+12345678901234567890523897987")
+ assert x.tolong() == 12345678901234567890523897987L
+ assert x.tobool() is True
+ x = rbigint.fromdecimalstr("-12345678901234567890523897987")
+ assert x.tolong() == -12345678901234567890523897987L
+ assert x.tobool() is True
+ x = rbigint.fromdecimalstr("+0")
+ assert x.tolong() == 0
+ assert x.tobool() is False
+ x = rbigint.fromdecimalstr("-0")
+ assert x.tolong() == 0
+ assert x.tobool() is False
+
def test_add(self):
x = 123456789123456789000000L
y = 123858582373821923936744221L
@@ -448,6 +465,35 @@
assert (rbigint.fromlong(-9**50).ulonglongmask() ==
r_ulonglong(-9**50))
+ def test_parse_digit_string(self):
+ from pypy.rlib.rbigint import parse_digit_string
+ class Parser:
+ def __init__(self, base, sign, digits):
+ self.base = base
+ self.sign = sign
+ self.next_digit = iter(digits + [-1]).next
+ x = parse_digit_string(Parser(10, 1, [6]))
+ assert x.eq(rbigint.fromint(6))
+ x = parse_digit_string(Parser(10, 1, [6, 2, 3]))
+ assert x.eq(rbigint.fromint(623))
+ x = parse_digit_string(Parser(10, -1, [6, 2, 3]))
+ assert x.eq(rbigint.fromint(-623))
+ x = parse_digit_string(Parser(16, 1, [0xA, 0x4, 0xF]))
+ assert x.eq(rbigint.fromint(0xA4F))
+ num = 0
+ for i in range(36):
+ x = parse_digit_string(Parser(36, 1, range(i)))
+ assert x.eq(rbigint.fromlong(num))
+ num = num * 36 + i
+ x = parse_digit_string(Parser(16, -1, range(15,-1,-1)*99))
+ assert x.eq(rbigint.fromlong(long('-0x' + 'FEDCBA9876543210'*99, 16)))
+ assert x.tobool() is True
+ x = parse_digit_string(Parser(7, 1, [0, 0, 0]))
+ assert x.tobool() is False
+ x = parse_digit_string(Parser(7, -1, [0, 0, 0]))
+ assert x.tobool() is False
+
+
BASE = 2 ** SHIFT
class TestTranslatable(object):
Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rdynload.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/test/test_rdynload.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/test/test_rdynload.py Tue Nov 23 10:12:47 2010
@@ -1,15 +1,22 @@
from pypy.rlib.rdynload import *
-from pypy.rlib.libffi import get_libc_name
+from pypy.rlib.clibffi import get_libc_name
from pypy.rpython.lltypesystem import rffi, lltype
import py
class TestDLOperations:
def test_dlopen(self):
- py.test.raises(DLOpenError, "dlopen(rffi.str2charp('xxxxxxxxxxxx'))")
- assert dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp('xxxxxxxxxxxx')
+ py.test.raises(DLOpenError, "dlopen(s)")
+ rffi.free_charp(s)
+ #
+ s = rffi.str2charp(get_libc_name())
+ assert dlopen(s)
+ rffi.free_charp(s)
def test_dlsym(self):
- lib = dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp(get_libc_name())
+ lib = dlopen(s)
+ rffi.free_charp(s)
handle = rffi.cast(lltype.Ptr(lltype.FuncType([lltype.Signed],
lltype.Signed)), dlsym(lib, 'abs'))
assert 1 == handle(1)
Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rsocket.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/test/test_rsocket.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/test/test_rsocket.py Tue Nov 23 10:12:47 2010
@@ -437,3 +437,31 @@
foo = self.serv.accept()
py.test.raises(SocketError, raise_error)
+def _test_cond_include(cond):
+ # Test that _rsocket_rffi is importable even on platforms where
+ # AF_PACKET or AF_NETLINK is not defined.
+ import re
+ from pypy.rlib import _rsocket_rffi
+ srcfile = _rsocket_rffi.__file__
+ if srcfile.lower().endswith('c') or srcfile.lower().endswith('o'):
+ srcfile = srcfile[:-1] # .pyc => .py
+ assert srcfile.lower().endswith('.py')
+ sourcelines = open(srcfile, 'rb').read().splitlines()
+ found = False
+ for i, line in enumerate(sourcelines):
+ line2 = re.sub(r"(\s*COND_HEADER\s*=)",
+ r"\1'#undef %s\\n'+" % cond,
+ line)
+ if line2 != line:
+ found = True
+ sourcelines[i] = line2
+ assert found
+ d = {}
+ sourcelines.append('')
+ exec '\n'.join(sourcelines) in d
+
+def test_no_AF_PACKET():
+ _test_cond_include('AF_PACKET')
+
+def test_no_AF_NETLINK():
+ _test_cond_include('AF_NETLINK')
Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rstring.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/test/test_rstring.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/test/test_rstring.py Tue Nov 23 10:12:47 2010
@@ -29,6 +29,7 @@
s = StringBuilder()
s.append("a")
s.append("abc")
+ assert s.getlength() == len('aabc')
s.append("a")
s.append_slice("abc", 1, 2)
s.append_multiple_char('d', 4)
@@ -39,6 +40,7 @@
s.append(u'a')
s.append(u'abc')
s.append_slice(u'abcdef', 1, 2)
+ assert s.getlength() == len('aabcb')
s.append_multiple_char('d', 4)
assert s.build() == 'aabcbdddd'
assert isinstance(s.build(), unicode)
Modified: pypy/branch/arm-backend/pypy/rlib/test/test_runicode.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/test/test_runicode.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/test/test_runicode.py Tue Nov 23 10:12:47 2010
@@ -76,7 +76,7 @@
assert start == startingpos
assert stop == endingpos
return u"42424242", stop
- return "", endingpos
+ return u"", endingpos
decoder = self.getdecoder(encoding)
if addstuff:
s += "some rest in ascii"
Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rzlib.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rlib/test/test_rzlib.py (original)
+++ pypy/branch/arm-backend/pypy/rlib/test/test_rzlib.py Tue Nov 23 10:12:47 2010
@@ -189,6 +189,8 @@
assert unused3 == len('more_garbage')
assert data3 == ''
+ rzlib.deflateEnd(stream)
+
def test_decompress_max_length():
"""
@@ -205,6 +207,8 @@
assert finished2 is True
assert unused2 == 0
+ rzlib.deflateEnd(stream)
+
def test_cornercases():
"""
@@ -215,6 +219,7 @@
bytes += rzlib.compress(stream, "")
bytes += rzlib.compress(stream, "", rzlib.Z_FINISH)
assert zlib.decompress(bytes) == ""
+ rzlib.deflateEnd(stream)
stream = rzlib.inflateInit()
data, finished, unused = rzlib.decompress(stream, "")
@@ -228,3 +233,4 @@
assert finished is False
assert unused > 0
buf = buf[-unused:]
+ rzlib.deflateEnd(stream)
Modified: pypy/branch/arm-backend/pypy/rpython/annlowlevel.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/annlowlevel.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/annlowlevel.py Tue Nov 23 10:12:47 2010
@@ -136,7 +136,7 @@
return funcdesc.cachedgraph(TYPE, alt_name=valid_identifier(alt_name))
-class MixLevelHelperAnnotator:
+class MixLevelHelperAnnotator(object):
def __init__(self, rtyper):
self.rtyper = rtyper
Modified: pypy/branch/arm-backend/pypy/rpython/llinterp.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/llinterp.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/llinterp.py Tue Nov 23 10:12:47 2010
@@ -48,8 +48,7 @@
current_interpreter = None
- def __init__(self, typer, tracing=True, exc_data_ptr=None,
- malloc_check=True):
+ def __init__(self, typer, tracing=True, exc_data_ptr=None):
self.bindings = {}
self.typer = typer
# 'heap' is module or object that provides malloc, etc for lltype ops
@@ -57,9 +56,7 @@
self.exc_data_ptr = exc_data_ptr
self.frame_stack = []
self.tracer = None
- self.malloc_check = malloc_check
self.frame_class = LLFrame
- self.mallocs = {}
if tracing:
self.tracer = Tracer()
@@ -163,24 +160,6 @@
return self.exc_data_ptr
return None
- def remember_malloc(self, ptr, llframe):
- # err....
- self.mallocs[ptr._obj] = llframe
-
- def remember_free(self, ptr):
- try:
- del self.mallocs[ptr._obj]
- except KeyError:
- self._rehash_mallocs()
- del self.mallocs[ptr._obj]
-
- def _rehash_mallocs(self):
- # rehashing is needed because some objects' hash may change
- # when being turned to <C object>
- items = self.mallocs.items()
- self.mallocs = {}
- self.mallocs.update(items)
-
def _store_exception(self, exc):
raise PleaseOverwriteStoreException("You just invoked ll2ctypes callback without overwriting _store_exception on llinterpreter")
@@ -726,23 +705,23 @@
def op_malloc(self, obj, flags):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
if flavor == "stack":
result = self.heap.malloc(obj, zero=zero, flavor='raw')
self.alloca_objects.append(result)
return result
- ptr = self.heap.malloc(obj, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
+ ptr = self.heap.malloc(obj, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
def op_malloc_varsize(self, obj, flags, size):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
assert flavor in ('gc', 'raw')
try:
- ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
+ ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
except MemoryError:
self.make_llexception()
@@ -759,11 +738,10 @@
zero = flags.get('zero', False)
return self.heap.malloc_nonmovable(TYPE, size, zero=zero)
- def op_free(self, obj, flavor):
- assert isinstance(flavor, str)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_free(obj)
- self.heap.free(obj, flavor=flavor)
+ def op_free(self, obj, flags):
+ assert flags['flavor'] == 'raw'
+ track_allocation = flags.get('track_allocation', True)
+ self.heap.free(obj, flavor='raw', track_allocation=track_allocation)
def op_shrink_array(self, obj, smallersize):
return self.heap.shrink_array(obj, smallersize)
@@ -934,6 +912,9 @@
def op_gc_dump_rpy_heap(self):
raise NotImplementedError("gc_dump_rpy_heap")
+ def op_gc_typeids_z(self):
+ raise NotImplementedError("gc_typeids_z")
+
def op_do_malloc_fixedsize_clear(self):
raise NotImplementedError("do_malloc_fixedsize_clear")
@@ -1037,6 +1018,13 @@
def op_stack_malloc(self, size): # mmh
raise NotImplementedError("backend only")
+ def op_track_alloc_start(self, addr):
+ # we don't do tracking at this level
+ checkadr(addr)
+
+ def op_track_alloc_stop(self, addr):
+ checkadr(addr)
+
# ____________________________________________________________
# Overflow-detecting variants
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py Tue Nov 23 10:12:47 2010
@@ -69,7 +69,7 @@
PIECESIZE = 0x08000000
PIECES = 10
m = rmmap.mmap(-1, PIECES * PIECESIZE,
- rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS,
+ rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS|rmmap.MAP_NORESERVE,
rmmap.PROT_READ|rmmap.PROT_WRITE)
m.close = lambda : None # leak instead of giving a spurious
# error at CPython's shutdown
@@ -823,6 +823,8 @@
except (ValueError, OverflowError):
for tc in 'HIL':
if array(tc).itemsize == array('u').itemsize:
+ import struct
+ cobj &= 256 ** struct.calcsize(tc) - 1
llobj = array('u', array(tc, (cobj,)).tostring())[0]
break
else:
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/llarena.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/llarena.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/llarena.py Tue Nov 23 10:12:47 2010
@@ -441,6 +441,7 @@
rffi.INT,
sandboxsafe=True, _nowrapper=True)
_dev_zero = rffi.str2charp('/dev/zero') # prebuilt
+ lltype.render_immortal(_dev_zero)
def clear_large_memory_chunk(baseaddr, size):
# on some Unixy platforms, reading from /dev/zero is the fastest way
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/llmemory.py Tue Nov 23 10:12:47 2010
@@ -105,11 +105,13 @@
if (isinstance(self.TYPE, lltype.ContainerType)
and self.TYPE._gckind == 'gc'):
assert self.repeat == 1
- p = lltype.malloc(self.TYPE, flavor='raw', zero=zero)
+ p = lltype.malloc(self.TYPE, flavor='raw', zero=zero,
+ track_allocation=False)
return cast_ptr_to_adr(p)
else:
T = lltype.FixedSizeArray(self.TYPE, self.repeat)
- p = lltype.malloc(T, flavor='raw', zero=zero)
+ p = lltype.malloc(T, flavor='raw', zero=zero,
+ track_allocation=False)
array_adr = cast_ptr_to_adr(p)
return array_adr + ArrayItemsOffset(T)
@@ -288,7 +290,8 @@
count = 0
p = lltype.malloc(parenttype or self.TYPE, count,
immortal = self.TYPE._gckind == 'raw',
- zero = zero)
+ zero = zero,
+ track_allocation = False)
return cast_ptr_to_adr(p)
def raw_memcopy(self, srcadr, dstadr):
@@ -763,8 +766,10 @@
return '<%s>' % (self,)
def __str__(self):
return 'gctransformed_wref(%s)' % (self._ptr,)
- def _normalizedcontainer(self):
- return self._ptr._obj
+ def _normalizedcontainer(self, check=True):
+ return self._ptr._getobj(check=check)._normalizedcontainer(check=check)
+ def _was_freed(self):
+ return self._ptr._was_freed()
# ____________________________________________________________
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/lloperation.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/lloperation.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/lloperation.py Tue Nov 23 10:12:47 2010
@@ -403,6 +403,8 @@
'raw_load': LLOp(sideeffects=False),
'raw_store': LLOp(),
'stack_malloc': LLOp(), # mmh
+ 'track_alloc_start': LLOp(),
+ 'track_alloc_stop': LLOp(),
'adr_add': LLOp(canfold=True),
'adr_sub': LLOp(canfold=True),
'adr_delta': LLOp(canfold=True),
@@ -474,6 +476,7 @@
'gc_get_rpy_type_index': LLOp(),
'gc_is_rpy_instance' : LLOp(),
'gc_dump_rpy_heap' : LLOp(),
+ 'gc_typeids_z' : LLOp(),
# ------- JIT & GC interaction, only for some GCs ----------
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/lltype.py Tue Nov 23 10:12:47 2010
@@ -1,7 +1,3 @@
-import StringIO
-import traceback
-import sys
-
import py
from pypy.rlib.rarithmetic import (r_int, r_uint, intmask, r_singlefloat,
r_ulonglong, r_longlong, base_int,
@@ -10,25 +6,13 @@
from pypy.tool.uid import Hashable
from pypy.tool.tls import tlsobject
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
from types import NoneType
from sys import maxint
import weakref
TLS = tlsobject()
-# Track allocations to detect memory leaks
-# Don't track 'gc' and immortal mallocs
-TRACK_ALLOCATIONS = False
-ALLOCATED = identity_dict()
-
-def start_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = True
- ALLOCATED.clear()
-
-def stop_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = False
class _uninitialized(object):
def __init__(self, TYPE):
@@ -794,6 +778,8 @@
return llmemory.cast_adr_to_ptr(value, TGT)
elif TGT == llmemory.Address and isinstance(ORIG, Ptr):
return llmemory.cast_ptr_to_adr(value)
+ elif TGT == Signed and isinstance(ORIG, Ptr) and ORIG.TO._gckind == 'raw':
+ return llmemory.cast_adr_to_int(llmemory.cast_ptr_to_adr(value), 'symbolic')
raise TypeError("don't know how to cast from %r to %r" % (ORIG, TGT))
@@ -1380,41 +1366,21 @@
__slots__ = ('_TYPE',
'_parent_type', '_parent_index', '_keepparent',
'_wrparent',
- '__weakref__', '_traceback',
- '__storage')
+ '__weakref__',
+ '_storage')
- def __init__(self, TYPE, track_allocation=None):
+ def __init__(self, TYPE):
self._wrparent = None
self._TYPE = TYPE
self._storage = True # means "use default storage", as opposed to:
# None - container was freed
# <ctypes object> - using ctypes
# (see ll2ctypes.py)
- if track_allocation is not False and TRACK_ALLOCATIONS:
- self._traceback = self._get_traceback()
- ALLOCATED[self] = None
- else:
- self._traceback = None
-
- def _get_traceback(self):
- frame = sys._getframe().f_back.f_back.f_back.f_back
- sio = StringIO.StringIO()
- traceback.print_stack(frame, file=sio)
- return sio.getvalue()
def _free(self):
self._check() # no double-frees
self._storage = None
- def _storage_get(self):
- return self.__storage
-
- def _storage_set(self, value):
- self.__storage = value
- if value is not True and self in ALLOCATED:
- del ALLOCATED[self]
- _storage = property(_storage_get, _storage_set)
-
def _was_freed(self):
if self._storage is None:
return True
@@ -1493,12 +1459,12 @@
__slots__ = ('_hash_cache_', '_compilation_info')
- def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
my_variety = _struct_variety(TYPE._names)
return object.__new__(my_variety)
- def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
- _parentable.__init__(self, TYPE, track_allocation)
+ def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
+ _parentable.__init__(self, TYPE)
if n is not None and TYPE._arrayfld is None:
raise TypeError("%r is not variable-sized" % (TYPE,))
if n is None and TYPE._arrayfld is not None:
@@ -1506,8 +1472,7 @@
first, FIRSTTYPE = TYPE._first_struct()
for fld, typ in TYPE._flds.items():
if fld == TYPE._arrayfld:
- value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld,
- track_allocation=track_allocation)
+ value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld)
else:
value = typ._allocate(initialization=initialization, parent=self, parentindex=fld)
setattr(self, fld, value)
@@ -1568,12 +1533,12 @@
__slots__ = ('items',)
- def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None):
if not isinstance(n, int):
raise TypeError, "array length must be an int"
if n < 0:
raise ValueError, "negative array length"
- _parentable.__init__(self, TYPE, track_allocation)
+ _parentable.__init__(self, TYPE)
try:
myrange = range(n)
except OverflowError:
@@ -1640,7 +1605,7 @@
_cache = weakref.WeakKeyDictionary() # parentarray -> {subarrays}
def __init__(self, TYPE, parent, baseoffset_or_fieldname):
- _parentable.__init__(self, TYPE, track_allocation=False)
+ _parentable.__init__(self, TYPE)
self._setparentstructure(parent, baseoffset_or_fieldname)
# Keep the parent array alive, we share the same allocation.
# Don't do it if we are inside a GC object, though -- it's someone
@@ -1648,6 +1613,13 @@
if typeOf(top_container(parent))._gckind == 'raw':
self._keepparent = parent
+ def __str__(self):
+ parent = self._wrparent()
+ if parent is None:
+ return '_subarray at %s in already freed' % (self._parent_index,)
+ return '_subarray at %r in %s' % (self._parent_index,
+ parent._TYPE)
+
def __repr__(self):
parent = self._wrparent()
if parent is None:
@@ -1861,8 +1833,9 @@
return id(self.value)
-def malloc(T, n=None, flavor='gc', immortal=False, zero=False):
- assert flavor != 'cpy'
+def malloc(T, n=None, flavor='gc', immortal=False, zero=False,
+ track_allocation=True):
+ assert flavor in ('gc', 'raw')
if zero or immortal:
initialization = 'example'
elif flavor == 'raw':
@@ -1870,9 +1843,9 @@
else:
initialization = 'malloc'
if isinstance(T, Struct):
- o = _struct(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _struct(T, n, initialization=initialization)
elif isinstance(T, Array):
- o = _array(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _array(T, n, initialization=initialization)
elif isinstance(T, OpaqueType):
assert n is None
o = _opaque(T, initialization=initialization)
@@ -1880,17 +1853,57 @@
raise TypeError, "malloc for Structs and Arrays only"
if T._gckind != 'gc' and not immortal and flavor.startswith('gc'):
raise TypeError, "gc flavor malloc of a non-GC non-immortal structure"
+ if flavor == "raw" and not immortal and track_allocation:
+ leakfinder.remember_malloc(o, framedepth=2)
solid = immortal or not flavor.startswith('gc') # immortal or non-gc case
return _ptr(Ptr(T), o, solid)
-def free(p, flavor):
+def free(p, flavor, track_allocation=True):
if flavor.startswith('gc'):
raise TypeError, "gc flavor free"
T = typeOf(p)
if not isinstance(T, Ptr) or p._togckind() != 'raw':
raise TypeError, "free(): only for pointers to non-gc containers"
+ if track_allocation:
+ leakfinder.remember_free(p._obj0)
p._obj0._free()
+def render_immortal(p, track_allocation=True):
+ T = typeOf(p)
+ if not isinstance(T, Ptr) or p._togckind() != 'raw':
+ raise TypeError, "free(): only for pointers to non-gc containers"
+ if track_allocation:
+ leakfinder.remember_free(p._obj0)
+
+def _make_scoped_allocator(T):
+ class ScopedAlloc:
+ def __init__(self, n=None, zero=False):
+ if n is None:
+ self.buf = malloc(T, flavor='raw', zero=zero)
+ else:
+ self.buf = malloc(T, n, flavor='raw', zero=zero)
+
+ def __enter__(self):
+ return self.buf
+
+ def __exit__(self, *args):
+ free(self.buf, flavor='raw')
+
+ ScopedAlloc.__name__ = 'ScopedAlloc_%s' % (T,)
+ return ScopedAlloc
+_make_scoped_allocator._annspecialcase_ = 'specialize:memo'
+
+def scoped_alloc(T, n=None, zero=False):
+ """Returns a context manager which handles allocation and
+ deallocation of temporary memory. Use it in a with statement::
+
+ with scoped_alloc(Array(Signed), 1) as array:
+ ...use array...
+ ...it's freed now.
+ """
+ return _make_scoped_allocator(T)(n=n, zero=zero)
+scoped_alloc._annspecialcase_ = 'specialize:arg(0)'
+
def functionptr(TYPE, name, **attrs):
if not isinstance(TYPE, FuncType):
raise TypeError, "functionptr() for FuncTypes only"
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/rbuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/rbuilder.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/rbuilder.py Tue Nov 23 10:12:47 2010
@@ -100,6 +100,10 @@
ll_builder.used = used
@staticmethod
+ def ll_getlength(ll_builder):
+ return ll_builder.used
+
+ @staticmethod
def ll_build(ll_builder):
final_size = ll_builder.used
assert final_size >= 0
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/rclass.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/rclass.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/rclass.py Tue Nov 23 10:12:47 2010
@@ -329,16 +329,33 @@
fields['__class__'] = 'typeptr', get_type_repr(self.rtyper)
else:
# instance attributes
- if llfields is None:
- llfields = []
attrs = self.classdef.attrs.items()
attrs.sort()
+ myllfields = []
for name, attrdef in attrs:
if not attrdef.readonly:
r = self.rtyper.getrepr(attrdef.s_value)
mangled_name = 'inst_' + name
fields[name] = mangled_name, r
- llfields.append((mangled_name, r.lowleveltype))
+ myllfields.append((mangled_name, r.lowleveltype))
+
+ # Sort the instance attributes by decreasing "likely size",
+ # as reported by rffi.sizeof(), to minimize padding holes in C.
+ # Fields of the same size are sorted by name (by attrs.sort()
+ # above) just to minimize randomness.
+ def keysize((_, T)):
+ if T is lltype.Void:
+ return None
+ from pypy.rpython.lltypesystem.rffi import sizeof
+ try:
+ return -sizeof(T)
+ except StandardError:
+ return None
+ myllfields.sort(key = keysize)
+ if llfields is None:
+ llfields = myllfields
+ else:
+ llfields = llfields + myllfields
self.rbase = getinstancerepr(self.rtyper, self.classdef.basedef,
self.gcflavor)
@@ -403,7 +420,7 @@
return cast_pointer(self.lowleveltype, result)
def create_instance(self):
- return malloc(self.object_type, flavor=self.gcflavor)
+ return malloc(self.object_type, flavor=self.gcflavor, immortal=True)
def initialize_prebuilt_data(self, value, classdef, result):
if self.classdef is not None:
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/rdict.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/rdict.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/rdict.py Tue Nov 23 10:12:47 2010
@@ -306,6 +306,13 @@
hop.exception_cannot_occur()
return hop.gendirectcall(ll_clear, v_dict)
+ def rtype_method_popitem(self, hop):
+ v_dict, = hop.inputargs(self)
+ r_tuple = hop.r_result
+ cTUPLE = hop.inputconst(lltype.Void, r_tuple.lowleveltype)
+ hop.exception_is_here()
+ return hop.gendirectcall(ll_popitem, cTUPLE, v_dict)
+
class __extend__(pairtype(DictRepr, rmodel.Repr)):
def rtype_getitem((r_dict, r_key), hop):
@@ -465,6 +472,10 @@
i = ll_dict_lookup(d, key, d.keyhash(key))
if not d.entries.valid(i):
raise KeyError
+ _ll_dict_del(d, i)
+ll_dict_delitem.oopspec = 'dict.delitem(d, key)'
+
+def _ll_dict_del(d, i):
d.entries.mark_deleted(i)
d.num_items -= 1
# clear the key and the value if they are GC pointers
@@ -481,7 +492,6 @@
num_entries = len(d.entries)
if num_entries > DICT_INITSIZE and d.num_items < num_entries / 4:
ll_dict_resize(d)
-ll_dict_delitem.oopspec = 'dict.delitem(d, key)'
def ll_dict_resize(d):
old_entries = d.entries
@@ -810,3 +820,26 @@
i = ll_dict_lookup(d, key, d.keyhash(key))
return d.entries.valid(i)
ll_contains.oopspec = 'dict.contains(d, key)'
+
+POPITEMINDEX = lltype.Struct('PopItemIndex', ('nextindex', lltype.Signed))
+global_popitem_index = lltype.malloc(POPITEMINDEX, zero=True, immortal=True)
+
+def ll_popitem(ELEM, dic):
+ entries = dic.entries
+ dmask = len(entries) - 1
+ base = global_popitem_index.nextindex
+ counter = 0
+ while counter <= dmask:
+ i = (base + counter) & dmask
+ counter += 1
+ if entries.valid(i):
+ break
+ else:
+ raise KeyError
+ global_popitem_index.nextindex += counter
+ entry = entries[i]
+ r = lltype.malloc(ELEM.TO)
+ r.item0 = recast(ELEM.TO.item0, entry.key)
+ r.item1 = recast(ELEM.TO.item1, entry.value)
+ _ll_dict_del(dic, i)
+ return r
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/rffi.py Tue Nov 23 10:12:47 2010
@@ -644,10 +644,18 @@
"""
Either free a non-moving buffer or keep the original storage alive.
"""
- if rgc.can_move(data):
+ # We cannot rely on rgc.can_move(data) here, because its result
+ # might have changed since get_nonmovingbuffer(). Instead we check
+ # if 'buf' points inside 'data'. This is only possible if we
+ # followed the 2nd case in get_nonmovingbuffer(); in the first case,
+ # 'buf' points to its own raw-malloced memory.
+ data = llstrtype(data)
+ data_start = cast_ptr_to_adr(data) + \
+ offsetof(STRTYPE, 'chars') + itemoffsetof(STRTYPE.chars, 0)
+ followed_2nd_path = (buf == cast(TYPEP, data_start))
+ keepalive_until_here(data)
+ if not followed_2nd_path:
lltype.free(buf, flavor='raw')
- else:
- keepalive_until_here(data)
# int -> (char*, str)
def alloc_buffer(count):
@@ -916,3 +924,11 @@
"""
return cast(lltype.Signed, getattr(pdst, fieldname))
getintfield._annspecialcase_ = 'specialize:ll_and_arg(1)'
+
+class scoped_str2charp:
+ def __init__(self, value):
+ self.buf = str2charp(value)
+ def __enter__(self):
+ return self.buf
+ def __exit__(self, *args):
+ free_charp(self.buf)
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_ll2ctypes.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Tue Nov 23 10:12:47 2010
@@ -765,6 +765,7 @@
assert abs(float(b[1]) - 1.1) < 1E-6
assert isinstance(b[2], rffi.r_singlefloat)
assert abs(float(b[2]) - 2.2) < 1E-6
+ lltype.free(a, flavor='raw')
def test_different_signatures(self):
if sys.platform=='win32':
@@ -879,6 +880,7 @@
qsort(rffi.cast(rffi.VOIDP, a), 5, rffi.sizeof(rffi.INT), compare)
for i in range(5):
assert a[i] == i + 1
+ lltype.free(a, flavor='raw')
def test_array_type_bug(self):
A = lltype.Array(lltype.Signed)
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_llmemory.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_llmemory.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_llmemory.py Tue Nov 23 10:12:47 2010
@@ -324,12 +324,14 @@
p_t = lltype.malloc(T)
assert p_t.s == lltype.nullptr(S)
# raw malloc does not
- p_raw_t = lltype.malloc(T, flavor="raw")
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
+ U = lltype.Struct("U", ('x', lltype.Signed))
+ p_raw_t = lltype.malloc(U, flavor="raw")
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+ lltype.free(p_raw_t, flavor="raw")
# this sort of raw_malloc too
- p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(T)), lltype.Ptr(T))
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
-
+ p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(U)), lltype.Ptr(U))
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+
def test_raw_malloc_signed_bunch():
adr = raw_malloc(sizeof(lltype.Signed) * 50)
@@ -601,7 +603,8 @@
a = lltype.malloc(A, flavor='raw')
src = cast_ptr_to_adr(a) + itemoffsetof(A, 0)
raw_memclear(src, sizeof(lltype.Signed) * 0)
-
+ lltype.free(a, flavor="raw")
+
def test_nonneg():
S1 = lltype.GcStruct('S1', ('x', lltype.Float))
A1 = lltype.GcArray(lltype.Float)
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_lltype.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_lltype.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_lltype.py Tue Nov 23 10:12:47 2010
@@ -1,7 +1,9 @@
+from __future__ import with_statement
import py
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
def isweak(p, T):
try:
@@ -804,22 +806,20 @@
class TestTrackAllocation:
- def setup_method(self, func):
- start_tracking_allocations()
-
- def teardown_method(self, func):
- assert not lltype.ALLOCATED, "Memory was not correctly freed"
- stop_tracking_allocations()
+ def test_automatic_tracking(self):
+ # calls to start_tracking_allocations/stop_tracking_allocations
+ # should occur automatically from pypy/conftest.py. Check that.
+ assert leakfinder.TRACK_ALLOCATIONS
def test_track_allocation(self):
"""A malloc'd buffer fills the ALLOCATED dictionary"""
- assert lltype.TRACK_ALLOCATIONS
- assert not lltype.ALLOCATED
+ assert leakfinder.TRACK_ALLOCATIONS
+ assert not leakfinder.ALLOCATED
buf = malloc(Array(Signed), 1, flavor="raw")
- assert len(lltype.ALLOCATED) == 1
- assert lltype.ALLOCATED.keys() == [buf._obj]
+ assert len(leakfinder.ALLOCATED) == 1
+ assert leakfinder.ALLOCATED.keys() == [buf._obj]
free(buf, flavor="raw")
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_str_from_buffer(self):
"""gc-managed memory does not need to be freed"""
@@ -828,16 +828,28 @@
for i in range(size): raw_buf[i] = 'a'
rstr = rffi.str_from_buffer(raw_buf, gc_buf, size, size)
rffi.keep_buffer_alive_until_here(raw_buf, gc_buf)
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_leak_traceback(self):
"""Test info stored for allocated items"""
buf = malloc(Array(Signed), 1, flavor="raw")
- traceback = lltype.ALLOCATED.keys()[0]._traceback
+ traceback = leakfinder.ALLOCATED.values()[0]
lines = traceback.splitlines()
assert 'malloc(' in lines[-1] and 'flavor="raw")' in lines[-1]
- # XXX The traceback should not be too long
+ # The traceback should not be too long
print traceback
free(buf, flavor="raw")
+
+ def test_no_tracking(self):
+ p1 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ p2 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ free(p2, flavor='raw', track_allocation=False)
+ # p1 is not freed
+
+ def test_scoped_allocator(self):
+ with scoped_alloc(Array(Signed), 1) as array:
+ array[0] = -42
+ x = array[0]
+ assert x == -42
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_rffi.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_rffi.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_rffi.py Tue Nov 23 10:12:47 2010
@@ -9,7 +9,7 @@
from pypy.rpython.lltypesystem.rstr import STR
from pypy.rpython.lltypesystem import lltype
from pypy.tool.udir import udir
-from pypy.rpython.test.test_llinterp import interpret, MallocMismatch
+from pypy.rpython.test.test_llinterp import interpret
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.annotation.annrpython import RPythonAnnotator
from pypy.rpython.rtyper import RPythonTyper
@@ -787,3 +787,10 @@
mixann.getgraph(f2, [], s_None)
mixann.finish()
+def test_force_cast_unichar():
+ x = cast(lltype.UniChar, -1)
+ assert isinstance(x, unicode)
+ if sys.maxunicode == 65535:
+ assert cast(LONG, x) == 65535
+ else:
+ assert cast(LONG, cast(INT, x)) == -1
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gc/base.py Tue Nov 23 10:12:47 2010
@@ -20,12 +20,15 @@
prebuilt_gc_objects_are_static_roots = True
object_minimal_size = 0
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE):
+ def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ translated_to_c=True):
self.gcheaderbuilder = GCHeaderBuilder(self.HDR)
self.AddressStack = get_address_stack(chunk_size)
self.AddressDeque = get_address_deque(chunk_size)
self.AddressDict = AddressDict
self.config = config
+ assert isinstance(translated_to_c, bool)
+ self.translated_to_c = translated_to_c
def setup(self):
# all runtime mutable values' setup should happen here
@@ -79,7 +82,7 @@
def set_root_walker(self, root_walker):
self.root_walker = root_walker
- def write_barrier(self, addr_struct):
+ def write_barrier(self, newvalue, addr_struct):
pass
def statistics(self, index):
@@ -244,6 +247,21 @@
(not self.config.taggedpointers or
llmemory.cast_adr_to_int(addr) & 1 == 0))
+ def enumerate_all_roots(self, callback, arg):
+ """For each root object, invoke callback(obj, arg).
+ 'callback' should not be a bound method.
+ Note that this method is not suitable for actually doing the
+ collection in a moving GC, because you cannot write back a
+ modified address. It is there only for inspection.
+ """
+ # overridden in some subclasses, for GCs which have an additional
+ # list of last generation roots
+ callback2, attrname = _convert_callback_formats(callback) # :-/
+ setattr(self, attrname, arg)
+ self.root_walker.walk_roots(callback2, callback2, callback2)
+ self.run_finalizers.foreach(callback, arg)
+ enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)'
+
def debug_check_consistency(self):
"""To use after a collection. If self.DEBUG is set, this
enumerates all roots and traces all objects to check if we didn't
@@ -257,8 +275,7 @@
self._debug_pending = self.AddressStack()
if not we_are_translated():
self.root_walker._walk_prebuilt_gc(self._debug_record)
- callback = GCBase._debug_callback
- self.root_walker.walk_roots(callback, callback, callback)
+ self.enumerate_all_roots(GCBase._debug_callback, self)
pending = self._debug_pending
while pending.non_empty():
obj = pending.pop()
@@ -272,9 +289,8 @@
seen.add(obj)
self.debug_check_object(obj)
self._debug_pending.append(obj)
- def _debug_callback(self, root):
- obj = root.address[0]
- ll_assert(bool(obj), "NULL address from walk_roots()")
+ @staticmethod
+ def _debug_callback(obj, self):
self._debug_record(obj)
def _debug_callback2(self, pointer, ignored):
obj = pointer.address[0]
@@ -429,3 +445,17 @@
if factor != 1:
return 0.0
return value
+
+def _convert_callback_formats(callback):
+ callback = getattr(callback, 'im_func', callback)
+ if callback not in _converted_callback_formats:
+ def callback2(gc, root):
+ obj = root.address[0]
+ ll_assert(bool(obj), "NULL address from walk_roots()")
+ callback(obj, getattr(gc, attrname))
+ attrname = '_callback2_arg%d' % len(_converted_callback_formats)
+ _converted_callback_formats[callback] = callback2, attrname
+ return _converted_callback_formats[callback]
+
+_convert_callback_formats._annspecialcase_ = 'specialize:memo'
+_converted_callback_formats = {}
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/generation.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gc/generation.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gc/generation.py Tue Nov 23 10:12:47 2010
@@ -5,7 +5,6 @@
from pypy.rpython.memory.gc.base import read_from_env
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
from pypy.rpython.lltypesystem import lltype, llmemory, llarena
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rlib.objectmodel import free_non_gc_object
from pypy.rlib.debug import ll_assert
from pypy.rlib.debug import debug_print, debug_start, debug_stop
@@ -49,15 +48,17 @@
nursery_hash_base = -1
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ def __init__(self, config,
nursery_size=32*WORD,
min_nursery_size=32*WORD,
auto_nursery_size=False,
space_size=1024*WORD,
- max_space_size=sys.maxint//2+1):
- SemiSpaceGC.__init__(self, config, chunk_size = chunk_size,
+ max_space_size=sys.maxint//2+1,
+ **kwds):
+ SemiSpaceGC.__init__(self, config,
space_size = space_size,
- max_space_size = max_space_size)
+ max_space_size = max_space_size,
+ **kwds)
assert min_nursery_size <= nursery_size <= space_size // 2
self.initial_nursery_size = nursery_size
self.auto_nursery_size = auto_nursery_size
@@ -157,6 +158,14 @@
"odd-valued (i.e. tagged) pointer unexpected here")
return self.nursery <= addr < self.nursery_top
+ def appears_to_be_in_nursery(self, addr):
+ # same as is_in_nursery(), but may return True accidentally if
+ # 'addr' is a tagged pointer with just the wrong value.
+ if not self.translated_to_c:
+ if not self.is_valid_gc_object(addr):
+ return False
+ return self.nursery <= addr < self.nursery_top
+
def malloc_fixedsize_clear(self, typeid, size, can_collect,
has_finalizer=False, contains_weakptr=False):
if (has_finalizer or not can_collect or
@@ -326,7 +335,7 @@
addr = pointer.address[0]
newaddr = self.copy(addr)
pointer.address[0] = newaddr
- self.write_into_last_generation_obj(obj)
+ self.write_into_last_generation_obj(obj, newaddr)
# ____________________________________________________________
# Implementation of nursery-only collections
@@ -457,9 +466,9 @@
# "if addr_struct.int0 & JIT_WB_IF_FLAG: remember_young_pointer()")
JIT_WB_IF_FLAG = GCFLAG_NO_YOUNG_PTRS
- def write_barrier(self, addr_struct):
- if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
- self.remember_young_pointer(addr_struct)
+ def write_barrier(self, newvalue, addr_struct):
+ if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
+ self.remember_young_pointer(addr_struct, newvalue)
def _setup_wb(self):
DEBUG = self.DEBUG
@@ -470,23 +479,33 @@
# For x86, there is also an extra requirement: when the JIT calls
# remember_young_pointer(), it assumes that it will not touch the SSE
# registers, so it does not save and restore them (that's a *hack*!).
- def remember_young_pointer(addr_struct):
+ def remember_young_pointer(addr_struct, addr):
#llop.debug_print(lltype.Void, "\tremember_young_pointer",
# addr_struct, "<-", addr)
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
- self.old_objects_pointing_to_young.append(addr_struct)
- self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS
- self.write_into_last_generation_obj(addr_struct)
+ #
+ # What is important in this function is that it *must*
+ # clear the flag GCFLAG_NO_YOUNG_PTRS from 'addr_struct'
+ # if 'addr' is in the nursery. It is ok if, accidentally,
+ # it also clears the flag in some more rare cases, like
+ # 'addr' being a tagged pointer whose value happens to be
+ # a large integer that fools is_in_nursery().
+ if self.appears_to_be_in_nursery(addr):
+ self.old_objects_pointing_to_young.append(addr_struct)
+ self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS
+ self.write_into_last_generation_obj(addr_struct, addr)
remember_young_pointer._dont_inline_ = True
self.remember_young_pointer = remember_young_pointer
- def write_into_last_generation_obj(self, addr_struct):
+ def write_into_last_generation_obj(self, addr_struct, addr):
objhdr = self.header(addr_struct)
if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
- objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
- self.last_generation_root_objects.append(addr_struct)
+ if (self.is_valid_gc_object(addr) and
+ not self.is_last_generation(addr)):
+ objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
+ self.last_generation_root_objects.append(addr_struct)
write_into_last_generation_obj._always_inline_ = True
def assume_young_pointers(self, addr_struct):
@@ -553,16 +572,10 @@
def _compute_current_nursery_hash(self, obj):
return intmask(llmemory.cast_adr_to_int(obj) + self.nursery_hash_base)
- def heap_stats_walk_roots(self):
- self.last_generation_root_objects.foreach(
- self._track_heap_ext, None)
- self.root_walker.walk_roots(
- SemiSpaceGC._track_heap_root,
- SemiSpaceGC._track_heap_root,
- SemiSpaceGC._track_heap_root)
-
- def _track_heap_ext(self, adr, ignored):
- self.trace(adr, self.track_heap_parent, adr)
+ def enumerate_all_roots(self, callback, arg):
+ self.last_generation_root_objects.foreach(callback, arg)
+ SemiSpaceGC.enumerate_all_roots(self, callback, arg)
+ enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)'
def debug_check_object(self, obj):
"""Check the invariants about 'obj' that should be true
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/inspector.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gc/inspector.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gc/inspector.py Tue Nov 23 10:12:47 2010
@@ -4,25 +4,22 @@
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rlib.objectmodel import free_non_gc_object
from pypy.rpython.module.ll_os import underscore_on_windows
-from pypy.rlib import rposix
+from pypy.rlib import rposix, rgc
from pypy.rpython.memory.support import AddressDict, get_address_stack
# ---------- implementation of pypy.rlib.rgc.get_rpy_roots() ----------
-def _counting_rpy_root(gc, root):
+def _counting_rpy_root(obj, gc):
gc._count_rpy += 1
def _do_count_rpy_roots(gc):
gc._count_rpy = 0
- gc.root_walker.walk_roots(
- _counting_rpy_root,
- _counting_rpy_root,
- _counting_rpy_root)
+ gc.enumerate_all_roots(_counting_rpy_root, gc)
return gc._count_rpy
-def _append_rpy_root(gc, root):
+def _append_rpy_root(obj, gc):
# Can use the gc list, but should not allocate!
# It is essential that the list is not resizable!
lst = gc._list_rpy
@@ -30,15 +27,12 @@
if index >= len(lst):
raise ValueError
gc._count_rpy = index + 1
- lst[index] = llmemory.cast_adr_to_ptr(root.address[0], llmemory.GCREF)
+ lst[index] = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF)
def _do_append_rpy_roots(gc, lst):
gc._count_rpy = 0
gc._list_rpy = lst
- gc.root_walker.walk_roots(
- _append_rpy_root,
- _append_rpy_root,
- _append_rpy_root)
+ gc.enumerate_all_roots(_append_rpy_root, gc)
gc._list_rpy = None
def get_rpy_roots(gc):
@@ -172,12 +166,7 @@
self.pending.append(obj)
def add_roots(self):
- self.gc._heap_dumper = self
- self.gc.root_walker.walk_roots(
- _hd_add_root,
- _hd_add_root,
- _hd_add_root)
- self.gc._heap_dumper = None
+ self.gc.enumerate_all_roots(_hd_add_root, self)
pendingroots = self.pending
self.pending = AddressStack()
self.walk(pendingroots)
@@ -188,8 +177,8 @@
while pending.non_empty():
self.writeobj(pending.pop())
-def _hd_add_root(gc, root):
- gc._heap_dumper.add(root.address[0])
+def _hd_add_root(obj, heap_dumper):
+ heap_dumper.add(obj)
def dump_rpy_heap(gc, fd):
heapdumper = HeapDumper(gc, fd)
@@ -198,3 +187,7 @@
heapdumper.flush()
heapdumper.delete()
return True
+
+def get_typeids_z(gc):
+ srcaddress = gc.root_walker.gcdata.typeids_z
+ return llmemory.cast_adr_to_ptr(srcaddress, lltype.Ptr(rgc.ARRAY_OF_CHAR))
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/markcompact.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gc/markcompact.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gc/markcompact.py Tue Nov 23 10:12:47 2010
@@ -2,7 +2,6 @@
from pypy.rpython.memory.gc.base import MovingGCBase, read_from_env
from pypy.rlib.debug import ll_assert, have_debug_prints
from pypy.rlib.debug import debug_print, debug_start, debug_stop
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack, get_address_deque
from pypy.rpython.memory.support import AddressDict
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
@@ -86,9 +85,9 @@
free = NULL
next_collect_after = -1
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096,
- min_next_collect_after=128):
- MovingGCBase.__init__(self, config, chunk_size)
+ def __init__(self, config, space_size=4096,
+ min_next_collect_after=128, **kwds):
+ MovingGCBase.__init__(self, config, **kwds)
self.space_size = space_size
self.min_next_collect_after = min_next_collect_after
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/marksweep.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gc/marksweep.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gc/marksweep.py Tue Nov 23 10:12:47 2010
@@ -1,7 +1,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free
from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack
from pypy.rpython.memory.gcheader import GCHeaderBuilder
from pypy.rpython.lltypesystem import lltype, llmemory, rffi, llgroup
@@ -48,9 +47,9 @@
# translating to a real backend.
TRANSLATION_PARAMS = {'start_heap_size': 8*1024*1024} # XXX adjust
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096):
+ def __init__(self, config, start_heap_size=4096, **kwds):
self.param_start_heap_size = start_heap_size
- GCBase.__init__(self, config, chunk_size)
+ GCBase.__init__(self, config, **kwds)
def setup(self):
GCBase.setup(self)
@@ -714,8 +713,8 @@
_alloc_flavor_ = "raw"
COLLECT_EVERY = 2000
- def __init__(self, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096):
- MarkSweepGC.__init__(self, chunk_size, start_heap_size)
+ def __init__(self, config, **kwds):
+ MarkSweepGC.__init__(self, config, **kwds)
self.count_mallocs = 0
def maybe_collect(self):
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gc/minimark.py Tue Nov 23 10:12:47 2010
@@ -1,10 +1,42 @@
+""" MiniMark GC.
+
+Environment variables can be used to fine-tune the following parameters:
+
+ PYPY_GC_NURSERY The nursery size. Defaults to half the size of
+ the L2 cache. Try values like '1.2MB'.
+
+ PYPY_GC_MAJOR_COLLECT Major collection memory factor. Default is '1.82',
+ which means trigger a major collection when the
+ memory consumed equals 1.82 times the memory
+ really used at the end of the previous major
+ collection.
+
+ PYPY_GC_GROWTH Major collection threshold's max growth rate.
+ Default is '1.3'. Useful to collect more often
+ than normally on sudden memory growth, e.g. when
+ there is a temporary peak in memory usage.
+
+ PYPY_GC_MAX The max heap size. If coming near this limit, it
+ will first collect more often, then raise an
+ RPython MemoryError, and if that is not enough,
+ crash the program with a fatal error. Try values
+ like '1.6GB'.
+
+ PYPY_GC_MIN Don't collect while the memory size is below this
+ limit. Useful to avoid spending all the time in
+ the GC in very small programs. Defaults to 8
+ times the nursery.
+"""
+# XXX Should find a way to bound the major collection threshold by the
+# XXX total addressable size. Maybe by keeping some minimarkpage arenas
+# XXX pre-reserved, enough for a few nursery collections? What about
+# XXX raw-malloced memory?
import sys
from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup
from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage
from pypy.rpython.memory.gc.base import GCBase, MovingGCBase
from pypy.rpython.memory.gc import minimarkpage, base, generation
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint
from pypy.rlib.rarithmetic import LONG_BIT_SHIFT
from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop
@@ -78,7 +110,7 @@
# During a minor collection, the objects in the nursery that are
# moved outside are changed in-place: their header is replaced with
- # the value -1, and the following word is set to the address of
+ # the value -42, and the following word is set to the address of
# where the object was moved. This means that all objects in the
# nursery need to be at least 2 words long, but objects outside the
# nursery don't need to.
@@ -88,13 +120,8 @@
TRANSLATION_PARAMS = {
# Automatically adjust the size of the nursery and the
- # 'major_collection_threshold' from the environment. For
- # 'nursery_size' it will look it up in the env var
- # PYPY_GC_NURSERY and fall back to half the size of
- # the L2 cache. For 'major_collection_threshold' it will look
- # it up in the env var PYPY_GC_MAJOR_COLLECT. It also sets
- # 'max_heap_size' to PYPY_GC_MAX. Finally, PYPY_GC_MIN sets
- # the minimal value of 'next_major_collection_threshold'.
+ # 'major_collection_threshold' from the environment.
+ # See docstring at the start of the file.
"read_from_env": True,
# The size of the nursery. Note that this is only used as a
@@ -122,6 +149,13 @@
# we trigger the next major collection.
"major_collection_threshold": 1.82,
+ # Threshold to avoid that the total heap size grows by a factor of
+ # major_collection_threshold at every collection: it can only
+ # grow at most by the following factor from one collection to the
+ # next. Used e.g. when there is a sudden, temporary peak in memory
+ # usage; this avoids that the upper bound grows too fast.
+ "growth_rate_max": 1.3,
+
# The number of array indices that are mapped to a single bit in
# write_barrier_from_array(). Must be a power of two. The default
# value of 128 means that card pages are 512 bytes (1024 on 64-bits)
@@ -140,23 +174,26 @@
"large_object_gcptrs": 8250*WORD,
}
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ def __init__(self, config,
read_from_env=False,
nursery_size=32*WORD,
page_size=16*WORD,
arena_size=64*WORD,
small_request_threshold=5*WORD,
major_collection_threshold=2.5,
+ growth_rate_max=2.5, # for tests
card_page_indices=0,
large_object=8*WORD,
large_object_gcptrs=10*WORD,
- ArenaCollectionClass=None):
- MovingGCBase.__init__(self, config, chunk_size)
+ ArenaCollectionClass=None,
+ **kwds):
+ MovingGCBase.__init__(self, config, **kwds)
assert small_request_threshold % WORD == 0
self.read_from_env = read_from_env
self.nursery_size = nursery_size
self.small_request_threshold = small_request_threshold
self.major_collection_threshold = major_collection_threshold
+ self.growth_rate_max = growth_rate_max
self.num_major_collects = 0
self.min_heap_size = 0.0
self.max_heap_size = 0.0
@@ -178,6 +215,7 @@
self.nursery = NULL
self.nursery_free = NULL
self.nursery_top = NULL
+ self.debug_always_do_minor_collect = False
#
# The ArenaCollection() handles the nonmovable objects allocation.
if ArenaCollectionClass is None:
@@ -245,6 +283,10 @@
# From there on, the GC is fully initialized and the code
# below can use it
newsize = base.read_from_env('PYPY_GC_NURSERY')
+ # PYPY_GC_NURSERY=1 forces a minor collect for every malloc.
+ # Useful to debug external factors, like trackgcroot or the
+ # handling of the write barrier.
+ self.debug_always_do_minor_collect = newsize == 1
if newsize <= 0:
newsize = generation.estimate_best_nursery_size()
if newsize <= 0:
@@ -252,9 +294,13 @@
newsize = max(newsize, minsize)
#
major_coll = base.read_float_from_env('PYPY_GC_MAJOR_COLLECT')
- if major_coll >= 1.0:
+ if major_coll > 1.0:
self.major_collection_threshold = major_coll
#
+ growth = base.read_float_from_env('PYPY_GC_GROWTH')
+ if growth > 1.0:
+ self.growth_rate_max = growth
+ #
min_heap_size = base.read_uint_from_env('PYPY_GC_MIN')
if min_heap_size > 0:
self.min_heap_size = float(min_heap_size)
@@ -290,11 +336,19 @@
# initialize the threshold
self.min_heap_size = max(self.min_heap_size, self.nursery_size *
self.major_collection_threshold)
+ self.next_major_collection_threshold = self.min_heap_size
self.set_major_threshold_from(0.0)
debug_stop("gc-set-nursery-size")
- def set_major_threshold_from(self, threshold):
+
+ def set_major_threshold_from(self, threshold, reserving_size=0):
# Set the next_major_collection_threshold.
+ threshold_max = (self.next_major_collection_threshold *
+ self.growth_rate_max)
+ if threshold > threshold_max:
+ threshold = threshold_max
+ #
+ threshold += reserving_size
if threshold < self.min_heap_size:
threshold = self.min_heap_size
#
@@ -444,6 +498,10 @@
result = self.nursery_free
self.nursery_free = result + totalsize
ll_assert(self.nursery_free <= self.nursery_top, "nursery overflow")
+ #
+ if self.debug_always_do_minor_collect:
+ self.nursery_free = self.nursery_top
+ #
return result
collect_and_reserve._dont_inline_ = True
@@ -636,13 +694,24 @@
"odd-valued (i.e. tagged) pointer unexpected here")
return self.nursery <= addr < self.nursery_top
+ def appears_to_be_in_nursery(self, addr):
+ # same as is_in_nursery(), but may return True accidentally if
+ # 'addr' is a tagged pointer with just the wrong value.
+ if not self.translated_to_c:
+ if not self.is_valid_gc_object(addr):
+ return False
+ return self.nursery <= addr < self.nursery_top
+
def is_forwarded(self, obj):
"""Returns True if the nursery obj is marked as forwarded.
Implemented a bit obscurely by checking an unrelated flag
- that can never be set on a young object -- except if tid == -1.
+ that can never be set on a young object -- except if tid == -42.
"""
assert self.is_in_nursery(obj)
- return self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING
+ result = (self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING != 0)
+ if result:
+ ll_assert(self.header(obj).tid == -42, "bogus header for young obj")
+ return result
def get_forwarding_address(self, obj):
return llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw
@@ -726,16 +795,20 @@
def JIT_max_size_of_young_obj(cls):
return cls.TRANSLATION_PARAMS['large_object']
- def write_barrier(self, addr_struct):
+ @classmethod
+ def JIT_minimal_size_in_nursery(cls):
+ return cls.minimal_size_in_nursery
+
+ def write_barrier(self, newvalue, addr_struct):
if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
- self.remember_young_pointer(addr_struct)
+ self.remember_young_pointer(addr_struct, newvalue)
- def write_barrier_from_array(self, addr_array, index):
+ def write_barrier_from_array(self, newvalue, addr_array, index):
if self.header(addr_array).tid & GCFLAG_NO_YOUNG_PTRS:
if self.card_page_indices > 0: # <- constant-folded
self.remember_young_pointer_from_array(addr_array, index)
else:
- self.remember_young_pointer(addr_array)
+ self.remember_young_pointer(addr_array, newvalue)
def _init_writebarrier_logic(self):
DEBUG = self.DEBUG
@@ -746,27 +819,28 @@
# For x86, there is also an extra requirement: when the JIT calls
# remember_young_pointer(), it assumes that it will not touch the SSE
# registers, so it does not save and restore them (that's a *hack*!).
- def remember_young_pointer(addr_struct):
+ def remember_young_pointer(addr_struct, newvalue):
# 'addr_struct' is the address of the object in which we write.
+ # 'newvalue' is the address that we are going to write in there.
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
#
- # We assume that what we are writing is a pointer to the nursery
- # (and don't care for the fact that this new pointer may not
- # actually point to the nursery, which seems ok). What we need is
+ # If it seems that what we are writing is a pointer to the nursery
+ # (as checked with appears_to_be_in_nursery()), then we need
# to remove the flag GCFLAG_NO_YOUNG_PTRS and add the old object
# to the list 'old_objects_pointing_to_young'. We know that
# 'addr_struct' cannot be in the nursery, because nursery objects
# never have the flag GCFLAG_NO_YOUNG_PTRS to start with.
- self.old_objects_pointing_to_young.append(addr_struct)
objhdr = self.header(addr_struct)
- objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
+ if self.appears_to_be_in_nursery(newvalue):
+ self.old_objects_pointing_to_young.append(addr_struct)
+ objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
#
# Second part: if 'addr_struct' is actually a prebuilt GC
# object and it's the first time we see a write to it, we
# add it to the list 'prebuilt_root_objects'. Note that we
- # do it even in the (rare?) case of 'addr' being another
+ # do it even in the (rare?) case of 'addr' being NULL or another
# prebuilt object, to simplify code.
if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
@@ -780,16 +854,24 @@
def _init_writebarrier_with_card_marker(self):
+ DEBUG = self.DEBUG
def remember_young_pointer_from_array(addr_array, index):
# 'addr_array' is the address of the object in which we write,
# which must have an array part; 'index' is the index of the
# item that is (or contains) the pointer that we write.
+ if DEBUG:
+ ll_assert(not self.is_in_nursery(addr_array),
+ "nursery array with GCFLAG_NO_YOUNG_PTRS")
objhdr = self.header(addr_array)
if objhdr.tid & GCFLAG_HAS_CARDS == 0:
#
- # no cards, use default logic. The 'nocard_logic()' is just
- # 'remember_young_pointer()', but forced to be inlined here.
- nocard_logic(addr_array)
+ # no cards, use default logic. Mostly copied from above.
+ self.old_objects_pointing_to_young.append(addr_array)
+ objhdr = self.header(addr_array)
+ objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
+ if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
+ objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
+ self.prebuilt_root_objects.append(addr_array)
return
#
# 'addr_array' is a raw_malloc'ed array with card markers
@@ -807,17 +889,15 @@
return
#
# We set the flag (even if the newly written address does not
- # actually point to the nursery -- like remember_young_pointer()).
+ # actually point to the nursery, which seems to be ok -- actually
+ # it seems more important that remember_young_pointer_from_array()
+ # does not take 3 arguments).
addr_byte.char[0] = chr(byte | bitmask)
#
if objhdr.tid & GCFLAG_CARDS_SET == 0:
self.old_objects_with_cards_set.append(addr_array)
objhdr.tid |= GCFLAG_CARDS_SET
- nocard_logic = func_with_new_name(self.remember_young_pointer,
- 'remember_young_pointer_nocard')
- del nocard_logic._dont_inline_
- nocard_logic._always_inline_ = True
remember_young_pointer_from_array._dont_inline_ = True
self.remember_young_pointer_from_array = (
remember_young_pointer_from_array)
@@ -891,7 +971,7 @@
#
# Now all live nursery objects should be out. Update the
# young weakrefs' targets.
- if self.young_objects_with_weakrefs.length() > 0:
+ if self.young_objects_with_weakrefs.non_empty():
self.invalidate_young_weakrefs()
#
# Clear this mapping.
@@ -985,7 +1065,7 @@
obj = oldlist.pop()
#
# Add the flag GCFLAG_NO_YOUNG_PTRS. All live objects should have
- # this flag after a nursery collection.
+ # this flag set after a nursery collection.
self.header(obj).tid |= GCFLAG_NO_YOUNG_PTRS
#
# Trace the 'obj' to replace pointers to nursery with pointers
@@ -1048,7 +1128,7 @@
# nursery are kept unchanged in this step.
llmemory.raw_memcopy(obj - size_gc_header, newhdr, totalsize)
#
- # Set the old object's tid to -1 (containing all flags) and
+ # Set the old object's tid to -42 (containing all flags) and
# replace the old object's content with the target address.
# A bit of no-ops to convince llarena that we are changing
# the layout, in non-translated versions.
@@ -1056,7 +1136,7 @@
llarena.arena_reset(obj - size_gc_header, totalsize, 0)
llarena.arena_reserve(obj - size_gc_header,
size_gc_header + llmemory.sizeof(FORWARDSTUB))
- self.header(obj).tid = -1
+ self.header(obj).tid = -42
newobj = newhdr + size_gc_header
llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw = newobj
#
@@ -1167,8 +1247,8 @@
# have allocated 'major_collection_threshold' times more than
# we currently have.
bounded = self.set_major_threshold_from(
- (self.get_total_memory_used() * self.major_collection_threshold)
- + reserving_size)
+ self.get_total_memory_used() * self.major_collection_threshold,
+ reserving_size)
#
# Max heap size: gives an upper bound on the threshold. If we
# already have at least this much allocated, raise MemoryError.
@@ -1257,6 +1337,11 @@
self.run_finalizers.foreach(self._collect_obj,
self.objects_to_trace)
+ def enumerate_all_roots(self, callback, arg):
+ self.prebuilt_root_objects.foreach(callback, arg)
+ MovingGCBase.enumerate_all_roots(self, callback, arg)
+ enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)'
+
@staticmethod
def _collect_obj(obj, objects_to_trace):
objects_to_trace.append(obj)
@@ -1308,7 +1393,7 @@
if self.is_valid_gc_object(obj):
if self.is_in_nursery(obj):
#
- # The object not a tagged pointer, and is it still in the
+ # The object is not a tagged pointer, and it is still in the
# nursery. Find or allocate a "shadow" object, which is
# where the object will be moved by the next minor
# collection
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/minimarkpage.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gc/minimarkpage.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gc/minimarkpage.py Tue Nov 23 10:12:47 2010
@@ -100,11 +100,14 @@
# allocation of the given size.
length = small_request_threshold / WORD + 1
self.page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.full_page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.nblocks_for_size = lltype.malloc(rffi.CArray(lltype.Signed),
- length, flavor='raw')
+ length, flavor='raw',
+ immortal=True)
self.hdrsize = llmemory.raw_malloc_usage(llmemory.sizeof(PAGE_HEADER))
assert page_size > self.hdrsize
self.nblocks_for_size[0] = 0 # unused
@@ -114,11 +117,13 @@
self.max_pages_per_arena = arena_size // page_size
self.arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
# this is used in mass_free() only
self.old_arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
#
# the arena currently consumed; it must have at least one page
# available, or be NULL. The arena object that we point to is
@@ -281,7 +286,7 @@
npages = (arena_end - firstpage) // self.page_size
#
# Allocate an ARENA object and initialize it
- arena = lltype.malloc(ARENA, flavor='raw')
+ arena = lltype.malloc(ARENA, flavor='raw', track_allocation=False)
arena.base = arena_base
arena.nfreepages = 0 # they are all uninitialized pages
arena.totalpages = npages
@@ -332,7 +337,7 @@
#
# The whole arena is empty. Free it.
llarena.arena_free(arena.base)
- lltype.free(arena, flavor='raw')
+ lltype.free(arena, flavor='raw', track_allocation=False)
#
else:
# Insert 'arena' in the correct arenas_lists[n]
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/semispace.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gc/semispace.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gc/semispace.py Tue Nov 23 10:12:47 2010
@@ -1,7 +1,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free
from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack, get_address_deque
from pypy.rpython.memory.support import AddressDict
from pypy.rpython.lltypesystem import lltype, llmemory, llarena, rffi, llgroup
@@ -59,11 +58,11 @@
# translating to a real backend.
TRANSLATION_PARAMS = {'space_size': 8*1024*1024} # XXX adjust
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096,
- max_space_size=sys.maxint//2+1):
+ def __init__(self, config, space_size=4096, max_space_size=sys.maxint//2+1,
+ **kwds):
self.param_space_size = space_size
self.param_max_space_size = max_space_size
- MovingGCBase.__init__(self, config, chunk_size)
+ MovingGCBase.__init__(self, config, **kwds)
def setup(self):
#self.total_collection_time = 0.0
@@ -231,6 +230,10 @@
while self.max_space_size > size:
self.max_space_size >>= 1
+ @classmethod
+ def JIT_minimal_size_in_nursery(cls):
+ return cls.object_minimal_size
+
def collect(self, gen=0):
self.debug_check_consistency()
self.semispace_collect()
@@ -690,15 +693,10 @@
self._ll_typeid_map[idx].size += llmemory.raw_malloc_usage(totsize)
self.trace(adr, self.track_heap_parent, adr)
- def _track_heap_root(self, root):
- self.track_heap(root.address[0])
+ @staticmethod
+ def _track_heap_root(obj, self):
+ self.track_heap(obj)
- def heap_stats_walk_roots(self):
- self.root_walker.walk_roots(
- SemiSpaceGC._track_heap_root,
- SemiSpaceGC._track_heap_root,
- SemiSpaceGC._track_heap_root)
-
def heap_stats(self):
self._tracked_dict = self.AddressDict()
max_tid = self.root_walker.gcdata.max_type_id
@@ -711,7 +709,7 @@
while i < max_tid:
self._tracked_dict.add(llmemory.cast_ptr_to_adr(ll_typeid_map[i]))
i += 1
- self.heap_stats_walk_roots()
+ self.enumerate_all_roots(SemiSpaceGC._track_heap_root, self)
self._ll_typeid_map = lltype.nullptr(ARRAY_TYPEID_MAP)
self._tracked_dict.delete()
return ll_typeid_map
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_direct.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_direct.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_direct.py Tue Nov 23 10:12:47 2010
@@ -70,6 +70,7 @@
GC_PARAMS = self.GC_PARAMS.copy()
if hasattr(meth, 'GC_PARAMS'):
GC_PARAMS.update(meth.GC_PARAMS)
+ GC_PARAMS['translated_to_c'] = False
self.gc = self.GCClass(config, **GC_PARAMS)
self.gc.DEBUG = True
self.rootwalker = DirectRootWalker(self)
@@ -86,17 +87,19 @@
def write(self, p, fieldname, newvalue):
if self.gc.needs_write_barrier:
+ newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
- self.gc.write_barrier(addr_struct)
+ self.gc.write_barrier(newaddr, addr_struct)
setattr(p, fieldname, newvalue)
def writearray(self, p, index, newvalue):
if self.gc.needs_write_barrier:
+ newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
if hasattr(self.gc, 'write_barrier_from_array'):
- self.gc.write_barrier_from_array(addr_struct, index)
+ self.gc.write_barrier_from_array(newaddr, addr_struct, index)
else:
- self.gc.write_barrier(addr_struct)
+ self.gc.write_barrier(newaddr, addr_struct)
p[index] = newvalue
def malloc(self, TYPE, n=None):
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_minimark.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_minimark.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_minimark.py Tue Nov 23 10:12:47 2010
@@ -28,3 +28,42 @@
assert gc.card_marking_bytes_for_length(P+P+1) == 3
assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P) == 8
assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P+1) == 9
+
+def test_set_major_threshold():
+ gc = MiniMarkGC(None, major_collection_threshold=2.0,
+ growth_rate_max=1.5)
+ gc.min_heap_size = 100.0
+ gc.max_heap_size = 300.0
+ gc.next_major_collection_threshold = 0.0
+ # first, we don't grow past min_heap_size
+ for i in range(5):
+ gc.set_major_threshold_from(100.0)
+ assert gc.next_major_collection_threshold == 100.0
+ # then we grow a lot
+ b = gc.set_major_threshold_from(100 * 2.0)
+ assert b is False
+ assert gc.next_major_collection_threshold == 150.0
+ b = gc.set_major_threshold_from(150 * 2.0)
+ assert b is False
+ assert gc.next_major_collection_threshold == 225.0
+ b = gc.set_major_threshold_from(225 * 2.0)
+ assert b is True
+ assert gc.next_major_collection_threshold == 300.0 # max reached
+ b = gc.set_major_threshold_from(300 * 2.0)
+ assert b is True
+ assert gc.next_major_collection_threshold == 300.0
+ # then we shrink instantly
+ b = gc.set_major_threshold_from(100.0)
+ assert b is False
+ assert gc.next_major_collection_threshold == 100.0
+ # then we grow a bit
+ b = gc.set_major_threshold_from(100 * 1.25)
+ assert b is False
+ assert gc.next_major_collection_threshold == 125.0
+ b = gc.set_major_threshold_from(125 * 1.25)
+ assert b is False
+ assert gc.next_major_collection_threshold == 156.25
+ # check that we cannot shrink below min_heap_size
+ b = gc.set_major_threshold_from(42.7)
+ assert b is False
+ assert gc.next_major_collection_threshold == 100.0
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gctransform/asmgcroot.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gctransform/asmgcroot.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gctransform/asmgcroot.py Tue Nov 23 10:12:47 2010
@@ -75,7 +75,8 @@
key = (TYPE, num)
if key not in sradict:
CONTAINER = lltype.FixedSizeArray(TYPE, 1)
- p = lltype.malloc(CONTAINER, flavor='raw', zero=True)
+ p = lltype.malloc(CONTAINER, flavor='raw', zero=True,
+ immortal=True)
sradict[key] = Constant(p, lltype.Ptr(CONTAINER))
sra.append(sradict[key])
#
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gctransform/framework.py Tue Nov 23 10:12:47 2010
@@ -172,6 +172,7 @@
gcdata.static_root_nongcend = a_random_address # patched in finish()
gcdata.static_root_end = a_random_address # patched in finish()
gcdata.max_type_id = 13 # patched in finish()
+ gcdata.typeids_z = a_random_address # patched in finish()
self.gcdata = gcdata
self.malloc_fnptr_cache = {}
@@ -212,6 +213,9 @@
data_classdef.generalize_attr(
'max_type_id',
annmodel.SomeInteger())
+ data_classdef.generalize_attr(
+ 'typeids_z',
+ annmodel.SomeAddress())
annhelper = annlowlevel.MixLevelHelperAnnotator(self.translator.rtyper)
@@ -415,6 +419,11 @@
[s_gc, annmodel.SomeInteger()],
annmodel.s_Bool,
minimal_transform=False)
+ self.get_typeids_z_ptr = getfn(inspector.get_typeids_z,
+ [s_gc],
+ annmodel.SomePtr(
+ lltype.Ptr(rgc.ARRAY_OF_CHAR)),
+ minimal_transform=False)
self.set_max_heap_size_ptr = getfn(GCClass.set_max_heap_size.im_func,
[s_gc,
@@ -426,6 +435,7 @@
if GCClass.needs_write_barrier:
self.write_barrier_ptr = getfn(GCClass.write_barrier.im_func,
[s_gc,
+ annmodel.SomeAddress(),
annmodel.SomeAddress()],
annmodel.s_None,
inline=True)
@@ -434,13 +444,15 @@
# func should not be a bound method, but a real function
assert isinstance(func, types.FunctionType)
self.write_barrier_failing_case_ptr = getfn(func,
- [annmodel.SomeAddress()],
+ [annmodel.SomeAddress(),
+ annmodel.SomeAddress()],
annmodel.s_None)
func = getattr(GCClass, 'write_barrier_from_array', None)
if func is not None:
self.write_barrier_from_array_ptr = getfn(func.im_func,
[s_gc,
annmodel.SomeAddress(),
+ annmodel.SomeAddress(),
annmodel.SomeInteger()],
annmodel.s_None,
inline=True)
@@ -569,7 +581,14 @@
newgcdependencies = []
newgcdependencies.append(ll_static_roots_inside)
ll_instance.inst_max_type_id = len(group.members)
- self.write_typeid_list()
+ typeids_z = self.write_typeid_list()
+ ll_typeids_z = lltype.malloc(rgc.ARRAY_OF_CHAR,
+ len(typeids_z),
+ immortal=True)
+ for i in range(len(typeids_z)):
+ ll_typeids_z[i] = typeids_z[i]
+ ll_instance.inst_typeids_z = llmemory.cast_ptr_to_adr(ll_typeids_z)
+ newgcdependencies.append(ll_typeids_z)
return newgcdependencies
def get_finish_tables(self):
@@ -596,6 +615,11 @@
for index in range(len(self.layoutbuilder.type_info_group.members)):
f.write("member%-4d %s\n" % (index, all_ids.get(index, '?')))
f.close()
+ try:
+ import zlib
+ return zlib.compress(udir.join("typeids.txt").read(), 9)
+ except ImportError:
+ return ''
def transform_graph(self, graph):
func = getattr(graph, 'func', None)
@@ -985,6 +1009,13 @@
resultvar=hop.spaceop.result)
self.pop_roots(hop, livevars)
+ def gct_gc_typeids_z(self, hop):
+ livevars = self.push_roots(hop)
+ hop.genop("direct_call",
+ [self.get_typeids_z_ptr, self.c_const_gc],
+ resultvar=hop.spaceop.result)
+ self.pop_roots(hop, livevars)
+
def gct_malloc_nonmovable_varsize(self, hop):
TYPE = hop.spaceop.result.concretetype
if self.gcdata.gc.can_malloc_nonmovable():
@@ -1021,6 +1052,8 @@
and not isinstance(v_newvalue, Constant)
and v_struct.concretetype.TO._gckind == "gc"
and hop.spaceop not in self.clean_sets):
+ v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
+ resulttype = llmemory.Address)
v_structaddr = hop.genop("cast_ptr_to_adr", [v_struct],
resulttype = llmemory.Address)
if (self.write_barrier_from_array_ptr is not None and
@@ -1030,12 +1063,14 @@
assert v_index.concretetype == lltype.Signed
hop.genop("direct_call", [self.write_barrier_from_array_ptr,
self.c_const_gc,
+ v_newvalue,
v_structaddr,
v_index])
else:
self.write_barrier_calls += 1
hop.genop("direct_call", [self.write_barrier_ptr,
self.c_const_gc,
+ v_newvalue,
v_structaddr])
hop.rename('bare_' + opname)
@@ -1203,7 +1238,7 @@
sizeofaddr = llmemory.sizeof(llmemory.Address)
-class BaseRootWalker:
+class BaseRootWalker(object):
need_root_stack = False
def __init__(self, gctransformer):
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gctransform/transform.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gctransform/transform.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gctransform/transform.py Tue Nov 23 10:12:47 2010
@@ -430,7 +430,8 @@
return self.parenttransformer.gct_malloc_varsize(hop)
def gct_free(self, hop):
- flavor = hop.spaceop.args[1].value
+ flags = hop.spaceop.args[1].value
+ flavor = flags['flavor']
assert flavor == 'raw'
return self.parenttransformer.gct_free(hop)
@@ -532,6 +533,8 @@
resulttype=llmemory.Address)
if flags.get('zero'):
hop.genop("raw_memclear", [v_raw, c_size])
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_fv_stack_malloc(self, hop, flags, TYPE, c_size):
@@ -602,15 +605,20 @@
[self.raw_malloc_varsize_ptr, v_length,
c_const_size, c_item_size, c_offset_to_length],
resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_free(self, hop):
op = hop.spaceop
- flavor = op.args[1].value
+ flags = op.args[1].value
+ flavor = flags['flavor']
v = op.args[0]
assert flavor != 'cpy', "cannot free CPython objects directly"
if flavor == 'raw':
v = hop.genop("cast_ptr_to_adr", [v], resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_stop", [v])
hop.genop('raw_free', [v])
else:
assert False, "%s has no support for free with flavor %r" % (self, flavor)
Modified: pypy/branch/arm-backend/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/gcwrapper.py Tue Nov 23 10:12:47 2010
@@ -9,7 +9,10 @@
def __init__(self, llinterp, flowgraphs, gc_class, GC_PARAMS={}):
translator = llinterp.typer.annotator.translator
config = translator.config.translation
- self.gc = gc_class(config, chunk_size = 10, **GC_PARAMS)
+ self.gc = gc_class(config,
+ chunk_size = 10,
+ translated_to_c = False,
+ **GC_PARAMS)
self.gc.set_root_walker(LLInterpRootWalker(self))
self.gc.DEBUG = True
self.llinterp = llinterp
@@ -39,7 +42,8 @@
#
# Interface for the llinterp
#
- def malloc(self, TYPE, n=None, flavor='gc', zero=False):
+ def malloc(self, TYPE, n=None, flavor='gc', zero=False,
+ track_allocation=True):
if flavor == 'gc':
typeid = self.get_type_id(TYPE)
addr = self.gc.malloc(typeid, n, zero=zero)
@@ -48,7 +52,8 @@
gctypelayout.zero_gc_pointers(result)
return result
else:
- return lltype.malloc(TYPE, n, flavor=flavor, zero=zero)
+ return lltype.malloc(TYPE, n, flavor=flavor, zero=zero,
+ track_allocation=track_allocation)
def malloc_nonmovable(self, TYPE, n=None, zero=False):
typeid = self.get_type_id(TYPE)
@@ -66,9 +71,10 @@
return self.gc.shrink_array(addr, smallersize)
return False
- def free(self, TYPE, flavor='gc'):
+ def free(self, TYPE, flavor='gc', track_allocation=True):
assert flavor != 'gc'
- return lltype.free(TYPE, flavor=flavor)
+ return lltype.free(TYPE, flavor=flavor,
+ track_allocation=track_allocation)
def setfield(self, obj, fieldname, fieldvalue):
STRUCT = lltype.typeOf(obj).TO
@@ -94,6 +100,7 @@
assert (type(index) is int # <- fast path
or lltype.typeOf(index) == lltype.Signed)
self.gc.write_barrier_from_array(
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer),
index)
wb = False
@@ -101,6 +108,7 @@
#
if wb:
self.gc.write_barrier(
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer))
llheap.setinterior(toplevelcontainer, inneraddr, INNERTYPE, newvalue)
Modified: pypy/branch/arm-backend/pypy/rpython/memory/support.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/support.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/support.py Tue Nov 23 10:12:47 2010
@@ -30,7 +30,8 @@
# we zero-initialize the chunks to make the translation
# backends happy, but we don't need to do it at run-time.
zero = not we_are_translated()
- return lltype.malloc(CHUNK, flavor="raw", zero=zero)
+ return lltype.malloc(CHUNK, flavor="raw", zero=zero,
+ track_allocation=False)
result = self.free_list
self.free_list = result.next
@@ -44,7 +45,7 @@
# Don't cache the old chunks but free them immediately.
# Helps debugging, and avoids that old chunks full of
# addresses left behind by a test end up in genc...
- lltype.free(chunk, flavor="raw")
+ lltype.free(chunk, flavor="raw", track_allocation=False)
unused_chunks = FreeList()
cache[chunk_size] = unused_chunks, null_chunk
@@ -111,7 +112,7 @@
cur = next
free_non_gc_object(self)
- def length(self):
+ def _length_estimate(self):
chunk = self.chunk
count = self.used_in_last_chunk
while chunk:
@@ -134,7 +135,7 @@
foreach._annspecialcase_ = 'specialize:arg(1)'
def stack2dict(self):
- result = AddressDict(self.length())
+ result = AddressDict(self._length_estimate())
self.foreach(_add_in_dict, result)
return result
Modified: pypy/branch/arm-backend/pypy/rpython/memory/test/test_gc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/test/test_gc.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/test/test_gc.py Tue Nov 23 10:12:47 2010
@@ -278,6 +278,105 @@
res = self.interpret(f, [])
assert res
+ def test_bug_1(self):
+ import weakref
+ class B(object):
+ pass
+ def g():
+ b = B()
+ llop.gc__collect(lltype.Void) # force 'b' to be old
+ ref = weakref.ref(B())
+ b.ref = ref
+ return ref
+ def f():
+ ref = g()
+ llop.gc__collect(lltype.Void)
+ llop.gc__collect(lltype.Void)
+ result = (ref() is None)
+ return result
+ res = self.interpret(f, [])
+ assert res
+
+ def test_cycle_with_weakref_and_del(self):
+ import weakref, gc
+ class A(object):
+ count = 0
+ a = A()
+ class B(object):
+ def __del__(self):
+ # when __del__ is called, the weakref to c should be dead
+ if self.ref() is None:
+ a.count += 10 # ok
+ else:
+ a.count = 666 # not ok
+ class C(object):
+ pass
+ def g():
+ c = C()
+ c.b = B()
+ ref = weakref.ref(c)
+ c.b.ref = ref
+ return ref
+ def f():
+ ref = g()
+ llop.gc__collect(lltype.Void)
+ llop.gc__collect(lltype.Void)
+ result = a.count + (ref() is None)
+ return result
+ res = self.interpret(f, [])
+ assert res == 11
+
+ def test_weakref_to_object_with_finalizer_ordering(self):
+ import weakref, gc
+ class A(object):
+ count = 0
+ a = A()
+ class B(object):
+ def __del__(self):
+ # when __del__ is called, the weakref to myself is still valid
+ # in RPython (at least with most GCs; this test might be
+ # skipped for specific GCs)
+ if self.ref() is self:
+ a.count += 10 # ok
+ else:
+ a.count = 666 # not ok
+ def g():
+ b = B()
+ ref = weakref.ref(b)
+ b.ref = ref
+ return ref
+ def f():
+ ref = g()
+ llop.gc__collect(lltype.Void)
+ llop.gc__collect(lltype.Void)
+ result = a.count + (ref() is None)
+ return result
+ res = self.interpret(f, [])
+ assert res == 11
+
+ def test_weakref_bug_1(self):
+ import weakref
+ class A(object):
+ pass
+ class B(object):
+ def __del__(self):
+ self.wref().x += 1
+ def g(a):
+ b = B()
+ b.wref = weakref.ref(a)
+ # the only way to reach this weakref is via B, which is an
+ # object with finalizer (but the weakref itself points to
+ # a, which does not go away but will move during the next
+ # gc.collect)
+ def f():
+ a = A()
+ a.x = 10
+ g(a)
+ llop.gc__collect(lltype.Void)
+ return a.x
+ res = self.interpret(f, [])
+ assert res == 11
+
def test_id(self):
class A(object):
pass
@@ -368,6 +467,14 @@
res = self.interpret(f, [4, 42])
assert res == 12
+ def test_print_leak(self):
+ def f(n):
+ for i in range(n):
+ print i
+ return 42
+ res = self.interpret(f, [10])
+ assert res == 42
+
def test_weakref_across_minor_collection(self):
import weakref
class A:
@@ -627,6 +734,9 @@
class TestMarkSweepGC(GCTest):
from pypy.rpython.memory.gc.marksweep import MarkSweepGC as GCClass
+ def test_weakref_to_object_with_finalizer_ordering(self):
+ py.test.skip("Does not work")
+
class TestSemiSpaceGC(GCTest, snippet.SemiSpaceGCTests):
from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass
GC_CAN_MOVE = True
Modified: pypy/branch/arm-backend/pypy/rpython/memory/test/test_transformed_gc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/memory/test/test_transformed_gc.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/memory/test/test_transformed_gc.py Tue Nov 23 10:12:47 2010
@@ -906,7 +906,8 @@
gcname = "marksweep"
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
@@ -1144,7 +1145,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.marksweep import PrintingMarkSweepGC as GCClass
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestSemiSpaceGC(GenericMovingGCTests):
@@ -1154,7 +1156,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass
- GC_PARAMS = {'space_size': 512*WORD}
+ GC_PARAMS = {'space_size': 512*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestMarkCompactGC(GenericMovingGCTests):
@@ -1163,7 +1166,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass
- GC_PARAMS = {'space_size': 4096*WORD}
+ GC_PARAMS = {'space_size': 4096*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestGenerationGC(GenericMovingGCTests):
@@ -1175,7 +1179,8 @@
from pypy.rpython.memory.gc.generation import GenerationGC as \
GCClass
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 32*WORD}
+ 'nursery_size': 32*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_weakref_across_minor_collection(cls):
@@ -1372,7 +1377,8 @@
GenerationGC._teardown(self)
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 128*WORD}
+ 'nursery_size': 128*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_working_nursery(cls):
@@ -1404,7 +1410,8 @@
from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass
GC_PARAMS = {'space_size': 512*WORD,
'nursery_size': 32*WORD,
- 'large_object': 8*WORD}
+ 'large_object': 8*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_ref_from_rawmalloced_to_regular(cls):
@@ -1477,6 +1484,7 @@
'large_object': 8*WORD,
'large_object_gcptrs': 10*WORD,
'card_page_indices': 4,
+ 'translated_to_c': False,
}
root_stack_depth = 200
@@ -1585,7 +1593,8 @@
gcname = "marksweep"
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestHybridTaggedPointerGC(TaggedPointerGCTests):
@@ -1596,7 +1605,8 @@
from pypy.rpython.memory.gc.generation import GenerationGC as \
GCClass
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 32*WORD}
+ 'nursery_size': 32*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestMarkCompactTaggedpointerGC(TaggedPointerGCTests):
@@ -1605,5 +1615,6 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass
- GC_PARAMS = {'space_size': 4096*WORD}
+ GC_PARAMS = {'space_size': 4096*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
Modified: pypy/branch/arm-backend/pypy/rpython/module/ll_time.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/module/ll_time.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/module/ll_time.py Tue Nov 23 10:12:47 2010
@@ -108,7 +108,7 @@
errcode = -1
if self.GETTIMEOFDAY_NO_TZ:
- errcode = g_gettimeofday(t)
+ errcode = c_gettimeofday(t)
else:
errcode = c_gettimeofday(t, void)
Modified: pypy/branch/arm-backend/pypy/rpython/rbuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/rbuilder.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/rbuilder.py Tue Nov 23 10:12:47 2010
@@ -36,8 +36,12 @@
hop.exception_cannot_occur()
return hop.gendirectcall(self.ll_append_multiple_char, *vlist)
+ def rtype_method_getlength(self, hop):
+ vlist = hop.inputargs(self)
+ hop.exception_cannot_occur()
+ return hop.gendirectcall(self.ll_getlength, *vlist)
+
def rtype_method_build(self, hop):
vlist = hop.inputargs(self)
hop.exception_cannot_occur()
return hop.gendirectcall(self.ll_build, *vlist)
-
Modified: pypy/branch/arm-backend/pypy/rpython/rbuiltin.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/rbuiltin.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/rbuiltin.py Tue Nov 23 10:12:47 2010
@@ -345,17 +345,22 @@
BUILTIN_TYPER[object.__init__] = rtype_object__init__
# annotation of low-level types
-def rtype_malloc(hop, i_flavor=None, i_zero=None):
+def rtype_malloc(hop, i_flavor=None, i_zero=None, i_track_allocation=None):
assert hop.args_s[0].is_constant()
vlist = [hop.inputarg(lltype.Void, arg=0)]
opname = 'malloc'
- v_flavor, v_zero = parse_kwds(hop, (i_flavor, lltype.Void), (i_zero, None))
+ v_flavor, v_zero, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_zero, None),
+ (i_track_allocation, None))
flags = {'flavor': 'gc'}
if v_flavor is not None:
flags['flavor'] = v_flavor.value
if i_zero is not None:
flags['zero'] = v_zero.value
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
vlist.append(hop.inputconst(lltype.Void, flags))
if hop.nb_args == 2:
@@ -366,12 +371,29 @@
hop.exception_is_here()
return hop.genop(opname, vlist, resulttype = hop.r_result.lowleveltype)
-def rtype_free(hop, i_flavor):
- assert i_flavor == 1
+def rtype_free(hop, i_flavor, i_track_allocation=None):
+ vlist = [hop.inputarg(hop.args_r[0], arg=0)]
+ v_flavor, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_track_allocation, None))
+ #
+ assert v_flavor is not None and v_flavor.value == 'raw'
+ flags = {'flavor': 'raw'}
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
+ vlist.append(hop.inputconst(lltype.Void, flags))
+ #
hop.exception_cannot_occur()
- vlist = hop.inputargs(hop.args_r[0], lltype.Void)
hop.genop('free', vlist)
+def rtype_render_immortal(hop, i_track_allocation=None):
+ vlist = [hop.inputarg(hop.args_r[0], arg=0)]
+ v_track_allocation = parse_kwds(hop,
+ (i_track_allocation, None))
+ hop.exception_cannot_occur()
+ if i_track_allocation is None or v_track_allocation.value:
+ hop.genop('track_alloc_stop', vlist)
+
def rtype_const_result(hop):
hop.exception_cannot_occur()
return hop.inputconst(hop.r_result.lowleveltype, hop.s_result.const)
@@ -509,6 +531,7 @@
BUILTIN_TYPER[lltype.malloc] = rtype_malloc
BUILTIN_TYPER[lltype.free] = rtype_free
+BUILTIN_TYPER[lltype.render_immortal] = rtype_render_immortal
BUILTIN_TYPER[lltype.cast_primitive] = rtype_cast_primitive
BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer
BUILTIN_TYPER[lltype.cast_opaque_ptr] = rtype_cast_opaque_ptr
@@ -584,8 +607,9 @@
vinst, = hop.inputargs(hop.args_r[0])
flavor = hop.args_r[0].gcflavor
assert flavor != 'gc'
- cflavor = hop.inputconst(lltype.Void, flavor)
- return hop.genop('free', [vinst, cflavor])
+ flags = {'flavor': flavor}
+ cflags = hop.inputconst(lltype.Void, flags)
+ return hop.genop('free', [vinst, cflags])
BUILTIN_TYPER[objectmodel.free_non_gc_object] = rtype_free_non_gc_object
Modified: pypy/branch/arm-backend/pypy/rpython/rmodel.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/rmodel.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/rmodel.py Tue Nov 23 10:12:47 2010
@@ -11,14 +11,14 @@
# initialization states for Repr instances
-class setupstate:
+class setupstate(object):
NOTINITIALIZED = 0
INPROGRESS = 1
BROKEN = 2
FINISHED = 3
DELAYED = 4
-class Repr:
+class Repr(object):
""" An instance of Repr is associated with each instance of SomeXxx.
It defines the chosen representation for the SomeXxx. The Repr subclasses
generally follows the SomeXxx subclass hierarchy, but there are numerous
Modified: pypy/branch/arm-backend/pypy/rpython/rpbc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/rpbc.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/rpbc.py Tue Nov 23 10:12:47 2010
@@ -256,6 +256,8 @@
def convert_const(self, value):
if isinstance(value, types.MethodType) and value.im_self is None:
value = value.im_func # unbound method -> bare function
+ elif isinstance(value, staticmethod):
+ value = value.__get__(42) # hackish, get the function wrapped by staticmethod
if self.lowleveltype is Void:
return None
if value is None:
Modified: pypy/branch/arm-backend/pypy/rpython/rtyper.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/rtyper.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/rtyper.py Tue Nov 23 10:12:47 2010
@@ -421,7 +421,7 @@
assert noexclink.exitcase is None
if pos == "removed":
# the exception cannot actually occur at all.
- # See for example rspecialcase.rtype_call_specialcase().
+ # This is set by calling exception_cannot_occur().
# We just remove all exception links.
block.exitswitch = None
block.exits = block.exits[:1]
@@ -1019,7 +1019,7 @@
from pypy.rpython import rint, rbool, rfloat
from pypy.rpython import rrange
from pypy.rpython import rstr, rdict, rlist
-from pypy.rpython import rclass, rbuiltin, rpbc, rspecialcase
+from pypy.rpython import rclass, rbuiltin, rpbc
from pypy.rpython import rexternalobj
from pypy.rpython import rptr
from pypy.rpython import rgeneric
Modified: pypy/branch/arm-backend/pypy/rpython/test/test_llinterp.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/test/test_llinterp.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/test/test_llinterp.py Tue Nov 23 10:12:47 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import py
import sys
from pypy.rpython.lltypesystem.lltype import typeOf, pyobjectptr, Ptr,\
@@ -5,6 +6,7 @@
from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rpython.llinterp import LLInterpreter, LLException, log
from pypy.rpython.rmodel import inputconst
+from pypy.rpython.annlowlevel import hlstr
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.rint import signed_repr
from pypy.rpython.lltypesystem import rstr, lltype
@@ -12,13 +14,11 @@
from pypy.annotation.model import lltype_to_annotation
from pypy.rlib.rarithmetic import r_uint, ovfcheck
from pypy.rpython.ootypesystem import ootype
+from pypy.tool import leakfinder
from pypy import conftest
# switch on logging of interp to show more info on failing tests
-class MallocMismatch(Exception):
- pass
-
def setup_module(mod):
mod.logstate = py.log._getstate()
py.log.setconsumer("llinterp", py.log.STDOUT)
@@ -72,7 +72,7 @@
def get_interpreter(func, values, view='auto', viewbefore='auto', policy=None,
someobjects=False, type_system="lltype", backendopt=False,
- config=None, malloc_check=True, **extraconfigopts):
+ config=None, **extraconfigopts):
extra_key = [(key, value) for key, value in extraconfigopts.iteritems()]
extra_key.sort()
extra_key = tuple(extra_key)
@@ -97,7 +97,7 @@
viewbefore, policy, type_system=type_system,
backendopt=backendopt, config=config,
**extraconfigopts)
- interp = LLInterpreter(typer, malloc_check=malloc_check)
+ interp = LLInterpreter(typer)
_tcache[key] = (t, interp, graph)
# keep the cache small
_lastinterpreted.append(key)
@@ -115,10 +115,17 @@
interp, graph = get_interpreter(func, values, view, viewbefore, policy,
someobjects, type_system=type_system,
backendopt=backendopt, config=config,
- malloc_check=malloc_check, **kwargs)
- result = interp.eval_graph(graph, values)
- if malloc_check and interp.mallocs:
- raise MallocMismatch(interp.mallocs)
+ **kwargs)
+ if not malloc_check:
+ result = interp.eval_graph(graph, values)
+ else:
+ prev = leakfinder.start_tracking_allocations()
+ try:
+ result = interp.eval_graph(graph, values)
+ finally:
+ leaks = leakfinder.stop_tracking_allocations(False, prev)
+ if leaks:
+ raise leakfinder.MallocMismatch(leaks)
return result
def interpret_raises(exc, func, values, view='auto', viewbefore='auto',
@@ -418,6 +425,7 @@
assert result
def test_stack_malloc():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
def f():
@@ -430,6 +438,7 @@
assert result == 1
def test_invalid_stack_access():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
globala = A()
@@ -605,7 +614,7 @@
if x:
free(t, flavor='raw')
interpret(f, [1])
- py.test.raises(MallocMismatch, "interpret(f, [0])")
+ py.test.raises(leakfinder.MallocMismatch, "interpret(f, [0])")
def f():
t1 = malloc(T, flavor='raw')
@@ -615,3 +624,37 @@
interpret(f, [])
+def test_context_manager():
+ state = []
+ class C:
+ def __enter__(self):
+ state.append('acquire')
+ return self
+ def __exit__(self, *args):
+ if args[1] is not None:
+ state.append('raised')
+ state.append('release')
+ def f():
+ try:
+ with C() as c:
+ state.append('use')
+ raise ValueError
+ except ValueError:
+ pass
+ return ', '.join(state)
+ res = interpret(f, [])
+ assert hlstr(res) == 'acquire, use, raised, release'
+
+
+def test_scoped_allocator():
+ from pypy.rpython.lltypesystem.lltype import scoped_alloc, Array, Signed
+ T = Array(Signed)
+
+ def f():
+ x = 0
+ with scoped_alloc(T, 1) as array:
+ array[0] = -42
+ x = array[0]
+ assert x == -42
+
+ res = interpret(f, [])
Modified: pypy/branch/arm-backend/pypy/rpython/test/test_nongc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/test/test_nongc.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/test/test_nongc.py Tue Nov 23 10:12:47 2010
@@ -79,7 +79,7 @@
py.test.raises(TypeError,rtyper.specialize) # results in an invalid cast
def test_isinstance():
- class A:
+ class A(object):
_alloc_flavor_ = "raw"
class B(A):
pass
@@ -95,7 +95,24 @@
o = B()
else:
o = C()
- return 100*isinstance(o, A)+10*isinstance(o, B)+1*isinstance(o ,C)
+ res = 100*isinstance(o, A) + 10*isinstance(o, B) + 1*isinstance(o, C)
+ if i == 0:
+ pass
+ elif i == 1:
+ assert isinstance(o, A)
+ free_non_gc_object(o)
+ elif i == 2:
+ assert isinstance(o, B)
+ free_non_gc_object(o)
+ else:
+ assert isinstance(o, C)
+ free_non_gc_object(o)
+ return res
+
+ assert f(1) == 100
+ assert f(2) == 110
+ assert f(3) == 111
+ assert f(0) == 0
a = RPythonAnnotator()
#does not raise:
@@ -131,10 +148,14 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ free_non_gc_object(c)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
@@ -169,10 +190,13 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
Modified: pypy/branch/arm-backend/pypy/rpython/test/test_rbuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/test/test_rbuilder.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/test/test_rbuilder.py Tue Nov 23 10:12:47 2010
@@ -1,4 +1,4 @@
-
+import py
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem.rbuilder import *
from pypy.rpython.annlowlevel import llstr, hlstr
@@ -55,8 +55,29 @@
assert res == 'aabcabcdefbuuuu'
assert isinstance(res, unicode)
+ def test_string_getlength(self):
+ def func():
+ s = StringBuilder()
+ s.append("a")
+ s.append("abc")
+ return s.getlength()
+ res = self.interpret(func, [])
+ assert res == 4
+
+ def test_unicode_getlength(self):
+ def func():
+ s = UnicodeBuilder()
+ s.append(u"a")
+ s.append(u"abc")
+ return s.getlength()
+ res = self.interpret(func, [])
+ assert res == 4
+
class TestLLtype(BaseTestStringBuilder, LLRtypeMixin):
pass
class TestOOtype(BaseTestStringBuilder, OORtypeMixin):
- pass
+ def test_string_getlength(self):
+ py.test.skip("getlength(): not implemented on ootype")
+ def test_unicode_getlength(self):
+ py.test.skip("getlength(): not implemented on ootype")
Modified: pypy/branch/arm-backend/pypy/rpython/test/test_rclass.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/test/test_rclass.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/test/test_rclass.py Tue Nov 23 10:12:47 2010
@@ -3,7 +3,7 @@
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.ootypesystem import ootype
-from pypy.rlib.rarithmetic import intmask
+from pypy.rlib.rarithmetic import intmask, r_longlong
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.objspace.flow.model import summary
@@ -319,6 +319,17 @@
res = self.interpret(f, [])
assert res == 42
+ def test_staticmethod2(self):
+ class A(object):
+ f = staticmethod(lambda x, y: x*y)
+ class B(A):
+ f = staticmethod(lambda x, y: x+y)
+ def f():
+ b = B()
+ return b.f(6, 7)
+ res = self.interpret(f, [])
+ assert res == 13
+
def test_is(self):
class A: pass
class B(A): pass
@@ -1001,6 +1012,40 @@
res = self.interpret(f, [5])
assert res == 0
+ def test_order_of_fields(self):
+ class A(object):
+ pass
+ def f(n):
+ a = A()
+ a.as_int = n
+ a.as_char = chr(n)
+ a.as_unichar = unichr(n)
+ a.as_double = n + 0.5
+ a.as_bool = bool(n)
+ a.as_void = None
+ a.as_longlong = r_longlong(n)
+ a.as_reference = A()
+ return a
+
+ res = self.interpret(f, [5])
+ names = list(typeOf(res).TO._names)
+ i = names.index('inst_as_int')
+ c = names.index('inst_as_char')
+ u = names.index('inst_as_unichar')
+ d = names.index('inst_as_double')
+ b = names.index('inst_as_bool')
+ v = names.index('inst_as_void')
+ l = names.index('inst_as_longlong')
+ r = names.index('inst_as_reference')
+ assert v == 1 # void fields are first
+ assert sorted([c, b]) == [7, 8]
+ if sys.maxint == 2147483647:
+ assert sorted([u, i, r]) == [4, 5, 6] # 32-bit types
+ assert sorted([d, l]) == [2, 3] # 64-bit types
+ else:
+ assert sorted([u]) == [6] # 32-bit types
+ assert sorted([i, r, d, l]) == [2, 3, 4, 5] # 64-bit types
+
class TestOOtype(BaseTestRclass, OORtypeMixin):
Modified: pypy/branch/arm-backend/pypy/rpython/test/test_rdict.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/test/test_rdict.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/test/test_rdict.py Tue Nov 23 10:12:47 2010
@@ -682,6 +682,26 @@
# if it does not crash, we are fine. It crashes if you forget the hash field.
self.interpret(func, [])
+ def test_dict_popitem(self):
+ def func():
+ d = {}
+ d[5] = 2
+ d[6] = 3
+ k1, v1 = d.popitem()
+ assert len(d) == 1
+ k2, v2 = d.popitem()
+ try:
+ d.popitem()
+ except KeyError:
+ pass
+ else:
+ assert 0, "should have raised KeyError"
+ assert len(d) == 0
+ return k1*1000 + v1*100 + k2*10 + v2
+
+ res = self.interpret(func, [])
+ assert res in [5263, 6352]
+
# ____________________________________________________________
def test_opt_nullkeymarker(self):
Modified: pypy/branch/arm-backend/pypy/rpython/test/test_rpbc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/test/test_rpbc.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/test/test_rpbc.py Tue Nov 23 10:12:47 2010
@@ -1547,7 +1547,7 @@
def test_always_raising_methods(self):
class Base:
def m(self):
- raise NotImplementedError
+ raise KeyError
class A(Base):
def m(self):
return 42
@@ -1560,11 +1560,11 @@
o = B()
try:
o.m()
- except NotImplementedError:
- pass
+ except KeyError:
+ assert 0
return B().m()
- self.interpret_raises(NotImplementedError, f, [7])
+ self.interpret_raises(KeyError, f, [7])
def test_possible_missing_attribute_access(self):
py.test.skip("Should explode or give some warning")
Modified: pypy/branch/arm-backend/pypy/rpython/test/test_rptr.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/test/test_rptr.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/test/test_rptr.py Tue Nov 23 10:12:47 2010
@@ -212,10 +212,31 @@
S = Struct('S', ('x', Signed))
def fn(n):
- p = malloc(S, flavor='whatever')
+ p = malloc(S, flavor='raw')
p.x = n
result = p.x
- free(p, flavor='whatever')
+ free(p, flavor='raw')
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ free(p, flavor='raw', track_allocation=False)
return n
res = interpret(fn, [23])
Modified: pypy/branch/arm-backend/pypy/tool/alarm.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/alarm.py (original)
+++ pypy/branch/arm-backend/pypy/tool/alarm.py Tue Nov 23 10:12:47 2010
@@ -38,8 +38,9 @@
sys.path.insert(0, os.path.dirname(sys.argv[0]))
return sys.argv[0]
-finished = []
-try:
- execfile(_main_with_alarm(finished))
-finally:
- finished.append(True)
+if __name__ == '__main__':
+ finished = []
+ try:
+ execfile(_main_with_alarm(finished))
+ finally:
+ finished.append(True)
Modified: pypy/branch/arm-backend/pypy/tool/readdictinfo.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/readdictinfo.py (original)
+++ pypy/branch/arm-backend/pypy/tool/readdictinfo.py Tue Nov 23 10:12:47 2010
@@ -7,37 +7,38 @@
import sys
-infile = open(sys.argv[1])
+if __name__ == '__main__':
+ infile = open(sys.argv[1])
-curr = None
-slots = []
-for line in infile:
- if line == '------------------\n':
- if curr:
- break
- curr = 1
- else:
- attr, val = [s.strip() for s in line.split(':')]
- slots.append(attr)
+ curr = None
+ slots = []
+ for line in infile:
+ if line == '------------------\n':
+ if curr:
+ break
+ curr = 1
+ else:
+ attr, val = [s.strip() for s in line.split(':')]
+ slots.append(attr)
-class DictInfo(object):
- __slots__ = slots
+ class DictInfo(object):
+ __slots__ = slots
-infile = open(sys.argv[1])
+ infile = open(sys.argv[1])
-infos = []
+ infos = []
-for line in infile:
- if line == '------------------\n':
- curr = object.__new__(DictInfo)
- infos.append(curr)
- else:
- attr, val = [s.strip() for s in line.split(':')]
- if '.' in val:
- val = float(val)
+ for line in infile:
+ if line == '------------------\n':
+ curr = object.__new__(DictInfo)
+ infos.append(curr)
else:
- val = int(val)
- setattr(curr, attr, val)
+ attr, val = [s.strip() for s in line.split(':')]
+ if '.' in val:
+ val = float(val)
+ else:
+ val = int(val)
+ setattr(curr, attr, val)
def histogram(infos, keyattr, *attrs):
r = {}
Modified: pypy/branch/arm-backend/pypy/tool/release/force-builds.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/release/force-builds.py (original)
+++ pypy/branch/arm-backend/pypy/tool/release/force-builds.py Tue Nov 23 10:12:47 2010
@@ -20,11 +20,12 @@
'own-linux-x86-32',
'own-linux-x86-64',
# 'own-macosx-x86-32',
- 'pypy-c-app-level-linux-x86-32',
- 'pypy-c-app-level-linux-x86-64',
+# 'pypy-c-app-level-linux-x86-32',
+# 'pypy-c-app-level-linux-x86-64',
'pypy-c-stackless-app-level-linux-x86-32',
'pypy-c-app-level-win-x86-32',
'pypy-c-jit-linux-x86-32',
+ 'pypy-c-jit-linux-x86-64',
# 'pypy-c-jit-macosx-x86-32',
'pypy-c-jit-win-x86-32',
]
Modified: pypy/branch/arm-backend/pypy/tool/release/make_release.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/release/make_release.py (original)
+++ pypy/branch/arm-backend/pypy/tool/release/make_release.py Tue Nov 23 10:12:47 2010
@@ -4,7 +4,7 @@
into release packages. Note: you must run apropriate buildbots first and
make sure there are no failures. Use force-builds.py from the same directory.
-Usage: make_release.py release/<release name>
+Usage: make_release.py release/<release name> release_version
"""
import autopath
@@ -30,7 +30,8 @@
else:
xml = override_xml
dom = minidom.parseString(xml)
- refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a')]
+ refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a')
+ if 'pypy' in node.getAttribute('href')]
# all refs are of form: pypy-{type}-{revision}-{platform}.tar.bz2
r = re.compile('pypy-c-([\w\d]+)-(\d+)-([\w\d]+).tar.bz2$')
d = {}
@@ -76,7 +77,7 @@
t.add('pypy-%s' % release)
alltars.append(name)
t.close()
- shutil.rmtree(str(tmpdir.join('pypy-1.3')))
+ shutil.rmtree(str(tmpdir.join('pypy-' + release)))
for name in alltars:
print "Uploading %s" % name
os.system('scp %s codespeak.net:/www/pypy.org/htdocs/download' % name)
@@ -84,8 +85,8 @@
os.chdir(olddir)
if __name__ == '__main__':
- if len(sys.argv) != 2:
+ if len(sys.argv) != 3:
print __doc__
sys.exit(1)
- main(sys.argv[1], release='1.3')
+ main(sys.argv[1], release=sys.argv[2])
Modified: pypy/branch/arm-backend/pypy/tool/release/package.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/release/package.py (original)
+++ pypy/branch/arm-backend/pypy/tool/release/package.py Tue Nov 23 10:12:47 2010
@@ -1,8 +1,12 @@
#!/usr/bin/env python
""" A sample script that packages PyPy, provided that it's already built.
-Usage:
+It uses 'pypy/translator/goal/pypy-c' and parts of the rest of the working
+copy. Usage:
-package.py pypydir [name-of-archive] [name-of-pypy-c]
+ package.py root-pypy-dir [name-of-archive] [name-of-pypy-c]
+
+Usually you would do: package.py ../../.. pypy-VER-PLATFORM.
+The output is found in the directory /tmp/usession-YOURNAME/build/.
"""
import autopath
@@ -32,7 +36,7 @@
class PyPyCNotFound(Exception):
pass
-def package(basedir, name='pypy-nightly', rename_pypy_c='pypy-c',
+def package(basedir, name='pypy-nightly', rename_pypy_c='pypy',
copy_to_dir = None, override_pypy_c = None):
basedir = py.path.local(basedir)
if sys.platform == 'win32':
@@ -64,6 +68,10 @@
headers = includedir.listdir('*.h') + includedir.listdir('*.inl')
for n in headers:
shutil.copy(str(n), str(pypydir.join('include')))
+ #
+ spdir = pypydir.ensure('site-packages', dir=True)
+ shutil.copy(str(basedir.join('site-packages', 'README')), str(spdir))
+ #
pypydir.ensure('bin', dir=True)
archive_pypy_c = pypydir.join('bin', rename_pypy_c)
shutil.copy(str(pypy_c), str(archive_pypy_c))
Modified: pypy/branch/arm-backend/pypy/tool/release/test/test_package.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/release/test/test_package.py (original)
+++ pypy/branch/arm-backend/pypy/tool/release/test/test_package.py Tue Nov 23 10:12:47 2010
@@ -18,7 +18,7 @@
prefix = builddir.join(test)
cpyver = '%d.%d.%d' % CPYTHON_VERSION[:3]
assert prefix.join('lib-python', cpyver, 'test').check()
- assert prefix.join('bin', 'pypy-c').check()
+ assert prefix.join('bin', 'pypy').check()
assert prefix.join('lib_pypy', 'syslog.py').check()
assert not prefix.join('lib_pypy', 'py').check()
assert not prefix.join('lib_pypy', 'ctypes_configure').check()
Modified: pypy/branch/arm-backend/pypy/tool/rundictbenchmarks.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/rundictbenchmarks.py (original)
+++ pypy/branch/arm-backend/pypy/tool/rundictbenchmarks.py Tue Nov 23 10:12:47 2010
@@ -7,20 +7,21 @@
# need to hack a copy of rst2html for yourself (svn docutils
# required).
-try:
- os.unlink("dictinfo.txt")
-except os.error:
- pass
+if __name__ == '__main__':
+ try:
+ os.unlink("dictinfo.txt")
+ except os.error:
+ pass
-progs = [('pystone', ['-c', 'from test import pystone; pystone.main()']),
- ('richards', ['richards.py']),
- ('docutils', ['rst2html.py', '../../doc/coding-guide.txt', 'foo.html']),
- ('translate', ['translate.py', '--backendopt', '--no-compile', '--batch',
- 'targetrpystonedalone.py'])
- ]
+ progs = [('pystone', ['-c', 'from test import pystone; pystone.main()']),
+ ('richards', ['richards.py']),
+ ('docutils', ['rst2html.py', '../../doc/coding-guide.txt', 'foo.html']),
+ ('translate', ['translate.py', '--backendopt', '--no-compile', '--batch',
+ 'targetrpystonedalone.py'])
+ ]
-EXE = sys.argv[1]
+ EXE = sys.argv[1]
-for suffix, args in progs:
- os.spawnv(os.P_WAIT, EXE, [EXE] + args)
- os.rename('dictinfo.txt', 'dictinfo-%s.txt'%suffix)
+ for suffix, args in progs:
+ os.spawnv(os.P_WAIT, EXE, [EXE] + args)
+ os.rename('dictinfo.txt', 'dictinfo-%s.txt'%suffix)
Modified: pypy/branch/arm-backend/pypy/tool/statistic_irc_log.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/statistic_irc_log.py (original)
+++ pypy/branch/arm-backend/pypy/tool/statistic_irc_log.py Tue Nov 23 10:12:47 2010
@@ -2,53 +2,54 @@
from os import system, chdir
from urllib import urlopen
-log_URL = 'http://tismerysoft.de/pypy/irc-logs/'
-archive_FILENAME = 'pypy.tar.gz'
-
-tempdir = py.test.ensuretemp("irc-log")
-
-# get compressed archive
-chdir( str(tempdir))
-system('wget -q %s%s' % (log_URL, archive_FILENAME))
-system('tar xzf %s' % archive_FILENAME)
-chdir('pypy')
-
-# get more recent daily logs
-pypydir = tempdir.join('pypy')
-for line in urlopen(log_URL + 'pypy/').readlines():
- i = line.find('%23pypy.log.')
- if i == -1:
- continue
- filename = line[i:].split('"')[0]
- system('wget -q %spypy/%s' % (log_URL, filename))
-
-# rename to YYYYMMDD
-for log_filename in pypydir.listdir('#pypy.log.*'):
- rename_to = None
- b = log_filename.basename
- if '-' in b:
- rename_to = log_filename.basename.replace('-', '')
- elif len(b) == 19:
- months= 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split()
- day = b[10:12]
- month = months.index(b[12:15]) + 1
- year = b[15:20]
- rename_to = '#pypy.log.%04s%02d%02s' % (year, month, day)
-
- if rename_to:
- log_filename.rename(rename_to)
- #print 'RENAMED', log_filename, 'TO', rename_to
-
-# print sorted list of filenames of daily logs
-print 'irc://irc.freenode.org/pypy'
-print 'date, messages, visitors'
-for log_filename in pypydir.listdir('#pypy.log.*'):
- n_messages, visitors = 0, {}
- f = str(log_filename)
- for s in file(f):
- if '<' in s and '>' in s:
- n_messages += 1
- elif ' joined #pypy' in s:
- v = s.split()[1]
- visitors[v] = True
- print '%04s-%02s-%02s, %d, %d' % (f[-8:-4], f[-4:-2], f[-2:], n_messages, len(visitors.keys()))
+if __name__ == '__main__':
+ log_URL = 'http://tismerysoft.de/pypy/irc-logs/'
+ archive_FILENAME = 'pypy.tar.gz'
+
+ tempdir = py.test.ensuretemp("irc-log")
+
+ # get compressed archive
+ chdir( str(tempdir))
+ system('wget -q %s%s' % (log_URL, archive_FILENAME))
+ system('tar xzf %s' % archive_FILENAME)
+ chdir('pypy')
+
+ # get more recent daily logs
+ pypydir = tempdir.join('pypy')
+ for line in urlopen(log_URL + 'pypy/').readlines():
+ i = line.find('%23pypy.log.')
+ if i == -1:
+ continue
+ filename = line[i:].split('"')[0]
+ system('wget -q %spypy/%s' % (log_URL, filename))
+
+ # rename to YYYYMMDD
+ for log_filename in pypydir.listdir('#pypy.log.*'):
+ rename_to = None
+ b = log_filename.basename
+ if '-' in b:
+ rename_to = log_filename.basename.replace('-', '')
+ elif len(b) == 19:
+ months= 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split()
+ day = b[10:12]
+ month = months.index(b[12:15]) + 1
+ year = b[15:20]
+ rename_to = '#pypy.log.%04s%02d%02s' % (year, month, day)
+
+ if rename_to:
+ log_filename.rename(rename_to)
+ #print 'RENAMED', log_filename, 'TO', rename_to
+
+ # print sorted list of filenames of daily logs
+ print 'irc://irc.freenode.org/pypy'
+ print 'date, messages, visitors'
+ for log_filename in pypydir.listdir('#pypy.log.*'):
+ n_messages, visitors = 0, {}
+ f = str(log_filename)
+ for s in file(f):
+ if '<' in s and '>' in s:
+ n_messages += 1
+ elif ' joined #pypy' in s:
+ v = s.split()[1]
+ visitors[v] = True
+ print '%04s-%02s-%02s, %d, %d' % (f[-8:-4], f[-4:-2], f[-2:], n_messages, len(visitors.keys()))
Modified: pypy/branch/arm-backend/pypy/tool/terminal.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/terminal.py (original)
+++ pypy/branch/arm-backend/pypy/tool/terminal.py Tue Nov 23 10:12:47 2010
@@ -62,9 +62,10 @@
for control in CONTROLS:
# Set the control escape sequence
setattr(MODULE, control, curses.tigetstr(CONTROLS[control]) or '')
- for value in VALUES:
- # Set terminal related values
- setattr(MODULE, value, curses.tigetnum(VALUES[value]))
+ if hasattr(curses, 'tigetnum'):
+ for value in VALUES:
+ # Set terminal related values
+ setattr(MODULE, value, curses.tigetnum(VALUES[value]))
def render(text):
"""Helper function to apply controls easily
@@ -74,7 +75,16 @@
return text % MODULE.__dict__
try:
- import curses
+ if '__pypy__' in sys.builtin_module_names:
+ # this is not really the whole curses, but our _minimal_curses it's
+ # better than nothing
+ import _minimal_curses as curses
+ # a bit of a hack: we have tigetstr but not tigetnum, so we call
+ # default() to have default values, then setup() will overwrite the
+ # ones it can
+ default()
+ else:
+ import curses
setup()
except Exception, e:
# There is a failure; set all attributes to default
Modified: pypy/branch/arm-backend/pypy/tool/watchdog.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/watchdog.py (original)
+++ pypy/branch/arm-backend/pypy/tool/watchdog.py Tue Nov 23 10:12:47 2010
@@ -7,9 +7,6 @@
return name
return 'signal %d' % (n,)
-timeout = float(sys.argv[1])
-timedout = False
-
def childkill():
global timedout
timedout = True
@@ -20,31 +17,35 @@
except OSError:
pass
-pid = os.fork()
-if pid == 0:
- os.execvp(sys.argv[2], sys.argv[2:])
-else: # parent
- t = threading.Timer(timeout, childkill)
- t.start()
- while True:
- try:
- pid, status = os.waitpid(pid, 0)
- except KeyboardInterrupt:
- continue
+if __name__ == '__main__':
+ timeout = float(sys.argv[1])
+ timedout = False
+
+ pid = os.fork()
+ if pid == 0:
+ os.execvp(sys.argv[2], sys.argv[2:])
+ else: # parent
+ t = threading.Timer(timeout, childkill)
+ t.start()
+ while True:
+ try:
+ pid, status = os.waitpid(pid, 0)
+ except KeyboardInterrupt:
+ continue
+ else:
+ t.cancel()
+ break
+ if os.WIFEXITED(status):
+ sys.exit(os.WEXITSTATUS(status))
else:
- t.cancel()
- break
- if os.WIFEXITED(status):
- sys.exit(os.WEXITSTATUS(status))
- else:
- assert os.WIFSIGNALED(status)
- sign = os.WTERMSIG(status)
- if timedout and sign == signal.SIGTERM:
+ assert os.WIFSIGNALED(status)
+ sign = os.WTERMSIG(status)
+ if timedout and sign == signal.SIGTERM:
+ sys.exit(1)
+ signame = getsignalname(sign)
+ sys.stderr.write("="*26 + "timedout" + "="*26 + "\n")
+ sys.stderr.write("="*25 + " %-08s " % signame + "="*25 + "\n")
sys.exit(1)
- signame = getsignalname(sign)
- sys.stderr.write("="*26 + "timedout" + "="*26 + "\n")
- sys.stderr.write("="*25 + " %-08s " % signame + "="*25 + "\n")
- sys.exit(1)
-
-
+
+
Modified: pypy/branch/arm-backend/pypy/tool/watchdog_nt.py
==============================================================================
--- pypy/branch/arm-backend/pypy/tool/watchdog_nt.py (original)
+++ pypy/branch/arm-backend/pypy/tool/watchdog_nt.py Tue Nov 23 10:12:47 2010
@@ -2,11 +2,6 @@
import threading
import ctypes
-PROCESS_TERMINATE = 0x1
-
-timeout = float(sys.argv[1])
-timedout = False
-
def childkill(pid):
global timedout
timedout = True
@@ -14,19 +9,25 @@
sys.stderr.write("="*26 + "timedout" + "="*26 + "\n")
ctypes.windll.kernel32.TerminateProcess(pid, 1)
-pid = os.spawnv(os.P_NOWAIT, sys.argv[2], sys.argv[2:])
+if __name__ == '__main__':
+ PROCESS_TERMINATE = 0x1
+
+ timeout = float(sys.argv[1])
+ timedout = False
+
+ pid = os.spawnv(os.P_NOWAIT, sys.argv[2], sys.argv[2:])
+
+ t = threading.Timer(timeout, childkill, (pid,))
+ t.start()
+ while True:
+ try:
+ pid, status = os.waitpid(pid, 0)
+ except KeyboardInterrupt:
+ continue
+ else:
+ t.cancel()
+ break
+
+ #print 'status ', status >> 8
+ sys.exit(status >> 8)
-t = threading.Timer(timeout, childkill, (pid,))
-t.start()
-while True:
- try:
- pid, status = os.waitpid(pid, 0)
- except KeyboardInterrupt:
- continue
- else:
- t.cancel()
- break
-
-#print 'status ', status >> 8
-sys.exit(status >> 8)
-
Modified: pypy/branch/arm-backend/pypy/translator/c/funcgen.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/funcgen.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/funcgen.py Tue Nov 23 10:12:47 2010
@@ -427,7 +427,7 @@
r = self.expr(op.result)
return 'OP_CALL_ARGS((%s), %s);' % (', '.join(args), r)
- def generic_call(self, FUNC, fnexpr, args_v, v_result):
+ def generic_call(self, FUNC, fnexpr, args_v, v_result, targets=None):
args = []
assert len(args_v) == len(FUNC.TO.ARGS)
for v, ARGTYPE in zip(args_v, FUNC.TO.ARGS):
@@ -444,17 +444,26 @@
# skip assignment of 'void' return value
r = self.expr(v_result)
line = '%s = %s' % (r, line)
+ if targets:
+ for graph in targets:
+ if getattr(graph, 'inhibit_tail_call', False):
+ line += '\nPYPY_INHIBIT_TAIL_CALL();'
+ break
return line
def OP_DIRECT_CALL(self, op):
fn = op.args[0]
+ try:
+ targets = [fn.value._obj.graph]
+ except AttributeError:
+ targets = None
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:], op.result)
+ op.args[1:], op.result, targets)
def OP_INDIRECT_CALL(self, op):
fn = op.args[0]
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:-1], op.result)
+ op.args[1:-1], op.result, op.args[-1].value)
def OP_ADR_CALL(self, op):
ARGTYPES = [v.concretetype for v in op.args[1:]]
Modified: pypy/branch/arm-backend/pypy/translator/c/gcc/instruction.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/gcc/instruction.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/gcc/instruction.py Tue Nov 23 10:12:47 2010
@@ -82,6 +82,11 @@
def all_sources_of(self, localvar):
return [localvar]
+class InsnCondJump(Insn): # only for debugging; not used internally
+ _args_ = ['label']
+ def __init__(self, label):
+ self.label = label
+
class Label(Insn):
_args_ = ['label', 'lineno']
def __init__(self, label, lineno):
@@ -170,9 +175,12 @@
self.delta = -7 # use an odd value as marker
class InsnStop(Insn):
- pass
+ _args_ = ['reason']
+ def __init__(self, reason='?'):
+ self.reason = reason
class InsnRet(InsnStop):
+ _args_ = []
framesize = 0
def __init__(self, registers):
self.registers = registers
Modified: pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track5.s
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track5.s (original)
+++ pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track5.s Tue Nov 23 10:12:47 2010
@@ -44,7 +44,7 @@
addl %eax, %ebx
jmp .L1221
.L1227:
- call RPyAbort
+ ;;call RPyAbort
cmpl 12(%esi), %ebx
jb .L1229
addl $20, %esp
Modified: pypy/branch/arm-backend/pypy/translator/c/gcc/test/test_trackgcroot.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/gcc/test/test_trackgcroot.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/gcc/test/test_trackgcroot.py Tue Nov 23 10:12:47 2010
@@ -98,18 +98,17 @@
"""
lines = source.splitlines(True)
parts = list(DarwinAssemblerParser().find_functions(iter(lines)))
- assert len(parts) == 7
+ assert len(parts) == 6
assert parts[0] == (False, lines[:3])
assert parts[1] == (True, lines[3:7])
assert parts[2] == (True, lines[7:11])
- assert parts[3] == (True, lines[11:13])
- assert parts[4] == (False, lines[13:18])
- assert parts[5] == (True, lines[18:20])
- assert parts[6] == (False, lines[20:])
+ assert parts[3] == (True, lines[11:18])
+ assert parts[4] == (True, lines[18:20])
+ assert parts[5] == (False, lines[20:])
def test_computegcmaptable():
tests = []
- for format in ('elf', 'darwin', 'msvc', 'elf64'):
+ for format in ('elf', 'elf64', 'darwin', 'darwin64', 'msvc'):
for path in this_dir.join(format).listdir("track*.s"):
n = path.purebasename[5:]
try:
Modified: pypy/branch/arm-backend/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/gcc/trackgcroot.py Tue Nov 23 10:12:47 2010
@@ -6,7 +6,7 @@
from pypy.translator.c.gcc.instruction import InsnFunctionStart, InsnStop
from pypy.translator.c.gcc.instruction import InsnSetLocal, InsnCopyLocal
from pypy.translator.c.gcc.instruction import InsnPrologue, InsnEpilogue
-from pypy.translator.c.gcc.instruction import InsnGCROOT
+from pypy.translator.c.gcc.instruction import InsnGCROOT, InsnCondJump
from pypy.translator.c.gcc.instruction import InsnStackAdjust
from pypy.translator.c.gcc.instruction import InsnCannotFollowEsp
from pypy.translator.c.gcc.instruction import LocalVar, somenewvalue
@@ -46,6 +46,7 @@
self.findlabels()
self.parse_instructions()
try:
+ self.trim_unreachable_instructions()
self.find_noncollecting_calls()
if not self.list_collecting_call_insns():
return []
@@ -122,19 +123,36 @@
assert label not in self.labels, "duplicate label: %s" % label
self.labels[label] = Label(label, lineno)
+ def trim_unreachable_instructions(self):
+ reached = set([self.insns[0]])
+ prevlen = 0
+ while len(reached) > prevlen:
+ prevlen = len(reached)
+ for insn in self.insns:
+ if insn not in reached:
+ for previnsn in insn.previous_insns:
+ if previnsn in reached:
+ # this instruction is reachable too
+ reached.add(insn)
+ break
+ # now kill all unreachable instructions
+ i = 0
+ while i < len(self.insns):
+ if self.insns[i] in reached:
+ i += 1
+ else:
+ del self.insns[i]
+
def find_noncollecting_calls(self):
- cannot_collect = self.CANNOT_COLLECT.copy()
+ cannot_collect = {}
for line in self.lines:
match = self.r_gcnocollect_marker.search(line)
if match:
name = match.group(1)
cannot_collect[name] = True
#
- if self.format in ('darwin', 'mingw32', 'msvc'):
- self.cannot_collect = dict.fromkeys(
- ['_' + name for name in cannot_collect])
- else:
- self.cannot_collect = cannot_collect
+ self.cannot_collect = dict.fromkeys(
+ [self.function_names_prefix + name for name in cannot_collect])
def append_instruction(self, insn):
# Add the instruction to the list, and link it to the previous one.
@@ -410,7 +428,8 @@
return result
# ____________________________________________________________
- CANNOT_COLLECT = { # some of the most used functions that cannot collect
+ BASE_FUNCTIONS_NOT_RETURNING = {
+ 'abort': None,
'pypy_debug_catch_fatal_exception': None,
'RPyAbort': None,
'RPyAssertFailed': None,
@@ -436,7 +455,7 @@
'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc',
'shl', 'shr', 'sal', 'sar', 'rol', 'ror', 'mul', 'imul', 'div', 'idiv',
'bswap', 'bt', 'rdtsc',
- 'punpck', 'pshufd',
+ 'punpck', 'pshufd', 'psll',
# zero-extending moves should not produce GC pointers
'movz',
])
@@ -644,7 +663,7 @@
if label != '0':
self.register_jump_to(label)
tablelin += 1
- return InsnStop()
+ return InsnStop("jump table")
if self.r_unaryinsn_star.match(line):
# that looks like an indirect tail-call.
# tail-calls are equivalent to RET for us
@@ -658,7 +677,7 @@
assert not target.startswith('.')
# tail-calls are equivalent to RET for us
return InsnRet(self.CALLEE_SAVE_REGISTERS)
- return InsnStop()
+ return InsnStop("jump")
def register_jump_to(self, label):
if not isinstance(self.insns[-1], InsnStop):
@@ -682,7 +701,7 @@
else:
label = match.group(1)
self.register_jump_to(label)
- return []
+ return [InsnCondJump(label)]
visit_jmpl = visit_jmp
visit_je = conditional_jump
@@ -754,7 +773,7 @@
target, = sources
if target in self.FUNCTIONS_NOT_RETURNING:
- return [InsnStop(), InsnCannotFollowEsp()]
+ return [InsnStop(target)]
if self.format == 'mingw32' and target == '__alloca':
# in functions with large stack requirements, windows
# needs a call to _alloca(), to turn reserved pages
@@ -885,7 +904,7 @@
# statically known pointer to a register
# %eax -> %rax
- new_line = re.sub(r"%e(ax|bx|cx|dx|di|si)$", r"%r\1", line)
+ new_line = re.sub(r"%e(ax|bx|cx|dx|di|si|bp)$", r"%r\1", line)
# %r10d -> %r10
new_line = re.sub(r"%r(\d+)d$", r"%r\1", new_line)
return func(self, new_line)
@@ -951,6 +970,7 @@
class ElfFunctionGcRootTracker32(FunctionGcRootTracker32):
format = 'elf'
+ function_names_prefix = ''
ESP = '%esp'
EBP = '%ebp'
@@ -984,13 +1004,14 @@
r_bottom_marker = re.compile(r"\t/[*] GC_STACK_BOTTOM [*]/")
FUNCTIONS_NOT_RETURNING = {
- 'abort': None,
'_exit': None,
'__assert_fail': None,
'___assert_rtn': None,
'L___assert_rtn$stub': None,
'L___eprintf$stub': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING[_name] = None
def __init__(self, lines, filetag=0):
match = self.r_functionstart.match(lines[0])
@@ -1010,6 +1031,8 @@
class ElfFunctionGcRootTracker64(FunctionGcRootTracker64):
format = 'elf64'
+ function_names_prefix = ''
+
ESP = '%rsp'
EBP = '%rbp'
EAX = '%rax'
@@ -1042,13 +1065,14 @@
r_bottom_marker = re.compile(r"\t/[*] GC_STACK_BOTTOM [*]/")
FUNCTIONS_NOT_RETURNING = {
- 'abort': None,
'_exit': None,
'__assert_fail': None,
'___assert_rtn': None,
'L___assert_rtn$stub': None,
'L___eprintf$stub': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING[_name] = None
def __init__(self, lines, filetag=0):
match = self.r_functionstart.match(lines[0])
@@ -1066,8 +1090,9 @@
ElfFunctionGcRootTracker64.init_regexp()
-class DarwinFunctionGcRootTracker(ElfFunctionGcRootTracker32):
+class DarwinFunctionGcRootTracker32(ElfFunctionGcRootTracker32):
format = 'darwin'
+ function_names_prefix = '_'
r_functionstart = re.compile(r"_(\w+):\s*$")
OFFSET_LABELS = 0
@@ -1077,17 +1102,36 @@
funcname = '_' + match.group(1)
FunctionGcRootTracker32.__init__(self, funcname, lines, filetag)
-class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker):
+class DarwinFunctionGcRootTracker64(ElfFunctionGcRootTracker64):
+ format = 'darwin64'
+ function_names_prefix = '_'
+
+ LABEL = ElfFunctionGcRootTracker32.LABEL
+ r_jmptable_item = re.compile(r"\t.(?:long|quad)\t"+LABEL+"(-\"?[A-Za-z0-9$]+\"?)?\s*$")
+
+ r_functionstart = re.compile(r"_(\w+):\s*$")
+ OFFSET_LABELS = 0
+
+ def __init__(self, lines, filetag=0):
+ match = self.r_functionstart.match(lines[0])
+ funcname = '_' + match.group(1)
+ FunctionGcRootTracker64.__init__(self, funcname, lines, filetag)
+
+class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker32):
format = 'mingw32'
+ function_names_prefix = '_'
FUNCTIONS_NOT_RETURNING = {
- '_abort': None,
'_exit': None,
'__assert': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING['_' + _name] = None
class MsvcFunctionGcRootTracker(FunctionGcRootTracker32):
format = 'msvc'
+ function_names_prefix = '_'
+
ESP = 'esp'
EBP = 'ebp'
EAX = 'eax'
@@ -1127,7 +1171,6 @@
r_bottom_marker = re.compile(r"; .+\tpypy_asm_stack_bottom\(\);")
FUNCTIONS_NOT_RETURNING = {
- '_abort': None,
'__exit': None,
'__assert': None,
'__wassert': None,
@@ -1136,6 +1179,8 @@
'DWORD PTR __imp__abort': None,
'DWORD PTR __imp___wassert': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING['_' + _name] = None
@classmethod
def init_regexp(cls):
@@ -1343,7 +1388,7 @@
class DarwinAssemblerParser(AssemblerParser):
format = "darwin"
- FunctionGcRootTracker = DarwinFunctionGcRootTracker
+ FunctionGcRootTracker = DarwinFunctionGcRootTracker32
r_textstart = re.compile(r"\t.text\s*$")
@@ -1360,6 +1405,7 @@
'const_data'
]
r_sectionstart = re.compile(r"\t\.("+'|'.join(OTHERSECTIONS)+").*$")
+ sections_doesnt_end_function = {'cstring': True, 'const': True}
def find_functions(self, iterlines):
functionlines = []
@@ -1367,20 +1413,20 @@
in_function = False
for n, line in enumerate(iterlines):
if self.r_textstart.match(line):
- assert not in_text, "unexpected repeated .text start: %d" % n
in_text = True
elif self.r_sectionstart.match(line):
- if in_function:
+ sectionname = self.r_sectionstart.match(line).group(1)
+ if (in_function and
+ sectionname not in self.sections_doesnt_end_function):
yield in_function, functionlines
functionlines = []
+ in_function = False
in_text = False
- in_function = False
elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line):
yield in_function, functionlines
functionlines = []
in_function = True
functionlines.append(line)
-
if functionlines:
yield in_function, functionlines
@@ -1389,27 +1435,14 @@
return super(DarwinAssemblerParser, self).process_function(
lines, entrypoint, filename)
+class DarwinAssemblerParser64(DarwinAssemblerParser):
+ format = "darwin64"
+ FunctionGcRootTracker = DarwinFunctionGcRootTracker64
+
class Mingw32AssemblerParser(DarwinAssemblerParser):
format = "mingw32"
FunctionGcRootTracker = Mingw32FunctionGcRootTracker
- def find_functions(self, iterlines):
- functionlines = []
- in_text = False
- in_function = False
- for n, line in enumerate(iterlines):
- if self.r_textstart.match(line):
- in_text = True
- elif self.r_sectionstart.match(line):
- in_text = False
- elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line):
- yield in_function, functionlines
- functionlines = []
- in_function = True
- functionlines.append(line)
- if functionlines:
- yield in_function, functionlines
-
class MsvcAssemblerParser(AssemblerParser):
format = "msvc"
FunctionGcRootTracker = MsvcFunctionGcRootTracker
@@ -1512,6 +1545,7 @@
'elf': ElfAssemblerParser,
'elf64': ElfAssemblerParser64,
'darwin': DarwinAssemblerParser,
+ 'darwin64': DarwinAssemblerParser64,
'mingw32': Mingw32AssemblerParser,
'msvc': MsvcAssemblerParser,
}
@@ -1543,15 +1577,13 @@
assert self.seen_main
def _globalname(name, disp=""):
- if self.format in ('darwin', 'mingw32', 'msvc'):
- name = '_' + name
- return name
+ return tracker_cls.function_names_prefix + name
def _variant(**kwargs):
txt = kwargs[self.format]
print >> output, "\t%s" % txt
- if self.format == 'elf64':
+ if self.format in ('elf64', 'darwin64'):
word_decl = '.quad'
else:
word_decl = '.long'
@@ -1604,10 +1636,11 @@
}
}
"""
- elif self.format == 'elf64':
+ elif self.format in ('elf64', 'darwin64'):
print >> output, "\t.text"
print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk')
- print >> output, "\t.type pypy_asm_stackwalk, @function"
+ _variant(elf64='.type pypy_asm_stackwalk, @function',
+ darwin64='')
print >> output, "%s:" % _globalname('pypy_asm_stackwalk')
print >> output, """\
@@ -1652,8 +1685,9 @@
/* the return value is the one of the 'call' above, */
/* because %rax (and possibly %rdx) are unmodified */
ret
- .size pypy_asm_stackwalk, .-pypy_asm_stackwalk
"""
+ _variant(elf64='.size pypy_asm_stackwalk, .-pypy_asm_stackwalk',
+ darwin64='')
else:
print >> output, "\t.text"
print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk')
@@ -1780,6 +1814,7 @@
_variant(elf='.section\t.rodata',
elf64='.section\t.rodata',
darwin='.const',
+ darwin64='.const',
mingw32='')
print >> output, """\
@@ -1853,11 +1888,14 @@
if __name__ == '__main__':
- verbose = 1
+ verbose = 0
shuffle = False
output_raw_table = False
if sys.platform == 'darwin':
- format = 'darwin'
+ if sys.maxint > 2147483647:
+ format = 'darwin64'
+ else:
+ format = 'darwin'
elif sys.platform == 'win32':
format = 'mingw32'
else:
Modified: pypy/branch/arm-backend/pypy/translator/c/genc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/genc.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/genc.py Tue Nov 23 10:12:47 2010
@@ -553,7 +553,7 @@
('debug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT" $(TARGET)'),
('debug_exc', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DDO_LOG_EXC" $(TARGET)'),
('debug_mem', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DTRIVIAL_MALLOC_DEBUG" $(TARGET)'),
- ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O1 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'),
+ ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O2 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'),
('linuxmemchk', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DLINUXMEMCHK" $(TARGET)'),
('llsafer', '', '$(MAKE) CFLAGS="-O2 -DRPY_LL_ASSERT" $(TARGET)'),
('lldebug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DRPY_LL_ASSERT" $(TARGET)'),
@@ -623,9 +623,16 @@
mk.definition('OBJECTS', '$(ASMLBLFILES) gcmaptable.s')
mk.rule('%.s', '%.c', '$(CC) $(CFLAGS) $(CFLAGSEXTRA) -frandom-seed=$< -o $@ -S $< $(INCLUDEDIRS)')
mk.rule('%.lbl.s %.gcmap', '%.s',
- python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py -m$(PYPY_MAIN_FUNCTION) -t $< > $*.gcmap')
+ [python +
+ '$(PYPYDIR)/translator/c/gcc/trackgcroot.py '
+ '-m$(PYPY_MAIN_FUNCTION) -t $< > $*.gctmp',
+ 'mv $*.gctmp $*.gcmap'])
mk.rule('gcmaptable.s', '$(GCMAPFILES)',
- python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py $(GCMAPFILES) > $@')
+ [python +
+ '$(PYPYDIR)/translator/c/gcc/trackgcroot.py '
+ '$(GCMAPFILES) > $@.tmp',
+ 'mv $@.tmp $@'])
+ mk.rule('.PRECIOUS', '%.s', "# don't remove .s files if Ctrl-C'ed")
else:
mk.definition('DEBUGFLAGS', '-O1 -g')
Modified: pypy/branch/arm-backend/pypy/translator/c/node.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/node.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/node.py Tue Nov 23 10:12:47 2010
@@ -714,7 +714,11 @@
s = ''.join([self.obj.getitem(i) for i in range(len(self.obj.items))])
else:
s = ''.join(self.obj.items)
- yield '\t%s%s' % (length, c_char_array_constant(s))
+ array_constant = c_char_array_constant(s)
+ if array_constant.startswith('{') and barebonearray(T):
+ assert array_constant.endswith('}')
+ array_constant = array_constant[1:-1].strip()
+ yield '\t%s%s' % (length, array_constant)
yield '}'
else:
barebone = barebonearray(T)
Modified: pypy/branch/arm-backend/pypy/translator/c/src/asm_gcc_x86.h
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/src/asm_gcc_x86.h (original)
+++ pypy/branch/arm-backend/pypy/translator/c/src/asm_gcc_x86.h Tue Nov 23 10:12:47 2010
@@ -2,6 +2,8 @@
* It replaces some complex macros with native assembler instructions.
*/
+#if 0 /* --- disabled: does not give any speed-up --- */
+
#undef OP_INT_ADD_OVF
#define OP_INT_ADD_OVF(x,y,r) \
asm volatile( \
@@ -50,6 +52,13 @@
: "0"(x), "g"(y) /* inputs */ \
: "cc", "memory") /* clobber */
+extern void op_int_overflowed(void)
+ asm ("_op_int_overflowed")
+ __attribute__((used));
+
+#endif /* 0 */
+
+
/* Pentium only! */
#define READ_TIMESTAMP(val) \
asm volatile("rdtsc" : "=A" (val))
@@ -62,19 +71,15 @@
// I don't know how important it is, comment talks about time warps
-/* prototypes */
-
-extern void op_int_overflowed(void)
- asm ("_op_int_overflowed")
- __attribute__((used));
-
/* implementations */
#ifndef PYPY_NOT_MAIN_FILE
+# if 0 /* disabled */
void op_int_overflowed(void)
{
FAIL_OVF("integer operation");
}
+# endif
#endif
Modified: pypy/branch/arm-backend/pypy/translator/c/src/debug_print.h
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/src/debug_print.h (original)
+++ pypy/branch/arm-backend/pypy/translator/c/src/debug_print.h Tue Nov 23 10:12:47 2010
@@ -10,6 +10,7 @@
but not any nested debug_print
:fname full logging
prefix:fname conditional logging
+ prefix1,prefix2:fname conditional logging with multiple selections
Conditional logging means that it only includes the debug_start/debug_stop
sections whose name match 'prefix'. Other sections are ignored, including
@@ -70,6 +71,8 @@
static void pypy_debug_open(void)
{
char *filename = getenv("PYPYLOG");
+ if (filename)
+ unsetenv("PYPYLOG"); /* don't pass it to subprocesses */
if (filename && filename[0])
{
char *colon = strchr(filename, ':');
@@ -139,12 +142,22 @@
#endif
-static bool_t startswith(const char *str, const char *substr)
+static bool_t startswithoneof(const char *str, const char *substr)
{
- while (*substr)
- if (*str++ != *substr++)
- return 0;
- return 1;
+ const char *p = str;
+ for (; *substr; substr++)
+ {
+ if (*substr != ',')
+ {
+ if (p && *p++ != *substr)
+ p = NULL; /* mismatch */
+ }
+ else if (p != NULL)
+ return 1; /* match */
+ else
+ p = str; /* mismatched, retry with the next */
+ }
+ return p != NULL;
}
#if defined(_MSC_VER) || defined(__MINGW32__)
@@ -175,7 +188,7 @@
if (!debug_profile)
{
/* non-profiling version */
- if (!debug_prefix || !startswith(category, debug_prefix))
+ if (!debug_prefix || !startswithoneof(category, debug_prefix))
{
/* wrong section name, or no PYPYLOG at all, skip it */
return;
Modified: pypy/branch/arm-backend/pypy/translator/c/src/g_include.h
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/src/g_include.h (original)
+++ pypy/branch/arm-backend/pypy/translator/c/src/g_include.h Tue Nov 23 10:12:47 2010
@@ -43,6 +43,10 @@
# include "src/asm_gcc_x86.h"
#endif
+#if defined(__GNUC__) && defined(__amd64__)
+# include "src/asm_gcc_x86_64.h"
+#endif
+
#if defined(__GNUC__) && defined(__ppc__)
# include "src/asm_ppc.h"
#endif
@@ -53,6 +57,7 @@
# include "src/rtyper.h"
# include "src/debug_print.h"
# include "src/debug_traceback.h"
+# include "src/debug_alloc.h"
#ifndef AVR
# include "src/ll_os.h"
# include "src/ll_strtod.h"
Modified: pypy/branch/arm-backend/pypy/translator/c/src/int.h
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/src/int.h (original)
+++ pypy/branch/arm-backend/pypy/translator/c/src/int.h Tue Nov 23 10:12:47 2010
@@ -2,40 +2,27 @@
/************************************************************/
/*** C header subsection: operations between ints ***/
-#ifndef LLONG_MAX
-# if SIZEOF_LONG_LONG == 8
-# define LLONG_MAX 0X7FFFFFFFFFFFFFFFLL
-# else
-# error "fix LLONG_MAX"
-# endif
-#endif
-
-#ifndef LLONG_MIN
-# define LLONG_MIN (-LLONG_MAX-1)
-#endif
/*** unary operations ***/
-#define OP_INT_IS_TRUE(x,r) OP_INT_NE(x,0,r)
-
-#define OP_INT_INVERT(x,r) r = ~((x))
-
-#define OP_INT_NEG(x,r) r = -(x)
+#define OP_INT_IS_TRUE(x,r) r = ((x) != 0)
+#define OP_INT_INVERT(x,r) r = ~(x)
+#define OP_INT_NEG(x,r) r = -(x)
#define OP_INT_NEG_OVF(x,r) \
- if ((x) == LONG_MIN) FAIL_OVF("integer negate"); \
+ if ((x) == LONG_MIN) FAIL_OVF("integer negate"); \
OP_INT_NEG(x,r)
#define OP_LLONG_NEG_OVF(x,r) \
- if ((x) == LLONG_MIN) FAIL_OVF("integer negate"); \
+ if ((x) == LLONG_MIN) FAIL_OVF("integer negate"); \
OP_LLONG_NEG(x,r)
#define OP_INT_ABS(x,r) r = (x) >= 0 ? x : -(x)
#define OP_INT_ABS_OVF(x,r) \
- if ((x) == LONG_MIN) FAIL_OVF("integer absolute"); \
+ if ((x) == LONG_MIN) FAIL_OVF("integer absolute"); \
OP_INT_ABS(x,r)
#define OP_LLONG_ABS_OVF(x,r) \
- if ((x) == LLONG_MIN) FAIL_OVF("integer absolute"); \
+ if ((x) == LLONG_MIN) FAIL_OVF("integer absolute"); \
OP_LLONG_ABS(x,r)
/*** binary operations ***/
@@ -59,50 +46,46 @@
#define OP_INT_ADD(x,y,r) r = (x) + (y)
+/* cast to avoid undefined behaviour on overflow */
#define OP_INT_ADD_OVF(x,y,r) \
- OP_INT_ADD(x,y,r); \
- if ((r^(x)) >= 0 || (r^(y)) >= 0); \
- else FAIL_OVF("integer addition")
+ r = (long)((unsigned long)x + y); \
+ if ((r^x) < 0 && (r^y) < 0) FAIL_OVF("integer addition")
+
+#define OP_LLONG_ADD_OVF(x,y,r) \
+ r = (long long)((unsigned long long)x + y); \
+ if ((r^x) < 0 && (r^y) < 0) FAIL_OVF("integer addition")
#define OP_INT_ADD_NONNEG_OVF(x,y,r) /* y can be assumed >= 0 */ \
- r = (long)((unsigned long)x + (unsigned long)y); \
- if (r >= (x)); \
- else FAIL_OVF("integer addition")
-/* Can a C compiler be too clever and think it can "prove" that
- * r >= x always holds above? Yes. Hence the casting. */
+ r = (long)((unsigned long)x + y); \
+ if ((r&~x) < 0) FAIL_OVF("integer addition")
#define OP_INT_SUB(x,y,r) r = (x) - (y)
#define OP_INT_SUB_OVF(x,y,r) \
- OP_INT_SUB(x,y,r); \
- if ((r^(x)) >= 0 || (r^~(y)) >= 0); \
- else FAIL_OVF("integer subtraction")
-
-#define OP_INT_MUL(x,y,r) r = (x) * (y)
-
-#if defined(HAVE_LONG_LONG) && SIZE_OF_LONG_LONG < SIZE_OF_LONG
-# define OP_INT_MUL_OVF_LL 1
-#lse
-# define OP_INT_MUL_OVF_LL 0
-#endif
+ r = (long)((unsigned long)x - y); \
+ if ((r^x) < 0 && (r^~y) < 0) FAIL_OVF("integer subtraction")
-#if !OP_INT_MUL_OVF_LL
+#define OP_LLONG_SUB_OVF(x,y,r) \
+ r = (long long)((unsigned long long)x - y); \
+ if ((r^x) < 0 && (r^~y) < 0) FAIL_OVF("integer subtraction")
-#define OP_INT_MUL_OVF(x,y,r) \
- if (op_int_mul_ovf(x,y,&r)); \
- else FAIL_OVF("integer multiplication")
-
-#else
+#define OP_INT_MUL(x,y,r) r = (x) * (y)
+#if SIZEOF_LONG * 2 <= SIZEOF_LONG_LONG
#define OP_INT_MUL_OVF(x,y,r) \
{ \
- PY_LONG_LONG lr = (PY_LONG_LONG)(x) * (PY_LONG_LONG)(y); \
- r = (long)lr; \
- if ((PY_LONG_LONG)r == lr); \
- else FAIL_OVF("integer multiplication"); \
+ long long _lr = (long long)x * y; \
+ r = (long)_lr; \
+ if (_lr != (long long)r) FAIL_OVF("integer multiplication"); \
}
+#else
+#define OP_INT_MUL_OVF(x,y,r) \
+ r = op_llong_mul_ovf(x, y) /* long == long long */
#endif
+#define OP_LLONG_MUL_OVF(x,y,r) \
+ r = op_llong_mul_ovf(x, y)
+
/* shifting */
/* NB. shifting has same limitations as C: the shift count must be
@@ -121,6 +104,10 @@
OP_INT_LSHIFT(x,y,r); \
if ((x) != Py_ARITHMETIC_RIGHT_SHIFT(long, r, (y))) \
FAIL_OVF("x<<y losing bits or changing sign")
+#define OP_LLONG_LSHIFT_OVF(x,y,r) \
+ OP_LLONG_LSHIFT(x,y,r); \
+ if ((x) != Py_ARITHMETIC_RIGHT_SHIFT(PY_LONG_LONG, r, (y))) \
+ FAIL_OVF("x<<y losing bits or changing sign")
/* floor division */
@@ -129,27 +116,48 @@
#define OP_LLONG_FLOORDIV(x,y,r) r = (x) / (y)
#define OP_ULLONG_FLOORDIV(x,y,r) r = (x) / (y)
-#define OP_INT_FLOORDIV_OVF(x,y,r) \
- if ((y) == -1 && (x) == LONG_MIN) \
- { FAIL_OVF("integer division"); } \
- else OP_INT_FLOORDIV(x,y,r)
-
-#define OP_INT_FLOORDIV_ZER(x,y,r) \
- if ((y)) { OP_INT_FLOORDIV(x,y,r); } \
- else FAIL_ZER("integer division")
-#define OP_UINT_FLOORDIV_ZER(x,y,r) \
- if ((y)) { OP_UINT_FLOORDIV(x,y,r); } \
- else FAIL_ZER("unsigned integer division")
-#define OP_LLONG_FLOORDIV_ZER(x,y,r) \
- if ((y)) { OP_LLONG_FLOORDIV(x,y,r); } \
- else FAIL_ZER("integer division")
-#define OP_ULLONG_FLOORDIV_ZER(x,y,r) \
- if ((y)) { OP_ULLONG_FLOORDIV(x,y,r); } \
- else FAIL_ZER("unsigned integer division")
-
-#define OP_INT_FLOORDIV_OVF_ZER(x,y,r) \
- if ((y)) { OP_INT_FLOORDIV_OVF(x,y,r); } \
- else FAIL_ZER("integer division")
+#define OP_INT_FLOORDIV_OVF(x,y,r) \
+ if ((y) == -1 && (x) == LONG_MIN) \
+ { FAIL_OVF("integer division"); r=0; } \
+ else \
+ r = (x) / (y)
+#define OP_LLONG_FLOORDIV_OVF(x,y,r) \
+ if ((y) == -1 && (x) == LLONG_MIN) \
+ { FAIL_OVF("integer division"); r=0; } \
+ else \
+ r = (x) / (y)
+
+#define OP_INT_FLOORDIV_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("integer division"); r=0; } \
+ else \
+ r = (x) / (y)
+#define OP_UINT_FLOORDIV_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("unsigned integer division"); r=0; } \
+ else \
+ r = (x) / (y)
+#define OP_LLONG_FLOORDIV_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("integer division"); r=0; } \
+ else \
+ r = (x) / (y)
+#define OP_ULLONG_FLOORDIV_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("unsigned integer division"); r=0; } \
+ else \
+ r = (x) / (y)
+
+#define OP_INT_FLOORDIV_OVF_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("integer division"); r=0; } \
+ else \
+ { OP_INT_FLOORDIV_OVF(x,y,r); }
+#define OP_LLONG_FLOORDIV_OVF_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("integer division"); r=0; } \
+ else \
+ { OP_LLONG_FLOORDIV_OVF(x,y,r); }
/* modulus */
@@ -158,27 +166,47 @@
#define OP_LLONG_MOD(x,y,r) r = (x) % (y)
#define OP_ULLONG_MOD(x,y,r) r = (x) % (y)
-#define OP_INT_MOD_OVF(x,y,r) \
- if ((y) == -1 && (x) == LONG_MIN) \
- { FAIL_OVF("integer modulo"); }\
- else OP_INT_MOD(x,y,r)
-
-#define OP_INT_MOD_ZER(x,y,r) \
- if ((y)) { OP_INT_MOD(x,y,r); } \
- else FAIL_ZER("integer modulo")
-#define OP_UINT_MOD_ZER(x,y,r) \
- if ((y)) { OP_UINT_MOD(x,y,r); } \
- else FAIL_ZER("unsigned integer modulo")
-#define OP_LLONG_MOD_ZER(x,y,r) \
- if ((y)) { OP_LLONG_MOD(x,y,r); } \
- else FAIL_ZER("integer modulo")
-#define OP_ULLONG_MOD_ZER(x,y,r) \
- if ((y)) { OP_ULLONG_MOD(x,y,r); } \
- else FAIL_ZER("integer modulo")
-
-#define OP_INT_MOD_OVF_ZER(x,y,r) \
- if ((y)) { OP_INT_MOD_OVF(x,y,r); } \
- else FAIL_ZER("integer modulo")
+#define OP_INT_MOD_OVF(x,y,r) \
+ if ((y) == -1 && (x) == LONG_MIN) \
+ { FAIL_OVF("integer modulo"); r=0; } \
+ else \
+ r = (x) % (y)
+#define OP_LLONG_MOD_OVF(x,y,r) \
+ if ((y) == -1 && (x) == LLONG_MIN) \
+ { FAIL_OVF("integer modulo"); r=0; } \
+ else \
+ r = (x) % (y)
+#define OP_INT_MOD_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("integer modulo"); r=0; } \
+ else \
+ r = (x) % (y)
+#define OP_UINT_MOD_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("unsigned integer modulo"); r=0; } \
+ else \
+ r = (x) % (y)
+#define OP_LLONG_MOD_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("integer modulo"); r=0; } \
+ else \
+ r = (x) % (y)
+#define OP_ULLONG_MOD_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("unsigned integer modulo"); r=0; } \
+ else \
+ r = (x) % (y)
+
+#define OP_INT_MOD_OVF_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("integer modulo"); r=0; } \
+ else \
+ { OP_INT_MOD_OVF(x,y,r); }
+#define OP_LLONG_MOD_OVF_ZER(x,y,r) \
+ if ((y) == 0) \
+ { FAIL_ZER("integer modulo"); r=0; } \
+ else \
+ { OP_LLONG_MOD_OVF(x,y,r); }
/* bit operations */
@@ -208,31 +236,30 @@
/* _________________ certain implementations __________________ */
-#if !OP_INT_MUL_OVF_LL
/* adjusted from intobject.c, Python 2.3.3 */
/* prototypes */
-int op_int_mul_ovf(long a, long b, long *longprod);
+long long op_llong_mul_ovf(long long a, long long b);
/* implementations */
#ifndef PYPY_NOT_MAIN_FILE
-int
-op_int_mul_ovf(long a, long b, long *longprod)
+long long op_llong_mul_ovf(long long a, long long b)
{
double doubled_longprod; /* (double)longprod */
double doubleprod; /* (double)a * (double)b */
+ long long longprod;
- *longprod = a * b;
+ longprod = a * b;
doubleprod = (double)a * (double)b;
- doubled_longprod = (double)*longprod;
+ doubled_longprod = (double)longprod;
/* Fast path for normal case: small multiplicands, and no info
is lost in either method. */
if (doubled_longprod == doubleprod)
- return 1;
+ return longprod;
/* Somebody somewhere lost info. Close enough, or way off? Note
that a != 0 and b != 0 (else doubled_longprod == doubleprod == 0).
@@ -247,15 +274,15 @@
/* absdiff/absprod <= 1/32 iff
32 * absdiff <= absprod -- 5 good bits is "close enough" */
if (32.0 * absdiff <= absprod)
- return 1;
- return 0;
+ return longprod;
+
+ FAIL_OVF("integer multiplication");
+ return -1;
}
}
#endif /* PYPY_NOT_MAIN_FILE */
-#endif /* !OP_INT_MUL_OVF_LL */
-
/* implementations */
#define OP_UINT_IS_TRUE OP_INT_IS_TRUE
Modified: pypy/branch/arm-backend/pypy/translator/c/src/main.h
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/src/main.h (original)
+++ pypy/branch/arm-backend/pypy/translator/c/src/main.h Tue Nov 23 10:12:47 2010
@@ -53,10 +53,16 @@
}
exitcode = STANDALONE_ENTRY_POINT(list);
+
+#ifdef RPY_ASSERT
+ pypy_debug_alloc_results();
+#endif
+
if (RPyExceptionOccurred()) {
/* print the RPython traceback */
pypy_debug_catch_fatal_exception();
}
+
return exitcode;
memory_out:
Modified: pypy/branch/arm-backend/pypy/translator/c/src/signals.h
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/src/signals.h (original)
+++ pypy/branch/arm-backend/pypy/translator/c/src/signals.h Tue Nov 23 10:12:47 2010
@@ -6,20 +6,6 @@
#include <limits.h>
-#ifndef LONG_MAX
-#if SIZEOF_LONG == 4
-#define LONG_MAX 0X7FFFFFFFL
-#elif SIZEOF_LONG == 8
-#define LONG_MAX 0X7FFFFFFFFFFFFFFFL
-#else
-#error "could not set LONG_MAX in pyport.h"
-#endif
-#endif
-
-#ifndef LONG_MIN
-#define LONG_MIN (-LONG_MAX-1)
-#endif
-
#include <stdlib.h>
#ifdef MS_WINDOWS
@@ -28,10 +14,6 @@
#include <signal.h>
-#ifndef SIG_ERR
-#define SIG_ERR ((PyOS_sighandler_t)(-1))
-#endif
-
#if defined(PYOS_OS2) && !defined(PYCC_GCC)
#define NSIG 12
#include <process.h>
@@ -65,16 +47,12 @@
/* utility to poll for signals that arrived */
int pypysig_poll(void); /* => signum or -1 */
-/* When a signal is received, the high bit of pypysig_occurred is set.
- After all signals are processed by pypysig_poll(), the high bit is
- cleared again. The variable is exposed and RPython code is free to
- use the other bits in any way. */
-#define PENDING_SIGNAL_BIT (LONG_MIN) /* high bit */
+/* When a signal is received, pypysig_counter is set to -1. */
/* This is a struct for the JIT. See interp_signal.py. */
struct pypysig_long_struct {
long value;
};
-extern struct pypysig_long_struct pypysig_occurred;
+extern struct pypysig_long_struct pypysig_counter;
/* some C tricks to get/set the variable as efficiently as possible:
use macros when compiling as a stand-alone program, but still
@@ -82,18 +60,20 @@
#undef pypysig_getaddr_occurred
void *pypysig_getaddr_occurred(void);
#ifndef PYPY_NOT_MAIN_FILE
-void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_occurred); }
+void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_counter); }
#endif
-#define pypysig_getaddr_occurred() ((void *)(&pypysig_occurred))
+#define pypysig_getaddr_occurred() ((void *)(&pypysig_counter))
/************************************************************/
/* Implementation */
#ifndef PYPY_NOT_MAIN_FILE
-struct pypysig_long_struct pypysig_occurred;
-static volatile long *pypysig_occurred_v = (volatile long *)&pypysig_occurred.value;
-static volatile int pypysig_flags[NSIG];
+struct pypysig_long_struct pypysig_counter = {0};
+static char volatile pypysig_flags[NSIG] = {0};
+static int volatile pypysig_occurred = 0;
+/* pypysig_occurred is only an optimization: it tells if any
+ pypysig_flags could be set. */
void pypysig_ignore(int signum)
{
@@ -126,10 +106,11 @@
static void signal_setflag_handler(int signum)
{
if (0 <= signum && signum < NSIG)
+ {
pypysig_flags[signum] = 1;
- /* the point of "*pypysig_occurred_v" instead of just "pypysig_occurred"
- is the volatile declaration */
- *pypysig_occurred_v |= PENDING_SIGNAL_BIT;
+ pypysig_occurred = 1;
+ pypysig_counter.value = -1;
+ }
}
void pypysig_setflag(int signum)
@@ -148,27 +129,21 @@
int pypysig_poll(void)
{
- /* the two commented out lines below are useful for performance in
- normal usage of pypysig_poll(); however, pypy/module/signal/ is
- not normal usage. It only calls pypysig_poll() if the
- PENDING_SIGNAL_BIT is set, and it clears that bit first. */
-
-/* if (pypysig_occurred & PENDING_SIGNAL_BIT) */
+ if (pypysig_occurred)
{
- int i;
-/* pypysig_occurred &= ~PENDING_SIGNAL_BIT; */
- for (i=0; i<NSIG; i++)
- if (pypysig_flags[i])
- {
- pypysig_flags[i] = 0;
- /* maybe another signal is pending: */
- pypysig_occurred.value |= PENDING_SIGNAL_BIT;
- return i;
- }
+ int i;
+ pypysig_occurred = 0;
+ for (i=0; i<NSIG; i++)
+ if (pypysig_flags[i])
+ {
+ pypysig_flags[i] = 0;
+ pypysig_occurred = 1; /* maybe another signal is pending */
+ return i;
+ }
}
- return -1; /* no pending signal */
+ return -1; /* no pending signal */
}
-#endif
+#endif /* !PYPY_NOT_MAIN_FILE */
#endif
Modified: pypy/branch/arm-backend/pypy/translator/c/src/stack.h
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/src/stack.h (original)
+++ pypy/branch/arm-backend/pypy/translator/c/src/stack.h Tue Nov 23 10:12:47 2010
@@ -33,6 +33,12 @@
&& LL_stack_too_big_slowpath());
}
+#ifdef __GNUC__
+# define PYPY_INHIBIT_TAIL_CALL() asm("/* inhibit_tail_call */")
+#else
+# define PYPY_INHIBIT_TAIL_CALL() /* add hints for other compilers here */
+#endif
+
#ifndef PYPY_NOT_MAIN_FILE
#include <stdio.h>
Modified: pypy/branch/arm-backend/pypy/translator/c/test/test_genc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/test/test_genc.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/test/test_genc.py Tue Nov 23 10:12:47 2010
@@ -426,6 +426,7 @@
if py.test.config.option.view:
t.view()
assert ' BarStruct ' in t.driver.cbuilder.c_source_filename.read()
+ free(foo, flavor="raw")
def test_recursive_llhelper():
from pypy.rpython.annlowlevel import llhelper
@@ -473,7 +474,27 @@
return f(s)
a_f = A(f, "f")
a_g = A(g, "g")
- t = lltype.malloc(STRUCT, flavor="raw")
+ t = lltype.malloc(STRUCT, flavor="raw", immortal=True)
t.bar = llhelper(FTPTR, a_f.make_func())
fn = compile(chooser, [bool])
assert fn(True)
+
+def test_inhibit_tail_call():
+ from pypy.rpython.lltypesystem import lltype
+ def foobar_fn(n):
+ return 42
+ foobar_fn._dont_inline_ = True
+ def main(n):
+ return foobar_fn(n)
+ #
+ t = Translation(main, [int], backend="c")
+ t.rtype()
+ t.context._graphof(foobar_fn).inhibit_tail_call = True
+ t.source_c()
+ lines = t.driver.cbuilder.c_source_filename.readlines()
+ for i, line in enumerate(lines):
+ if '= pypy_g_foobar_fn' in line:
+ break
+ else:
+ assert 0, "the call was not found in the C source"
+ assert 'PYPY_INHIBIT_TAIL_CALL();' in lines[i+1]
Modified: pypy/branch/arm-backend/pypy/translator/c/test/test_lltyped.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/test/test_lltyped.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/test/test_lltyped.py Tue Nov 23 10:12:47 2010
@@ -1,6 +1,7 @@
import py
from pypy.rpython.lltypesystem.lltype import *
from pypy.translator.c.test import test_typed
+from pypy.tool.sourcetools import func_with_new_name
class TestLowLevelType(test_typed.CompilationTestCase):
@@ -401,6 +402,7 @@
for i in range(n):
p = malloc(S, flavor='raw', zero=True)
if p.x != 0 or p.y != 0:
+ free(p, flavor='raw')
return -1
p.x = i
p.y = i
@@ -418,14 +420,16 @@
def f(n):
for length in range(n-1, -1, -1):
p = malloc(S, length, flavor='raw', zero=True)
- if p.x != 0:
- return -1
- p.x = n
- for j in range(length):
- if p.y[j] != 0:
- return -3
- p.y[j] = n^j
- free(p, flavor='raw')
+ try:
+ if p.x != 0:
+ return -1
+ p.x = n
+ for j in range(length):
+ if p.y[j] != 0:
+ return -3
+ p.y[j] = n^j
+ finally:
+ free(p, flavor='raw')
return 42
fn = self.getcompiled(f, [int])
@@ -652,10 +656,49 @@
fn = self.getcompiled(llf)
fn()
+ def test_prebuilt_raw_arrays(self):
+ from pypy.rpython.lltypesystem import rffi, ll2ctypes
+ #
+ def make_test_function(cast, haslength, length):
+ a = malloc(A, length, flavor='raw', immortal=True)
+ # two cases: a zero-terminated array if length == 6 or 1030,
+ # a non-zero-terminated array if length == 557 or 1031
+ for i in range(length):
+ a[i] = cast(256 - 5 + i)
+ def llf():
+ for i in range(length):
+ if a[i] != cast(256 - 5 + i):
+ return False
+ if haslength and len(a) != length:
+ return False
+ return True
+ return func_with_new_name(llf, repr((A, haslength, length)))
+ #
+ testfns = []
+ records = []
+ for OF, cast in [(Void, lambda n: None),
+ (Char, lambda n: chr(n & 0xFF)),
+ (Signed, lambda n: n)]:
+ for A, haslength in [(rffi.CArray(OF), False),
+ (Array(OF), True)]:
+ for length in [0, 6, 557, 1030, 1031]:
+ testfns.append(make_test_function(cast, haslength, length))
+ records.append((A, haslength, length))
+ def llf():
+ i = 0
+ for fn in testfns:
+ if not fn():
+ return i # returns the index of the failing function
+ i += 1
+ return -42
+ fn = self.getcompiled(llf)
+ res = fn()
+ assert res == -42, "failing function: %r" % (records[res],)
+
def test_prebuilt_ll2ctypes_array(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
@@ -676,7 +719,7 @@
def test_ll2ctypes_array_from_c(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
@@ -838,3 +881,17 @@
assert res == -98765432
res = fn(1)
assert res == -9999999
+
+ def test_render_immortal(self):
+ A = FixedSizeArray(Signed, 1)
+ a1 = malloc(A, flavor='raw')
+ render_immortal(a1)
+ a1[0] = 42
+ def llf():
+ a2 = malloc(A, flavor='raw')
+ render_immortal(a2)
+ a2[0] = 3
+ return a1[0] + a2[0]
+ fn = self.getcompiled(llf)
+ assert fn() == 45
+
Modified: pypy/branch/arm-backend/pypy/translator/c/test/test_newgc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/test/test_newgc.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/test/test_newgc.py Tue Nov 23 10:12:47 2010
@@ -624,13 +624,13 @@
os.unlink(self.filename)
def define_callback_with_collect(cls):
- from pypy.rlib.libffi import ffi_type_pointer, cast_type_to_ffitype,\
+ from pypy.rlib.clibffi import ffi_type_pointer, cast_type_to_ffitype,\
CDLL, ffi_type_void, CallbackFuncPtr, ffi_type_sint
from pypy.rpython.lltypesystem import rffi, ll2ctypes
import gc
ffi_size_t = cast_type_to_ffitype(rffi.SIZE_T)
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
def callback(ll_args, ll_res, stuff):
gc.collect()
@@ -1066,7 +1066,9 @@
filename_dump = str(udir.join('test_dump_rpy_heap'))
def define_dump_rpy_heap(self):
- U = lltype.GcStruct('U', ('x', lltype.Signed))
+ U = lltype.GcForwardReference()
+ U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)),
+ ('x', lltype.Signed)))
S = lltype.GcStruct('S', ('u', lltype.Ptr(U)))
A = lltype.GcArray(lltype.Ptr(S))
filename = self.filename_dump
@@ -1074,11 +1076,16 @@
def fn():
s = lltype.malloc(S)
s.u = lltype.malloc(U)
+ s.u.next = lltype.malloc(U)
+ s.u.next.next = lltype.malloc(U)
a = lltype.malloc(A, 1000)
s2 = lltype.malloc(S)
#
fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666)
rgc.dump_rpy_heap(fd)
+ keepalive_until_here(s2)
+ keepalive_until_here(s)
+ keepalive_until_here(a)
os.close(fd)
return 0
@@ -1087,8 +1094,43 @@
def test_dump_rpy_heap(self):
self.run("dump_rpy_heap")
assert os.path.exists(self.filename_dump)
- assert os.path.getsize(self.filename_dump) > 0 # minimal test
+ assert os.path.getsize(self.filename_dump) > 64
+
+ filename_dump_typeids_z = str(udir.join('test_typeids_z'))
+ def define_write_typeids_z(self):
+ U = lltype.GcForwardReference()
+ U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)),
+ ('x', lltype.Signed)))
+ S = lltype.GcStruct('S', ('u', lltype.Ptr(U)))
+ A = lltype.GcArray(lltype.Ptr(S))
+ filename = self.filename_dump_typeids_z
+
+ def fn():
+ s = lltype.malloc(S)
+ s.u = lltype.malloc(U)
+ s.u.next = lltype.malloc(U)
+ s.u.next.next = lltype.malloc(U)
+ a = lltype.malloc(A, 1000)
+ s2 = lltype.malloc(S)
+ #
+ p = rgc.get_typeids_z()
+ s = ''.join([p[i] for i in range(len(p))])
+ fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666)
+ os.write(fd, s)
+ os.close(fd)
+ return 0
+
+ return fn
+ def test_write_typeids_z(self):
+ self.run("write_typeids_z")
+ f = open(self.filename_dump_typeids_z)
+ data_z = f.read()
+ f.close()
+ import zlib
+ data = zlib.decompress(data_z)
+ assert data.startswith('member0')
+ assert 'GcArray of * GcStruct S {' in data
class TestSemiSpaceGC(TestUsingFramework, snippet.SemiSpaceGCTestDefines):
gcpolicy = "semispace"
@@ -1173,21 +1215,22 @@
b = 0
c = 0
for i in range(len(tb)):
- if tb[i].count == 10:
+ if tb[i].count == 10: # the type of S
a += 1
nr = i
for i in range(len(tb)):
- if tb[i].count == 3:
+ if tb[i].count == 3: # the type GcArray(Ptr(S))
b += 1
c += tb[i].links[nr]
- # we don't count b here since there can be more singletons,
+ # b can be 1 or 2 here since _heap_stats() is free to return or
+ # ignore the three GcStructs that point to the GcArray(Ptr(S)).
# important one is c, a is for check
return c * 100 + b * 10 + a
return f
def test_gc_heap_stats(self):
res = self.run("gc_heap_stats")
- assert res == 3011
+ assert res == 3011 or res == 3021
def definestr_string_builder(cls):
def fn(_):
Modified: pypy/branch/arm-backend/pypy/translator/c/test/test_standalone.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/test/test_standalone.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/test/test_standalone.py Tue Nov 23 10:12:47 2010
@@ -16,11 +16,16 @@
class StandaloneTests(object):
config = None
- def compile(self, entry_point, debug=True, shared=False):
+ def compile(self, entry_point, debug=True, shared=False,
+ stackcheck=False):
t = TranslationContext(self.config)
t.buildannotator().build_types(entry_point, [s_list_of_strings])
t.buildrtyper().specialize()
+ if stackcheck:
+ from pypy.translator.transform import insert_ll_stackcheck
+ insert_ll_stackcheck(t)
+
t.config.translation.shared = shared
cbuilder = CStandaloneBuilder(t, entry_point, t.config)
@@ -363,12 +368,27 @@
assert not err
assert path.check(file=1)
data = path.read()
- assert 'toplevel' in path.read()
- assert 'mycat' not in path.read()
- assert 'foo 2 bar 3' not in path.read()
+ assert 'toplevel' in data
+ assert 'mycat' not in data
+ assert 'foo 2 bar 3' not in data
assert 'cat2' in data
assert 'baz' in data
assert 'bok' not in data
+ # check with PYPYLOG=myc,cat2:somefilename (includes mycat and cat2)
+ path = udir.join('test_debug_xxx_myc_cat2.log')
+ out, err = cbuilder.cmdexec("", err=True,
+ env={'PYPYLOG': 'myc,cat2:%s' % path})
+ assert out.strip() == 'got:bcda.'
+ assert not err
+ assert path.check(file=1)
+ data = path.read()
+ assert 'toplevel' in data
+ assert '{mycat' in data
+ assert 'mycat}' in data
+ assert 'foo 2 bar 3' in data
+ assert 'cat2' in data
+ assert 'baz' in data
+ assert 'bok' in data
#
# finally, check compiling with logging disabled
from pypy.config.pypyoption import get_pypy_config
@@ -630,6 +650,22 @@
else:
os.environ['CC'] = old_cc
+ def test_inhibit_tail_call(self):
+ # the point is to check that the f()->f() recursion stops
+ from pypy.rlib.rstackovf import StackOverflow
+ def f(n):
+ if n <= 0:
+ return 42
+ return f(n+1)
+ def entry_point(argv):
+ try:
+ return f(1)
+ except StackOverflow:
+ print 'hi!'
+ return 0
+ t, cbuilder = self.compile(entry_point, stackcheck=True)
+ out = cbuilder.cmdexec("")
+ assert out.strip() == "hi!"
class TestMaemo(TestStandalone):
def setup_class(cls):
Modified: pypy/branch/arm-backend/pypy/translator/c/test/test_typed.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/c/test/test_typed.py (original)
+++ pypy/branch/arm-backend/pypy/translator/c/test/test_typed.py Tue Nov 23 10:12:47 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import autopath
import sys
import math
@@ -823,3 +824,41 @@
while int(x + frac) >= -sys.maxint-1:
x -= 1
assert f(x + frac) == -666
+
+ def test_context_manager(self):
+ state = []
+ class C:
+ def __init__(self, name):
+ self.name = name
+ def __enter__(self):
+ state.append('acquire')
+ return self
+ def __exit__(self, typ, value, tb):
+ if typ is not None:
+ if value is None:
+ raise RuntimeError('test failed')
+ state.append('raised')
+ else:
+ if value is not None:
+ raise RuntimeError('test failed')
+ state.append('release')
+
+ def func(n):
+ del state[:]
+ try:
+ with C('hello') as c:
+ state.append(c.name)
+ if n == 1:
+ raise ValueError
+ elif n == 2:
+ raise TypeError
+ except (ValueError, TypeError):
+ pass
+ return ', '.join(state)
+ f = self.getcompiled(func, [int])
+ res = f(0)
+ assert res == 'acquire, hello, release'
+ res = f(1)
+ assert res == 'acquire, hello, raised, release'
+ res = f(2)
+ assert res == 'acquire, hello, raised, release'
Modified: pypy/branch/arm-backend/pypy/translator/cli/src/pypylib.cs
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/cli/src/pypylib.cs (original)
+++ pypy/branch/arm-backend/pypy/translator/cli/src/pypylib.cs Tue Nov 23 10:12:47 2010
@@ -83,8 +83,12 @@
return Double.NegativeInfinity;
else if (s == "nan")
return Double.NaN;
- else
- return System.Convert.ToDouble(s);
+ else {
+ System.Globalization.NumberFormatInfo formatter;
+ formatter = new System.Globalization.NumberFormatInfo();
+ formatter.NumberDecimalSeparator = ".";
+ return System.Convert.ToDouble(s, formatter);
+ }
}
}
Modified: pypy/branch/arm-backend/pypy/translator/driver.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/driver.py (original)
+++ pypy/branch/arm-backend/pypy/translator/driver.py Tue Nov 23 10:12:47 2010
@@ -426,6 +426,22 @@
[OOTYPE],
"JIT compiler generation")
+ def task_jittest_lltype(self):
+ """ Run with the JIT on top of the llgraph backend
+ """
+ # parent process loop: spawn a child, wait for the child to finish,
+ # print a message, and restart
+ from pypy.translator.goal import unixcheckpoint
+ unixcheckpoint.restartable_point(auto='run')
+ # load the module pypy/jit/tl/jittest.py, which you can hack at
+ # and restart without needing to restart the whole translation process
+ from pypy.jit.tl import jittest
+ jittest.jittest(self)
+ #
+ task_jittest_lltype = taskdef(task_jittest_lltype,
+ [RTYPE],
+ "test of the JIT on the llgraph backend")
+
def task_backendopt_lltype(self):
""" Run all backend optimizations - lltype version
"""
@@ -433,7 +449,8 @@
backend_optimizations(self.translator)
#
task_backendopt_lltype = taskdef(task_backendopt_lltype,
- [RTYPE, '??pyjitpl_lltype'],
+ [RTYPE, '??pyjitpl_lltype',
+ '??jittest_lltype'],
"lltype back-end optimisations")
BACKENDOPT = 'backendopt_lltype'
Modified: pypy/branch/arm-backend/pypy/translator/goal/ann_override.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/goal/ann_override.py (original)
+++ pypy/branch/arm-backend/pypy/translator/goal/ann_override.py Tue Nov 23 10:12:47 2010
@@ -27,21 +27,6 @@
pol.pypytypes = {}
pol.single_space = single_space
- #def override__wrap_exception_cls(pol, space, x):
- # import pypy.objspace.std.typeobject as typeobject
- # clsdef = getbookkeeper().getuniqueclassdef(typeobject.W_TypeObject)
- # return annmodel.SomeInstance(clsdef, can_be_None=True)
- #
- #def override__fake_object(pol, space, x):
- # from pypy.interpreter import typedef
- # clsdef = getbookkeeper().getuniqueclassdef(typedef.W_Root)
- # return annmodel.SomeInstance(clsdef)
- #
- #def override__cpy_compile(pol, self, source, filename, mode, flags):
- # from pypy.interpreter import pycode
- # clsdef = getbookkeeper().getuniqueclassdef(pycode.PyCode)
- # return annmodel.SomeInstance(clsdef)
-
def specialize__wrap(pol, funcdesc, args_s):
from pypy.interpreter.baseobjspace import Wrappable
from pypy.annotation.classdef import ClassDef
Modified: pypy/branch/arm-backend/pypy/translator/goal/app_main.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/goal/app_main.py (original)
+++ pypy/branch/arm-backend/pypy/translator/goal/app_main.py Tue Nov 23 10:12:47 2010
@@ -255,7 +255,7 @@
break
elif arg == '-u':
unbuffered = True
- elif arg == '-O':
+ elif arg == '-O' or arg == '-OO':
pass
elif arg == '--version' or arg == '-V':
print "Python", sys.version
@@ -326,10 +326,6 @@
except:
print >> sys.stderr, "'import site' failed"
- # update sys.path *after* loading site.py, in case there is a
- # "site.py" file in the script's directory.
- sys.path.insert(0, '')
-
if warnoptions:
sys.warnoptions.append(warnoptions)
from warnings import _processoptions
@@ -366,6 +362,9 @@
try:
if run_command:
# handle the "-c" command
+ # Put '' on sys.path
+ sys.path.insert(0, '')
+
def run_it():
exec cmd in mainmodule.__dict__
success = run_toplevel(run_it)
@@ -378,6 +377,13 @@
elif run_stdin:
# handle the case where no command/filename/module is specified
# on the command-line.
+
+ # update sys.path *after* loading site.py, in case there is a
+ # "site.py" file in the script's directory. Only run this if we're
+ # executing the interactive prompt, if we're running a script we
+ # put it's directory on sys.path
+ sys.path.insert(0, '')
+
if go_interactive or sys.stdin.isatty():
# If stdin is a tty or if "-i" is specified, we print
# a banner and run $PYTHONSTARTUP.
@@ -385,9 +391,12 @@
python_startup = os.getenv('PYTHONSTARTUP')
if python_startup:
try:
- startup = open(python_startup).read()
- except IOError:
- pass
+ f = open(python_startup)
+ startup = f.read()
+ f.close()
+ except IOError, e:
+ print >> sys.stderr, "Could not open PYTHONSTARTUP"
+ print >> sys.stderr, "IOError:", e
else:
def run_it():
co_python_startup = compile(startup,
Modified: pypy/branch/arm-backend/pypy/translator/goal/targetpypystandalone.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/goal/targetpypystandalone.py (original)
+++ pypy/branch/arm-backend/pypy/translator/goal/targetpypystandalone.py Tue Nov 23 10:12:47 2010
@@ -28,9 +28,14 @@
w_call_finish_gateway = space.wrap(gateway.interp2app(call_finish))
w_call_startup_gateway = space.wrap(gateway.interp2app(call_startup))
w_os = setup_nanos(space)
+ withjit = space.config.objspace.usemodules.pypyjit
def entry_point(argv):
space.timer.start("Entrypoint")
+ if withjit:
+ from pypy.jit.backend.hlinfo import highleveljitinfo
+ highleveljitinfo.sys_executable = argv[0]
+
#debug("entry point starting")
#for arg in argv:
# debug(" argv -> " + arg)
@@ -150,6 +155,9 @@
if config.objspace.allworkingmodules:
from pypy.config.pypyoption import enable_allworkingmodules
enable_allworkingmodules(config)
+ if config.objspace.translationmodules:
+ from pypy.config.pypyoption import enable_translationmodules
+ enable_translationmodules(config)
if config.translation.type_system == 'ootype':
config.objspace.usemodules.suggest(rbench=True)
Modified: pypy/branch/arm-backend/pypy/translator/goal/test2/test_app_main.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/goal/test2/test_app_main.py (original)
+++ pypy/branch/arm-backend/pypy/translator/goal/test2/test_app_main.py Tue Nov 23 10:12:47 2010
@@ -95,6 +95,11 @@
child.expect('>>> ')
child.sendline('__name__')
child.expect("'__main__'")
+ child.expect('>>> ')
+ child.sendline('import sys')
+ child.expect('>>> ')
+ child.sendline("'' in sys.path")
+ child.expect("True")
def test_run_script(self):
child = self.spawn([demo_script])
@@ -463,8 +468,10 @@
yield
finally:
old_cwd.chdir()
- os.putenv('PYTHONPATH', old_pythonpath)
-
+ # Can't call putenv with a None argument.
+ if old_pythonpath is not None:
+ os.putenv('PYTHONPATH', old_pythonpath)
+
tmpdir.join('site.py').write('print "SHOULD NOT RUN"')
runme_py = tmpdir.join('runme.py')
runme_py.write('print "some text"')
@@ -485,9 +492,12 @@
with chdir_and_unset_pythonpath(tmpdir):
data = self.run(cmdline2, python_flags='-S')
-
assert data.startswith("some new text\n")
assert repr(str(tmpdir.join('otherpath'))) in data
+ assert "''" not in data
+
+ data = self.run('-c "import sys; print sys.path"')
+ assert data.startswith("[''")
class AppTestAppMain:
@@ -524,7 +534,8 @@
newpath = app_main.get_library_path('/tmp/pypy-c') # stdlib not found
assert newpath == sys.path
newpath = app_main.get_library_path(self.fake_exe)
- assert newpath == self.expected_path
+ # we get at least 'expected_path', and maybe more (e.g.plat-linux2)
+ assert newpath[:len(self.expected_path)] == self.expected_path
finally:
sys.path.pop()
@@ -537,7 +548,9 @@
app_main.os = os
pypy_c = os.path.join(self.trunkdir, 'pypy', 'translator', 'goal', 'pypy-c')
newpath = app_main.get_library_path(pypy_c)
- assert len(newpath) == 3
+ # we get at least lib_pypy, lib-python/modified-X.Y.Z,
+ # lib-python/X.Y.Z, and maybe more (e.g. plat-linux2)
+ assert len(newpath) >= 3
for p in newpath:
assert p.startswith(self.trunkdir)
finally:
Modified: pypy/branch/arm-backend/pypy/translator/goal/translate.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/goal/translate.py (original)
+++ pypy/branch/arm-backend/pypy/translator/goal/translate.py Tue Nov 23 10:12:47 2010
@@ -27,6 +27,7 @@
("annotate", "do type inference", "-a --annotate", ""),
("rtype", "do rtyping", "-t --rtype", ""),
("pyjitpl", "JIT generation step", "--pyjitpl", ""),
+ ("jittest", "JIT test with llgraph backend", "--jittest", ""),
("backendopt", "do backend optimizations", "--backendopt", ""),
("source", "create source", "-s --source", ""),
("compile", "compile", "-c --compile", " (default goal)"),
Modified: pypy/branch/arm-backend/pypy/translator/jvm/test/test_class.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/jvm/test/test_class.py (original)
+++ pypy/branch/arm-backend/pypy/translator/jvm/test/test_class.py Tue Nov 23 10:12:47 2010
@@ -1,6 +1,6 @@
import py
from pypy.translator.jvm.test.runtest import JvmTest
-from pypy.translator.oosupport.test_template.class_ import BaseTestClass, BaseTestSpecialcase
+from pypy.translator.oosupport.test_template.class_ import BaseTestClass
class TestJvmClass(JvmTest, BaseTestClass):
def test_overridden_classattr_as_defaults(self):
@@ -26,6 +26,3 @@
def test_specialize_methods(self):
py.test.skip('ABSTRACT METHOD FIX: RE-TEST AFTER MERGE')
-
-class TestJvmSpecialCase(JvmTest, BaseTestSpecialcase):
- pass
Modified: pypy/branch/arm-backend/pypy/translator/oosupport/test_template/class_.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/oosupport/test_template/class_.py (original)
+++ pypy/branch/arm-backend/pypy/translator/oosupport/test_template/class_.py Tue Nov 23 10:12:47 2010
@@ -1,6 +1,5 @@
import py
from pypy.rpython.test import test_rclass
-from pypy.rpython.test.test_rspecialcase import BaseTestRspecialcase
class BaseTestClass(test_rclass.TestOOtype):
def test_abstract_method(self):
@@ -66,6 +65,3 @@
def test_cast_object_mix_null(self):
py.test.skip('cannot return ootype.NULL from translated functions')
-
-class BaseTestSpecialcase(BaseTestRspecialcase):
- pass
Modified: pypy/branch/arm-backend/pypy/translator/platform/darwin.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/platform/darwin.py (original)
+++ pypy/branch/arm-backend/pypy/translator/platform/darwin.py Tue Nov 23 10:12:47 2010
@@ -14,7 +14,10 @@
def __init__(self, cc=None):
if cc is None:
- cc = 'gcc'
+ try:
+ cc = os.environ['CC']
+ except KeyError:
+ cc = 'gcc'
self.cc = cc
def _args_for_shared(self, args):
Modified: pypy/branch/arm-backend/pypy/translator/platform/linux.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/platform/linux.py (original)
+++ pypy/branch/arm-backend/pypy/translator/platform/linux.py Tue Nov 23 10:12:47 2010
@@ -27,6 +27,8 @@
class Linux(BaseLinux):
+ shared_only = () # it seems that on 32-bit linux, compiling with -fPIC
+ # gives assembler that asmgcc is not happy about.
def library_dirs_for_libffi_a(self):
# places where we need to look for libffi.a
return self.library_dirs_for_libffi() + ['/usr/lib']
Modified: pypy/branch/arm-backend/pypy/translator/platform/posix.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/platform/posix.py (original)
+++ pypy/branch/arm-backend/pypy/translator/platform/posix.py Tue Nov 23 10:12:47 2010
@@ -104,12 +104,10 @@
else:
target_name = exe_name.basename
- cflags = self.cflags
- if sys.maxint > 2147483647: # XXX XXX XXX sort this out
- if shared:
- cflags = self.cflags + self.shared_only
- else:
- cflags = self.cflags + self.standalone_only
+ if shared:
+ cflags = self.cflags + self.shared_only
+ else:
+ cflags = self.cflags + self.standalone_only
m = GnuMakefile(path)
m.exe_name = exe_name
Modified: pypy/branch/arm-backend/pypy/translator/transform.py
==============================================================================
--- pypy/branch/arm-backend/pypy/translator/transform.py (original)
+++ pypy/branch/arm-backend/pypy/translator/transform.py Tue Nov 23 10:12:47 2010
@@ -221,15 +221,19 @@
stack_check_ptr_const = Constant(stack_check_ptr, lltype.typeOf(stack_check_ptr))
edges = set()
insert_in = set()
+ block2graph = {}
for caller in translator.graphs:
for block, callee in find_calls_from(translator, caller):
if getattr(getattr(callee, 'func', None),
'insert_stack_check_here', False):
insert_in.add(callee.startblock)
+ block2graph[callee.startblock] = callee
continue
if block is not caller.startblock:
edges.add((caller.startblock, block))
+ block2graph[caller.startblock] = caller
edges.add((block, callee.startblock))
+ block2graph[block] = caller
edgelist = [Edge(block1, block2) for (block1, block2) in edges]
edgedict = make_edge_dict(edgelist)
@@ -241,6 +245,10 @@
v.concretetype = lltype.Void
unwind_op = SpaceOperation('direct_call', [stack_check_ptr_const], v)
block.operations.insert(0, unwind_op)
+ # prevents cycles of tail calls from occurring -- such cycles would
+ # not consume any stack, so would turn into potentially infinite loops
+ graph = block2graph[block]
+ graph.inhibit_tail_call = True
return len(insert_in)
More information about the Pypy-commit
mailing list