[pypy-svn] r78228 - in pypy/branch/fast-forward: dotviewer lib-python lib_pypy/_ctypes pypy pypy/annotation pypy/annotation/test pypy/config pypy/doc pypy/doc/config pypy/interpreter pypy/interpreter/pyparser pypy/jit/backend/llgraph pypy/jit/backend/llsupport pypy/jit/backend/llsupport/test pypy/jit/backend/test pypy/jit/backend/x86 pypy/jit/backend/x86/test pypy/jit/codewriter pypy/jit/codewriter/test pypy/jit/metainterp pypy/jit/metainterp/optimizeopt pypy/jit/metainterp/test pypy/jit/tl pypy/jit/tool pypy/module/__builtin__ pypy/module/__builtin__/test pypy/module/_ffi pypy/module/_ffi/test pypy/module/_rawffi pypy/module/_rawffi/test pypy/module/_socket/test pypy/module/_sre pypy/module/_weakref pypy/module/_winreg pypy/module/array pypy/module/bz2 pypy/module/cpyext pypy/module/cpyext/test pypy/module/gc pypy/module/imp/test pypy/module/pypyjit pypy/module/pypyjit/test pypy/module/signal pypy/module/sys pypy/module/test_lib_pypy/ctypes_tests pypy/module/thread pypy/objspace/flow pypy/objspace/flow/test pypy/objspace/std pypy/objspace/std/test pypy/rlib pypy/rlib/rsre pypy/rlib/rsre/test pypy/rlib/test pypy/rpython pypy/rpython/lltypesystem pypy/rpython/lltypesystem/test pypy/rpython/memory pypy/rpython/memory/gc pypy/rpython/memory/gctransform pypy/rpython/memory/test pypy/rpython/module pypy/rpython/test pypy/tool pypy/tool/test pypy/translator pypy/translator/c pypy/translator/c/gcc pypy/translator/c/gcc/test/elf pypy/translator/c/gcc/test/elf64 pypy/translator/c/src pypy/translator/c/test pypy/translator/goal pypy/translator/jvm/test pypy/translator/oosupport/test_template pypy/translator/platform
afa at codespeak.net
afa at codespeak.net
Fri Oct 22 23:09:54 CEST 2010
Author: afa
Date: Fri Oct 22 23:09:43 2010
New Revision: 78228
Added:
pypy/branch/fast-forward/pypy/doc/config/objspace.std.withmapdict.txt
- copied unchanged from r78227, pypy/trunk/pypy/doc/config/objspace.std.withmapdict.txt
pypy/branch/fast-forward/pypy/doc/config/objspace.std.withstrbuf.txt
- copied unchanged from r78227, pypy/trunk/pypy/doc/config/objspace.std.withstrbuf.txt
pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._ffi.txt
- copied unchanged from r78227, pypy/trunk/pypy/doc/config/objspace.usemodules._ffi.txt
pypy/branch/fast-forward/pypy/jit/backend/llsupport/ffisupport.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/backend/llsupport/ffisupport.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_ffisupport.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/backend/llsupport/test/test_ffisupport.py
pypy/branch/fast-forward/pypy/jit/metainterp/greenfield.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/metainterp/greenfield.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/fficall.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_fficall.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/metainterp/test/test_fficall.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_greenfield.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/metainterp/test/test_greenfield.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizefficall.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/metainterp/test/test_optimizefficall.py
pypy/branch/fast-forward/pypy/jit/tool/oparser.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/tool/oparser.py
pypy/branch/fast-forward/pypy/module/_ffi/
- copied from r78227, pypy/trunk/pypy/module/_ffi/
pypy/branch/fast-forward/pypy/module/_ffi/__init__.py
- copied unchanged from r78227, pypy/trunk/pypy/module/_ffi/__init__.py
pypy/branch/fast-forward/pypy/module/_ffi/interp_ffi.py
- copied unchanged from r78227, pypy/trunk/pypy/module/_ffi/interp_ffi.py
pypy/branch/fast-forward/pypy/module/_ffi/test/
- copied from r78227, pypy/trunk/pypy/module/_ffi/test/
pypy/branch/fast-forward/pypy/module/_ffi/test/__init__.py
- copied unchanged from r78227, pypy/trunk/pypy/module/_ffi/test/__init__.py
pypy/branch/fast-forward/pypy/module/_ffi/test/test__ffi.py
- copied unchanged from r78227, pypy/trunk/pypy/module/_ffi/test/test__ffi.py
pypy/branch/fast-forward/pypy/objspace/std/mapdict.py
- copied unchanged from r78227, pypy/trunk/pypy/objspace/std/mapdict.py
pypy/branch/fast-forward/pypy/objspace/std/strbufobject.py
- copied unchanged from r78227, pypy/trunk/pypy/objspace/std/strbufobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py
- copied unchanged from r78227, pypy/trunk/pypy/objspace/std/test/test_mapdict.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_strbufobject.py
- copied unchanged from r78227, pypy/trunk/pypy/objspace/std/test/test_strbufobject.py
pypy/branch/fast-forward/pypy/rlib/clibffi.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/clibffi.py
pypy/branch/fast-forward/pypy/rlib/libffi.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/libffi.py
pypy/branch/fast-forward/pypy/rlib/rerased.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/rerased.py
pypy/branch/fast-forward/pypy/rlib/rsre/rsre_jit.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/rsre/rsre_jit.py
pypy/branch/fast-forward/pypy/rlib/rsre/test/conftest.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/rsre/test/conftest.py
pypy/branch/fast-forward/pypy/rlib/rsre/test/test_zjit.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/rsre/test/test_zjit.py
pypy/branch/fast-forward/pypy/rlib/test/test_clibffi.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/test/test_clibffi.py
pypy/branch/fast-forward/pypy/rlib/test/test_libffi.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/test/test_libffi.py
pypy/branch/fast-forward/pypy/rlib/test/test_rerased.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/test/test_rerased.py
pypy/branch/fast-forward/pypy/tool/leakfinder.py
- copied unchanged from r78227, pypy/trunk/pypy/tool/leakfinder.py
pypy/branch/fast-forward/pypy/tool/test/test_leakfinder.py
- copied unchanged from r78227, pypy/trunk/pypy/tool/test/test_leakfinder.py
pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track10.s
- copied unchanged from r78227, pypy/trunk/pypy/translator/c/gcc/test/elf/track10.s
pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track11.s
- copied unchanged from r78227, pypy/trunk/pypy/translator/c/gcc/test/elf/track11.s
pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf64/track_loadconst.s
- copied unchanged from r78227, pypy/trunk/pypy/translator/c/gcc/test/elf64/track_loadconst.s
pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf64/track_rpyassertfailed.s
- copied unchanged from r78227, pypy/trunk/pypy/translator/c/gcc/test/elf64/track_rpyassertfailed.s
pypy/branch/fast-forward/pypy/translator/c/src/debug_alloc.h
- copied unchanged from r78227, pypy/trunk/pypy/translator/c/src/debug_alloc.h
Removed:
pypy/branch/fast-forward/pypy/jit/metainterp/test/oparser.py
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_find.py
pypy/branch/fast-forward/pypy/rpython/rspecialcase.py
pypy/branch/fast-forward/pypy/rpython/test/test_rspecialcase.py
Modified:
pypy/branch/fast-forward/dotviewer/drawgraph.py
pypy/branch/fast-forward/lib-python/conftest.py
pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py
pypy/branch/fast-forward/lib_pypy/_ctypes/union.py
pypy/branch/fast-forward/pypy/ (props changed)
pypy/branch/fast-forward/pypy/annotation/builtin.py
pypy/branch/fast-forward/pypy/annotation/model.py
pypy/branch/fast-forward/pypy/annotation/policy.py
pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py
pypy/branch/fast-forward/pypy/annotation/test/test_model.py
pypy/branch/fast-forward/pypy/config/pypyoption.py
pypy/branch/fast-forward/pypy/conftest.py
pypy/branch/fast-forward/pypy/doc/docindex.txt
pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py
pypy/branch/fast-forward/pypy/interpreter/pycode.py
pypy/branch/fast-forward/pypy/interpreter/pyopcode.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
pypy/branch/fast-forward/pypy/interpreter/typedef.py
pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py
pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/llmodel.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/symbolic.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py
pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py
pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py
pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py
pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py
pypy/branch/fast-forward/pypy/jit/backend/x86/regloc.py
pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py
pypy/branch/fast-forward/pypy/jit/backend/x86/rx86.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_assembler.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_gc_integration.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_string.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py
pypy/branch/fast-forward/pypy/jit/codewriter/call.py
pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py
pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py
pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py
pypy/branch/fast-forward/pypy/jit/codewriter/support.py
pypy/branch/fast-forward/pypy/jit/codewriter/test/test_codewriter.py
pypy/branch/fast-forward/pypy/jit/codewriter/test/test_effectinfo.py
pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py
pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py
pypy/branch/fast-forward/pypy/jit/metainterp/compile.py
pypy/branch/fast-forward/pypy/jit/metainterp/executor.py
pypy/branch/fast-forward/pypy/jit/metainterp/graphpage.py
pypy/branch/fast-forward/pypy/jit/metainterp/history.py
pypy/branch/fast-forward/pypy/jit/metainterp/jitdriver.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimize_nopspec.py (contents, props changed)
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/intbounds.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py
pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py
pypy/branch/fast-forward/pypy/jit/metainterp/resume.py
pypy/branch/fast-forward/pypy/jit/metainterp/simple_optimize.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_jitdriver.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_loop_nopspec.py (props changed)
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizefindnode.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_string.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py
pypy/branch/fast-forward/pypy/jit/metainterp/virtualref.py
pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py
pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py
pypy/branch/fast-forward/pypy/jit/tl/jittest.py (props changed)
pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py
pypy/branch/fast-forward/pypy/jit/tl/pypyjit_child.py
pypy/branch/fast-forward/pypy/jit/tool/loopviewer.py
pypy/branch/fast-forward/pypy/jit/tool/showstats.py
pypy/branch/fast-forward/pypy/jit/tool/traceviewer.py
pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py
pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py
pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py
pypy/branch/fast-forward/pypy/module/_rawffi/array.py
pypy/branch/fast-forward/pypy/module/_rawffi/callback.py
pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py
pypy/branch/fast-forward/pypy/module/_rawffi/structure.py
pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py
pypy/branch/fast-forward/pypy/module/_socket/test/test_sock_app.py
pypy/branch/fast-forward/pypy/module/_sre/interp_sre.py
pypy/branch/fast-forward/pypy/module/_weakref/interp__weakref.py
pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py
pypy/branch/fast-forward/pypy/module/array/interp_array.py
pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py
pypy/branch/fast-forward/pypy/module/cpyext/classobject.py
pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py
pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py
pypy/branch/fast-forward/pypy/module/gc/referents.py
pypy/branch/fast-forward/pypy/module/imp/test/test_import.py
pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py
pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py
pypy/branch/fast-forward/pypy/module/pypyjit/policy.py
pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py
pypy/branch/fast-forward/pypy/module/signal/interp_signal.py
pypy/branch/fast-forward/pypy/module/sys/__init__.py
pypy/branch/fast-forward/pypy/module/sys/state.py
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py
pypy/branch/fast-forward/pypy/module/thread/ll_thread.py
pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py
pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py
pypy/branch/fast-forward/pypy/objspace/std/celldict.py
pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py
pypy/branch/fast-forward/pypy/objspace/std/fake.py
pypy/branch/fast-forward/pypy/objspace/std/model.py
pypy/branch/fast-forward/pypy/objspace/std/objspace.py
pypy/branch/fast-forward/pypy/objspace/std/sharingdict.py
pypy/branch/fast-forward/pypy/objspace/std/stringobject.py
pypy/branch/fast-forward/pypy/objspace/std/stringtype.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py
pypy/branch/fast-forward/pypy/objspace/std/typeobject.py
pypy/branch/fast-forward/pypy/rlib/_rsocket_rffi.py
pypy/branch/fast-forward/pypy/rlib/jit.py
pypy/branch/fast-forward/pypy/rlib/rgc.py
pypy/branch/fast-forward/pypy/rlib/rmmap.py
pypy/branch/fast-forward/pypy/rlib/rsre/rsre_char.py
pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py
pypy/branch/fast-forward/pypy/rlib/rsre/test/test_match.py
pypy/branch/fast-forward/pypy/rlib/rstring.py
pypy/branch/fast-forward/pypy/rlib/test/test_jit.py
pypy/branch/fast-forward/pypy/rlib/test/test_rdynload.py
pypy/branch/fast-forward/pypy/rlib/test/test_rsocket.py
pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py
pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py
pypy/branch/fast-forward/pypy/rpython/llinterp.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/ll2ctypes.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/rbuilder.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/rclass.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_llmemory.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_lltype.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_rffi.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py
pypy/branch/fast-forward/pypy/rpython/memory/gctransform/asmgcroot.py
pypy/branch/fast-forward/pypy/rpython/memory/gctransform/transform.py
pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py
pypy/branch/fast-forward/pypy/rpython/memory/support.py
pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py
pypy/branch/fast-forward/pypy/rpython/module/ll_time.py
pypy/branch/fast-forward/pypy/rpython/rbuilder.py
pypy/branch/fast-forward/pypy/rpython/rbuiltin.py
pypy/branch/fast-forward/pypy/rpython/rpbc.py
pypy/branch/fast-forward/pypy/rpython/rtyper.py
pypy/branch/fast-forward/pypy/rpython/test/test_llinterp.py
pypy/branch/fast-forward/pypy/rpython/test/test_nongc.py
pypy/branch/fast-forward/pypy/rpython/test/test_rbuilder.py
pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py
pypy/branch/fast-forward/pypy/rpython/test/test_rptr.py
pypy/branch/fast-forward/pypy/translator/c/funcgen.py
pypy/branch/fast-forward/pypy/translator/c/gcc/instruction.py
pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track5.s
pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py
pypy/branch/fast-forward/pypy/translator/c/src/g_include.h
pypy/branch/fast-forward/pypy/translator/c/src/main.h
pypy/branch/fast-forward/pypy/translator/c/src/signals.h
pypy/branch/fast-forward/pypy/translator/c/src/stack.h
pypy/branch/fast-forward/pypy/translator/c/test/test_genc.py
pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py
pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py
pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py
pypy/branch/fast-forward/pypy/translator/c/test/test_typed.py
pypy/branch/fast-forward/pypy/translator/goal/ann_override.py
pypy/branch/fast-forward/pypy/translator/goal/app_main.py
pypy/branch/fast-forward/pypy/translator/jvm/test/test_class.py
pypy/branch/fast-forward/pypy/translator/oosupport/test_template/class_.py
pypy/branch/fast-forward/pypy/translator/platform/darwin.py
pypy/branch/fast-forward/pypy/translator/transform.py
Log:
Merge from trunk:
svn merge -r77543:78227 ../trunk/ .
Modified: pypy/branch/fast-forward/dotviewer/drawgraph.py
==============================================================================
--- pypy/branch/fast-forward/dotviewer/drawgraph.py (original)
+++ pypy/branch/fast-forward/dotviewer/drawgraph.py Fri Oct 22 23:09:43 2010
@@ -423,20 +423,43 @@
else:
for line in lines:
raw_line = line.replace('\\l','').replace('\r','') or ' '
- img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
- w, h = img.get_size()
- if w>wmax: wmax = w
- if raw_line.strip():
- if line.endswith('\\l'):
- def cmd(img=img, y=hmax):
- img.draw(xleft, ytop+y)
- elif line.endswith('\r'):
- def cmd(img=img, y=hmax, w=w):
- img.draw(xright-w, ytop+y)
- else:
- def cmd(img=img, y=hmax, w=w):
- img.draw(xcenter-w//2, ytop+y)
+ if '\f' in raw_line: # grayed out parts of the line
+ imgs = []
+ graytext = True
+ h = 16
+ w_total = 0
+ for linepart in raw_line.split('\f'):
+ graytext = not graytext
+ if not linepart.strip():
+ continue
+ if graytext:
+ fgcolor = (128, 160, 160)
+ else:
+ fgcolor = (0, 0, 0)
+ img = TextSnippet(self, linepart, fgcolor, bgcolor)
+ imgs.append((w_total, img))
+ w, h = img.get_size()
+ w_total += w
+ if w_total > wmax: wmax = w_total
+ def cmd(imgs=imgs, y=hmax):
+ for x, img in imgs:
+ img.draw(xleft+x, ytop+y)
commands.append(cmd)
+ else:
+ img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
+ w, h = img.get_size()
+ if w>wmax: wmax = w
+ if raw_line.strip():
+ if line.endswith('\\l'):
+ def cmd(img=img, y=hmax):
+ img.draw(xleft, ytop+y)
+ elif line.endswith('\r'):
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xright-w, ytop+y)
+ else:
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xcenter-w//2, ytop+y)
+ commands.append(cmd)
hmax += h
#hmax += 8
Modified: pypy/branch/fast-forward/lib-python/conftest.py
==============================================================================
--- pypy/branch/fast-forward/lib-python/conftest.py (original)
+++ pypy/branch/fast-forward/lib-python/conftest.py Fri Oct 22 23:09:43 2010
@@ -496,7 +496,12 @@
RegrTest('test_coding.py'),
RegrTest('test_complex_args.py'),
RegrTest('test_contextlib.py', usemodules="thread"),
- RegrTest('test_ctypes.py', usemodules="_rawffi"),
+ # we skip test ctypes, since we adapted it massively in order
+ # to test what we want to support. There are real failures,
+ # but it's about missing features that we don't want to support
+ # now
+ RegrTest('test_ctypes.py', usemodules="_rawffi",
+ skip="missing features that we don't want to support now"),
RegrTest('test_defaultdict.py'),
RegrTest('test_email_renamed.py'),
RegrTest('test_exception_variations.py'),
Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Fri Oct 22 23:09:43 2010
@@ -7,7 +7,7 @@
def round_up(size, alignment):
return (size + alignment - 1) & -alignment
-def size_alignment_pos(fields):
+def size_alignment_pos(fields, is_union=False):
import ctypes
size = 0
alignment = 1
@@ -15,14 +15,19 @@
for fieldname, ctype in fields:
fieldsize = ctypes.sizeof(ctype)
fieldalignment = ctypes.alignment(ctype)
- size = round_up(size, fieldalignment)
alignment = max(alignment, fieldalignment)
- pos.append(size)
- size += fieldsize
+ if is_union:
+ pos.append(0)
+ size = max(size, fieldsize)
+ else:
+ size = round_up(size, fieldalignment)
+ pos.append(size)
+ size += fieldsize
size = round_up(size, alignment)
return size, alignment, pos
-def names_and_fields(_fields_, superclass, zero_offset=False, anon=None):
+def names_and_fields(_fields_, superclass, zero_offset=False, anon=None,
+ is_union=False):
if isinstance(_fields_, tuple):
_fields_ = list(_fields_)
for _, tp in _fields_:
@@ -36,7 +41,7 @@
rawfields = [(name, ctype._ffishape)
for name, ctype in all_fields]
if not zero_offset:
- _, _, pos = size_alignment_pos(all_fields)
+ _, _, pos = size_alignment_pos(all_fields, is_union)
else:
pos = [0] * len(all_fields)
fields = {}
@@ -73,8 +78,8 @@
# ________________________________________________________________
-def _set_shape(tp, rawfields):
- tp._ffistruct = _rawffi.Structure(rawfields)
+def _set_shape(tp, rawfields, is_union=False):
+ tp._ffistruct = _rawffi.Structure(rawfields, is_union)
tp._ffiargshape = tp._ffishape = (tp._ffistruct, 1)
tp._fficompositesize = tp._ffistruct.size
@@ -92,13 +97,14 @@
raise AttributeError("Structure or union cannot contain itself")
self._names, rawfields, self._fieldtypes = names_and_fields(
value, self.__bases__[0], False,
- self.__dict__.get('_anonymous_', None))
+ self.__dict__.get('_anonymous_', None), self._is_union)
_CDataMeta.__setattr__(self, '_fields_', value)
- _set_shape(self, rawfields)
+ _set_shape(self, rawfields, self._is_union)
return
_CDataMeta.__setattr__(self, name, value)
-class StructureMeta(_CDataMeta):
+class StructOrUnionMeta(_CDataMeta):
+
def __new__(self, name, cls, typedict):
res = type.__new__(self, name, cls, typedict)
if '_fields_' in typedict:
@@ -109,8 +115,8 @@
raise AttributeError("Anonymous field not found")
res._names, rawfields, res._fieldtypes = names_and_fields(
typedict['_fields_'], cls[0], False,
- typedict.get('_anonymous_', None))
- _set_shape(res, rawfields)
+ typedict.get('_anonymous_', None), self._is_union)
+ _set_shape(res, rawfields, self._is_union)
return res
@@ -150,8 +156,8 @@
res.__dict__['_index'] = -1
return res
-class Structure(_CData):
- __metaclass__ = StructureMeta
+class StructOrUnion(_CData):
+ __metaclass__ = StructOrUnionMeta
def __new__(cls, *args, **kwds):
if not hasattr(cls, '_ffistruct'):
@@ -213,3 +219,10 @@
def _get_buffer_value(self):
return self._buffer.buffer
+
+
+class StructureMeta(StructOrUnionMeta):
+ _is_union = False
+
+class Structure(StructOrUnion):
+ __metaclass__ = StructureMeta
Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/union.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_ctypes/union.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_ctypes/union.py Fri Oct 22 23:09:43 2010
@@ -1,118 +1,7 @@
+from _ctypes import structure
+class UnionMeta(structure.StructOrUnionMeta):
+ _is_union = True
-import _rawffi
-from _ctypes.basics import _CData, _CDataMeta, store_reference, keepalive_key
-from _ctypes.basics import ensure_objects
-from _ctypes.structure import round_up, names_and_fields, struct_getattr,\
- struct_setattr
-
-
-def _set_shape(tp):
- size = tp._sizeofinstances()
- alignment = tp._alignmentofinstances()
- tp._ffiopaque = _rawffi.Structure((size, alignment)) # opaque
- tp._ffiargshape = tp._ffishape = (tp._ffiopaque, 1)
- tp._fficompositesize = tp._ffiopaque.size
- # we need to create an array of size one for each
- # of our elements
- tp._ffiarrays = {}
- for name, field in tp._fieldtypes.iteritems():
- tp._ffiarrays[name] = _rawffi.Array(field.ctype._ffishape)
-
-class UnionMeta(_CDataMeta):
- def __new__(self, name, cls, typedict):
- res = type.__new__(self, name, cls, typedict)
- if '_fields_' in typedict:
- res._names, rawfields, res._fieldtypes = names_and_fields(
- typedict['_fields_'], cls[0], True,
- typedict.get('_anonymous_', None))
- _set_shape(res)
-
- def __init__(self): # don't allow arguments by now
- if not hasattr(self, '_ffiarrays'):
- raise TypeError("Cannot instantiate union, has no type")
- # malloc size
- size = self.__class__._sizeofinstances()
- self.__dict__['_objects'] = {}
- self.__dict__['_buffer'] = self._ffiopaque(autofree=True)
- res.__init__ = __init__
- return res
-
- def _sizeofinstances(self):
- if not hasattr(self, '_size_'):
- self._size_ = max([field.size for field in
- self._fieldtypes.values()] + [0])
- return self._size_
-
- def _alignmentofinstances(self):
- from ctypes import alignment
- if not hasattr(self, '_alignment_'):
- self._alignment_ = max([alignment(field.ctype) for field in
- self._fieldtypes.values()] + [1])
- return self._alignment_
-
- __getattr__ = struct_getattr
-
- def __setattr__(self, name, value):
- if name == '_fields_':
- if self.__dict__.get('_fields_', None):
- raise AttributeError("_fields_ is final")
- if self in [v for k, v in value]:
- raise AttributeError("Union cannot contain itself")
- self._names, rawfields, self._fieldtypes = names_and_fields(
- value, self.__bases__[0], True,
- self.__dict__.get('_anonymous_', None))
- _CDataMeta.__setattr__(self, '_fields_', value)
- _set_shape(self)
- _CDataMeta.__setattr__(self, name, value)
-
- def _CData_output(self, resarray, base=None, index=-1):
- res = self.__new__(self)
- ffiopaque = self._ffiopaque.fromaddress(resarray.buffer)
- res.__dict__['_buffer'] = ffiopaque
- res.__dict__['_base'] = base
- res.__dict__['_index'] = index
- return res
-
- def _CData_retval(self, resbuffer):
- res = self.__new__(self)
- res.__dict__['_buffer'] = resbuffer
- res.__dict__['_base'] = None
- res.__dict__['_index'] = -1
- return res
-
-
-class Union(_CData):
+class Union(structure.StructOrUnion):
__metaclass__ = UnionMeta
-
- def __getattr__(self, name):
- try:
- field = self._fieldtypes[name]
- except KeyError:
- raise AttributeError(name)
- fieldtype = field.ctype
- val = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1)
- offset = field.num
- return fieldtype._CData_output(val, self, offset)
-
- def __setattr__(self, name, value):
- try:
- field = self._fieldtypes[name]
- except KeyError:
- raise AttributeError(name)
- fieldtype = field.ctype
- cobj = fieldtype.from_param(value)
- if ensure_objects(cobj) is not None:
- key = keepalive_key(field.num)
- store_reference(self, key, cobj._objects)
- arg = cobj._get_buffer_value()
- if fieldtype._fficompositesize is not None:
- from ctypes import memmove
- dest = self._buffer.buffer
- memmove(dest, arg, fieldtype._fficompositesize)
- else:
- buf = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1)
- buf[0] = arg
-
- def _get_buffer_value(self):
- return self._buffer.buffer
Modified: pypy/branch/fast-forward/pypy/annotation/builtin.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/builtin.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/builtin.py Fri Oct 22 23:09:43 2010
@@ -423,7 +423,7 @@
from pypy.annotation.model import SomePtr
from pypy.rpython.lltypesystem import lltype
-def malloc(s_T, s_n=None, s_flavor=None, s_zero=None):
+def malloc(s_T, s_n=None, s_flavor=None, s_zero=None, s_track_allocation=None):
assert (s_n is None or s_n.knowntype == int
or issubclass(s_n.knowntype, pypy.rlib.rarithmetic.base_int))
assert s_T.is_constant()
@@ -438,13 +438,15 @@
r = SomePtr(lltype.typeOf(p))
else:
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# not sure how to call malloc() for the example 'p' in the
# presence of s_extraargs
r = SomePtr(lltype.Ptr(s_T.const))
return r
-def free(s_p, s_flavor):
+def free(s_p, s_flavor, s_track_allocation=None):
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# same problem as in malloc(): some flavors are not easy to
# malloc-by-example
#T = s_p.ll_ptrtype.TO
Modified: pypy/branch/fast-forward/pypy/annotation/model.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/model.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/model.py Fri Oct 22 23:09:43 2010
@@ -584,11 +584,11 @@
NUMBER = object()
annotation_to_ll_map = [
+ (SomeSingleFloat(), lltype.SingleFloat),
(s_None, lltype.Void), # also matches SomeImpossibleValue()
(s_Bool, lltype.Bool),
(SomeInteger(knowntype=r_ulonglong), NUMBER),
(SomeFloat(), lltype.Float),
- (SomeSingleFloat(), lltype.SingleFloat),
(SomeLongFloat(), lltype.LongFloat),
(SomeChar(), lltype.Char),
(SomeUnicodeCodePoint(), lltype.UniChar),
Modified: pypy/branch/fast-forward/pypy/annotation/policy.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/policy.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/policy.py Fri Oct 22 23:09:43 2010
@@ -1,4 +1,4 @@
-# base annotation policy for overrides and specialization
+# base annotation policy for specialization
from pypy.annotation.specialize import default_specialize as default
from pypy.annotation.specialize import specialize_argvalue, specialize_argtype, specialize_arglistitemtype
from pypy.annotation.specialize import memo
@@ -41,7 +41,7 @@
if directive is None:
return pol.default_specialize
- # specialize|override:name[(args)]
+ # specialize[(args)]
directive_parts = directive.split('(', 1)
if len(directive_parts) == 1:
[name] = directive_parts
@@ -60,14 +60,6 @@
except AttributeError:
raise AttributeError("%r specialize tag not defined in annotation"
"policy %s" % (name, pol))
- if directive.startswith('override:'):
- # different signature: override__xyz(*args_s)
- if parms:
- raise Exception, "override:* specialisations don't support parameters"
- def specialize_override(funcdesc, args_s):
- funcdesc.overridden = True
- return specializer(*args_s)
- return specialize_override
else:
if not parms:
return specializer
@@ -92,9 +84,5 @@
from pypy.rpython.annlowlevel import LowLevelAnnotatorPolicy
return LowLevelAnnotatorPolicy.specialize__ll_and_arg(*args)
- def override__ignore(pol, *args):
- bk = getbookkeeper()
- return bk.immutablevalue(None)
-
class StrictAnnotatorPolicy(AnnotatorPolicy):
allow_someobjects = False
Modified: pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py Fri Oct 22 23:09:43 2010
@@ -766,28 +766,6 @@
s = a.build_types(f, [list])
assert s.classdef is a.bookkeeper.getuniqueclassdef(IndexError) # KeyError ignored because l is a list
- def test_overrides(self):
- excs = []
- def record_exc(e):
- """NOT_RPYTHON"""
- excs.append(sys.exc_info)
- record_exc._annspecialcase_ = "override:record_exc"
- def g():
- pass
- def f():
- try:
- g()
- except Exception, e:
- record_exc(e)
- class MyAnnotatorPolicy(policy.AnnotatorPolicy):
-
- def override__record_exc(pol, s_e):
- return a.bookkeeper.immutablevalue(None)
-
- a = self.RPythonAnnotator(policy=MyAnnotatorPolicy())
- s = a.build_types(f, [])
- assert s.const is None
-
def test_freeze_protocol(self):
class Stuff:
def __init__(self, flag):
@@ -3359,6 +3337,26 @@
s = a.build_types(f, [])
assert isinstance(s, annmodel.SomeChar)
+ def test_context_manager(self):
+ class C:
+ def __init__(self):
+ pass
+ def __enter__(self):
+ self.x = 1
+ def __exit__(self, *args):
+ self.x = 3
+ def f():
+ c = C()
+ with c:
+ pass
+ return c.x
+
+ a = self.RPythonAnnotator()
+ s = a.build_types(f, [])
+ assert isinstance(s, annmodel.SomeInteger)
+ # not a constant: both __enter__ and __exit__ have been annotated
+ assert not s.is_constant()
+
def g(n):
return [0,1,2,n]
Modified: pypy/branch/fast-forward/pypy/annotation/test/test_model.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/test/test_model.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/test/test_model.py Fri Oct 22 23:09:43 2010
@@ -128,7 +128,7 @@
assert isinstance(s_p, SomeOOInstance) and s_p.ootype == C
def test_annotation_to_lltype():
- from pypy.rlib.rarithmetic import r_uint
+ from pypy.rlib.rarithmetic import r_uint, r_singlefloat
s_i = SomeInteger()
s_pos = SomeInteger(nonneg=True)
s_1 = SomeInteger(nonneg=True); s_1.const = 1
@@ -151,6 +151,9 @@
C = ootype.Instance('C', ROOT, {})
ref = SomeOOInstance(C)
assert annotation_to_lltype(ref) == C
+ s_singlefloat = SomeSingleFloat()
+ s_singlefloat.const = r_singlefloat(0.0)
+ assert annotation_to_lltype(s_singlefloat) == lltype.SingleFloat
def test_ll_union():
PS1 = lltype.Ptr(lltype.GcStruct('s'))
Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/fast-forward/pypy/config/pypyoption.py (original)
+++ pypy/branch/fast-forward/pypy/config/pypyoption.py Fri Oct 22 23:09:43 2010
@@ -73,9 +73,9 @@
}
module_import_dependencies = {
- # no _rawffi if importing pypy.rlib.libffi raises ImportError
+ # no _rawffi if importing pypy.rlib.clibffi raises ImportError
# or CompilationError
- "_rawffi" : ["pypy.rlib.libffi"],
+ "_rawffi" : ["pypy.rlib.clibffi"],
"zlib" : ["pypy.rlib.rzlib"],
"bz2" : ["pypy.module.bz2.interp_bz2"],
@@ -198,6 +198,9 @@
BoolOption("withstrslice", "use strings optimized for slicing",
default=False),
+ BoolOption("withstrbuf", "use strings optimized for addition (ver 2)",
+ default=False),
+
BoolOption("withprebuiltchar",
"use prebuilt single-character string objects",
default=False),
@@ -210,7 +213,8 @@
BoolOption("withrope", "use ropes as the string implementation",
default=False,
requires=[("objspace.std.withstrslice", False),
- ("objspace.std.withstrjoin", False)],
+ ("objspace.std.withstrjoin", False),
+ ("objspace.std.withstrbuf", False)],
suggests=[("objspace.std.withprebuiltchar", True),
("objspace.std.sharesmallstr", True)]),
@@ -238,6 +242,16 @@
default=False,
requires=[("objspace.std.withshadowtracking", False)]),
+ BoolOption("withmapdict",
+ "make instances really small but slow without the JIT",
+ default=False,
+ requires=[("objspace.std.withshadowtracking", False),
+ ("objspace.std.withinlineddict", False),
+ ("objspace.std.withsharingdict", False),
+ ("objspace.std.getattributeshortcut", True),
+ ("objspace.std.withtypeversion", True),
+ ]),
+
BoolOption("withrangelist",
"enable special range list implementation that does not "
"actually create the full list until the resulting "
@@ -343,7 +357,7 @@
config.objspace.std.suggest(withprebuiltint=True)
config.objspace.std.suggest(withrangelist=True)
config.objspace.std.suggest(withprebuiltchar=True)
- config.objspace.std.suggest(withinlineddict=True)
+ config.objspace.std.suggest(withmapdict=True)
config.objspace.std.suggest(withstrslice=True)
config.objspace.std.suggest(withstrjoin=True)
# xxx other options? ropes maybe?
@@ -359,6 +373,7 @@
# extra optimizations with the JIT
if level == 'jit':
config.objspace.std.suggest(withcelldict=True)
+ #config.objspace.std.suggest(withmapdict=True)
def enable_allworkingmodules(config):
Modified: pypy/branch/fast-forward/pypy/conftest.py
==============================================================================
--- pypy/branch/fast-forward/pypy/conftest.py (original)
+++ pypy/branch/fast-forward/pypy/conftest.py Fri Oct 22 23:09:43 2010
@@ -7,6 +7,7 @@
from inspect import isclass, getmro
from pypy.tool.udir import udir
from pypy.tool.autopath import pypydir
+from pypy.tool import leakfinder
# pytest settings
pytest_plugins = "resultlog",
@@ -358,7 +359,14 @@
def runtest(self):
try:
- super(IntTestFunction, self).runtest()
+ leakfinder.start_tracking_allocations()
+ try:
+ super(IntTestFunction, self).runtest()
+ finally:
+ if leakfinder.TRACK_ALLOCATIONS:
+ leaks = leakfinder.stop_tracking_allocations(False)
+ else:
+ leaks = None # stop_tracking_allocations() already called
except OperationError, e:
check_keyboard_interrupt(e)
raise
@@ -377,6 +385,8 @@
_pygame_imported = True
assert option.view, ("should not invoke Pygame "
"if conftest.option.view is False")
+ if leaks: # check for leaks, but only if the test passed so far
+ raise leakfinder.MallocMismatch(leaks)
class AppTestFunction(PyPyTestFunction):
def _prunetraceback(self, traceback):
Modified: pypy/branch/fast-forward/pypy/doc/docindex.txt
==============================================================================
--- pypy/branch/fast-forward/pypy/doc/docindex.txt (original)
+++ pypy/branch/fast-forward/pypy/doc/docindex.txt Fri Oct 22 23:09:43 2010
@@ -84,7 +84,7 @@
PyPy's own tests `summary`_, daily updated, run through BuildBot infrastructure.
You can also find CPython's compliance tests run with compiled ``pypy-c``
-exeuctables there.
+executables there.
information dating from early 2007:
Modified: pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py Fri Oct 22 23:09:43 2010
@@ -168,6 +168,20 @@
def _call_builtin_destructor(self):
pass # method overridden in typedef.py
+ # hooks that the mapdict implementations needs:
+ def _get_mapdict_map(self):
+ return None
+ def _set_mapdict_map(self, map):
+ raise NotImplementedError
+ def _mapdict_read_storage(self, index):
+ raise NotImplementedError
+ def _mapdict_write_storage(self, index, value):
+ raise NotImplementedError
+ def _mapdict_storage_length(self):
+ raise NotImplementedError
+ def _set_mapdict_storage_and_map(self, storage, map):
+ raise NotImplementedError
+
class Wrappable(W_Root):
"""A subclass of Wrappable is an internal, interpreter-level class
Modified: pypy/branch/fast-forward/pypy/interpreter/pycode.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pycode.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pycode.py Fri Oct 22 23:09:43 2010
@@ -117,6 +117,10 @@
self._compute_flatcall()
+ if self.space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import init_mapdict_cache
+ init_mapdict_cache(self)
+
def _freeze_(self):
if (self.magic == cpython_magic and
'__pypy__' not in sys.builtin_module_names):
Modified: pypy/branch/fast-forward/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyopcode.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyopcode.py Fri Oct 22 23:09:43 2010
@@ -164,6 +164,9 @@
next_instr = block.handle(self, unroller)
return next_instr
+ def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
+ return self.space.call_function(w_func, w_typ, w_val, w_tb)
+
@jit.unroll_safe
def dispatch_bytecode(self, co_code, next_instr, ec):
space = self.space
@@ -710,9 +713,14 @@
def LOAD_ATTR(self, nameindex, next_instr):
"obj.attributename"
- w_attributename = self.getname_w(nameindex)
w_obj = self.popvalue()
- w_value = self.space.getattr(w_obj, w_attributename)
+ if (self.space.config.objspace.std.withmapdict
+ and not jit.we_are_jitted()):
+ from pypy.objspace.std.mapdict import LOAD_ATTR_caching
+ w_value = LOAD_ATTR_caching(self.getcode(), w_obj, nameindex)
+ else:
+ w_attributename = self.getname_w(nameindex)
+ w_value = self.space.getattr(w_obj, w_attributename)
self.pushvalue(w_value)
LOAD_ATTR._always_inline_ = True
@@ -904,28 +912,47 @@
self.pushvalue(w_result)
def WITH_CLEANUP(self, oparg, next_instr):
- self.dropvalues(2)
- w_unroller = self.popvalue()
+ # see comment in END_FINALLY for stack state
+ # This opcode changed a lot between CPython versions
+ if (self.pycode.magic >= 0xa0df2ef
+ # Implementation since 2.7a0: 62191 (introduce SETUP_WITH)
+ or self.pycode.magic >= 0xa0df2d1):
+ # implementation since 2.6a1: 62161 (WITH_CLEANUP optimization)
+ self.popvalue()
+ self.popvalue()
+ w_unroller = self.popvalue()
+ w_exitfunc = self.popvalue()
+ self.pushvalue(w_unroller)
+ self.pushvalue(self.space.w_None)
+ self.pushvalue(self.space.w_None)
+ elif self.pycode.magic >= 0xa0df28c:
+ # Implementation since 2.5a0: 62092 (changed WITH_CLEANUP opcode)
+ w_exitfunc = self.popvalue()
+ w_unroller = self.peekvalue(2)
+ else:
+ raise NotImplementedError("WITH_CLEANUP for CPython <= 2.4")
+
unroller = self.space.interpclass_w(w_unroller)
w_exit = self.popvalue()
is_app_exc = (unroller is not None and
isinstance(unroller, SApplicationException))
if is_app_exc:
operr = unroller.operr
- w_type = operr.w_type
- w_value = operr.get_w_value(self.space)
- w_tb = self.space.wrap(operr.application_traceback)
- else:
- w_type = w_value = w_tb = self.space.w_None
- w_suppress = self.space.call_function(w_exit, w_type, w_value, w_tb)
- if is_app_exc and self.space.is_true(w_suppress):
- self.pushvalue(self.space.w_None)
- self.pushvalue(self.space.w_None)
- self.pushvalue(self.space.w_None)
+ w_traceback = self.space.wrap(operr.application_traceback)
+ w_suppress = self.call_contextmanager_exit_function(
+ w_exitfunc,
+ operr.w_type,
+ operr.get_w_value(self.space),
+ w_traceback)
+ if self.space.is_true(w_suppress):
+ # __exit__() returned True -> Swallow the exception.
+ self.settopvalue(self.space.w_None, 2)
else:
- self.pushvalue(w_unroller)
- self.pushvalue(w_value)
- self.pushvalue(w_type)
+ self.call_contextmanager_exit_function(
+ w_exitfunc,
+ self.space.w_None,
+ self.space.w_None,
+ self.space.w_None)
@jit.unroll_safe
def call_function(self, oparg, w_star=None, w_starstar=None):
@@ -1345,10 +1372,10 @@
unroller.operr.normalize_exception(frame.space)
return FinallyBlock.really_handle(self, frame, unroller)
-
block_classes = {'SETUP_LOOP': LoopBlock,
'SETUP_EXCEPT': ExceptBlock,
- 'SETUP_FINALLY': FinallyBlock}
+ 'SETUP_FINALLY': FinallyBlock,
+ 'SETUP_WITH': WithBlock}
### helpers written at the application-level ###
# Some of these functions are expected to be generally useful if other
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py Fri Oct 22 23:09:43 2010
@@ -268,7 +268,8 @@
i += 1
import StringIO
print "states = ["
- for state in dfa.states:
+ for numstate, state in enumerate(dfa.states):
+ print " #", numstate
s = StringIO.StringIO()
i = 0
for k, v in sorted(state.items()):
@@ -284,14 +285,17 @@
s.write(', ')
s.write('},')
i = 0
- for line in textwrap.wrap(s.getvalue(), width=35):
+ if len(state) <= 4:
+ text = [s.getvalue()]
+ else:
+ text = textwrap.wrap(s.getvalue(), width=36)
+ for line in text:
line = line.replace('::', ': ')
if i == 0:
print ' {' + line
else:
print ' ' + line
i += 1
- print
print " ]"
print "%s = automata.%s(states, accepts)" % (name, dfa_class)
print
@@ -302,16 +306,8 @@
endDFAMap = makePyEndDFAMap()
output("double3DFA", "NonGreedyDFA", endDFAMap['"""'])
output("single3DFA", "NonGreedyDFA", endDFAMap["'''"])
- output("doubleDFA", "DFA", endDFAMap['"'])
output("singleDFA", "DFA", endDFAMap["'"])
- print "endDFAs = {\"'\" : singleDFA,"
- print " '\"' : doubleDFA,"
- print " 'r' : None,"
- print " 'R' : None,"
- print " 'u' : None,"
- print " 'U' : None,"
- print " 'b' : None,"
- print " 'B' : None}"
+ output("doubleDFA", "DFA", endDFAMap['"'])
# ______________________________________________________________________
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py Fri Oct 22 23:09:43 2010
@@ -30,6 +30,7 @@
True, False, False, False, False, True, False,
False, False, True]
states = [
+ # 0
{'\t': 0, '\n': 13, '\x0c': 0,
'\r': 14, ' ': 0, '!': 10, '"': 16,
'#': 18, '%': 12, '&': 12, "'": 15,
@@ -55,7 +56,7 @@
'u': 2, 'v': 1, 'w': 1, 'x': 1,
'y': 1, 'z': 1, '{': 13, '|': 12,
'}': 13, '~': 13},
-
+ # 1
{'0': 1, '1': 1, '2': 1, '3': 1,
'4': 1, '5': 1, '6': 1, '7': 1,
'8': 1, '9': 1, 'A': 1, 'B': 1,
@@ -72,7 +73,7 @@
'p': 1, 'q': 1, 'r': 1, 's': 1,
't': 1, 'u': 1, 'v': 1, 'w': 1,
'x': 1, 'y': 1, 'z': 1},
-
+ # 2
{'"': 16, "'": 15, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
@@ -90,7 +91,7 @@
'r': 3, 's': 1, 't': 1, 'u': 1,
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
-
+ # 3
{'"': 16, "'": 15, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
@@ -108,215 +109,188 @@
'r': 1, 's': 1, 't': 1, 'u': 1,
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
-
+ # 4
{'.': 24, '0': 21, '1': 21, '2': 21,
'3': 21, '4': 21, '5': 21, '6': 21,
'7': 21, '8': 23, '9': 23, 'B': 22,
'E': 25, 'J': 13, 'L': 13, 'O': 20,
'X': 19, 'b': 22, 'e': 25, 'j': 13,
'l': 13, 'o': 20, 'x': 19},
-
+ # 5
{'.': 24, '0': 5, '1': 5, '2': 5,
'3': 5, '4': 5, '5': 5, '6': 5,
'7': 5, '8': 5, '9': 5, 'E': 25,
'J': 13, 'L': 13, 'e': 25, 'j': 13,
'l': 13},
-
+ # 6
{'0': 26, '1': 26, '2': 26, '3': 26,
'4': 26, '5': 26, '6': 26, '7': 26,
'8': 26, '9': 26},
-
+ # 7
{'*': 12, '=': 13},
-
+ # 8
{'=': 13, '>': 12},
-
+ # 9
{'<': 12, '=': 13, '>': 13},
-
+ # 10
{'=': 13},
-
+ # 11
{'/': 12, '=': 13},
-
+ # 12
{'=': 13},
-
+ # 13
{},
-
+ # 14
{'\n': 13},
-
- {automata.DEFAULT: 30, '\n': 27,
- "'": 28, '\\': 29},
-
- {automata.DEFAULT: 33, '\n': 27,
- '"': 31, '\\': 32},
-
+ # 15
+ {automata.DEFAULT: 30, '\n': 27, "'": 28, '\\': 29},
+ # 16
+ {automata.DEFAULT: 33, '\n': 27, '"': 31, '\\': 32},
+ # 17
{'\n': 13, '\r': 14},
-
- {automata.DEFAULT: 18, '\n': 27,
- '\r': 27},
-
+ # 18
+ {automata.DEFAULT: 18, '\n': 27, '\r': 27},
+ # 19
{'0': 19, '1': 19, '2': 19, '3': 19,
'4': 19, '5': 19, '6': 19, '7': 19,
'8': 19, '9': 19, 'A': 19, 'B': 19,
'C': 19, 'D': 19, 'E': 19, 'F': 19,
'L': 13, 'a': 19, 'b': 19, 'c': 19,
- 'd': 19, 'e': 19, 'f': 19,
- 'l': 13},
-
+ 'd': 19, 'e': 19, 'f': 19, 'l': 13},
+ # 20
{'0': 20, '1': 20, '2': 20, '3': 20,
'4': 20, '5': 20, '6': 20, '7': 20,
'L': 13, 'l': 13},
-
+ # 21
{'.': 24, '0': 21, '1': 21, '2': 21,
'3': 21, '4': 21, '5': 21, '6': 21,
'7': 21, '8': 23, '9': 23, 'E': 25,
'J': 13, 'L': 13, 'e': 25, 'j': 13,
'l': 13},
-
- {'0': 22, '1': 22, 'L': 13,
- 'l': 13},
-
+ # 22
+ {'0': 22, '1': 22, 'L': 13, 'l': 13},
+ # 23
{'.': 24, '0': 23, '1': 23, '2': 23,
'3': 23, '4': 23, '5': 23, '6': 23,
'7': 23, '8': 23, '9': 23, 'E': 25,
'J': 13, 'e': 25, 'j': 13},
-
+ # 24
{'0': 24, '1': 24, '2': 24, '3': 24,
'4': 24, '5': 24, '6': 24, '7': 24,
'8': 24, '9': 24, 'E': 34, 'J': 13,
'e': 34, 'j': 13},
-
+ # 25
{'+': 35, '-': 35, '0': 36, '1': 36,
'2': 36, '3': 36, '4': 36, '5': 36,
- '6': 36, '7': 36, '8': 36,
- '9': 36},
-
+ '6': 36, '7': 36, '8': 36, '9': 36},
+ # 26
{'0': 26, '1': 26, '2': 26, '3': 26,
'4': 26, '5': 26, '6': 26, '7': 26,
'8': 26, '9': 26, 'E': 34, 'J': 13,
'e': 34, 'j': 13},
-
+ # 27
{},
-
+ # 28
{"'": 13},
-
- {automata.DEFAULT: 37, '\n': 13,
- '\r': 14},
-
- {automata.DEFAULT: 30, '\n': 27,
- "'": 13, '\\': 29},
-
+ # 29
+ {automata.DEFAULT: 37, '\n': 13, '\r': 14},
+ # 30
+ {automata.DEFAULT: 30, '\n': 27, "'": 13, '\\': 29},
+ # 31
{'"': 13},
-
- {automata.DEFAULT: 38, '\n': 13,
- '\r': 14},
-
- {automata.DEFAULT: 33, '\n': 27,
- '"': 13, '\\': 32},
-
+ # 32
+ {automata.DEFAULT: 38, '\n': 13, '\r': 14},
+ # 33
+ {automata.DEFAULT: 33, '\n': 27, '"': 13, '\\': 32},
+ # 34
{'+': 39, '-': 39, '0': 40, '1': 40,
'2': 40, '3': 40, '4': 40, '5': 40,
- '6': 40, '7': 40, '8': 40,
- '9': 40},
-
+ '6': 40, '7': 40, '8': 40, '9': 40},
+ # 35
{'0': 36, '1': 36, '2': 36, '3': 36,
'4': 36, '5': 36, '6': 36, '7': 36,
'8': 36, '9': 36},
-
+ # 36
{'0': 36, '1': 36, '2': 36, '3': 36,
'4': 36, '5': 36, '6': 36, '7': 36,
- '8': 36, '9': 36, 'J': 13,
- 'j': 13},
-
- {automata.DEFAULT: 37, '\n': 27,
- "'": 13, '\\': 29},
-
- {automata.DEFAULT: 38, '\n': 27,
- '"': 13, '\\': 32},
-
+ '8': 36, '9': 36, 'J': 13, 'j': 13},
+ # 37
+ {automata.DEFAULT: 37, '\n': 27, "'": 13, '\\': 29},
+ # 38
+ {automata.DEFAULT: 38, '\n': 27, '"': 13, '\\': 32},
+ # 39
{'0': 40, '1': 40, '2': 40, '3': 40,
'4': 40, '5': 40, '6': 40, '7': 40,
'8': 40, '9': 40},
-
+ # 40
{'0': 40, '1': 40, '2': 40, '3': 40,
'4': 40, '5': 40, '6': 40, '7': 40,
- '8': 40, '9': 40, 'J': 13,
- 'j': 13},
-
+ '8': 40, '9': 40, 'J': 13, 'j': 13},
]
pseudoDFA = automata.DFA(states, accepts)
accepts = [False, False, False, False, False, True]
states = [
- {automata.DEFAULT: 0, '"': 1,
- '\\': 2},
-
- {automata.DEFAULT: 4, '"': 3,
- '\\': 2},
-
+ # 0
+ {automata.DEFAULT: 0, '"': 1, '\\': 2},
+ # 1
+ {automata.DEFAULT: 4, '"': 3, '\\': 2},
+ # 2
{automata.DEFAULT: 4},
-
- {automata.DEFAULT: 4, '"': 5,
- '\\': 2},
-
- {automata.DEFAULT: 4, '"': 1,
- '\\': 2},
-
- {automata.DEFAULT: 4, '"': 5,
- '\\': 2},
-
+ # 3
+ {automata.DEFAULT: 4, '"': 5, '\\': 2},
+ # 4
+ {automata.DEFAULT: 4, '"': 1, '\\': 2},
+ # 5
+ {automata.DEFAULT: 4, '"': 5, '\\': 2},
]
double3DFA = automata.NonGreedyDFA(states, accepts)
accepts = [False, False, False, False, False, True]
states = [
- {automata.DEFAULT: 0, "'": 1,
- '\\': 2},
-
- {automata.DEFAULT: 4, "'": 3,
- '\\': 2},
-
+ # 0
+ {automata.DEFAULT: 0, "'": 1, '\\': 2},
+ # 1
+ {automata.DEFAULT: 4, "'": 3, '\\': 2},
+ # 2
{automata.DEFAULT: 4},
-
- {automata.DEFAULT: 4, "'": 5,
- '\\': 2},
-
- {automata.DEFAULT: 4, "'": 1,
- '\\': 2},
-
- {automata.DEFAULT: 4, "'": 5,
- '\\': 2},
-
+ # 3
+ {automata.DEFAULT: 4, "'": 5, '\\': 2},
+ # 4
+ {automata.DEFAULT: 4, "'": 1, '\\': 2},
+ # 5
+ {automata.DEFAULT: 4, "'": 5, '\\': 2},
]
single3DFA = automata.NonGreedyDFA(states, accepts)
accepts = [False, True, False, False]
states = [
- {automata.DEFAULT: 0, '"': 1,
- '\\': 2},
-
+ # 0
+ {automata.DEFAULT: 0, "'": 1, '\\': 2},
+ # 1
{},
-
+ # 2
{automata.DEFAULT: 3},
-
- {automata.DEFAULT: 3, '"': 1,
- '\\': 2},
-
+ # 3
+ {automata.DEFAULT: 3, "'": 1, '\\': 2},
]
-doubleDFA = automata.DFA(states, accepts)
+singleDFA = automata.DFA(states, accepts)
accepts = [False, True, False, False]
states = [
- {automata.DEFAULT: 0, "'": 1,
- '\\': 2},
-
+ # 0
+ {automata.DEFAULT: 0, '"': 1, '\\': 2},
+ # 1
{},
-
+ # 2
{automata.DEFAULT: 3},
-
- {automata.DEFAULT: 3, "'": 1,
- '\\': 2},
-
+ # 3
+ {automata.DEFAULT: 3, '"': 1, '\\': 2},
]
-singleDFA = automata.DFA(states, accepts)
+doubleDFA = automata.DFA(states, accepts)
+
+#_______________________________________________________________________
+# End of automatically generated DFA's
endDFAs = {"'" : singleDFA,
'"' : doubleDFA,
Modified: pypy/branch/fast-forward/pypy/interpreter/typedef.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/typedef.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/typedef.py Fri Oct 22 23:09:43 2010
@@ -127,6 +127,13 @@
typedef = cls.typedef
if wants_dict and typedef.hasdict:
wants_dict = False
+ if config.objspace.std.withmapdict and not typedef.hasdict:
+ # mapdict only works if the type does not already have a dict
+ if wants_del:
+ parentcls = get_unique_interplevel_subclass(config, cls, True, True,
+ False, True)
+ return _usersubclswithfeature(config, parentcls, "del")
+ return _usersubclswithfeature(config, cls, "user", "dict", "weakref", "slots")
# Forest of if's - see the comment above.
if wants_del:
if wants_dict:
@@ -180,10 +187,20 @@
def add(Proto):
for key, value in Proto.__dict__.items():
- if not key.startswith('__') or key == '__del__':
+ if (not key.startswith('__') and not key.startswith('_mixin_')
+ or key == '__del__'):
+ if hasattr(value, "func_name"):
+ value = func_with_new_name(value, value.func_name)
body[key] = value
+ if (config.objspace.std.withmapdict and "dict" in features):
+ from pypy.objspace.std.mapdict import BaseMapdictObject, ObjectMixin
+ add(BaseMapdictObject)
+ add(ObjectMixin)
+ features = ()
+
if "user" in features: # generic feature needed by all subcls
+
class Proto(object):
user_overridden_class = True
@@ -249,6 +266,9 @@
wantdict = False
if wantdict:
+ base_user_setup = supercls.user_setup.im_func
+ if "user_setup" in body:
+ base_user_setup = body["user_setup"]
class Proto(object):
def getdict(self):
return self.w__dict__
@@ -257,11 +277,9 @@
self.w__dict__ = check_new_dictionary(space, w_dict)
def user_setup(self, space, w_subtype):
- self.space = space
- self.w__class__ = w_subtype
self.w__dict__ = space.newdict(
instance=True, classofinstance=w_subtype)
- self.user_setup_slots(w_subtype.nslots)
+ base_user_setup(self, space, w_subtype)
def setclass(self, space, w_subtype):
# only used by descr_set___class__
Modified: pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py Fri Oct 22 23:09:43 2010
@@ -10,7 +10,7 @@
BoxInt, BoxPtr, BoxObj, BoxFloat,
REF, INT, FLOAT)
from pypy.jit.codewriter import heaptracker
-from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.module.support import LLSupport, OOSupport
from pypy.rpython.llinterp import LLException
@@ -305,12 +305,12 @@
loop = _from_opaque(loop)
loop.operations.append(Operation(opnum))
-def compile_add_descr(loop, ofs, type):
+def compile_add_descr(loop, ofs, type, arg_types):
from pypy.jit.backend.llgraph.runner import Descr
loop = _from_opaque(loop)
op = loop.operations[-1]
assert isinstance(type, str) and len(type) == 1
- op.descr = Descr(ofs, type)
+ op.descr = Descr(ofs, type, arg_types=arg_types)
def compile_add_loop_token(loop, descr):
if we_are_translated():
@@ -801,7 +801,7 @@
else:
raise TypeError(x)
try:
- return _do_call_common(func, args_in_order)
+ return _do_call_common(func, args_in_order, calldescr)
except LLException, lle:
_last_exception = lle
d = {'v': None,
@@ -1018,6 +1018,9 @@
if isinstance(TYPE, lltype.Ptr):
if isinstance(x, (int, long, llmemory.AddressAsInt)):
x = llmemory.cast_int_to_adr(x)
+ if TYPE is rffi.VOIDP:
+ # assume that we want a "C-style" cast, without typechecking the value
+ return rffi.cast(TYPE, x)
return llmemory.cast_adr_to_ptr(x, TYPE)
elif TYPE == llmemory.Address:
if isinstance(x, (int, long, llmemory.AddressAsInt)):
@@ -1411,10 +1414,26 @@
def do_call_pushfloat(x):
_call_args_f.append(x)
-def _do_call_common(f, args_in_order=None):
+kind2TYPE = {
+ 'i': lltype.Signed,
+ 'f': lltype.Float,
+ 'v': lltype.Void,
+ }
+
+def _do_call_common(f, args_in_order=None, calldescr=None):
ptr = llmemory.cast_int_to_adr(f).ptr
- FUNC = lltype.typeOf(ptr).TO
- ARGS = FUNC.ARGS
+ PTR = lltype.typeOf(ptr)
+ if PTR == rffi.VOIDP:
+ # it's a pointer to a C function, so we don't have a precise
+ # signature: create one from the descr
+ ARGS = map(kind2TYPE.get, calldescr.arg_types)
+ RESULT = kind2TYPE[calldescr.typeinfo]
+ FUNC = lltype.FuncType(ARGS, RESULT)
+ func_to_call = rffi.cast(lltype.Ptr(FUNC), ptr)
+ else:
+ FUNC = PTR.TO
+ ARGS = FUNC.ARGS
+ func_to_call = ptr._obj._callable
args = cast_call_args(ARGS, _call_args_i, _call_args_r, _call_args_f,
args_in_order)
del _call_args_i[:]
@@ -1426,7 +1445,7 @@
result = llinterp.eval_graph(ptr._obj.graph, args)
# ^^^ may raise, in which case we get an LLException
else:
- result = ptr._obj._callable(*args)
+ result = func_to_call(*args)
return result
def do_call_void(f):
Modified: pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py Fri Oct 22 23:09:43 2010
@@ -154,7 +154,7 @@
llimpl.compile_add(c, op.getopnum())
descr = op.getdescr()
if isinstance(descr, Descr):
- llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo)
+ llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo, descr.arg_types)
if isinstance(descr, history.LoopToken) and op.getopnum() != rop.JUMP:
llimpl.compile_add_loop_token(c, descr)
if self.is_oo and isinstance(descr, (OODescr, MethDescr)):
@@ -297,6 +297,18 @@
return self.getdescr(0, token[0], extrainfo=extrainfo,
arg_types=''.join(arg_types))
+ def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+ from pypy.jit.backend.llsupport.ffisupport import get_ffi_type_kind
+ arg_types = []
+ for arg in ffi_args:
+ kind = get_ffi_type_kind(arg)
+ if kind != history.VOID:
+ arg_types.append(kind)
+ reskind = get_ffi_type_kind(ffi_result)
+ return self.getdescr(0, reskind, extrainfo=extrainfo,
+ arg_types=''.join(arg_types))
+
+
def grab_exc_value(self):
return llimpl.grab_exc_value()
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py Fri Oct 22 23:09:43 2010
@@ -82,6 +82,7 @@
_is_pointer_field = False # unless overridden by GcPtrFieldDescr
_is_float_field = False # unless overridden by FloatFieldDescr
+ _is_field_signed = False # unless overridden by XxxFieldDescr
def is_pointer_field(self):
return self._is_pointer_field
@@ -89,6 +90,9 @@
def is_float_field(self):
return self._is_float_field
+ def is_field_signed(self):
+ return self._is_field_signed
+
def repr_of_descr(self):
return '<%s %s %s>' % (self._clsname, self.name, self.offset)
@@ -105,7 +109,7 @@
def getFieldDescrClass(TYPE):
return getDescrClass(TYPE, BaseFieldDescr, GcPtrFieldDescr,
NonGcPtrFieldDescr, 'Field', 'get_field_size',
- '_is_float_field')
+ '_is_float_field', '_is_field_signed')
def get_field_descr(gccache, STRUCT, fieldname):
cache = gccache._cache_field
@@ -144,6 +148,7 @@
_is_array_of_pointers = False # unless overridden by GcPtrArrayDescr
_is_array_of_floats = False # unless overridden by FloatArrayDescr
+ _is_item_signed = False # unless overridden by XxxArrayDescr
def is_array_of_pointers(self):
return self._is_array_of_pointers
@@ -151,6 +156,9 @@
def is_array_of_floats(self):
return self._is_array_of_floats
+ def is_item_signed(self):
+ return self._is_item_signed
+
def repr_of_descr(self):
return '<%s>' % self._clsname
@@ -186,12 +194,12 @@
def getArrayDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr,
NonGcPtrArrayDescr, 'Array', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def getArrayNoLengthDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayNoLengthDescr, GcPtrArrayNoLengthDescr,
NonGcPtrArrayNoLengthDescr, 'ArrayNoLength', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def get_array_descr(gccache, ARRAY):
cache = gccache._cache_array
@@ -242,6 +250,9 @@
def get_result_size(self, translate_support_code):
raise NotImplementedError
+ def is_result_signed(self):
+ return False # unless overridden
+
def create_call_stub(self, rtyper, RESULT):
def process(c):
arg = 'args_%s[%d]' % (c, seen[c])
@@ -307,6 +318,30 @@
_return_type = history.INT
call_stub = staticmethod(lambda func, args_i, args_r, args_f: 0)
+ _is_result_signed = False # can be overridden in XxxCallDescr
+ def is_result_signed(self):
+ return self._is_result_signed
+
+class DynamicIntCallDescr(BaseIntCallDescr):
+ """
+ calldescr that works for every integer type, by explicitly passing it the
+ size of the result. Used only by get_call_descr_dynamic
+ """
+ _clsname = 'DynamicIntCallDescr'
+
+ def __init__(self, arg_classes, result_size, result_sign, extrainfo=None):
+ BaseIntCallDescr.__init__(self, arg_classes, extrainfo)
+ assert isinstance(result_sign, bool)
+ self._result_size = chr(result_size)
+ self._result_sign = result_sign
+
+ def get_result_size(self, translate_support_code):
+ return ord(self._result_size)
+
+ def is_result_signed(self):
+ return self._result_sign
+
+
class NonGcPtrCallDescr(BaseIntCallDescr):
_clsname = 'NonGcPtrCallDescr'
def get_result_size(self, translate_support_code):
@@ -341,7 +376,8 @@
return FloatCallDescr
return getDescrClass(RESULT, BaseIntCallDescr, GcPtrCallDescr,
NonGcPtrCallDescr, 'Call', 'get_result_size',
- Ellipsis) # <= floatattrname should not be used here
+ Ellipsis, # <= floatattrname should not be used here
+ '_is_result_signed')
def get_call_descr(gccache, ARGS, RESULT, extrainfo=None):
arg_classes = []
@@ -368,7 +404,8 @@
# ____________________________________________________________
def getDescrClass(TYPE, BaseDescr, GcPtrDescr, NonGcPtrDescr,
- nameprefix, methodname, floatattrname, _cache={}):
+ nameprefix, methodname, floatattrname, signedattrname,
+ _cache={}):
if isinstance(TYPE, lltype.Ptr):
if TYPE.TO._gckind == 'gc':
return GcPtrDescr
@@ -388,6 +425,8 @@
#
if TYPE is lltype.Float:
setattr(Descr, floatattrname, True)
+ elif TYPE is not lltype.Bool and rffi.cast(TYPE, -1) == -1:
+ setattr(Descr, signedattrname, True)
#
_cache[nameprefix, TYPE] = Descr
return Descr
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py Fri Oct 22 23:09:43 2010
@@ -158,7 +158,7 @@
# used to avoid too many duplications in the GCREF_LISTs.
self.hashtable = lltype.malloc(self.HASHTABLE,
self.HASHTABLE_SIZE+1,
- flavor='raw')
+ flavor='raw', track_allocation=False)
dummy = lltype.direct_ptradd(lltype.direct_arrayitems(self.hashtable),
self.HASHTABLE_SIZE)
dummy = llmemory.cast_ptr_to_adr(dummy)
@@ -252,14 +252,15 @@
def _enlarge_gcmap(self):
newlength = 250 + self._gcmap_maxlength * 2
- newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw')
+ newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw',
+ track_allocation=False)
oldgcmap = self._gcmap
for i in range(self._gcmap_curlength):
newgcmap[i] = oldgcmap[i]
self._gcmap = newgcmap
self._gcmap_maxlength = newlength
if oldgcmap:
- lltype.free(oldgcmap, flavor='raw')
+ lltype.free(oldgcmap, flavor='raw', track_allocation=False)
def get_basic_shape(self, is_64_bit=False):
# XXX: Should this code even really know about stack frame layout of
@@ -308,7 +309,8 @@
# them inside bigger arrays) and we never try to share them.
length = len(shape)
compressed = lltype.malloc(self.CALLSHAPE_ARRAY, length,
- flavor='raw')
+ flavor='raw',
+ track_allocation=False) # memory leak
for i in range(length):
compressed[length-1-i] = rffi.cast(rffi.UCHAR, shape[i])
return llmemory.cast_ptr_to_adr(compressed)
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/llmodel.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/llmodel.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/llmodel.py Fri Oct 22 23:09:43 2010
@@ -17,6 +17,7 @@
from pypy.jit.backend.llsupport.descr import get_call_descr
from pypy.jit.backend.llsupport.descr import BaseIntCallDescr, GcPtrCallDescr
from pypy.jit.backend.llsupport.descr import FloatCallDescr, VoidCallDescr
+from pypy.jit.backend.llsupport.ffisupport import get_call_descr_dynamic
from pypy.rpython.annlowlevel import cast_instance_to_base_ptr
@@ -82,7 +83,8 @@
# read back by the machine code reading at the address given by
# pos_exception() and pos_exc_value().
_exception_emulator = lltype.malloc(rffi.CArray(lltype.Signed), 2,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self._exception_emulator = _exception_emulator
def _store_exception(lle):
@@ -210,7 +212,8 @@
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
- return ofs, size
+ sign = fielddescr.is_field_signed()
+ return ofs, size, sign
unpack_fielddescr_size._always_inline_ = True
def arraydescrof(self, A):
@@ -225,12 +228,16 @@
assert isinstance(arraydescr, BaseArrayDescr)
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
- return ofs, size
+ sign = arraydescr.is_item_signed()
+ return ofs, size, sign
unpack_arraydescr_size._always_inline_ = True
def calldescrof(self, FUNC, ARGS, RESULT, extrainfo=None):
return get_call_descr(self.gc_ll_descr, ARGS, RESULT, extrainfo)
+ def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+ return get_call_descr_dynamic(ffi_args, ffi_result, extrainfo)
+
def get_overflow_error(self):
ovf_vtable = self.cast_adr_to_int(self._ovf_error_vtable)
ovf_inst = lltype.cast_opaque_ptr(llmemory.GCREF,
@@ -252,15 +259,21 @@
@specialize.argtype(2)
def bh_getarrayitem_gc_i(self, arraydescr, gcref, itemindex):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- items = rffi.cast(rffi.CArrayPtr(TYPE), items)
- val = items[itemindex]
+ if sign:
+ items = rffi.cast(rffi.CArrayPtr(STYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ items = rffi.cast(rffi.CArrayPtr(UTYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -285,10 +298,10 @@
@specialize.argtype(2)
def bh_setarrayitem_gc_i(self, arraydescr, gcref, itemindex, newvalue):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
items = rffi.cast(rffi.CArrayPtr(TYPE), items)
items[itemindex] = rffi.cast(TYPE, newvalue)
@@ -339,14 +352,22 @@
@specialize.argtype(1)
def _base_do_getfield_i(self, struct, fielddescr):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- val = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)[0]
+ # Note that in the common case where size==sizeof(Signed),
+ # both cases of what follows are doing the same thing.
+ # But gcc is clever enough to figure this out :-)
+ if sign:
+ val = rffi.cast(rffi.CArrayPtr(STYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ val = rffi.cast(rffi.CArrayPtr(UTYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -378,10 +399,10 @@
@specialize.argtype(1)
def _base_do_setfield_i(self, struct, fielddescr, newvalue):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
fieldptr = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)
fieldptr[0] = rffi.cast(TYPE, newvalue)
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/symbolic.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/symbolic.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/symbolic.py Fri Oct 22 23:09:43 2010
@@ -69,8 +69,9 @@
SIZEOF_INT = get_size(rffi.INT, False)
SIZEOF_FLOAT = get_size(lltype.Float, False)
-unroll_basic_sizes = unrolling_iterable([(lltype.Signed, WORD),
- (lltype.Char, SIZEOF_CHAR),
- (rffi.SHORT, SIZEOF_SHORT),
- (rffi.INT, SIZEOF_INT)])
+unroll_basic_sizes = unrolling_iterable([
+ (lltype.Signed, lltype.Unsigned, WORD),
+ (rffi.SIGNEDCHAR, lltype.Char, SIZEOF_CHAR),
+ (rffi.SHORT, rffi.USHORT, SIZEOF_SHORT),
+ (rffi.INT, rffi.UINT, SIZEOF_INT)])
# does not contain Float ^^^ which must be special-cased
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py Fri Oct 22 23:09:43 2010
@@ -83,6 +83,18 @@
assert descr_f.is_float_field()
+def test_get_field_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr_x = get_field_descr(c2, S, 'x')
+ assert descr_x.is_field_signed() == signed
+
+
def test_get_array_descr():
U = lltype.Struct('U')
T = lltype.GcStruct('T')
@@ -164,6 +176,25 @@
assert descr.get_base_size(False) == 0
assert descr.get_ofs_length(False) == -1
+
+def test_get_array_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, A)
+ assert arraydescr.is_item_signed() == signed
+ #
+ RA = rffi.CArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, RA)
+ assert arraydescr.is_item_signed() == signed
+
+
def test_get_call_descr_not_translated():
c0 = GcCache(False)
descr1 = get_call_descr(c0, [lltype.Char, lltype.Signed], lltype.Char)
@@ -219,6 +250,17 @@
extrainfo = descr3.get_extra_info()
assert extrainfo is None
+def test_get_call_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr1 = get_call_descr(c2, [], RESTYPE)
+ assert descr1.is_result_signed() == signed
+
def test_repr_of_descr():
c0 = GcCache(False)
Modified: pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py Fri Oct 22 23:09:43 2010
@@ -9,12 +9,13 @@
ConstObj, BoxFloat, ConstFloat)
from pypy.jit.metainterp.resoperation import ResOperation, rop
from pypy.jit.metainterp.typesystem import deref
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi, rclass
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.llinterp import LLException
from pypy.jit.codewriter import heaptracker
+from pypy.rlib.rarithmetic import intmask
class Runner(object):
@@ -421,6 +422,7 @@
assert x == 3.5 - 42
def test_call(self):
+ from pypy.rlib.libffi import types
def func_int(a, b):
return a + b
@@ -428,23 +430,31 @@
return chr(ord(c) + ord(c1))
functions = [
- (func_int, lltype.Signed, 655360),
- (func_int, rffi.SHORT, 1213),
- (func_char, lltype.Char, 12)
+ (func_int, lltype.Signed, types.sint, 655360),
+ (func_int, rffi.SHORT, types.sint16, 1213),
+ (func_char, lltype.Char, types.uchar, 12)
]
- for func, TP, num in functions:
+ for func, TP, ffi_type, num in functions:
cpu = self.cpu
#
FPTR = self.Ptr(self.FuncType([TP, TP], TP))
func_ptr = llhelper(FPTR, func)
FUNC = deref(FPTR)
- calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
funcbox = self.get_funcbox(cpu, func_ptr)
+ # first, try it with the "normal" calldescr
+ calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
res = self.execute_operation(rop.CALL,
[funcbox, BoxInt(num), BoxInt(num)],
'int', descr=calldescr)
assert res.value == 2 * num
+ # then, try it with the dynamic calldescr
+ dyn_calldescr = cpu.calldescrof_dynamic([ffi_type, ffi_type], ffi_type)
+ res = self.execute_operation(rop.CALL,
+ [funcbox, BoxInt(num), BoxInt(num)],
+ 'int', descr=dyn_calldescr)
+ assert res.value == 2 * num
+
if cpu.supports_floats:
def func(f0, f1, f2, f3, f4, f5, f6, i0, i1, f7, f8, f9):
@@ -507,6 +517,24 @@
'int', descr=calldescr)
assert res.value == func_ints(*args)
+ def test_call_to_c_function(self):
+ from pypy.rlib.libffi import CDLL, types, ArgChain
+ from pypy.rpython.lltypesystem.ll2ctypes import libc_name
+ libc = CDLL(libc_name)
+ c_tolower = libc.getpointer('tolower', [types.uchar], types.sint)
+ argchain = ArgChain().arg(ord('A'))
+ assert c_tolower.call(argchain, rffi.INT) == ord('a')
+
+ func_adr = llmemory.cast_ptr_to_adr(c_tolower.funcsym)
+ funcbox = ConstInt(heaptracker.adr2int(func_adr))
+ calldescr = self.cpu.calldescrof_dynamic([types.uchar], types.sint)
+ res = self.execute_operation(rop.CALL,
+ [funcbox, BoxInt(ord('A'))],
+ 'int',
+ descr=calldescr)
+ assert res.value == ord('a')
+
+
def test_field_basic(self):
t_box, T_box = self.alloc_instance(self.T)
fielddescr = self.cpu.fielddescrof(self.S, 'value')
@@ -833,6 +861,23 @@
length_box], 'void')
assert self.look_string(r_box) == "!??cdef?!"
+ def test_copyunicodecontent(self):
+ s_box = self.alloc_unicode(u"abcdef")
+ for s_box in [s_box, s_box.constbox()]:
+ for srcstart_box in [BoxInt(2), ConstInt(2)]:
+ for dststart_box in [BoxInt(3), ConstInt(3)]:
+ for length_box in [BoxInt(4), ConstInt(4)]:
+ for r_box_is_const in [False, True]:
+ r_box = self.alloc_unicode(u"!???????!")
+ if r_box_is_const:
+ r_box = r_box.constbox()
+ self.execute_operation(rop.COPYUNICODECONTENT,
+ [s_box, r_box,
+ srcstart_box,
+ dststart_box,
+ length_box], 'void')
+ assert self.look_unicode(r_box) == u"!??cdef?!"
+
def test_do_unicode_basic(self):
u = self.cpu.bh_newunicode(5)
self.cpu.bh_unicodesetitem(u, 4, 123)
@@ -1227,6 +1272,10 @@
u_box = BoxPtr(lltype.cast_opaque_ptr(llmemory.GCREF, u))
return u_box
+ def look_unicode(self, unicode_box):
+ u = unicode_box.getref(lltype.Ptr(rstr.UNICODE))
+ return u''.join(u.chars)
+
def test_casts(self):
py.test.skip("xxx fix or kill")
@@ -1283,6 +1332,7 @@
descr=fd)
res = self.execute_operation(get_op, [s_box], 'int', descr=fd)
assert res.getint() == 32
+ lltype.free(s, flavor='raw')
def test_new_with_vtable(self):
cpu = self.cpu
@@ -1980,6 +2030,196 @@
assert self.cpu.get_latest_value_float(0) == 13.5
assert called
+ def test_short_result_of_getfield_direct(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getfield_gc_i(lltype.cast_opaque_ptr(llmemory.GCREF, s),
+ descrfld_x)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getfield_compiled(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
+ res = self.execute_operation(rop.GETFIELD_GC, [BoxPtr(s_gcref)],
+ 'int', descr=descrfld_x)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getarrayitem_gc_i(
+ descrarray, lltype.cast_opaque_ptr(llmemory.GCREF, a), 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getarrayitem_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, a)
+ res = self.execute_operation(rop.GETARRAYITEM_GC,
+ [BoxPtr(a_gcref), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_raw_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ x = cpu.bh_getarrayitem_raw_i(descrarray, a_rawint, 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_getarrayitem_raw_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ res = self.execute_operation(rop.GETARRAYITEM_RAW,
+ [BoxInt(a_rawint), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_call_direct(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ x = self.cpu.bh_call_i(self.get_funcbox(self.cpu, f).value,
+ calldescr, [value], None, None)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_call_compiled(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ funcbox = self.get_funcbox(self.cpu, f)
+ res = self.execute_operation(rop.CALL, [funcbox, BoxInt(value)],
+ 'int', descr=calldescr)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
class OOtypeBackendTest(BaseBackendTest):
Modified: pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py Fri Oct 22 23:09:43 2010
@@ -599,7 +599,7 @@
OPERATIONS.append(StrLenOperation(rop.STRLEN))
OPERATIONS.append(UnicodeLenOperation(rop.UNICODELEN))
OPERATIONS.append(CopyStrContentOperation(rop.COPYSTRCONTENT))
- #OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT))
+ OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT))
for i in range(2):
OPERATIONS.append(GuardClassOperation(rop.GUARD_CLASS))
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py Fri Oct 22 23:09:43 2010
@@ -8,7 +8,8 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.tool.uid import fixid
from pypy.jit.backend.x86.regalloc import (RegAlloc, X86RegisterManager,
- X86XMMRegisterManager, get_ebp_ofs)
+ X86XMMRegisterManager, get_ebp_ofs,
+ _get_scale)
from pypy.jit.backend.x86.arch import (FRAME_FIXED_SIZE, FORCE_INDEX_OFS, WORD,
IS_X86_32, IS_X86_64)
@@ -22,7 +23,8 @@
X86_64_SCRATCH_REG,
X86_64_XMM_SCRATCH_REG,
RegLoc, StackLoc, ConstFloatLoc,
- ImmedLoc, AddressLoc, imm)
+ ImmedLoc, AddressLoc, imm,
+ imm0, imm1)
from pypy.rlib.objectmodel import we_are_translated, specialize
from pypy.jit.backend.x86 import rx86, regloc, codebuf
@@ -244,12 +246,13 @@
f = open_file_as_stream(output_log, "w")
for i in range(len(self.loop_run_counters)):
name, struct = self.loop_run_counters[i]
- f.write(str(struct.i) + " " * (8 - len(str(struct.i))) + name + "\n")
+ f.write(str(name) + ":" + str(struct.i) + "\n")
f.close()
def _build_float_constants(self):
# 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment
- addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw')
+ addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw',
+ track_allocation=False)
if not we_are_translated():
self._keepalive_malloced_float_consts = addr
float_constants = rffi.cast(lltype.Signed, addr)
@@ -399,9 +402,10 @@
funcname = "<loop %d>" % len(self.loop_run_counters)
# invent the counter, so we don't get too confused
if self._debug:
- struct = lltype.malloc(DEBUG_COUNTER, flavor='raw')
+ struct = lltype.malloc(DEBUG_COUNTER, flavor='raw',
+ track_allocation=False) # known to leak
struct.i = 0
- self.loop_run_counters.append((funcname, struct))
+ self.loop_run_counters.append((len(self.loop_run_counters), struct))
return funcname
def patch_jump_for_descr(self, faildescr, adr_new_target):
@@ -440,7 +444,7 @@
# self.mc.PUSH(eax)
# adr = rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])
# self.mc.MOV(eax, heap(adr))
- # self.mc.ADD(eax, imm(1))
+ # self.mc.ADD(eax, imm1)
# self.mc.MOV(heap(adr), eax)
# self.mc.POP(eax)
return operations
@@ -711,7 +715,7 @@
self.regalloc_perform_with_guard(None, guard_op, faillocs, arglocs,
resloc, current_depths)
- def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm(0)):
+ def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm0):
self.mc.LEA(result, addr_add(frm, sizereg, baseofs, scale))
def _unaryop(asmop):
@@ -973,28 +977,28 @@
def genop_guard_int_is_true(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
return self.implement_guard(guard_token, 'Z')
else:
return self.implement_guard(guard_token, 'NZ')
def genop_int_is_true(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['NE'], rl.value)
self.mc.MOVZX8(resloc, rl)
def genop_guard_int_is_zero(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
return self.implement_guard(guard_token, 'NZ')
else:
return self.implement_guard(guard_token, 'Z')
def genop_int_is_zero(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['E'], rl.value)
self.mc.MOVZX8(resloc, rl)
@@ -1050,50 +1054,66 @@
assert result_loc is eax
self.call(self.malloc_unicode_func_addr, arglocs, eax)
- def genop_getfield_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, size_loc = arglocs
- assert isinstance(size_loc, ImmedLoc)
+ # ----------
+
+ def load_from_mem(self, resloc, source_addr, size_loc, sign_loc):
assert isinstance(resloc, RegLoc)
size = size_loc.value
-
- source_addr = AddressLoc(base_loc, ofs_loc)
+ sign = sign_loc.value
if resloc.is_xmm:
self.mc.MOVSD(resloc, source_addr)
+ elif size == WORD:
+ self.mc.MOV(resloc, source_addr)
+ elif size == 1:
+ if sign:
+ self.mc.MOVSX8(resloc, source_addr)
+ else:
+ self.mc.MOVZX8(resloc, source_addr)
+ elif size == 2:
+ if sign:
+ self.mc.MOVSX16(resloc, source_addr)
+ else:
+ self.mc.MOVZX16(resloc, source_addr)
+ elif IS_X86_64 and size == 4:
+ if sign:
+ self.mc.MOVSX32(resloc, source_addr)
+ else:
+ self.mc.MOV32(resloc, source_addr) # zero-extending
+ else:
+ not_implemented("load_from_mem size = %d" % size)
+
+ def save_into_mem(self, dest_addr, value_loc, size_loc):
+ size = size_loc.value
+ if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
+ self.mc.MOVSD(dest_addr, value_loc)
elif size == 1:
- self.mc.MOVZX8(resloc, source_addr)
+ self.mc.MOV8(dest_addr, value_loc.lowest8bits())
elif size == 2:
- self.mc.MOVZX16(resloc, source_addr)
+ self.mc.MOV16(dest_addr, value_loc)
elif size == 4:
- # MOV32 is zero-extending on 64-bit, so this is okay
- self.mc.MOV32(resloc, source_addr)
+ self.mc.MOV32(dest_addr, value_loc)
elif IS_X86_64 and size == 8:
- self.mc.MOV(resloc, source_addr)
+ self.mc.MOV(dest_addr, value_loc)
else:
- raise NotImplementedError("getfield size = %d" % size)
+ not_implemented("save_into_mem size = %d" % size)
+
+ def genop_getfield_gc(self, op, arglocs, resloc):
+ base_loc, ofs_loc, size_loc, sign_loc = arglocs
+ assert isinstance(size_loc, ImmedLoc)
+ source_addr = AddressLoc(base_loc, ofs_loc)
+ self.load_from_mem(resloc, source_addr, size_loc, sign_loc)
genop_getfield_raw = genop_getfield_gc
genop_getfield_raw_pure = genop_getfield_gc
genop_getfield_gc_pure = genop_getfield_gc
def genop_getarrayitem_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, scale, ofs = arglocs
+ base_loc, ofs_loc, size_loc, ofs, sign_loc = arglocs
assert isinstance(ofs, ImmedLoc)
- assert isinstance(scale, ImmedLoc)
- src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale.value)
- if op.result.type == FLOAT:
- self.mc.MOVSD(resloc, src_addr)
- else:
- if scale.value == 0:
- self.mc.MOVZX8(resloc, src_addr)
- elif scale.value == 1:
- self.mc.MOVZX16(resloc, src_addr)
- elif scale.value == 2:
- self.mc.MOV32(resloc, src_addr)
- elif IS_X86_64 and scale.value == 3:
- self.mc.MOV(resloc, src_addr)
- else:
- print "[asmgen]getarrayitem unsupported size: %d" % scale.value
- raise NotImplementedError()
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale)
+ self.load_from_mem(resloc, src_addr, size_loc, sign_loc)
genop_getarrayitem_gc_pure = genop_getarrayitem_gc
genop_getarrayitem_raw = genop_getarrayitem_gc
@@ -1101,40 +1121,16 @@
def genop_discard_setfield_gc(self, op, arglocs):
base_loc, ofs_loc, size_loc, value_loc = arglocs
assert isinstance(size_loc, ImmedLoc)
- size = size_loc.value
dest_addr = AddressLoc(base_loc, ofs_loc)
- if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
- self.mc.MOVSD(dest_addr, value_loc)
- elif IS_X86_64 and size == 8:
- self.mc.MOV(dest_addr, value_loc)
- elif size == 4:
- self.mc.MOV32(dest_addr, value_loc)
- elif size == 2:
- self.mc.MOV16(dest_addr, value_loc)
- elif size == 1:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- print "[asmgen]setfield addr size %d" % size
- raise NotImplementedError("Addr size %d" % size)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_setarrayitem_gc(self, op, arglocs):
- base_loc, ofs_loc, value_loc, scale_loc, baseofs = arglocs
+ base_loc, ofs_loc, value_loc, size_loc, baseofs = arglocs
assert isinstance(baseofs, ImmedLoc)
- assert isinstance(scale_loc, ImmedLoc)
- dest_addr = AddressLoc(base_loc, ofs_loc, scale_loc.value, baseofs.value)
- if op.getarg(2).type == FLOAT:
- self.mc.MOVSD(dest_addr, value_loc)
- else:
- if IS_X86_64 and scale_loc.value == 3:
- self.mc.MOV(dest_addr, value_loc)
- elif scale_loc.value == 2:
- self.mc.MOV32(dest_addr, value_loc)
- elif scale_loc.value == 1:
- self.mc.MOV16(dest_addr, value_loc)
- elif scale_loc.value == 0:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- raise NotImplementedError("scale = %d" % scale_loc.value)
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ dest_addr = AddressLoc(base_loc, ofs_loc, scale, baseofs.value)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_strsetitem(self, op, arglocs):
base_loc, ofs_loc, val_loc = arglocs
@@ -1201,7 +1197,7 @@
def genop_guard_guard_no_exception(self, ign_1, guard_op, guard_token,
locs, ign_2):
- self.mc.CMP(heap(self.cpu.pos_exception()), imm(0))
+ self.mc.CMP(heap(self.cpu.pos_exception()), imm0)
return self.implement_guard(guard_token, 'NZ')
def genop_guard_guard_exception(self, ign_1, guard_op, guard_token,
@@ -1213,8 +1209,8 @@
addr = self.implement_guard(guard_token, 'NE')
if resloc is not None:
self.mc.MOV(resloc, heap(self.cpu.pos_exc_value()))
- self.mc.MOV(heap(self.cpu.pos_exception()), imm(0))
- self.mc.MOV(heap(self.cpu.pos_exc_value()), imm(0))
+ self.mc.MOV(heap(self.cpu.pos_exception()), imm0)
+ self.mc.MOV(heap(self.cpu.pos_exc_value()), imm0)
return addr
def _gen_guard_overflow(self, guard_op, guard_token):
@@ -1224,8 +1220,8 @@
elif guard_opnum == rop.GUARD_OVERFLOW:
return self.implement_guard(guard_token, 'NO')
else:
- print "int_xxx_ovf followed by", guard_op.getopname()
- raise AssertionError
+ not_implemented("int_xxx_ovf followed by %s" %
+ guard_op.getopname())
def genop_guard_int_add_ovf(self, op, guard_op, guard_token, arglocs, result_loc):
self.genop_int_add(op, arglocs, result_loc)
@@ -1288,7 +1284,7 @@
def genop_guard_guard_nonnull_class(self, ign_1, guard_op,
guard_token, locs, ign_2):
self.mc.ensure_bytes_available(256)
- self.mc.CMP(locs[0], imm(1))
+ self.mc.CMP(locs[0], imm1)
# Patched below
self.mc.J_il8(rx86.Conditions['B'], 0)
jb_location = self.mc.get_relative_pos()
@@ -1637,25 +1633,34 @@
sizeloc = arglocs[0]
assert isinstance(sizeloc, ImmedLoc)
size = sizeloc.value
+ signloc = arglocs[1]
if isinstance(op.getarg(0), Const):
x = imm(op.getarg(0).getint())
else:
- x = arglocs[1]
+ x = arglocs[2]
if x is eax:
tmp = ecx
else:
tmp = eax
- self._emit_call(x, arglocs, 2, tmp=tmp)
+ self._emit_call(x, arglocs, 3, tmp=tmp)
+
+ if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
+ self.mc.FSTP_b(resloc.value) # float return
+ elif size == WORD:
+ assert resloc is eax or resloc is xmm0 # a full word
+ elif size == 0:
+ pass # void return
+ else:
+ # use the code in load_from_mem to do the zero- or sign-extension
+ assert resloc is eax
+ if size == 1:
+ srcloc = eax.lowest8bits()
+ else:
+ srcloc = eax
+ self.load_from_mem(eax, srcloc, sizeloc, signloc)
- if isinstance(resloc, StackLoc) and resloc.width == 8 and IS_X86_32:
- self.mc.FSTP_b(resloc.value)
- elif size == 1:
- self.mc.AND_ri(eax.value, 0xff)
- elif size == 2:
- self.mc.AND_ri(eax.value, 0xffff)
-
def genop_guard_call_may_force(self, op, guard_op, guard_token,
arglocs, result_loc):
faildescr = guard_op.getdescr()
@@ -1808,20 +1813,16 @@
self.mc.LEA_rb(resloc.value, FORCE_INDEX_OFS)
def not_implemented_op_discard(self, op, arglocs):
- msg = "not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op(self, op, arglocs, resloc):
- msg = "not implemented operation with res: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with res: %s" %
+ op.getopname())
def not_implemented_op_guard(self, op, guard_op,
failaddr, arglocs, resloc):
- msg = "not implemented operation (guard): %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation (guard): %s" %
+ op.getopname())
def mark_gc_roots(self):
gcrootmap = self.cpu.gc_ll_descr.gcrootmap
@@ -1905,3 +1906,7 @@
def heap(addr):
return AddressLoc(ImmedLoc(addr), ImmedLoc(0), 0, 0)
+
+def not_implemented(msg):
+ os.write(2, '[x86/asm] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py Fri Oct 22 23:09:43 2010
@@ -2,6 +2,7 @@
""" Register allocation scheme.
"""
+import os
from pypy.jit.metainterp.history import (Box, Const, ConstInt, ConstPtr,
ResOperation, BoxPtr,
LoopToken, INT, REF, FLOAT)
@@ -40,12 +41,10 @@
return imm(c.value)
elif isinstance(c, ConstPtr):
if we_are_translated() and c.value and rgc.can_move(c.value):
- print "convert_to_imm: ConstPtr needs special care"
- raise AssertionError
+ not_implemented("convert_to_imm: ConstPtr needs special care")
return imm(rffi.cast(lltype.Signed, c.value))
else:
- print "convert_to_imm: got a %s" % c
- raise AssertionError
+ not_implemented("convert_to_imm: got a %s" % c)
class X86_64_RegisterManager(X86RegisterManager):
# r11 omitted because it's used as scratch
@@ -70,8 +69,9 @@
def _get_new_array(self):
n = self.BASE_CONSTANT_SIZE
+ # known to leak
self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n,
- flavor='raw')
+ flavor='raw', track_allocation=False)
self.cur_array_free = n
_get_new_array._dont_inline_ = True
@@ -359,8 +359,8 @@
if op.is_ovf():
if (operations[i + 1].getopnum() != rop.GUARD_NO_OVERFLOW and
operations[i + 1].getopnum() != rop.GUARD_OVERFLOW):
- print "int_xxx_ovf not followed by guard_(no)_overflow"
- raise AssertionError
+ not_implemented("int_xxx_ovf not followed by "
+ "guard_(no)_overflow")
return True
return False
if (operations[i + 1].getopnum() != rop.GUARD_TRUE and
@@ -412,8 +412,8 @@
arg = op.getarg(j)
if isinstance(arg, Box):
if arg not in start_live:
- print "Bogus arg in operation %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in operation %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
if op.is_guard():
for arg in op.getfailargs():
@@ -421,8 +421,8 @@
continue
assert isinstance(arg, Box)
if arg not in start_live:
- print "Bogus arg in guard %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in guard %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
for arg in inputargs:
if arg not in longevity:
@@ -667,7 +667,13 @@
assert isinstance(calldescr, BaseCallDescr)
assert len(calldescr.arg_classes) == op.numargs() - 1
size = calldescr.get_result_size(self.translate_support_code)
- self._call(op, [imm(size)] + [self.loc(op.getarg(i)) for i in range(op.numargs())],
+ sign = calldescr.is_result_signed()
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self._call(op, [imm(size), sign_loc] +
+ [self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_not_forced_op)
def consider_call(self, op):
@@ -688,7 +694,7 @@
self.rm._sync_var(op.getarg(vable_index))
vable = self.fm.loc(op.getarg(vable_index))
else:
- vable = imm(0)
+ vable = imm0
self._call(op, [imm(size), vable] +
[self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_op)
@@ -778,15 +784,11 @@
loc = self.loc(op.getarg(0))
return self._call(op, [loc])
# boehm GC (XXX kill the following code at some point)
- ofs_items, itemsize, ofs = symbolic.get_array_token(rstr.UNICODE, self.translate_support_code)
- if itemsize == 4:
- return self._malloc_varsize(ofs_items, ofs, 2, op.getarg(0),
- op.result)
- elif itemsize == 2:
- return self._malloc_varsize(ofs_items, ofs, 1, op.getarg(0),
- op.result)
- else:
- assert False, itemsize
+ ofs_items, _, ofs = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ scale = self._get_unicode_item_scale()
+ return self._malloc_varsize(ofs_items, ofs, scale, op.getarg(0),
+ op.result)
def _malloc_varsize(self, ofs_items, ofs_length, scale, v, res_v):
# XXX kill this function at some point
@@ -818,8 +820,9 @@
arglocs.append(self.loc(op.getarg(0)))
return self._call(op, arglocs)
# boehm GC (XXX kill the following code at some point)
- scale_of_field, basesize, ofs_length, _ = (
+ itemsize, basesize, ofs_length, _, _ = (
self._unpack_arraydescr(op.getdescr()))
+ scale_of_field = _get_scale(itemsize)
return self._malloc_varsize(basesize, ofs_length, scale_of_field,
op.getarg(0), op.result)
@@ -829,21 +832,19 @@
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
ptr = arraydescr.is_array_of_pointers()
- scale = 0
- while (1 << scale) < size:
- scale += 1
- assert (1 << scale) == size
- return scale, ofs, ofs_length, ptr
+ sign = arraydescr.is_item_signed()
+ return size, ofs, ofs_length, ptr, sign
def _unpack_fielddescr(self, fielddescr):
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
ptr = fielddescr.is_pointer_field()
- return imm(ofs), imm(size), ptr
+ sign = fielddescr.is_field_signed()
+ return imm(ofs), imm(size), ptr, sign
def consider_setfield_gc(self, op):
- ofs_loc, size_loc, ptr = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, _ = self._unpack_fielddescr(op.getdescr())
assert isinstance(size_loc, ImmedLoc)
if size_loc.value == 1:
need_lower_byte = True
@@ -870,10 +871,10 @@
consider_unicodesetitem = consider_strsetitem
def consider_setarrayitem_gc(self, op):
- scale, ofs, _, ptr = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, _ = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
- if scale == 0:
+ if itemsize == 1:
need_lower_byte = True
else:
need_lower_byte = False
@@ -882,30 +883,39 @@
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.possibly_free_vars(args)
self.PerformDiscard(op, [base_loc, ofs_loc, value_loc,
- imm(scale), imm(ofs)])
+ imm(itemsize), imm(ofs)])
consider_setarrayitem_raw = consider_setarrayitem_gc
def consider_getfield_gc(self, op):
- ofs_loc, size_loc, _ = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, sign = self._unpack_fielddescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
self.rm.possibly_free_vars(args)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, size_loc], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, size_loc, sign_loc], result_loc)
consider_getfield_raw = consider_getfield_gc
consider_getfield_raw_pure = consider_getfield_gc
consider_getfield_gc_pure = consider_getfield_gc
def consider_getarrayitem_gc(self, op):
- scale, ofs, _, _ = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, sign = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.rm.possibly_free_vars_for_op(op)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, imm(scale), imm(ofs)], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, imm(itemsize), imm(ofs),
+ sign_loc], result_loc)
consider_getarrayitem_raw = consider_getarrayitem_gc
consider_getarrayitem_gc_pure = consider_getarrayitem_gc
@@ -959,6 +969,12 @@
consider_unicodegetitem = consider_strgetitem
def consider_copystrcontent(self, op):
+ self._consider_copystrcontent(op, is_unicode=False)
+
+ def consider_copyunicodecontent(self, op):
+ self._consider_copystrcontent(op, is_unicode=True)
+
+ def _consider_copystrcontent(self, op, is_unicode):
# compute the source address
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(args[0], args)
@@ -970,7 +986,8 @@
srcaddr_box = TempBox()
forbidden_vars = [args[1], args[3], args[4], srcaddr_box]
srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box, forbidden_vars)
- self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc)
+ self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc,
+ is_unicode=is_unicode)
# compute the destination address
base_loc = self.rm.make_sure_var_in_reg(args[1], forbidden_vars)
ofs_loc = self.rm.make_sure_var_in_reg(args[3], forbidden_vars)
@@ -980,25 +997,57 @@
forbidden_vars = [args[4], srcaddr_box]
dstaddr_box = TempBox()
dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box, forbidden_vars)
- self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc)
+ self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc,
+ is_unicode=is_unicode)
+ # compute the length in bytes
+ length_box = args[4]
+ length_loc = self.loc(length_box)
+ if is_unicode:
+ self.rm.possibly_free_var(length_box)
+ forbidden_vars = [srcaddr_box, dstaddr_box]
+ bytes_box = TempBox()
+ bytes_loc = self.rm.force_allocate_reg(bytes_box, forbidden_vars)
+ scale = self._get_unicode_item_scale()
+ if not (isinstance(length_loc, ImmedLoc) or
+ isinstance(length_loc, RegLoc)):
+ self.assembler.mov(length_loc, bytes_loc)
+ length_loc = bytes_loc
+ self.assembler.load_effective_addr(length_loc, 0, scale, bytes_loc)
+ length_box = bytes_box
+ length_loc = bytes_loc
# call memcpy()
- length_loc = self.loc(args[4])
self.rm.before_call()
self.xrm.before_call()
self.assembler._emit_call(imm(self.assembler.memcpy_addr),
[dstaddr_loc, srcaddr_loc, length_loc])
- self.rm.possibly_free_var(args[4])
+ self.rm.possibly_free_var(length_box)
self.rm.possibly_free_var(dstaddr_box)
self.rm.possibly_free_var(srcaddr_box)
- def _gen_address_inside_string(self, baseloc, ofsloc, resloc):
+ def _gen_address_inside_string(self, baseloc, ofsloc, resloc, is_unicode):
cpu = self.assembler.cpu
- ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR,
+ if is_unicode:
+ ofs_items, _, _ = symbolic.get_array_token(rstr.UNICODE,
self.translate_support_code)
- assert itemsize == 1
- self.assembler.load_effective_addr(ofsloc, ofs_items, 0,
+ scale = self._get_unicode_item_scale()
+ else:
+ ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR,
+ self.translate_support_code)
+ assert itemsize == 1
+ scale = 0
+ self.assembler.load_effective_addr(ofsloc, ofs_items, scale,
resloc, baseloc)
+ def _get_unicode_item_scale(self):
+ _, itemsize, _ = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ if itemsize == 4:
+ return 2
+ elif itemsize == 2:
+ return 1
+ else:
+ raise AssertionError("bad unicode item size")
+
def consider_jump(self, op):
assembler = self.assembler
assert self.jump_target_descr is None
@@ -1033,6 +1082,9 @@
def consider_debug_merge_point(self, op):
pass
+ def consider_jit_debug(self, op):
+ pass
+
def get_mark_gc_roots(self, gcrootmap):
shape = gcrootmap.get_basic_shape(IS_X86_64)
for v, val in self.fm.frame_bindings.items():
@@ -1052,15 +1104,11 @@
self.Perform(op, [], loc)
def not_implemented_op(self, op):
- msg = "[regalloc] Not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op_with_guard(self, op, guard_op):
- msg = "[regalloc] Not implemented operation with guard: %s" % (
- op.getopname(),)
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with guard: %s" % (
+ op.getopname(),))
oplist = [RegAlloc.not_implemented_op] * rop._LAST
oplist_with_guard = [RegAlloc.not_implemented_op_with_guard] * rop._LAST
@@ -1094,3 +1142,14 @@
# Returns (ebp-20), (ebp-24), (ebp-28)...
# i.e. the n'th word beyond the fixed frame size.
return -WORD * (FRAME_FIXED_SIZE + position)
+
+def _get_scale(size):
+ assert size == 1 or size == 2 or size == 4 or size == 8
+ if size < 4:
+ return size - 1 # 1, 2 => 0, 1
+ else:
+ return (size >> 2) + 1 # 4, 8 => 2, 3
+
+def not_implemented(msg):
+ os.write(2, '[x86/regalloc] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/regloc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/regloc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/regloc.py Fri Oct 22 23:09:43 2010
@@ -442,8 +442,11 @@
MOV8 = _binaryop('MOV8')
MOV16 = _16_bit_binaryop('MOV')
MOVZX8 = _binaryop('MOVZX8')
+ MOVSX8 = _binaryop('MOVSX8')
MOVZX16 = _binaryop('MOVZX16')
+ MOVSX16 = _binaryop('MOVSX16')
MOV32 = _binaryop('MOV32')
+ MOVSX32 = _binaryop('MOVSX32')
XCHG = _binaryop('XCHG')
PUSH = _unaryop('PUSH')
@@ -473,6 +476,9 @@
else:
return ImmedLoc(x)
+imm0 = imm(0)
+imm1 = imm(1)
+
all_extra_instructions = [name for name in LocationCodeBuilder.__dict__
if name[0].isupper()]
all_extra_instructions.sort()
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py Fri Oct 22 23:09:43 2010
@@ -113,7 +113,8 @@
return CPU386.cast_adr_to_int(adr)
all_null_registers = lltype.malloc(rffi.LONGP.TO, 24,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
def force(self, addr_of_force_index):
TP = rffi.CArrayPtr(lltype.Signed)
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/rx86.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/rx86.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/rx86.py Fri Oct 22 23:09:43 2010
@@ -642,7 +642,10 @@
define_modrm_modes('MOV8_*i', [rex_w, '\xC6', orbyte(0<<3)], [immediate(2, 'b')], regtype='BYTE')
define_modrm_modes('MOVZX8_r*', [rex_w, '\x0F\xB6', register(1, 8)], regtype='BYTE')
+define_modrm_modes('MOVSX8_r*', [rex_w, '\x0F\xBE', register(1, 8)], regtype='BYTE')
define_modrm_modes('MOVZX16_r*', [rex_w, '\x0F\xB7', register(1, 8)])
+define_modrm_modes('MOVSX16_r*', [rex_w, '\x0F\xBF', register(1, 8)])
+define_modrm_modes('MOVSX32_r*', [rex_w, '\x63', register(1, 8)])
define_modrm_modes('MOVSD_x*', ['\xF2', rex_nw, '\x0F\x10', register(1,8)], regtype='XMM')
define_modrm_modes('MOVSD_*x', ['\xF2', rex_nw, '\x0F\x11', register(2,8)], regtype='XMM')
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_assembler.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_assembler.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_assembler.py Fri Oct 22 23:09:43 2010
@@ -81,7 +81,8 @@
# also test rebuild_faillocs_from_descr(), which should not
# reproduce the holes at all
- bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw')
+ bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw',
+ immortal=True)
for i in range(len(mc.content)):
assert 0 <= mc.content[i] <= 255
bytecode[i] = rffi.cast(rffi.UCHAR, mc.content[i])
@@ -115,7 +116,8 @@
assert withfloats
value = random.random() - 0.5
# make sure it fits into 64 bits
- tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw')
+ tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw',
+ track_allocation=False)
rffi.cast(rffi.DOUBLEP, tmp)[0] = value
return rffi.cast(rffi.DOUBLEP, tmp)[0], tmp[0], tmp[1]
@@ -152,10 +154,12 @@
# prepare the expected target arrays, the descr_bytecode,
# the 'registers' and the 'stack' arrays according to 'content'
- xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1, flavor='raw')
+ xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1,
+ flavor='raw', immortal=True)
registers = rffi.ptradd(xmmregisters, 16)
stacklen = baseloc + 10
- stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw')
+ stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw',
+ immortal=True)
expected_ints = [0] * len(content)
expected_ptrs = [lltype.nullptr(llmemory.GCREF.TO)] * len(content)
expected_floats = [0.0] * len(content)
@@ -221,7 +225,7 @@
descr_bytecode.append(0x00)
descr_bytecode.append(0xCC) # end marker
descr_bytes = lltype.malloc(rffi.UCHARP.TO, len(descr_bytecode),
- flavor='raw')
+ flavor='raw', immortal=True)
for i in range(len(descr_bytecode)):
assert 0 <= descr_bytecode[i] <= 255
descr_bytes[i] = rffi.cast(rffi.UCHAR, descr_bytecode[i])
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_gc_integration.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_gc_integration.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_gc_integration.py Fri Oct 22 23:09:43 2010
@@ -12,7 +12,7 @@
from pypy.jit.backend.detect_cpu import getcpuclass
from pypy.jit.backend.x86.regalloc import RegAlloc
from pypy.jit.backend.x86.arch import WORD, FRAME_FIXED_SIZE
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import rclass, rstr
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py Fri Oct 22 23:09:43 2010
@@ -11,7 +11,7 @@
from pypy.jit.backend.x86.regalloc import RegAlloc, X86RegisterManager,\
FloatConstants, is_comparison_or_ovf_op
from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import rclass, rstr
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py Fri Oct 22 23:09:43 2010
@@ -10,7 +10,7 @@
from pypy.jit.metainterp.resoperation import rop
from pypy.jit.metainterp.executor import execute
from pypy.jit.backend.test.runner_test import LLtypeBackendTest
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.tool.udir import udir
import ctypes
import sys
@@ -506,8 +506,8 @@
self.cpu.execute_token(ops.token)
# check debugging info
name, struct = self.cpu.assembler.loop_run_counters[0]
- assert name == 'xyz'
+ assert name == 0 # 'xyz'
assert struct.i == 10
self.cpu.finish_once()
lines = py.path.local(self.logfile + ".count").readlines()
- assert lines[0] == '10 xyz\n'
+ assert lines[0] == '0:10\n' # '10 xyz\n'
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_string.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_string.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_string.py Fri Oct 22 23:09:43 2010
@@ -2,8 +2,12 @@
from pypy.jit.metainterp.test import test_string
from pypy.jit.backend.x86.test.test_basic import Jit386Mixin
-class TestString(Jit386Mixin, test_string.StringTests):
+class TestString(Jit386Mixin, test_string.TestLLtype):
# for the individual tests see
# ====> ../../../metainterp/test/test_string.py
- CALL = 'call'
- CALL_PURE = 'call_pure'
+ pass
+
+class TestUnicode(Jit386Mixin, test_string.TestLLtypeUnicode):
+ # for the individual tests see
+ # ====> ../../../metainterp/test/test_string.py
+ pass
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py Fri Oct 22 23:09:43 2010
@@ -191,6 +191,33 @@
def run_orig(self, name, n, x):
self.main_allfuncs(name, n, x)
+ def define_libffi_workaround(cls):
+ # XXX: this is a workaround for a bug in database.py. It seems that
+ # the problem is triggered by optimizeopt/fficall.py, and in
+ # particular by the ``cast_base_ptr_to_instance(Func, llfunc)``: in
+ # these tests, that line is the only place where libffi.Func is
+ # referenced.
+ #
+ # The problem occurs because the gctransformer tries to annotate a
+ # low-level helper to call the __del__ of libffi.Func when it's too
+ # late.
+ #
+ # This workaround works by forcing the annotator (and all the rest of
+ # the toolchain) to see libffi.Func in a "proper" context, not just as
+ # the target of cast_base_ptr_to_instance. Note that the function
+ # below is *never* called by any actual test, it's just annotated.
+ #
+ from pypy.rlib.libffi import get_libc_name, CDLL, types, ArgChain
+ libc_name = get_libc_name()
+ def f(n, x, *args):
+ libc = CDLL(libc_name)
+ ptr = libc.getpointer('labs', [types.slong], types.slong)
+ chain = ArgChain()
+ chain.arg(n)
+ n = ptr.call(chain, lltype.Signed)
+ return (n, x) + args
+ return None, f, None
+
def define_compile_framework_1(cls):
# a moving GC. Supports malloc_varsize_nonmovable. Simple test, works
# without write_barriers and root stack enumeration.
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py Fri Oct 22 23:09:43 2010
@@ -1,4 +1,4 @@
-import py, os
+import py, os, sys
from pypy.tool.udir import udir
from pypy.rlib.jit import JitDriver, OPTIMIZER_FULL, unroll_parameters
from pypy.rlib.jit import PARAMETERS, dont_look_inside
@@ -63,8 +63,32 @@
if k - abs(j): raise ValueError
if k - abs(-j): raise ValueError
return total * 10
+ #
+ from pypy.rpython.lltypesystem import lltype, rffi
+ from pypy.rlib.libffi import types, CDLL, ArgChain
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ jitdriver2 = JitDriver(greens=[], reds = ['i', 'func', 'res', 'x'])
+ def libffi_stuff(i, j):
+ lib = CDLL(libm_name)
+ func = lib.getpointer('fabs', [types.double], types.double)
+ res = 0.0
+ x = float(j)
+ while i > 0:
+ jitdriver2.jit_merge_point(i=i, res=res, func=func, x=x)
+ jitdriver2.can_enter_jit(i=i, res=res, func=func, x=x)
+ func = hint(func, promote=True)
+ argchain = ArgChain()
+ argchain.arg(x)
+ res = func.call(argchain, rffi.DOUBLE)
+ i -= 1
+ return res
+ #
+ def main(i, j):
+ return f(i, j) + libffi_stuff(i, j)
+ expected = f(40, -49)
res = self.meta_interp(f, [40, -49])
- assert res == f(40, -49)
+ assert res == expected
def test_direct_assembler_call_translates(self):
class Thing(object):
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/call.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/call.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/call.py Fri Oct 22 23:09:43 2010
@@ -277,3 +277,11 @@
return seen.pop()
else:
return None
+
+ def could_be_green_field(self, GTYPE, fieldname):
+ GTYPE_fieldname = (GTYPE, fieldname)
+ for jd in self.jitdrivers_sd:
+ if jd.greenfield_info is not None:
+ if GTYPE_fieldname in jd.greenfield_info.green_fields:
+ return True
+ return False
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py Fri Oct 22 23:09:43 2010
@@ -95,18 +95,18 @@
print '%s:' % (ssarepr.name,)
print format_assembler(ssarepr)
else:
- dir = udir.ensure("jitcodes", dir=1)
- if portal_jitdriver:
- name = "%02d_portal_runner" % (portal_jitdriver.index,)
- elif ssarepr.name and ssarepr.name != '?':
- name = ssarepr.name
- else:
- name = 'unnamed' % id(ssarepr)
- i = 1
- extra = ''
- while name+extra in self._seen_files:
- i += 1
- extra = '.%d' % i
- self._seen_files.add(name+extra)
- dir.join(name+extra).write(format_assembler(ssarepr))
log.dot()
+ dir = udir.ensure("jitcodes", dir=1)
+ if portal_jitdriver:
+ name = "%02d_portal_runner" % (portal_jitdriver.index,)
+ elif ssarepr.name and ssarepr.name != '?':
+ name = ssarepr.name
+ else:
+ name = 'unnamed' % id(ssarepr)
+ i = 1
+ extra = ''
+ while name+extra in self._seen_files:
+ i += 1
+ extra = '.%d' % i
+ self._seen_files.add(name+extra)
+ dir.join(name+extra).write(format_assembler(ssarepr))
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py Fri Oct 22 23:09:43 2010
@@ -18,19 +18,34 @@
# the 'oopspecindex' field is one of the following values:
OS_NONE = 0 # normal case, no oopspec
OS_ARRAYCOPY = 1 # "list.ll_arraycopy"
- OS_STR_CONCAT = 2 # "stroruni.concat"
- OS_UNI_CONCAT = 3 # "stroruni.concat"
- OS_STR_SLICE = 4 # "stroruni.slice"
- OS_UNI_SLICE = 5 # "stroruni.slice"
- OS_STR_EQUAL = 6 # "stroruni.equal"
- OS_UNI_EQUAL = 7 # "stroruni.equal"
- OS_STREQ_SLICE_CHECKNULL = 8 # s2!=NULL and s1[x:x+length]==s2
- OS_STREQ_SLICE_NONNULL = 9 # s1[x:x+length]==s2 (assert s2!=NULL)
- OS_STREQ_SLICE_CHAR = 10 # s1[x:x+length]==char
- OS_STREQ_NONNULL = 11 # s1 == s2 (assert s1!=NULL,s2!=NULL)
- OS_STREQ_NONNULL_CHAR = 12 # s1 == char (assert s1!=NULL)
- OS_STREQ_CHECKNULL_CHAR = 13 # s1!=NULL and s1==char
- OS_STREQ_LENGTHOK = 14 # s1 == s2 (assert len(s1)==len(s2))
+ OS_STR2UNICODE = 2 # "str.str2unicode"
+ #
+ OS_STR_CONCAT = 22 # "stroruni.concat"
+ OS_STR_SLICE = 23 # "stroruni.slice"
+ OS_STR_EQUAL = 24 # "stroruni.equal"
+ OS_STREQ_SLICE_CHECKNULL = 25 # s2!=NULL and s1[x:x+length]==s2
+ OS_STREQ_SLICE_NONNULL = 26 # s1[x:x+length]==s2 (assert s2!=NULL)
+ OS_STREQ_SLICE_CHAR = 27 # s1[x:x+length]==char
+ OS_STREQ_NONNULL = 28 # s1 == s2 (assert s1!=NULL,s2!=NULL)
+ OS_STREQ_NONNULL_CHAR = 29 # s1 == char (assert s1!=NULL)
+ OS_STREQ_CHECKNULL_CHAR = 30 # s1!=NULL and s1==char
+ OS_STREQ_LENGTHOK = 31 # s1 == s2 (assert len(s1)==len(s2))
+ #
+ OS_UNI_CONCAT = 42 #
+ OS_UNI_SLICE = 43 #
+ OS_UNI_EQUAL = 44 #
+ OS_UNIEQ_SLICE_CHECKNULL = 45 #
+ OS_UNIEQ_SLICE_NONNULL = 46 #
+ OS_UNIEQ_SLICE_CHAR = 47 #
+ OS_UNIEQ_NONNULL = 48 # the same for unicode
+ OS_UNIEQ_NONNULL_CHAR = 49 # (must be the same amount as for
+ OS_UNIEQ_CHECKNULL_CHAR = 50 # STR, in the same order)
+ OS_UNIEQ_LENGTHOK = 51 #
+ _OS_offset_uni = OS_UNI_CONCAT - OS_STR_CONCAT
+ #
+ OS_LIBFFI_PREPARE = 60
+ OS_LIBFFI_PUSH_ARG = 61
+ OS_LIBFFI_CALL = 62
def __new__(cls, readonly_descrs_fields,
write_descrs_fields, write_descrs_arrays,
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py Fri Oct 22 23:09:43 2010
@@ -316,8 +316,14 @@
prepare = self._handle_list_call
elif oopspec_name.startswith('stroruni.'):
prepare = self._handle_stroruni_call
+ elif oopspec_name == 'str.str2unicode':
+ prepare = self._handle_str2unicode_call
elif oopspec_name.startswith('virtual_ref'):
prepare = self._handle_virtual_ref_call
+ elif oopspec_name.startswith('jit.'):
+ prepare = self._handle_jit_call
+ elif oopspec_name.startswith('libffi_'):
+ prepare = self._handle_libffi_call
else:
prepare = self.prepare_builtin_call
try:
@@ -427,7 +433,8 @@
op.result)
def rewrite_op_free(self, op):
- assert op.args[1].value == 'raw'
+ flags = op.args[1].value
+ assert flags['flavor'] == 'raw'
ARRAY = op.args[0].concretetype.TO
return self._do_builtin_call(op, 'raw_free', [op.args[0]],
extra = (ARRAY,), extrakey = ARRAY)
@@ -519,7 +526,12 @@
# check for deepfrozen structures that force constant-folding
immut = v_inst.concretetype.TO._immutable_field(c_fieldname.value)
if immut:
- pure = '_pure'
+ if (self.callcontrol is not None and
+ self.callcontrol.could_be_green_field(v_inst.concretetype.TO,
+ c_fieldname.value)):
+ pure = '_greenfield'
+ else:
+ pure = '_pure'
if immut == "[*]":
self.immutable_arrays[op.result] = True
else:
@@ -819,6 +831,8 @@
def rewrite_op_jit_marker(self, op):
key = op.args[0].value
jitdriver = op.args[1].value
+ if not jitdriver.active:
+ return []
return getattr(self, 'handle_jit_marker__%s' % key)(op, jitdriver)
def handle_jit_marker__jit_merge_point(self, op, jitdriver):
@@ -852,6 +866,15 @@
(self.graph,))
return []
+ def _handle_jit_call(self, op, oopspec_name, args):
+ if oopspec_name == 'jit.debug':
+ return SpaceOperation('jit_debug', args, None)
+ elif oopspec_name == 'jit.assert_green':
+ kind = getkind(args[0].concretetype)
+ return SpaceOperation('%s_assert_green' % kind, args, None)
+ else:
+ raise AssertionError("missing support for %r" % oopspec_name)
+
# ----------
# Lists.
@@ -1028,8 +1051,10 @@
# ----------
# Strings and Unicodes.
- def _handle_oopspec_call(self, op, args, oopspecindex):
+ def _handle_oopspec_call(self, op, args, oopspecindex, extraeffect=None):
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
+ if extraeffect:
+ calldescr.get_extra_info().extraeffect = extraeffect
if isinstance(op.args[0].value, str):
pass # for tests only
else:
@@ -1055,28 +1080,32 @@
[c_func] + [varoftype(T) for T in argtypes],
varoftype(resulttype))
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
- func = heaptracker.adr2int(
- llmemory.cast_ptr_to_adr(c_func.value))
+ if isinstance(c_func.value, str): # in tests only
+ func = c_func.value
+ else:
+ func = heaptracker.adr2int(
+ llmemory.cast_ptr_to_adr(c_func.value))
_callinfo_for_oopspec[oopspecindex] = calldescr, func
def _handle_stroruni_call(self, op, oopspec_name, args):
- if args[0].concretetype.TO == rstr.STR:
+ SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
+ if SoU.TO == rstr.STR:
dict = {"stroruni.concat": EffectInfo.OS_STR_CONCAT,
"stroruni.slice": EffectInfo.OS_STR_SLICE,
"stroruni.equal": EffectInfo.OS_STR_EQUAL,
}
- elif args[0].concretetype.TO == rstr.UNICODE:
+ CHR = lltype.Char
+ elif SoU.TO == rstr.UNICODE:
dict = {"stroruni.concat": EffectInfo.OS_UNI_CONCAT,
"stroruni.slice": EffectInfo.OS_UNI_SLICE,
"stroruni.equal": EffectInfo.OS_UNI_EQUAL,
}
+ CHR = lltype.UniChar
else:
assert 0, "args[0].concretetype must be STR or UNICODE"
#
if oopspec_name == "stroruni.equal":
- SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
for otherindex, othername, argtypes, resulttype in [
-
(EffectInfo.OS_STREQ_SLICE_CHECKNULL,
"str.eq_slice_checknull",
[SoU, lltype.Signed, lltype.Signed, SoU],
@@ -1087,7 +1116,7 @@
lltype.Signed),
(EffectInfo.OS_STREQ_SLICE_CHAR,
"str.eq_slice_char",
- [SoU, lltype.Signed, lltype.Signed, lltype.Char],
+ [SoU, lltype.Signed, lltype.Signed, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL,
"str.eq_nonnull",
@@ -1095,22 +1124,27 @@
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL_CHAR,
"str.eq_nonnull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_CHECKNULL_CHAR,
"str.eq_checknull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_LENGTHOK,
"str.eq_lengthok",
[SoU, SoU],
lltype.Signed),
]:
+ if args[0].concretetype.TO == rstr.UNICODE:
+ otherindex += EffectInfo._OS_offset_uni
self._register_extra_helper(otherindex, othername,
argtypes, resulttype)
#
return self._handle_oopspec_call(op, args, dict[oopspec_name])
+ def _handle_str2unicode_call(self, op, oopspec_name, args):
+ return self._handle_oopspec_call(op, args, EffectInfo.OS_STR2UNICODE)
+
# ----------
# VirtualRefs.
@@ -1121,6 +1155,23 @@
vrefinfo.JIT_VIRTUAL_REF)
return SpaceOperation(oopspec_name, list(args), op.result)
+ # -----------
+ # rlib.libffi
+
+ def _handle_libffi_call(self, op, oopspec_name, args):
+ if oopspec_name == 'libffi_prepare_call':
+ oopspecindex = EffectInfo.OS_LIBFFI_PREPARE
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
+ elif oopspec_name.startswith('libffi_push_'):
+ oopspecindex = EffectInfo.OS_LIBFFI_PUSH_ARG
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
+ elif oopspec_name.startswith('libffi_call_'):
+ oopspecindex = EffectInfo.OS_LIBFFI_CALL
+ extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
+ else:
+ assert False, 'unsupported oopspec: %s' % oopspec_name
+ return self._handle_oopspec_call(op, args, oopspecindex, extraeffect)
+
def rewrite_op_jit_force_virtual(self, op):
return self._do_builtin_call(op)
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/support.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/support.py Fri Oct 22 23:09:43 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, rclass
+from pypy.rpython.lltypesystem import lltype, rclass, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython import rlist
from pypy.rpython.lltypesystem import rstr as ll_rstr, rdict as ll_rdict
@@ -8,6 +8,7 @@
from pypy.rpython.ootypesystem import rdict as oo_rdict
from pypy.rpython.llinterp import LLInterpreter
from pypy.rpython.extregistry import ExtRegistryEntry
+from pypy.rpython.annlowlevel import cast_base_ptr_to_instance
from pypy.translator.simplify import get_funcobj
from pypy.translator.unsimplify import split_block
from pypy.objspace.flow.model import Constant
@@ -60,7 +61,7 @@
return rtyper.annotator.translator.graphs[0]
def split_before_jit_merge_point(graph, portalblock, portalopindex):
- """Find the block with 'jit_merge_point' and split just before,
+ """Split the block just before the 'jit_merge_point',
making sure the input args are in the canonical order.
"""
# split the block just before the jit_merge_point()
@@ -217,6 +218,33 @@
else:
return x
+
+# libffi support
+# --------------
+
+def func(llfunc):
+ from pypy.rlib.libffi import Func
+ return cast_base_ptr_to_instance(Func, llfunc)
+
+def _ll_1_libffi_prepare_call(llfunc):
+ return func(llfunc)._prepare()
+
+def _ll_4_libffi_push_int(llfunc, value, ll_args, i):
+ return func(llfunc)._push_int(value, ll_args, i)
+
+def _ll_4_libffi_push_float(llfunc, value, ll_args, i):
+ return func(llfunc)._push_float(value, ll_args, i)
+
+def _ll_3_libffi_call_int(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.LONG)
+
+def _ll_3_libffi_call_float(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.DOUBLE)
+
+def _ll_3_libffi_call_void(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, lltype.Void)
+
+
# in the following calls to builtins, the JIT is allowed to look inside:
inline_calls_to = [
('int_floordiv_ovf_zer', [lltype.Signed, lltype.Signed], lltype.Signed),
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/test/test_codewriter.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/test/test_codewriter.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/test/test_codewriter.py Fri Oct 22 23:09:43 2010
@@ -45,6 +45,7 @@
self.portal_graph = portal_graph
self.portal_runner_ptr = "???"
self.virtualizable_info = None
+ self.greenfield_info = None
def test_loop():
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/test/test_effectinfo.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/test/test_effectinfo.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/test/test_effectinfo.py Fri Oct 22 23:09:43 2010
@@ -1,7 +1,8 @@
from pypy.rpython.lltypesystem.rclass import OBJECT
from pypy.rpython.lltypesystem import lltype
from pypy.rpython.ootypesystem import ootype
-from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze
+from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze,\
+ EffectInfo
class FakeCPU:
def fielddescrof(self, T, fieldname):
@@ -9,6 +10,14 @@
def arraydescrof(self, A):
return ('arraydescr', A)
+def test_no_oopspec_duplicate():
+ # check that all the various EffectInfo.OS_* have unique values
+ oopspecs = set()
+ for name, value in EffectInfo.__dict__.iteritems():
+ if name.startswith('OS_'):
+ assert value not in oopspecs
+ oopspecs.add(value)
+
def test_include_read_field():
S = lltype.GcStruct("S", ("a", lltype.Signed))
effects = frozenset([("readstruct", lltype.Ptr(S), "a")])
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py Fri Oct 22 23:09:43 2010
@@ -77,7 +77,32 @@
class FakeBuiltinCallControl:
def guess_call_kind(self, op):
return 'builtin'
- def getcalldescr(self, op, oopspecindex):
+ def getcalldescr(self, op, oopspecindex=None):
+ assert oopspecindex is not None # in this test
+ EI = effectinfo.EffectInfo
+ if oopspecindex != EI.OS_ARRAYCOPY:
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ INT = lltype.Signed
+ UNICHAR = lltype.UniChar
+ argtypes = {
+ EI.OS_STR2UNICODE:([PSTR], PUNICODE),
+ EI.OS_STR_CONCAT: ([PSTR, PSTR], PSTR),
+ EI.OS_STR_SLICE: ([PSTR, INT, INT], PSTR),
+ EI.OS_UNI_CONCAT: ([PUNICODE, PUNICODE], PUNICODE),
+ EI.OS_UNI_SLICE: ([PUNICODE, INT, INT], PUNICODE),
+ EI.OS_UNI_EQUAL: ([PUNICODE, PUNICODE], lltype.Bool),
+ EI.OS_UNIEQ_SLICE_CHECKNULL:([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_NONNULL: ([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_CHAR: ([PUNICODE, INT, INT, UNICHAR], INT),
+ EI.OS_UNIEQ_NONNULL: ([PUNICODE, PUNICODE], INT),
+ EI.OS_UNIEQ_NONNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_CHECKNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_LENGTHOK: ([PUNICODE, PUNICODE], INT),
+ }
+ argtypes = argtypes[oopspecindex]
+ assert argtypes[0] == [v.concretetype for v in op.args[1:]]
+ assert argtypes[1] == op.result.concretetype
return 'calldescr-%d' % oopspecindex
def calldescr_canraise(self, calldescr):
return False
@@ -662,6 +687,79 @@
assert block.operations[1].result is None
assert block.exits[0].args == [v1]
+def test_jit_merge_point_1():
+ class FakeJitDriverSD:
+ index = 42
+ class jitdriver:
+ greens = ['green1', 'green2', 'voidgreen3']
+ reds = ['red1', 'red2', 'voidred3']
+ jd = FakeJitDriverSD()
+ v1 = varoftype(lltype.Signed)
+ v2 = varoftype(lltype.Signed)
+ vvoid1 = varoftype(lltype.Void)
+ v3 = varoftype(lltype.Signed)
+ v4 = varoftype(lltype.Signed)
+ vvoid2 = varoftype(lltype.Void)
+ v5 = varoftype(lltype.Void)
+ op = SpaceOperation('jit_marker',
+ [Constant('jit_merge_point', lltype.Void),
+ Constant(jd.jitdriver, lltype.Void),
+ v1, v2, vvoid1, v3, v4, vvoid2], v5)
+ tr = Transformer()
+ tr.portal_jd = jd
+ oplist = tr.rewrite_operation(op)
+ assert len(oplist) == 6
+ assert oplist[0].opname == '-live-'
+ assert oplist[1].opname == 'int_guard_value'
+ assert oplist[1].args == [v1]
+ assert oplist[2].opname == '-live-'
+ assert oplist[3].opname == 'int_guard_value'
+ assert oplist[3].args == [v2]
+ assert oplist[4].opname == 'jit_merge_point'
+ assert oplist[4].args[0].value == 42
+ assert list(oplist[4].args[1]) == [v1, v2]
+ assert list(oplist[4].args[4]) == [v3, v4]
+ assert oplist[5].opname == '-live-'
+
+def test_getfield_gc():
+ S = lltype.GcStruct('S', ('x', lltype.Char))
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_pure():
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_pure'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_greenfield():
+ class FakeCC:
+ def get_vinfo(self, v):
+ return None
+ def could_be_green_field(self, S1, name1):
+ assert S1 is S
+ assert name1 == 'x'
+ return True
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU(), FakeCC()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_greenfield'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
def test_int_abs():
v1 = varoftype(lltype.Signed)
v2 = varoftype(lltype.Signed)
@@ -766,6 +864,46 @@
assert op1.args[3] == ListOfKind('ref', [v1])
assert op1.result == v4
+def test_str2unicode():
+ # test that the oopspec is present and correctly transformed
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PSTR], PUNICODE)
+ func = lltype.functionptr(FUNC, 'll_str2unicode',
+ _callable=rstr.LLHelpers.ll_str2unicode)
+ v1 = varoftype(PSTR)
+ v2 = varoftype(PUNICODE)
+ op = SpaceOperation('direct_call', [const(func), v1], v2)
+ tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_r'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_STR2UNICODE
+ assert op1.args[2] == ListOfKind('ref', [v1])
+ assert op1.result == v2
+
+def test_unicode_eq_checknull_char():
+ # test that the oopspec is present and correctly transformed
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PUNICODE, PUNICODE], lltype.Bool)
+ func = lltype.functionptr(FUNC, 'll_streq',
+ _callable=rstr.LLHelpers.ll_streq)
+ v1 = varoftype(PUNICODE)
+ v2 = varoftype(PUNICODE)
+ v3 = varoftype(lltype.Bool)
+ op = SpaceOperation('direct_call', [const(func), v1, v2], v3)
+ tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_i'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_UNI_EQUAL
+ assert op1.args[2] == ListOfKind('ref', [v1, v2])
+ assert op1.result == v3
+ # test that the OS_UNIEQ_* functions are registered
+ cifo = effectinfo._callinfo_for_oopspec
+ assert effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL in cifo
+ assert effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR in cifo
+
def test_list_ll_arraycopy():
from pypy.rlib.rgc import ll_arraycopy
LIST = lltype.GcArray(lltype.Signed)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py Fri Oct 22 23:09:43 2010
@@ -760,6 +760,20 @@
def bhimpl_debug_fatalerror(msg):
llop.debug_fatalerror(lltype.Void, msg)
+ @arguments("r", "i", "i", "i", "i")
+ def bhimpl_jit_debug(string, arg1=0, arg2=0, arg3=0, arg4=0):
+ pass
+
+ @arguments("i")
+ def bhimpl_int_assert_green(x):
+ pass
+ @arguments("r")
+ def bhimpl_ref_assert_green(x):
+ pass
+ @arguments("f")
+ def bhimpl_float_assert_green(x):
+ pass
+
# ----------
# the main hints and recursive calls
@@ -1073,6 +1087,10 @@
bhimpl_getfield_vable_r = bhimpl_getfield_gc_r
bhimpl_getfield_vable_f = bhimpl_getfield_gc_f
+ bhimpl_getfield_gc_i_greenfield = bhimpl_getfield_gc_i
+ bhimpl_getfield_gc_r_greenfield = bhimpl_getfield_gc_r
+ bhimpl_getfield_gc_f_greenfield = bhimpl_getfield_gc_f
+
@arguments("cpu", "i", "d", returns="i")
def bhimpl_getfield_raw_i(cpu, struct, fielddescr):
return cpu.bh_getfield_raw_i(struct, fielddescr)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/compile.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/compile.py Fri Oct 22 23:09:43 2010
@@ -370,7 +370,8 @@
from pypy.jit.metainterp.resume import force_from_resumedata
metainterp_sd = self.metainterp_sd
vinfo = self.jitdriver_sd.virtualizable_info
- all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo)
+ ginfo = self.jitdriver_sd.greenfield_info
+ all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo, ginfo)
# The virtualizable data was stored on the real virtualizable above.
# Handle all_virtuals: keep them for later blackholing from the
# future failure of the GUARD_NOT_FORCED
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/executor.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/executor.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/executor.py Fri Oct 22 23:09:43 2010
@@ -80,6 +80,9 @@
do_call_loopinvariant = do_call
do_call_may_force = do_call
+def do_call_c(cpu, metainterp, argboxes, descr):
+ raise NotImplementedError("Should never be called directly")
+
def do_getarrayitem_gc(cpu, _, arraybox, indexbox, arraydescr):
array = arraybox.getref_base()
index = indexbox.getint()
@@ -304,6 +307,7 @@
rop.CALL_ASSEMBLER,
rop.COND_CALL_GC_WB,
rop.DEBUG_MERGE_POINT,
+ rop.JIT_DEBUG,
rop.SETARRAYITEM_RAW,
): # list of opcodes never executed by pyjitpl
continue
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/graphpage.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/graphpage.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/graphpage.py Fri Oct 22 23:09:43 2010
@@ -153,7 +153,7 @@
opindex = opstartindex
while True:
op = operations[opindex]
- lines.append(repr(op))
+ lines.append(op.repr(graytext=True))
if is_interesting_guard(op):
tgt = op.getdescr()._debug_suboperations[0]
tgt_g, tgt_i = self.all_operations[tgt]
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/history.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/history.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/history.py Fri Oct 22 23:09:43 2010
@@ -698,6 +698,21 @@
return result
_const_ptr_for_string = {}
+def get_const_ptr_for_unicode(s):
+ from pypy.rpython.annlowlevel import llunicode
+ if not we_are_translated():
+ try:
+ return _const_ptr_for_unicode[s]
+ except KeyError:
+ pass
+ if isinstance(s, str):
+ s = unicode(s)
+ result = ConstPtr(lltype.cast_opaque_ptr(llmemory.GCREF, llunicode(s)))
+ if not we_are_translated():
+ _const_ptr_for_unicode[s] = result
+ return result
+_const_ptr_for_unicode = {}
+
# ____________________________________________________________
# The TreeLoop class contains a loop or a generalized loop, i.e. a tree
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/jitdriver.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/jitdriver.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/jitdriver.py Fri Oct 22 23:09:43 2010
@@ -13,8 +13,10 @@
# self.num_red_args ... pypy.jit.metainterp.warmspot
# self.result_type ... pypy.jit.metainterp.warmspot
# self.virtualizable_info... pypy.jit.metainterp.warmspot
+ # self.greenfield_info ... pypy.jit.metainterp.warmspot
# self.warmstate ... pypy.jit.metainterp.warmspot
# self.handle_jitexc_from_bh pypy.jit.metainterp.warmspot
+ # self.no_loop_header ... pypy.jit.metainterp.warmspot
# self.portal_finishtoken... pypy.jit.metainterp.pyjitpl
# self.index ... pypy.jit.codewriter.call
# self.mainjitcode ... pypy.jit.codewriter.call
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimize_nopspec.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimize_nopspec.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimize_nopspec.py Fri Oct 22 23:09:43 2010
@@ -14,6 +14,9 @@
def _optimize_loop(metainterp_sd, old_loop_tokens, loop):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(loop.inputargs, loop.operations)
+ # XXX the following lines are probably still needed, to discard invalid
+ # loops. bit silly to run a full perfect specialization and throw the
+ # result away.
finder = PerfectSpecializationFinder(cpu)
finder.find_nodes_loop(loop, False)
if old_loop_tokens:
@@ -31,6 +34,7 @@
def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations)
+ # XXX same comment as above applies
finder = BridgeSpecializationFinder(cpu)
finder.find_nodes_bridge(bridge)
if old_loop_tokens:
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/__init__.py Fri Oct 22 23:09:43 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.optimizeopt.intbounds import OptIntBounds
from pypy.jit.metainterp.optimizeopt.virtualize import OptVirtualize
from pypy.jit.metainterp.optimizeopt.heap import OptHeap
+from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall
from pypy.jit.metainterp.optimizeopt.string import OptString
def optimize_loop_1(metainterp_sd, loop, virtuals=True):
@@ -16,6 +17,7 @@
OptVirtualize(),
OptString(),
OptHeap(),
+ OptFfiCall(),
]
optimizer = Optimizer(metainterp_sd, loop, optimizations, virtuals)
optimizer.propagate_all_forward()
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/intbounds.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/intbounds.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/intbounds.py Fri Oct 22 23:09:43 2010
@@ -191,6 +191,7 @@
v1.intbound.make_ge(IntLowerBound(0))
optimize_STRLEN = optimize_ARRAYLEN_GC
+ optimize_UNICODELEN = optimize_ARRAYLEN_GC
def make_int_lt(self, box1, box2):
v1 = self.getvalue(box1)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py Fri Oct 22 23:09:43 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.history import ConstInt
from pypy.jit.metainterp.optimizeutil import _findall
from pypy.jit.metainterp.resoperation import rop, ResOperation
+from pypy.jit.codewriter.effectinfo import EffectInfo
class OptRewrite(Optimization):
"""Rewrite operations into equivalent, cheaper operations.
@@ -245,6 +246,9 @@
def optimize_CALL_LOOPINVARIANT(self, op):
funcvalue = self.getvalue(op.getarg(0))
if not funcvalue.is_constant():
+ # XXX this code path is never executed in tests nor in production.
+ # in fact, it can't even happen since residual_call in codewriter
+ # expects a compile-time constant
self.emit_operation(op)
return
key = make_hashable_int(op.getarg(0).getint())
@@ -323,8 +327,37 @@
## return
## self.emit_operation(op)
-optimize_ops = _findall(OptRewrite, 'optimize_')
-
+ def optimize_CALL(self, op):
+ # dispatch based on 'oopspecindex' to a method that handles
+ # specifically the given oopspec call. For non-oopspec calls,
+ # oopspecindex is just zero.
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ oopspecindex = effectinfo.oopspecindex
+ if oopspecindex == EffectInfo.OS_ARRAYCOPY:
+ if self._optimize_CALL_ARRAYCOPY(op):
+ return
+ self.emit_operation(op)
+ def _optimize_CALL_ARRAYCOPY(self, op):
+ source_value = self.getvalue(op.getarg(1))
+ dest_value = self.getvalue(op.getarg(2))
+ source_start_box = self.get_constant_box(op.getarg(3))
+ dest_start_box = self.get_constant_box(op.getarg(4))
+ length = self.get_constant_box(op.getarg(5))
+ if (source_value.is_virtual() and source_start_box and dest_start_box
+ and length and dest_value.is_virtual()):
+ # XXX optimize the case where dest value is not virtual,
+ # but we still can avoid a mess
+ source_start = source_start_box.getint()
+ dest_start = dest_start_box.getint()
+ for index in range(length.getint()):
+ val = source_value.getitem(index + source_start)
+ dest_value.setitem(index + dest_start, val)
+ return True
+ if length and length.getint() == 0:
+ return True # 0-length arraycopy
+ return False
+
+optimize_ops = _findall(OptRewrite, 'optimize_')
-
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py Fri Oct 22 23:09:43 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.history import Box, BoxInt, BoxPtr
from pypy.jit.metainterp.history import Const, ConstInt, ConstPtr
from pypy.jit.metainterp.history import get_const_ptr_for_string
+from pypy.jit.metainterp.history import get_const_ptr_for_unicode
from pypy.jit.metainterp.resoperation import rop, ResOperation
from pypy.jit.metainterp.optimizeopt import optimizer, virtualize
from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1
@@ -11,56 +12,106 @@
from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec
from pypy.jit.codewriter import heaptracker
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib.objectmodel import specialize, we_are_translated
+
+
+class StrOrUnicode(object):
+ def __init__(self, LLTYPE, hlstr, emptystr, chr,
+ NEWSTR, STRLEN, STRGETITEM, STRSETITEM, COPYSTRCONTENT,
+ OS_offset):
+ self.LLTYPE = LLTYPE
+ self.hlstr = hlstr
+ self.emptystr = emptystr
+ self.chr = chr
+ self.NEWSTR = NEWSTR
+ self.STRLEN = STRLEN
+ self.STRGETITEM = STRGETITEM
+ self.STRSETITEM = STRSETITEM
+ self.COPYSTRCONTENT = COPYSTRCONTENT
+ self.OS_offset = OS_offset
+
+ def _freeze_(self):
+ return True
+
+mode_string = StrOrUnicode(rstr.STR, annlowlevel.hlstr, '', chr,
+ rop.NEWSTR, rop.STRLEN, rop.STRGETITEM,
+ rop.STRSETITEM, rop.COPYSTRCONTENT, 0)
+mode_unicode = StrOrUnicode(rstr.UNICODE, annlowlevel.hlunicode, u'', unichr,
+ rop.NEWUNICODE, rop.UNICODELEN, rop.UNICODEGETITEM,
+ rop.UNICODESETITEM, rop.COPYUNICODECONTENT,
+ EffectInfo._OS_offset_uni)
+
+# ____________________________________________________________
class __extend__(optimizer.OptValue):
"""New methods added to the base class OptValue for this file."""
- def getstrlen(self, newoperations):
- s = self.get_constant_string()
- if s is not None:
- return ConstInt(len(s))
+ def getstrlen(self, newoperations, mode):
+ if mode is mode_string:
+ s = self.get_constant_string_spec(mode_string)
+ if s is not None:
+ return ConstInt(len(s))
else:
- if newoperations is None:
- return None
- self.ensure_nonnull()
- box = self.force_box()
- lengthbox = BoxInt()
- newoperations.append(ResOperation(rop.STRLEN, [box], lengthbox))
- return lengthbox
+ s = self.get_constant_string_spec(mode_unicode)
+ if s is not None:
+ return ConstInt(len(s))
+ if newoperations is None:
+ return None
+ self.ensure_nonnull()
+ box = self.force_box()
+ lengthbox = BoxInt()
+ newoperations.append(ResOperation(mode.STRLEN, [box], lengthbox))
+ return lengthbox
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
if self.is_constant():
- s = self.box.getref(lltype.Ptr(rstr.STR))
- return annlowlevel.hlstr(s)
+ s = self.box.getref(lltype.Ptr(mode.LLTYPE))
+ return mode.hlstr(s)
else:
return None
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
# Copies the pointer-to-string 'self' into the target string
# given by 'targetbox', at the specified offset. Returns the offset
# at the end of the copy.
- lengthbox = self.getstrlen(newoperations)
+ lengthbox = self.getstrlen(newoperations, mode)
srcbox = self.force_box()
return copy_str_content(newoperations, srcbox, targetbox,
- CONST_0, offsetbox, lengthbox)
+ CONST_0, offsetbox, lengthbox, mode)
class VAbstractStringValue(virtualize.AbstractVirtualValue):
- _attrs_ = ()
+ _attrs_ = ('mode',)
+
+ def __init__(self, optimizer, keybox, source_op, mode):
+ virtualize.AbstractVirtualValue.__init__(self, optimizer, keybox,
+ source_op)
+ self.mode = mode
def _really_force(self):
- s = self.get_constant_string()
- if s is not None:
- c_s = get_const_ptr_for_string(s)
- self.make_constant(c_s)
- return
+ if self.mode is mode_string:
+ s = self.get_constant_string_spec(mode_string)
+ if s is not None:
+ c_s = get_const_ptr_for_string(s)
+ self.make_constant(c_s)
+ return
+ else:
+ s = self.get_constant_string_spec(mode_unicode)
+ if s is not None:
+ c_s = get_const_ptr_for_unicode(s)
+ self.make_constant(c_s)
+ return
assert self.source_op is not None
self.box = box = self.source_op.result
newoperations = self.optimizer.newoperations
- lengthbox = self.getstrlen(newoperations)
- newoperations.append(ResOperation(rop.NEWSTR, [lengthbox], box))
- self.string_copy_parts(newoperations, box, CONST_0)
+ lengthbox = self.getstrlen(newoperations, self.mode)
+ op = ResOperation(self.mode.NEWSTR, [lengthbox], box)
+ if not we_are_translated():
+ op.name = 'FORCE'
+ newoperations.append(op)
+ self.string_copy_parts(newoperations, box, CONST_0, self.mode)
class VStringPlainValue(VAbstractStringValue):
@@ -74,7 +125,7 @@
assert 0 <= start <= stop <= len(longerlist)
self._chars = longerlist[start:stop]
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
if self._lengthbox is None:
self._lengthbox = ConstInt(len(self._chars))
return self._lengthbox
@@ -86,18 +137,21 @@
assert isinstance(charvalue, optimizer.OptValue)
self._chars[index] = charvalue
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
for c in self._chars:
if c is optimizer.CVAL_UNINITIALIZED_ZERO or not c.is_constant():
return None
- return ''.join([chr(c.box.getint()) for c in self._chars])
+ return mode.emptystr.join([mode.chr(c.box.getint())
+ for c in self._chars])
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
for i in range(len(self._chars)):
charbox = self._chars[i].force_box()
- newoperations.append(ResOperation(rop.STRSETITEM, [targetbox,
- offsetbox,
- charbox], None))
+ newoperations.append(ResOperation(mode.STRSETITEM, [targetbox,
+ offsetbox,
+ charbox],
+ None))
offsetbox = _int_add(newoperations, offsetbox, CONST_1)
return offsetbox
@@ -109,7 +163,7 @@
value.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrplain()
+ return modifier.make_vstrplain(self.mode is mode_unicode)
class VStringConcatValue(VAbstractStringValue):
@@ -120,23 +174,24 @@
self.right = right
self.lengthbox = lengthbox
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
return self.lengthbox
- def get_constant_string(self):
- s1 = self.left.get_constant_string()
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
+ s1 = self.left.get_constant_string_spec(mode)
if s1 is None:
return None
- s2 = self.right.get_constant_string()
+ s2 = self.right.get_constant_string_spec(mode)
if s2 is None:
return None
return s1 + s2
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
offsetbox = self.left.string_copy_parts(newoperations, targetbox,
- offsetbox)
+ offsetbox, mode)
offsetbox = self.right.string_copy_parts(newoperations, targetbox,
- offsetbox)
+ offsetbox, mode)
return offsetbox
def get_args_for_fail(self, modifier):
@@ -150,7 +205,7 @@
self.right.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrconcat()
+ return modifier.make_vstrconcat(self.mode is mode_unicode)
class VStringSliceValue(VAbstractStringValue):
@@ -162,12 +217,13 @@
self.vstart = vstart
self.vlength = vlength
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
return self.vlength.force_box()
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
if self.vstart.is_constant() and self.vlength.is_constant():
- s1 = self.vstr.get_constant_string()
+ s1 = self.vstr.get_constant_string_spec(mode)
if s1 is None:
return None
start = self.vstart.box.getint()
@@ -177,12 +233,12 @@
return s1[start : start + length]
return None
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
- lengthbox = self.getstrlen(newoperations)
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
+ lengthbox = self.getstrlen(newoperations, mode)
return copy_str_content(newoperations,
self.vstr.force_box(), targetbox,
self.vstart.force_box(), offsetbox,
- lengthbox)
+ lengthbox, mode)
def get_args_for_fail(self, modifier):
if self.box is None and not modifier.already_seen_virtual(self.keybox):
@@ -195,11 +251,11 @@
self.vlength.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrslice()
+ return modifier.make_vstrslice(self.mode is mode_unicode)
def copy_str_content(newoperations, srcbox, targetbox,
- srcoffsetbox, offsetbox, lengthbox):
+ srcoffsetbox, offsetbox, lengthbox, mode):
if isinstance(srcbox, ConstPtr) and isinstance(srcoffsetbox, Const):
M = 5
else:
@@ -208,17 +264,18 @@
# up to M characters are done "inline", i.e. with STRGETITEM/STRSETITEM
# instead of just a COPYSTRCONTENT.
for i in range(lengthbox.value):
- charbox = _strgetitem(newoperations, srcbox, srcoffsetbox)
+ charbox = _strgetitem(newoperations, srcbox, srcoffsetbox, mode)
srcoffsetbox = _int_add(newoperations, srcoffsetbox, CONST_1)
- newoperations.append(ResOperation(rop.STRSETITEM, [targetbox,
- offsetbox,
- charbox], None))
+ newoperations.append(ResOperation(mode.STRSETITEM, [targetbox,
+ offsetbox,
+ charbox],
+ None))
offsetbox = _int_add(newoperations, offsetbox, CONST_1)
else:
nextoffsetbox = _int_add(newoperations, offsetbox, lengthbox)
- op = ResOperation(rop.COPYSTRCONTENT, [srcbox, targetbox,
- srcoffsetbox, offsetbox,
- lengthbox], None)
+ op = ResOperation(mode.COPYSTRCONTENT, [srcbox, targetbox,
+ srcoffsetbox, offsetbox,
+ lengthbox], None)
newoperations.append(op)
offsetbox = nextoffsetbox
return offsetbox
@@ -245,12 +302,16 @@
newoperations.append(ResOperation(rop.INT_SUB, [box1, box2], resbox))
return resbox
-def _strgetitem(newoperations, strbox, indexbox):
+def _strgetitem(newoperations, strbox, indexbox, mode):
if isinstance(strbox, ConstPtr) and isinstance(indexbox, ConstInt):
- s = strbox.getref(lltype.Ptr(rstr.STR))
- return ConstInt(ord(s.chars[indexbox.getint()]))
+ if mode is mode_string:
+ s = strbox.getref(lltype.Ptr(rstr.STR))
+ return ConstInt(ord(s.chars[indexbox.getint()]))
+ else:
+ s = strbox.getref(lltype.Ptr(rstr.UNICODE))
+ return ConstInt(ord(s.chars[indexbox.getint()]))
resbox = BoxInt()
- newoperations.append(ResOperation(rop.STRGETITEM, [strbox, indexbox],
+ newoperations.append(ResOperation(mode.STRGETITEM, [strbox, indexbox],
resbox))
return resbox
@@ -258,62 +319,34 @@
class OptString(optimizer.Optimization):
"Handling of strings and unicodes."
- def make_vstring_plain(self, box, source_op=None):
- vvalue = VStringPlainValue(self.optimizer, box, source_op)
+ def make_vstring_plain(self, box, source_op, mode):
+ vvalue = VStringPlainValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def make_vstring_concat(self, box, source_op=None):
- vvalue = VStringConcatValue(self.optimizer, box, source_op)
+ def make_vstring_concat(self, box, source_op, mode):
+ vvalue = VStringConcatValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def make_vstring_slice(self, box, source_op=None):
- vvalue = VStringSliceValue(self.optimizer, box, source_op)
+ def make_vstring_slice(self, box, source_op, mode):
+ vvalue = VStringSliceValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def optimize_CALL(self, op):
- # dispatch based on 'oopspecindex' to a method that handles
- # specifically the given oopspec call. For non-oopspec calls,
- # oopspecindex is just zero.
- effectinfo = op.getdescr().get_extra_info()
- if effectinfo is not None:
- oopspecindex = effectinfo.oopspecindex
- for value, meth in opt_call_oopspec_ops:
- if oopspecindex == value:
- if meth(self, op):
- return
- self.emit_operation(op)
-
- def opt_call_oopspec_ARRAYCOPY(self, op):
- source_value = self.getvalue(op.getarg(1))
- dest_value = self.getvalue(op.getarg(2))
- source_start_box = self.get_constant_box(op.getarg(3))
- dest_start_box = self.get_constant_box(op.getarg(4))
- length = self.get_constant_box(op.getarg(5))
- if (source_value.is_virtual() and source_start_box and dest_start_box
- and length and dest_value.is_virtual()):
- # XXX optimize the case where dest value is not virtual,
- # but we still can avoid a mess
- source_start = source_start_box.getint()
- dest_start = dest_start_box.getint()
- for index in range(length.getint()):
- val = source_value.getitem(index + source_start)
- dest_value.setitem(index + dest_start, val)
- return True
- if length and length.getint() == 0:
- return True # 0-length arraycopy
- return False
-
def optimize_NEWSTR(self, op):
+ self._optimize_NEWSTR(op, mode_string)
+ def optimize_NEWUNICODE(self, op):
+ self._optimize_NEWSTR(op, mode_unicode)
+
+ def _optimize_NEWSTR(self, op, mode):
length_box = self.get_constant_box(op.getarg(0))
if length_box:
# if the original 'op' did not have a ConstInt as argument,
# build a new one with the ConstInt argument
if not isinstance(op.getarg(0), ConstInt):
- op = ResOperation(rop.NEWSTR, [length_box], op.result)
- vvalue = self.make_vstring_plain(op.result, op)
+ op = ResOperation(mode.NEWSTR, [length_box], op.result)
+ vvalue = self.make_vstring_plain(op.result, op, mode)
vvalue.setup(length_box.getint())
else:
self.getvalue(op.result).ensure_nonnull()
@@ -329,13 +362,20 @@
value.ensure_nonnull()
self.emit_operation(op)
+ optimize_UNICODESETITEM = optimize_STRSETITEM
+
def optimize_STRGETITEM(self, op):
+ self._optimize_STRGETITEM(op, mode_string)
+ def optimize_UNICODEGETITEM(self, op):
+ self._optimize_STRGETITEM(op, mode_unicode)
+
+ def _optimize_STRGETITEM(self, op, mode):
value = self.getvalue(op.getarg(0))
vindex = self.getvalue(op.getarg(1))
- vresult = self.strgetitem(value, vindex)
+ vresult = self.strgetitem(value, vindex, mode)
self.make_equal_to(op.result, vresult)
- def strgetitem(self, value, vindex):
+ def strgetitem(self, value, vindex, mode):
value.ensure_nonnull()
#
if value.is_virtual() and isinstance(value, VStringSliceValue):
@@ -350,28 +390,71 @@
return value.getitem(vindex.box.getint())
#
resbox = _strgetitem(self.optimizer.newoperations,
- value.force_box(),vindex.force_box())
+ value.force_box(),vindex.force_box(), mode)
return self.getvalue(resbox)
def optimize_STRLEN(self, op):
+ self._optimize_STRLEN(op, mode_string)
+ def optimize_UNICODELEN(self, op):
+ self._optimize_STRLEN(op, mode_unicode)
+
+ def _optimize_STRLEN(self, op, mode):
value = self.getvalue(op.getarg(0))
- lengthbox = value.getstrlen(self.optimizer.newoperations)
+ lengthbox = value.getstrlen(self.optimizer.newoperations, mode)
self.make_equal_to(op.result, self.getvalue(lengthbox))
- def opt_call_oopspec_STR_CONCAT(self, op):
+ def optimize_CALL(self, op):
+ # dispatch based on 'oopspecindex' to a method that handles
+ # specifically the given oopspec call. For non-oopspec calls,
+ # oopspecindex is just zero.
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ oopspecindex = effectinfo.oopspecindex
+ for value, meth in opt_call_oopspec_ops:
+ if oopspecindex == value: # a match with the OS_STR_xxx
+ if meth(self, op, mode_string):
+ return
+ break
+ if oopspecindex == value + EffectInfo._OS_offset_uni:
+ # a match with the OS_UNI_xxx
+ if meth(self, op, mode_unicode):
+ return
+ break
+ if oopspecindex == EffectInfo.OS_STR2UNICODE:
+ if self.opt_call_str_STR2UNICODE(op):
+ return
+ self.emit_operation(op)
+
+ def opt_call_str_STR2UNICODE(self, op):
+ # Constant-fold unicode("constant string").
+ # More generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
+ varg = self.getvalue(op.getarg(1))
+ s = varg.get_constant_string_spec(mode_string)
+ if s is None:
+ return False
+ try:
+ u = unicode(s)
+ except UnicodeDecodeError:
+ return False
+ self.make_constant(op.result, get_const_ptr_for_unicode(u))
+ return True
+
+ def opt_call_stroruni_STR_CONCAT(self, op, mode):
vleft = self.getvalue(op.getarg(1))
vright = self.getvalue(op.getarg(2))
vleft.ensure_nonnull()
vright.ensure_nonnull()
newoperations = self.optimizer.newoperations
- len1box = vleft.getstrlen(newoperations)
- len2box = vright.getstrlen(newoperations)
+ len1box = vleft.getstrlen(newoperations, mode)
+ len2box = vright.getstrlen(newoperations, mode)
lengthbox = _int_add(newoperations, len1box, len2box)
- value = self.make_vstring_concat(op.result, op)
+ value = self.make_vstring_concat(op.result, op, mode)
value.setup(vleft, vright, lengthbox)
return True
- def opt_call_oopspec_STR_SLICE(self, op):
+ def opt_call_stroruni_STR_SLICE(self, op, mode):
newoperations = self.optimizer.newoperations
vstr = self.getvalue(op.getarg(1))
vstart = self.getvalue(op.getarg(2))
@@ -380,7 +463,7 @@
if (isinstance(vstr, VStringPlainValue) and vstart.is_constant()
and vstop.is_constant()):
# slicing with constant bounds of a VStringPlainValue
- value = self.make_vstring_plain(op.result, op)
+ value = self.make_vstring_plain(op.result, op, mode)
value.setup_slice(vstr._chars, vstart.box.getint(),
vstop.box.getint())
return True
@@ -398,16 +481,16 @@
vstart.force_box())
vstart = self.getvalue(startbox)
#
- value = self.make_vstring_slice(op.result, op)
+ value = self.make_vstring_slice(op.result, op, mode)
value.setup(vstr, vstart, self.getvalue(lengthbox))
return True
- def opt_call_oopspec_STR_EQUAL(self, op):
+ def opt_call_stroruni_STR_EQUAL(self, op, mode):
v1 = self.getvalue(op.getarg(1))
v2 = self.getvalue(op.getarg(2))
#
- l1box = v1.getstrlen(None)
- l2box = v2.getstrlen(None)
+ l1box = v1.getstrlen(None, mode)
+ l2box = v2.getstrlen(None, mode)
if (l1box is not None and l2box is not None and
isinstance(l1box, ConstInt) and
isinstance(l2box, ConstInt) and
@@ -416,13 +499,13 @@
self.make_constant(op.result, CONST_0)
return True
#
- if self.handle_str_equal_level1(v1, v2, op.result):
+ if self.handle_str_equal_level1(v1, v2, op.result, mode):
return True
- if self.handle_str_equal_level1(v2, v1, op.result):
+ if self.handle_str_equal_level1(v2, v1, op.result, mode):
return True
- if self.handle_str_equal_level2(v1, v2, op.result):
+ if self.handle_str_equal_level2(v1, v2, op.result, mode):
return True
- if self.handle_str_equal_level2(v2, v1, op.result):
+ if self.handle_str_equal_level2(v2, v1, op.result, mode):
return True
#
if v1.is_nonnull() and v2.is_nonnull():
@@ -434,37 +517,37 @@
else:
do = EffectInfo.OS_STREQ_NONNULL
self.generate_modified_call(do, [v1.force_box(),
- v2.force_box()], op.result)
+ v2.force_box()], op.result, mode)
return True
return False
- def handle_str_equal_level1(self, v1, v2, resultbox):
- l2box = v2.getstrlen(None)
+ def handle_str_equal_level1(self, v1, v2, resultbox, mode):
+ l2box = v2.getstrlen(None, mode)
if isinstance(l2box, ConstInt):
if l2box.value == 0:
- lengthbox = v1.getstrlen(self.optimizer.newoperations)
+ lengthbox = v1.getstrlen(self.optimizer.newoperations, mode)
seo = self.optimizer.send_extra_operation
seo(ResOperation(rop.INT_EQ, [lengthbox, CONST_0], resultbox))
return True
if l2box.value == 1:
- l1box = v1.getstrlen(None)
+ l1box = v1.getstrlen(None, mode)
if isinstance(l1box, ConstInt) and l1box.value == 1:
# comparing two single chars
- vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO)
- vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO, mode)
+ vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
seo = self.optimizer.send_extra_operation
seo(ResOperation(rop.INT_EQ, [vchar1.force_box(),
vchar2.force_box()],
resultbox))
return True
if isinstance(v1, VStringSliceValue):
- vchar = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
do = EffectInfo.OS_STREQ_SLICE_CHAR
self.generate_modified_call(do, [v1.vstr.force_box(),
v1.vstart.force_box(),
v1.vlength.force_box(),
vchar.force_box()],
- resultbox)
+ resultbox, mode)
return True
#
if v2.is_null():
@@ -482,17 +565,18 @@
#
return False
- def handle_str_equal_level2(self, v1, v2, resultbox):
- l2box = v2.getstrlen(None)
+ def handle_str_equal_level2(self, v1, v2, resultbox, mode):
+ l2box = v2.getstrlen(None, mode)
if isinstance(l2box, ConstInt):
if l2box.value == 1:
- vchar = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
if v1.is_nonnull():
do = EffectInfo.OS_STREQ_NONNULL_CHAR
else:
do = EffectInfo.OS_STREQ_CHECKNULL_CHAR
self.generate_modified_call(do, [v1.force_box(),
- vchar.force_box()], resultbox)
+ vchar.force_box()], resultbox,
+ mode)
return True
#
if v1.is_virtual() and isinstance(v1, VStringSliceValue):
@@ -503,11 +587,12 @@
self.generate_modified_call(do, [v1.vstr.force_box(),
v1.vstart.force_box(),
v1.vlength.force_box(),
- v2.force_box()], resultbox)
+ v2.force_box()], resultbox, mode)
return True
return False
- def generate_modified_call(self, oopspecindex, args, result):
+ def generate_modified_call(self, oopspecindex, args, result, mode):
+ oopspecindex += mode.OS_offset
calldescr, func = callinfo_for_oopspec(oopspecindex)
op = ResOperation(rop.CALL, [ConstInt(func)] + args, result,
descr=calldescr)
@@ -525,7 +610,7 @@
optimize_ops = _findall(OptString, 'optimize_')
def _findall_call_oopspec():
- prefix = 'opt_call_oopspec_'
+ prefix = 'opt_call_stroruni_'
result = []
for name in dir(OptString):
if name.startswith(prefix):
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/virtualize.py Fri Oct 22 23:09:43 2010
@@ -74,6 +74,8 @@
assert self.source_op is not None
# ^^^ This case should not occur any more (see test_bug_3).
#
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
@@ -134,6 +136,11 @@
fielddescrs = self._get_field_descr_list()
return modifier.make_virtual(self.known_class, fielddescrs)
+ def __repr__(self):
+ cls_name = self.known_class.value.adr.ptr._obj._TYPE._name
+ field_names = [field.name for field in self._fields]
+ return "<VirtualValue cls=%s fields=%s>" % (cls_name, field_names)
+
class VStructValue(AbstractVirtualStructValue):
def __init__(self, optimizer, structdescr, keybox, source_op=None):
@@ -165,6 +172,8 @@
def _really_force(self):
assert self.source_op is not None
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py Fri Oct 22 23:09:43 2010
@@ -1,4 +1,4 @@
-import py, os
+import py, os, sys
from pypy.rpython.lltypesystem import lltype, llmemory, rclass
from pypy.rlib.objectmodel import we_are_translated
from pypy.rlib.unroll import unrolling_iterable
@@ -498,6 +498,22 @@
opimpl_getfield_gc_r_pure = _opimpl_getfield_gc_pure_any
opimpl_getfield_gc_f_pure = _opimpl_getfield_gc_pure_any
+ @arguments("orgpc", "box", "descr")
+ def _opimpl_getfield_gc_greenfield_any(self, pc, box, fielddescr):
+ ginfo = self.metainterp.jitdriver_sd.greenfield_info
+ if (ginfo is not None and fielddescr in ginfo.green_field_descrs
+ and not self._nonstandard_virtualizable(pc, box)):
+ # fetch the result, but consider it as a Const box and don't
+ # record any operation
+ resbox = executor.execute(self.metainterp.cpu, self.metainterp,
+ rop.GETFIELD_GC_PURE, fielddescr, box)
+ return resbox.constbox()
+ # fall-back
+ return self.execute_with_descr(rop.GETFIELD_GC_PURE, fielddescr, box)
+ opimpl_getfield_gc_i_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_r_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_f_greenfield = _opimpl_getfield_gc_greenfield_any
+
@arguments("box", "descr", "box")
def _opimpl_setfield_gc_any(self, box, fielddescr, valuebox):
self.execute_with_descr(rop.SETFIELD_GC, fielddescr, box, valuebox)
@@ -529,7 +545,8 @@
def _nonstandard_virtualizable(self, pc, box):
# returns True if 'box' is actually not the "standard" virtualizable
# that is stored in metainterp.virtualizable_boxes[-1]
- if self.metainterp.jitdriver_sd.virtualizable_info is None:
+ if (self.metainterp.jitdriver_sd.virtualizable_info is None and
+ self.metainterp.jitdriver_sd.greenfield_info is None):
return True # can occur in case of multiple JITs
standard_box = self.metainterp.virtualizable_boxes[-1]
if standard_box is box:
@@ -799,12 +816,16 @@
@arguments("orgpc", "int", "boxes3", "boxes3")
def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, redboxes):
+ any_operation = len(self.metainterp.history.operations) > 0
jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex]
self.verify_green_args(jitdriver_sd, greenboxes)
# xxx we may disable the following line in some context later
self.debug_merge_point(jitdriver_sd, greenboxes)
if self.metainterp.seen_loop_header_for_jdindex < 0:
- return
+ if not jitdriver_sd.no_loop_header or not any_operation:
+ return
+ # automatically add a loop_header if there is none
+ self.metainterp.seen_loop_header_for_jdindex = jdindex
#
assert self.metainterp.seen_loop_header_for_jdindex == jdindex, (
"found a loop_header for a JitDriver that does not match "
@@ -893,6 +914,40 @@
msg = box.getref(lltype.Ptr(rstr.STR))
lloperation.llop.debug_fatalerror(msg)
+ @arguments("box", "box", "box", "box", "box")
+ def opimpl_jit_debug(self, stringbox, arg1box, arg2box, arg3box, arg4box):
+ from pypy.rpython.lltypesystem import rstr
+ from pypy.rpython.annlowlevel import hlstr
+ msg = stringbox.getref(lltype.Ptr(rstr.STR))
+ debug_print('jit_debug:', hlstr(msg),
+ arg1box.getint(), arg2box.getint(),
+ arg3box.getint(), arg4box.getint())
+ args = [stringbox, arg1box, arg2box, arg3box, arg4box]
+ i = 4
+ while i > 0 and args[i].getint() == -sys.maxint-1:
+ i -= 1
+ assert i >= 0
+ op = self.metainterp.history.record(rop.JIT_DEBUG, args[:i+1], None)
+ self.metainterp.attach_debug_info(op)
+
+ @arguments("box")
+ def _opimpl_assert_green(self, box):
+ if not isinstance(box, Const):
+ msg = "assert_green failed at %s:%d" % (
+ self.jitcode.name,
+ self.pc)
+ if we_are_translated():
+ from pypy.rpython.annlowlevel import llstr
+ from pypy.rpython.lltypesystem import lloperation
+ lloperation.llop.debug_fatalerror(lltype.Void, llstr(msg))
+ else:
+ from pypy.rlib.jit import AssertGreenFailed
+ raise AssertGreenFailed(msg)
+
+ opimpl_int_assert_green = _opimpl_assert_green
+ opimpl_ref_assert_green = _opimpl_assert_green
+ opimpl_float_assert_green = _opimpl_assert_green
+
@arguments("box")
def opimpl_virtual_ref(self, box):
# Details on the content of metainterp.virtualref_boxes:
@@ -998,7 +1053,8 @@
guard_op = metainterp.history.record(opnum, moreargs, None,
descr=resumedescr)
virtualizable_boxes = None
- if metainterp.jitdriver_sd.virtualizable_info is not None:
+ if (metainterp.jitdriver_sd.virtualizable_info is not None or
+ metainterp.jitdriver_sd.greenfield_info is not None):
virtualizable_boxes = metainterp.virtualizable_boxes
saved_pc = self.pc
if resumepc >= 0:
@@ -1646,6 +1702,7 @@
duplicates)
live_arg_boxes += self.virtualizable_boxes
live_arg_boxes.pop()
+ #
assert len(self.virtualref_boxes) == 0, "missing virtual_ref_finish()?"
# Called whenever we reach the 'loop_header' hint.
# First, attempt to make a bridge:
@@ -1832,6 +1889,7 @@
f.setup_call(original_boxes)
assert self.in_recursion == 0
self.virtualref_boxes = []
+ self.initialize_withgreenfields(original_boxes)
self.initialize_virtualizable(original_boxes)
def initialize_state_from_guard_failure(self, resumedescr):
@@ -1856,6 +1914,14 @@
self.virtualizable_boxes.append(virtualizable_box)
self.initialize_virtualizable_enter()
+ def initialize_withgreenfields(self, original_boxes):
+ ginfo = self.jitdriver_sd.greenfield_info
+ if ginfo is not None:
+ assert self.jitdriver_sd.virtualizable_info is None
+ index = (self.jitdriver_sd.num_green_args +
+ ginfo.red_index)
+ self.virtualizable_boxes = [original_boxes[index]]
+
def initialize_virtualizable_enter(self):
vinfo = self.jitdriver_sd.virtualizable_info
virtualizable_box = self.virtualizable_boxes[-1]
@@ -1949,8 +2015,10 @@
def rebuild_state_after_failure(self, resumedescr):
vinfo = self.jitdriver_sd.virtualizable_info
+ ginfo = self.jitdriver_sd.greenfield_info
self.framestack = []
- boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo)
+ boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo,
+ ginfo)
inputargs_and_holes, virtualizable_boxes, virtualref_boxes = boxlists
#
# virtual refs: make the vrefs point to the freshly allocated virtuals
@@ -1975,6 +2043,12 @@
assert not virtualizable.vable_token
# fill the virtualizable with the local boxes
self.synchronize_virtualizable()
+ #
+ elif self.jitdriver_sd.greenfield_info:
+ self.virtualizable_boxes = virtualizable_boxes
+ else:
+ assert not virtualizable_boxes
+ #
return inputargs_and_holes
def check_synchronized_virtualizable(self):
@@ -2048,7 +2122,8 @@
for i in range(len(boxes)):
if boxes[i] is oldbox:
boxes[i] = newbox
- if self.jitdriver_sd.virtualizable_info is not None:
+ if (self.jitdriver_sd.virtualizable_info is not None or
+ self.jitdriver_sd.greenfield_info is not None):
boxes = self.virtualizable_boxes
for i in range(len(boxes)):
if boxes[i] is oldbox:
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py Fri Oct 22 23:09:43 2010
@@ -93,7 +93,7 @@
def __repr__(self):
return self.repr()
- def repr(self):
+ def repr(self, graytext=False):
# RPython-friendly version
if self.result is not None:
sres = '%s = ' % (self.result,)
@@ -101,6 +101,8 @@
sres = ''
if self.name:
prefix = "%s:%s " % (self.name, self.pc)
+ if graytext:
+ prefix = "\f%s\f" % prefix
else:
prefix = ""
args = self.getarglist()
@@ -457,13 +459,14 @@
#'RUNTIMENEW/1', # ootype operation
'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier)
'DEBUG_MERGE_POINT/1', # debugging only
+ 'JIT_DEBUG/*', # debugging only
'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend
'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length
'COPYUNICODECONTENT/5',
'_CANRAISE_FIRST', # ----- start of can_raise operations -----
'CALL/*d',
- 'CALL_ASSEMBLER/*d',
+ 'CALL_ASSEMBLER/*d', # call already compiled assembler
'CALL_MAY_FORCE/*d',
'CALL_LOOPINVARIANT/*d',
#'OOSEND', # ootype operation
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/resume.py Fri Oct 22 23:09:43 2010
@@ -255,13 +255,19 @@
def make_varray(self, arraydescr):
return VArrayInfo(arraydescr)
- def make_vstrplain(self):
+ def make_vstrplain(self, is_unicode=False):
+ if is_unicode:
+ return VUniPlainInfo()
return VStrPlainInfo()
- def make_vstrconcat(self):
+ def make_vstrconcat(self, is_unicode=False):
+ if is_unicode:
+ return VUniConcatInfo()
return VStrConcatInfo()
- def make_vstrslice(self):
+ def make_vstrslice(self, is_unicode=False):
+ if is_unicode:
+ return VUniSliceInfo()
return VStrSliceInfo()
def register_virtual_fields(self, virtualbox, fieldboxes):
@@ -550,6 +556,60 @@
for i in self.fieldnums:
debug_print("\t\t", str(untag(i)))
+
+class VUniPlainInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of the characters of all
+ fieldnums."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ length = len(self.fieldnums)
+ string = decoder.allocate_unicode(length)
+ decoder.virtuals_cache[index] = string
+ for i in range(length):
+ decoder.unicode_setitem(string, i, self.fieldnums[i])
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvuniplaininfo length", len(self.fieldnums))
+
+
+class VUniConcatInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of the concatenation of two
+ other unicode strings."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ # xxx for blackhole resuming, this will build all intermediate
+ # strings and throw them away immediately, which is a bit sub-
+ # efficient. Not sure we care.
+ left, right = self.fieldnums
+ string = decoder.concat_unicodes(left, right)
+ decoder.virtuals_cache[index] = string
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvuniconcatinfo")
+ for i in self.fieldnums:
+ debug_print("\t\t", str(untag(i)))
+
+
+class VUniSliceInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of slicing another
+ unicode string."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ largerstr, start, length = self.fieldnums
+ string = decoder.slice_unicode(largerstr, start, length)
+ decoder.virtuals_cache[index] = string
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvunisliceinfo")
+ for i in self.fieldnums:
+ debug_print("\t\t", str(untag(i)))
+
# ____________________________________________________________
class AbstractResumeDataReader(object):
@@ -629,9 +689,11 @@
# ---------- when resuming for pyjitpl.py, make boxes ----------
-def rebuild_from_resumedata(metainterp, storage, virtualizable_info):
+def rebuild_from_resumedata(metainterp, storage, virtualizable_info,
+ greenfield_info):
resumereader = ResumeDataBoxReader(storage, metainterp)
- boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info)
+ boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info,
+ greenfield_info)
virtualizable_boxes, virtualref_boxes = boxes
frameinfo = storage.rd_frame_info_list
while True:
@@ -676,15 +738,18 @@
assert (end & 1) == 0
return [self.decode_ref(nums[i]) for i in range(end)]
- def consume_vref_and_vable_boxes(self, vinfo):
+ def consume_vref_and_vable_boxes(self, vinfo, ginfo):
nums = self.cur_numb.nums
self.cur_numb = self.cur_numb.prev
- if vinfo is None:
- virtualizable_boxes = None
- end = len(nums)
- else:
+ if vinfo is not None:
virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums)
end = len(nums) - len(virtualizable_boxes)
+ elif ginfo is not None:
+ virtualizable_boxes = [self.decode_ref(nums[-1])]
+ end = len(nums) - 1
+ else:
+ virtualizable_boxes = None
+ end = len(nums)
virtualref_boxes = self.consume_virtualref_boxes(nums, end)
return virtualizable_boxes, virtualref_boxes
@@ -725,6 +790,32 @@
return self.metainterp.execute_and_record_varargs(
rop.CALL, [ConstInt(func), strbox, startbox, stopbox], calldescr)
+ def allocate_unicode(self, length):
+ return self.metainterp.execute_and_record(rop.NEWUNICODE,
+ None, ConstInt(length))
+
+ def unicode_setitem(self, strbox, index, charnum):
+ charbox = self.decode_box(charnum, INT)
+ self.metainterp.execute_and_record(rop.UNICODESETITEM, None,
+ strbox, ConstInt(index), charbox)
+
+ def concat_unicodes(self, str1num, str2num):
+ calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT)
+ str1box = self.decode_box(str1num, REF)
+ str2box = self.decode_box(str2num, REF)
+ return self.metainterp.execute_and_record_varargs(
+ rop.CALL, [ConstInt(func), str1box, str2box], calldescr)
+
+ def slice_unicode(self, strnum, startnum, lengthnum):
+ calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE)
+ strbox = self.decode_box(strnum, REF)
+ startbox = self.decode_box(startnum, INT)
+ lengthbox = self.decode_box(lengthnum, INT)
+ stopbox = self.metainterp.execute_and_record(rop.INT_ADD, None,
+ startbox, lengthbox)
+ return self.metainterp.execute_and_record_varargs(
+ rop.CALL, [ConstInt(func), strbox, startbox, stopbox], calldescr)
+
def setfield(self, descr, structbox, fieldnum):
if descr.is_pointer_field():
kind = REF
@@ -815,8 +906,9 @@
resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage,
all_virtuals)
vinfo = jitdriver_sd.virtualizable_info
+ ginfo = jitdriver_sd.greenfield_info
vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
#
# First get a chain of blackhole interpreters whose length is given
# by the depth of rd_frame_info_list. The first one we get must be
@@ -846,11 +938,11 @@
resumereader.done()
return firstbh
-def force_from_resumedata(metainterp_sd, storage, vinfo=None):
+def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo):
resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage)
resumereader.handling_async_forcing()
vrefinfo = metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
return resumereader.force_all_virtuals()
class ResumeDataDirectReader(AbstractResumeDataReader):
@@ -925,11 +1017,12 @@
return specialize_value(TYPE, x)
load_value_of_type._annspecialcase_ = 'specialize:arg(1)'
- def consume_vref_and_vable(self, vrefinfo, vinfo):
+ def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo):
nums = self.cur_numb.nums
self.cur_numb = self.cur_numb.prev
if self.resume_after_guard_not_forced != 2:
end_vref = self.consume_vable_info(vinfo, nums)
+ if ginfo is not None: end_vref -= 1
self.consume_virtualref_info(vrefinfo, nums, end_vref)
def allocate_with_vtable(self, known_class):
@@ -967,6 +1060,31 @@
result = funcptr(str, start, start + length)
return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+ def allocate_unicode(self, length):
+ return self.cpu.bh_newunicode(length)
+
+ def unicode_setitem(self, str, index, charnum):
+ char = self.decode_int(charnum)
+ self.cpu.bh_unicodesetitem(str, index, char)
+
+ def concat_unicodes(self, str1num, str2num):
+ str1 = self.decode_ref(str1num)
+ str2 = self.decode_ref(str2num)
+ str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1)
+ str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2)
+ funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT)
+ result = funcptr(str1, str2)
+ return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+
+ def slice_unicode(self, strnum, startnum, lengthnum):
+ str = self.decode_ref(strnum)
+ start = self.decode_int(startnum)
+ length = self.decode_int(lengthnum)
+ str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str)
+ funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE)
+ result = funcptr(str, start, start + length)
+ return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+
def setfield(self, descr, struct, fieldnum):
if descr.is_pointer_field():
newvalue = self.decode_ref(fieldnum)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/simple_optimize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/simple_optimize.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/simple_optimize.py Fri Oct 22 23:09:43 2010
@@ -9,11 +9,13 @@
def transform(op):
from pypy.jit.metainterp.history import AbstractDescr
- # Rename CALL_PURE to CALL.
+ # Rename CALL_PURE and CALL_LOOPINVARIANT to CALL.
# Simplify the VIRTUAL_REF_* so that they don't show up in the backend.
if op.getopnum() == rop.CALL_PURE:
op = ResOperation(rop.CALL, op.getarglist()[1:], op.result,
op.getdescr())
+ elif op.getopnum() == rop.CALL_LOOPINVARIANT:
+ op = op.copy_and_change(rop.CALL)
elif op.getopnum() == rop.VIRTUAL_REF:
op = ResOperation(rop.SAME_AS, [op.getarg(0)], op.result)
elif op.getopnum() == rop.VIRTUAL_REF_FINISH:
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py Fri Oct 22 23:09:43 2010
@@ -2,6 +2,7 @@
import sys
from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside
from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant
+from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed
from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats
from pypy.jit.backend.llgraph import runner
from pypy.jit.metainterp import pyjitpl, history
@@ -44,6 +45,7 @@
num_green_args = 0
portal_graph = graphs[0]
virtualizable_info = None
+ greenfield_info = None
result_type = result_kind
portal_runner_ptr = "???"
@@ -1644,6 +1646,33 @@
res = self.interp_operations(f, [10, 3.5])
assert res == 3.5
+ def test_jit_debug(self):
+ myjitdriver = JitDriver(greens = [], reds = ['x'])
+ class A:
+ pass
+ def f(x):
+ while x > 0:
+ myjitdriver.can_enter_jit(x=x)
+ myjitdriver.jit_merge_point(x=x)
+ jit_debug("hi there:", x)
+ jit_debug("foobar")
+ x -= 1
+ return x
+ res = self.meta_interp(f, [8])
+ assert res == 0
+ self.check_loops(jit_debug=2)
+
+ def test_assert_green(self):
+ def f(x, promote):
+ if promote:
+ x = hint(x, promote=True)
+ assert_green(x)
+ return x
+ res = self.interp_operations(f, [8, 1])
+ assert res == 8
+ py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0])
+
+
class TestOOtype(BasicTests, OOJitMixin):
def test_oohash(self):
@@ -1751,7 +1780,7 @@
c = bool(p1)
d = not bool(p2)
return 1000*a + 100*b + 10*c + d
- prebuilt = [lltype.malloc(TP, flavor='raw')] * 2
+ prebuilt = [lltype.malloc(TP, flavor='raw', immortal=True)] * 2
expected = f(0, 1)
assert self.interp_operations(f, [0, 1]) == expected
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py Fri Oct 22 23:09:43 2010
@@ -6,8 +6,8 @@
from pypy.jit.metainterp.compile import ResumeGuardCountersInt
from pypy.jit.metainterp.compile import compile_tmp_callback
from pypy.jit.metainterp import optimize, jitprof, typesystem, compile
-from pypy.jit.metainterp.test.oparser import parse
from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin
+from pypy.jit.tool.oparser import parse
def test_insert_loop_token():
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_jitdriver.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_jitdriver.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_jitdriver.py Fri Oct 22 23:09:43 2010
@@ -72,6 +72,33 @@
# we expect no loop at all for 'loop1': it should always be inlined
self.check_tree_loop_count(2) # 1 x loop, 1 x enter bridge
+ def test_inactive_jitdriver(self):
+ myjitdriver1 = JitDriver(greens=[], reds=['n', 'm'],
+ get_printable_location = getloc1)
+ myjitdriver2 = JitDriver(greens=['g'], reds=['r'],
+ get_printable_location = getloc2)
+ #
+ myjitdriver1.active = False # <===
+ #
+ def loop1(n, m):
+ while n > 0:
+ myjitdriver1.can_enter_jit(n=n, m=m)
+ myjitdriver1.jit_merge_point(n=n, m=m)
+ n -= m
+ return n
+ #
+ def loop2(g, r):
+ while r > 0:
+ myjitdriver2.can_enter_jit(g=g, r=r)
+ myjitdriver2.jit_merge_point(g=g, r=r)
+ r += loop1(r, g) + (-1)
+ return r
+ #
+ res = self.meta_interp(loop2, [4, 40], repeat=7, inline=True)
+ assert res == loop2(4, 40)
+ # we expect no int_sub, but a residual call
+ self.check_loops(int_sub=0, call=1)
+
class TestLLtype(MultipleJitDriversTests, LLJitMixin):
pass
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py Fri Oct 22 23:09:43 2010
@@ -1,6 +1,6 @@
import sys
from pypy.rlib import debug
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
from pypy.jit.metainterp import logger
from pypy.jit.metainterp.typesystem import llhelper
from StringIO import StringIO
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py Fri Oct 22 23:09:43 2010
@@ -1,7 +1,7 @@
from pypy.rpython.lltypesystem import lltype, llmemory
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.resoperation import rop
from pypy.jit.metainterp.history import AbstractDescr, BoxInt, LoopToken,\
BoxFloat
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizefindnode.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizefindnode.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizefindnode.py Fri Oct 22 23:09:43 2010
@@ -18,7 +18,7 @@
from pypy.jit.metainterp.specnode import ConstantSpecNode
from pypy.jit.codewriter.effectinfo import EffectInfo
from pypy.jit.codewriter.heaptracker import register_known_gctype, adr2int
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
def test_sort_descrs():
class PseudoDescr(AbstractDescr):
@@ -117,33 +117,32 @@
EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE))
arraycopydescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
EffectInfo([], [], [], oopspecindex=EffectInfo.OS_ARRAYCOPY))
- strconcatdescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_CONCAT))
- slicedescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_SLICE))
- strequaldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_EQUAL))
- streq_slice_checknull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHECKNULL))
- streq_slice_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_NONNULL))
- streq_slice_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHAR))
- streq_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL))
- streq_nonnull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL_CHAR))
- streq_checknull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_CHECKNULL_CHAR))
- streq_lengthok_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_LENGTHOK))
+
+ for _name, _os in [
+ ('strconcatdescr', 'OS_STR_CONCAT'),
+ ('strslicedescr', 'OS_STR_SLICE'),
+ ('strequaldescr', 'OS_STR_EQUAL'),
+ ('streq_slice_checknull_descr', 'OS_STREQ_SLICE_CHECKNULL'),
+ ('streq_slice_nonnull_descr', 'OS_STREQ_SLICE_NONNULL'),
+ ('streq_slice_char_descr', 'OS_STREQ_SLICE_CHAR'),
+ ('streq_nonnull_descr', 'OS_STREQ_NONNULL'),
+ ('streq_nonnull_char_descr', 'OS_STREQ_NONNULL_CHAR'),
+ ('streq_checknull_char_descr', 'OS_STREQ_CHECKNULL_CHAR'),
+ ('streq_lengthok_descr', 'OS_STREQ_LENGTHOK'),
+ ]:
+ _oopspecindex = getattr(EffectInfo, _os)
+ locals()[_name] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+ #
+ _oopspecindex = getattr(EffectInfo, _os.replace('STR', 'UNI'))
+ locals()[_name.replace('str', 'unicode')] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+
+ s2u_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR2UNICODE))
+ #
class LoopToken(AbstractDescr):
pass
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py Fri Oct 22 23:09:43 2010
@@ -12,7 +12,7 @@
from pypy.jit.metainterp.jitprof import EmptyProfiler
from pypy.jit.metainterp import executor, compile, resume, history
from pypy.jit.metainterp.resoperation import rop, opname, ResOperation
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
##class FakeFrame(object):
## parent_resumedata_snapshot = None
@@ -132,14 +132,21 @@
# ____________________________________________________________
def equaloplists(oplist1, oplist2, strict_fail_args=True, remap={}):
- print '-'*20, 'Comparing lists', '-'*20
+ # try to use the full width of the terminal to display the list
+ # unfortunately, does not work with the default capture method of py.test
+ # (which is fd), you you need to use either -s or --capture=sys, else you
+ # get the standard 80 columns width
+ totwidth = py.io.get_terminal_width()
+ width = totwidth / 2 - 1
+ print ' Comparing lists '.center(totwidth, '-')
+ print '%s| %s' % ('optimized'.center(width), 'expected'.center(width))
for op1, op2 in zip(oplist1, oplist2):
txt1 = str(op1)
txt2 = str(op2)
while txt1 or txt2:
- print '%-39s| %s' % (txt1[:39], txt2[:39])
- txt1 = txt1[39:]
- txt2 = txt2[39:]
+ print '%s| %s' % (txt1[:width].ljust(width), txt2[:width])
+ txt1 = txt1[width:]
+ txt2 = txt2[width:]
assert op1.getopnum() == op2.getopnum()
assert op1.numargs() == op2.numargs()
for i in range(op1.numargs()):
@@ -262,6 +269,10 @@
expected = self.parse(optops)
print '\n'.join([str(o) for o in loop.operations])
self.assert_equal(loop, expected)
+ return loop
+
+
+class OptimizeOptTest(BaseTestOptimizeOpt):
def test_simple(self):
ops = """
@@ -2643,7 +2654,7 @@
''', rop.GUARD_TRUE)
-class TestLLtype(BaseTestOptimizeOpt, LLtypeMixin):
+class TestLLtype(OptimizeOptTest, LLtypeMixin):
def test_residual_call_does_not_invalidate_caches(self):
ops = """
@@ -3487,7 +3498,7 @@
i0 = strlen(p0)
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_addsub_const(self):
ops = """
@@ -3893,6 +3904,15 @@
"""
self.optimize_loop(ops, 'Not, Not', expected)
+ # ----------
+ def optimize_strunicode_loop(self, ops, spectext, optops):
+ # check with the arguments passed in
+ self.optimize_loop(ops, spectext, optops)
+ # check with replacing 'str' with 'unicode' everywhere
+ self.optimize_loop(ops.replace('str','unicode').replace('s"', 'u"'),
+ spectext,
+ optops.replace('str','unicode').replace('s"', 'u"'))
+
def test_newstr_1(self):
ops = """
[i0]
@@ -3905,7 +3925,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_newstr_2(self):
ops = """
@@ -3921,7 +3941,7 @@
[i0, i1]
jump(i1, i0)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_concat_1(self):
ops = """
@@ -3942,7 +3962,7 @@
copystrcontent(p2, p3, 0, i4, i5)
jump(p2, p3)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_concat_vstr2_str(self):
ops = """
@@ -3965,7 +3985,7 @@
copystrcontent(p2, p3, 0, 2, i4)
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_vstr2(self):
ops = """
@@ -3989,7 +4009,7 @@
i6 = int_add(i5, 1) # will be killed by the backend
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_str_str(self):
ops = """
@@ -4016,12 +4036,12 @@
copystrcontent(p3, p5, 0, i12b, i3b)
jump(p2, p3, p5)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_cstr1(self):
ops = """
[p2]
- p3 = call(0, p2, "x", descr=strconcatdescr)
+ p3 = call(0, p2, s"x", descr=strconcatdescr)
jump(p3)
"""
expected = """
@@ -4035,28 +4055,28 @@
i5 = int_add(i4, 1) # will be killed by the backend
jump(p3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_str_concat_consts(self):
ops = """
[]
- p1 = same_as("ab")
- p2 = same_as("cde")
+ p1 = same_as(s"ab")
+ p2 = same_as(s"cde")
p3 = call(0, p1, p2, descr=strconcatdescr)
escape(p3)
jump()
"""
expected = """
[]
- escape("abcde")
+ escape(s"abcde")
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_strunicode_loop(ops, '', expected)
def test_str_slice_1(self):
ops = """
[p1, i1, i2]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
jump(p2, i1, i2)
"""
expected = """
@@ -4066,12 +4086,12 @@
copystrcontent(p1, p2, i1, 0, i3)
jump(p2, i1, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_slice_2(self):
ops = """
[p1, i2]
- p2 = call(0, p1, 0, i2, descr=slicedescr)
+ p2 = call(0, p1, 0, i2, descr=strslicedescr)
jump(p2, i2)
"""
expected = """
@@ -4080,13 +4100,13 @@
copystrcontent(p1, p2, 0, 0, i2)
jump(p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_slice_3(self):
ops = """
[p1, i1, i2, i3, i4]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
- p3 = call(0, p2, i3, i4, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
+ p3 = call(0, p2, i3, i4, descr=strslicedescr)
jump(p3, i1, i2, i3, i4)
"""
expected = """
@@ -4098,12 +4118,12 @@
copystrcontent(p1, p3, i6, 0, i5)
jump(p3, i1, i2, i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not, Not', expected)
def test_str_slice_getitem1(self):
ops = """
[p1, i1, i2, i3]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
i4 = strgetitem(p2, i3)
escape(i4)
jump(p1, i1, i2, i3)
@@ -4116,7 +4136,7 @@
escape(i4)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
def test_str_slice_plain(self):
ops = """
@@ -4124,7 +4144,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i3)
strsetitem(p1, 1, i4)
- p2 = call(0, p1, 1, 2, descr=slicedescr)
+ p2 = call(0, p1, 1, 2, descr=strslicedescr)
i5 = strgetitem(p2, 0)
escape(i5)
jump(i3, i4)
@@ -4134,12 +4154,12 @@
escape(i4)
jump(i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_slice_concat(self):
ops = """
[p1, i1, i2, p2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
p4 = call(0, p3, p2, descr=strconcatdescr)
jump(p4, i1, i2, p2)
"""
@@ -4155,10 +4175,10 @@
copystrcontent(p2, p4, 0, i3, i4b)
jump(p4, i1, i2, p2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
# ----------
- def optimize_loop_extradescrs(self, ops, spectext, optops):
+ def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops):
from pypy.jit.metainterp.optimizeopt import string
def my_callinfo_for_oopspec(oopspecindex):
calldescrtype = type(LLtypeMixin.strequaldescr)
@@ -4173,7 +4193,7 @@
saved = string.callinfo_for_oopspec
try:
string.callinfo_for_oopspec = my_callinfo_for_oopspec
- self.optimize_loop(ops, spectext, optops)
+ self.optimize_strunicode_loop(ops, spectext, optops)
finally:
string.callinfo_for_oopspec = saved
@@ -4184,7 +4204,7 @@
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', ops)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', ops)
def test_str_equal_noop2(self):
ops = """
@@ -4209,12 +4229,13 @@
escape(i0)
jump(p1, p2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ expected)
def test_str_equal_slice1(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p4, p3, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4226,12 +4247,13 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice2(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4243,13 +4265,14 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice3(self):
ops = """
[p1, i1, i2, p3]
guard_nonnull(p3) []
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4262,13 +4285,14 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice4(self):
ops = """
[p1, i1, i2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
- i0 = call(0, p3, "x", descr=strequaldescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
+ i0 = call(0, p3, s"x", descr=strequaldescr)
escape(i0)
jump(p1, i1, i2)
"""
@@ -4279,12 +4303,13 @@
escape(i0)
jump(p1, i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ expected)
def test_str_equal_slice5(self):
ops = """
[p1, i1, i2, i3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
p5 = newstr(1)
strsetitem(p5, 0, i3)
i0 = call(0, p5, p4, descr=strequaldescr)
@@ -4298,7 +4323,8 @@
escape(i0)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_none1(self):
ops = """
@@ -4313,7 +4339,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_none2(self):
ops = """
@@ -4328,30 +4354,30 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull1(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=strequaldescr)
+ i0 = call(0, p1, s"hello world", descr=strequaldescr)
escape(i0)
jump(p1)
"""
expected = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=streq_nonnull_descr)
+ i0 = call(0, p1, s"hello world", descr=streq_nonnull_descr)
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull2(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4363,13 +4389,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull3(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4380,13 +4406,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull4(self):
ops = """
[p1, p2]
p4 = call(0, p1, p2, descr=strconcatdescr)
- i0 = call(0, "hello world", p4, descr=strequaldescr)
+ i0 = call(0, s"hello world", p4, descr=strequaldescr)
escape(i0)
jump(p1, p2)
"""
@@ -4401,17 +4427,17 @@
i5 = strlen(p2)
i6 = int_add(i4, i5) # will be killed by the backend
copystrcontent(p2, p4, 0, i4, i5)
- i0 = call(0, "hello world", p4, descr=streq_nonnull_descr)
+ i0 = call(0, s"hello world", p4, descr=streq_nonnull_descr)
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
def test_str_equal_chars0(self):
ops = """
[i1]
p1 = newstr(0)
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4420,14 +4446,14 @@
escape(1)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_chars1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4437,7 +4463,7 @@
escape(i0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_chars2(self):
ops = """
@@ -4445,7 +4471,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=strequaldescr)
+ i0 = call(0, p1, s"xy", descr=strequaldescr)
escape(i0)
jump(i1, i2)
"""
@@ -4454,16 +4480,16 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=streq_lengthok_descr)
+ i0 = call(0, p1, s"xy", descr=streq_lengthok_descr)
escape(i0)
jump(i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
def test_str_equal_chars3(self):
ops = """
[p1]
- i0 = call(0, "x", p1, descr=strequaldescr)
+ i0 = call(0, s"x", p1, descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4473,14 +4499,14 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_lengthmismatch1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, "xy", p1, descr=strequaldescr)
+ i0 = call(0, s"xy", p1, descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4489,13 +4515,36 @@
escape(0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
- # XXX unicode operations
- # XXX str2unicode
+ def test_str2unicode_constant(self):
+ ops = """
+ []
+ p0 = call(0, "xy", descr=s2u_descr) # string -> unicode
+ escape(p0)
+ jump()
+ """
+ expected = """
+ []
+ escape(u"xy")
+ jump()
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, '', expected)
+
+ def test_str2unicode_nonconstant(self):
+ ops = """
+ [p0]
+ p1 = call(0, p0, descr=s2u_descr) # string -> unicode
+ escape(p1)
+ jump(p1)
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', ops)
+ # more generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
-##class TestOOtype(BaseTestOptimizeOpt, OOtypeMixin):
+##class TestOOtype(OptimizeOptTest, OOtypeMixin):
## def test_instanceof(self):
## ops = """
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py Fri Oct 22 23:09:43 2010
@@ -240,6 +240,17 @@
return FakeBuiltObject(strconcat=[left, right])
def slice_string(self, str, start, length):
return FakeBuiltObject(strslice=[str, start, length])
+ def allocate_unicode(self, length):
+ return FakeBuiltObject(unistring=[None]*length)
+ def unicode_setitem(self, unistring, i, fieldnum):
+ value, tag = untag(fieldnum)
+ assert tag == TAGINT
+ assert 0 <= i < len(unistring.unistring)
+ unistring.unistring[i] = value
+ def concat_unicodes(self, left, right):
+ return FakeBuiltObject(uniconcat=[left, right])
+ def slice_unicode(self, str, start, length):
+ return FakeBuiltObject(unislice=[str, start, length])
class FakeBuiltObject(object):
def __init__(self, **kwds):
@@ -304,6 +315,30 @@
assert reader.force_all_virtuals() == [
FakeBuiltObject(strslice=info.fieldnums)]
+def test_vuniplaininfo():
+ info = VUniPlainInfo()
+ info.fieldnums = [tag(60, TAGINT)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(unistring=[60])]
+
+def test_vuniconcatinfo():
+ info = VUniConcatInfo()
+ info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(uniconcat=info.fieldnums)]
+
+def test_vunisliceinfo():
+ info = VUniSliceInfo()
+ info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX), tag(30, TAGBOX)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(unislice=info.fieldnums)]
+
# ____________________________________________________________
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_string.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_string.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_string.py Fri Oct 22 23:09:43 2010
@@ -6,14 +6,17 @@
class StringTests:
+ _str, _chr = str, chr
+
def test_eq_residual(self):
+ _str = self._str
jitdriver = JitDriver(greens = [], reds = ['n', 'i', 's'])
- global_s = "hello"
+ global_s = _str("hello")
def f(n, b, s):
if b:
- s += "ello"
+ s += _str("ello")
else:
- s += "allo"
+ s += _str("allo")
i = 0
while n > 0:
jitdriver.can_enter_jit(s=s, n=n, i=i)
@@ -21,18 +24,19 @@
n -= 1 + (s == global_s)
i += 1
return i
- res = self.meta_interp(f, [10, True, 'h'], listops=True)
+ res = self.meta_interp(f, [10, True, _str('h')], listops=True)
assert res == 5
self.check_loops(**{self.CALL: 1, self.CALL_PURE: 0})
def test_eq_folded(self):
+ _str = self._str
jitdriver = JitDriver(greens = ['s'], reds = ['n', 'i'])
- global_s = "hello"
+ global_s = _str("hello")
def f(n, b, s):
if b:
- s += "ello"
+ s += _str("ello")
else:
- s += "allo"
+ s += _str("allo")
i = 0
while n > 0:
jitdriver.can_enter_jit(s=s, n=n, i=i)
@@ -40,31 +44,18 @@
n -= 1 + (s == global_s)
i += 1
return i
- res = self.meta_interp(f, [10, True, 'h'], listops=True)
+ res = self.meta_interp(f, [10, True, _str('h')], listops=True)
assert res == 5
self.check_loops(**{self.CALL: 0, self.CALL_PURE: 0})
def test_newstr(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['n', 'm'])
def f(n, m):
while True:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- bytecode = 'adlfkj' + chr(n)
- res = bytecode[n]
- m -= 1
- if m < 0:
- return ord(res)
- res = self.meta_interp(f, [6, 10])
- assert res == 6
-
- def test_newunicode(self):
- jitdriver = JitDriver(greens = [], reds = ['n', 'm'])
- def f(n, m):
- while True:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- bytecode = u'adlfkj' + unichr(n)
+ bytecode = _str('adlfkj') + _chr(n)
res = bytecode[n]
m -= 1
if m < 0:
@@ -73,95 +64,96 @@
assert res == 6
def test_char2string_pure(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n):
- while n > 0:
- jitdriver.can_enter_jit(n=n)
- jitdriver.jit_merge_point(n=n)
- s = dochr(n)
- if not we_are_jitted():
- s += s # forces to be a string
- if n > 100:
- escape(s)
- n -= 1
- return 42
- self.meta_interp(f, [6])
- self.check_loops(newstr=0, strsetitem=0, strlen=0,
- newunicode=0, unicodesetitem=0, unicodelen=0)
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n):
+ while n > 0:
+ jitdriver.can_enter_jit(n=n)
+ jitdriver.jit_merge_point(n=n)
+ s = _chr(n)
+ if not we_are_jitted():
+ s += s # forces to be a string
+ if n > 100:
+ escape(s)
+ n -= 1
+ return 42
+ self.meta_interp(f, [6])
+ self.check_loops(newstr=0, strsetitem=0, strlen=0,
+ newunicode=0, unicodesetitem=0, unicodelen=0)
def test_char2string_escape(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['n', 'total'])
- @dont_look_inside
- def escape(x):
- return ord(x[0])
- def f(n):
- total = 0
- while n > 0:
- jitdriver.can_enter_jit(n=n, total=total)
- jitdriver.jit_merge_point(n=n, total=total)
- s = dochr(n)
- if not we_are_jitted():
- s += s # forces to be a string
- total += escape(s)
- n -= 1
- return total
- res = self.meta_interp(f, [6])
- assert res == 21
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['n', 'total'])
+ @dont_look_inside
+ def escape(x):
+ return ord(x[0])
+ def f(n):
+ total = 0
+ while n > 0:
+ jitdriver.can_enter_jit(n=n, total=total)
+ jitdriver.jit_merge_point(n=n, total=total)
+ s = _chr(n)
+ if not we_are_jitted():
+ s += s # forces to be a string
+ total += escape(s)
+ n -= 1
+ return total
+ res = self.meta_interp(f, [6])
+ assert res == 21
def test_char2string2char(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'total'])
- def f(m):
- total = 0
- while m > 0:
- jitdriver.can_enter_jit(m=m, total=total)
- jitdriver.jit_merge_point(m=m, total=total)
- string = dochr(m)
- if m > 100:
- string += string # forces to be a string
- # read back the character
- c = string[0]
- total += ord(c)
- m -= 1
- return total
- res = self.meta_interp(f, [6])
- assert res == 21
- self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0,
- newunicode=0, unicodegetitem=0, unicodesetitem=0,
- unicodelen=0)
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['m', 'total'])
+ def f(m):
+ total = 0
+ while m > 0:
+ jitdriver.can_enter_jit(m=m, total=total)
+ jitdriver.jit_merge_point(m=m, total=total)
+ string = _chr(m)
+ if m > 100:
+ string += string # forces to be a string
+ # read back the character
+ c = string[0]
+ total += ord(c)
+ m -= 1
+ return total
+ res = self.meta_interp(f, [6])
+ assert res == 21
+ self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0,
+ newunicode=0, unicodegetitem=0, unicodesetitem=0,
+ unicodelen=0)
def test_strconcat_pure(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- mylist = [somestr+str(i) for i in range(10)]
- def f(n, m):
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + mylist[m]
- if m > 100:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [6, 7])
- self.check_loops(newstr=0, strsetitem=0,
- newunicode=0, unicodesetitem=0,
- call=0, call_pure=0)
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ mylist = [_str("abc") + _str(i) for i in range(10)]
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = mylist[n] + mylist[m]
+ if m > 100:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(newstr=0, strsetitem=0,
+ newunicode=0, unicodesetitem=0,
+ call=0, call_pure=0)
def test_strconcat_escape_str_str(self):
+ _str = self._str
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
@@ -171,46 +163,64 @@
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=0, copystrcontent=2,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=0, copystrcontent=2,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=0,
+ copyunicodecontent=2,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_str_char(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + chr(m)
+ s = mylist[n] + _chr(m)
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=1,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_char_str(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = chr(n) + mylist[m]
+ s = _chr(n) + mylist[m]
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=1,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_char_char(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
@@ -219,91 +229,132 @@
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = chr(n) + chr(m)
+ s = _chr(n) + _chr(m)
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=2, copystrcontent=0,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=2, copystrcontent=0,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=2,
+ copyunicodecontent=0,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_str_char_str(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + chr(n) + mylist[m]
+ s = mylist[n] + _chr(n) + mylist[m]
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=2,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=2,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=2,
+ call=1, call_pure=0) # escape
def test_strconcat_guard_fail(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- mylist = [somestr+str(i) for i in range(12)]
- def f(n, m):
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + mylist[m]
- if m & 1:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [6, 10])
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ mylist = [_str("abc") + _str(i) for i in range(12)]
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = mylist[n] + mylist[m]
+ if m & 1:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 10])
def test_strslice(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n, m):
- assert n >= 0
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = "foobarbazetc"[m:n]
- if m <= 5:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [10, 10])
+ _str = self._str
+ longstring = _str("foobarbazetc")
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n, m):
+ assert n >= 0
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = longstring[m:n]
+ if m <= 5:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [10, 10])
def test_streq_char(self):
- for somestr in ["?abcdefg", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n, m):
- assert n >= 0
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = somestr[:m]
- escape(s == "?")
- m -= 1
- return 42
- self.meta_interp(f, [6, 7])
- self.check_loops(newstr=0, newunicode=0)
-
-
-class TestOOtype(StringTests, OOJitMixin):
- CALL = "oosend"
- CALL_PURE = "oosend_pure"
+ _str = self._str
+ longstring = _str("?abcdefg")
+ somechar = _str("?")
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n, m):
+ assert n >= 0
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = longstring[:m]
+ escape(s == somechar)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(newstr=0, newunicode=0)
+
+
+#class TestOOtype(StringTests, OOJitMixin):
+# CALL = "oosend"
+# CALL_PURE = "oosend_pure"
class TestLLtype(StringTests, LLJitMixin):
CALL = "call"
CALL_PURE = "call_pure"
+
+class TestLLtypeUnicode(TestLLtype):
+ _str, _chr = unicode, unichr
+
+ def test_str2unicode(self):
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ class Foo:
+ pass
+ @dont_look_inside
+ def escape(x):
+ assert x == _str("6y")
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ foo = Foo()
+ foo.y = chr(m)
+ foo.y = "y"
+ s = _str(str(n)) + _str(foo.y)
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(call=3, # str(), _str(), escape()
+ newunicode=1, unicodegetitem=0,
+ unicodesetitem=1, copyunicodecontent=1)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py Fri Oct 22 23:09:43 2010
@@ -93,7 +93,7 @@
lst = []
vrefinfo.continue_tracing = lambda vref, virtual: \
lst.append((vref, virtual))
- resumereader.consume_vref_and_vable(vrefinfo, None)
+ resumereader.consume_vref_and_vable(vrefinfo, None, None)
del vrefinfo.continue_tracing
assert len(lst) == 1
lltype.cast_opaque_ptr(lltype.Ptr(JIT_VIRTUAL_REF),
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py Fri Oct 22 23:09:43 2010
@@ -296,6 +296,69 @@
assert res == 1
self.check_loops(int_add=1) # I get 13 without the loop_header()
+ def test_omit_can_enter_jit(self):
+ # Simple test comparing the effects of always giving a can_enter_jit(),
+ # or not giving any. Mostly equivalent, except that if given, it is
+ # ignored the first time, and so it ends up taking one extra loop to
+ # start JITting.
+ mydriver = JitDriver(greens=[], reds=['m'])
+ #
+ for i2 in range(10):
+ def f2(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f2, [i2])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f2: no loop generated for i2==%d" % i2
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ for i1 in range(10):
+ def f1(m):
+ while m > 0:
+ mydriver.can_enter_jit(m=m)
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f1, [i1])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f1: no loop generated for i1==%d" % i1
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ assert i1 - 1 == i2
+
+ def test_no_loop_at_all(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f2(m):
+ mydriver.jit_merge_point(m=m)
+ return m - 1
+ def f1(m):
+ while m > 0:
+ m = f2(m)
+ self.meta_interp(f1, [8])
+ # it should generate one "loop" only, which ends in a FINISH
+ # corresponding to the return from f2.
+ self.check_tree_loop_count(1)
+ self.check_loop_count(0)
+
+ def test_simple_loop(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f1(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m = m - 1
+ self.meta_interp(f1, [8])
+ self.check_loop_count(1)
+ self.check_loops({'int_sub': 1, 'int_gt': 1, 'guard_true': 1,
+ 'jump': 1})
+
class TestLLWarmspot(WarmspotTests, LLJitMixin):
CPUClass = runner.LLtypeCPU
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/virtualref.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/virtualref.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/virtualref.py Fri Oct 22 23:09:43 2010
@@ -16,7 +16,8 @@
('virtualref_index', lltype.Signed),
('forced', rclass.OBJECTPTR))
self.jit_virtual_ref_vtable = lltype.malloc(rclass.OBJECT_VTABLE,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self.jit_virtual_ref_vtable.name = rclass.alloc_array_name(
'jit_virtual_ref')
# build some constants
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py Fri Oct 22 23:09:43 2010
@@ -110,15 +110,16 @@
for i in range(len(block.operations)):
op = block.operations[i]
if (op.opname == 'jit_marker' and
- op.args[0].value == marker_name):
+ op.args[0].value == marker_name and
+ op.args[1].value.active): # the jitdriver
results.append((graph, block, i))
return results
def find_can_enter_jit(graphs):
- results = _find_jit_marker(graphs, 'can_enter_jit')
- if not results:
- raise Exception("no can_enter_jit found!")
- return results
+ return _find_jit_marker(graphs, 'can_enter_jit')
+
+def find_loop_headers(graphs):
+ return _find_jit_marker(graphs, 'loop_header')
def find_jit_merge_points(graphs):
results = _find_jit_marker(graphs, 'jit_merge_point')
@@ -211,9 +212,9 @@
"there are multiple jit_merge_points with the same jitdriver"
def split_graph_and_record_jitdriver(self, graph, block, pos):
- jd = JitDriverStaticData()
- jd._jit_merge_point_pos = (graph, block, pos)
op = block.operations[pos]
+ jd = JitDriverStaticData()
+ jd._jit_merge_point_pos = (graph, op)
args = op.args[2:]
s_binding = self.translator.annotator.binding
jd._portal_args_s = [s_binding(v) for v in args]
@@ -286,10 +287,20 @@
def make_virtualizable_infos(self):
vinfos = {}
for jd in self.jitdrivers_sd:
+ #
+ jd.greenfield_info = None
+ for name in jd.jitdriver.greens:
+ if '.' in name:
+ from pypy.jit.metainterp.greenfield import GreenFieldInfo
+ jd.greenfield_info = GreenFieldInfo(self.cpu, jd)
+ break
+ #
if not jd.jitdriver.virtualizables:
jd.virtualizable_info = None
jd.index_of_virtualizable = -1
continue
+ else:
+ assert jd.greenfield_info is None, "XXX not supported yet"
#
jitdriver = jd.jitdriver
assert len(jitdriver.virtualizables) == 1 # for now
@@ -457,8 +468,7 @@
self.make_args_specification(jd)
def make_args_specification(self, jd):
- graph, block, index = jd._jit_merge_point_pos
- op = block.operations[index]
+ graph, op = jd._jit_merge_point_pos
greens_v, reds_v = support.decode_hp_hint_args(op)
ALLARGS = [v.concretetype for v in (greens_v + reds_v)]
jd._green_args_spec = [v.concretetype for v in greens_v]
@@ -474,26 +484,37 @@
[lltype.Signed, llmemory.GCREF], RESTYPE)
def rewrite_can_enter_jits(self):
- can_enter_jits = find_can_enter_jit(self.translator.graphs)
sublists = {}
for jd in self.jitdrivers_sd:
- sublists[jd.jitdriver] = []
+ sublists[jd.jitdriver] = jd, []
+ jd.no_loop_header = True
+ #
+ loop_headers = find_loop_headers(self.translator.graphs)
+ for graph, block, index in loop_headers:
+ op = block.operations[index]
+ jitdriver = op.args[1].value
+ assert jitdriver in sublists, \
+ "loop_header with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
+ jd.no_loop_header = False
+ #
+ can_enter_jits = find_can_enter_jit(self.translator.graphs)
for graph, block, index in can_enter_jits:
op = block.operations[index]
jitdriver = op.args[1].value
assert jitdriver in sublists, \
"can_enter_jit with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
origportalgraph = jd._jit_merge_point_pos[0]
if graph is not origportalgraph:
- sublists[jitdriver].append((graph, block, index))
+ sublist.append((graph, block, index))
+ jd.no_loop_header = False
else:
pass # a 'can_enter_jit' before the 'jit-merge_point', but
# originally in the same function: we ignore it here
# see e.g. test_jitdriver.test_simple
for jd in self.jitdrivers_sd:
- sublist = sublists[jd.jitdriver]
- assert len(sublist) > 0, \
- "found no can_enter_jit for %r" % (jd.jitdriver,)
+ _, sublist = sublists[jd.jitdriver]
self.rewrite_can_enter_jit(jd, sublist)
def rewrite_can_enter_jit(self, jd, can_enter_jits):
@@ -501,6 +522,19 @@
FUNCPTR = jd._PTR_JIT_ENTER_FUNCTYPE
jit_enter_fnptr = self.helper_func(FUNCPTR, jd._maybe_enter_jit_fn)
+ if len(can_enter_jits) == 0:
+ # see test_warmspot.test_no_loop_at_all
+ operations = jd.portal_graph.startblock.operations
+ op1 = operations[0]
+ assert (op1.opname == 'jit_marker' and
+ op1.args[0].value == 'jit_merge_point')
+ op0 = SpaceOperation(
+ 'jit_marker',
+ [Constant('can_enter_jit', lltype.Void)] + op1.args[1:],
+ None)
+ operations.insert(0, op0)
+ can_enter_jits = [(jd.portal_graph, jd.portal_graph.startblock, 0)]
+
for graph, block, index in can_enter_jits:
if graph is jd._jit_merge_point_pos[0]:
continue
@@ -709,8 +743,14 @@
# ____________________________________________________________
# Now mutate origportalgraph to end with a call to portal_runner_ptr
#
- _, origblock, origindex = jd._jit_merge_point_pos
- op = origblock.operations[origindex]
+ _, op = jd._jit_merge_point_pos
+ for origblock in origportalgraph.iterblocks():
+ if op in origblock.operations:
+ break
+ else:
+ assert False, "lost the operation %r in the graph %r" % (
+ op, origportalgraph)
+ origindex = origblock.operations.index(op)
assert op.opname == 'jit_marker'
assert op.args[0].value == 'jit_merge_point'
greens_v, reds_v = support.decode_hp_hint_args(op)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py Fri Oct 22 23:09:43 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, llmemory, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import hlstr, llstr, cast_base_ptr_to_instance
from pypy.rpython.annlowlevel import cast_object_to_ptr
@@ -24,7 +24,11 @@
"""
INPUT = lltype.typeOf(x)
if INPUT is lltype.Signed:
- return lltype.cast_primitive(TYPE, x) # XXX missing: Ptr(non-gc)
+ if isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'raw':
+ # non-gc pointer
+ return rffi.cast(TYPE, x)
+ else:
+ return lltype.cast_primitive(TYPE, x)
elif INPUT is lltype.Float:
assert TYPE is lltype.Float
return x
@@ -172,6 +176,9 @@
meth(default_value)
def set_param_threshold(self, threshold):
+ if threshold < 0:
+ self.increment_threshold = 0 # never reach the THRESHOLD_LIMIT
+ return
if threshold < 2:
threshold = 2
self.increment_threshold = (self.THRESHOLD_LIMIT // threshold) + 1
Modified: pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py Fri Oct 22 23:09:43 2010
@@ -40,6 +40,9 @@
config.objspace.usemodules.array = True
config.objspace.usemodules._weakref = False
config.objspace.usemodules._sre = False
+#
+config.objspace.usemodules._ffi = True
+#
set_pypy_opt_level(config, level='jit')
config.objspace.std.withinlineddict = True
Modified: pypy/branch/fast-forward/pypy/jit/tl/pypyjit_child.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tl/pypyjit_child.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tl/pypyjit_child.py Fri Oct 22 23:09:43 2010
@@ -16,7 +16,7 @@
interp.heap.malloc_nonmovable = returns_null # XXX
from pypy.jit.backend.llgraph.runner import LLtypeCPU
- LLtypeCPU.supports_floats = False # for now
+ #LLtypeCPU.supports_floats = False # for now
apply_jit(interp, graph, LLtypeCPU)
Modified: pypy/branch/fast-forward/pypy/jit/tool/loopviewer.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tool/loopviewer.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tool/loopviewer.py Fri Oct 22 23:09:43 2010
@@ -9,7 +9,7 @@
import optparse
from pprint import pprint
from pypy.tool import logparser
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.history import ConstInt
from pypy.rpython.lltypesystem import llmemory, lltype
Modified: pypy/branch/fast-forward/pypy/jit/tool/showstats.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tool/showstats.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tool/showstats.py Fri Oct 22 23:09:43 2010
@@ -4,7 +4,7 @@
import autopath
import sys, py
from pypy.tool import logparser
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.resoperation import rop
from pypy.rpython.lltypesystem import lltype, llmemory
Modified: pypy/branch/fast-forward/pypy/jit/tool/traceviewer.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tool/traceviewer.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tool/traceviewer.py Fri Oct 22 23:09:43 2010
@@ -253,10 +253,10 @@
def main(loopfile, use_threshold, view=True):
countname = py.path.local(loopfile + '.count')
if countname.check():
- counts = [re.split('(<code)|(<loop)', line, maxsplit=1)
- for line in countname.readlines()]
- counts = Counts([('<code' + k.strip("\n"), int(v.strip('\n').strip()))
- for v, _, _, k in counts])
+ #counts = [line.split(':', 1) for line in countname.readlines()]
+ #counts = Counts([('<code' + k.strip("\n"), int(v.strip('\n').strip()))
+ # for v, k in counts])
+ counts = Counts([])
l = list(sorted(counts.values()))
if len(l) > 20 and use_threshold:
counts.threshold = l[-20]
Modified: pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py (original)
+++ pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py Fri Oct 22 23:09:43 2010
@@ -2,9 +2,11 @@
from pypy.interpreter.error import OperationError, operationerrfmt
from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, applevel
from pypy.interpreter.gateway import interp2app, ObjSpace
-from pypy.interpreter.typedef import TypeDef, make_weakref_descr
+from pypy.interpreter.typedef import TypeDef
from pypy.interpreter.argument import Arguments
from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.typedef import GetSetProperty, descr_get_dict
+from pypy.interpreter.typedef import descr_set_dict
from pypy.rlib.rarithmetic import r_uint, intmask
from pypy.rlib.objectmodel import compute_identity_hash
from pypy.rlib.debug import make_sure_not_resized
@@ -57,6 +59,14 @@
self.bases_w = bases
self.w_dict = w_dict
+ def instantiate(self, space):
+ cache = space.fromcache(Cache)
+ if self.lookup(space, '__del__') is not None:
+ w_inst = cache.cls_with_del(space, self)
+ else:
+ w_inst = cache.cls_without_del(space, self)
+ return w_inst
+
def getdict(self):
return self.w_dict
@@ -100,15 +110,15 @@
return False
@jit.unroll_safe
- def lookup(self, space, w_attr):
+ def lookup(self, space, attr):
# returns w_value or interplevel None
- w_result = space.finditem(self.w_dict, w_attr)
+ w_result = space.finditem_str(self.w_dict, attr)
if w_result is not None:
return w_result
for base in self.bases_w:
# XXX fix annotation of bases_w to be a list of W_ClassObjects
assert isinstance(base, W_ClassObject)
- w_result = base.lookup(space, w_attr)
+ w_result = base.lookup(space, attr)
if w_result is not None:
return w_result
return None
@@ -122,7 +132,7 @@
return space.wrap(self.name)
elif name == "__bases__":
return space.newtuple(self.bases_w)
- w_value = self.lookup(space, w_attr)
+ w_value = self.lookup(space, name)
if w_value is None:
raise operationerrfmt(
space.w_AttributeError,
@@ -147,7 +157,7 @@
self.setbases(space, w_value)
return
elif name == "__del__":
- if self.lookup(space, w_attr) is None:
+ if self.lookup(space, name) is None:
msg = ("a __del__ method added to an existing class "
"will not be called")
space.warn(msg, space.w_RuntimeWarning)
@@ -195,13 +205,20 @@
# NOT_RPYTHON
return '<W_ClassObject(%s)>' % self.name
+class Cache:
+ def __init__(self, space):
+ from pypy.interpreter.typedef import _usersubclswithfeature
+ # evil
+ self.cls_without_del = _usersubclswithfeature(
+ space.config, W_InstanceObject, "dict", "weakref")
+ self.cls_with_del = _usersubclswithfeature(
+ space.config, self.cls_without_del, "del")
+
+
def class_descr_call(space, w_self, __args__):
self = space.interp_w(W_ClassObject, w_self)
- if self.lookup(space, space.wrap('__del__')) is not None:
- w_inst = W_InstanceObjectWithDel(space, self)
- else:
- w_inst = W_InstanceObject(space, self)
- w_init = w_inst.getattr_from_class(space, space.wrap('__init__'))
+ w_inst = self.instantiate(space)
+ w_init = w_inst.getattr_from_class(space, '__init__')
if w_init is not None:
w_result = space.call_args(w_init, __args__)
if not space.is_w(w_result, space.w_None):
@@ -235,7 +252,7 @@
def make_unary_instance_method(name):
def unaryop(self, space):
- w_meth = self.getattr(space, space.wrap(name), True)
+ w_meth = self.getattr(space, name, True)
return space.call_function(w_meth)
unaryop.func_name = name
return unaryop
@@ -243,7 +260,7 @@
def make_binary_returning_notimplemented_instance_method(name):
def binaryop(self, space, w_other):
try:
- w_meth = self.getattr(space, space.wrap(name), False)
+ w_meth = self.getattr(space, name, False)
except OperationError, e:
if e.match(space, space.w_AttributeError):
return space.w_NotImplemented
@@ -268,7 +285,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_meth = self.getattr(space, space.wrap(specialname), False)
+ w_meth = self.getattr(space, specialname, False)
if w_meth is None:
return space.w_NotImplemented
return space.call_function(w_meth, w_b)
@@ -279,7 +296,7 @@
def rbinaryop(self, space, w_other):
w_a, w_b = _coerce_helper(space, self, w_other)
if w_a is None or w_a is self:
- w_meth = self.getattr(space, space.wrap(rspecialname), False)
+ w_meth = self.getattr(space, rspecialname, False)
if w_meth is None:
return space.w_NotImplemented
return space.call_function(w_meth, w_other)
@@ -303,46 +320,34 @@
raise OperationError(
space.w_TypeError,
space.wrap("instance() first arg must be class"))
- if space.is_w(w_dict, space.w_None):
- w_dict = None
- elif not space.is_true(space.isinstance(w_dict, space.w_dict)):
- raise OperationError(
- space.w_TypeError,
- space.wrap("instance() second arg must be dictionary or None"))
- return W_InstanceObject(space, w_class, w_dict)
+ w_result = w_class.instantiate(space)
+ if not space.is_w(w_dict, space.w_None):
+ w_result.setdict(space, w_dict)
+ return w_result
class W_InstanceObject(Wrappable):
- def __init__(self, space, w_class, w_dict=None):
- if w_dict is None:
- w_dict = space.newdict(instance=True)
+ def __init__(self, space, w_class):
+ # note that user_setup is overridden by the typedef.py machinery
+ self.user_setup(space, space.gettypeobject(self.typedef))
assert isinstance(w_class, W_ClassObject)
self.w_class = w_class
- self.w_dict = w_dict
- self.space = space
-
- def getdict(self):
- return self.w_dict
- def setdict(self, space, w_dict):
- if (w_dict is None or
- not space.is_true(space.isinstance(w_dict, space.w_dict))):
- raise OperationError(
- space.w_TypeError,
- space.wrap("__dict__ must be a dictionary object"))
- self.w_dict = w_dict
+ def user_setup(self, space, w_subtype):
+ self.space = space
- def setclass(self, space, w_class):
+ def set_oldstyle_class(self, space, w_class):
if w_class is None or not isinstance(w_class, W_ClassObject):
raise OperationError(
space.w_TypeError,
space.wrap("__class__ must be set to a class"))
self.w_class = w_class
- def getattr_from_class(self, space, w_name):
+ def getattr_from_class(self, space, name):
# Look up w_name in the class dict, and call its __get__.
# This method ignores the instance dict and the __getattr__.
# Returns None if not found.
- w_value = self.w_class.lookup(space, w_name)
+ assert isinstance(name, str)
+ w_value = self.w_class.lookup(space, name)
if w_value is None:
return None
w_descr_get = space.lookup(w_value, '__get__')
@@ -350,19 +355,20 @@
return w_value
return space.call_function(w_descr_get, w_value, self, self.w_class)
- def getattr(self, space, w_name, exc=True):
+ def getattr(self, space, name, exc=True):
# Normal getattr rules: look up w_name in the instance dict,
# in the class dict, and then via a call to __getatttr__.
- w_result = space.finditem(self.w_dict, w_name)
+ assert isinstance(name, str)
+ w_result = self.getdictvalue(space, name)
if w_result is not None:
return w_result
- w_result = self.getattr_from_class(space, w_name)
+ w_result = self.getattr_from_class(space, name)
if w_result is not None:
return w_result
- w_meth = self.getattr_from_class(space, space.wrap('__getattr__'))
+ w_meth = self.getattr_from_class(space, '__getattr__')
if w_meth is not None:
try:
- return space.call_function(w_meth, w_name)
+ return space.call_function(w_meth, space.wrap(name))
except OperationError, e:
if not exc and e.match(space, space.w_AttributeError):
return None # eat the AttributeError
@@ -372,7 +378,7 @@
raise operationerrfmt(
space.w_AttributeError,
"%s instance has no attribute '%s'",
- self.w_class.name, space.str_w(w_name))
+ self.w_class.name, name)
else:
return None
@@ -380,44 +386,46 @@
name = space.str_w(w_attr)
if len(name) >= 8 and name[0] == '_':
if name == "__dict__":
- return self.w_dict
+ return self.getdict()
elif name == "__class__":
return self.w_class
- return self.getattr(space, w_attr)
+ return self.getattr(space, name)
def descr_setattr(self, space, w_name, w_value):
name = unwrap_attr(space, w_name)
- w_meth = self.getattr_from_class(space, space.wrap('__setattr__'))
+ w_meth = self.getattr_from_class(space, '__setattr__')
if name and name[0] == "_":
if name == '__dict__':
self.setdict(space, w_value)
return
if name == '__class__':
- self.setclass(space, w_value)
+ self.set_oldstyle_class(space, w_value)
return
if name == '__del__' and w_meth is None:
- if (not isinstance(self, W_InstanceObjectWithDel)
- and space.finditem(self.w_dict, w_name) is None):
+ cache = space.fromcache(Cache)
+ if (not isinstance(self, cache.cls_with_del)
+ and self.getdictvalue(space, '__del__') is None):
msg = ("a __del__ method added to an instance "
"with no __del__ in the class will not be called")
space.warn(msg, space.w_RuntimeWarning)
if w_meth is not None:
space.call_function(w_meth, w_name, w_value)
else:
- self.setdictvalue(space, name, w_value)
+ # bit obscure: appease normalization
+ self.setdictvalue(space, name, w_value, True)
def descr_delattr(self, space, w_name):
name = unwrap_attr(space, w_name)
if name and name[0] == "_":
if name == '__dict__':
# use setdict to raise the error
- self.setdict(space, None)
+ self.setdict(space, space.w_None)
return
elif name == '__class__':
- # use setclass to raise the error
- self.setclass(space, None)
+ # use set_oldstyle_class to raise the error
+ self.set_oldstyle_class(space, None)
return
- w_meth = self.getattr_from_class(space, space.wrap('__delattr__'))
+ w_meth = self.getattr_from_class(space, '__delattr__')
if w_meth is not None:
space.call_function(w_meth, w_name)
else:
@@ -428,7 +436,7 @@
self.w_class.name, name)
def descr_repr(self, space):
- w_meth = self.getattr(space, space.wrap('__repr__'), False)
+ w_meth = self.getattr(space, '__repr__', False)
if w_meth is None:
w_class = self.w_class
mod = w_class.get_module_string(space)
@@ -436,19 +444,19 @@
return space.call_function(w_meth)
def descr_str(self, space):
- w_meth = self.getattr(space, space.wrap('__str__'), False)
+ w_meth = self.getattr(space, '__str__', False)
if w_meth is None:
return self.descr_repr(space)
return space.call_function(w_meth)
def descr_unicode(self, space):
- w_meth = self.getattr(space, space.wrap('__unicode__'), False)
+ w_meth = self.getattr(space, '__unicode__', False)
if w_meth is None:
return self.descr_str(space)
return space.call_function(w_meth)
def descr_len(self, space):
- w_meth = self.getattr(space, space.wrap('__len__'))
+ w_meth = self.getattr(space, '__len__')
w_result = space.call_function(w_meth)
if space.is_true(space.isinstance(w_result, space.w_int)):
if space.is_true(space.lt(w_result, space.wrap(0))):
@@ -461,22 +469,22 @@
space.wrap("__len__() should return an int"))
def descr_getitem(self, space, w_key):
- w_meth = self.getattr(space, space.wrap('__getitem__'))
+ w_meth = self.getattr(space, '__getitem__')
return space.call_function(w_meth, w_key)
def descr_setitem(self, space, w_key, w_value):
- w_meth = self.getattr(space, space.wrap('__setitem__'))
+ w_meth = self.getattr(space, '__setitem__')
space.call_function(w_meth, w_key, w_value)
def descr_delitem(self, space, w_key):
- w_meth = self.getattr(space, space.wrap('__delitem__'))
+ w_meth = self.getattr(space, '__delitem__')
space.call_function(w_meth, w_key)
def descr_iter(self, space):
- w_meth = self.getattr(space, space.wrap('__iter__'), False)
+ w_meth = self.getattr(space, '__iter__', False)
if w_meth is not None:
return space.call_function(w_meth)
- w_meth = self.getattr(space, space.wrap('__getitem__'), False)
+ w_meth = self.getattr(space, '__getitem__', False)
if w_meth is None:
raise OperationError(
space.w_TypeError,
@@ -486,14 +494,14 @@
# don't see the point
def descr_getslice(self, space, w_i, w_j):
- w_meth = self.getattr(space, space.wrap('__getslice__'), False)
+ w_meth = self.getattr(space, '__getslice__', False)
if w_meth is not None:
return space.call_function(w_meth, w_i, w_j)
else:
return space.getitem(self, space.newslice(w_i, w_j, space.w_None))
def descr_setslice(self, space, w_i, w_j, w_sequence):
- w_meth = self.getattr(space, space.wrap('__setslice__'), False)
+ w_meth = self.getattr(space, '__setslice__', False)
if w_meth is not None:
space.call_function(w_meth, w_i, w_j, w_sequence)
else:
@@ -501,20 +509,20 @@
w_sequence)
def descr_delslice(self, space, w_i, w_j):
- w_meth = self.getattr(space, space.wrap('__delslice__'), False)
+ w_meth = self.getattr(space, '__delslice__', False)
if w_meth is not None:
space.call_function(w_meth, w_i, w_j)
else:
return space.delitem(self, space.newslice(w_i, w_j, space.w_None))
def descr_call(self, space, __args__):
- w_meth = self.getattr(space, space.wrap('__call__'))
+ w_meth = self.getattr(space, '__call__')
return space.call_args(w_meth, __args__)
def descr_nonzero(self, space):
- w_func = self.getattr(space, space.wrap('__nonzero__'), False)
+ w_func = self.getattr(space, '__nonzero__', False)
if w_func is None:
- w_func = self.getattr(space, space.wrap('__len__'), False)
+ w_func = self.getattr(space, '__len__', False)
if w_func is None:
return space.w_True
w_result = space.call_function(w_func)
@@ -538,7 +546,7 @@
not isinstance(w_b, W_InstanceObject)):
return space.cmp(w_a, w_b)
if isinstance(w_a, W_InstanceObject):
- w_func = w_a.getattr(space, space.wrap('__cmp__'), False)
+ w_func = w_a.getattr(space, '__cmp__', False)
if w_func is not None:
w_res = space.call_function(w_func, w_b)
if space.is_w(w_res, space.w_NotImplemented):
@@ -557,7 +565,7 @@
return space.wrap(-1)
return space.wrap(0)
if isinstance(w_b, W_InstanceObject):
- w_func = w_b.getattr(space, space.wrap('__cmp__'), False)
+ w_func = w_b.getattr(space, '__cmp__', False)
if w_func is not None:
w_res = space.call_function(w_func, w_a)
if space.is_w(w_res, space.w_NotImplemented):
@@ -578,10 +586,10 @@
return space.w_NotImplemented
def descr_hash(self, space):
- w_func = self.getattr(space, space.wrap('__hash__'), False)
+ w_func = self.getattr(space, '__hash__', False)
if w_func is None:
- w_eq = self.getattr(space, space.wrap('__eq__'), False)
- w_cmp = self.getattr(space, space.wrap('__cmp__'), False)
+ w_eq = self.getattr(space, '__eq__', False)
+ w_cmp = self.getattr(space, '__cmp__', False)
if w_eq is not None or w_cmp is not None:
raise OperationError(space.w_TypeError,
space.wrap("unhashable instance"))
@@ -617,7 +625,7 @@
return self.descr_int(space)
def descr_index(self, space):
- w_func = self.getattr(space, space.wrap('__index__'), False)
+ w_func = self.getattr(space, '__index__', False)
if w_func is not None:
return space.call_function(w_func)
raise OperationError(
@@ -625,7 +633,7 @@
space.wrap("object cannot be interpreted as an index"))
def descr_contains(self, space, w_obj):
- w_func = self.getattr(space, space.wrap('__contains__'), False)
+ w_func = self.getattr(space, '__contains__', False)
if w_func is not None:
return space.wrap(space.is_true(space.call_function(w_func, w_obj)))
# now do it ourselves
@@ -648,7 +656,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_func = self.getattr(space, space.wrap('__pow__'), False)
+ w_func = self.getattr(space, '__pow__', False)
if w_func is not None:
return space.call_function(w_func, w_other)
return space.w_NotImplemented
@@ -656,7 +664,7 @@
return space.pow(w_a, w_b, space.w_None)
else:
# CPython also doesn't try coercion in this case
- w_func = self.getattr(space, space.wrap('__pow__'), False)
+ w_func = self.getattr(space, '__pow__', False)
if w_func is not None:
return space.call_function(w_func, w_other, w_modulo)
return space.w_NotImplemented
@@ -668,7 +676,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_func = self.getattr(space, space.wrap('__rpow__'), False)
+ w_func = self.getattr(space, '__rpow__', False)
if w_func is not None:
return space.call_function(w_func, w_other)
return space.w_NotImplemented
@@ -676,13 +684,13 @@
return space.pow(w_b, w_a, space.w_None)
else:
# CPython also doesn't try coercion in this case
- w_func = self.getattr(space, space.wrap('__rpow__'), False)
+ w_func = self.getattr(space, '__rpow__', False)
if w_func is not None:
return space.call_function(w_func, w_other, w_modulo)
return space.w_NotImplemented
def descr_next(self, space):
- w_func = self.getattr(space, space.wrap('next'), False)
+ w_func = self.getattr(space, 'next', False)
if w_func is None:
raise OperationError(space.w_TypeError,
space.wrap("instance has no next() method"))
@@ -691,10 +699,9 @@
def descr_del(self, space):
# Note that this is called from executioncontext.UserDelAction
# via the space.userdel() method.
- w_name = space.wrap('__del__')
- w_func = space.finditem(self.w_dict, w_name)
+ w_func = self.getdictvalue(space, '__del__')
if w_func is None:
- w_func = self.getattr_from_class(space, w_name)
+ w_func = self.getattr_from_class(space, '__del__')
if w_func is not None:
space.call_function(w_func)
@@ -744,6 +751,14 @@
rmeth,
unwrap_spec=["self", ObjSpace, W_Root])
+
+def descr_del_dict(space, w_inst):
+ # use setdict to raise the error
+ w_inst.setdict(space, space.w_None)
+
+dict_descr = GetSetProperty(descr_get_dict, descr_set_dict, descr_del_dict)
+dict_descr.name = '__dict__'
+
W_InstanceObject.typedef = TypeDef("instance",
__new__ = interp2app(descr_instance_new),
__getattribute__ = interp2app(W_InstanceObject.descr_getattribute,
@@ -797,14 +812,11 @@
unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
next = interp2app(W_InstanceObject.descr_next,
unwrap_spec=['self', ObjSpace]),
- __weakref__ = make_weakref_descr(W_InstanceObject),
__del__ = interp2app(W_InstanceObject.descr_del,
unwrap_spec=['self', ObjSpace]),
__exit__ = interp2app(W_InstanceObject.descr_exit,
unwrap_spec=['self', ObjSpace, W_Root, W_Root, W_Root]),
+ __dict__ = dict_descr,
**rawdict
)
-
-class W_InstanceObjectWithDel(W_InstanceObject):
- def __del__(self):
- self._enqueue_for_destruction(self.space)
+W_InstanceObject.typedef.acceptable_as_base_class = False
Modified: pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py (original)
+++ pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py Fri Oct 22 23:09:43 2010
@@ -955,6 +955,31 @@
assert x is b
assert y == 5
+ def test_cant_subclass_instance(self):
+ class A:
+ pass
+ try:
+ class B(type(A())):
+ pass
+ except TypeError:
+ pass
+ else:
+ assert 0, "should have raised"
+
+ def test_dict_descriptor(self):
+ import sys
+ if not hasattr(sys, 'pypy_objspaceclass'):
+ skip("on CPython old-style instances don't have a __dict__ descriptor")
+ class A:
+ pass
+ a = A()
+ a.x = 1
+ descr = type(a).__dict__['__dict__']
+ assert descr.__get__(a) == {'x': 1}
+ descr.__set__(a, {'x': 2})
+ assert a.x == 2
+ raises(TypeError, descr.__delete__, a)
+
class AppTestOldStyleSharing(AppTestOldstyle):
def setup_class(cls):
@@ -993,3 +1018,22 @@
a = 1
b = 2
assert self.is_strdict(A)
+
+class AppTestOldStyleMapDict(AppTestOldstyle):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withmapdict": True})
+ if option.runappdirect:
+ py.test.skip("can only be run on py.py")
+ def has_mapdict(space, w_inst):
+ return space.wrap(w_inst._get_mapdict_map() is not None)
+ cls.w_has_mapdict = cls.space.wrap(gateway.interp2app(has_mapdict))
+
+
+ def test_has_mapdict(self):
+ class A:
+ def __init__(self):
+ self.x = 42
+ a = A()
+ assert a.x == 42
+ assert self.has_mapdict(a)
+
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py Fri Oct 22 23:09:43 2010
@@ -1,5 +1,5 @@
-""" Low-level interface to libffi
+""" Low-level interface to clibffi
"""
from pypy.interpreter.mixedmodule import MixedModule
@@ -46,12 +46,12 @@
if hasattr(interp_rawffi, 'check_HRESULT'):
Module.interpleveldefs['check_HRESULT'] = 'interp_rawffi.check_HRESULT'
- from pypy.rlib import libffi
+ from pypy.rlib import clibffi
for name in ['FUNCFLAG_STDCALL', 'FUNCFLAG_CDECL', 'FUNCFLAG_PYTHONAPI',
'FUNCFLAG_USE_ERRNO', 'FUNCFLAG_USE_LASTERROR',
]:
- if hasattr(libffi, name):
- Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(libffi, name)
+ if hasattr(clibffi, name):
+ Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(clibffi, name)
super(Module, cls).buildloaders()
buildloaders = classmethod(buildloaders)
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/array.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/array.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/array.py Fri Oct 22 23:09:43 2010
@@ -97,7 +97,15 @@
class W_ArrayInstance(W_DataInstance):
def __init__(self, space, shape, length, address=r_uint(0)):
- W_DataInstance.__init__(self, space, shape.size * length, address)
+ # Workaround for a strange behavior of libffi: make sure that
+ # we always have at least 8 bytes. For W_ArrayInstances that are
+ # used as the result value of a function call, ffi_call() writes
+ # 8 bytes into it even if the function's result type asks for less.
+ # This strange behavior is documented.
+ memsize = shape.size * length
+ if memsize < 8:
+ memsize = 8
+ W_DataInstance.__init__(self, space, memsize, address)
self.length = length
self.shape = shape
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/callback.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/callback.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/callback.py Fri Oct 22 23:09:43 2010
@@ -8,8 +8,8 @@
from pypy.module._rawffi.array import get_elem, push_elem
from pypy.module._rawffi.interp_rawffi import W_DataInstance, letter2tp, \
wrap_value, unwrap_value, unwrap_truncate_int
-from pypy.rlib.libffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
-from pypy.rlib.libffi import ffi_type_void
+from pypy.rlib.clibffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
+from pypy.rlib.clibffi import ffi_type_void
from pypy.module._rawffi.tracker import tracker
from pypy.interpreter.error import OperationError
from pypy.interpreter import gateway
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Fri Oct 22 23:09:43 2010
@@ -5,7 +5,7 @@
from pypy.interpreter.gateway import interp2app, NoneNotWrapped
from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.rlib.libffi import *
+from pypy.rlib.clibffi import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.rlib.unroll import unrolling_iterable
import pypy.rlib.rposix as rposix
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/structure.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/structure.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/structure.py Fri Oct 22 23:09:43 2010
@@ -15,7 +15,7 @@
from pypy.module._rawffi.interp_rawffi import wrap_value, unwrap_value
from pypy.module._rawffi.interp_rawffi import unpack_shape_with_length
from pypy.module._rawffi.interp_rawffi import size_alignment
-from pypy.rlib import libffi
+from pypy.rlib import clibffi
from pypy.rlib.rarithmetic import intmask, r_uint
def unpack_fields(space, w_fields):
@@ -34,7 +34,7 @@
def round_up(size, alignment):
return (size + alignment - 1) & -alignment
-def size_alignment_pos(fields):
+def size_alignment_pos(fields, is_union=False):
size = 0
alignment = 1
pos = []
@@ -42,16 +42,20 @@
# fieldtype is a W_Array
fieldsize = fieldtype.size
fieldalignment = fieldtype.alignment
- size = round_up(size, fieldalignment)
alignment = max(alignment, fieldalignment)
- pos.append(size)
- size += intmask(fieldsize)
+ if is_union:
+ pos.append(0)
+ size = max(size, fieldsize)
+ else:
+ size = round_up(size, fieldalignment)
+ pos.append(size)
+ size += intmask(fieldsize)
size = round_up(size, alignment)
return size, alignment, pos
class W_Structure(W_DataShape):
- def __init__(self, space, fields, size, alignment):
+ def __init__(self, space, fields, size, alignment, is_union=False):
name_to_index = {}
if fields is not None:
for i in range(len(fields)):
@@ -60,7 +64,7 @@
raise operationerrfmt(space.w_ValueError,
"duplicate field name %s", name)
name_to_index[name] = i
- size, alignment, pos = size_alignment_pos(fields)
+ size, alignment, pos = size_alignment_pos(fields, is_union)
else: # opaque case
fields = []
pos = []
@@ -104,14 +108,14 @@
descr_fieldoffset.unwrap_spec = ['self', ObjSpace, str]
# get the corresponding ffi_type
- ffi_struct = lltype.nullptr(libffi.FFI_STRUCT_P.TO)
+ ffi_struct = lltype.nullptr(clibffi.FFI_STRUCT_P.TO)
def get_basic_ffi_type(self):
if not self.ffi_struct:
# Repeated fields are delicate. Consider for example
# struct { int a[5]; }
# or struct { struct {int x;} a[5]; }
- # Seeing no corresponding doc in libffi, let's just repeat
+ # Seeing no corresponding doc in clibffi, let's just repeat
# the field 5 times...
fieldtypes = []
for name, tp in self.fields:
@@ -122,7 +126,7 @@
while count + basic_size <= total_size:
fieldtypes.append(basic_ffi_type)
count += basic_size
- self.ffi_struct = libffi.make_struct_ffitype_e(self.size,
+ self.ffi_struct = clibffi.make_struct_ffitype_e(self.size,
self.alignment,
fieldtypes)
return self.ffi_struct.ffistruct
@@ -133,15 +137,17 @@
-def descr_new_structure(space, w_type, w_shapeinfo):
+def descr_new_structure(space, w_type, w_shapeinfo, union=0):
+ is_union = bool(union)
if space.is_true(space.isinstance(w_shapeinfo, space.w_tuple)):
w_size, w_alignment = space.fixedview(w_shapeinfo, expected_length=2)
S = W_Structure(space, None, space.int_w(w_size),
- space.int_w(w_alignment))
+ space.int_w(w_alignment), is_union)
else:
fields = unpack_fields(space, w_shapeinfo)
- S = W_Structure(space, fields, 0, 0)
+ S = W_Structure(space, fields, 0, 0, is_union)
return space.wrap(S)
+descr_new_structure.unwrap_spec = [ObjSpace, W_Root, W_Root, int]
W_Structure.typedef = TypeDef(
'Structure',
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py Fri Oct 22 23:09:43 2010
@@ -71,7 +71,7 @@
return s[num];
}
- char *char_check(char x, char y)
+ const char *char_check(char x, char y)
{
if (y == static_str[0])
return static_str;
@@ -186,13 +186,14 @@
sum_x_y
give perturb get_s2a check_s2a
AAA_first_ordinal_function
+ ret_un_func
""".split()
eci = ExternalCompilationInfo(export_symbols=symbols)
return str(platform.compile([c_file], eci, 'x', standalone=False))
prepare_c_example = staticmethod(prepare_c_example)
def setup_class(cls):
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
space = gettestobjspace(usemodules=('_rawffi', 'struct'))
cls.space = space
cls.w_lib_name = space.wrap(cls.prepare_c_example())
@@ -948,14 +949,15 @@
assert a[4] == 't'
def test_union(self):
- skip("segfaulting")
import _rawffi
longsize = _rawffi.sizeof('l')
- S = _rawffi.Structure((longsize, longsize))
+ S = _rawffi.Structure([('x', 'h'), ('y', 'l')], union=True)
s = S(autofree=False)
+ s.x = 12345
lib = _rawffi.CDLL(self.lib_name)
f = lib.ptr('ret_un_func', [(S, 1)], (S, 1))
ret = f(s)
+ assert ret.y == 1234500, "ret.y == %d" % (ret.y,)
s.free()
class AppTestAutoFree:
Modified: pypy/branch/fast-forward/pypy/module/_socket/test/test_sock_app.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_socket/test/test_sock_app.py (original)
+++ pypy/branch/fast-forward/pypy/module/_socket/test/test_sock_app.py Fri Oct 22 23:09:43 2010
@@ -254,6 +254,7 @@
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
fd = s.fileno()
w_obj = rsocket.make_address(c_addr, addrlen).as_object(fd, space)
+ lltype.free(c_addr_ll, flavor='raw')
assert space.is_true(space.eq(w_obj, space.newtuple([
space.wrap('lo'),
space.wrap(socket.ntohs(8)),
Modified: pypy/branch/fast-forward/pypy/module/_sre/interp_sre.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_sre/interp_sre.py (original)
+++ pypy/branch/fast-forward/pypy/module/_sre/interp_sre.py Fri Oct 22 23:09:43 2010
@@ -290,10 +290,8 @@
def SRE_Pattern__new__(space, w_subtype, w_pattern, flags, w_code,
groups=0, w_groupindex=None, w_indexgroup=None):
n = space.int_w(space.len(w_code))
- code = [0] * n
- for i in range(n):
- x = space.uint_w(space.getitem(w_code, space.wrap(i)))
- code[i] = intmask(x)
+ code = [intmask(space.uint_w(space.getitem(w_code, space.wrap(i))))
+ for i in range(n)]
#
w_srepat = space.allocate_instance(W_SRE_Pattern, w_subtype)
srepat = space.interp_w(W_SRE_Pattern, w_srepat)
Modified: pypy/branch/fast-forward/pypy/module/_weakref/interp__weakref.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_weakref/interp__weakref.py (original)
+++ pypy/branch/fast-forward/pypy/module/_weakref/interp__weakref.py Fri Oct 22 23:09:43 2010
@@ -7,7 +7,7 @@
import weakref
-class WeakrefLifeline(object):
+class WeakrefLifeline(W_Root):
def __init__(self, space):
self.space = space # this is here for W_Root.clear_all_weakrefs()
self.refs_weak = []
Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original)
+++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Fri Oct 22 23:09:43 2010
@@ -213,13 +213,10 @@
subkey = None
else:
subkey = space.str_w(w_subkey)
- dataptr = rffi.str2charp(value)
- try:
+ with rffi.scoped_str2charp(value) as dataptr:
ret = rwinreg.RegSetValue(hkey, subkey, rwinreg.REG_SZ, dataptr, len(value))
- finally:
- rffi.free_charp(dataptr)
- if ret != 0:
- raiseWindowsError(space, ret, 'RegSetValue')
+ if ret != 0:
+ raiseWindowsError(space, ret, 'RegSetValue')
SetValue.unwrap_spec = [ObjSpace, W_Root, W_Root, int, str]
def QueryValue(space, w_hkey, w_subkey):
@@ -238,23 +235,15 @@
subkey = None
else:
subkey = space.str_w(w_subkey)
- bufsize_p = lltype.malloc(rwin32.PLONG.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.PLONG.TO, 1) as bufsize_p:
ret = rwinreg.RegQueryValue(hkey, subkey, None, bufsize_p)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValue')
- buf = lltype.malloc(rffi.CCHARP.TO, bufsize_p[0], flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, bufsize_p[0]) as buf:
ret = rwinreg.RegQueryValue(hkey, subkey, buf, bufsize_p)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValue')
return space.wrap(rffi.charp2strn(buf, bufsize_p[0] - 1))
- finally:
- lltype.free(buf, flavor='raw')
- finally:
- lltype.free(bufsize_p, flavor='raw')
- if ret != 0:
- raiseWindowsError(space, ret, 'RegQueryValue')
QueryValue.unwrap_spec = [ObjSpace, W_Root, W_Root]
def convert_to_regdata(space, w_value, typ):
@@ -413,16 +402,14 @@
value_name is a string indicating the value to query"""
hkey = hkey_w(w_hkey, space)
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
- retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize:
ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, null_dword,
None, retDataSize)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValueEx')
- databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0], flavor='raw')
- try:
- retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType:
ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword,
retType, databuf, retDataSize)
@@ -433,12 +420,6 @@
retDataSize[0], retType[0]),
space.wrap(retType[0]),
])
- finally:
- lltype.free(retType, flavor='raw')
- finally:
- lltype.free(databuf, flavor='raw')
- finally:
- lltype.free(retDataSize, flavor='raw')
QueryValueEx.unwrap_spec = [ObjSpace, W_Root, str]
@@ -455,14 +436,11 @@
The return value is the handle of the opened key.
If the function fails, an exception is raised."""
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegCreateKey(hkey, subkey, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'CreateKey')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
CreateKey.unwrap_spec = [ObjSpace, W_Root, str]
def DeleteKey(space, w_hkey, subkey):
@@ -505,14 +483,11 @@
The result is a new handle to the specified key
If the function fails, an EnvironmentError exception is raised."""
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegOpenKeyEx(hkey, subkey, res, sam, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'RegOpenKeyEx')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
OpenKey.unwrap_spec = [ObjSpace, W_Root, str, int, rffi.r_uint]
def EnumValue(space, w_hkey, index):
@@ -532,10 +507,8 @@
hkey = hkey_w(w_hkey, space)
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
- retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize:
ret = rwinreg.RegQueryInfoKey(
hkey, None, null_dword, null_dword,
null_dword, null_dword, null_dword,
@@ -547,14 +520,9 @@
retValueSize[0] += 1
retDataSize[0] += 1
- valuebuf = lltype.malloc(rffi.CCHARP.TO, retValueSize[0],
- flavor='raw')
- try:
- databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0],
- flavor='raw')
- try:
- retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retValueSize[0]) as valuebuf:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType:
ret = rwinreg.RegEnumValue(
hkey, index, valuebuf, retValueSize,
null_dword, retType, databuf, retDataSize)
@@ -567,16 +535,6 @@
retDataSize[0], retType[0]),
space.wrap(retType[0]),
])
- finally:
- lltype.free(retType, flavor='raw')
- finally:
- lltype.free(databuf, flavor='raw')
- finally:
- lltype.free(valuebuf, flavor='raw')
- finally:
- lltype.free(retDataSize, flavor='raw')
- finally:
- lltype.free(retValueSize, flavor='raw')
EnumValue.unwrap_spec = [ObjSpace, W_Root, int]
@@ -593,10 +551,8 @@
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
# max key name length is 255
- buf = lltype.malloc(rffi.CCHARP.TO, 256, flavor='raw')
- try:
- retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, 256) as buf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize:
retValueSize[0] = r_uint(256) # includes NULL terminator
ret = rwinreg.RegEnumKeyEx(hkey, index, buf, retValueSize,
null_dword, None, null_dword,
@@ -604,10 +560,6 @@
if ret != 0:
raiseWindowsError(space, ret, 'RegEnumKeyEx')
return space.wrap(rffi.charp2str(buf))
- finally:
- lltype.free(retValueSize, flavor='raw')
- finally:
- lltype.free(buf, flavor='raw')
EnumKey.unwrap_spec = [ObjSpace, W_Root, int]
@@ -622,12 +574,9 @@
A long integer that identifies when the key was last modified (if available)
as 100's of nanoseconds since Jan 1, 1600."""
hkey = hkey_w(w_hkey, space)
- nSubKeys = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- nValues = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- ft = lltype.malloc(rwin32.PFILETIME.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nSubKeys:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nValues:
+ with lltype.scoped_alloc(rwin32.PFILETIME.TO, 1) as ft:
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
ret = rwinreg.RegQueryInfoKey(
hkey, None, null_dword, null_dword,
@@ -641,12 +590,6 @@
return space.newtuple([space.wrap(nSubKeys[0]),
space.wrap(nValues[0]),
space.wrap(l)])
- finally:
- lltype.free(ft, flavor='raw')
- finally:
- lltype.free(nValues, flavor='raw')
- finally:
- lltype.free(nSubKeys, flavor='raw')
QueryInfoKey.unwrap_spec = [ObjSpace, W_Root]
def str_or_None_w(space, w_obj):
@@ -667,12 +610,9 @@
If the function fails, an EnvironmentError exception is raised."""
machine = str_or_None_w(space, w_machine)
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegConnectRegistry(machine, hkey, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'RegConnectRegistry')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
ConnectRegistry.unwrap_spec = [ObjSpace, W_Root, W_Root]
Modified: pypy/branch/fast-forward/pypy/module/array/interp_array.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/array/interp_array.py (original)
+++ pypy/branch/fast-forward/pypy/module/array/interp_array.py Fri Oct 22 23:09:43 2010
@@ -193,32 +193,30 @@
mytype.bytes
raise OperationError(space.w_OverflowError,
space.wrap(msg))
- elif mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w':
+ return rffi.cast(mytype.itemtype, item)
+ if mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w':
if len(item) != 1:
msg = 'array item must be char'
raise OperationError(space.w_TypeError, space.wrap(msg))
item = item[0]
-
+ return rffi.cast(mytype.itemtype, item)
+ #
+ # "regular" case: it fits in an rpython integer (lltype.Signed)
+ result = rffi.cast(mytype.itemtype, item)
if mytype.canoverflow:
- msg = None
- if mytype.signed:
- if item < -1 << (mytype.bytes * 8 - 1):
+ if rffi.cast(lltype.Signed, result) != item:
+ # overflow. build the correct message
+ if item < 0:
msg = ('signed %d-byte integer is less than minimum' %
mytype.bytes)
- elif item > (1 << (mytype.bytes * 8 - 1)) - 1:
+ else:
msg = ('signed %d-byte integer is greater than maximum'
% mytype.bytes)
- else:
- if item < 0:
- msg = ('unsigned %d-byte integer is less than minimum'
- % mytype.bytes)
- elif item > (1 << (mytype.bytes * 8)) - 1:
- msg = ('unsigned %d-byte integer is greater'
- ' than maximum' % mytype.bytes)
- if msg is not None:
+ if not mytype.signed:
+ msg = 'un' + msg # 'signed' => 'unsigned'
raise OperationError(space.w_OverflowError,
space.wrap(msg))
- return rffi.cast(mytype.itemtype, item)
+ return result
def __del__(self):
self.setlen(0)
Modified: pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py (original)
+++ pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py Fri Oct 22 23:09:43 2010
@@ -225,6 +225,11 @@
if self.current_size > 0:
rffi.keep_buffer_alive_until_here(self.raw_buf, self.gc_buf)
+ def __enter__(self):
+ return self
+ def __exit__(self, *args):
+ self.free()
+
# ____________________________________________________________
#
# Make the BZ2File type by internally inheriting from W_File.
@@ -531,33 +536,30 @@
if not self.running:
raise OperationError(self.space.w_ValueError,
self.space.wrap("this object was already flushed"))
-
- out = OutBuffer(self.bzs)
+
in_bufsize = datasize
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(datasize):
- in_buf[i] = data[i]
- try:
-
- self.bzs.c_next_in = in_buf
- rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
+ with OutBuffer(self.bzs) as out:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
- while True:
- bzerror = BZ2_bzCompress(self.bzs, BZ_RUN)
- if bzerror != BZ_RUN_OK:
- _catch_bz2_error(self.space, bzerror)
+ for i in range(datasize):
+ in_buf[i] = data[i]
- if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
- break
- elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
+ self.bzs.c_next_in = in_buf
+ rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
- res = out.make_result_string()
- return self.space.wrap(res)
- finally:
- lltype.free(in_buf, flavor='raw')
- out.free()
+ while True:
+ bzerror = BZ2_bzCompress(self.bzs, BZ_RUN)
+ if bzerror != BZ_RUN_OK:
+ _catch_bz2_error(self.space, bzerror)
+
+ if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
+ break
+ elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
+ return self.space.wrap(res)
compress.unwrap_spec = ['self', 'bufferstr']
@@ -566,9 +568,8 @@
raise OperationError(self.space.w_ValueError,
self.space.wrap("this object was already flushed"))
self.running = False
-
- out = OutBuffer(self.bzs)
- try:
+
+ with OutBuffer(self.bzs) as out:
while True:
bzerror = BZ2_bzCompress(self.bzs, BZ_FINISH)
if bzerror == BZ_STREAM_END:
@@ -581,8 +582,6 @@
res = out.make_result_string()
return self.space.wrap(res)
- finally:
- out.free()
flush.unwrap_spec = ['self']
W_BZ2Compressor.typedef = TypeDef("BZ2Compressor",
@@ -641,38 +640,37 @@
if not self.running:
raise OperationError(self.space.w_EOFError,
self.space.wrap("end of stream was already found"))
-
+
in_bufsize = len(data)
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
- out = OutBuffer(self.bzs)
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
self.bzs.c_next_in = in_buf
rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
- while True:
- bzerror = BZ2_bzDecompress(self.bzs)
- if bzerror == BZ_STREAM_END:
- if rffi.getintfield(self.bzs, 'c_avail_in') != 0:
- unused = [self.bzs.c_next_in[i] for i in range(rffi.getintfield(self.bzs, 'c_avail_in'))]
- self.unused_data = "".join(unused)
- self.running = False
- break
- if bzerror != BZ_OK:
- _catch_bz2_error(self.space, bzerror)
-
- if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
- break
- elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
+ with OutBuffer(self.bzs) as out:
+ while True:
+ bzerror = BZ2_bzDecompress(self.bzs)
+ if bzerror == BZ_STREAM_END:
+ if rffi.getintfield(self.bzs, 'c_avail_in') != 0:
+ unused = [self.bzs.c_next_in[i]
+ for i in range(
+ rffi.getintfield(self.bzs,
+ 'c_avail_in'))]
+ self.unused_data = "".join(unused)
+ self.running = False
+ break
+ if bzerror != BZ_OK:
+ _catch_bz2_error(self.space, bzerror)
+
+ if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
+ break
+ elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
- res = out.make_result_string()
- return self.space.wrap(res)
- finally:
- lltype.free(in_buf, flavor='raw')
- out.free()
+ res = out.make_result_string()
+ return self.space.wrap(res)
decompress.unwrap_spec = ['self', 'bufferstr']
@@ -695,43 +693,39 @@
if compresslevel < 1 or compresslevel > 9:
raise OperationError(space.w_ValueError,
space.wrap("compresslevel must be between 1 and 9"))
-
- bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
- in_bufsize = len(data)
- # conforming to bz2 manual, this is large enough to fit compressed
- # data in one shot. We will check it later anyway.
- out = OutBuffer(bzs, in_bufsize + (in_bufsize / 100 + 1) + 600)
-
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
-
- try:
- bzs.c_next_in = in_buf
- rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
- bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0)
- if bzerror != BZ_OK:
- _catch_bz2_error(space, bzerror)
+ with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs:
+ in_bufsize = len(data)
+
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
+ bzs.c_next_in = in_buf
+ rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
+
+ # conforming to bz2 manual, this is large enough to fit compressed
+ # data in one shot. We will check it later anyway.
+ with OutBuffer(bzs,
+ in_bufsize + (in_bufsize / 100 + 1) + 600) as out:
+
+ bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0)
+ if bzerror != BZ_OK:
+ _catch_bz2_error(space, bzerror)
- while True:
- bzerror = BZ2_bzCompress(bzs, BZ_FINISH)
- if bzerror == BZ_STREAM_END:
- break
- elif bzerror != BZ_FINISH_OK:
+ while True:
+ bzerror = BZ2_bzCompress(bzs, BZ_FINISH)
+ if bzerror == BZ_STREAM_END:
+ break
+ elif bzerror != BZ_FINISH_OK:
+ BZ2_bzCompressEnd(bzs)
+ _catch_bz2_error(space, bzerror)
+
+ if rffi.getintfield(bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
BZ2_bzCompressEnd(bzs)
- _catch_bz2_error(space, bzerror)
-
- if rffi.getintfield(bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
-
- res = out.make_result_string()
- BZ2_bzCompressEnd(bzs)
- return space.wrap(res)
- finally:
- lltype.free(bzs, flavor='raw')
- lltype.free(in_buf, flavor='raw')
- out.free()
+ return space.wrap(res)
compress.unwrap_spec = [ObjSpace, 'bufferstr', int]
def decompress(space, data):
@@ -744,40 +738,34 @@
if in_bufsize == 0:
return space.wrap("")
- bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
-
- out = OutBuffer(bzs)
- try:
- bzs.c_next_in = in_buf
- rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
-
- bzerror = BZ2_bzDecompressInit(bzs, 0, 0)
- if bzerror != BZ_OK:
- _catch_bz2_error(space, bzerror)
-
- while True:
- bzerror = BZ2_bzDecompress(bzs)
- if bzerror == BZ_STREAM_END:
- break
- if bzerror != BZ_OK:
- BZ2_bzDecompressEnd(bzs)
- _catch_bz2_error(space, bzerror)
-
- if rffi.getintfield(bzs, 'c_avail_in') == 0:
+ with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
+ bzs.c_next_in = in_buf
+ rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
+
+ with OutBuffer(bzs) as out:
+ bzerror = BZ2_bzDecompressInit(bzs, 0, 0)
+ if bzerror != BZ_OK:
+ _catch_bz2_error(space, bzerror)
+
+ while True:
+ bzerror = BZ2_bzDecompress(bzs)
+ if bzerror == BZ_STREAM_END:
+ break
+ if bzerror != BZ_OK:
+ BZ2_bzDecompressEnd(bzs)
+ _catch_bz2_error(space, bzerror)
+
+ if rffi.getintfield(bzs, 'c_avail_in') == 0:
+ BZ2_bzDecompressEnd(bzs)
+ raise OperationError(space.w_ValueError, space.wrap(
+ "couldn't find end of stream"))
+ elif rffi.getintfield(bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
BZ2_bzDecompressEnd(bzs)
- raise OperationError(space.w_ValueError,
- space.wrap("couldn't find end of stream"))
- elif rffi.getintfield(bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
-
- res = out.make_result_string()
- BZ2_bzDecompressEnd(bzs)
- return space.wrap(res)
- finally:
- lltype.free(bzs, flavor='raw')
- lltype.free(in_buf, flavor='raw')
- out.free()
+ return space.wrap(res)
decompress.unwrap_spec = [ObjSpace, 'bufferstr']
Modified: pypy/branch/fast-forward/pypy/module/cpyext/classobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/cpyext/classobject.py (original)
+++ pypy/branch/fast-forward/pypy/module/cpyext/classobject.py Fri Oct 22 23:09:43 2010
@@ -15,16 +15,20 @@
class is the class of new object. The dict parameter will be used as the
object's __dict__; if NULL, a new dictionary will be created for the
instance."""
- if not PyClass_Check(space, w_class):
+ if not isinstance(w_class, W_ClassObject):
return PyErr_BadInternalCall(space)
- return W_InstanceObject(space, w_class, w_dict)
+ w_result = w_class.instantiate(space)
+ if w_dict is not None:
+ w_result.setdict(space, w_dict)
+ return w_result
@cpython_api([PyObject, PyObject], PyObject, error=CANNOT_FAIL)
def _PyInstance_Lookup(space, w_instance, w_name):
+ name = space.str_w(w_name)
assert isinstance(w_instance, W_InstanceObject)
- w_result = space.finditem(w_instance.w_dict, w_name)
+ w_result = w_instance.getdictvalue(space, name)
if w_result is not None:
return w_result
- return w_instance.w_class.lookup(space, w_name)
+ return w_instance.w_class.lookup(space, name)
Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py (original)
+++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py Fri Oct 22 23:09:43 2010
@@ -16,6 +16,7 @@
from pypy.module.cpyext.pyobject import Py_DecRef, InvalidPointerException
from pypy.translator.goal import autopath
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
@api.cpython_api([], api.PyObject)
def PyPy_Crash1(space):
@@ -78,7 +79,6 @@
self.frozen_refcounts[w_obj] = obj.c_ob_refcnt
#state.print_refcounts()
self.frozen_ll2callocations = set(ll2ctypes.ALLOCATED.values())
- lltype.start_tracking_allocations()
class LeakCheckingTest(object):
def check_and_print_leaks(self):
@@ -126,17 +126,8 @@
for w_obj in lost_objects_w:
print >>sys.stderr, "Lost object %r" % (w_obj, )
leaking = True
- for llvalue in set(ll2ctypes.ALLOCATED.values()) - self.frozen_ll2callocations:
- if getattr(llvalue, "_traceback", None): # this means that the allocation should be tracked
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (ll2ctypes)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
- for llvalue in lltype.ALLOCATED.keys():
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (llvalue)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
-
- lltype.stop_tracking_allocations()
+ # the actual low-level leak checking is done by pypy.tool.leakfinder,
+ # enabled automatically by pypy.conftest.
return leaking
class AppTestCpythonExtensionBase(LeakCheckingTest):
Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py (original)
+++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py Fri Oct 22 23:09:43 2010
@@ -177,13 +177,14 @@
encoded_charp = rffi.str2charp(encoded)
strict_charp = rffi.str2charp("strict")
if endian is not None:
- pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
if endian < 0:
- pendian[0] = -1
+ value = -1
elif endian > 0:
- pendian[0] = 1
+ value = 1
else:
- pendian[0] = 0
+ value = 0
+ pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
+ pendian[0] = rffi.cast(rffi.INT, value)
else:
pendian = None
Modified: pypy/branch/fast-forward/pypy/module/gc/referents.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/gc/referents.py (original)
+++ pypy/branch/fast-forward/pypy/module/gc/referents.py Fri Oct 22 23:09:43 2010
@@ -15,8 +15,16 @@
def try_cast_gcref_to_w_root(gcref):
w_obj = rgc.try_cast_gcref_to_instance(W_Root, gcref)
- if not we_are_translated() and not hasattr(w_obj, 'typedef'):
- w_obj = None
+ # Ignore the instances of W_Root that are not really valid as Python
+ # objects. There is e.g. WeakrefLifeline in module/_weakref that
+ # inherits from W_Root for internal reasons. Such instances don't
+ # have a typedef at all (or have a null typedef after translation).
+ if not we_are_translated():
+ if not hasattr(w_obj, 'typedef'):
+ return None
+ else:
+ if w_obj is None or not w_obj.typedef:
+ return None
return w_obj
def wrap(space, gcref):
Modified: pypy/branch/fast-forward/pypy/module/imp/test/test_import.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/test/test_import.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/test/test_import.py Fri Oct 22 23:09:43 2010
@@ -792,6 +792,7 @@
extrapath = udir.ensure("pythonpath", dir=1)
extrapath.join("urllib.py").write("print 42\n")
old = os.environ.get('PYTHONPATH', None)
+ oldlang = os.environ.pop('LANG', None)
try:
os.environ['PYTHONPATH'] = str(extrapath)
output = py.process.cmdexec('''"%s" "%s" -c "import urllib"''' %
@@ -800,6 +801,8 @@
finally:
if old:
os.environ['PYTHONPATH'] = old
+ if oldlang:
+ os.environ['LANG'] = oldlang
class AppTestImportHooks(object):
def test_meta_path(self):
Modified: pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py Fri Oct 22 23:09:43 2010
@@ -6,6 +6,7 @@
interpleveldefs = {
'set_param': 'interp_jit.set_param',
+ 'residual_call': 'interp_jit.residual_call',
}
def setup_after_space_initialization(self):
Modified: pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py (original)
+++ pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py Fri Oct 22 23:09:43 2010
@@ -5,10 +5,10 @@
from pypy.tool.pairtype import extendabletype
from pypy.rlib.rarithmetic import r_uint, intmask
-from pypy.rlib.jit import JitDriver, hint, we_are_jitted
+from pypy.rlib.jit import JitDriver, hint, we_are_jitted, dont_look_inside
import pypy.interpreter.pyopcode # for side-effects
from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import ObjSpace, Arguments
+from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root
from pypy.interpreter.pycode import PyCode, CO_GENERATOR
from pypy.interpreter.pyframe import PyFrame
from pypy.interpreter.pyopcode import ExitFrame
@@ -131,3 +131,10 @@
"no JIT parameter '%s'", key)
set_param.unwrap_spec = [ObjSpace, Arguments]
+
+ at dont_look_inside
+def residual_call(space, w_callable, args):
+ '''For testing. Invokes callable(...), but without letting
+ the JIT follow the call.'''
+ return space.call_args(w_callable, args)
+residual_call.unwrap_spec = [ObjSpace, W_Root, Arguments]
Modified: pypy/branch/fast-forward/pypy/module/pypyjit/policy.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pypyjit/policy.py (original)
+++ pypy/branch/fast-forward/pypy/module/pypyjit/policy.py Fri Oct 22 23:09:43 2010
@@ -11,7 +11,7 @@
if '.' in modname:
modname, _ = modname.split('.', 1)
if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions',
- 'imp', 'sys', 'array']:
+ 'imp', 'sys', 'array', '_ffi']:
return True
return False
Modified: pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py Fri Oct 22 23:09:43 2010
@@ -79,8 +79,11 @@
class PyPyCJITTests(object):
- def run_source(self, source, expected_max_ops, *testcases):
+ def run_source(self, source, expected_max_ops, *testcases, **kwds):
assert isinstance(expected_max_ops, int)
+ threshold = kwds.pop('threshold', 3)
+ if kwds:
+ raise TypeError, 'Unsupported keyword arguments: %s' % kwds.keys()
source = py.code.Source(source)
filepath = self.tmpdir.join('case%d.py' % self.counter)
logfilepath = filepath.new(ext='.log')
@@ -92,7 +95,7 @@
import sys
try: # make the file runnable by CPython
import pypyjit
- pypyjit.set_param(threshold=3)
+ pypyjit.set_param(threshold=%d)
except ImportError:
pass
@@ -102,7 +105,7 @@
print >> sys.stderr, 'got:', repr(result)
assert result == expected
assert type(result) is type(expected)
- """)
+ """ % threshold)
for testcase in testcases * 2:
print >> f, "check(%r, %r)" % testcase
print >> f, "print 'OK :-)'"
@@ -116,6 +119,8 @@
result = child_stdout.read()
child_stdout.close()
assert result
+ if result.strip().startswith('SKIP:'):
+ py.test.skip(result.strip())
assert result.splitlines()[-1].strip() == 'OK :-)'
self.parse_loops(logfilepath)
self.print_loops()
@@ -123,9 +128,10 @@
if self.total_ops > expected_max_ops:
assert 0, "too many operations: got %d, expected maximum %d" % (
self.total_ops, expected_max_ops)
+ return result
def parse_loops(self, opslogfile):
- from pypy.jit.metainterp.test.oparser import parse
+ from pypy.jit.tool.oparser import parse
from pypy.tool import logparser
assert opslogfile.check()
log = logparser.parse_log_file(str(opslogfile))
@@ -272,7 +278,7 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 7
+ assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict
assert not ops[1] # second LOOKUP_METHOD folded away
ops = self.get_by_bytecode("CALL_METHOD")
@@ -283,7 +289,7 @@
else:
assert not bytecode.get_opnames("call")
assert not bytecode.get_opnames("new")
- assert len(bytecode.get_opnames("guard")) <= 9
+ assert len(bytecode.get_opnames("guard")) <= 6
assert len(ops[1]) < len(ops[0])
ops = self.get_by_bytecode("LOAD_ATTR")
@@ -317,8 +323,8 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 7
- assert len(ops[0].get_opnames("getfield")) < 6
+ assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict
+ assert len(ops[0].get_opnames("getfield")) <= 5 # we get <5 withmapdict
assert not ops[1] # second LOOKUP_METHOD folded away
def test_default_and_kw(self):
@@ -382,7 +388,7 @@
a.x = 2
i = i + a.x
return i
- ''', 67,
+ ''', 69,
([20], 20),
([31], 32))
@@ -390,7 +396,7 @@
self.get_by_bytecode("CALL_FUNCTION"))
assert not callA.get_opnames("call")
assert not callA.get_opnames("new")
- assert len(callA.get_opnames("guard")) <= 8
+ assert len(callA.get_opnames("guard")) <= 2
assert not callisinstance1.get_opnames("call")
assert not callisinstance1.get_opnames("new")
assert len(callisinstance1.get_opnames("guard")) <= 2
@@ -554,17 +560,13 @@
def test_blockstack_virtualizable(self):
self.run_source('''
- def g(k):
- s = 0
- for i in range(k, k+2):
- s += 1
- return s
+ from pypyjit import residual_call
def main():
i = 0
while i < 100:
try:
- g(i)
+ residual_call(len, [])
except:
pass
i += 1
@@ -606,16 +608,17 @@
# call that can raise is not exchanged into getarrayitem_gc
def test_overflow_checking(self):
+ startvalue = sys.maxint - 2147483647
self.run_source('''
def main():
def f(a,b):
if a < 0: return -1
return a-b
- total = 0
+ total = %d
for i in range(100000):
total += f(i, 5)
return total
- ''', 170, ([], 4999450000L))
+ ''' % startvalue, 170, ([], startvalue + 4999450000L))
def test_boolrewrite_invers(self):
for a, b, res, ops in (('2000', '2000', 20001000, 51),
@@ -742,6 +745,8 @@
'''%(op1, float(a)/4.0, float(b)/4.0, op2), 109, ([], res))
def test_boolrewrite_ptr(self):
+ # XXX this test is way too imprecise in what it is actually testing
+ # it should count the number of guards instead
compares = ('a == b', 'b == a', 'a != b', 'b != a', 'a == c', 'c != b')
for e1 in compares:
for e2 in compares:
@@ -765,7 +770,7 @@
print
print 'Test:', e1, e2, n, res
self.run_source('''
- class tst:
+ class tst(object):
pass
def main():
a = tst()
@@ -781,24 +786,6 @@
return sa
'''%(e1, e2), n, ([], res))
- def test_boolrewrite_ptr_single(self):
- self.run_source('''
- class tst:
- pass
- def main():
- a = tst()
- b = tst()
- c = tst()
- sa = 0
- for i in range(1000):
- if a == b: sa += 1
- else: sa += 2
- if a != b: sa += 10000
- else: sa += 20000
- if i > 750: a = b
- return sa
- ''', 215, ([], 12481752))
-
def test_array_sum(self):
for tc, maxops in zip('bhilBHILfd', (38,) * 6 + (40, 40, 41, 38)):
res = 19352859
@@ -847,7 +834,12 @@
''', 65, ([], 122880))
def test_array_intimg(self):
- for tc, maxops in zip('ilILd', (67, 67, 69, 69, 61)):
+ # XXX this test is way too imprecise in what it is actually testing
+ # it should count the number of guards instead
+ for tc, maxops in zip('ilILd', (67, 67, 70, 70, 61)):
+ print
+ print '='*65
+ print '='*20, 'running test for tc=%r' % (tc,), '='*20
res = 73574560
if tc in 'IL':
res = long(res)
@@ -1130,6 +1122,44 @@
return sa
''', 88, ([], 1997001))
+ def test__ffi_call(self):
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ out = self.run_source('''
+ def main():
+ try:
+ from _ffi import CDLL, types
+ except ImportError:
+ sys.stdout.write('SKIP: cannot import _ffi')
+ return 0
+
+ libm = CDLL('%(libm_name)s')
+ pow = libm.getfunc('pow', [types.double, types.double],
+ types.double)
+ print pow.getaddr()
+ i = 0
+ res = 0
+ while i < 2000:
+ res += pow(2, 3)
+ i += 1
+ return res
+ ''' % locals(),
+ 76, ([], 8.0*2000), threshold=1000)
+ pow_addr = int(out.splitlines()[0])
+ ops = self.get_by_bytecode('CALL_FUNCTION')
+ assert len(ops) == 2 # we get two loops, because of specialization
+ call_function = ops[0]
+ last_ops = [op.getopname() for op in call_function[-5:]]
+ assert last_ops == ['force_token',
+ 'setfield_gc',
+ 'call_may_force',
+ 'guard_not_forced',
+ 'guard_no_exception']
+ call = call_function[-3]
+ assert call.getarg(0).value == pow_addr
+ assert call.getarg(1).value == 2.0
+ assert call.getarg(2).value == 3.0
+
# test_circular
class AppTestJIT(PyPyCJITTests):
@@ -1155,6 +1185,17 @@
cls.pypy_c = option.pypy_c
+def test_interface_residual_call():
+ space = gettestobjspace(usemodules=['pypyjit'])
+ space.appexec([], """():
+ import pypyjit
+ def f(*args, **kwds):
+ return (args, kwds)
+ res = pypyjit.residual_call(f, 4, x=6)
+ assert res == ((4,), {'x': 6})
+ """)
+
+
def has_info(pypy_c, option):
g = os.popen('"%s" --info' % pypy_c, 'r')
lines = g.readlines()
Modified: pypy/branch/fast-forward/pypy/module/signal/interp_signal.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/signal/interp_signal.py (original)
+++ pypy/branch/fast-forward/pypy/module/signal/interp_signal.py Fri Oct 22 23:09:43 2010
@@ -1,7 +1,6 @@
from pypy.interpreter.error import OperationError
from pypy.interpreter.baseobjspace import W_Root, ObjSpace
from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag
-from pypy.rlib.rarithmetic import LONG_BIT, intmask
import signal as cpy_signal
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.translator.tool.cbuild import ExternalCompilationInfo
@@ -64,8 +63,8 @@
class CheckSignalAction(AsyncAction):
"""An action that is automatically invoked when a signal is received."""
- # The C-level signal handler sets the highest bit of pypysig_occurred:
- bitmask = intmask(1 << (LONG_BIT-1))
+ # The C-level signal handler sets the bit 30 of pypysig_occurred:
+ bitmask = 1 << 30
def __init__(self, space):
AsyncAction.__init__(self, space)
Modified: pypy/branch/fast-forward/pypy/module/sys/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/sys/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/sys/__init__.py Fri Oct 22 23:09:43 2010
@@ -7,13 +7,15 @@
"""Sys Builtin Module. """
def __init__(self, space, w_name):
"""NOT_RPYTHON""" # because parent __init__ isn't
+ if space.config.translating:
+ del self.__class__.interpleveldefs['pypy_getudir']
super(Module, self).__init__(space, w_name)
self.checkinterval = 100
self.recursionlimit = 100
self.w_default_encoder = None
self.defaultencoding = "ascii"
self.filesystemencoding = None
-
+
interpleveldefs = {
'__name__' : '(space.wrap("sys"))',
'__doc__' : '(space.wrap("PyPy sys module"))',
@@ -39,7 +41,7 @@
'py3kwarning' : 'space.w_False',
'warnoptions' : 'state.get(space).w_warnoptions',
'builtin_module_names' : 'state.w_None',
- 'pypy_getudir' : 'state.pypy_getudir',
+ 'pypy_getudir' : 'state.pypy_getudir', # not translated
'pypy_initial_path' : 'state.pypy_initial_path',
'_getframe' : 'vm._getframe',
Modified: pypy/branch/fast-forward/pypy/module/sys/state.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/sys/state.py (original)
+++ pypy/branch/fast-forward/pypy/module/sys/state.py Fri Oct 22 23:09:43 2010
@@ -95,15 +95,8 @@
def getio(space):
return space.fromcache(IOState)
-def _pypy_getudir(space):
- """NOT_RPYTHON"""
+def pypy_getudir(space):
+ """NOT_RPYTHON
+ (should be removed from interpleveldefs before translation)"""
from pypy.tool.udir import udir
return space.wrap(str(udir))
-_pypy_getudir._annspecialcase_ = "override:ignore"
-
-# we need the indirection because this function will live in a dictionary with other
-# RPYTHON functions and share call sites with them. Better it not be a special-case
-# directly.
-def pypy_getudir(space):
- return _pypy_getudir(space)
-
Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
==============================================================================
--- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c (original)
+++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c Fri Oct 22 23:09:43 2010
@@ -533,3 +533,7 @@
return inp;
}
+int my_unused_function(void)
+{
+ return 42;
+}
Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py (original)
+++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py Fri Oct 22 23:09:43 2010
@@ -148,3 +148,7 @@
# but it segfaults for some reason.
if sys.platform == 'win32':
assert f() == 0x12345678
+
+ def test_restype(self):
+ foo = lib.my_unused_function
+ assert foo.restype is c_int # by default
Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py (original)
+++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py Fri Oct 22 23:09:43 2010
@@ -10,7 +10,7 @@
"""
def test_array_of_pointers(self):
# tests array item assignements & pointer.contents = ...
- A = POINTER(c_int) * 24
+ A = POINTER(c_long) * 24
a = A()
l = c_long(2)
p = pointer(l)
Modified: pypy/branch/fast-forward/pypy/module/thread/ll_thread.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/thread/ll_thread.py (original)
+++ pypy/branch/fast-forward/pypy/module/thread/ll_thread.py Fri Oct 22 23:09:43 2010
@@ -111,7 +111,7 @@
c_thread_releaselock(self._lock)
def __del__(self):
- lltype.free(self._lock, flavor='raw')
+ lltype.free(self._lock, flavor='raw', track_allocation=False)
# ____________________________________________________________
#
@@ -128,10 +128,13 @@
null_ll_lock = lltype.nullptr(TLOCKP.TO)
def allocate_ll_lock():
- ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw')
+ # track_allocation=False here; be careful to lltype.free() it. The
+ # reason it is set to False is that we get it from all app-level
+ # lock objects, as well as from the GIL, which exists at shutdown.
+ ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw', track_allocation=False)
res = c_thread_lock_init(ll_lock)
if res == -1:
- lltype.free(ll_lock, flavor='raw')
+ lltype.free(ll_lock, flavor='raw', track_allocation=False)
raise error("out of resources")
return ll_lock
Modified: pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py Fri Oct 22 23:09:43 2010
@@ -398,6 +398,20 @@
break
class FlowSpaceFrame(pyframe.CPythonFrame):
+
+ def SETUP_WITH(self, offsettoend, next_instr):
+ # A simpler version than the 'real' 2.7 one:
+ # directly call manager.__enter__(), don't use special lookup functions
+ # which don't make sense on the RPython type system.
+ from pypy.interpreter.pyopcode import WithBlock
+ w_manager = self.peekvalue()
+ w_exit = self.space.getattr(w_manager, self.space.wrap("__exit__"))
+ self.settopvalue(w_exit)
+ w_result = self.space.call_method(w_manager, "__enter__")
+ block = WithBlock(self, next_instr + offsettoend)
+ self.append_block(block)
+ self.pushvalue(w_result)
+
# XXX Unimplemented 2.7 opcodes ----------------
# Set literals, set comprehensions
@@ -413,9 +427,6 @@
def MAP_ADD(self, oparg, next_instr):
raise NotImplementedError("MAP_ADD")
- # `with` statement
-
-
def make_arguments(self, nargs):
return ArgumentsForTranslation(self.space, self.peekvalues(nargs))
def argument_factory(self, *args):
@@ -427,3 +438,12 @@
raise operr
return pyframe.PyFrame.handle_operation_error(self, ec, operr,
*args, **kwds)
+
+ def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
+ if w_typ is not self.space.w_None:
+ # The annotator won't allow to merge exception types with None.
+ # Replace it with an object which will break translation when used
+ # (except maybe with 'exc_typ is None')
+ w_typ = self.space.wrap(self.space)
+ return self.space.call_function(w_func, w_typ, w_val, w_tb)
+
Modified: pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py Fri Oct 22 23:09:43 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import new
import py
from pypy.objspace.flow.model import Constant, Block, Link, Variable, traverse
@@ -828,6 +829,25 @@
simplify_graph(graph)
assert self.all_operations(graph) == {'getitem': 1}
+ def test_context_manager(self):
+ def f(c, x):
+ with x:
+ pass
+ graph = self.codetest(f)
+ # 2 method calls: x.__enter__() and x.__exit__(None, None, None)
+ assert self.all_operations(graph) == {'getattr': 2,
+ 'simple_call': 2}
+ #
+ def g(): pass
+ def f(c, x):
+ with x:
+ g()
+ graph = self.codetest(f)
+ assert self.all_operations(graph) == {
+ 'getattr': 2, # __enter__ and __exit__
+ 'simple_call': 4, # __enter__, g and 2 possible calls to __exit__
+ 'is_true': 1} # check the result of __exit__()
+
def monkey_patch_code(self, code, stacksize, flags, codestring, names, varnames):
c = code
return new.code(c.co_argcount, c.co_nlocals, stacksize, flags,
Modified: pypy/branch/fast-forward/pypy/objspace/std/celldict.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/celldict.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/celldict.py Fri Oct 22 23:09:43 2010
@@ -45,7 +45,7 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
def impl_setitem_str(self, name, w_value, shadows_type=True):
self.getcell(name, True).w_value = w_value
@@ -66,7 +66,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
# inefficient, but do we care?
@@ -85,7 +85,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_lookup)
+ return self._as_rdict().impl_fallback_getitem(w_lookup)
def impl_getitem_str(self, lookup):
res = self.getcell(lookup, False)
Modified: pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py Fri Oct 22 23:09:43 2010
@@ -102,17 +102,17 @@
else:
return None
- # _________________________________________________________________
+ # _________________________________________________________________
# implementation methods
def impl_getitem(self, w_key):
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_getitem_str(self, w_key):
+ def impl_getitem_str(self, key):
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value, shadows_type=True):
raise NotImplementedError("abstract base class")
def impl_setitem(self, w_key, w_value):
@@ -120,7 +120,7 @@
def impl_delitem(self, w_key):
raise NotImplementedError("abstract base class")
-
+
def impl_length(self):
raise NotImplementedError("abstract base class")
@@ -310,7 +310,7 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
def impl_setitem_str(self, key, w_value, shadows_type=True):
self.content[key] = w_value
@@ -324,7 +324,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
return len(self.content)
@@ -344,7 +344,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_key)
+ return self._as_rdict().impl_fallback_getitem(w_key)
def impl_iter(self):
return StrIteratorImplementation(self.space, self)
@@ -414,7 +414,7 @@
StrDictImplementation.impl_setitem_str(
self, self.space.str_w(w_key), w_value, False)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
def impl_shadows_anything(self):
return (self._shadows_anything or
@@ -446,7 +446,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_get_builtin_indexed(self, i):
return self.shadowed[i]
Modified: pypy/branch/fast-forward/pypy/objspace/std/fake.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/fake.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/fake.py Fri Oct 22 23:09:43 2010
@@ -21,7 +21,6 @@
#debug_print("faking obj %s" % x)
ft = fake_type(type(x))
return ft(space, x)
-fake_object._annspecialcase_ = "override:fake_object"
import sys
@@ -47,7 +46,6 @@
w_exc = space.wrap(exc)
w_value = space.wrap(value)
raise OperationError, OperationError(w_exc, w_value), tb
-wrap_exception._annspecialcase_ = "override:ignore"
def fake_type(cpy_type):
assert type(cpy_type) is type
Modified: pypy/branch/fast-forward/pypy/objspace/std/model.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/model.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/model.py Fri Oct 22 23:09:43 2010
@@ -18,6 +18,7 @@
"withsmallint" : ["smallintobject.W_SmallIntObject"],
"withstrslice" : ["strsliceobject.W_StringSliceObject"],
"withstrjoin" : ["strjoinobject.W_StringJoinObject"],
+ "withstrbuf" : ["strbufobject.W_StringBufferObject"],
"withrope" : ["ropeobject.W_RopeObject",
"ropeobject.W_RopeIterObject"],
"withropeunicode": ["ropeunicodeobject.W_RopeUnicodeObject",
@@ -77,6 +78,7 @@
from pypy.objspace.std import ropeunicodeobject
from pypy.objspace.std import strsliceobject
from pypy.objspace.std import strjoinobject
+ from pypy.objspace.std import strbufobject
from pypy.objspace.std import typeobject
from pypy.objspace.std import sliceobject
from pypy.objspace.std import longobject
@@ -228,6 +230,13 @@
(unicodeobject.W_UnicodeObject,
strjoinobject.delegate_join2unicode)
]
+ elif config.objspace.std.withstrbuf:
+ self.typeorder[strbufobject.W_StringBufferObject] += [
+ (stringobject.W_StringObject,
+ strbufobject.delegate_buf2str),
+ (unicodeobject.W_UnicodeObject,
+ strbufobject.delegate_buf2unicode)
+ ]
if config.objspace.std.withrangelist:
self.typeorder[rangeobject.W_RangeListObject] += [
(listobject.W_ListObject,
Modified: pypy/branch/fast-forward/pypy/objspace/std/objspace.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/objspace.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/objspace.py Fri Oct 22 23:09:43 2010
@@ -23,6 +23,7 @@
from pypy.objspace.std.listobject import W_ListObject
from pypy.objspace.std.longobject import W_LongObject
from pypy.objspace.std.noneobject import W_NoneObject
+from pypy.objspace.std.objectobject import W_ObjectObject
from pypy.objspace.std.ropeobject import W_RopeObject
from pypy.objspace.std.iterobject import W_SeqIterObject
from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
@@ -242,7 +243,6 @@
w_result = getattr(self, 'w_' + x.__name__)
return w_result
return None
- wrap_exception_cls._annspecialcase_ = "override:wrap_exception_cls"
def unwrap(self, w_obj):
if isinstance(w_obj, Wrappable):
@@ -318,9 +318,14 @@
w_subtype = w_type.check_user_subclass(w_subtype)
if cls.typedef.applevel_subclasses_base is not None:
cls = cls.typedef.applevel_subclasses_base
- subcls = get_unique_interplevel_subclass(
- self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0,
- w_subtype.needsdel, w_subtype.weakrefable)
+ if (self.config.objspace.std.withmapdict and cls is W_ObjectObject
+ and not w_subtype.needsdel):
+ from pypy.objspace.std.mapdict import get_subclass_of_correct_size
+ subcls = get_subclass_of_correct_size(self, cls, w_subtype)
+ else:
+ subcls = get_unique_interplevel_subclass(
+ self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0,
+ w_subtype.needsdel, w_subtype.weakrefable)
instance = instantiate(subcls)
assert isinstance(instance, cls)
instance.user_setup(self, w_subtype)
Modified: pypy/branch/fast-forward/pypy/objspace/std/sharingdict.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/sharingdict.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/sharingdict.py Fri Oct 22 23:09:43 2010
@@ -71,7 +71,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_lookup)
+ return self._as_rdict().impl_fallback_getitem(w_lookup)
def impl_getitem_str(self, lookup):
i = self.structure.lookup_position(lookup)
@@ -84,7 +84,7 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
@unroll_safe
def impl_setitem_str(self, key, w_value, shadows_type=True):
@@ -132,7 +132,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
return self.structure.length
Modified: pypy/branch/fast-forward/pypy/objspace/std/stringobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/stringobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/stringobject.py Fri Oct 22 23:09:43 2010
@@ -14,7 +14,7 @@
from pypy.rlib.rstring import StringBuilder, string_repeat
from pypy.interpreter.buffer import StringBuffer
-from pypy.objspace.std.stringtype import sliced, joined, wrapstr, wrapchar, \
+from pypy.objspace.std.stringtype import sliced, wrapstr, wrapchar, \
stringendswith, stringstartswith, joined2
from pypy.objspace.std.formatting import mod_format
Modified: pypy/branch/fast-forward/pypy/objspace/std/stringtype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/stringtype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/stringtype.py Fri Oct 22 23:09:43 2010
@@ -55,19 +55,14 @@
return W_StringSliceObject(s, start, stop)
return wrapstr(space, s[start:stop])
-def joined(space, strlist):
- assert not space.config.objspace.std.withrope
- if space.config.objspace.std.withstrjoin:
- from pypy.objspace.std.strjoinobject import W_StringJoinObject
- return W_StringJoinObject(strlist)
- else:
- return wrapstr(space, "".join(strlist))
-
def joined2(space, str1, str2):
assert not space.config.objspace.std.withrope
if space.config.objspace.std.withstrjoin:
from pypy.objspace.std.strjoinobject import W_StringJoinObject
return W_StringJoinObject([str1, str2])
+ elif space.config.objspace.std.withstrbuf:
+ from pypy.objspace.std.strbufobject import joined2
+ return joined2(str1, str2)
else:
return wrapstr(space, str1 + str2)
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py Fri Oct 22 23:09:43 2010
@@ -602,6 +602,15 @@
classofinstance=classofinstance,
from_strdict_shared=from_strdict_shared)
+ def finditem_str(self, w_dict, s):
+ return w_dict.getitem_str(s) # assume it's a multidict
+
+ def setitem_str(self, w_dict, s, w_value):
+ return w_dict.setitem_str(s, w_value) # assume it's a multidict
+
+ def delitem(self, w_dict, w_s):
+ return w_dict.delitem(w_s) # assume it's a multidict
+
def allocate_instance(self, cls, type):
return object.__new__(cls)
@@ -611,7 +620,7 @@
w_StopIteration = StopIteration
w_None = None
StringObjectCls = FakeString
- w_dict = None
+ w_dict = W_DictMultiObject
iter = iter
fixedview = list
listview = list
@@ -687,6 +696,14 @@
assert self.impl.length() == 0
self.check_not_devolved()
+ def test_clear(self):
+ self.fill_impl()
+ assert self.impl.length() == 2
+ self.impl.clear()
+ assert self.impl.length() == 0
+ self.check_not_devolved()
+
+
def test_keys(self):
self.fill_impl()
keys = self.impl.keys()
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py Fri Oct 22 23:09:43 2010
@@ -3,7 +3,8 @@
class TestShadowTracking(object):
def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withshadowtracking": True})
+ cls.space = gettestobjspace(**{"objspace.std.withshadowtracking": True,
+ "objspace.std.withmapdict": False})
def test_simple_shadowing(self):
space = self.space
Modified: pypy/branch/fast-forward/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/typeobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/typeobject.py Fri Oct 22 23:09:43 2010
@@ -76,7 +76,9 @@
'weakrefable',
'hasdict',
'nslots',
- 'instancetypedef']
+ 'instancetypedef',
+ 'terminator',
+ ]
# for config.objspace.std.getattributeshortcut
# (False is a conservative default, fixed during real usage)
@@ -118,6 +120,12 @@
# dict_w of any of the types in the mro changes, or if the mro
# itself changes
w_self._version_tag = VersionTag()
+ if space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import DictTerminator, NoDictTerminator
+ if w_self.hasdict:
+ w_self.terminator = DictTerminator(space, w_self)
+ else:
+ w_self.terminator = NoDictTerminator(space, w_self)
def mutated(w_self):
space = w_self.space
Modified: pypy/branch/fast-forward/pypy/rlib/_rsocket_rffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/_rsocket_rffi.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/_rsocket_rffi.py Fri Oct 22 23:09:43 2010
@@ -332,10 +332,11 @@
('sll_hatype', rffi.INT),
('sll_addr', rffi.CFixedArray(rffi.CHAR, 8)),
('sll_halen', rffi.INT)],
- )
+ ifdef='AF_PACKET')
CConfig.ifreq = platform.Struct('struct ifreq', [('ifr_ifindex', rffi.INT),
- ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))])
+ ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))],
+ ifdef='AF_PACKET')
if _WIN32:
CConfig.WSAEVENT = platform.SimpleType('WSAEVENT', rffi.VOIDP)
@@ -546,8 +547,9 @@
socketpair_t = rffi.CArray(socketfd_type)
socketpair = external('socketpair', [rffi.INT, rffi.INT, rffi.INT,
lltype.Ptr(socketpair_t)], rffi.INT)
- ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)],
- rffi.INT)
+ if ifreq is not None:
+ ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)],
+ rffi.INT)
if _WIN32:
ioctlsocket = external('ioctlsocket',
Modified: pypy/branch/fast-forward/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/jit.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/jit.py Fri Oct 22 23:09:43 2010
@@ -77,6 +77,12 @@
return result
return decorator
+def oopspec(spec):
+ def decorator(func):
+ func.oopspec = spec
+ return func
+ return decorator
+
class Entry(ExtRegistryEntry):
_about_ = hint
@@ -139,6 +145,24 @@
return hop.inputconst(lltype.Signed, _we_are_jitted)
+def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1,
+ arg3=-sys.maxint-1, arg4=-sys.maxint-1):
+ """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in
+ the graphs. Should not be left after debugging."""
+ keepalive_until_here(string) # otherwise the whole function call is removed
+jit_debug.oopspec = 'jit.debug(string, arg1, arg2, arg3, arg4)'
+
+def assert_green(value):
+ """Very strong assert: checks that 'value' is a green
+ (a JIT compile-time constant)."""
+ keepalive_until_here(value)
+assert_green._annspecialcase_ = 'specialize:argtype(0)'
+assert_green.oopspec = 'jit.assert_green(value)'
+
+class AssertGreenFailed(Exception):
+ pass
+
+
##def force_virtualizable(virtualizable):
## pass
@@ -250,8 +274,9 @@
several independent JITting interpreters in it.
"""
+ active = True # if set to False, this JitDriver is ignored
virtualizables = []
-
+
def __init__(self, greens=None, reds=None, virtualizables=None,
get_jitcell_at=None, set_jitcell_at=None,
get_printable_location=None, confirm_enter_jit=None,
@@ -266,7 +291,8 @@
self.virtualizables = virtualizables
for v in self.virtualizables:
assert v in self.reds
- self._alllivevars = dict.fromkeys(self.greens + self.reds)
+ self._alllivevars = dict.fromkeys(
+ [name for name in self.greens + self.reds if '.' not in name])
self._make_extregistryentries()
self.get_jitcell_at = get_jitcell_at
self.set_jitcell_at = set_jitcell_at
@@ -355,10 +381,16 @@
def compute_result_annotation(self, **kwds_s):
from pypy.annotation import model as annmodel
+
+ if self.instance.__name__ == 'jit_merge_point':
+ if not self.annotate_hooks(**kwds_s):
+ return None # wrong order, try again later
+
driver = self.instance.im_self
keys = kwds_s.keys()
keys.sort()
- expected = ['s_' + name for name in driver.greens + driver.reds]
+ expected = ['s_' + name for name in driver.greens + driver.reds
+ if '.' not in name]
expected.sort()
if keys != expected:
raise JitHintError("%s expects the following keyword "
@@ -382,30 +414,35 @@
key[2:])
cache[key] = s_value
- if self.instance.__name__ == 'jit_merge_point':
- self.annotate_hooks(**kwds_s)
-
return annmodel.s_None
def annotate_hooks(self, **kwds_s):
driver = self.instance.im_self
s_jitcell = self.bookkeeper.valueoftype(BaseJitCell)
- self.annotate_hook(driver.get_jitcell_at, driver.greens, **kwds_s)
- self.annotate_hook(driver.set_jitcell_at, driver.greens, [s_jitcell],
- **kwds_s)
- self.annotate_hook(driver.get_printable_location, driver.greens, **kwds_s)
+ h = self.annotate_hook
+ return (h(driver.get_jitcell_at, driver.greens, **kwds_s)
+ and h(driver.set_jitcell_at, driver.greens, [s_jitcell], **kwds_s)
+ and h(driver.get_printable_location, driver.greens, **kwds_s))
def annotate_hook(self, func, variables, args_s=[], **kwds_s):
if func is None:
- return
+ return True
bk = self.bookkeeper
s_func = bk.immutablevalue(func)
uniquekey = 'jitdriver.%s' % func.func_name
args_s = args_s[:]
for name in variables:
- s_arg = kwds_s['s_' + name]
+ if '.' not in name:
+ s_arg = kwds_s['s_' + name]
+ else:
+ objname, fieldname = name.split('.')
+ s_instance = kwds_s['s_' + objname]
+ s_arg = s_instance.classdef.about_attribute(fieldname)
+ if s_arg is None:
+ return False # wrong order, try again later
args_s.append(s_arg)
bk.emulate_pbc_call(uniquekey, s_func, args_s)
+ return True
def specialize_call(self, hop, **kwds_i):
# XXX to be complete, this could also check that the concretetype
@@ -416,9 +453,42 @@
greens_v = []
reds_v = []
for name in driver.greens:
- i = kwds_i['i_' + name]
- r_green = hop.args_r[i]
- v_green = hop.inputarg(r_green, arg=i)
+ if '.' not in name:
+ i = kwds_i['i_' + name]
+ r_green = hop.args_r[i]
+ v_green = hop.inputarg(r_green, arg=i)
+ else:
+ if hop.rtyper.type_system.name == 'ootypesystem':
+ py.test.skip("lltype only")
+ objname, fieldname = name.split('.') # see test_green_field
+ assert objname in driver.reds
+ i = kwds_i['i_' + objname]
+ s_red = hop.args_s[i]
+ r_red = hop.args_r[i]
+ while True:
+ try:
+ mangled_name, r_field = r_red._get_field(fieldname)
+ break
+ except KeyError:
+ pass
+ assert r_red.rbase is not None, (
+ "field %r not found in %r" % (name,
+ r_red.lowleveltype.TO))
+ r_red = r_red.rbase
+ GTYPE = r_red.lowleveltype.TO
+ assert GTYPE._immutable_field(mangled_name), (
+ "field %r must be declared as immutable" % name)
+ if not hasattr(driver, 'll_greenfields'):
+ driver.ll_greenfields = {}
+ driver.ll_greenfields[name] = GTYPE, mangled_name
+ #
+ v_red = hop.inputarg(r_red, arg=i)
+ c_llname = hop.inputconst(lltype.Void, mangled_name)
+ v_green = hop.genop('getfield', [v_red, c_llname],
+ resulttype = r_field)
+ s_green = s_red.classdef.about_attribute(fieldname)
+ assert s_green is not None
+ hop.rtyper.annotator.setbinding(v_green, s_green)
greens_v.append(v_green)
for name in driver.reds:
i = kwds_i['i_' + name]
Modified: pypy/branch/fast-forward/pypy/rlib/rgc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rgc.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rgc.py Fri Oct 22 23:09:43 2010
@@ -170,7 +170,14 @@
return hop.genop('gc_set_max_heap_size', [v_nbytes],
resulttype=lltype.Void)
-def can_move(p): # NB. must not be called with NULL pointers
+def can_move(p):
+ """Check if the GC object 'p' is at an address that can move.
+ Must not be called with None. With non-moving GCs, it is always False.
+ With some moving GCs like the SemiSpace GC, it is always True.
+ With other moving GCs like the MiniMark GC, it can be True for some
+ time, then False for the same object, when we are sure that it won't
+ move any more.
+ """
return True
class CanMoveEntry(ExtRegistryEntry):
Modified: pypy/branch/fast-forward/pypy/rlib/rmmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rmmap.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rmmap.py Fri Oct 22 23:09:43 2010
@@ -50,7 +50,7 @@
constant_names = ['MAP_SHARED', 'MAP_PRIVATE',
'PROT_READ', 'PROT_WRITE',
'MS_SYNC']
- opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS',
+ opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS', 'MAP_NORESERVE',
'PROT_EXEC',
'MAP_DENYWRITE', 'MAP_EXECUTABLE']
for name in constant_names:
Modified: pypy/branch/fast-forward/pypy/rlib/rsre/rsre_char.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rsre/rsre_char.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rsre/rsre_char.py Fri Oct 22 23:09:43 2010
@@ -4,6 +4,7 @@
import sys
from pypy.rlib.rlocale import tolower, isalnum
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib import jit
# Note: the unicode parts of this module require you to call
# rsre_char.set_unicode_db() first, to select one of the modules
@@ -43,6 +44,7 @@
# XXX can we import those safely from sre_constants?
SRE_INFO_PREFIX = 1
SRE_INFO_LITERAL = 2
+SRE_INFO_CHARSET = 4
SRE_FLAG_LOCALE = 4 # honour system locale
SRE_FLAG_UNICODE = 32 # use unicode locale
OPCODE_INFO = 17
@@ -64,33 +66,27 @@
#### Category helpers
-ascii_char_info = [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 6, 2,
-2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 25, 25, 25, 25, 25, 25, 25,
-25, 25, 0, 0, 0, 0, 0, 0, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0,
-0, 0, 16, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0, 0, 0, 0 ]
-
+is_a_word = [(chr(i).isalnum() or chr(i) == '_') for i in range(256)]
linebreak = ord("\n")
underline = ord("_")
def is_digit(code):
- return code < 128 and (ascii_char_info[code] & 1 != 0)
+ return code <= 57 and code >= 48
def is_uni_digit(code):
assert unicodedb is not None
return unicodedb.isdecimal(code)
def is_space(code):
- return code < 128 and (ascii_char_info[code] & 2 != 0)
+ return code == 32 or (code <= 13 and code >= 9)
def is_uni_space(code):
assert unicodedb is not None
return unicodedb.isspace(code)
def is_word(code):
- return code < 128 and (ascii_char_info[code] & 16 != 0)
+ assert code >= 0
+ return code < 256 and is_a_word[code]
def is_uni_word(code):
assert unicodedb is not None
@@ -142,6 +138,7 @@
SET_OK = -1
SET_NOT_OK = -2
+ at jit.unroll_safe
def check_charset(pattern, ppos, char_code):
"""Checks whether a character matches set of arbitrary length.
The set starts at pattern[ppos]."""
Modified: pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py Fri Oct 22 23:09:43 2010
@@ -1,9 +1,11 @@
import sys
-from pypy.rlib.debug import check_nonneg
+from pypy.rlib.debug import check_nonneg, make_sure_not_modified
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib.rsre import rsre_char
from pypy.tool.sourcetools import func_with_new_name
from pypy.rlib.objectmodel import we_are_translated
+from pypy.rlib import jit
+from pypy.rlib.rsre.rsre_jit import install_jitdriver, install_jitdriver_spec
OPCODE_FAILURE = 0
@@ -56,16 +58,19 @@
_seen_specname[specname] = True
# Install a copy of the function under the name '_spec_funcname' in each
# concrete subclass
+ specialized_methods = []
for prefix, concreteclass in [('str', StrMatchContext),
('uni', UnicodeMatchContext)]:
newfunc = func_with_new_name(func, prefix + specname)
assert not hasattr(concreteclass, specname)
setattr(concreteclass, specname, newfunc)
+ specialized_methods.append(newfunc)
# Return a dispatcher function, specialized on the exact type of 'ctx'
def dispatch(ctx, *args):
return getattr(ctx, specname)(*args)
dispatch._annspecialcase_ = 'specialize:argtype(0)'
- return dispatch
+ dispatch._specialized_methods_ = specialized_methods
+ return func_with_new_name(dispatch, specname)
# ____________________________________________________________
@@ -75,6 +80,7 @@
class AbstractMatchContext(object):
"""Abstract base class"""
+ _immutable_fields_ = ['pattern[*]', 'flags', 'end']
match_start = 0
match_end = 0
match_marks = None
@@ -164,6 +170,8 @@
def __init__(self, pattern, string, match_start, end, flags):
AbstractMatchContext.__init__(self, pattern, match_start, end, flags)
self._string = string
+ if not we_are_translated() and isinstance(string, unicode):
+ self.flags |= rsre_char.SRE_FLAG_UNICODE # for rsre_re.py
def str(self, index):
check_nonneg(index)
@@ -238,8 +246,9 @@
self.start_ptr = ptr
self.start_marks = marks
+ @jit.unroll_safe
def find_first_result(self, ctx):
- ppos = self.ppos
+ ppos = jit.hint(self.ppos, promote=True)
while ctx.pat(ppos):
result = sre_match(ctx, ppos + 1, self.start_ptr, self.start_marks)
ppos += ctx.pat(ppos)
@@ -250,6 +259,10 @@
find_next_result = find_first_result
class RepeatOneMatchResult(MatchResult):
+ install_jitdriver('RepeatOne',
+ greens=['nextppos', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(1, 0)) # indices in 'greens'
def __init__(self, nextppos, minptr, ptr, marks):
self.nextppos = nextppos
@@ -259,8 +272,11 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
while ptr >= self.minptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ ctx.jitdriver_RepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos)
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
ptr -= 1
if result is not None:
self.subresult = result
@@ -270,6 +286,10 @@
class MinRepeatOneMatchResult(MatchResult):
+ install_jitdriver('MinRepeatOne',
+ greens=['nextppos', 'ppos3', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(2, 0)) # indices in 'greens'
def __init__(self, nextppos, ppos3, maxptr, ptr, marks):
self.nextppos = nextppos
@@ -280,29 +300,32 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
+ ppos3 = self.ppos3
while ptr <= self.maxptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ ctx.jitdriver_MinRepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos, ppos3=ppos3)
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
if result is not None:
self.subresult = result
self.start_ptr = ptr
return self
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, ppos3):
break
ptr += 1
def find_next_result(self, ctx):
ptr = self.start_ptr
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, self.ppos3):
return
self.start_ptr = ptr + 1
return self.find_first_result(ctx)
- def next_char_ok(self, ctx, ptr):
+ def next_char_ok(self, ctx, ptr, ppos):
if ptr == ctx.end:
return False
- ppos = self.ppos3
op = ctx.pat(ppos)
- for op1, (checkerfn, _) in unroll_char_checker:
+ for op1, checkerfn in unroll_char_checker:
if op1 == op:
return checkerfn(ctx, ptr, ppos)
raise Error("next_char_ok[%d]" % op)
@@ -325,41 +348,34 @@
self.next = next # chained list
class MaxUntilMatchResult(AbstractUntilMatchResult):
+ install_jitdriver('MaxUntil',
+ greens=['ppos', 'tailppos', 'match_more', 'ctx.pattern'],
+ reds=['ptr', 'marks', 'self', 'ctx'],
+ debugprint=(3, 0, 2))
def find_first_result(self, ctx):
- enum = sre_match(ctx, self.ppos + 3, self.cur_ptr, self.cur_marks)
- return self.search_next(ctx, enum, resume=False)
+ return self.search_next(ctx, match_more=True)
def find_next_result(self, ctx):
- return self.search_next(ctx, None, resume=True)
+ return self.search_next(ctx, match_more=False)
- def search_next(self, ctx, enum, resume):
+ def search_next(self, ctx, match_more):
ppos = self.ppos
- min = ctx.pat(ppos+1)
- max = ctx.pat(ppos+2)
+ tailppos = self.tailppos
ptr = self.cur_ptr
marks = self.cur_marks
while True:
- while True:
- if (enum is not None and
- (ptr != ctx.match_end or self.num_pending < min)):
- # ^^^^^^^^^^ zero-width match protection
- # matched one more 'item'. record it and continue.
- self.pending = Pending(ptr, marks, enum, self.pending)
- self.num_pending += 1
- ptr = ctx.match_end
- marks = ctx.match_marks
- break
- # 'item' no longer matches.
- if not resume and self.num_pending >= min:
- # try to match 'tail' if we have enough 'item'
- result = sre_match(ctx, self.tailppos, ptr, marks)
- if result is not None:
- self.subresult = result
- self.cur_ptr = ptr
- self.cur_marks = marks
- return self
- resume = False
+ ctx.jitdriver_MaxUntil.jit_merge_point(
+ ppos=ppos, tailppos=tailppos, match_more=match_more,
+ ptr=ptr, marks=marks, self=self, ctx=ctx)
+ if match_more:
+ max = ctx.pat(ppos+2)
+ if max == 65535 or self.num_pending < max:
+ # try to match one more 'item'
+ enum = sre_match(ctx, ppos + 3, ptr, marks)
+ else:
+ enum = None # 'max' reached, no more matches
+ else:
p = self.pending
if p is None:
return
@@ -369,11 +385,27 @@
marks = p.marks
enum = p.enum.move_to_next_result(ctx)
#
- if max == 65535 or self.num_pending < max:
- # try to match one more 'item'
- enum = sre_match(ctx, ppos + 3, ptr, marks)
+ min = ctx.pat(ppos+1)
+ if (enum is not None and
+ (ptr != ctx.match_end or self.num_pending < min)):
+ # ^^^^^^^^^^ zero-width match protection
+ # matched one more 'item'. record it and continue.
+ self.pending = Pending(ptr, marks, enum, self.pending)
+ self.num_pending += 1
+ ptr = ctx.match_end
+ marks = ctx.match_marks
+ match_more = True
else:
- enum = None # 'max' reached, no more matches
+ # 'item' no longer matches.
+ if self.num_pending >= min:
+ # try to match 'tail' if we have enough 'item'
+ result = sre_match(ctx, tailppos, ptr, marks)
+ if result is not None:
+ self.subresult = result
+ self.cur_ptr = ptr
+ self.cur_marks = marks
+ return self
+ match_more = False
class MinUntilMatchResult(AbstractUntilMatchResult):
@@ -384,6 +416,7 @@
return self.search_next(ctx, resume=True)
def search_next(self, ctx, resume):
+ # XXX missing jit support here
ppos = self.ppos
min = ctx.pat(ppos+1)
max = ctx.pat(ppos+2)
@@ -429,6 +462,7 @@
# ____________________________________________________________
@specializectx
+ at jit.unroll_safe
def sre_match(ctx, ppos, ptr, marks):
"""Returns either None or a MatchResult object. Usually we only need
the first result, but there is the case of REPEAT...UNTIL where we
@@ -437,6 +471,13 @@
while True:
op = ctx.pat(ppos)
ppos += 1
+ make_sure_not_modified(ctx.pattern)
+
+ #jit.jit_debug("sre_match", op, ppos, ptr)
+ #
+ # When using the JIT, calls to sre_match() must always have a constant
+ # (green) argument for 'ppos'. If not, the following assert fails.
+ jit.assert_green(op)
if op == OPCODE_FAILURE:
return
@@ -712,13 +753,23 @@
@specializectx
def find_repetition_end(ctx, ppos, ptr, maxcount):
end = ctx.end
- # adjust end
- if maxcount != 65535:
+ if maxcount <= 1:
+ if maxcount == 1 and ptr < end:
+ # Relatively common case: maxcount == 1. If we are not at the
+ # end of the string, it's done by a single direct check.
+ op = ctx.pat(ppos)
+ for op1, checkerfn in unroll_char_checker:
+ if op1 == op:
+ if checkerfn(ctx, ptr, ppos):
+ return ptr + 1
+ return ptr
+ elif maxcount != 65535:
+ # adjust end
end1 = ptr + maxcount
if end1 <= end:
end = end1
op = ctx.pat(ppos)
- for op1, (_, fre) in unroll_char_checker:
+ for op1, fre in unroll_fre_checker:
if op1 == op:
return fre(ctx, ptr, end, ppos)
raise Error("rsre.find_repetition_end[%d]" % op)
@@ -751,23 +802,60 @@
if checkerfn == match_ANY_ALL:
def fre(ctx, ptr, end, ppos):
return end
+ elif checkerfn == match_IN:
+ install_jitdriver_spec('MatchIn',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchIn.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
+ elif checkerfn == match_IN_IGNORE:
+ install_jitdriver_spec('MatchInIgnore',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchInIgnore.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
else:
+ # in the other cases, the fre() function is not JITted at all
+ # and is present as a residual call.
+ @specializectx
def fre(ctx, ptr, end, ppos):
while ptr < end and checkerfn(ctx, ptr, ppos):
ptr += 1
return ptr
- return checkerfn, fre
+ fre = func_with_new_name(fre, 'fre_' + checkerfn.__name__)
+ return fre
+
+unroll_char_checker = [
+ (OPCODE_ANY, match_ANY),
+ (OPCODE_ANY_ALL, match_ANY_ALL),
+ (OPCODE_IN, match_IN),
+ (OPCODE_IN_IGNORE, match_IN_IGNORE),
+ (OPCODE_LITERAL, match_LITERAL),
+ (OPCODE_LITERAL_IGNORE, match_LITERAL_IGNORE),
+ (OPCODE_NOT_LITERAL, match_NOT_LITERAL),
+ (OPCODE_NOT_LITERAL_IGNORE, match_NOT_LITERAL_IGNORE),
+ ]
+unroll_fre_checker = [(_op, _make_fre(_fn))
+ for (_op, _fn) in unroll_char_checker]
-unroll_char_checker = unrolling_iterable([
- (OPCODE_ANY, _make_fre(match_ANY)),
- (OPCODE_ANY_ALL, _make_fre(match_ANY_ALL)),
- (OPCODE_IN, _make_fre(match_IN)),
- (OPCODE_IN_IGNORE, _make_fre(match_IN_IGNORE)),
- (OPCODE_LITERAL, _make_fre(match_LITERAL)),
- (OPCODE_LITERAL_IGNORE, _make_fre(match_LITERAL_IGNORE)),
- (OPCODE_NOT_LITERAL, _make_fre(match_NOT_LITERAL)),
- (OPCODE_NOT_LITERAL_IGNORE, _make_fre(match_NOT_LITERAL_IGNORE)),
- ])
+unroll_char_checker = unrolling_iterable(unroll_char_checker)
+unroll_fre_checker = unrolling_iterable(unroll_fre_checker)
##### At dispatch
@@ -873,74 +961,139 @@
else:
return None
+install_jitdriver('Match',
+ greens=['ctx.pattern'], reds=['ctx'],
+ debugprint=(0,))
+
def match_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
+ ctx.jitdriver_Match.jit_merge_point(ctx=ctx)
return sre_match(ctx, 0, ctx.match_start, None) is not None
def search_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
- if ctx.pat(0) == OPCODE_INFO:
- if ctx.pat(2) & rsre_char.SRE_INFO_PREFIX and ctx.pat(5) > 1:
- return fast_search(ctx)
- return regular_search(ctx)
+ base = 0
+ charset = False
+ if ctx.pat(base) == OPCODE_INFO:
+ flags = ctx.pat(2)
+ if flags & rsre_char.SRE_INFO_PREFIX:
+ if ctx.pat(5) > 1:
+ return fast_search(ctx)
+ else:
+ charset = (flags & rsre_char.SRE_INFO_CHARSET)
+ base += 1 + ctx.pat(1)
+ if ctx.pat(base) == OPCODE_LITERAL:
+ return literal_search(ctx, base)
+ if charset:
+ return charset_search(ctx, base)
+ return regular_search(ctx, base)
+
+install_jitdriver('RegularSearch',
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
-def regular_search(ctx):
+def regular_search(ctx, base):
start = ctx.match_start
while start <= ctx.end:
- if sre_match(ctx, 0, start, None) is not None:
+ ctx.jitdriver_RegularSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if sre_match(ctx, base, start, None) is not None:
ctx.match_start = start
return True
start += 1
return False
+install_jitdriver_spec("LiteralSearch",
+ greens=['base', 'character', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(2, 0, 1))
+ at specializectx
+def literal_search(ctx, base):
+ # pattern starts with a literal character. this is used
+ # for short prefixes, and if fast search is disabled
+ character = ctx.pat(base + 1)
+ base += 2
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_LiteralSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base, character=character)
+ if ctx.str(start) == character:
+ if sre_match(ctx, base, start + 1, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec("CharsetSearch",
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
+ at specializectx
+def charset_search(ctx, base):
+ # pattern starts with a character from a known set
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_CharsetSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if rsre_char.check_charset(ctx.pattern, 5, ctx.str(start)):
+ if sre_match(ctx, base, start, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec('FastSearch',
+ greens=['i', 'prefix_len', 'ctx.pattern'],
+ reds=['string_position', 'ctx'],
+ debugprint=(2, 0))
@specializectx
def fast_search(ctx):
# skips forward in a string as fast as possible using information from
# an optimization info block
# <INFO> <1=skip> <2=flags> <3=min> <4=...>
# <5=length> <6=skip> <7=prefix data> <overlap data>
- flags = ctx.pat(2)
+ string_position = ctx.match_start
+ if string_position >= ctx.end:
+ return False
prefix_len = ctx.pat(5)
assert prefix_len >= 0
- prefix_skip = ctx.pat(6)
- assert prefix_skip >= 0
- overlap_offset = 7 + prefix_len - 1
- assert overlap_offset >= 0
- pattern_offset = ctx.pat(1) + 1
- ppos_start = pattern_offset + 2 * prefix_skip
- assert ppos_start >= 0
i = 0
- string_position = ctx.match_start
- end = ctx.end
- while string_position < end:
- while True:
- char_ord = ctx.str(string_position)
- if char_ord != ctx.pat(7 + i):
- if i == 0:
- break
- else:
- i = ctx.pat(overlap_offset + i)
- else:
- i += 1
- if i == prefix_len:
- # found a potential match
- start = string_position + 1 - prefix_len
- assert start >= 0
- ptr = start + prefix_skip
- if flags & rsre_char.SRE_INFO_LITERAL:
- # matched all of pure literal pattern
- ctx.match_start = start
- ctx.match_end = ptr
- ctx.match_marks = None
- return True
- if sre_match(ctx, ppos_start, ptr, None) is not None:
- ctx.match_start = start
- return True
- i = ctx.pat(overlap_offset + i)
- break
+ while True:
+ ctx.jitdriver_FastSearch.jit_merge_point(ctx=ctx,
+ string_position=string_position, i=i, prefix_len=prefix_len)
+ char_ord = ctx.str(string_position)
+ if char_ord != ctx.pat(7 + i):
+ if i > 0:
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
+ continue
+ else:
+ i += 1
+ if i == prefix_len:
+ # found a potential match
+ start = string_position + 1 - prefix_len
+ assert start >= 0
+ prefix_skip = ctx.pat(6)
+ ptr = start + prefix_skip
+ #flags = ctx.pat(2)
+ #if flags & rsre_char.SRE_INFO_LITERAL:
+ # # matched all of pure literal pattern
+ # ctx.match_start = start
+ # ctx.match_end = ptr
+ # ctx.match_marks = None
+ # return True
+ pattern_offset = ctx.pat(1) + 1
+ ppos_start = pattern_offset + 2 * prefix_skip
+ if sre_match(ctx, ppos_start, ptr, None) is not None:
+ ctx.match_start = start
+ return True
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
string_position += 1
- return False
+ if string_position >= ctx.end:
+ return False
Modified: pypy/branch/fast-forward/pypy/rlib/rsre/test/test_match.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rsre/test/test_match.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rsre/test/test_match.py Fri Oct 22 23:09:43 2010
@@ -1,24 +1,49 @@
-import _sre, re, sre_compile
-from pypy.rlib.rsre import rsre_core
+import re
+from pypy.rlib.rsre import rsre_core, rsre_char
-def get_code(regexp, flags=0, allargs=False):
- class GotIt(Exception):
- pass
- def my_compile(pattern, flags, code, *args):
- print code
- raise GotIt(code, flags, args)
- saved = _sre.compile
- try:
- _sre.compile = my_compile
- try:
- sre_compile.compile(regexp, flags)
- except GotIt, e:
- pass
+def get_hacked_sre_compile(my_compile):
+ """Return a copy of the sre_compile module for which the _sre
+ module is a custom module that has _sre.compile == my_compile
+ and CODESIZE == rsre_char.CODESIZE.
+ """
+ import sre_compile, __builtin__, new
+ sre_hacked = new.module("_sre_hacked")
+ sre_hacked.compile = my_compile
+ sre_hacked.MAGIC = sre_compile.MAGIC
+ sre_hacked.CODESIZE = rsre_char.CODESIZE
+ sre_hacked.getlower = rsre_char.getlower
+ def my_import(name, *args):
+ if name == '_sre':
+ return sre_hacked
else:
- raise ValueError("did not reach _sre.compile()!")
+ return default_import(name, *args)
+ src = sre_compile.__file__
+ if src.lower().endswith('.pyc') or src.lower().endswith('.pyo'):
+ src = src[:-1]
+ mod = new.module("sre_compile_hacked")
+ default_import = __import__
+ try:
+ __builtin__.__import__ = my_import
+ execfile(src, mod.__dict__)
finally:
- _sre.compile = saved
+ __builtin__.__import__ = default_import
+ return mod
+
+class GotIt(Exception):
+ pass
+def my_compile(pattern, flags, code, *args):
+ print code
+ raise GotIt(code, flags, args)
+sre_compile_hacked = get_hacked_sre_compile(my_compile)
+
+def get_code(regexp, flags=0, allargs=False):
+ try:
+ sre_compile_hacked.compile(regexp, flags)
+ except GotIt, e:
+ pass
+ else:
+ raise ValueError("did not reach _sre.compile()!")
if allargs:
return e.args
else:
Modified: pypy/branch/fast-forward/pypy/rlib/rstring.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rstring.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rstring.py Fri Oct 22 23:09:43 2010
@@ -69,6 +69,9 @@
def build(self):
return self.tp("").join(self.l)
+ def getlength(self):
+ return len(self.build())
+
class StringBuilder(AbstractStringBuilder):
tp = str
@@ -122,9 +125,12 @@
assert isinstance(s_times, SomeInteger)
return s_None
+ def method_getlength(self):
+ return SomeInteger(nonneg=True)
+
def method_build(self):
return SomeString()
-
+
def rtyper_makerepr(self, rtyper):
return rtyper.type_system.rbuilder.stringbuilder_repr
@@ -144,6 +150,9 @@
assert isinstance(s_times, SomeInteger)
return s_None
+ def method_getlength(self):
+ return SomeInteger(nonneg=True)
+
def method_build(self):
return SomeUnicodeString()
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_jit.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_jit.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_jit.py Fri Oct 22 23:09:43 2010
@@ -1,10 +1,17 @@
import py
+from pypy.conftest import option
from pypy.rlib.jit import hint, we_are_jitted, JitDriver, purefunction_promote
-from pypy.rlib.jit import JitHintError
+from pypy.rlib.jit import JitHintError, oopspec
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem import lltype
+def test_oopspec():
+ @oopspec('foobar')
+ def fn():
+ pass
+ assert fn.oopspec == 'foobar'
+
class BaseTestJIT(BaseRtypingTest):
def test_hint(self):
def f():
@@ -104,6 +111,26 @@
return n
py.test.raises(JitHintError, self.gengraph, fn, [int])
+ def test_green_field(self):
+ def get_printable_location(xfoo):
+ return str(ord(xfoo)) # xfoo must be annotated as a character
+ myjitdriver = JitDriver(greens=['x.foo'], reds=['n', 'x'],
+ get_printable_location=get_printable_location)
+ class A(object):
+ _immutable_fields_ = ['foo']
+ def fn(n):
+ x = A()
+ x.foo = chr(n)
+ while n > 0:
+ myjitdriver.can_enter_jit(x=x, n=n)
+ myjitdriver.jit_merge_point(x=x, n=n)
+ n -= 1
+ return n
+ t = self.gengraph(fn, [int])[0]
+ if option.view:
+ t.view()
+ # assert did not raise
+
class TestJITLLtype(BaseTestJIT, LLRtypeMixin):
pass
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rdynload.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rdynload.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rdynload.py Fri Oct 22 23:09:43 2010
@@ -1,15 +1,22 @@
from pypy.rlib.rdynload import *
-from pypy.rlib.libffi import get_libc_name
+from pypy.rlib.clibffi import get_libc_name
from pypy.rpython.lltypesystem import rffi, lltype
import py
class TestDLOperations:
def test_dlopen(self):
- py.test.raises(DLOpenError, "dlopen(rffi.str2charp('xxxxxxxxxxxx'))")
- assert dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp('xxxxxxxxxxxx')
+ py.test.raises(DLOpenError, "dlopen(s)")
+ rffi.free_charp(s)
+ #
+ s = rffi.str2charp(get_libc_name())
+ assert dlopen(s)
+ rffi.free_charp(s)
def test_dlsym(self):
- lib = dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp(get_libc_name())
+ lib = dlopen(s)
+ rffi.free_charp(s)
handle = rffi.cast(lltype.Ptr(lltype.FuncType([lltype.Signed],
lltype.Signed)), dlsym(lib, 'abs'))
assert 1 == handle(1)
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rsocket.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rsocket.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rsocket.py Fri Oct 22 23:09:43 2010
@@ -437,3 +437,31 @@
foo = self.serv.accept()
py.test.raises(SocketError, raise_error)
+def _test_cond_include(cond):
+ # Test that _rsocket_rffi is importable even on platforms where
+ # AF_PACKET or AF_NETLINK is not defined.
+ import re
+ from pypy.rlib import _rsocket_rffi
+ srcfile = _rsocket_rffi.__file__
+ if srcfile.lower().endswith('c') or srcfile.lower().endswith('o'):
+ srcfile = srcfile[:-1] # .pyc => .py
+ assert srcfile.lower().endswith('.py')
+ sourcelines = open(srcfile, 'rb').read().splitlines()
+ found = False
+ for i, line in enumerate(sourcelines):
+ line2 = re.sub(r"(\s*COND_HEADER\s*=)",
+ r"\1'#undef %s\\n'+" % cond,
+ line)
+ if line2 != line:
+ found = True
+ sourcelines[i] = line2
+ assert found
+ d = {}
+ sourcelines.append('')
+ exec '\n'.join(sourcelines) in d
+
+def test_no_AF_PACKET():
+ _test_cond_include('AF_PACKET')
+
+def test_no_AF_NETLINK():
+ _test_cond_include('AF_NETLINK')
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py Fri Oct 22 23:09:43 2010
@@ -29,6 +29,7 @@
s = StringBuilder()
s.append("a")
s.append("abc")
+ assert s.getlength() == len('aabc')
s.append("a")
s.append_slice("abc", 1, 2)
s.append_multiple_char('d', 4)
@@ -39,6 +40,7 @@
s.append(u'a')
s.append(u'abc')
s.append_slice(u'abcdef', 1, 2)
+ assert s.getlength() == len('aabcb')
s.append_multiple_char(u'd', 4)
assert s.build() == 'aabcbdddd'
assert isinstance(s.build(), unicode)
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py Fri Oct 22 23:09:43 2010
@@ -189,6 +189,8 @@
assert unused3 == len('more_garbage')
assert data3 == ''
+ rzlib.deflateEnd(stream)
+
def test_decompress_max_length():
"""
@@ -205,6 +207,8 @@
assert finished2 is True
assert unused2 == 0
+ rzlib.deflateEnd(stream)
+
def test_cornercases():
"""
@@ -215,6 +219,7 @@
bytes += rzlib.compress(stream, "")
bytes += rzlib.compress(stream, "", rzlib.Z_FINISH)
assert zlib.decompress(bytes) == ""
+ rzlib.deflateEnd(stream)
stream = rzlib.inflateInit()
data, finished, unused = rzlib.decompress(stream, "")
@@ -228,3 +233,4 @@
assert finished is False
assert unused > 0
buf = buf[-unused:]
+ rzlib.deflateEnd(stream)
Modified: pypy/branch/fast-forward/pypy/rpython/llinterp.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/llinterp.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/llinterp.py Fri Oct 22 23:09:43 2010
@@ -48,8 +48,7 @@
current_interpreter = None
- def __init__(self, typer, tracing=True, exc_data_ptr=None,
- malloc_check=True):
+ def __init__(self, typer, tracing=True, exc_data_ptr=None):
self.bindings = {}
self.typer = typer
# 'heap' is module or object that provides malloc, etc for lltype ops
@@ -57,9 +56,7 @@
self.exc_data_ptr = exc_data_ptr
self.frame_stack = []
self.tracer = None
- self.malloc_check = malloc_check
self.frame_class = LLFrame
- self.mallocs = {}
if tracing:
self.tracer = Tracer()
@@ -163,24 +160,6 @@
return self.exc_data_ptr
return None
- def remember_malloc(self, ptr, llframe):
- # err....
- self.mallocs[ptr._obj] = llframe
-
- def remember_free(self, ptr):
- try:
- del self.mallocs[ptr._obj]
- except KeyError:
- self._rehash_mallocs()
- del self.mallocs[ptr._obj]
-
- def _rehash_mallocs(self):
- # rehashing is needed because some objects' hash may change
- # when being turned to <C object>
- items = self.mallocs.items()
- self.mallocs = {}
- self.mallocs.update(items)
-
def _store_exception(self, exc):
raise PleaseOverwriteStoreException("You just invoked ll2ctypes callback without overwriting _store_exception on llinterpreter")
@@ -726,23 +705,23 @@
def op_malloc(self, obj, flags):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
if flavor == "stack":
result = self.heap.malloc(obj, zero=zero, flavor='raw')
self.alloca_objects.append(result)
return result
- ptr = self.heap.malloc(obj, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
+ ptr = self.heap.malloc(obj, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
def op_malloc_varsize(self, obj, flags, size):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
assert flavor in ('gc', 'raw')
try:
- ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
+ ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
except MemoryError:
self.make_llexception()
@@ -759,11 +738,10 @@
zero = flags.get('zero', False)
return self.heap.malloc_nonmovable(TYPE, size, zero=zero)
- def op_free(self, obj, flavor):
- assert isinstance(flavor, str)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_free(obj)
- self.heap.free(obj, flavor=flavor)
+ def op_free(self, obj, flags):
+ assert flags['flavor'] == 'raw'
+ track_allocation = flags.get('track_allocation', True)
+ self.heap.free(obj, flavor='raw', track_allocation=track_allocation)
def op_shrink_array(self, obj, smallersize):
return self.heap.shrink_array(obj, smallersize)
@@ -1037,6 +1015,13 @@
def op_stack_malloc(self, size): # mmh
raise NotImplementedError("backend only")
+ def op_track_alloc_start(self, addr):
+ # we don't do tracking at this level
+ checkadr(addr)
+
+ def op_track_alloc_stop(self, addr):
+ checkadr(addr)
+
# ____________________________________________________________
# Overflow-detecting variants
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/ll2ctypes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/ll2ctypes.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/ll2ctypes.py Fri Oct 22 23:09:43 2010
@@ -72,7 +72,7 @@
PIECESIZE = 0x08000000
PIECES = 10
m = rmmap.mmap(-1, PIECES * PIECESIZE,
- rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS,
+ rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS|rmmap.MAP_NORESERVE,
rmmap.PROT_READ|rmmap.PROT_WRITE)
m.close = lambda : None # leak instead of giving a spurious
# error at CPython's shutdown
@@ -827,6 +827,8 @@
except (ValueError, OverflowError):
for tc in 'HIL':
if array(tc).itemsize == array('u').itemsize:
+ import struct
+ cobj &= 256 ** struct.calcsize(tc) - 1
llobj = array('u', array(tc, (cobj,)).tostring())[0]
break
else:
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py Fri Oct 22 23:09:43 2010
@@ -440,7 +440,7 @@
[rffi.INT],
rffi.INT,
sandboxsafe=True, _nowrapper=True)
- _dev_zero = rffi.str2charp('/dev/zero') # prebuilt
+ _dev_zero = rffi.str2charp_immortal('/dev/zero') # prebuilt
def clear_large_memory_chunk(baseaddr, size):
# on some Unixy platforms, reading from /dev/zero is the fastest way
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py Fri Oct 22 23:09:43 2010
@@ -105,11 +105,13 @@
if (isinstance(self.TYPE, lltype.ContainerType)
and self.TYPE._gckind == 'gc'):
assert self.repeat == 1
- p = lltype.malloc(self.TYPE, flavor='raw', zero=zero)
+ p = lltype.malloc(self.TYPE, flavor='raw', zero=zero,
+ track_allocation=False)
return cast_ptr_to_adr(p)
else:
T = lltype.FixedSizeArray(self.TYPE, self.repeat)
- p = lltype.malloc(T, flavor='raw', zero=zero)
+ p = lltype.malloc(T, flavor='raw', zero=zero,
+ track_allocation=False)
array_adr = cast_ptr_to_adr(p)
return array_adr + ArrayItemsOffset(T)
@@ -288,7 +290,8 @@
count = 0
p = lltype.malloc(parenttype or self.TYPE, count,
immortal = self.TYPE._gckind == 'raw',
- zero = zero)
+ zero = zero,
+ track_allocation = False)
return cast_ptr_to_adr(p)
def raw_memcopy(self, srcadr, dstadr):
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py Fri Oct 22 23:09:43 2010
@@ -405,6 +405,8 @@
'raw_load': LLOp(sideeffects=False),
'raw_store': LLOp(),
'stack_malloc': LLOp(), # mmh
+ 'track_alloc_start': LLOp(),
+ 'track_alloc_stop': LLOp(),
'adr_add': LLOp(canfold=True),
'adr_sub': LLOp(canfold=True),
'adr_delta': LLOp(canfold=True),
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py Fri Oct 22 23:09:43 2010
@@ -1,7 +1,3 @@
-import StringIO
-import traceback
-import sys
-
import py
from pypy.rlib.rarithmetic import (r_int, r_uint, intmask, r_singlefloat,
r_ulonglong, r_longlong, r_longfloat,
@@ -10,25 +6,13 @@
from pypy.tool.uid import Hashable
from pypy.tool.tls import tlsobject
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
from types import NoneType
from sys import maxint
import weakref
TLS = tlsobject()
-# Track allocations to detect memory leaks
-# Don't track 'gc' and immortal mallocs
-TRACK_ALLOCATIONS = False
-ALLOCATED = identity_dict()
-
-def start_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = True
- ALLOCATED.clear()
-
-def stop_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = False
class _uninitialized(object):
def __init__(self, TYPE):
@@ -801,6 +785,8 @@
return llmemory.cast_adr_to_ptr(value, TGT)
elif TGT == llmemory.Address and isinstance(ORIG, Ptr):
return llmemory.cast_ptr_to_adr(value)
+ elif TGT == Signed and isinstance(ORIG, Ptr) and ORIG.TO._gckind == 'raw':
+ return llmemory.cast_adr_to_int(llmemory.cast_ptr_to_adr(value), 'symbolic')
raise TypeError("don't know how to cast from %r to %r" % (ORIG, TGT))
@@ -1387,41 +1373,21 @@
__slots__ = ('_TYPE',
'_parent_type', '_parent_index', '_keepparent',
'_wrparent',
- '__weakref__', '_traceback',
- '__storage')
+ '__weakref__',
+ '_storage')
- def __init__(self, TYPE, track_allocation=None):
+ def __init__(self, TYPE):
self._wrparent = None
self._TYPE = TYPE
self._storage = True # means "use default storage", as opposed to:
# None - container was freed
# <ctypes object> - using ctypes
# (see ll2ctypes.py)
- if track_allocation is not False and TRACK_ALLOCATIONS:
- self._traceback = self._get_traceback()
- ALLOCATED[self] = None
- else:
- self._traceback = None
-
- def _get_traceback(self):
- frame = sys._getframe().f_back.f_back.f_back.f_back
- sio = StringIO.StringIO()
- traceback.print_stack(frame, file=sio)
- return sio.getvalue()
def _free(self):
self._check() # no double-frees
self._storage = None
- def _storage_get(self):
- return self.__storage
-
- def _storage_set(self, value):
- self.__storage = value
- if value is not True and self in ALLOCATED:
- del ALLOCATED[self]
- _storage = property(_storage_get, _storage_set)
-
def _was_freed(self):
if self._storage is None:
return True
@@ -1500,12 +1466,12 @@
__slots__ = ('_hash_cache_', '_compilation_info')
- def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
my_variety = _struct_variety(TYPE._names)
return object.__new__(my_variety)
- def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
- _parentable.__init__(self, TYPE, track_allocation)
+ def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
+ _parentable.__init__(self, TYPE)
if n is not None and TYPE._arrayfld is None:
raise TypeError("%r is not variable-sized" % (TYPE,))
if n is None and TYPE._arrayfld is not None:
@@ -1513,8 +1479,7 @@
first, FIRSTTYPE = TYPE._first_struct()
for fld, typ in TYPE._flds.items():
if fld == TYPE._arrayfld:
- value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld,
- track_allocation=track_allocation)
+ value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld)
else:
value = typ._allocate(initialization=initialization, parent=self, parentindex=fld)
setattr(self, fld, value)
@@ -1575,12 +1540,12 @@
__slots__ = ('items',)
- def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None):
if not isinstance(n, int):
raise TypeError, "array length must be an int"
if n < 0:
raise ValueError, "negative array length"
- _parentable.__init__(self, TYPE, track_allocation)
+ _parentable.__init__(self, TYPE)
try:
myrange = range(n)
except OverflowError:
@@ -1647,7 +1612,7 @@
_cache = weakref.WeakKeyDictionary() # parentarray -> {subarrays}
def __init__(self, TYPE, parent, baseoffset_or_fieldname):
- _parentable.__init__(self, TYPE, track_allocation=False)
+ _parentable.__init__(self, TYPE)
self._setparentstructure(parent, baseoffset_or_fieldname)
# Keep the parent array alive, we share the same allocation.
# Don't do it if we are inside a GC object, though -- it's someone
@@ -1655,6 +1620,13 @@
if typeOf(top_container(parent))._gckind == 'raw':
self._keepparent = parent
+ def __str__(self):
+ parent = self._wrparent()
+ if parent is None:
+ return '_subarray at %s in already freed' % (self._parent_index,)
+ return '_subarray at %r in %s' % (self._parent_index,
+ parent._TYPE)
+
def __repr__(self):
parent = self._wrparent()
if parent is None:
@@ -1868,8 +1840,9 @@
return id(self.value)
-def malloc(T, n=None, flavor='gc', immortal=False, zero=False):
- assert flavor != 'cpy'
+def malloc(T, n=None, flavor='gc', immortal=False, zero=False,
+ track_allocation=True):
+ assert flavor in ('gc', 'raw')
if zero or immortal:
initialization = 'example'
elif flavor == 'raw':
@@ -1877,9 +1850,9 @@
else:
initialization = 'malloc'
if isinstance(T, Struct):
- o = _struct(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _struct(T, n, initialization=initialization)
elif isinstance(T, Array):
- o = _array(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _array(T, n, initialization=initialization)
elif isinstance(T, OpaqueType):
assert n is None
o = _opaque(T, initialization=initialization)
@@ -1887,17 +1860,50 @@
raise TypeError, "malloc for Structs and Arrays only"
if T._gckind != 'gc' and not immortal and flavor.startswith('gc'):
raise TypeError, "gc flavor malloc of a non-GC non-immortal structure"
+ if flavor == "raw" and not immortal and track_allocation:
+ leakfinder.remember_malloc(o, framedepth=2)
solid = immortal or not flavor.startswith('gc') # immortal or non-gc case
return _ptr(Ptr(T), o, solid)
-def free(p, flavor):
+def free(p, flavor, track_allocation=True):
if flavor.startswith('gc'):
raise TypeError, "gc flavor free"
T = typeOf(p)
if not isinstance(T, Ptr) or p._togckind() != 'raw':
raise TypeError, "free(): only for pointers to non-gc containers"
+ if track_allocation:
+ leakfinder.remember_free(p._obj0)
p._obj0._free()
+def _make_scoped_allocator(T):
+ class ScopedAlloc:
+ def __init__(self, n=None, zero=False):
+ if n is None:
+ self.buf = malloc(T, flavor='raw', zero=zero)
+ else:
+ self.buf = malloc(T, n, flavor='raw', zero=zero)
+
+ def __enter__(self):
+ return self.buf
+
+ def __exit__(self, *args):
+ free(self.buf, flavor='raw')
+
+ ScopedAlloc.__name__ = 'ScopedAlloc_%s' % (T,)
+ return ScopedAlloc
+_make_scoped_allocator._annspecialcase_ = 'specialize:memo'
+
+def scoped_alloc(T, n=None, zero=False):
+ """Returns a context manager which handles allocation and
+ deallocation of temporary memory. Use it in a with statement::
+
+ with scoped_alloc(Array(Signed), 1) as array:
+ ...use array...
+ ...it's freed now.
+ """
+ return _make_scoped_allocator(T)(n=n, zero=zero)
+scoped_alloc._annspecialcase_ = 'specialize:arg(0)'
+
def functionptr(TYPE, name, **attrs):
if not isinstance(TYPE, FuncType):
raise TypeError, "functionptr() for FuncTypes only"
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rbuilder.py Fri Oct 22 23:09:43 2010
@@ -100,6 +100,10 @@
ll_builder.used = used
@staticmethod
+ def ll_getlength(ll_builder):
+ return ll_builder.used
+
+ @staticmethod
def ll_build(ll_builder):
final_size = ll_builder.used
assert final_size >= 0
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rclass.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rclass.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rclass.py Fri Oct 22 23:09:43 2010
@@ -329,16 +329,33 @@
fields['__class__'] = 'typeptr', get_type_repr(self.rtyper)
else:
# instance attributes
- if llfields is None:
- llfields = []
attrs = self.classdef.attrs.items()
attrs.sort()
+ myllfields = []
for name, attrdef in attrs:
if not attrdef.readonly:
r = self.rtyper.getrepr(attrdef.s_value)
mangled_name = 'inst_' + name
fields[name] = mangled_name, r
- llfields.append((mangled_name, r.lowleveltype))
+ myllfields.append((mangled_name, r.lowleveltype))
+
+ # Sort the instance attributes by decreasing "likely size",
+ # as reported by rffi.sizeof(), to minimize padding holes in C.
+ # Fields of the same size are sorted by name (by attrs.sort()
+ # above) just to minimize randomness.
+ def keysize((_, T)):
+ if T is lltype.Void:
+ return None
+ from pypy.rpython.lltypesystem.rffi import sizeof
+ try:
+ return -sizeof(T)
+ except StandardError:
+ return None
+ myllfields.sort(key = keysize)
+ if llfields is None:
+ llfields = myllfields
+ else:
+ llfields = llfields + myllfields
self.rbase = getinstancerepr(self.rtyper, self.classdef.basedef,
self.gcflavor)
@@ -403,7 +420,7 @@
return cast_pointer(self.lowleveltype, result)
def create_instance(self):
- return malloc(self.object_type, flavor=self.gcflavor)
+ return malloc(self.object_type, flavor=self.gcflavor, immortal=True)
def initialize_prebuilt_data(self, value, classdef, result):
if self.classdef is not None:
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py Fri Oct 22 23:09:43 2010
@@ -609,6 +609,15 @@
return array
str2charp._annenforceargs_ = [strtype]
+ def str2charp_immortal(s):
+ "NOT_RPYTHON"
+ array = lltype.malloc(TYPEP.TO, len(s) + 1, flavor='raw',
+ immortal=True)
+ for i in range(len(s)):
+ array[i] = s[i]
+ array[len(s)] = lastchar
+ return array
+
def free_charp(cp):
lltype.free(cp, flavor='raw')
@@ -646,10 +655,18 @@
"""
Either free a non-moving buffer or keep the original storage alive.
"""
- if rgc.can_move(data):
+ # We cannot rely on rgc.can_move(data) here, because its result
+ # might have changed since get_nonmovingbuffer(). Instead we check
+ # if 'buf' points inside 'data'. This is only possible if we
+ # followed the 2nd case in get_nonmovingbuffer(); in the first case,
+ # 'buf' points to its own raw-malloced memory.
+ data = llstrtype(data)
+ data_start = cast_ptr_to_adr(data) + \
+ offsetof(STRTYPE, 'chars') + itemoffsetof(STRTYPE.chars, 0)
+ followed_2nd_path = (buf == cast(TYPEP, data_start))
+ keepalive_until_here(data)
+ if not followed_2nd_path:
lltype.free(buf, flavor='raw')
- else:
- keepalive_until_here(data)
# int -> (char*, str)
def alloc_buffer(count):
@@ -719,19 +736,19 @@
l = [cp[i] for i in range(size)]
return emptystr.join(l)
- return (str2charp, free_charp, charp2str,
+ return (str2charp, str2charp_immortal, free_charp, charp2str,
get_nonmovingbuffer, free_nonmovingbuffer,
alloc_buffer, str_from_buffer, keep_buffer_alive_until_here,
charp2strn, charpsize2str,
)
-(str2charp, free_charp, charp2str,
+(str2charp, str2charp_immortal, free_charp, charp2str,
get_nonmovingbuffer, free_nonmovingbuffer,
alloc_buffer, str_from_buffer, keep_buffer_alive_until_here,
charp2strn, charpsize2str,
) = make_string_mappings(str)
-(unicode2wcharp, free_wcharp, wcharp2unicode,
+(unicode2wcharp, unicode2wcharp_immortal, free_wcharp, wcharp2unicode,
get_nonmoving_unicodebuffer, free_nonmoving_unicodebuffer,
alloc_unicodebuffer, unicode_from_buffer, keep_unicodebuffer_alive_until_here,
wcharp2unicoden, wcharpsize2unicode,
@@ -918,3 +935,11 @@
"""
return cast(lltype.Signed, getattr(pdst, fieldname))
getintfield._annspecialcase_ = 'specialize:ll_and_arg(1)'
+
+class scoped_str2charp:
+ def __init__(self, value):
+ self.buf = str2charp(value)
+ def __enter__(self):
+ return self.buf
+ def __exit__(self, *args):
+ free_charp(self.buf)
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_ll2ctypes.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Fri Oct 22 23:09:43 2010
@@ -765,6 +765,7 @@
assert abs(float(b[1]) - 1.1) < 1E-6
assert isinstance(b[2], rffi.r_singlefloat)
assert abs(float(b[2]) - 2.2) < 1E-6
+ lltype.free(a, flavor='raw')
def test_different_signatures(self):
if sys.platform=='win32':
@@ -879,6 +880,7 @@
qsort(rffi.cast(rffi.VOIDP, a), 5, rffi.sizeof(rffi.INT), compare)
for i in range(5):
assert a[i] == i + 1
+ lltype.free(a, flavor='raw')
def test_array_type_bug(self):
A = lltype.Array(lltype.Signed)
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_llmemory.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_llmemory.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_llmemory.py Fri Oct 22 23:09:43 2010
@@ -324,12 +324,14 @@
p_t = lltype.malloc(T)
assert p_t.s == lltype.nullptr(S)
# raw malloc does not
- p_raw_t = lltype.malloc(T, flavor="raw")
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
+ U = lltype.Struct("U", ('x', lltype.Signed))
+ p_raw_t = lltype.malloc(U, flavor="raw")
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+ lltype.free(p_raw_t, flavor="raw")
# this sort of raw_malloc too
- p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(T)), lltype.Ptr(T))
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
-
+ p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(U)), lltype.Ptr(U))
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+
def test_raw_malloc_signed_bunch():
adr = raw_malloc(sizeof(lltype.Signed) * 50)
@@ -601,7 +603,8 @@
a = lltype.malloc(A, flavor='raw')
src = cast_ptr_to_adr(a) + itemoffsetof(A, 0)
raw_memclear(src, sizeof(lltype.Signed) * 0)
-
+ lltype.free(a, flavor="raw")
+
def test_nonneg():
S1 = lltype.GcStruct('S1', ('x', lltype.Float))
A1 = lltype.GcArray(lltype.Float)
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_lltype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_lltype.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_lltype.py Fri Oct 22 23:09:43 2010
@@ -1,7 +1,9 @@
+from __future__ import with_statement
import py
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
def isweak(p, T):
try:
@@ -804,22 +806,20 @@
class TestTrackAllocation:
- def setup_method(self, func):
- start_tracking_allocations()
-
- def teardown_method(self, func):
- assert not lltype.ALLOCATED, "Memory was not correctly freed"
- stop_tracking_allocations()
+ def test_automatic_tracking(self):
+ # calls to start_tracking_allocations/stop_tracking_allocations
+ # should occur automatically from pypy/conftest.py. Check that.
+ assert leakfinder.TRACK_ALLOCATIONS
def test_track_allocation(self):
"""A malloc'd buffer fills the ALLOCATED dictionary"""
- assert lltype.TRACK_ALLOCATIONS
- assert not lltype.ALLOCATED
+ assert leakfinder.TRACK_ALLOCATIONS
+ assert not leakfinder.ALLOCATED
buf = malloc(Array(Signed), 1, flavor="raw")
- assert len(lltype.ALLOCATED) == 1
- assert lltype.ALLOCATED.keys() == [buf._obj]
+ assert len(leakfinder.ALLOCATED) == 1
+ assert leakfinder.ALLOCATED.keys() == [buf._obj]
free(buf, flavor="raw")
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_str_from_buffer(self):
"""gc-managed memory does not need to be freed"""
@@ -828,16 +828,28 @@
for i in range(size): raw_buf[i] = 'a'
rstr = rffi.str_from_buffer(raw_buf, gc_buf, size, size)
rffi.keep_buffer_alive_until_here(raw_buf, gc_buf)
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_leak_traceback(self):
"""Test info stored for allocated items"""
buf = malloc(Array(Signed), 1, flavor="raw")
- traceback = lltype.ALLOCATED.keys()[0]._traceback
+ traceback = leakfinder.ALLOCATED.values()[0]
lines = traceback.splitlines()
assert 'malloc(' in lines[-1] and 'flavor="raw")' in lines[-1]
- # XXX The traceback should not be too long
+ # The traceback should not be too long
print traceback
free(buf, flavor="raw")
+
+ def test_no_tracking(self):
+ p1 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ p2 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ free(p2, flavor='raw', track_allocation=False)
+ # p1 is not freed
+
+ def test_scoped_allocator(self):
+ with scoped_alloc(Array(Signed), 1) as array:
+ array[0] = -42
+ x = array[0]
+ assert x == -42
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_rffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_rffi.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_rffi.py Fri Oct 22 23:09:43 2010
@@ -9,7 +9,7 @@
from pypy.rpython.lltypesystem.rstr import STR
from pypy.rpython.lltypesystem import lltype
from pypy.tool.udir import udir
-from pypy.rpython.test.test_llinterp import interpret, MallocMismatch
+from pypy.rpython.test.test_llinterp import interpret
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.annotation.annrpython import RPythonAnnotator
from pypy.rpython.rtyper import RPythonTyper
@@ -787,3 +787,10 @@
mixann.getgraph(f2, [], s_None)
mixann.finish()
+def test_force_cast_unichar():
+ x = cast(lltype.UniChar, -1)
+ assert isinstance(x, unicode)
+ if sys.maxunicode == 65535:
+ assert cast(LONG, x) == 65535
+ else:
+ assert cast(LONG, cast(INT, x)) == -1
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py Fri Oct 22 23:09:43 2010
@@ -100,11 +100,14 @@
# allocation of the given size.
length = small_request_threshold / WORD + 1
self.page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.full_page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.nblocks_for_size = lltype.malloc(rffi.CArray(lltype.Signed),
- length, flavor='raw')
+ length, flavor='raw',
+ immortal=True)
self.hdrsize = llmemory.raw_malloc_usage(llmemory.sizeof(PAGE_HEADER))
assert page_size > self.hdrsize
self.nblocks_for_size[0] = 0 # unused
@@ -114,11 +117,13 @@
self.max_pages_per_arena = arena_size // page_size
self.arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
# this is used in mass_free() only
self.old_arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
#
# the arena currently consumed; it must have at least one page
# available, or be NULL. The arena object that we point to is
@@ -281,7 +286,7 @@
npages = (arena_end - firstpage) // self.page_size
#
# Allocate an ARENA object and initialize it
- arena = lltype.malloc(ARENA, flavor='raw')
+ arena = lltype.malloc(ARENA, flavor='raw', track_allocation=False)
arena.base = arena_base
arena.nfreepages = 0 # they are all uninitialized pages
arena.totalpages = npages
@@ -332,7 +337,7 @@
#
# The whole arena is empty. Free it.
llarena.arena_free(arena.base)
- lltype.free(arena, flavor='raw')
+ lltype.free(arena, flavor='raw', track_allocation=False)
#
else:
# Insert 'arena' in the correct arenas_lists[n]
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gctransform/asmgcroot.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gctransform/asmgcroot.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gctransform/asmgcroot.py Fri Oct 22 23:09:43 2010
@@ -75,7 +75,8 @@
key = (TYPE, num)
if key not in sradict:
CONTAINER = lltype.FixedSizeArray(TYPE, 1)
- p = lltype.malloc(CONTAINER, flavor='raw', zero=True)
+ p = lltype.malloc(CONTAINER, flavor='raw', zero=True,
+ immortal=True)
sradict[key] = Constant(p, lltype.Ptr(CONTAINER))
sra.append(sradict[key])
#
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gctransform/transform.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gctransform/transform.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gctransform/transform.py Fri Oct 22 23:09:43 2010
@@ -430,7 +430,8 @@
return self.parenttransformer.gct_malloc_varsize(hop)
def gct_free(self, hop):
- flavor = hop.spaceop.args[1].value
+ flags = hop.spaceop.args[1].value
+ flavor = flags['flavor']
assert flavor == 'raw'
return self.parenttransformer.gct_free(hop)
@@ -532,6 +533,8 @@
resulttype=llmemory.Address)
if flags.get('zero'):
hop.genop("raw_memclear", [v_raw, c_size])
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_fv_stack_malloc(self, hop, flags, TYPE, c_size):
@@ -602,15 +605,20 @@
[self.raw_malloc_varsize_ptr, v_length,
c_const_size, c_item_size, c_offset_to_length],
resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_free(self, hop):
op = hop.spaceop
- flavor = op.args[1].value
+ flags = op.args[1].value
+ flavor = flags['flavor']
v = op.args[0]
assert flavor != 'cpy', "cannot free CPython objects directly"
if flavor == 'raw':
v = hop.genop("cast_ptr_to_adr", [v], resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_stop", [v])
hop.genop('raw_free', [v])
else:
assert False, "%s has no support for free with flavor %r" % (self, flavor)
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py Fri Oct 22 23:09:43 2010
@@ -42,7 +42,8 @@
#
# Interface for the llinterp
#
- def malloc(self, TYPE, n=None, flavor='gc', zero=False):
+ def malloc(self, TYPE, n=None, flavor='gc', zero=False,
+ track_allocation=True):
if flavor == 'gc':
typeid = self.get_type_id(TYPE)
addr = self.gc.malloc(typeid, n, zero=zero)
@@ -51,7 +52,8 @@
gctypelayout.zero_gc_pointers(result)
return result
else:
- return lltype.malloc(TYPE, n, flavor=flavor, zero=zero)
+ return lltype.malloc(TYPE, n, flavor=flavor, zero=zero,
+ track_allocation=track_allocation)
def malloc_nonmovable(self, TYPE, n=None, zero=False):
typeid = self.get_type_id(TYPE)
@@ -69,9 +71,10 @@
return self.gc.shrink_array(addr, smallersize)
return False
- def free(self, TYPE, flavor='gc'):
+ def free(self, TYPE, flavor='gc', track_allocation=True):
assert flavor != 'gc'
- return lltype.free(TYPE, flavor=flavor)
+ return lltype.free(TYPE, flavor=flavor,
+ track_allocation=track_allocation)
def setfield(self, obj, fieldname, fieldvalue):
STRUCT = lltype.typeOf(obj).TO
Modified: pypy/branch/fast-forward/pypy/rpython/memory/support.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/support.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/support.py Fri Oct 22 23:09:43 2010
@@ -30,7 +30,8 @@
# we zero-initialize the chunks to make the translation
# backends happy, but we don't need to do it at run-time.
zero = not we_are_translated()
- return lltype.malloc(CHUNK, flavor="raw", zero=zero)
+ return lltype.malloc(CHUNK, flavor="raw", zero=zero,
+ track_allocation=False)
result = self.free_list
self.free_list = result.next
@@ -44,7 +45,7 @@
# Don't cache the old chunks but free them immediately.
# Helps debugging, and avoids that old chunks full of
# addresses left behind by a test end up in genc...
- lltype.free(chunk, flavor="raw")
+ lltype.free(chunk, flavor="raw", track_allocation=False)
unused_chunks = FreeList()
cache[chunk_size] = unused_chunks, null_chunk
Modified: pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py Fri Oct 22 23:09:43 2010
@@ -368,6 +368,14 @@
res = self.interpret(f, [4, 42])
assert res == 12
+ def test_print_leak(self):
+ def f(n):
+ for i in range(n):
+ print i
+ return 42
+ res = self.interpret(f, [10])
+ assert res == 42
+
def test_weakref_across_minor_collection(self):
import weakref
class A:
Modified: pypy/branch/fast-forward/pypy/rpython/module/ll_time.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/module/ll_time.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/module/ll_time.py Fri Oct 22 23:09:43 2010
@@ -108,7 +108,7 @@
errcode = -1
if self.GETTIMEOFDAY_NO_TZ:
- errcode = g_gettimeofday(t)
+ errcode = c_gettimeofday(t)
else:
errcode = c_gettimeofday(t, void)
Modified: pypy/branch/fast-forward/pypy/rpython/rbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/rbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/rbuilder.py Fri Oct 22 23:09:43 2010
@@ -36,8 +36,12 @@
hop.exception_cannot_occur()
return hop.gendirectcall(self.ll_append_multiple_char, *vlist)
+ def rtype_method_getlength(self, hop):
+ vlist = hop.inputargs(self)
+ hop.exception_cannot_occur()
+ return hop.gendirectcall(self.ll_getlength, *vlist)
+
def rtype_method_build(self, hop):
vlist = hop.inputargs(self)
hop.exception_cannot_occur()
return hop.gendirectcall(self.ll_build, *vlist)
-
Modified: pypy/branch/fast-forward/pypy/rpython/rbuiltin.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/rbuiltin.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/rbuiltin.py Fri Oct 22 23:09:43 2010
@@ -345,17 +345,22 @@
BUILTIN_TYPER[object.__init__] = rtype_object__init__
# annotation of low-level types
-def rtype_malloc(hop, i_flavor=None, i_zero=None):
+def rtype_malloc(hop, i_flavor=None, i_zero=None, i_track_allocation=None):
assert hop.args_s[0].is_constant()
vlist = [hop.inputarg(lltype.Void, arg=0)]
opname = 'malloc'
- v_flavor, v_zero = parse_kwds(hop, (i_flavor, lltype.Void), (i_zero, None))
+ v_flavor, v_zero, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_zero, None),
+ (i_track_allocation, None))
flags = {'flavor': 'gc'}
if v_flavor is not None:
flags['flavor'] = v_flavor.value
if i_zero is not None:
flags['zero'] = v_zero.value
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
vlist.append(hop.inputconst(lltype.Void, flags))
if hop.nb_args == 2:
@@ -366,10 +371,19 @@
hop.exception_is_here()
return hop.genop(opname, vlist, resulttype = hop.r_result.lowleveltype)
-def rtype_free(hop, i_flavor):
- assert i_flavor == 1
+def rtype_free(hop, i_flavor, i_track_allocation=None):
+ vlist = [hop.inputarg(hop.args_r[0], arg=0)]
+ v_flavor, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_track_allocation, None))
+ #
+ assert v_flavor is not None and v_flavor.value == 'raw'
+ flags = {'flavor': 'raw'}
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
+ vlist.append(hop.inputconst(lltype.Void, flags))
+ #
hop.exception_cannot_occur()
- vlist = hop.inputargs(hop.args_r[0], lltype.Void)
hop.genop('free', vlist)
def rtype_const_result(hop):
@@ -584,8 +598,9 @@
vinst, = hop.inputargs(hop.args_r[0])
flavor = hop.args_r[0].gcflavor
assert flavor != 'gc'
- cflavor = hop.inputconst(lltype.Void, flavor)
- return hop.genop('free', [vinst, cflavor])
+ flags = {'flavor': flavor}
+ cflags = hop.inputconst(lltype.Void, flags)
+ return hop.genop('free', [vinst, cflags])
BUILTIN_TYPER[objectmodel.free_non_gc_object] = rtype_free_non_gc_object
Modified: pypy/branch/fast-forward/pypy/rpython/rpbc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/rpbc.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/rpbc.py Fri Oct 22 23:09:43 2010
@@ -256,6 +256,8 @@
def convert_const(self, value):
if isinstance(value, types.MethodType) and value.im_self is None:
value = value.im_func # unbound method -> bare function
+ elif isinstance(value, staticmethod):
+ value = value.__get__(42) # hackish, get the function wrapped by staticmethod
if self.lowleveltype is Void:
return None
if value is None:
Modified: pypy/branch/fast-forward/pypy/rpython/rtyper.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/rtyper.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/rtyper.py Fri Oct 22 23:09:43 2010
@@ -421,7 +421,7 @@
assert noexclink.exitcase is None
if pos == "removed":
# the exception cannot actually occur at all.
- # See for example rspecialcase.rtype_call_specialcase().
+ # This is set by calling exception_cannot_occur().
# We just remove all exception links.
block.exitswitch = None
block.exits = block.exits[:1]
@@ -1019,7 +1019,7 @@
from pypy.rpython import rint, rbool, rfloat
from pypy.rpython import rrange
from pypy.rpython import rstr, rdict, rlist
-from pypy.rpython import rclass, rbuiltin, rpbc, rspecialcase
+from pypy.rpython import rclass, rbuiltin, rpbc
from pypy.rpython import rexternalobj
from pypy.rpython import rptr
from pypy.rpython import rgeneric
Modified: pypy/branch/fast-forward/pypy/rpython/test/test_llinterp.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/test/test_llinterp.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/test/test_llinterp.py Fri Oct 22 23:09:43 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import py
import sys
from pypy.rpython.lltypesystem.lltype import typeOf, pyobjectptr, Ptr,\
@@ -5,6 +6,7 @@
from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rpython.llinterp import LLInterpreter, LLException, log
from pypy.rpython.rmodel import inputconst
+from pypy.rpython.annlowlevel import hlstr
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.rint import signed_repr
from pypy.rpython.lltypesystem import rstr, lltype
@@ -12,13 +14,11 @@
from pypy.annotation.model import lltype_to_annotation
from pypy.rlib.rarithmetic import r_uint, ovfcheck
from pypy.rpython.ootypesystem import ootype
+from pypy.tool import leakfinder
from pypy import conftest
# switch on logging of interp to show more info on failing tests
-class MallocMismatch(Exception):
- pass
-
def setup_module(mod):
mod.logstate = py.log._getstate()
py.log.setconsumer("llinterp", py.log.STDOUT)
@@ -72,7 +72,7 @@
def get_interpreter(func, values, view='auto', viewbefore='auto', policy=None,
someobjects=False, type_system="lltype", backendopt=False,
- config=None, malloc_check=True, **extraconfigopts):
+ config=None, **extraconfigopts):
extra_key = [(key, value) for key, value in extraconfigopts.iteritems()]
extra_key.sort()
extra_key = tuple(extra_key)
@@ -97,7 +97,7 @@
viewbefore, policy, type_system=type_system,
backendopt=backendopt, config=config,
**extraconfigopts)
- interp = LLInterpreter(typer, malloc_check=malloc_check)
+ interp = LLInterpreter(typer)
_tcache[key] = (t, interp, graph)
# keep the cache small
_lastinterpreted.append(key)
@@ -115,10 +115,17 @@
interp, graph = get_interpreter(func, values, view, viewbefore, policy,
someobjects, type_system=type_system,
backendopt=backendopt, config=config,
- malloc_check=malloc_check, **kwargs)
- result = interp.eval_graph(graph, values)
- if malloc_check and interp.mallocs:
- raise MallocMismatch(interp.mallocs)
+ **kwargs)
+ if not malloc_check:
+ result = interp.eval_graph(graph, values)
+ else:
+ prev = leakfinder.start_tracking_allocations()
+ try:
+ result = interp.eval_graph(graph, values)
+ finally:
+ leaks = leakfinder.stop_tracking_allocations(False, prev)
+ if leaks:
+ raise leakfinder.MallocMismatch(leaks)
return result
def interpret_raises(exc, func, values, view='auto', viewbefore='auto',
@@ -418,6 +425,7 @@
assert result
def test_stack_malloc():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
def f():
@@ -430,6 +438,7 @@
assert result == 1
def test_invalid_stack_access():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
globala = A()
@@ -605,7 +614,7 @@
if x:
free(t, flavor='raw')
interpret(f, [1])
- py.test.raises(MallocMismatch, "interpret(f, [0])")
+ py.test.raises(leakfinder.MallocMismatch, "interpret(f, [0])")
def f():
t1 = malloc(T, flavor='raw')
@@ -615,3 +624,37 @@
interpret(f, [])
+def test_context_manager():
+ state = []
+ class C:
+ def __enter__(self):
+ state.append('acquire')
+ return self
+ def __exit__(self, *args):
+ if args[1] is not None:
+ state.append('raised')
+ state.append('release')
+ def f():
+ try:
+ with C() as c:
+ state.append('use')
+ raise ValueError
+ except ValueError:
+ pass
+ return ', '.join(state)
+ res = interpret(f, [])
+ assert hlstr(res) == 'acquire, use, raised, release'
+
+
+def test_scoped_allocator():
+ from pypy.rpython.lltypesystem.lltype import scoped_alloc, Array, Signed
+ T = Array(Signed)
+
+ def f():
+ x = 0
+ with scoped_alloc(T, 1) as array:
+ array[0] = -42
+ x = array[0]
+ assert x == -42
+
+ res = interpret(f, [])
Modified: pypy/branch/fast-forward/pypy/rpython/test/test_nongc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/test/test_nongc.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/test/test_nongc.py Fri Oct 22 23:09:43 2010
@@ -79,7 +79,7 @@
py.test.raises(TypeError,rtyper.specialize) # results in an invalid cast
def test_isinstance():
- class A:
+ class A(object):
_alloc_flavor_ = "raw"
class B(A):
pass
@@ -95,7 +95,24 @@
o = B()
else:
o = C()
- return 100*isinstance(o, A)+10*isinstance(o, B)+1*isinstance(o ,C)
+ res = 100*isinstance(o, A) + 10*isinstance(o, B) + 1*isinstance(o, C)
+ if i == 0:
+ pass
+ elif i == 1:
+ assert isinstance(o, A)
+ free_non_gc_object(o)
+ elif i == 2:
+ assert isinstance(o, B)
+ free_non_gc_object(o)
+ else:
+ assert isinstance(o, C)
+ free_non_gc_object(o)
+ return res
+
+ assert f(1) == 100
+ assert f(2) == 110
+ assert f(3) == 111
+ assert f(0) == 0
a = RPythonAnnotator()
#does not raise:
@@ -131,10 +148,14 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ free_non_gc_object(c)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
@@ -169,10 +190,13 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
Modified: pypy/branch/fast-forward/pypy/rpython/test/test_rbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/test/test_rbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/test/test_rbuilder.py Fri Oct 22 23:09:43 2010
@@ -1,4 +1,4 @@
-
+import py
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem.rbuilder import *
from pypy.rpython.annlowlevel import llstr, hlstr
@@ -55,8 +55,29 @@
assert res == 'aabcabcdefbuuuu'
assert isinstance(res, unicode)
+ def test_string_getlength(self):
+ def func():
+ s = StringBuilder()
+ s.append("a")
+ s.append("abc")
+ return s.getlength()
+ res = self.interpret(func, [])
+ assert res == 4
+
+ def test_unicode_getlength(self):
+ def func():
+ s = UnicodeBuilder()
+ s.append(u"a")
+ s.append(u"abc")
+ return s.getlength()
+ res = self.interpret(func, [])
+ assert res == 4
+
class TestLLtype(BaseTestStringBuilder, LLRtypeMixin):
pass
class TestOOtype(BaseTestStringBuilder, OORtypeMixin):
- pass
+ def test_string_getlength(self):
+ py.test.skip("getlength(): not implemented on ootype")
+ def test_unicode_getlength(self):
+ py.test.skip("getlength(): not implemented on ootype")
Modified: pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py Fri Oct 22 23:09:43 2010
@@ -3,7 +3,7 @@
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.ootypesystem import ootype
-from pypy.rlib.rarithmetic import intmask
+from pypy.rlib.rarithmetic import intmask, r_longlong
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.objspace.flow.model import summary
@@ -319,6 +319,17 @@
res = self.interpret(f, [])
assert res == 42
+ def test_staticmethod2(self):
+ class A(object):
+ f = staticmethod(lambda x, y: x*y)
+ class B(A):
+ f = staticmethod(lambda x, y: x+y)
+ def f():
+ b = B()
+ return b.f(6, 7)
+ res = self.interpret(f, [])
+ assert res == 13
+
def test_is(self):
class A: pass
class B(A): pass
@@ -1001,6 +1012,40 @@
res = self.interpret(f, [5])
assert res == 0
+ def test_order_of_fields(self):
+ class A(object):
+ pass
+ def f(n):
+ a = A()
+ a.as_int = n
+ a.as_char = chr(n)
+ a.as_unichar = unichr(n)
+ a.as_double = n + 0.5
+ a.as_bool = bool(n)
+ a.as_void = None
+ a.as_longlong = r_longlong(n)
+ a.as_reference = A()
+ return a
+
+ res = self.interpret(f, [5])
+ names = list(typeOf(res).TO._names)
+ i = names.index('inst_as_int')
+ c = names.index('inst_as_char')
+ u = names.index('inst_as_unichar')
+ d = names.index('inst_as_double')
+ b = names.index('inst_as_bool')
+ v = names.index('inst_as_void')
+ l = names.index('inst_as_longlong')
+ r = names.index('inst_as_reference')
+ assert v == 1 # void fields are first
+ assert sorted([c, b]) == [7, 8]
+ if sys.maxint == 2147483647:
+ assert sorted([u, i, r]) == [4, 5, 6] # 32-bit types
+ assert sorted([d, l]) == [2, 3] # 64-bit types
+ else:
+ assert sorted([u]) == [6] # 32-bit types
+ assert sorted([i, r, d, l]) == [2, 3, 4, 5] # 64-bit types
+
class TestOOtype(BaseTestRclass, OORtypeMixin):
Modified: pypy/branch/fast-forward/pypy/rpython/test/test_rptr.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/test/test_rptr.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/test/test_rptr.py Fri Oct 22 23:09:43 2010
@@ -212,10 +212,31 @@
S = Struct('S', ('x', Signed))
def fn(n):
- p = malloc(S, flavor='whatever')
+ p = malloc(S, flavor='raw')
p.x = n
result = p.x
- free(p, flavor='whatever')
+ free(p, flavor='raw')
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ free(p, flavor='raw', track_allocation=False)
return n
res = interpret(fn, [23])
Modified: pypy/branch/fast-forward/pypy/translator/c/funcgen.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/funcgen.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/funcgen.py Fri Oct 22 23:09:43 2010
@@ -427,7 +427,7 @@
r = self.expr(op.result)
return 'OP_CALL_ARGS((%s), %s);' % (', '.join(args), r)
- def generic_call(self, FUNC, fnexpr, args_v, v_result):
+ def generic_call(self, FUNC, fnexpr, args_v, v_result, targets=None):
args = []
assert len(args_v) == len(FUNC.TO.ARGS)
for v, ARGTYPE in zip(args_v, FUNC.TO.ARGS):
@@ -444,17 +444,26 @@
# skip assignment of 'void' return value
r = self.expr(v_result)
line = '%s = %s' % (r, line)
+ if targets:
+ for graph in targets:
+ if getattr(graph, 'inhibit_tail_call', False):
+ line += '\nPYPY_INHIBIT_TAIL_CALL();'
+ break
return line
def OP_DIRECT_CALL(self, op):
fn = op.args[0]
+ try:
+ targets = [fn.value._obj.graph]
+ except AttributeError:
+ targets = None
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:], op.result)
+ op.args[1:], op.result, targets)
def OP_INDIRECT_CALL(self, op):
fn = op.args[0]
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:-1], op.result)
+ op.args[1:-1], op.result, op.args[-1].value)
def OP_ADR_CALL(self, op):
ARGTYPES = [v.concretetype for v in op.args[1:]]
Modified: pypy/branch/fast-forward/pypy/translator/c/gcc/instruction.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/gcc/instruction.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/gcc/instruction.py Fri Oct 22 23:09:43 2010
@@ -82,6 +82,11 @@
def all_sources_of(self, localvar):
return [localvar]
+class InsnCondJump(Insn): # only for debugging; not used internally
+ _args_ = ['label']
+ def __init__(self, label):
+ self.label = label
+
class Label(Insn):
_args_ = ['label', 'lineno']
def __init__(self, label, lineno):
@@ -170,9 +175,12 @@
self.delta = -7 # use an odd value as marker
class InsnStop(Insn):
- pass
+ _args_ = ['reason']
+ def __init__(self, reason='?'):
+ self.reason = reason
class InsnRet(InsnStop):
+ _args_ = []
framesize = 0
def __init__(self, registers):
self.registers = registers
Modified: pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track5.s
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track5.s (original)
+++ pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track5.s Fri Oct 22 23:09:43 2010
@@ -44,7 +44,7 @@
addl %eax, %ebx
jmp .L1221
.L1227:
- call RPyAbort
+ ;;call RPyAbort
cmpl 12(%esi), %ebx
jb .L1229
addl $20, %esp
Modified: pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py Fri Oct 22 23:09:43 2010
@@ -6,7 +6,7 @@
from pypy.translator.c.gcc.instruction import InsnFunctionStart, InsnStop
from pypy.translator.c.gcc.instruction import InsnSetLocal, InsnCopyLocal
from pypy.translator.c.gcc.instruction import InsnPrologue, InsnEpilogue
-from pypy.translator.c.gcc.instruction import InsnGCROOT
+from pypy.translator.c.gcc.instruction import InsnGCROOT, InsnCondJump
from pypy.translator.c.gcc.instruction import InsnStackAdjust
from pypy.translator.c.gcc.instruction import InsnCannotFollowEsp
from pypy.translator.c.gcc.instruction import LocalVar, somenewvalue
@@ -46,6 +46,7 @@
self.findlabels()
self.parse_instructions()
try:
+ self.trim_unreachable_instructions()
self.find_noncollecting_calls()
if not self.list_collecting_call_insns():
return []
@@ -122,19 +123,36 @@
assert label not in self.labels, "duplicate label: %s" % label
self.labels[label] = Label(label, lineno)
+ def trim_unreachable_instructions(self):
+ reached = set([self.insns[0]])
+ prevlen = 0
+ while len(reached) > prevlen:
+ prevlen = len(reached)
+ for insn in self.insns:
+ if insn not in reached:
+ for previnsn in insn.previous_insns:
+ if previnsn in reached:
+ # this instruction is reachable too
+ reached.add(insn)
+ break
+ # now kill all unreachable instructions
+ i = 0
+ while i < len(self.insns):
+ if self.insns[i] in reached:
+ i += 1
+ else:
+ del self.insns[i]
+
def find_noncollecting_calls(self):
- cannot_collect = self.CANNOT_COLLECT.copy()
+ cannot_collect = {}
for line in self.lines:
match = self.r_gcnocollect_marker.search(line)
if match:
name = match.group(1)
cannot_collect[name] = True
#
- if self.format in ('darwin', 'mingw32', 'msvc'):
- self.cannot_collect = dict.fromkeys(
- ['_' + name for name in cannot_collect])
- else:
- self.cannot_collect = cannot_collect
+ self.cannot_collect = dict.fromkeys(
+ [self.function_names_prefix + name for name in cannot_collect])
def append_instruction(self, insn):
# Add the instruction to the list, and link it to the previous one.
@@ -410,7 +428,8 @@
return result
# ____________________________________________________________
- CANNOT_COLLECT = { # some of the most used functions that cannot collect
+ BASE_FUNCTIONS_NOT_RETURNING = {
+ 'abort': None,
'pypy_debug_catch_fatal_exception': None,
'RPyAbort': None,
'RPyAssertFailed': None,
@@ -644,7 +663,7 @@
if label != '0':
self.register_jump_to(label)
tablelin += 1
- return InsnStop()
+ return InsnStop("jump table")
if self.r_unaryinsn_star.match(line):
# that looks like an indirect tail-call.
# tail-calls are equivalent to RET for us
@@ -658,7 +677,7 @@
assert not target.startswith('.')
# tail-calls are equivalent to RET for us
return InsnRet(self.CALLEE_SAVE_REGISTERS)
- return InsnStop()
+ return InsnStop("jump")
def register_jump_to(self, label):
if not isinstance(self.insns[-1], InsnStop):
@@ -682,7 +701,7 @@
else:
label = match.group(1)
self.register_jump_to(label)
- return []
+ return [InsnCondJump(label)]
visit_jmpl = visit_jmp
visit_je = conditional_jump
@@ -754,7 +773,7 @@
target, = sources
if target in self.FUNCTIONS_NOT_RETURNING:
- return [InsnStop(), InsnCannotFollowEsp()]
+ return [InsnStop(target)]
if self.format == 'mingw32' and target == '__alloca':
# in functions with large stack requirements, windows
# needs a call to _alloca(), to turn reserved pages
@@ -885,7 +904,7 @@
# statically known pointer to a register
# %eax -> %rax
- new_line = re.sub(r"%e(ax|bx|cx|dx|di|si)$", r"%r\1", line)
+ new_line = re.sub(r"%e(ax|bx|cx|dx|di|si|bp)$", r"%r\1", line)
# %r10d -> %r10
new_line = re.sub(r"%r(\d+)d$", r"%r\1", new_line)
return func(self, new_line)
@@ -951,6 +970,7 @@
class ElfFunctionGcRootTracker32(FunctionGcRootTracker32):
format = 'elf'
+ function_names_prefix = ''
ESP = '%esp'
EBP = '%ebp'
@@ -984,13 +1004,14 @@
r_bottom_marker = re.compile(r"\t/[*] GC_STACK_BOTTOM [*]/")
FUNCTIONS_NOT_RETURNING = {
- 'abort': None,
'_exit': None,
'__assert_fail': None,
'___assert_rtn': None,
'L___assert_rtn$stub': None,
'L___eprintf$stub': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING[_name] = None
def __init__(self, lines, filetag=0):
match = self.r_functionstart.match(lines[0])
@@ -1010,6 +1031,8 @@
class ElfFunctionGcRootTracker64(FunctionGcRootTracker64):
format = 'elf64'
+ function_names_prefix = ''
+
ESP = '%rsp'
EBP = '%rbp'
EAX = '%rax'
@@ -1042,13 +1065,14 @@
r_bottom_marker = re.compile(r"\t/[*] GC_STACK_BOTTOM [*]/")
FUNCTIONS_NOT_RETURNING = {
- 'abort': None,
'_exit': None,
'__assert_fail': None,
'___assert_rtn': None,
'L___assert_rtn$stub': None,
'L___eprintf$stub': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING[_name] = None
def __init__(self, lines, filetag=0):
match = self.r_functionstart.match(lines[0])
@@ -1068,6 +1092,7 @@
class DarwinFunctionGcRootTracker(ElfFunctionGcRootTracker32):
format = 'darwin'
+ function_names_prefix = '_'
r_functionstart = re.compile(r"_(\w+):\s*$")
OFFSET_LABELS = 0
@@ -1079,15 +1104,19 @@
class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker):
format = 'mingw32'
+ function_names_prefix = '_'
FUNCTIONS_NOT_RETURNING = {
- '_abort': None,
'_exit': None,
'__assert': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING['_' + _name] = None
class MsvcFunctionGcRootTracker(FunctionGcRootTracker32):
format = 'msvc'
+ function_names_prefix = '_'
+
ESP = 'esp'
EBP = 'ebp'
EAX = 'eax'
@@ -1127,7 +1156,6 @@
r_bottom_marker = re.compile(r"; .+\tpypy_asm_stack_bottom\(\);")
FUNCTIONS_NOT_RETURNING = {
- '_abort': None,
'__exit': None,
'__assert': None,
'__wassert': None,
@@ -1136,6 +1164,8 @@
'DWORD PTR __imp__abort': None,
'DWORD PTR __imp___wassert': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING['_' + _name] = None
@classmethod
def init_regexp(cls):
@@ -1543,9 +1573,7 @@
assert self.seen_main
def _globalname(name, disp=""):
- if self.format in ('darwin', 'mingw32', 'msvc'):
- name = '_' + name
- return name
+ return tracker_cls.function_names_prefix + name
def _variant(**kwargs):
txt = kwargs[self.format]
Modified: pypy/branch/fast-forward/pypy/translator/c/src/g_include.h
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/src/g_include.h (original)
+++ pypy/branch/fast-forward/pypy/translator/c/src/g_include.h Fri Oct 22 23:09:43 2010
@@ -53,6 +53,7 @@
# include "src/rtyper.h"
# include "src/debug_print.h"
# include "src/debug_traceback.h"
+# include "src/debug_alloc.h"
#ifndef AVR
# include "src/ll_os.h"
# include "src/ll_strtod.h"
Modified: pypy/branch/fast-forward/pypy/translator/c/src/main.h
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/src/main.h (original)
+++ pypy/branch/fast-forward/pypy/translator/c/src/main.h Fri Oct 22 23:09:43 2010
@@ -53,10 +53,16 @@
}
exitcode = STANDALONE_ENTRY_POINT(list);
+
+#ifdef RPY_ASSERT
+ pypy_debug_alloc_results();
+#endif
+
if (RPyExceptionOccurred()) {
/* print the RPython traceback */
pypy_debug_catch_fatal_exception();
}
+
return exitcode;
memory_out:
Modified: pypy/branch/fast-forward/pypy/translator/c/src/signals.h
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/src/signals.h (original)
+++ pypy/branch/fast-forward/pypy/translator/c/src/signals.h Fri Oct 22 23:09:43 2010
@@ -6,20 +6,6 @@
#include <limits.h>
-#ifndef LONG_MAX
-#if SIZEOF_LONG == 4
-#define LONG_MAX 0X7FFFFFFFL
-#elif SIZEOF_LONG == 8
-#define LONG_MAX 0X7FFFFFFFFFFFFFFFL
-#else
-#error "could not set LONG_MAX in pyport.h"
-#endif
-#endif
-
-#ifndef LONG_MIN
-#define LONG_MIN (-LONG_MAX-1)
-#endif
-
#include <stdlib.h>
#ifdef MS_WINDOWS
@@ -28,10 +14,6 @@
#include <signal.h>
-#ifndef SIG_ERR
-#define SIG_ERR ((PyOS_sighandler_t)(-1))
-#endif
-
#if defined(PYOS_OS2) && !defined(PYCC_GCC)
#define NSIG 12
#include <process.h>
@@ -65,11 +47,11 @@
/* utility to poll for signals that arrived */
int pypysig_poll(void); /* => signum or -1 */
-/* When a signal is received, the high bit of pypysig_occurred is set.
- After all signals are processed by pypysig_poll(), the high bit is
+/* When a signal is received, the bit 30 of pypysig_occurred is set.
+ After all signals are processed by pypysig_poll(), the bit 30 is
cleared again. The variable is exposed and RPython code is free to
use the other bits in any way. */
-#define PENDING_SIGNAL_BIT (LONG_MIN) /* high bit */
+#define PENDING_SIGNAL_BIT (1 << 30)
/* This is a struct for the JIT. See interp_signal.py. */
struct pypysig_long_struct {
long value;
Modified: pypy/branch/fast-forward/pypy/translator/c/src/stack.h
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/src/stack.h (original)
+++ pypy/branch/fast-forward/pypy/translator/c/src/stack.h Fri Oct 22 23:09:43 2010
@@ -33,6 +33,12 @@
&& LL_stack_too_big_slowpath());
}
+#ifdef __GNUC__
+# define PYPY_INHIBIT_TAIL_CALL() asm("/* inhibit_tail_call */")
+#else
+# define PYPY_INHIBIT_TAIL_CALL() /* add hints for other compilers here */
+#endif
+
#ifndef PYPY_NOT_MAIN_FILE
#include <stdio.h>
Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_genc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/test/test_genc.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/test/test_genc.py Fri Oct 22 23:09:43 2010
@@ -426,6 +426,7 @@
if py.test.config.option.view:
t.view()
assert ' BarStruct ' in t.driver.cbuilder.c_source_filename.read()
+ free(foo, flavor="raw")
def test_recursive_llhelper():
from pypy.rpython.annlowlevel import llhelper
@@ -473,7 +474,27 @@
return f(s)
a_f = A(f, "f")
a_g = A(g, "g")
- t = lltype.malloc(STRUCT, flavor="raw")
+ t = lltype.malloc(STRUCT, flavor="raw", immortal=True)
t.bar = llhelper(FTPTR, a_f.make_func())
fn = compile(chooser, [bool])
assert fn(True)
+
+def test_inhibit_tail_call():
+ from pypy.rpython.lltypesystem import lltype
+ def foobar_fn(n):
+ return 42
+ foobar_fn._dont_inline_ = True
+ def main(n):
+ return foobar_fn(n)
+ #
+ t = Translation(main, [int], backend="c")
+ t.rtype()
+ t.context._graphof(foobar_fn).inhibit_tail_call = True
+ t.source_c()
+ lines = t.driver.cbuilder.c_source_filename.readlines()
+ for i, line in enumerate(lines):
+ if '= pypy_g_foobar_fn' in line:
+ break
+ else:
+ assert 0, "the call was not found in the C source"
+ assert 'PYPY_INHIBIT_TAIL_CALL();' in lines[i+1]
Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py Fri Oct 22 23:09:43 2010
@@ -401,6 +401,7 @@
for i in range(n):
p = malloc(S, flavor='raw', zero=True)
if p.x != 0 or p.y != 0:
+ free(p, flavor='raw')
return -1
p.x = i
p.y = i
@@ -418,14 +419,16 @@
def f(n):
for length in range(n-1, -1, -1):
p = malloc(S, length, flavor='raw', zero=True)
- if p.x != 0:
- return -1
- p.x = n
- for j in range(length):
- if p.y[j] != 0:
- return -3
- p.y[j] = n^j
- free(p, flavor='raw')
+ try:
+ if p.x != 0:
+ return -1
+ p.x = n
+ for j in range(length):
+ if p.y[j] != 0:
+ return -3
+ p.y[j] = n^j
+ finally:
+ free(p, flavor='raw')
return 42
fn = self.getcompiled(f, [int])
@@ -655,7 +658,7 @@
def test_prebuilt_ll2ctypes_array(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
@@ -676,7 +679,7 @@
def test_ll2ctypes_array_from_c(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py Fri Oct 22 23:09:43 2010
@@ -624,13 +624,13 @@
os.unlink(self.filename)
def define_callback_with_collect(cls):
- from pypy.rlib.libffi import ffi_type_pointer, cast_type_to_ffitype,\
+ from pypy.rlib.clibffi import ffi_type_pointer, cast_type_to_ffitype,\
CDLL, ffi_type_void, CallbackFuncPtr, ffi_type_sint
from pypy.rpython.lltypesystem import rffi, ll2ctypes
import gc
ffi_size_t = cast_type_to_ffitype(rffi.SIZE_T)
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
def callback(ll_args, ll_res, stuff):
gc.collect()
Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py Fri Oct 22 23:09:43 2010
@@ -16,11 +16,16 @@
class StandaloneTests(object):
config = None
- def compile(self, entry_point, debug=True, shared=False):
+ def compile(self, entry_point, debug=True, shared=False,
+ stackcheck=False):
t = TranslationContext(self.config)
t.buildannotator().build_types(entry_point, [s_list_of_strings])
t.buildrtyper().specialize()
+ if stackcheck:
+ from pypy.translator.transform import insert_ll_stackcheck
+ insert_ll_stackcheck(t)
+
t.config.translation.shared = shared
cbuilder = CStandaloneBuilder(t, entry_point, t.config)
@@ -630,6 +635,22 @@
else:
os.environ['CC'] = old_cc
+ def test_inhibit_tail_call(self):
+ # the point is to check that the f()->f() recursion stops
+ from pypy.rlib.rstackovf import StackOverflow
+ def f(n):
+ if n <= 0:
+ return 42
+ return f(n+1)
+ def entry_point(argv):
+ try:
+ return f(1)
+ except StackOverflow:
+ print 'hi!'
+ return 0
+ t, cbuilder = self.compile(entry_point, stackcheck=True)
+ out = cbuilder.cmdexec("")
+ assert out.strip() == "hi!"
class TestMaemo(TestStandalone):
def setup_class(cls):
Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_typed.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/test/test_typed.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/test/test_typed.py Fri Oct 22 23:09:43 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import autopath
import sys
import math
@@ -823,3 +824,41 @@
while int(x + frac) >= -sys.maxint-1:
x -= 1
assert f(x + frac) == -666
+
+ def test_context_manager(self):
+ state = []
+ class C:
+ def __init__(self, name):
+ self.name = name
+ def __enter__(self):
+ state.append('acquire')
+ return self
+ def __exit__(self, typ, value, tb):
+ if typ is not None:
+ if value is None:
+ raise RuntimeError('test failed')
+ state.append('raised')
+ else:
+ if value is not None:
+ raise RuntimeError('test failed')
+ state.append('release')
+
+ def func(n):
+ del state[:]
+ try:
+ with C('hello') as c:
+ state.append(c.name)
+ if n == 1:
+ raise ValueError
+ elif n == 2:
+ raise TypeError
+ except (ValueError, TypeError):
+ pass
+ return ', '.join(state)
+ f = self.getcompiled(func, [int])
+ res = f(0)
+ assert res == 'acquire, hello, release'
+ res = f(1)
+ assert res == 'acquire, hello, raised, release'
+ res = f(2)
+ assert res == 'acquire, hello, raised, release'
Modified: pypy/branch/fast-forward/pypy/translator/goal/ann_override.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/goal/ann_override.py (original)
+++ pypy/branch/fast-forward/pypy/translator/goal/ann_override.py Fri Oct 22 23:09:43 2010
@@ -27,21 +27,6 @@
pol.pypytypes = {}
pol.single_space = single_space
- #def override__wrap_exception_cls(pol, space, x):
- # import pypy.objspace.std.typeobject as typeobject
- # clsdef = getbookkeeper().getuniqueclassdef(typeobject.W_TypeObject)
- # return annmodel.SomeInstance(clsdef, can_be_None=True)
- #
- #def override__fake_object(pol, space, x):
- # from pypy.interpreter import typedef
- # clsdef = getbookkeeper().getuniqueclassdef(typedef.W_Root)
- # return annmodel.SomeInstance(clsdef)
- #
- #def override__cpy_compile(pol, self, source, filename, mode, flags):
- # from pypy.interpreter import pycode
- # clsdef = getbookkeeper().getuniqueclassdef(pycode.PyCode)
- # return annmodel.SomeInstance(clsdef)
-
def specialize__wrap(pol, funcdesc, args_s):
from pypy.interpreter.baseobjspace import Wrappable
from pypy.annotation.classdef import ClassDef
Modified: pypy/branch/fast-forward/pypy/translator/goal/app_main.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/goal/app_main.py (original)
+++ pypy/branch/fast-forward/pypy/translator/goal/app_main.py Fri Oct 22 23:09:43 2010
@@ -306,7 +306,7 @@
break
elif arg == '-u':
options["unbuffered"] = True
- elif arg == '-O':
+ elif arg == '-O' or arg == '-OO':
pass
elif arg == '--version' or arg == '-V':
print "Python", sys.version
@@ -451,9 +451,12 @@
python_startup = os.getenv('PYTHONSTARTUP')
if python_startup:
try:
- startup = open(python_startup).read()
- except IOError:
- pass
+ f = open(python_startup)
+ startup = f.read()
+ f.close()
+ except IOError, e:
+ print >> sys.stderr, "Could not open PYTHONSTARTUP"
+ print >> sys.stderr, "IOError:", e
else:
def run_it():
co_python_startup = compile(startup,
Modified: pypy/branch/fast-forward/pypy/translator/jvm/test/test_class.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/jvm/test/test_class.py (original)
+++ pypy/branch/fast-forward/pypy/translator/jvm/test/test_class.py Fri Oct 22 23:09:43 2010
@@ -1,6 +1,6 @@
import py
from pypy.translator.jvm.test.runtest import JvmTest
-from pypy.translator.oosupport.test_template.class_ import BaseTestClass, BaseTestSpecialcase
+from pypy.translator.oosupport.test_template.class_ import BaseTestClass
class TestJvmClass(JvmTest, BaseTestClass):
def test_overridden_classattr_as_defaults(self):
@@ -26,6 +26,3 @@
def test_specialize_methods(self):
py.test.skip('ABSTRACT METHOD FIX: RE-TEST AFTER MERGE')
-
-class TestJvmSpecialCase(JvmTest, BaseTestSpecialcase):
- pass
Modified: pypy/branch/fast-forward/pypy/translator/oosupport/test_template/class_.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/oosupport/test_template/class_.py (original)
+++ pypy/branch/fast-forward/pypy/translator/oosupport/test_template/class_.py Fri Oct 22 23:09:43 2010
@@ -1,6 +1,5 @@
import py
from pypy.rpython.test import test_rclass
-from pypy.rpython.test.test_rspecialcase import BaseTestRspecialcase
class BaseTestClass(test_rclass.TestOOtype):
def test_abstract_method(self):
@@ -66,6 +65,3 @@
def test_cast_object_mix_null(self):
py.test.skip('cannot return ootype.NULL from translated functions')
-
-class BaseTestSpecialcase(BaseTestRspecialcase):
- pass
Modified: pypy/branch/fast-forward/pypy/translator/platform/darwin.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/platform/darwin.py (original)
+++ pypy/branch/fast-forward/pypy/translator/platform/darwin.py Fri Oct 22 23:09:43 2010
@@ -14,7 +14,10 @@
def __init__(self, cc=None):
if cc is None:
- cc = 'gcc'
+ try:
+ cc = os.environ['CC']
+ except KeyError:
+ cc = 'gcc'
self.cc = cc
def _args_for_shared(self, args):
Modified: pypy/branch/fast-forward/pypy/translator/transform.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/transform.py (original)
+++ pypy/branch/fast-forward/pypy/translator/transform.py Fri Oct 22 23:09:43 2010
@@ -221,15 +221,19 @@
stack_check_ptr_const = Constant(stack_check_ptr, lltype.typeOf(stack_check_ptr))
edges = set()
insert_in = set()
+ block2graph = {}
for caller in translator.graphs:
for block, callee in find_calls_from(translator, caller):
if getattr(getattr(callee, 'func', None),
'insert_stack_check_here', False):
insert_in.add(callee.startblock)
+ block2graph[callee.startblock] = callee
continue
if block is not caller.startblock:
edges.add((caller.startblock, block))
+ block2graph[caller.startblock] = caller
edges.add((block, callee.startblock))
+ block2graph[block] = caller
edgelist = [Edge(block1, block2) for (block1, block2) in edges]
edgedict = make_edge_dict(edgelist)
@@ -241,6 +245,10 @@
v.concretetype = lltype.Void
unwind_op = SpaceOperation('direct_call', [stack_check_ptr_const], v)
block.operations.insert(0, unwind_op)
+ # prevents cycles of tail calls from occurring -- such cycles would
+ # not consume any stack, so would turn into potentially infinite loops
+ graph = block2graph[block]
+ graph.inhibit_tail_call = True
return len(insert_in)
More information about the Pypy-commit
mailing list