[pypy-commit] pypy default: merged upstream
alex_gaynor
noreply at buildbot.pypy.org
Sun Jun 19 03:54:37 CEST 2011
Author: Alex Gaynor <alex.gaynor at gmail.com>
Branch:
Changeset: r44997:da2244d0d7ff
Date: 2011-06-18 18:58 -0700
http://bitbucket.org/pypy/pypy/changeset/da2244d0d7ff/
Log: merged upstream
diff --git a/pypy/doc/garbage_collection.rst b/pypy/doc/garbage_collection.rst
--- a/pypy/doc/garbage_collection.rst
+++ b/pypy/doc/garbage_collection.rst
@@ -212,90 +212,4 @@
becomes free garbage, to be collected at the next major collection.
-Minimark GC
------------
-
-This is a simplification and rewrite of the ideas from the Hybrid GC.
-It uses a nursery for the young objects, and mark-and-sweep for the old
-objects. This is a moving GC, but objects may only move once (from
-the nursery to the old stage).
-
-The main difference with the Hybrid GC is that the mark-and-sweep
-objects (the "old stage") are directly handled by the GC's custom
-allocator, instead of being handled by malloc() calls. The gain is that
-it is then possible, during a major collection, to walk through all old
-generation objects without needing to store a list of pointers to them.
-So as a first approximation, when compared to the Hybrid GC, the
-Minimark GC saves one word of memory per old object.
-
-There are a number of environment variables that can be tweaked to
-influence the GC. (Their default value should be ok for most usages.)
-You can read more about them at the start of
-`pypy/rpython/memory/gc/minimark.py`_.
-
-In more details:
-
-- The small newly malloced objects are allocated in the nursery (case 1).
- All objects living in the nursery are "young".
-
-- The big objects are always handled directly by the system malloc().
- But the big newly malloced objects are still "young" when they are
- allocated (case 2), even though they don't live in the nursery.
-
-- When the nursery is full, we do a minor collection, i.e. we find
- which "young" objects are still alive (from cases 1 and 2). The
- "young" flag is then removed. The surviving case 1 objects are moved
- to the old stage. The dying case 2 objects are immediately freed.
-
-- The old stage is an area of memory containing old (small) objects. It
- is handled by `pypy/rpython/memory/gc/minimarkpage.py`_. It is organized
- as "arenas" of 256KB or 512KB, subdivided into "pages" of 4KB or 8KB.
- Each page can either be free, or contain small objects of all the same
- size. Furthermore at any point in time each object location can be
- either allocated or freed. The basic design comes from ``obmalloc.c``
- from CPython (which itself comes from the same source as the Linux
- system malloc()).
-
-- New objects are added to the old stage at every minor collection.
- Immediately after a minor collection, when we reach some threshold, we
- trigger a major collection. This is the mark-and-sweep step. It walks
- over *all* objects (mark), and then frees some fraction of them (sweep).
- This means that the only time when we want to free objects is while
- walking over all of them; we never ask to free an object given just its
- address. This allows some simplifications and memory savings when
- compared to ``obmalloc.c``.
-
-- As with all generational collectors, this GC needs a write barrier to
- record which old objects have a reference to young objects.
-
-- Additionally, we found out that it is useful to handle the case of
- big arrays specially: when we allocate a big array (with the system
- malloc()), we reserve a small number of bytes before. When the array
- grows old, we use the extra bytes as a set of bits. Each bit
- represents 128 entries in the array. Whenever the write barrier is
- called to record a reference from the Nth entry of the array to some
- young object, we set the bit number ``(N/128)`` to 1. This can
- considerably speed up minor collections, because we then only have to
- scan 128 entries of the array instead of all of them.
-
-- As usual, we need special care about weak references, and objects with
- finalizers. Weak references are allocated in the nursery, and if they
- survive they move to the old stage, as usual for all objects; the
- difference is that the reference they contain must either follow the
- object, or be set to NULL if the object dies. And the objects with
- finalizers, considered rare enough, are immediately allocated old to
- simplify the design. In particular their ``__del__`` method can only
- be called just after a major collection.
-
-- The objects move once only, so we can use a trick to implement id()
- and hash(). If the object is not in the nursery, it won't move any
- more, so its id() and hash() are the object's address, cast to an
- integer. If the object is in the nursery, and we ask for its id()
- or its hash(), then we pre-reserve a location in the old stage, and
- return the address of that location. If the object survives the
- next minor collection, we move it there, and so its id() and hash()
- are preserved. If the object dies then the pre-reserved location
- becomes free garbage, to be collected at the next major collection.
-
-
.. include:: _ref.txt
diff --git a/pypy/tool/jitlogparser/parser.py b/pypy/tool/jitlogparser/parser.py
--- a/pypy/tool/jitlogparser/parser.py
+++ b/pypy/tool/jitlogparser/parser.py
@@ -98,7 +98,7 @@
def __init__(self, operations, storage):
if operations[0].name == 'debug_merge_point':
self.inline_level = int(operations[0].args[0])
- m = re.search('<code object ([<>\w]+), file \'(.+?)\', line (\d+)> #(\d+) (\w+)',
+ m = re.search('<code object ([<>\w]+)\. file \'(.+?)\'\. line (\d+)> #(\d+) (\w+)',
operations[0].getarg(1))
if m is None:
# a non-code loop, like StrLiteralSearch or something
diff --git a/pypy/tool/jitlogparser/test/test_parser.py b/pypy/tool/jitlogparser/test/test_parser.py
--- a/pypy/tool/jitlogparser/test/test_parser.py
+++ b/pypy/tool/jitlogparser/test/test_parser.py
@@ -38,10 +38,10 @@
def test_split():
ops = parse('''
[i0]
- debug_merge_point(0, "<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD")
- debug_merge_point(0, "<code object stuff, file '/I/dont/exist.py', line 200> #11 SUB")
+ debug_merge_point(0, "<code object stuff. file '/I/dont/exist.py'. line 200> #10 ADD")
+ debug_merge_point(0, "<code object stuff. file '/I/dont/exist.py'. line 200> #11 SUB")
i1 = int_add(i0, 1)
- debug_merge_point(0, "<code object stuff, file '/I/dont/exist.py', line 200> #11 SUB")
+ debug_merge_point(0, "<code object stuff. file '/I/dont/exist.py'. line 200> #11 SUB")
i2 = int_add(i1, 1)
''')
res = Function.from_operations(ops.operations, LoopStorage())
@@ -54,12 +54,12 @@
def test_inlined_call():
ops = parse("""
[]
- debug_merge_point(0, '<code object inlined_call, file 'source.py', line 12> #28 CALL_FUNCTION')
+ debug_merge_point(0, '<code object inlined_call. file 'source.py'. line 12> #28 CALL_FUNCTION')
i18 = getfield_gc(p0, descr=<BoolFieldDescr pypy.interpreter.pyframe.PyFrame.inst_is_being_profiled 89>)
- debug_merge_point(1, '<code object inner, file 'source.py', line 9> #0 LOAD_FAST')
- debug_merge_point(1, '<code object inner, file 'source.py', line 9> #3 LOAD_CONST')
- debug_merge_point(1, '<code object inner, file 'source.py', line 9> #7 RETURN_VALUE')
- debug_merge_point(0, '<code object inlined_call, file 'source.py', line 12> #31 STORE_FAST')
+ debug_merge_point(1, '<code object inner. file 'source.py'. line 9> #0 LOAD_FAST')
+ debug_merge_point(1, '<code object inner. file 'source.py'. line 9> #3 LOAD_CONST')
+ debug_merge_point(1, '<code object inner. file 'source.py'. line 9> #7 RETURN_VALUE')
+ debug_merge_point(0, '<code object inlined_call. file 'source.py'. line 12> #31 STORE_FAST')
""")
res = Function.from_operations(ops.operations, LoopStorage())
assert len(res.chunks) == 3 # two chunks + inlined call
@@ -72,10 +72,10 @@
def test_name():
ops = parse('''
[i0]
- debug_merge_point(0, "<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD")
- debug_merge_point(0, "<code object stuff, file '/I/dont/exist.py', line 201> #11 SUB")
+ debug_merge_point(0, "<code object stuff. file '/I/dont/exist.py'. line 200> #10 ADD")
+ debug_merge_point(0, "<code object stuff. file '/I/dont/exist.py'. line 201> #11 SUB")
i1 = int_add(i0, 1)
- debug_merge_point(0, "<code object stuff, file '/I/dont/exist.py', line 202> #11 SUB")
+ debug_merge_point(0, "<code object stuff. file '/I/dont/exist.py'. line 202> #11 SUB")
i2 = int_add(i1, 1)
''')
res = Function.from_operations(ops.operations, LoopStorage())
@@ -89,10 +89,10 @@
ops = parse('''
[i0]
i3 = int_add(i0, 1)
- debug_merge_point(0, "<code object stuff, file '/I/dont/exist.py', line 200> #10 ADD")
- debug_merge_point(0, "<code object stuff, file '/I/dont/exist.py', line 201> #11 SUB")
+ debug_merge_point(0, "<code object stuff. file '/I/dont/exist.py'. line 200> #10 ADD")
+ debug_merge_point(0, "<code object stuff. file '/I/dont/exist.py'. line 201> #11 SUB")
i1 = int_add(i0, 1)
- debug_merge_point(0, "<code object stuff, file '/I/dont/exist.py', line 202> #11 SUB")
+ debug_merge_point(0, "<code object stuff. file '/I/dont/exist.py'. line 202> #11 SUB")
i2 = int_add(i1, 1)
''')
res = Function.from_operations(ops.operations, LoopStorage())
@@ -102,10 +102,10 @@
fname = str(py.path.local(__file__).join('..', 'x.py'))
ops = parse('''
[i0, i1]
- debug_merge_point(0, "<code object f, file '%(fname)s', line 2> #0 LOAD_FAST")
- debug_merge_point(0, "<code object f, file '%(fname)s', line 2> #3 LOAD_FAST")
- debug_merge_point(0, "<code object f, file '%(fname)s', line 2> #6 BINARY_ADD")
- debug_merge_point(0, "<code object f, file '%(fname)s', line 2> #7 RETURN_VALUE")
+ debug_merge_point(0, "<code object f. file '%(fname)s'. line 2> #0 LOAD_FAST")
+ debug_merge_point(0, "<code object f. file '%(fname)s'. line 2> #3 LOAD_FAST")
+ debug_merge_point(0, "<code object f. file '%(fname)s'. line 2> #6 BINARY_ADD")
+ debug_merge_point(0, "<code object f. file '%(fname)s'. line 2> #7 RETURN_VALUE")
''' % locals())
res = Function.from_operations(ops.operations, LoopStorage())
assert res.chunks[1].lineno == 3
@@ -114,11 +114,11 @@
fname = str(py.path.local(__file__).join('..', 'x.py'))
ops = parse('''
[i0, i1]
- debug_merge_point(0, "<code object g, file '%(fname)s', line 5> #9 LOAD_FAST")
- debug_merge_point(0, "<code object g, file '%(fname)s', line 5> #12 LOAD_CONST")
- debug_merge_point(0, "<code object g, file '%(fname)s', line 5> #22 LOAD_CONST")
- debug_merge_point(0, "<code object g, file '%(fname)s', line 5> #28 LOAD_CONST")
- debug_merge_point(0, "<code object g, file '%(fname)s', line 5> #6 SETUP_LOOP")
+ debug_merge_point(0, "<code object g. file '%(fname)s'. line 5> #9 LOAD_FAST")
+ debug_merge_point(0, "<code object g. file '%(fname)s'. line 5> #12 LOAD_CONST")
+ debug_merge_point(0, "<code object g. file '%(fname)s'. line 5> #22 LOAD_CONST")
+ debug_merge_point(0, "<code object g. file '%(fname)s'. line 5> #28 LOAD_CONST")
+ debug_merge_point(0, "<code object g. file '%(fname)s'. line 5> #6 SETUP_LOOP")
''' % locals())
res = Function.from_operations(ops.operations, LoopStorage())
assert res.linerange == (7, 9)
@@ -128,7 +128,7 @@
fname = str(py.path.local(__file__).join('..', 'x.py'))
ops = parse("""
[p6, p1]
- debug_merge_point(0, '<code object h, file '%(fname)s', line 11> #17 FOR_ITER')
+ debug_merge_point(0, '<code object h. file '%(fname)s'. line 11> #17 FOR_ITER')
guard_class(p6, 144264192, descr=<Guard2>)
p12 = getfield_gc(p6, descr=<GcPtrFieldDescr pypy.objspace.std.iterobject.W_AbstractSeqIterObject.inst_w_seq 12>)
""" % locals())
diff --git a/pypy/translator/c/genc.py b/pypy/translator/c/genc.py
--- a/pypy/translator/c/genc.py
+++ b/pypy/translator/c/genc.py
@@ -570,7 +570,10 @@
mk.definition('ASMFILES', sfiles)
mk.definition('ASMLBLFILES', lblsfiles)
mk.definition('GCMAPFILES', gcmapfiles)
- mk.definition('DEBUGFLAGS', '-O2 -fomit-frame-pointer -g')
+ if sys.platform == 'win32':
+ mk.definition('DEBUGFLAGS', '/Zi')
+ else:
+ mk.definition('DEBUGFLAGS', '-O2 -fomit-frame-pointer -g')
if self.config.translation.shared:
mk.definition('PYPY_MAIN_FUNCTION', "pypy_main_startup")
@@ -623,7 +626,10 @@
mk.rule('.PRECIOUS', '%.s', "# don't remove .s files if Ctrl-C'ed")
else:
- mk.definition('DEBUGFLAGS', '-O1 -g')
+ if sys.platform == 'win32':
+ mk.definition('DEBUGFLAGS', '/Zi')
+ else:
+ mk.definition('DEBUGFLAGS', '-O1 -g')
mk.write()
#self.translator.platform,
# ,
diff --git a/pypy/translator/c/src/main.h b/pypy/translator/c/src/main.h
--- a/pypy/translator/c/src/main.h
+++ b/pypy/translator/c/src/main.h
@@ -79,6 +79,7 @@
fprintf(stderr, "Fatal error during initialization: %s\n", errmsg);
#endif
abort();
+ return 1;
}
int PYPY_MAIN_FUNCTION(int argc, char *argv[])
More information about the pypy-commit
mailing list