Where are the strings in gc.get_objects?
Edward K Ream
edreamleo at charter.net
Fri May 4 15:31:22 EDT 2007
The following script dumps all objects allocated since the last time it was
called. It suppresses the dump if more than 200 new objects were allocated.
g.app.idDict is a dict whose keys are id(obj) and whose values are obj.
(g.app.idDict will persist between invocations of the script). This allows
us to determine whether an object has already been allocated. It's not a
perfect test,but perfection is not required.
As expected, this stresses the gc a lot. The isLargeItem function is
important: it returns True if repr(obj) would be too big. It tries to avoid
actually generating repr(obj). Even so, sometimes the script appears to hang
while isLargeItem cranks. I thought it would be clever to call isLargeItem
for dict values, but that overflowed the stack.
Anyway, here is the script:
import gc,types
def isLargeItem (obj):
....if (
........type(obj) == types.CodeType or
........type(obj) == types.ListType and len(obj) > 500 or
........type(obj) == types.TupleType and len(obj) > 500
....):
........return True
....if type(obj) == types.DictType:
........n = 0 ; keys = obj.keys()
........if len(keys) > 100: return True
........for key in keys:
............val = obj.get(key)
............# if isLargeItem(val): return True
............n += len(repr(key))
............n += len(repr(val))
............if n > 1000: return True
........return False
............
....return len(repr(obj)) > 1000
try: d = g.app.idDict
except AttributeError: d = g.app.idDict = {}
new = {}
for obj in gc.get_objects():
....oldObj = d.get(id(obj))
....if oldObj is None:
........new[id(obj)] = obj
........
keys = new.keys()
print '----- %d new objects' % len(keys)
if len(keys) < 200:
....keys.sort()
....n = 0
....for key in keys:
........n += 1
........obj = new.get(key)
........print '%3d' % (n),key,type(obj),
........if isLargeItem(obj):
............print '***** large item'
........else:
............print repr(obj)
....print '====='
for key in keys:
....d[key] = new.get(key)
And here is some example output:
----- 70129 new objects [first run of script: nothing printed]
----- 118 new objects [second run of script]
1 13723776 <type 'frame'> <frame object at 0x00D16880>
2 17053392 <type 'frame'> <frame object at 0x010436D0>
3 20219840 <type 'frame'> <frame object at 0x013487C0>
4 21927208 <type 'frame'> <frame object at 0x014E9528>
5 23088536 <type 'tuple'> ('5780', '??', '??', '??', '66', '4', '4522765',
'??', '115', '174', '\x02', '-1', 'b', '98', '.232818
72.23282192.23282432.23282472.23282592.23282672.24029064.24051224.canvas',
'2', '524', '412', '66')
6 23248088 <type 'tuple'> ('gc', 'types', 'isLargeItem', 'g', 'app',
'idDict', 'd', 'AttributeError', 'new', 'get_objects', 'obj
', 'get', 'id', 'oldObj', 'None', 'keys', 'len', 'sort', 'n', 'key', 'type',
'repr')
7 23293152 <type 'tuple'> (<_Pmw.Pmw_1_2.lib.PmwBase.__TkinterCallWrapper
instance at 0x0175AB20>, '5780', '??', '??', '??', '66
', '4', '4522765', '??', '115', '174', '\x02', '-1', 'b', '98',
'.23281872.23282192.23282432.23282472.23282592.23282672.24029064.2
4051224.canvas', '2', '524', '412', '66')
8 23562680 <type 'frame'> <frame object at 0x016789B8>
9 24069264 <type 'list'> []
10 24116944 <type 'dict'> {'stack': [], 'v': <vnode
27563632:'getNewObjects'>}
11 24117936 <type 'list'> ['black', 'blue']
12 24118016 <type 'instancemethod'> <bound method
__TkinterCallWrapper.__call__ of
<_Pmw.Pmw_1_2.lib.PmwBase.__TkinterCallWrapper
instance at 0x0295CA08>>
13 24953360 <type 'tuple'> ('type', 'obj', 'types', 'CodeType', 'ListType',
'len', 'TupleType', 'True', 'DictType', 'n', 'keys',
'key', 'get', 'val', 'repr', 'False')
14 24959216 <type 'tuple'> (('command', None), ('insert', None),
('overwrite', None), ('button', None), ('body', 'body'), ('text'
, 'head'), ('tree', 'head'), ('tree', 'canvas'), ('log', 'log'), ('text',
'log'), ('text', None), ('all', None))
15 25585536 <type 'tuple'> ('obj', 'd', 'oldObj', 'isLargeItem', 'n',
'keys', 'gc', 'key', 'new', 'types')
16 26717792 <type 'frame'> <frame object at 0x0197AE60>
17 27405776 <type 'tupleiterator'> <tupleiterator object at 0x01A22DD0>
18 27405808 <class 'leoNodes.position'> <pos 27405808 lvl: 0 [0]
getNewObjects>
19 27405840 <class 'leoNodes.position'> <pos 27405840 lvl: 0 [0]
getNewObjects>
20 27405872 <class 'leoNodes.position'> <pos 27405872 lvl: 0 [0]
getNewObjects>
21 27405968 <class 'leoNodes.position'> <pos 27405968 [0] None>
22 27406000 <class 'leoNodes.position'> <pos 27406000 lvl: 0 [0] Startup>
23 27406032 <class 'leoNodes.position'> <pos 27406032 lvl: 0 [0]
Prototypes>
24 27406064 <class 'leoNodes.position'> <pos 27406064 lvl: 0 [0] Tests>
25 27406096 <class 'leoNodes.position'> <pos 27406096 lvl: 0 [0]
gc.get_objects>
26 27406128 <class 'leoNodes.position'> <pos 27406128 lvl: 0 [0] Java>
27 27406160 <class 'leoNodes.position'> <pos 27406160 lvl: 0 [0] gc tests>
28 27406192 <class 'leoNodes.position'> <pos 27406192 lvl: 0 [0] print sep>
29 27406224 <class 'leoNodes.position'> <pos 27406224 lvl: 0 [0]
print-plugins>
30 27406256 <class 'leoNodes.position'> <pos 27406256 lvl: 0 [0] byte-code
tests>
31 27406320 <class 'leoNodes.position'> <pos 27406320 lvl: 0 [0]
getNewObjects>
32 27564784 <class 'leoNodes.position'> <pos 27564784 lvl: 0 [0]
getNewObjects>
33 27571824 <type 'list'> ***** large item
34 27572704 <type 'list'> ['black', 'red', 'blue', 'purple']
35 27608960 <type 'dict'> {'c': Commander 23254680:
u'C:\\prog\\tigris-cvs\\leo\\test\\test.leo', 'widget': <Tkinter.Canvas
insta
nce at 0x016EFF80>, 'actualEvent': <Tkinter.Event instance at 0x0295C9B8>,
'char': '\x02', 'w': <Tkinter.Canvas instance at 0x016E
FF80>, 'y': 174, 'x': 115, 'keysym': 'b'}
36 27621536 <type 'dict'> {'subst': None, 'widget': <Tkinter.Tk instance at
0x016055F8>, 'func': <function callit at 0x0296A030>}
37 27622976 <type 'dict'> {}
38 27647312 <type 'tuple'> (<function idleTimeHookHandler at 0x00BBEAF0>,
(), <Tkinter.Tk instance at 0x016055F8>, ['24118016call
it'])
39 27647408 <type 'tuple'> (None, 500, 0, 100, 1000)
40 27647456 <type 'tuple'> ('obj', 'val', 'keys', 'n', 'key')
41 43151120 <type 'frame'> <frame object at 0x02926F10>
42 43368936 <type 'list'> []
43 43369296 <type 'list'> []
44 43369376 <type 'tuple'> (<Tkinter.Event instance at 0x0295C9B8>,)
45 43369496 <type 'list'> []
46 43369696 <type 'list'> ['24118016callit']
47 43369736 <type 'list'> []
48 43369816 <type 'list'> []
49 43369856 <type 'list'> []
50 43369896 <type 'tuple'> (leoTkTextWidget id: 25544304 name: head-6, 27)
51 43370096 <type 'tuple'> (<pos 27406032 lvl: 0 [0] Prototypes>, 8)
52 43370216 <type 'tuple'> ('\x02', u'Ctrl+b')
53 43370296 <type 'instance'> tkGui.leoKeyEvent: char: '\x02', keysym: 'b'
54 43370496 <type 'tuple'> (<function masterBindKeyCallback at 0x01756AF0>,
(<Tkinter.Event instance at 0x0295C9B8>,))
55 43370616 <type 'list'> []
56 43370656 <type 'tuple'> (leoTkTextWidget id: 27572864 name: head-10, 44)
57 43370696 <type 'tuple'> (<pos 27406064 lvl: 0 [0] Tests>, 8)
58 43370776 <type 'tuple'> (<pos 27406000 lvl: 0 [0] Startup>, 8)
59 43370856 <type 'tuple'> (<pos 27406128 lvl: 0 [0] Java>, 8)
60 43370936 <type 'instance'> <Tkinter.Event instance at 0x0295C9B8>
61 43370976 <type 'list'> [41, 46, 50, 51, 56, 1, 6, 11, 16, 21, 25, 29,
33]
62 43371016 <type 'instance'>
<_Pmw.Pmw_1_2.lib.PmwBase.__TkinterCallWrapper instance at 0x0295CA08>
63 43371056 <type 'list'> [7, 12, 17, 42, 52]
64 43371096 <type 'list'> [36, 40, 45, 49, 55, 5, 10, 15, 20, 24, 28]
65 43371136 <type 'list'> []
66 43371176 <type 'list'> [34, 38, 43, 47, 53, 3, 8, 13, 18, 22, 26]
67 43371216 <type 'list'> []
68 43371256 <type 'list'> []
69 43371296 <type 'list'> []
70 43371336 <type 'list'> []
71 43371576 <type 'tuple'> (leoTkTextWidget id: 25544784 name: head-8, 35)
72 43371696 <type 'list'> []
73 43371736 <type 'tuple'> (<pos 27406192 lvl: 0 [0] print sep>, 8)
74 43372016 <type 'tuple'> (leoTkTextWidget id: 25542984 name: head-2, 9)
75 43372136 <type 'list'> []
76 43372176 <type 'tuple'> (leoTkTextWidget id: 27572984 name: head-11, 48)
77 43372376 <type 'tuple'> (<pos 27406224 lvl: 0 [0] print-plugins>, 8)
78 43372592 <type 'dict'> {'stack': [], 'v': <vnode
27563632:'getNewObjects'>}
79 43372736 <type 'dict'> {'keysym_num': 98, 'widget': <Tkinter.Canvas
instance at 0x016EFF80>, 'x_root': 524, 'type': '2', 'delt
a': 66, 'y_root': 412, 'height': '??', 'char': '\x02', 'width': '??',
'state': 4, 'num': '??', 'time': 4522765, 'y': 174, 'x': 115
, 'serial': 5780, 'keysym': 'b', 'keycode': 66}
80 43373168 <type 'dict'> {'27563248:5.': (leoTkTextWidget id: 25543504
name: head-3, 14), '27563376:6.': (leoTkTextWidget id: 25
544544 name: head-7, 31), '27563632:9.': (leoTkTextWidget id: 27572424 name:
head-5, 23), '27563440:7.': (leoTkTextWidget id: 2757
2024 name: head-1, 4), '25508048:0.': (leoTkTextWidget id: 27572864 name:
head-10, 44), '27513232:2.': (leoTkTextWidget id: 255447
84 name: head-8, 35), '27562704:3.': (leoTkTextWidget id: 25542984 name:
head-2, 9), '27563152:4.': (leoTkTextWidget id: 27572984
name: head-11, 48), '26446064:1.': (leoTkTextWidget id: 25544304 name:
head-6, 27), '27563536:8.': (leoTkTextWidget id: 25543704 n
ame: head-4, 19), '27563728:0.': (leoTkTextWidget id: 27627608 name:
head-12, 54)}
81 43373312 <type 'dict'> ***** large item
82 43373456 <type 'dict'> {'stack': [], 'v': None}
83 43373600 <type 'dict'> {'stack': [], 'v': <vnode
27563632:'getNewObjects'>}
84 43373744 <type 'dict'> ***** large item
85 43373888 <type 'dict'> {'stack': [], 'v': <vnode
27563632:'getNewObjects'>}
86 43374032 <type 'dict'> {34: (<pos 27406000 lvl: 0 [0] Startup>, 8), 3:
(<pos 27406224 lvl: 0 [0] print-plugins>, 8), 38: (<pos
27406032 lvl: 0 [0] Prototypes>, 8), 8: (<pos 27406160 lvl: 0 [0] gc
tests>, 8), 43: (<pos 27406064 lvl: 0 [0] Tests>, 8), 13: (<
pos 27406256 lvl: 0 [0] byte-code tests>, 8), 47: (<pos 27406128 lvl: 0 [0]
Java>, 8), 18: (<pos 27406096 lvl: 0 [0] gc.get_object
s>, 8), 53: (<pos 27406192 lvl: 0 [0] print sep>, 8), 22: (<pos 27406320
lvl: 0 [0] getNewObjects>, 8), 26: (<pos 43384912 lvl: 1
[0] isLargeItem>, 8)}
87 43374176 <type 'dict'> {'stack': [], 'v': <vnode 27513232:'Tests'>}
88 43374464 <type 'dict'> {'stack': [], 'v': <vnode 25508048:'Startup'>}
89 43374608 <type 'dict'> {'stack': [], 'v': <vnode 27562704:'Java'>}
90 43374752 <type 'dict'> {'stack': [], 'v': <vnode 26446064:'Prototypes'>}
91 43374896 <type 'dict'> {'stack': [], 'v': <vnode 27563152:'print sep'>}
92 43375040 <type 'dict'> {'stack': [], 'v': <vnode
27563248:'print-plugins'>}
93 43375184 <type 'dict'> {'stack': [], 'v': <vnode 27563376:'gc tests'>}
94 43375328 <type 'dict'> {'stack': [], 'v': <vnode 27563440:'byte-code
tests'>}
95 43375472 <type 'dict'> {'stack': [], 'v': <vnode
27563536:'gc.get_objects'>}
96 43375616 <type 'dict'> {'stack': [], 'v': <vnode
27563632:'getNewObjects'>}
97 43376192 <type 'dict'> {'stack': [], 'v': <vnode
27563728:'isLargeItem'>}
98 43380784 <type 'list'> []
99 43380824 <type 'tuple'> (leoTkTextWidget id: 25543504 name: head-3, 14)
100 43381024 <type 'tuple'> (<pos 27406160 lvl: 0 [0] gc tests>, 8)
101 43381184 <type 'list'> []
102 43381224 <type 'tuple'> (leoTkTextWidget id: 25544544 name: head-7, 31)
103 43381424 <type 'tuple'> (<pos 27406256 lvl: 0 [0] byte-code tests>, 8)
104 43381584 <type 'list'> []
105 43381624 <type 'tuple'> (leoTkTextWidget id: 27572024 name: head-1, 4)
106 43381824 <type 'tuple'> (<pos 27406096 lvl: 0 [0] gc.get_objects>, 8)
107 43382024 <type 'tuple'> (leoTkTextWidget id: 25543704 name: head-4, 19)
108 43382224 <type 'tuple'> (<pos 27406320 lvl: 0 [0] getNewObjects>, 8)
109 43382424 <type 'tuple'> (<pos 43384912 lvl: 1 [0] isLargeItem>, 8)
110 43382704 <type 'tuple'> (leoTkTextWidget id: 27572424 name: head-5, 23)
111 43382824 <type 'list'> []
112 43382864 <type 'tuple'> (leoTkTextWidget id: 27627608 name: head-12, 54)
113 43383384 <type 'tuple'> ***** large item
114 43384912 <class 'leoNodes.position'> <pos 43384912 lvl: 1 [0]
isLargeItem>
115 43425840 <type 'function'> <function callit at 0x0296A030>
116 43426032 <type 'function'> <function isLargeItem at 0x0296A0F0>
117 43426224 <type 'tuple'> (None, <code object isLargeItem at 0296A160,
file "C:\prog\tigris-cvs\leo\test\scriptFile.py", line 7>
, '----- %d new objects', 200, 0, 1, '%3d', '***** large item', '=====')
118 43442512 <type 'dict'> ***** large item
=====
I think this answers the question about where the strings are: they are
enclosed in tuples. My guess is that the tuples are the way that Python's
bytecode accesses the strings. For example, there is a LOAD_CONST i
bytecode whose argument i is an index into the stack frame (or is it the
code objects). Anyway, my guess is that strings seldom (never?) exist
outside the tuple by which they are referenced. So I think this answers the
mystery, mostly.
Edward
--------------------------------------------------------------------
Edward K. Ream email: edreamleo at charter.net
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------
More information about the Python-list
mailing list