<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"><html><head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<style type="text/css">
BLOCKQUOTE{margin-Top: 0px; margin-Bottom: 0px; margin-Left: 2em}
</style>
<meta name="GENERATOR" content="MSHTML 11.00.10570.1001"><!-- flashmail style begin -->
<style type="text/css">
body {border-width:0;margin:0}
img {border:0;margin:0;padding:0}
</style>
<base target="_blank"><!-- flashmail style end --></head>
<body style="BORDER-LEFT-WIDTH: 0px; FONT-SIZE: 10.5pt; FONT-FAMILY: 微软雅黑; BORDER-RIGHT-WIDTH: 0px; BORDER-BOTTOM-WIDTH: 0px; COLOR: #000000; MARGIN: 12px; LINE-HEIGHT: 1.5; BORDER-TOP-WIDTH: 0px" marginheight="0" marginwidth="0">Hi hubo,<br>
<br>
Wow, that was a very good analysis, thanks a lot for finding that out! Armin is working on fixing the bug. <br>
<br>
Cheers, <br>
Carl Friedrich<br><br><div class="gmail_quote">On June 28, 2018 7:51:58 AM GMT+02:00, hubo <hubo@jiedaibao.com> wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div>This problem is caused by caching GeneratorExit exception:</div>
<div> </div>
<div><a href="https://bitbucket.org/pypy/pypy/src/8b43b50fbf61f46701d398bf514cf540201ffd03/pypy/interpreter/generator.py?at=py3.5&fileviewer=file-view-default#generator.py-454:457">https://bitbucket.org/pypy/pypy/src/8b43b50fbf61f46701d398bf514cf540201ffd03/pypy/interpreter/generator.py?at=py3.5&fileviewer=file-view-default#generator.py-454:457</a></div>
<div> </div>
<div>In Python 3, an exception saves its traceback in __traceback__ attribute.
When an exception object with __traceback__ is raised again, current
frame is appended to the __traceback__. When the GeneratorExit is raised inside
a iterator, the __traceback__ attribute of this object saves the frame, which
prevents the iterator from been collected by GC.</div>
<div> </div>
<div>Since __traceback__ attribute is mutable, it is generally a dangerous idea
to reuse an exception object in Python 3, the problem includes: creating memory
leaks with frames, wrong traceback, etc. I think remove the caching will solve
the problem.</div>
<div> </div>
<div>It's quite easy to be validated with the following script:</div>
<div> </div>
<div style="FONT-FAMILY: Courier New">def
test():<br> try:<br>
yield 1<br> except BaseException as
e:<br> global
ex<br> ex = e</div>
<div style="FONT-FAMILY: Courier New"> </div>
<div style="FONT-FAMILY: Courier New">for _ in range(10):<br>
t = test()<br> next(t)<br> t.close()</div>
<div style="FONT-FAMILY: Courier New"> </div>
<div style="FONT-FAMILY: Courier New">import traceback</div>
<div style="FONT-FAMILY: Courier New">traceback.print_tb(ex.__traceback__)</div>
<div> </div>
<div>You will see many frames in the traceback.</div>
<div> </div>
<div>Removing the __traceback__ attribute before each usage is not acceptable,
because the exception may be used concurrently in different threads,
or nested in a finalizer when GC is triggered inside the generator close.</div>
<div> </div>
<div style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; COLOR: #c0c0c0" align="left">2018-06-28
<hr id="SignNameHR" style="BORDER-TOP: #c0c0c0 1px solid; HEIGHT: 1px; BORDER-RIGHT: 0px; WIDTH: 122px; BORDER-BOTTOM: 0px; BORDER-LEFT: 0px" align="left">
<span id="_FlashSignName">hubo</span> </div>
<hr style="BORDER-TOP: #c0c0c0 1px solid; HEIGHT: 1px; BORDER-RIGHT: 0px; BORDER-BOTTOM: 0px; BORDER-LEFT: 0px">
<blockquote id="ntes-flashmail-quote" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; PADDING-LEFT: 0px; MARGIN-LEFT: 0px">
<div><strong>发件人：</strong>"hubo" <hubo@jiedaibao.com></div>
<div><strong>发送时间：</strong>2018-06-28 13:06</div>
<div><strong>主题：</strong>[pypy-dev] Memory leaking with iterator.close() in
PyPy3</div>
<div><strong>收件人：</strong>"PyPy Developer Mailing
List"<pypy-dev@python.org></div>
<div><strong>抄送：</strong></div>
<div> </div>
<div><stationery>
<div>In PyPy3, when an iterator is closed with "close()" method, the
iterator leaks and cannot be collected.</div>
<div> </div>
<div>Execute the following script in PyPy3, the memory usage is increasing
very fast, and gc.collect() cannot collect the memory</div>
<div> </div>
<div> </div>
<div style="FONT-FAMILY: Courier New">def test():</div>
<div style="FONT-FAMILY: Courier New"> yield 1</div>
<div style="FONT-FAMILY: Courier New"> </div>
<div style="FONT-FAMILY: Courier New">while True:</div>
<div style="FONT-FAMILY: Courier New"> t = test()</div>
<div style="FONT-FAMILY: Courier New"> t.close()</div>
<div style="FONT-FAMILY: Courier New"> </div>
<div><font face="Courier New"><span style="FONT-FAMILY: 微软雅黑"></span></font> </div>
<div>The tested version:</div>
<div> </div>
<div>Python 3.5.3 (fdd60ed87e94, Apr 24 2018, 06:10:04)<br>[PyPy 6.0.0 with
GCC 6.2.0 20160901]</div>
<div> </div>
<div>This is not reproduced in CPython 3.5 and PyPy2.</div>
<div> </div>
<div style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; COLOR: #c0c0c0">
<div align="left">2018-06-28</div>
<hr id="SignNameHR" style="BORDER-TOP: #c0c0c0 1px solid; HEIGHT: 1px; BORDER-RIGHT: 0px; WIDTH: 122px; BORDER-BOTTOM: 0px; BORDER-LEFT: 0px" align="left">
<span id="_FlashSignName">hubo</span>
</div></stationery></div></blockquote></blockquote></div></body></html>