[Python-Dev] AMD64 Windows8.1 Refleaks 3.x buildbot is back to green!

[Brett Cannon]

Thanks for tracking that down! Been bugging me as well and the one time I
tried to figure it out I got no where, so kudos for sticking with it!

On Wed, 9 Jan 2019 at 09:13, Victor Stinner <vstinner at redhat.com> wrote:

> Hi,
>
> The "AMD64 Windows 8.1 Refleaks 3.x" buildbot (which hunts reference
> leaks and memory leaks) was failing on test_asyncio for 1 year:
>
>    https://bugs.python.org/issue32710
>
> It was a leak of a single reference:
>
>    test_aiosend leaked [1, 1, 1] references, sum=3
>
> I tried multiple times since last year (January 2018) to understand
> the leak: it didn't make any sense (well, as any bug at the beginning,
> no?). I checked several times the complex asyncio code: all transports
> are cleanly closed, the event loop is properly closed, etc. The code
> looks correct.
>
> After a long sleepness night... I still failed to reproduce the bug
> :-( But I succeeded to get a way shorter reproducer script. Thanks to
> this script, I was able to loop to get 100 reference leaks instead of
> leaking a single reference. Using tracemalloc, I found the faulty
> line... but it still didn't make sense to me. After additional several
> hours of debug, I found that an overlapped object wasn't released
> properly: an asynchronous WSASend().
>
> The problem was when an overlapped WSASend() failed immediately, the
> internal buffer was not released, whereas it holds a reference to the
> input byte string. **It means that an asyncio send() failure using the
> ProactorEventLoop can cause a memory leak**... I'm very surprised that
> nobody noticed such **huge** memory leak previously!
>
> The _overlapped bugfix:
>
>
> https://github.com/python/cpython/commit/a234e148394c2c7419372ab65b773d53a57f3625
>
> Eventually, the "AMD64 Windows 8.1 Refleaks 3.x" buildbot is back to green!
>
>    https://buildbot.python.org/all/#/builders/80
>
> It means that it will be easier and faster to spot reference or memory
> leak regressions (specific to Windows, the Gentoo Refleaks buildbot
> was already green for several months!).
>
> Since ProactorEventLoop became the default event loop in Python 3.8
> (on Windows, it's specific to Windows), I hope that we fixed all most
> obvious bugs!
>
> This story also means that any very tiny buildbot failure (a single
> test method failure on a single very specific buildbot) can hide a
> giant bug ;-) Sadly, we have to fix *all* buildbots failures to find
> them... Hopefully, almost all buildbots are green currently.
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190109/9d259a9b/attachment.html>