[New-bugs-announce] [issue35291] duplicate of memoryview from io.BufferedWriter leaks

Jason Madden report at bugs.python.org
Wed Nov 21 10:12:32 EST 2018

New submission from Jason Madden <jason at nextthought.com>:

Using Python 2.7.15, if a BufferedWriter wraps an IO object that duplicates the memoryview passed to the IO object's `write` method, that memoryview leaks. 

This script demonstrates the problem by leaking a memoryview for each iteration of the loop (if the `flush` is skipped, the leaks are less frequent but still occur):

from __future__ import print_function

import io
import gc

def count_memoryview():
    result = 0
    for x in gc.get_objects():
        if type(x) is memoryview:
            result += 1
    return result

class FileLike(object):
    closed = False

    def writable(self):
        return True

    def write(self, data):
        memoryview(data) # XXX: This causes the problem
        return len(data)

bf = io.BufferedWriter(FileLike())

i = 0
memoryview_count = 0

while True:
    if i == 0 or i % 100 == 0:
        # This reports 100 new memoryview objects each time
        old = memoryview_count
        new = count_memoryview()
        print(i, "memoryview", new, "+%s" % (new - old))
        memoryview_count = new

    i += 1

The leak can also be observed using the operating system's memory monitoring tools for the process (seen on both Fedora and macOS).

Commenting out the line in `FileLike.write` that makes a duplicate memoryview of the given buffer solves the leak. 

Deleting the BufferedWriter doesn't appear to reclaim the leaked memoryviews.

I can't duplicate this in Python 3.4 or above.

Originally reported to gevent in https://github.com/gevent/gevent/issues/1318

Possibly related to Issue 26720 and Issue 15994

components: IO
messages: 330206
nosy: jmadden
priority: normal
severity: normal
status: open
title: duplicate of memoryview from io.BufferedWriter leaks
versions: Python 2.7

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list