[Python-Dev] smarter temporary file object (SF #415692)

[dustin at v.igoro.us]

I came across a complaint that PEP 0042 had become a graveyard of
neglected ideas, and decided to have a look through and implement
something.  Creating a smarter temporary file object seemed simple
enough.

Oddly, even after GvR re-opened it, I can't post an attachment to that
tracker item (it's under "Feature Requests" -- does it need to get moved
to "Patches" first?), but the implementation is short, so it's included
below.  This is intended to be appended to Lib/tempfile.py (and thus
assumes that module's globals are present).

I would appreciate it if the gurus of python-dev could take a peek and
let me know if this is unsuitable or incorrect for any reason.  It's not
the most straightforward implementatio -- I used the optimization
techniques I found in TemporaryFile.

If this looks good, I'll prepare a patch against trunk, including an
additional chunk of documentation and a unit test.

Dustin

                     -----cut-here-----

try:
  from cStringIO import StringIO
except:
  from StringIO import StringIO

class SpooledTemporaryFile:
    """Temporary file wrapper, specialized to switch from
    StringIO to a real file when it exceeds a certain size or
    when a fileno is needed.
    """
    _rolled = False

    def __init__(self, max_size=0, mode='w+b', bufsize=-1,
                 suffix="", prefix=template, dir=None):
        self._file = StringIO()
        self._max_size = max_size
        self._TemporaryFileArgs = (mode, bufsize, suffix, prefix, dir)

    def _check(self, file):
        if self._rolled: return
        if file.tell() > self.__dict__['_max_size']:
            self._rollover(file)

    def _rollover(self, file):
        args = self.__dict__['_TemporaryFileArgs']
        self.__dict__.clear() # clear attributes cached by __getattr__
        newfile = self._file = TemporaryFile(*args)
        newfile.write(file.getvalue())
        newfile.seek(file.tell(), 0)
        self._rolled = True
        
        # replace patched functions with the new file's methods
        self.write = newfile.write
        self.writelines = newfile.writelines
        self.fileno = newfile.fileno

    def write(self, s):
        file = self.__dict__['_file']
        rv = file.write(s)
        self._check(file)
        return rv

    def writelines(self, iterable):
        file = self.__dict__['_file']
        rv = file.writelines(iterable)
        self._check(file)
        return rv

    def fileno(self):
        self._rollover(self.__dict__['_file'])
        return self.fileno()

    def __getattr__(self, name):
        file = self.__dict__['_file']
        a = getattr(file, name)
        if type(a) != type(0):
            setattr(self, name, a)
        return a