smarter temporary file object (SF #415692)
I came across a complaint that PEP 0042 had become a graveyard of neglected ideas, and decided to have a look through and implement something. Creating a smarter temporary file object seemed simple enough. Oddly, even after GvR re-opened it, I can't post an attachment to that tracker item (it's under "Feature Requests" -- does it need to get moved to "Patches" first?), but the implementation is short, so it's included below. This is intended to be appended to Lib/tempfile.py (and thus assumes that module's globals are present). I would appreciate it if the gurus of python-dev could take a peek and let me know if this is unsuitable or incorrect for any reason. It's not the most straightforward implementatio -- I used the optimization techniques I found in TemporaryFile. If this looks good, I'll prepare a patch against trunk, including an additional chunk of documentation and a unit test. Dustin -----cut-here----- try: from cStringIO import StringIO except: from StringIO import StringIO class SpooledTemporaryFile: """Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed. """ _rolled = False def __init__(self, max_size=0, mode='w+b', bufsize=-1, suffix="", prefix=template, dir=None): self._file = StringIO() self._max_size = max_size self._TemporaryFileArgs = (mode, bufsize, suffix, prefix, dir) def _check(self, file): if self._rolled: return if file.tell() > self.__dict__['_max_size']: self._rollover(file) def _rollover(self, file): args = self.__dict__['_TemporaryFileArgs'] self.__dict__.clear() # clear attributes cached by __getattr__ newfile = self._file = TemporaryFile(*args) newfile.write(file.getvalue()) newfile.seek(file.tell(), 0) self._rolled = True # replace patched functions with the new file's methods self.write = newfile.write self.writelines = newfile.writelines self.fileno = newfile.fileno def write(self, s): file = self.__dict__['_file'] rv = file.write(s) self._check(file) return rv def writelines(self, iterable): file = self.__dict__['_file'] rv = file.writelines(iterable) self._check(file) return rv def fileno(self): self._rollover(self.__dict__['_file']) return self.fileno() def __getattr__(self, name): file = self.__dict__['_file'] a = getattr(file, name) if type(a) != type(0): setattr(self, name, a) return a
On 1/2/07, dustin@v.igoro.us
I came across a complaint that PEP 0042 had become a graveyard of neglected ideas, and decided to have a look through and implement something. Creating a smarter temporary file object seemed simple enough.
Oddly, even after GvR re-opened it, I can't post an attachment to that tracker item
Hi Dustin. Thanks for your patch! Unfortunately, SF doesn't allow anyone except developers and the tracker item owner to submit attachments. The best thing to do would be to create a new patch. Then you can make a comment in the old RFE that references your new patch. Something like this is sufficient: Patch available here: http://python.org/sf/NEW_PATCH_NUMBER Cheers, n
On Tue, Jan 02, 2007 at 10:07:58PM -0800, Neal Norwitz wrote:
Thanks for your patch!
With some advice from Jim Jewett, the addition of some test cases, and a paragraph of documentation, I've uploaded the corresponding patch at http://python.org/sf/1630118 if there are any other modifications or improvements anyone has in mind, please let me know. Dustin
participants (2)
-
dustin@v.igoro.us
-
Neal Norwitz