What are the limitations of cStringIO.StringIO() ?

[Barak, Ron]

Hi,

I think I'm up against a limitation in cStringIO.StringIO(), but could not find any clues on the web, especially not in http://docs.python.org/library/stringio.html.

What I have is the following function:

    def concatenate_files(self,filename_array):
        import types

        f = cStringIO.StringIO()
        for filename in filename_array:
            log_stream = LogStream(filename)
            if not isinstance(log_stream, types.NoneType):
                try:
                    string_ = log_stream.input_file.read()
                except AttributeError, e:
                        sys.stderr.write("AttributeError: "+str(e)+"\n")
                        sys.stderr.write("log_stream: "+str(log_stream)+"\n")
            else:
                return(None)

            f.write(string_)
        return (f)

And, when the list of files - in filename_array - is pointing to long/large enough files, I get the exception:

Traceback (most recent call last):
  File "svm_ts_tool_in_progress.py", line 244, in OnSelectCell
    self.InspectorViewController(row,col)
  File "svm_ts_tool_in_progress.py", line 1216, in InspectorViewController
    self.InspectorePaneGetRecords(self.req_table, rec)
  File "svm_ts_tool_in_progress.py", line 1315, in InspectorePaneGetRecords
    log_stream = ConcatenatedLogStream(self.events_dict[req_table]["db_dict"].keys())
  File "c:\Documents and Settings\rbarak\rbarak_devel\dpm16\ConcatenatedLogStream.py", line 31, in __init__
    self.input_file = self.concatenate_files(filename_array)
  File "c:\Documents and Settings\rbarak\rbarak_devel\dpm16\ConcatenatedLogStream.py", line 47, in concatenate_files
    string_ = log_stream.input_file.read()
MemoryError

 1.  Anyone knows whet's the limitation on cStringIO.StringIO() objects ?
 2.  Could you suggest a better way to create a string that contains the concatenation of several big files ?

Thanks,
Ron.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20090701/7a56bd37/attachment.html>