[issue9942] Allow memory sections to be OS MERGEABLE
report at bugs.python.org
Sat Sep 25 16:31:36 CEST 2010
Antoine Pitrou <pitrou at free.fr> added the comment:
> > Well, first, this would only work for large objects. [...]
> > Why do you think you might have such duplication in your workload?
> Some of the projects with which I work involve multiple manipulations
> of large datasets. Often, we use Python scripts as "first and third"
> stages in a pipeline. For example, in one current workflow, we read a
> large file into a cStringIO object, do a few manipulations with it,
> pass it off to a second process, and await the results.
Why do you read it into a cStringIO? A cStringIO has the same interface
as a file, so you could simply operate on the file directly.
(you could also try mmap if you need quick random access to various
portions of the file)
Python tracker <report at bugs.python.org>
More information about the Python-bugs-list