[issue9942] Allow memory sections to be OS MERGEABLE

Antoine Pitrou report at bugs.python.org
Sat Sep 25 16:31:36 CEST 2010

Antoine Pitrou <pitrou at free.fr> added the comment:

> > Well, first, this would only work for large objects. [...]
> > Why do you think you might have such duplication in your workload?
> Some of the projects with which I work involve multiple manipulations
> of large datasets.  Often, we use Python scripts as "first and third"
> stages in a pipeline.  For example, in one current workflow, we read a
> large file into a cStringIO object, do a few manipulations with it,
> pass it off to a second process, and await the results.

Why do you read it into a cStringIO? A cStringIO has the same interface
as a file, so you could simply operate on the file directly.

(you could also try mmap if you need quick random access to various
portions of the file)


Python tracker <report at bugs.python.org>

More information about the Python-bugs-list mailing list