speeding up reading files (possibly with cython)
sion at paintbox.UUCP
Mon Mar 9 14:55:57 CET 2009
Carl Banks <pavlovevidence at gmail.com> wrote:
>When building a very large structure like you're doing, the cyclic
>garbage collector can be a bottleneck. Try disabling the cyclic
>garbage collector before building the large dictionary, and re-
>enabling it afterwards.
> for line in file:
> split_values =3D line.strip().split('\t')
> # do stuff with split_values
Completely untested, but if you find yourself doing that a lot,
from contextlib import contextmanager
for line in file:
# ... etc.
be worth considering?
More information about the Python-list