python newbie - slicing a big memory chunk without GC penalties

Giovanni Bajo noway at sorry.com
Sun Feb 2 13:06:59 EST 2003


Hello,

Sorry if the question is trivial, but I am a newbie with Python. I have a
file read into memory within a 'sequence' (or whatever is returned by
file.read()), and I need to process it 512 bytes a time. Now, I was doing
something like:

for i in range(0, len(buf)/512):
    Process(buf[i*512 : (i+1)*512])

But it seems like a lot of time is wasted in the sequence slicing (before I
was processing everything in a shot, and it was much faster - and Process is
O(n) so it should not really matter that much). I tried also other
approacches like:

while len(buf):
    Process(buf[0:512])
    buf = buf[512:]

but it seems even worse. What's the best way to do this?

Thanks
Giovanni






More information about the Python-list mailing list