[Tutor] A file containing a string of 1 billion random digits.

[ALAN GAULD]

> Now that you see what I want to do with 1 billion random 
digits,
> please give me your suggestion(s). As I mentioned 
before, 
> I'm very new to reading from and writing to files.

The way old text editors used to do this - in the days when we only had 16K RAM etc!
was to use buffers and pages. Let's say you have a 100K buffer size and read in the 
first buffer full of text. When you get down to say 75K you delete the first 50K and 
load the next 50K from the file. Your cursor is now 25K into a new set of 100K.

This allows you to page back a certain amount and page forward a lot. And so 
you progress losing 50% from top or bottom and loading the next/previous chunk 
into RAM. Its non-trivial to get right but in your case you are only looking to 
sequentially process so you only need to look forward so not quite so bad...

You need markers to keep your location and other markers to keep track of 
the buffers location within the file.

If you really are only doing sequential processing you can dispense with 
the buffer/pages and just read small chunks such that you always have two 
in memory at once. The chunks might only be a few Kb in size depending 
on the longest sequence you are looking for (a chunk of twice that size is 
a good guide)

The seek() tell() and read() methods are your friends for this kind of work

 
Alan Gauld
Author of the Learn To Program website
http://www.alan-g.me.uk/