parse/slice/...

[Albert Hopkins]

On Tue, 2009-01-06 at 11:23 -0800, rcmn wrote:
> I'm not sure how to call it sorry for the subject description.
>    Here what i'm trying to accomplish.
> the script i'm working on, take a submitted list (for line in file)
> and generate thread for it. unfortunately winxp has a limit of 500
> thread . So I have to parse/slice the file by chunk of 500 and loop
> until the list is done.
> I though i would of done it in no time but i can't get started for
> some reason.
> And i can't find a good way to do it efficiently . Does anyone have
> something similar to this.

If your thread is smart enough to handle the file you can do something
like:

class Worker(threading.Thread):
    def __init__(self, file_object):
        self.file_object = file_object
        threading.Thread.__init__(self)

    def run(self):
        while True:
            line = self.file_object.readline()
            if line == '':
                return
            # process line

NUM_WORKERS = 500
bigfile = open('somefile', 'r')
workers = [Worker(bigfile) for i in range(NUM_WORKERS)]
for worker in workers:
    worker.start()