threading a 10 lines out of a file

alex goretoy aleksandr.goretoy at
Mon Jan 5 07:00:50 CET 2009

Hello All,

I have a question. I'm not sure exactly as how to explain it in any other
way then the way I will explain it. So I'm sorry if it's hard to understand
exactly what it is I'm trying to do. Maybe not. Anyway. Here goes.

Lets say I have a file that looks like this.

123,abc,testing is fun,yeah baby
456,qwe,python makes if funner, yeah baby
789,zxc,this is another line in this file, yeah baby

This file can have an unknown amount of lines in it. It can be 700,400 or
even 7,000.
I do this to read the file:

            reader = csv.reader(file(filename, "rb"))
                header =
                self.buffer = list(reader) # total = len(self.buffer)
                self.bufferp= [dict(zip(header,line)) for line in
                self.header = header
                #for row in reader:

                    #for j in range(len(self.buffer[0])):
       = i-1
            except csv.Error, e:
                sys.exit('file %s, line %d: %s' % (filename,
reader.line_num, e))
        except IOError, e:
            sys.exit('file %s, IOError: %s' % (filename, e))

What I am currently doing is looping over this file, performing a function
for every line. This function isbeing called from this class __init__(),

def loop_lines(self):

        for k in range(len(self.buffer)): #for every line in csv file

Now, for my question. Is it possible for me to thread this scenario somehow?

So that I can set a variable that says how many lines to work on at the same

lets say 10 lines at a time, once it finishes some it moves on to the next
ones that are not in the thread pool or something to that nature. Always
making sure that it works on 10 at the same time.

How would I achieve something like this in my program? Can someone please
recommend something for me. I would greatly appreciate. My program would
appreciate it too, seeing as it will be multi-threaded :) Thank you for your

-Alex Goretoy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the Python-list mailing list