Threads and temporary files

aiwarrior zubeido at yahoo.com.br
Fri Mar 13 22:07:46 CET 2009


Hi
I recently am meddling with threads and wanted to make a threaded
class that instead of processing anything just retrieves data from a
file and returns that data to a main thread that takes all the
gathered data and concatenates it sequentially.

An example is if we want to get various ranges of an http resource in
paralell

import threading

class download(threading.Thread):
  def __init__(self,queue_in,queue_out):
        threading.Thread.__init__( self )

        self.url = url
        self.starts = starts
        self.ends = ends
        self.content = 0

  def getBytesRange(self):
        request = urllib2.Request(self.url) # New Request object
        if self.ends is not None: # If the end of the desired range is
specified
            request.add_header("Range", "bytes=%d-%d" % (self.starts,
self.ends))
        else: # If you want everything from start up to the resource's
length
            request.add_header("Range", "bytes=%d-" % self.starts)
        response = urllib2.urlopen(request) # Make the request, get
the data
        self.response.read()
  def run(self):
    self.getBytesRange()

then when we create the threads we wait for them to complete and write
in a file. To reduce memory footprint we can write the self.response
to a tempifile and then when all threads complete concatenate the
temps in a single file in a specified location. The problem here is
the same, how to get a reference to the temporary file objects created
in the threads sequeatially. I hope i have made myself clear. Thanks
you in advance



More information about the Python-list mailing list