duplicate temporary file?

[Laszlo Nagy]

I try to write an iterator class:

class FileIterator(object):
     def __init__(self,file_object):
         self.fin = file_object
         # other initialization code here ...
     def __iter__(self):
         return self
     def next(self):
         # special code that reads from the file and converts to 
Pythonic data...

This iterator I want to use with regular binary file objects. My 
requirement is to be able to create many iterators for the same file 
object, and use them from many threads - which is not straightforward. 
Of course I do not want to save file positions in the iterators, use 
locking on the file, and call seek() every time I need to read from the 
file. Instead of that, I try to re-open the file for each iteator. The 
trivial way to do it is:

class FileIterator(object):
     def __init__(self,file_object):
         self.fin = open(file_object.name,"rb")
         self.fin.seek(file_object.tell())
     ...

However, it won't work for temporary files. Temporary files are just 
file-like objects, and their name is '<fdopen>'. I guess I could open a 
temporary file with os.open, and then use os.dup, but that is low level 
stuff. Then I have to use os.* methods, and the file object won't be 
iterable. tempfile.NamedTempFile is not better, because it cannot be 
reopened under Windows.

So the question is, how do I create my iterator so that:

    * under Windows and Unix
    * many iterators can be used for the same underlying file, from many
      threads
    * with a normal binary file
    * and also with temporary files

A workaround would be to create a temporary directory, and then create 
real files inside the temporary directory. If there is no better way to 
do it, I'll do that.

Thanks

    Laszlo


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20100503/ff965136/attachment.html>