fork after creating temporary file using NamedTemporaryFile

[Sebastian "lunar" Wiesner]

rparimi at gmail.com <rparimi at gmail.com>:

> When I create temporary file using the tempfile module, and forkI)
> later on in my program, I always see errors when the program exits. Is
> this because the child process deletes temp file?
> Here's a stripped down version of my script that exhibits this
> problem:
> 
> #!/usr/bin/python
> 
> import os
> import tempfile
> import sys
> 
> cmd = []
> cmd.append('/bin/ls')
> cmd.append('-l')
> cmd.append('/tmp')
> 
> foo = tempfile.NamedTemporaryFile(mode='w+b')
> 
> pid = os.fork()
> if pid:
>         print 'I am parent'
> else:
>         print 'I am child'
>         sys.exit(0)
> 
> $ python sub.py
> I am child
> I am parent
> Exception exceptions.OSError: (2, 'No such file or directory', '/tmp/
> tmp-mZTPq') in <bound method _TemporaryFileWrapper.__del__ of <closed
> file '<fdopen>', mode 'w+b' at 0xb7d2a578>> ignored

NamedTemporaryFile attempts to delete the file, when "close" is invoked
(which is done by the destructor in this case, because you're senselessly
not closing that file explicitly).  A "fork()" clones the whole process
memory, after fork() is done, there are *two* NamedTemporaryFile objects,
pointing to the *same* file, existing in two separate process (one in the
parent, the other, cloned one, in the child).  When the first process
exists, the file is removed cleanly, the second process can't remove
anything anymore, since there's nothing left to remove.

Since the order of process execution is not deterministic and completely up
to the systems scheduler, you can *never* say, which one will exit first.  

You should replace NamedTempraryFile with "mkstemp" and take care of
deletion yourself.  You could for instance call "os.wait" to wait for the
child's termination in the parent process and thus delete the temporary
file, once the child has gone.

Of course, you could also catch the exception, if you properly close the
file object (what you should be doing anyway!), but I'd consider this a not
very robust solution.  If one of the two processes exit unexpectedly early,
the directory entry is gone, though it might still be needed by the parent
process.

> How can these warnings be avoided? I tried to catch this exception
> using try/except but it didn't work.

If you want to catch that exception, you should properly close the named
temporary file and wrap the "close" call inside a try-except statement. 
Beginning with Python 2.5 you might also want to use the with-statement
here.  Relying on the destructor is *always* a bad idea, you should always
close files explicitly!

-- 
Freedom is always the freedom of dissenters.
                                      (Rosa Luxemburg)